Wednesday, June 8, 2016

The Importance of Quantitative Data Mining to Improve Practice

One way to use quantitative research to improve practice is to use the simple techniques for critiquing quantitative research in the top journals in terms of its practical significance; primarily in terms of how experimental students did on an actual basis. This is the emphasis in the first part of the alternative text Authentic Quantitative Analysis for Leadership Decision-Making and EdD Dissertations. Another less discussed way of using quantitative data is "Data Mining." While data mining is extensively employed in the business world to improve organizational performance, it also has wide applicability in education. Data mining simply means finding a key metric amidst all the data flowing through and within a school or district to initiate improvement action and to then monitor changes to that metric.

The importance of data mining comes across in a recent story on National Public Radio by Elissa Nadworny entitled:  What One District's Data Mining Did For Chronic Absence.

This story details how the superintendent asked the question: Do we have a serious chronic truancy problem?

This leads to the questions:

  • How should chronic truancy be defined?
  • How to access that data?

The subsequent analysis showed out that there was a huge unaddressed chronic truancy problem—40% of students fell into that category. The rest of the story deals with the steps taken both initially and subsequently to address the problem.

Throughout the improvement effort those involved continued to monitor progress on the metric. Several years later the number of chronic absentees had been cut in half. This case study has the following implications for EdD programs:

  • Data Mining is an important form of quantitative analysis that those in EdD programs are likely to apply in practice,
  • Such analyses do not require advanced statistics at any stage of the process, and
  • If you establish appropriate practices that are actively monitored and iteratively improved upon, within several years you can produce BIG improvements.

In terms of the last point, there is no need to calculate statistical significance or effect sizes. You have produced a large, highly visible improvement that does not require any statistic to tell you if the improvement is significant. It clearly is--and these are the types of improvements schools need to strive for.

At the same time, data mining by itself does not produce improvement. You need to incorporate research findings and local knowledge to develop an appropriate action plan. At the same time, if leaders do not have a data mining perspective/impulse, problems can easily be overlooked. It also becomes easy to overlook that a plan of action to solve a problem is not having the intended effect. In this example, the strategy used by the district had no effect in the first year and they then had to improvise changes to the action plan.

What this means is that quantitative methods courses in EdD programs should spend some time on applied data mining.  Topics could include:

  • Identifying and defining key metrics of improvement,
  • Having students collect data around a metric of their choice for their school(s), 
  • Evaluating the results and developing an initial action plan, and
  • Methods of communicating the quantitative data to the entire community in an easy to understand fashion that can be used to mobilize a broad-scale response to problems.  

The URL for the story is: http://www.npr.org/sections/ed/2016/05/30/477506418/what-one-districts-data-mining-did-for-chronic-absence