Thank you, Scott, for providing insight into how schools are using data to improve learning, not just test scores. Unfortunately, I’ve witnessed less cheerful data-driven decision making. Some schools are using benchmark tests and other newly available data to play the system and up their numbers. Let me mention a few of these bad and the ugly uses of data.
When I was teaching, my school ran a Saturday program for kids who were close to passing state tests. At the time, I patted myself on the back and thought we were helping our students. Now I understand that our principal was simply trying to increase our passing rates the fastest way she knew how. Other students were nowhere close to passing, and we didn’t roll the red carpet out for them.
The practice of focusing on kids who are close to passing has been well-documented by now. A RAND report on D3M identified this practice as one of the most common forms of data-driven decision making. In one of their studies, more than 75% of principals reported that their school or district encourages teachers to focus on these students, and between one-quarter and one-third of teachers said they actually do focus on these students.
A more expedient way to use data is to select out lower performing students before they even enter your school. In a system like New York City, where all students must apply to high school, even unscreened schools – schools that are prohibited from selecting students based on their test scores, prior grades, etc - have used data to screen out students. Until last year, all unscreened schools had access to individual students’ prior attendance, grades, their test scores, their date of birth, their address, their sending junior high schools, and their special education and English language learner status.
Interestingly, the Department of Education stopped providing this information beginning with admission for the 9th grade class of 2007. Why? One can imagine that the Dept of Ed finally figured out what many of us already knew – that some unscreened schools were using these data to pick off the best students. (For more info on the issue of creaming in NYC, see here, here, and here.)
Certainly data-driven decision making has a bright side, but it has a dark side as well, especially when schools feel intense pressure to quickly improve their scores. Scott makes the important point that we shouldn’t throw out the baby with the bath water, and I agree. But as more schools implement D3M-based approaches, we should be aware that its uses are not uniformly positive.