How Much Will Longitudinal Data Improve Instruction?
Earlier this week, I wrote about the challenges states face in preparing their data systems for the heavy lifting needed to implement Race to the Top and other major reform proposals.
Experts at the fall policy meeting of the American Educational Research Association's Organization of Institutional Affiliates this week echoed concerns about a disconnect between policymakers and the IT folks building these systems, but they also spoke of a deeper problem: Helping educators fish through the tsunami of data on its way for information that can help them in the classroom.
Even in districts where their performance isn't being publicly outed, "some educators are actively wary of state longitudinal data because they've seen data wielded as a weapon rather than as a source of illumination," said Jennifer L. Steele, an associate policy researcher at RAND Corp., who helped prepare the Institute of Education Sciences 2009 practice guide on data-driven decision-making.
In a separate three-year study of how superintendents, principals and teachers in California, Georgia and Pennsylvania, RAND researchers found educators and administrators used student data to determine instruction, but often not in the ways policymakers intended. For example, while about half of teachers used student data to target deeper content instruction to students, roughly the same proportion taught new test-taking strategies. Moreover, "there are nearly no studies about the causal inferences of which practices are most effective to teach educators how to use student performance data," Ms. Steele said. "We have better data than we had before, and more pressure to better align programs. The catch is that better data does not mean better decision-making."
Yet AERA executive director Felice J. Levine cautioned that professional development will not be an end in itself, and states may easily overload teachers by asking them to analyze longitudinal data. "Do we really want to make every teacher a scientist in the use of this data?" she argued.
Yet last week, Jay Pfifer told me educators have a lot to teach the data-system builders about how to make data work for the classroom. Mr. Pfifer helped design Florida's original longitudinal data system, before retiring to direct longitudinal data system programs for MPR Associates, Inc, and he said he got some of his best ideas for building the data system and its professional development by sitting down with groups of teachers.
"If you take a fairly complicated 'dashboarding' approach and teach teachers how to use it, they may use it for a little while following the release of a tool, but as soon as the school year gears up, they won't use it," Mr. Pfifer said. "If you ask teachers, 'How can we make your life easier?' ... You may find their responses aren't really around a highfalutin' dashboard approach, but around some more basic reports. I learned some things we could do that would require not an additional nickel, but would do things that teachers want in the classroom."
Let's see how many states start talking to teachers about the data they actually need and how to use it.