This post is by Caitlin Farrell, (@ccfarrell), director of the National Center for Research in Policy and Practice (@NCRPP).
Today’s post is written from the researcher perspective. Stay tuned: Thursday we will provide the practitioner perspective on this research.
Researchers, policymakers, and foundations often wonder why some research ends up being influential in school district policy and practice while other research is not. As part of a project funded by the William T. Grant Foundation, we asked over 60 district leaders in three school districts to identify pieces of research they have found valuable and to explain why that research was useful to them. From these conversations emerged a portrait of the research district leaders actually turn to in their day-to-day work—and whether you are a researcher or educator, the findings may surprise you.
Current Assumptions About What Makes Research Useful to District Leaders
Advocates of evidence use in education have long argued that the most useful research studies for school or district leaders are efficacy and effectiveness studies, as they help leaders engage in evidence-based decision making about what programs to select. This premise is shared by the recently passed Every Student Succeeds Act, which calls on leaders to use “evidence-based programs,” where “evidence” often focuses on knowledge generated by impact studies. In an environment where resources are scarce, the argument goes, policymakers need evidence that helps them choose between alternative strategies for improving achievement for all students.
Such policies present an overly narrow view of what, how, and when research might be useful for three reasons. First, it suggests that what policy makers need to know is limited to “what works.” Second, it suggests that research is useful for primarily instrumental purposes—to make decisions in the moment about program adoption. Third, it rests on a set of assumptions of what educational leaders do and how they make decisions. In this vision, district leaders primarily are in the business of selecting and adopting policies, programs, and practices that have been developed and studied elsewhere.
In our own work, we started with the idea that learning first about leaders’ ideas about what research they find useful might help us better understand their conceptions of what useful research is and when it is relevant to their practice. It’s only by understanding what research is in fact useful on the ground—versus assumptions about what research should be useful—that we can then identify and improve the bridges between research and practice. That is, market research on educational leaders’ actual practices and habits can help us better understand the “demand side” of research use.
What The Research Examines
We analyzed survey data and interviews with over 60 district leaders in three large urban districts (for full article, see here). For each study leaders identified as useful, we asked them why it was useful and identified the types of work activities mentioned in their answers.
What The Research Finds
We did find a few instances of district leaders referring to research as useful to them in the context of selecting programs. For example, in one district, leaders recognized the need for a reading assessment in early education. They partnered with local researchers to help select an assessment to pilot and study its implementation over the course of one year.
However, leaders reported using research for a much wider array of work practices beyond selecting among programs. Leaders read research in order to support their own professional learning. One leader reported wanting to “stay on top of best practices” in her field and area of responsibilities. They also engaged with research as part of efforts to provide instructional leadership to others. Sometimes, this involved sharing research within one’s own team or department or the central office. Other times, leaders used research when they designed professional development opportunities for teachers or school leaders.
A third activity where research was useful was when leaders designed district-wide strategies and programs. Many leaders reported that the research they used offered an overall framework to structure the vision and mission for the district as well as guide decision making in particular areas. One cabinet-level leader in one district said:
“We based our theory of action [in the strategic plan] around the work that Tony Bryk had done in his Lessons From Chicago. We paid attention to those drivers, and we were able to leverage our school improvement grant dollars to invest in a subset of . . . schools, really invest deeply.”
Finally, district leaders noted that many studies helped inform implementation strategies, either as a part of the implementation plan for an initiative or in order to foster buy-in in the roll-out of an initiative.
Across all of these activities, the forms of research that most leaders perceived to be most useful included only a small number of impact studies in peer-reviewed journals. Instead, district leaders found a broader range of research useful. They reported turning to scholarship that is largely conceptual or prescriptive, written by those both inside and outside of the academy, published in books, and covering a wide range of topics. Although there are specific authors who are popular across districts, it is also clear that district leaders prioritize research related to particular district-specific initiatives. For example, district leaders in a school system with a professional learning community initiative reported drawing on the work of the popular translators of research on teacher professional learning communities.
Implications For Practice
These findings suggest the need for a more nuanced understanding of the differences between district leaders’ actual engagement with research compared to conceptions of what research use should look like.
Taken together, these findings paint a picture of district leaders’ work where research—broadly defined—holds promise for broader influence than current policies promote. District leaders can and do make use of research to inform decision making, but they rely on pieces produced by scholars that provide actionable frameworks for guiding action, often from books. They draw on these resources to inform their multiple roles in school districts as designers of professional development, monitors of implementation, and more.
If policies regarding research use are to be effective, they will need to take into greater account a broader spectrum of research sources. Or, policies could promote use of research within the multiple contexts of school and district leaders’ practices. For example, we see the possibilities of the Tier 4 ESSA evidence standards (referred to as a “hidden gem” by one state leader). Under ESSA tier 4, district leaders have the opportunity to engage with research and strategies of continuous improvement to design, implement, and refine their own programs, policies, and practices. By drawing on this or other policy levers we can more effectively create bridges between policy and practice to support improvement efforts on the ground.
Thank to colleagues involved in this project, including Bill Penuel, Anna-Ruth Allen, Yukie Toyama, and Cynthia Coburn, as well as the William T. Grant Foundation (grant number 180922). Additional details are available in the full article in Educational Policy, available here).
Curious about other research topics partnerships and guest bloggers have written about for this blog? See this Guide to the NNERPP EdWeek Blog for all previous blog posts organized by research topic area to easily find other posts of particular interest to you!