Note: Andrew McEachin, an assistant professor of educational policy analysis and program evaluation at NC State University, is guest posting this week.
I break today’s discussion into two areas: the measurement of teacher and school quality and the process by which teachers and schools are held accountable.
As mentioned in a previous post, it is quite difficult to separate the within-school influences on students’ learning from the outside-of-school influences when measuring teacher and school quality. Because of this difficulty, many of the performance measures used today are tightly correlated with student and school demographics, including the growth models currently endorsed by the federal government (e.g., Student Growth Percentiles). And the Brookings report released a few weeks ago that found that teachers’ observation scores are positively correlated with their students’ prior achievement levels. Teachers and administrators in schools serving larger shares of traditionally under-served students therefore have an a priori disadvantage compared to their peers in more well-to-do settings. This disadvantage creates a disincentive for teachers to work in these schools. If teachers and schools are going to be held accountable for student outcomes, and it appears that they will for the foreseeable future, it is time to have serious, and likely uncomfortable, conversations about how to account for the variation in school contexts into our evaluation systems.
One approach is for systems to directly account for the contextual differences among teachers’ classrooms and schools. This means including controls for students’ race/ethnicity, socioeconomic status, language minority status, and so on. There are two potential benefits from this method. The first is that teachers and schools are only compared against observationally similar teachers and schools. The second is that every teacher and school, regardless of the students they serve, has a proportional chance of achieving a high rating. For more detail on this approach see some of the work done by the smart folks at the University of Missouri, led by Professor Cory Koedel, (for a non-technical intro see here and here, and for their more technical work here and here). This idea is not without criticism. Explicitly accounting for differences among teachers’ classrooms and schools can be seen as holding students to differential growth standards based on their demographic differences. For example, two schools serving different populations of students can be in the same percentile of growth (e.g., the 90th percentile) even though the school with predominately traditionally under-served students grew 10 points while the more well-to-do school grew 25 points. However, the benefit of this approach is that it may increase the likelihood of attracting talented teachers to the schools that need them the most.
We also need to leverage better the strengths and preferences of the local actors in the public school system during the process of creating and implementing teacher and school evaluation systems. The process in Kansas is a nice example. To satisfy the teacher evaluation provisions of the ESEA waiver application, Kansas has proposed an interesting process (although I cannot speak to the quality of the measures used in the system). Instead of creating a one-size-fits-all model, in its waiver application Kansas gave districts two choices. First, they can adopt a model plan developed by the state department of education. Or second, districts can propose their own plan with the provisions that teacher evaluations include at least some aspect of student achievement growth--although the share of the evaluation that comes from achievement growth is not pre-determined. This plan allows districts to respond to the local desires of their teachers, parents, students, and other stakeholders. They can decide what a good teacher looks like in their context, and develop a sound evaluation around this definition. It will be interesting to see how this idea plays out in practice.
I have been kicking around a related idea for some time. I guess you could call it a teacher report account model of accountability. Typically teachers are held accountable for an arbitrarily weighted index of value-add, observations, and so on. Using this index, teachers are placed into groups based on where their index falls within the population of teachers in the district. In this system, teachers’ ranks are vulnerable to changes in the choice of cutoffs and the weights given to measures of teacher quality. Generating multiple measures of teacher quality is a useful practice, especially if they provide teachers’ information about their practice during the school year. However, instead of weighting these measures into an arbitrary index, we can simply provide teacher report cards to principals that include teachers’ scores across the various measures, as well as their place within the comparative distribution. Principals can then decide which of the measure(s) are most important for their context, create professional development plans accordingly, and ultimately make staffing decisions based on the multiple measures of teacher quality. This idea would require that principals have the capacity to use this information to maximize their teaching workforce and it would require that principals are allowed to separate teachers if they are not making adequate progress. The plan could also help foster professional learning environments if the stakes of the teacher quality measures were reduced to more formative assessments of teacher progress. The teacher quality measures can also be aggregated up to the grade and/or school-level so parents can use the information when deciding where to send their kids. For example, for parents that value growth in test scores, they could seek the schools with the highest value-add. For parents that seek other teacher attributes, they would look for schools with higher levels of those attributes.
The teacher and school accountability world fascinates me. It is important to note that we spend too much time focusing on accountability and not enough time talking about how accountability policies fit within the larger standards-based reform (SBR). Initially, the tenets of SBR were to set learning outcomes, or standards, across grades and subject; align state policies with a common goal; provide capacity building interventions for struggling schools; support professional development; measure student outcomes; and provide schools with the local autonomy to make day-to-day decisions based on the student population they served. Only once these things were in place would states then hold schools accountable for their states outcomes. With that said, it is still a fruitful exercise to talk about the finer points of how to design teacher and school accountability policies. Tomorrow I am going to explore students’ summer and out-of-school time and the impact this has on education policy.
--Andrew McEachin