We Can't Neglect Professional Learning Evaluation
Demonstrating the impact of professional learning has never been more critical. We are at a juncture in our schools where educators are implementing rigorous content standards and assessments and experiencing new evaluation systems, even as they navigate a sea of other challenges and even as resources for professional learning are on the chopping block. For educators to be successful in ensuring that students are college- and career-ready, the support school systems provide must include high-quality professional learning. But without indications that professional learning has an impact, why would school systems continue to invest in it?
Knowing professional learning is achieving its intended outcomes is not a simple matter, but that's no excuse for neglecting its evaluation. Here I propose a three-stage approach to professional learning evaluation aligned with Learning Forward's definition and Standards for Professional Learning.
My approach is based on several assumptions. First, we must agree that the primary purpose of professional learning is to raise the performance levels of educators and their students. Agreeing on this purpose is the first step in addressing the question of how we will evaluate professional learning.
Next, I believe professional learning is the single most important strategy for improving the performance of educators and students. And, it is most effective when it is part of a comprehensive system focused on improving team, school, and system performance, as well as the performance of individuals.
However, it is challenging to isolate the effectiveness of professional development from other factors that promote school and individual improvement. Most school systems and schools lack the research capacity and time necessary to determine that a particular professional development activity produced a specific performance outcome. While they need information to make informed judgments about the results of professional learning, it is not useful to propose sophisticated research methodologies that most school systems and schools cannot and will not use. Instead, school system and school leaders need resources and tools that fit with the realities of their work environments, and enable them to gain greater understanding about whether professional development is solving their problems, improving their schoolwide, team and individual performance, and helping more students to achieve standards.
Given these assumptions, I propose the following approach in three stages.
Most professional learning consists of an educator participating in one or more structured learning experiences. Most commonly, these include (a) independent study; (b) listening to a presentation or lecture; (c) observing the real-time practice of another educator or a video of such practice; and (d) participating with peers in research, study, discussion, problem solving, or product creation.
Most educators responsible for professional development focus their attention on the organization and content of a learning experience. Rarely do school systems and schools make any effort to assess what educators learned from the experiences, or how they intend to use their learning to raise their own levels of performance or those of their students. School systems and schools must see value in making such an effort. If they do so, they can begin to evaluate educator learning by asking individuals questions following their participation in a professional learning experience.
This is not as easy as it looks and goes beyond standard post-learning experience surveys. For the questions to be appropriate, and for the people asking the questions to evaluate what participants learned and how they will likely use their learning, the questioners themselves will need preparation. However, this is a type of data collection that is feasible. Even if such questioning is conducted with only a sample of the educators participating in a particular professional learning activity, it will tell those responsible for professional learning more than they will know otherwise.
Assuming there is an expectation by school systems and schools that the educators who participate in professional learning should and will use their learning to change their practices and raise their levels of performance, there is then a need to evaluate whether and how effectively they do so. There is no substitute for periodically observing the educators' practice. Again, those responsible for conducting and documenting the observations will need excellent preparation to execute their roles effectively and productively. If this process is iterative between the observer and the educator, it has the potential to also provide data that the school system or school can use to improve its professional learning experiences, as well as to develop more realistic expectations of the educators who participate in such experiences.
(3) Analysis of student performance data
Assuming school systems and schools are definitive about how an educator's participation in professional learning should increase student performance (and if they aren't, why are they allocating significant resources to professional learning?), there is a need to determine whether and to what extent it does so. The first step is to collect data, whether that comes from results from teacher-made or standardized tests, observation of student performance, observation of students' classroom interactions, or samples of student work.
It will be necessary to determine which aspects of students' performances are most reasonably related to their teachers' professional learning, and to draw conclusions about the linkages. This type of evaluation should probably involve the teacher as one member of a three- or four-person evaluation team. The process will also require careful preparation of the participants as well as modifications over time. Even if this does not result in agreement about whether and to what extent the teacher's professional learning impacted student learning, it will cause focus, examination, and discussion to a much greater degree than now exists and certainly advance the understanding of the participants.
While these three stages have their own problems (availability of time, need to prepare people responsible for the evaluations, ongoing monitoring and refinement of the processes), this approach will engage educators because they respond to the questions and challenges educators face every day.
Though I outline these stages in part because they will provide a wealth of information most schools and school systems don't yet have and in part because they are feasible, I acknowledge that we haven't leveraged tools and structures that might help to begin this work. Learning Forward is ready to collaborate with one school or school system to co-develop and refine such approaches. Who stands with us?
Executive Director, Learning Forward