When the National Council on Teacher Quality put out its 2013 report savaging the quality of the nation’s teaching programs, critics pounced, claiming the ratings were flawed, meaningless, and should be ignored.
Two years, two reviews, and countless back-and-forths later, does that criticism hold up? Well, yes and no, according to an independent study released today by the University of North Carolina at Chapel Hill.
Overall, the council’s infamous four-star ratings didn’t have a discernable relationship to teaching quality, raising questions about whether it makes sense overall for programs to try to improve their performance on the standards.
But higher scores on two of the group’s 19 standards—teacher selection and using outcomes data—did seem to predict which programs produced better teachers, at least in North Carolina. And higher scores on at least two other benchmarks, of middle school content and classroom management, bore a negative relationship to principals’ opinions of good teaching.
How’s that for “mixed findings”?
The study, by Kevin C. Bastian of the University of North Carolina at Chapel Hill and Gary T. Henry of Vanderbilt University, was funded by the North Carolina General Assembly Administration’s Teacher Quality Research Initiative. The NCTQ participated in the study voluntarily, turning over all of its scores, indicators, and rating criteria to the researchers.
For the study, the researchers looked at program graduates who were in their first or second year of teaching in North Carolina public schools 2011-12 or 2012-13. They connected each college’s graduates to two measures of teaching abilities: value-added scores based on standardized tests, and teacher-evaluation ratings issued by principals. Then, they looked for correlations between each program’s ratings on 15 of the council’s standards.
Here’s the overall summary of that series of tests:
Note that these findings vary quite a lot from standard to standard. In all, the number of tests that found no relationship between rating and outcomes was far higher than those that did find a relationship. (That doesn’t mean those topics are unimportant—just that these particular outcomes don’t seem to be explained by the variations the NCTQ documented.)
In a few cases, the relationship did seem significant:
- Six of 10 tests found that programs that use outcomes data to inform their programming tended to do better as measured by student test scores;
- Stronger selection measures showed a relationship with both teacher-evaluation and the test-score results (this finding, however, related only to the use of SAT scores for selectivity, not GPA requirements).
- On the other hand, principals gave lower ratings to teachers who had what NCTQ deemed strong training in middle school math content and classroom management.
These are not particularly easy findings to interpret. That’s in part because, as the authors note, what the NCTQ was judging programs on was not always conceptually well aligned with what principals were looking for on the state’s teacher-evaluation framework.
Jayne Fleener, the dean of North Carolina State’s college of education, said that the study points to the challenges of measuring what really matters in teacher preparation.
“We all feel strongly that the student-teaching experience is really important, and I think one of the results or implications of the results is that just counting the number of visits or observations [by a supervisor] doesn’t seem to be related to other, later teacher success variables,” she said. “So the question is: What are the things we should be looking at?”
Findings in Context
The study is among the first to attempt to link a particular accountability framework for teacher preparation to program outcomes.
Accountability for teaching programs is among the most polarizing topic in our field today. For example, new standards promulgated by the specialty accreditor for teaching programs are now being questioned in some quarters. The American Association of Colleges for Teacher Education, which led the attack on the NCTQ project, also dislikes a federal proposal to grade programs. Meanwhile, almost every program gets a thumbs-up from its own state, rendering those audits useless for research purposes.
The North Carolina study is particularly interesting when taken in light of the debate over selection. The finding here suggests that the accreditor could be on the right track in calling for stronger selection standards.
The authors caution that the findings are limited to North Carolina colleges.
The NCTQ largely welcomed the findings.
The group said it will make revisions to its review as a result of the study. It plans to delete its middle school content standard, for example, and will put more weight on the gauge of whether programs collect and act on student data. The next review is due out in 2016.
“At the broadest level, we’re taking to heart the report’s suggestion to give greater emphasis to the quality of elements that the NCTQ believes make for strong design of teacher preparation over the quantity of those elements,” it said in a statement.
See also:
- Disputed Review Finds Disparities in Teacher Prep
- As Criticism Continues, NCTQ Changes Some Teacher-Prep Scores