« The Boston Pilot/Charter School Study: Some Good News, and Some Cautions | Main | Private Practice, Public Health, and the Autism/Vaccine Debate »

The Skillful Publicist


When a school district makes a big to-do about the use of "evidence to make decisions about how to help students learn, where to put our resources and how to manage our staff," is it fair to criticize it for implementing unproven and experimental programs? skoolboy supports modest experimental innovations, as long as they are evaluated carefully before expansion to a scale that would encompass an entire population. After all, students and teachers aren’t guinea pigs. The fact that schools are failing is not a justification to do any old thing, on the assumption that any innovation will be better than the status quo.

Speaking of any old thing … The Washington Post reported earlier this week that the Washington, DC Public Schools are abandoning support for National Board certification as a means of teacher professional development, shifting instead to, among other things, the Skillful Teacher program marketed by Research for Better Teaching, Inc. (RBT), founded by Jon Saphier in 1979. The program consists of a series of six one-day workshops; you can buy the book, which approaches 600 pages, for $70 at Amazon.

You might think that an organization that’s been peddling professional development for 30 years, with a book in its sixth edition, would have some compelling evidence of the effects of the jewel in its crown on teaching practice and student learning. If professional development doesn’t result in improvements in teaching and learning, what’s the point? But the RBT website doesn’t point to much evidence, emphasizing testimonials and brief "stories." skoolboy’s favorite is the account of Fairfax County, VA’s implementation, "Making Teacher Evaluation Substantive and Growth-Oriented." "In the first year of implementation, 162 teachers were dismissed or resigned compared to single digits the previous year," the website crows. Now, is that growth, or is it development? Sometimes skoolboy gets confused by the difference.

The blurb for Montgomery County, MD, on which the DC plan is based, touts an independent evaluation by Dr. Julia Koppich of the program’s effects on teachers and administrators, and claims that "in 2001 grade 2 students scored in the 68th percentile in math computation. In 2003 scores were in the 83rd percentile." The inference is that the Skillful Teacher program produced this change, but any reader of this blog knows that demonstrating program impact requires a careful design to rule out alternative explanations of changes over time in outcomes. (It also helps to have a good theory of how a program might plausibly produce particular changes.)

Montgomery County’s own internal evaluations of Studying Skillful Teaching aren’t as positive. Although 3rd grade teachers and Alegebra I teachers who took the course are "more likely to teach mastery lessons and less likely to miss opportunities to positively impact student learning" than comparable teachers who did not (Merchlinsky, 2006, 2007), there were no effects on elementary reading and math test scores or algebra performance.

And lest anyone think that people who live in glass houses shouldn’t throw stones, The Skillful Teacher isn’t the only professional development initiative that skoolboy, who teaches at the home of the Teachers College Reading and Writing Project, thinks could benefit from more rigorous evaluations before scaling up.


Am I reading this correctly? Are you saying the Teachers College Reading and Writing Project could "benefit from more rigorous evaluations"?

You have my complete attention. Keep talking.

Let me put it this way, Robert: I don't think you'll be seeing the TC Reading and Writing Project showing up in a What Works Clearinghouse review anytime soon.

I would love to see more rigorous evaluations of the TC Reading and Writing Project. While the program has some strengths it has some major weaknesses. Add to that the fact that my former school was using the 3-6 curriculum in middle school grades 7-8 (with students who had already utilized it for years) and you will understand why the teachers rebelled and fought for a new curriculum.

Skoolboy seems to have drunk the coolaid that says we can only consider bumps in test scores as evidence of efficacy. Back to AERA re-education camp. What about the value to a school system of having a language to anlyyze classroom practice, a language for teachers and administrators to talk about the quality of teaching and learning? What about the fact that teachers in Montgomery and Fairfax Counties value the collegiality and professionalism created through the Skillful Teacher? Why counterpoise National Board with The Skillful Teacher? They both begin with the same core philosophy. Many of us in DC who have been critical of Michelle Rhee, who feel the urgency of providing support for quality teaching, think that her plans to bring in The Skillful Teacher are the most encouraging development yet. It's true that she's wrong to reject the National Board for Professional Teaching Standards. But on the Skillful Teacher, the problem is that the investment planned is too small and too slow. Quantitiative reserch studies are fine, but everything of value doesn't have to raise test scores in and of itself.

This just in...Existence of AERA re-education camp confirmed. Film at 11.

Mark, I think most of us are willing to accept measurements of efficacy other than bumps in test scores. Can you show the data confirming your claims of the somewhat less tangible positive effects of The Skillful Teacher?

An article in yesterday’s Washington Post [ http://www.washingtonpost.com/wp-dyn/content/article/2009/01/04/AR2009010401534.html ] only mentioned one benefit of the Skillful teacher program: Saphier said the program “fosters teachers' BELIEF [emphasis added] in their power to lift student achievement despite conditions outside school” and that research shows that it’s changed the BELIEFS [emphasis added again] of Montgomery County teachers who took the course.

Back in April, on PBS, http://www.pbs.org/newshour/bb/education/jan-june08/challenge_04-02.html Cheryl Krehbiel, the DC Schools Professional Development deputy said this:

CHERYL KREHBIEL, District of Columbia Public Schools: We have a number of teachers who I don't believe will ever believe that kids can learn at high levels. And those are the teachers we need to move out quickly, rapidly, at whatever cost.

JOHN MERROW: Can you quantify -- I mean, what percentage of your roughly 4,000 teachers feel this way, have this problem?

CHERYL KREHBIEL: Fifty percent don't have the right mindset. And there's the possibility that more of them don't have the content knowledge to do the job.

So there you have it! The most important outcome is teacher "belief" – and they’ve quantified it -- or so they say.

The Montgomery County Professional Growth System, of which Saphier's Research for Better Teaching, "The Skillful Teacher" course and book, and the "Observing and Analyzing Teaching" course for administrators were central components, has been extensively studied. For four years, Dr. Julia Koppich led a collaborative evaluation team that also involved researchers from George Washington University. The one-year and four-year reports are available online. Data came from surveys, focus groups, case studies, and data analysis. The Professional Growth System, as a whole, has been considered a tremendous success, and the RBT training was singled out by principals in surveys and focus groups as being particularly valuable to their ability to communicate with teachers about teaching. The one concern expressed about the skillful teacher training for teachers was that not all teachers were able to experience it directly, some received the effects indirectly through the work of staff development teachers in their schools. While we don't have data on the effects of "the skillful teacher" in isolation, the value of Dr. Koppich's reports is that they reflect the broad experience of a school system that created an infrastructure through which teachers and administrators have been taking the craft of teaching very seriously. http://www.montgomeryschoolsmd.org/departments/development/documents/jk.pdf

I think most of us are willing to accept measurements of efficacy other than bumps in test scores.

I'd be interested in hearing what other measures are looked at. I'm don't find test score bumps very compelling, but not much else gets discussed.

I think improving communication among school staff is a worthy goal of a training course. If that's what it does, that's how it should be marketed - not as improving student performance or anything else that hasn't been adequately measured.

I do not think teacher indoctrination is a worthy goal of any teacher training course and this seems to be why the "Skillful Teacher" was attractive to the Rhee administration.

Thanks, Skoolboy, for another incisive and, umm, skillful dissection of the D.C. Improvement Juggernaut (great illustration, too). And while I hate to disagree with the estimable Mark Simon, National Board Certification has little in common with skill-building programs like the eponymous "Skillful Teacher."

National Board Certification is a rigorous, standards-based assessment of accomplished teaching. While more than 90% of National Board candidates say that it provides useful insights into effective teaching, it was not designed as professional development for novice or ineffective teachers. Which, if you believe Michelle Rhee, are endemic in the D.C. schools.

It's a shame that the National Board program was dropped in D.C., because urban schools also need exemplary, experienced veteran teachers to provide models and collegial leadership. Michelle Rhee is incorrect in stating that there research showing that the students are not positively impacted by having National Board Certified Teachers. Says who? The National Academies:


As usual, Nancy, you're right. NBPTS is a comprehensive national assessment of teaching practice whereas Skillful Teacher training is simply a useful tool for building professional culture. I only said that they both come from the same philosophical place. It is not a choice. Montgomery County maintains a huge commitment to the NBPTS process and uses the Skillful Teacher tools.

this seems to support my original premise that Rhee does not care what teachers know, she just cares what they believe - so she chucks a tool that measures ability and keeps one that supposedly molds beliefs. I'm guessing that the DC version of the training will have a special module on that. Probably neither course is tied by solid data to student achievement, but she uses that as an excuse to dump the course not designed to change teacher mindset.

this is starting to sound like the evolution/intelligent design argument

skoolboy supports modest experimental innovations, as long as they are evaluated carefully before expansion to a scale that would encompass an entire population.

Amen to that, skoolboy.

The year-four 2004 evaluation report from Montgomery County is interesting because it draws from a lot of data and describes a multi-pronged effort. It is hard to separate out the components to attribute causation, for example to "Studying Skillful Teaching". But the overall approach seems to be validated.


This post was not intended as a referendum on the Professional Growth System developed during your tenure as the president of the Montgomery County Education Association; there are many admirable features of the PGS. I was specifically criticizing the lack of evidence of the effects of Studying Skillful Teaching on student learning. Did you look at the evaluation reports I cited in the post? They address this specifically, and are more recent than the report prepared by Dr. Koppich.

I do have to say that I am not very impressed by teachers' self-reports of changes in their practices. We all have the capacity to internalize the rhetoric of reform initiatives, but observational studies frequently do not demonstrate changes in behaviors that correspond to the self-reports.

Of course I read and am familiar with the Merchlinsky studies you cited. And I agree that objective observation of changed practice is better than self reported impact. My point, evidently not communicated effectively, was that there are lots of things impacting teachers in MCPS at the same time. Therefore, its hard to single out cause and effect from one element, in this case, Studying Skillful Teaching. The effects of the course extended to teachers who didn't take it through other aspects of the Professional Growth System. Tools and methods from the SST course were brought to teachers who never took the course by staff development teachers assigned to each school. In addition, the teachers taking the course in MCPS are not a random sample of teachers. Teachers either self selected into the course or were required to take it.

What is troubling to me about your post has nothing to do with my tenure in Montgomery County. It is that from your academic perch, you chose the moment when Michelle Rhee is actually thinking of investing in DC in some deep training for teachers in the craft of teaching, to question the efficacy of the program she has chosen. You cite one small study with questionable data, but did not cite other more favorable data. I simply posted the Koppich 4-year study (which I had trouble finding).

It is not true that Michelle Rhee is dumping NBPTS in favor of Skillful Teacher. You made that up. There is no connection between the two. The upshot of your post might be that the Chancellor will decide that none of this training for teachers in the deep knowledge base of teaching can be proven to change teacher behavior, so best to stick to test prep, which is much of what schools have been doing on her watch to date.

Hi Mark,

Glad that the comments allow us to clarify matters. The point I was trying to make was: (a) Chancellor Rhee is championing a culture of evidence-based decision-making; (b) she's proposing introducing The Skillful Teacher as a key mode of professional development; (c) I can't find any credible evidence of the effect of The Skillful Teacher on student learning that is independent of teachers' self-reports.

I pointed to MCPS because it's one of the cases on Research for Better Teaching's website, and it was mentioned in the Washington Post article. I fully appreciate that there were, and are, many factors influencing both teacher professional development and student learning in MCPS, and indeed it's not MCPS' responsibility to demonstrate that the Skillful Teacher has effects on student learning -- it's RBT's. Shouldn't an organization that styles itself "Research for Better Teaching" have actually done some research on the program it's hawking?

You state that, "It is not true that Michelle Rhee is dumping NBPTS in favor of Skillful Teacher. You made that up." What I wrote was, "The Washington Post reported earlier this week that the Washington, DC Public Schools are abandoning support for National Board certification as a means of teacher professional development, shifting instead to, among other things, the Skillful Teacher program marketed by Research for Better Teaching, Inc." You were quoted in the article; do you think Bill Turque got the facts wrong?

And this brings me back to my point that Rhee's main, if not only real interest in "Skillful Teacher" is that it changes teacher mindset, as Saphier stated in the article, "...the program fosters teachers' belief in their power to lift student achievement despite conditions outside school."

What does it take to realize that Rhee is seeking mind control -- teachers' "belief in their power" -- over any research-based programs that actually improve student achievement or teacher skills.

Maybe because it sounds so wacky and far-out that even though it's written in plain English, you can't accept it? Even though everything she's done so far has been by executive fiat and not evidence-based research? How deep is the spell you're under?

For further information about her plans for the schools, read an article in Sunday's Post, in which a principal hired a whole new faculty of only of teachers who “…survived rigorous interviews designed to weed out those who weren't up for lots of extra work….” (without extra pay, I might add.) This is her plan - exploiting teachers in brave, new ways - especially the young idealistic ones. http://www.washingtonpost.com/wp-dyn/content/article/2009/01/10/AR2009011001772.html

E Favorite,

Keep those posts coming. You hit the key factor of why Rhee's approach can't work. As long as they keep the "thought police" mentality, evidence-based ddecision-making is impossible. Nobody will dare allow unpleasant news to go up the chain of command. That's why data-driven decision-making is largely incompatible with data-driven accountability. Create a "culture of accountability" especially with her plans for rapid change and you will have the same type of culture of compliance as was produced by SovietFive Year Plans. We pretend to teach and they pretend to learn, just so long as we make the numbers look good.

This is a shame because Rhee seems to be sincere. But poor children deserve the same benefits that flow from a culture of inquiry.

Education needs all types of people with all types of personalities and worldwiews. You can't turn teachers into robots or cogs in a wheel.

Rhee's vision reminds me of Vonegut's dystopia where ballet dancers have chains on their ankles. Would there be any room for Vonegut or Orwell in Rhee's school system? I'm reminded of the story of Locke High School where, as I recall, some TFA teachers saw Brave New World as the satire it is while others saw it as a concept that they sought to emulate.

John Thompson – Ha – I remember that Vonnegut story – it was one of my favorites. I wonder if Rhee has read it.

Good point about the thought police mentality making evidence-based decision-making impossible. I hadn’t thought of it that way. Did you know that she talks about her system being “data-driven” all the time, but hasn’t shown any data for anything yet? In fact all her firings so far have been because the people in question “didn’t fit.” The “fitting in” criteria has become more clear in a couple of recent WaPo articles that describe hiring practices used by her hand-picked principals. For instance, in a school that Rhee opened to journalist Jay Mathews, http://www.washingtonpost.com/wp-dyn/content/article/2008/11/30/AR2008113001929.html, the principal “…said he wanted a school full of ambitious, young teachers ‘before they were jaded.’ So he hired just two with more than five years of experience.”

New hires were asked what they would do to raise kids’ low reading and math scores. The kind who said this got the job: "Every kid can learn, and we all say that, but what is missing is the last part of the sentence: Every kid can learn given the motivation, given the supports, given the expectations. I will be motivating my kids, I will be giving my kids the support and I will be expecting them to do it." That is, the teacher committed to being completely responsible for her students’ progress. Applicants who didn’t get the job, “…suggested that those social ailments [undereducated parents and impoverished homes]would be hard to cure.”

In other words, teachers who think they are miracle workers were hired and teachers with a grasp on reality and the guilelessness to express it were not. Certainly teachers should have a positive attitude, in terms of trying their best to help their students learn, but it’s frightening to think people are not getting hired because they say the wrong words, when those words are fact-based. I suppose it will get around pretty fast that you have to say certain things and avoid certain things to get a job. Then, if you’re selected, your reward is a job that no human can ever do – and isn’t expected to, really. The whole system is set up on on the assumption that the young idealists who take these jobs will burn out in a few years and move on, to be replaced by the next group.

I’m waiting for the journalist who will do a thorough investigative, analytical report on Rhee’s methods.

E Favorite: You make some good points about the general attitude in hiring teachers in some under-performing schools: Expect miracles, and then expect the teachers to burn out. From my experience as a former teacher, I think one of the best things to do to help teachers avoid burnout is to assign them fewer classes each day and more prep periods (if possible). I've heard that some other high performing countries take this approach - it's more like a college professor, I imagine.

In addition, it helps to assign newer teachers only one subject at a time (e.g., don't give a new middle school teacher two periods of 8th grade English, one period of 7th grade English and two periods of 6th grade world history). In my experience, the reality was usually the opposite: new teachers were not given any additional prep periods, and were often given several different subjects to teach at once (the "left over" classes after the scheduling had been completed earlier that summer). This made no sense from a retention perspective and led to more teacher burnout, in my opinion.

Comments are now closed for this post.


Recent Comments

  • Attorney DC: E Favorite: You make some good points about the general read more
  • E Favorite: John Thompson – Ha – I remember that Vonnegut story read more
  • JOHN THOMPSON: E Favorite, Keep those posts coming. You hit the key read more
  • E Favorite: And this brings me back to my point that Rhee's read more
  • skoolboy: Hi Mark, Glad that the comments allow us to clarify read more




Technorati search

» Blogs that link here


8th grade retention
Fordham Foundation
The New Teacher Project
Tim Daly
absent teacher reserve
absent teacher reserve

accountability in Texas
accountability systems in education
achievement gap
achievement gap in New York City
acting white
AERA annual meetings
AERA conference
Alexander Russo
Algebra II
American Association of University Women
American Education Research Associatio
American Education Research Association
American Educational Research Journal
American Federation of Teachers
Andrew Ho
Art Siebens
Baltimore City Public Schools
Barack Obama
Bill Ayers
black-white achievement gap
books on educational research
boy crisis
brain-based education
Brian Jacob
bubble kids
Building on the Basics
Cambridge Education
carnival of education
Caroline Hoxby
Caroline Hoxby charter schools
cell phone plan
charter schools
Checker Finn
Chicago shooting
Chicago violence
Chris Cerf
class size
Coby Loup
college access
cool people you should know
credit recovery
curriculum narrowing
Dan Willingham
data driven
data-driven decision making
data-driven decision-making
David Cantor
Dean Millot
demographics of schoolchildren
Department of Assessment and Accountability
Department of Education budget
Diplomas Count
disadvantages of elite education
do schools matter
Doug Ready
Doug Staiger
dropout factories
dropout rate
education books
education policy
education policy thinktanks
educational equity
educational research
educational triage
effects of neighborhoods on education
effects of No Child Left Behind
effects of schools
effects of Teach for America
elite education
Everyday Antiracism
excessed teachers
exit exams
experienced teachers
Fordham and Ogbu
Fordham Foundation
Frederick Douglass High School
Gates Foundation
gender and education
gender and math
gender and science and mathematics
gifted and talented
gifted and talented admissions
gifted and talented program
gifted and talented programs in New York City
girls and math
good schools
graduate student union
graduation rate
graduation rates
guns in Chicago
health benefits for teachers
High Achievers
high school
high school dropouts
high school exit exams
high school graduates
high school graduation rate
high-stakes testing
high-stakes tests and science
higher ed
higher education
highly effective teachers
Houston Independent School District
how to choose a school
incentives in education
Institute for Education Sciences
is teaching a profession?
is the No Child Left Behind Act working
Jay Greene
Jim Liebman
Joel Klein
John Merrow
Jonah Rockoff
Kevin Carey
KIPP and boys
KIPP and gender
Lake Woebegon
Lars Lefgren
leaving teaching
Leonard Sax
Liam Julian

Marcus Winters
math achievement for girls
meaning of high school diploma
Mica Pollock
Michael Bloomberg
Michelle Rhee
Michelle Rhee teacher contract
Mike Bloomberg
Mike Klonsky
Mike Petrilli
narrowing the curriculum
National Center for Education Statistics Condition of Education
new teachers
New York City
New York City bonuses for principals
New York City budget
New York City budget cuts
New York City Budget cuts
New York City Department of Education
New York City Department of Education Truth Squad
New York City ELA and Math Results 2008
New York City gifted and talented
New York City Progress Report
New York City Quality Review
New York City school budget cuts
New York City school closing
New York City schools
New York City small schools
New York City social promotion
New York City teacher experiment
New York City teacher salaries
New York City teacher tenure
New York City Test scores 2008
New York City value-added
New York State ELA and Math 2008
New York State ELA and Math Results 2008
New York State ELA and Math Scores 2008
New York State ELA Exam
New York state ELA test
New York State Test scores
No Child Left Behind
No Child Left Behind Act
passing rates
picking a school
press office
principal bonuses
proficiency scores
push outs
qualitative educational research
qualitative research in education
quitting teaching
race and education
racial segregation in schools
Randall Reback
Randi Weingarten
Randy Reback
recovering credits in high school
Rick Hess
Robert Balfanz
Robert Pondiscio
Roland Fryer
Russ Whitehurst
Sarah Reckhow
school budget cuts in New York City
school choice
school effects
school integration
single sex education
small schools
small schools in New York City
social justice teaching
Sol Stern
Stefanie DeLuca
stereotype threat
talented and gifted
talking about race
talking about race in schools
Teach for America
teacher effectiveness
teacher effects
teacher quailty
teacher quality
teacher tenure
teachers and obesity
Teachers College
teachers versus doctors
teaching as career
teaching for social justice
teaching profession
test score inflation
test scores
test scores in New York City
testing and accountability
Texas accountability
The No Child Left Behind Act
The Persistence of Teacher-Induced Learning Gains
thinktanks in educational research
Thomas B. Fordham Foundation
Tom Kane
University of Iowa
Urban Institute study of Teach for America
Urban Institute Teach for America
value-added assessment
Wendy Kopp
women and graduate school science and engineering
women and science
women in math and science
Woodrow Wilson High School