June 2010 Archives

The Institute of Education Science's study on intensive teacher induction is getting a lot of buzz, and at least one scholar is worried that it may be giving an inaccurate picture of the effects of mentoring.

The study found positive effects in both reading and math that, relatively speaking, are quite large. As the New Teacher Center's Liam Goldrick points out on his blog, the effect sizes for mentoring given in this report are larger than those produced in other large-scale randomized studies, such as one on Teach For America.

But Jonah Rockoff, an associate professor at Columbia University's School of Business who's studied induction programs in New York City, said to be cautious about the results. For one, he noted that despite these large achievement boosts, the data don't show any other effects that would seem to confirm the results. Teachers didn't report feeling any more prepared, for instance.

He also directed my attention to this factor: The researchers used "covariates" to control for the effects of teachers' background characteristics and other factors on the data. Without such controls, the effect estimates drop and are no longer statistically significant for reading. You can find this on p. 95 of the report.

By Rockoff's read, covariates shouldn't be necessary in an experiment if the "treatment" and "control" group are really appropriate comparisons. One problem with large-scale education studies, he said, is that teachers often change classes, grade levels and subjects. In this case, as the population of teachers declined due to attrition and other movements, only a small population of teachers were tied to at least two years of student-achievement data. And if the declining group of teachers in the study resulted in a material difference in the composition of the treatment and control group, then that might skew the data.

"The big issue is whether treatment and control groups still look like one another, among the subset of teachers with multiple years of data," Rockoff told me.

UPDATE: Steven Glazerman of Mathematica, one of the number guys behind the study, submits this response:

"We agree with Jonah Rockoff that one should be cautious about interpreting the test score impact findings. We sounded that note of caution in our report and in its executive summary because the results are not robust to alternative ways to estimate the program's impact. However, the most credible estimates adjust for chance differences between student and teacher characteristics, including students' achievement before they entered the study teacher's classroom. These show a positive and significant impact on reading and math in two-year districts and no impact in one-year districts. Rockoff is correct that it is not necessary in a true experiment to adjust for these chance differences by including covariates. However, failing to do so would cause us to ignore data and rely on a less precise estimate. In this case especially, when the test score analysis sample is small, the precision gains are substantial: the standard error associated with our estimates decreases by 29% in a model with covariates compared to a model without covariates."

Posted by Stephen Sawchuk at 2:02 PM | Permalink | Leave a comment | Recommend

Politics K-12 has the potentially explosive news that House Democrats want to rescind money from several of the Obama administration's key reform programs, including the federal Teacher Incentive Fund, in order to help finance $10 billion to preserve education jobs. TIF supports local performance-based compensation systems for educators. Other programs they're eyeing include the Race to the Top Fund and a charter school innovation

The important implication here is that congressional Dems, and especially David Obey, the chairman of the House Appropriations Committee, aren't on board with the direction Obama and Education Secretary Arne Duncan are moving in education reform. Obey, in a hearing earlier this year, clearly telegraphed that he thought preserving jobs was more important than advancing reform.

Some folks are wondering if this is even legal. But my understanding, and please write in to let us know if this isn't correct, is that as long as discretionary funding isn't actually obligated or disbursed to a grantee, it's well within Congress' purview to yank it back. And we know from past experience with TIF that appropriators do pay attention to what agencies spend. Back in 2006-07, the Education Department dawdled in obligating its first TIF dollars, and appropriators refused to give the fund any more until the first batch was out the door.

The big winner in this situation? The teachers' unions, who have been pushing for an education jobs fund and have found lots to criticize in TIF, the Race to the Top, and charter schools. Cynics will no doubt claim that House Dems, in addition to putting a big "Kick Me" sign on Arne Duncan's back, are engaging in bit of pre-election pandering to the unions.

UPDATE: AFT officials say that while they support the bill, the union preferred the earlier $23 billion proposal that didn't cut TIF or RTT. The union did not know in advance which programs would be offset.

All of this still has a long legislative path to navigate before it's enacted. First, the proposal must make it through the Rules Committee, and then past the House floor and the Senate.

But no matter what happens, it's a sign that there's trouble in edu-paradise.

Posted by Stephen Sawchuk at 10:48 AM | Permalink | Leave a comment | Recommend

That's the bottom line of this new study out from the U.S. Department of Education's Institute of Education Sciences. It's an important study because the experimental design allows us to conclude that it was the intensive, two-year structured mentoring that "treatment" teachers received—and not some other factor—that led to these boosts.

But what's more striking is that this is the third-year report of an ongoing study, and neither of the first two years of study found any effect on student achievement. In fact, this year of study occurred after all the "treatment" schools no longer received the intensive support.

That leaves us with this perplexing question: Why did it take so long for the effects to show up? Does it, perhaps, take a while for teachers to emulate the good teaching practices they see during the induction years?

Overall, this is good news for induction programs, but there is a caveat: The report found that the programs are having "no effect" in keeping these teachers in their districts or in the professional longer.

Teachers may be more effective with this support, but they're not sticking around any longer, and that raises some real cost-benefit questions.

Check back soon at edweek.org for a full writeup.

Posted by Stephen Sawchuk at 1:01 PM | Permalink | Leave a comment | Recommend

Here's an important story from the Chicago Sun Times: The district has just approved a policy for laying off teachers that would dismiss tenured teachers rated "ineffective" before dipping into the ranks of higher-rated novices.

Newly elected Chicago Teachers Union head, Karen Lewis, says the policy is illegal; the district has said that a provision in state law allows it to deviate from the seniority-based layoff system spelled out in the contract. As this Chicago Tribune story explains, the two parties disagree about whether the state or the contract holds the trump.

In general, this is yet another example of the budget crunch calling into question some long-standing teacher-policy structures. But in this case, there's another important subtext: Lewis is part of a group from within the CTU that criticized the former president, Marilyn Stewart, as being too compliant with the district. Lewis has since asserted in the local media that things like charter schools and standardized testing reflect a desire by businesses to make money and to blame teachers for poor scores. Her election was viewed in some quarters as a reaction by rank-and-file teachers to elements of the Obama administration's reform agenda.

Lewis also said that the district's current evaluation system isn't appropriate. But the district is currently piloting a new evaluation system that apparently does a better job distinguishing performance. It will be interesting to see Lewis' views on the pilot—and whether she will support its expansion.

Posted by Stephen Sawchuk at 4:36 PM | Permalink | Leave a comment | Recommend

Denver's ProComp pay program may have helped attract more-effective teachers to the district and boosted retention in hard-to-serve schools, according to a report on the much-discussed system released recently by the University of Colorado at Boulder. Teachers opting into the program also appear to be slightly more effective on the whole.

The analysis was based on student and teacher data from eight school years, from 2001-02 through 2008-09. (ProComp began in 2005-06 , with opt-in periods for teachers in 2006, 2007, 2008, and 2009.)

Researchers compared each student's results with those of other students with similar achievement histories and traced the data back to the students' teachers. They also examined value-added data linked to specific teachers. And they reviewed data from surveys of teachers and principals in the district.

Here's a rundown of the findings:

— Student achievement improved over the time period studied, but all teachers appear to have seen increased achievement in the years subsequent to implementation and there was little evidence to indicate that those who opted into ProComp were more "productive" on average at boosting achievement than those who did not.
— Teachers who opted into ProComp do appear to have slightly outperformed their peers, but it's not clear whether this is because of ProComp itself or because of the self-selection of more-effective teachers to join the program.
— Teachers hired after ProComp went into effect consistently show higher first-year achievement in both reading and math than those hired prior to the program. The findings persist for those teachers' first three years, suggesting that ProComp may have served to attracted more able teachers to the district.
— Retention rates generally increased over the time period studied; schools with median levels of teachers in ProComp had higher rates of retention overall.
— Hard-to-serve schools with higher levels of teachers participating in ProComp experienced a sharp increase in retention rates in the first full year of ProComp and the greatest increase in retention, from 74 percent to 86 percent, again suggesting a possible ProComp effect.

Now, I know you're all wondering how this jives with that other big performance-pay study that came out recently, on the Chicago Teacher Advancement Program, which had no positive findings.

But it's important to note that these aren't comparable studies, and that neither is conclusive about the impact of performance pay. One of the reasons is because of major differences in how the programs were studied. The Chicago study used an experimental research design, allowing us to draw some cause-and-effect conclusions.

This Denver study is based mostly on observational data. That means that ProComp, student scores, and teacher hiring may be related, but the data don't say definitively whether the program actually caused those changes.

It is also worth pointing out that, like TAP, ProComp is far more than a performance-pay program. Both programs also include professional-development aspects and pay for other aspects—in TAP for taking on additional roles and responsibilities, and in ProComp for taking advanced coursework, teaching in hard-to-staff subjects, and working in high-need schools and fields.

The individual teacher-pay component of ProComp based on student-achievement results is less heavily emphasized than in TAP, boosting salaries only a little more over 7 percent of the base-pay level. (Compare that with 9 percent for advanced degrees and about 13 percent for incentives to take on the tougher assignments.)

In effect, with both studies, it's hard to determine exactly how bonuses for advancing student achievement might have affected the field separate and apart from all the other moving pieces.

The authors of this report note that an upcoming study will describe outcomes "at a finer level of granularity to better understand differential outcomes of the program's various elements," so let's hope more is coming our way on this. Stay tuned.

Posted by Stephen Sawchuk at 10:24 AM | Permalink | Leave a comment | Recommend

Once again proving the importance of having union backing, former Georgia Gov. Roy Barnes is actively courting teachers' association support in his bid for a comeback in the Peach State, according to the AP. He's even saying he might back off from a plan to put some teacher-pension funding into riskier investments.

While serving as governor, Barnes in 2000 advanced plans to dismantle due process for teachers. The move led to his loss of state teachers' association support, a situation that many credit with his subsequent re-election loss in 2002.

We just witnessed the inverse of that situation in Florida, where Gov. Charlie Crist vetoed an extremely unpopular bill that would have all but done away with tenure, and was rewarded with an endorsement by the Florida Education Association in his upcoming Senate race.

The Georgia news is particularly interesting given that Barnes co-chaired the Aspen Commission on No Child Left Behind. The group released a series of recommendations for renewing NCLB, one of which included denying Title I school positions for teachers deemed ineffective on test-score growth measures.

And while that recommendation didn't mess around with tenure, it wasn't a million miles away from Florida's SB 6 or all of the other state laws that have since moved in a similar direction.

Maybe the unions are aware of this, too. Despite the full-on courting, the Georgia Association of Educators has declined to endorse Barnes so far.

Posted by Stephen Sawchuk at 12:07 PM | Permalink | Leave a comment | Recommend

The Pittsburgh district and its American Federation of Teachers-affiliated local union have reached agreement on a five-year contract that contains three significant new pay elements: a school-based performance-pay plan, a pilot individual performance-pay plan, and a salary schedule that puts much more emphasis on student results rather than teacher credentials.

The school-based bonus plan will award bonuses to staff at schools that make the most progress in lifting student achievement (within the top 15 percent in the state).

The individual performance-pay element, a voluntary pilot program, will be based on two factors: a mandatory demonstration of student growth through value-added measures or other yet-to-be-determined growth measures, and a "choice" component giving teachers credit for high performance on the teacher-evaluation system, demonstrated leadership, and providing professional development to others. Teachers could earn up to $8,000 additionally through this program. And participating teachers could choose to opt out of the program when the pilot ends.

One interesting provision says that teachers at the top of the salary schedule who elect to participate in the individual performance-pay program would forfeit the raise at that step in exchange for an increase in their base salary equal to 40 percent of the bonus amount.

Various members from the union and district will meet to work out all the details in these plans, with both the schoolwide and individual programs to begin in 2011-12.

Finally, beginning July 1 of this year, new teachers will no longer enter the system on the old salary schedule but on a new one that's a bit of a hybrid between a traditional schedule and a performance-pay plan. Teachers would still earn "step" increases each year, but master's degree pay bumps would be a thing of the past.

Big pay boosts would come by satisfying a periodic review based on a combination of evaluation score and proof that a teacher's advanced student growth. (Just how this will be measured has to be determined by yet another committee.) It works like this: After meeting the four-year tenure mark, and each third year thereafter, teachers would be placed into one of four "professional growth" levels, the highest of which tops out at $100,000 annually. And after a certain number of years, they could qualify to earn between $10,000 and $14,000 in extra pay for taking on additional responsibilities for working with the highest-need students and working longer hours.

If this sounds complicated, it really is. Think of it as a bit of a cross between Cincinnati's defunct pay program of the mid-2000s, and this pay plan in Harrison School District Two, in Colorado Springs, Colo. In any case, the plan would offer beaucoup bucks for teachers who perform well, participate in the individual pay plan, and take on additional roles.

And as an aside, it's interesting to note that this contract went through relatively quietly compared with some of the other big contracts of late, as in D.C. And so far, AFT national has refrained from promoting the heck out of it as it did in New Haven, Conn., and elsewhere, even though this contract seems to reflect AFT Pres Randi Weingarten's favorite theme, collaboration.

In any case, Pittsburgh is one of the Bill & Melinda Gates Foundation's sites for its Intensive Partnerships for Teacher Quality, which has required a lot of relationship-building and commitment from both district and union. Perhaps that made all the difference here.

Posted by Stephen Sawchuk at 4:01 PM | Permalink | Leave a comment | Recommend

Results from year one of a pilot teacher-evaluation system in Chicago show a much broader range of ratings under the new system than under the district's existing one, with at least 8 percent of pre-tenured teachers receiving at least one "unsatisfactory" rating, according to a new paper out from the Consortium on Chicago School Research.

Although Chicago is not the only district putting a new teacher-evaluation system in place, it is certainly one of the few that's paying a lot of attention to implementation, studying it, and documenting the results. The system, based on Charlotte Danielson's Framework for Teaching, was rolled out in 44 schools in 2008-09 and expanded to 100 in 2009-10. The data here is from the first year of implementation and used "matched" observations to determine whether an administrator and external observer gave the same rating.

According to the data, over a third of teachers received all "proficient" or "distinguished" scores from their principals on the various strands of the observation framework, and about a third received a mix of "basic" and "proficient" scores.

Consistency among the parties performing the teacher observations was high in the aggregate, but less so on individual strands of the evaluations and in terms of outliers. For instance, at the high end of the ratings scale, principals were more likely than observers to give the highest rating of "distinguished" versus a rating of "proficient."

And before you go thinking that's an odd finding given all the rhetoric out there about principals, about 30 percent of principals were actually more severe graders on the whole than observers. Complicated stuff, and important to keep an eye on because of the high-stakes implications of inconsistent ratings.

Once you've digested the summary, read the much longer full report on the first year of implementation.

One thing to also keep in mind is that that the policy stakes here are huge. As I wrote in this story, the reform of teacher-evaluation systems has in mind the goal of connecting the results to other aspects of the teacher-quality continuum—things like pay, promotion, professional opportunities, tenure-granting, and dismissal.

So, if the teacher evaluation piece doesn't come together, the chances are these other elements won't, either.

Posted by Stephen Sawchuk at 8:41 AM | Permalink | Leave a comment | Recommend

Yesterday's post on the as-yet-unseen Bennet teacher bill got me thinking a bit about what other pieces of teacher legislation could be candidates for inclusion in a revised Elementary and Secondary Education Act draft. Here are a few proposals that might have legs.

• In what is, to my knowledge, the first actual bill to propose addressing the Title I comparability 'loophole,' Rep. Chaka Fattah, D-Pa., has introduced the ESEA Fiscal Fairness Act. One provision of note specifies that this change does not endorse or require the forced transfer of teachers, one of the concerns of teachers' unions. This bill could still be a tough sell on the Hill, but a bunch of groups have lined up in favor of it.

• Three senators introduced a bill to overhaul professional-development spending in the ESEA. Rep. Carolyn McCarthy, D-N.Y., has a similar (though not identical) bill in the House. Read my earlier item for some analysis on the proposal.

• Rep. George Miller's Teacher Excellence for All Children Act was the basis of the House education committee's proposed Title II teacher-quality rewrite in 2007. He once referred to it as covering a "soup to nuts" approach to teacher quality. It doesn't appear that Miller has reintroduced it this legislative session, but don't count it out yet.

• Late in 2009, Sen. Patty Murray, D-Wash., introduced a bill authorizing $2 billion for districts to reduce class sizes and, surprise, it's very similar to class-size language the National Education Association included in its own ESEA reauthorization proposal. The conventional wisdom has it that the halcyon days of federally subsidized class-size reduction are gone. But with a Democratic president in office and both chambers of Congress controlled by Democrats—for now—a class-size program might be an important bargaining chip with which to curry support from liberal Democrats and teachers' union for an ESEA rewrite.

Now—am I missing anything crucial? Write in and tell us all.

Posted by Stephen Sawchuk at 2:29 PM | Permalink | Leave a comment | Recommend

Over at Politics K-12, Alyson Klein has a very interesting item up about Sen. Michael Bennet, D-Colo., who recently introduced a school-turnaround bill. While some folks are poring over that, though, here in the teacher-quality universe the question on the table is: What the heck happened to Bennet's teacher bill?

For months, the senator was said to be working on a teacher-quality bill that would no doubt have generated a lot of attention, given his closeness to the current administration on education issues. As the former superintendent in Denver, he's also had a lot of experience working with teachers.

For now, this bill seems to be on the backburner. Alyson surmises that may be because of his precarious position in the upcoming elections and a desire to avoid tension with core constituencies like teachers' unions.

We do, however, have a few clues as to what might be in this bill. A few months ago, at a Center for American Progress event, I asked Bennet to talk a bit more about it, and here's what he had to say:

"I think that work is principally going to revolve around how we think about Title II [of the ESEA] and whether we think that Title II is really a very effective use of resources or whether there might be other ways to use it to engage our teachers, to engage more senior teachers as master teachers to bring people into the profession.

I think we need to think very strongly about alternative licensure regimes in this country so that we're getting the benefit of all the talent that we have in the United States of America to do the most important work that we have."

That's pretty broad stuff, but in her item Alyson notes that one proposal floating around was the idea of a national teacher license to ease teachers' ability to work across state lines and create a larger pool of talent for hard-to-staff schools or subject areas. This sounds quite similar to the America's Teacher Corps idea proposed by a high-powered group of scholars not long ago.

Interestingly, in a recent paper, Linda Darling-Hammond, an education professor much beloved by teachers' colleges, also floated the idea of a national teaching license predicated on a performance-based teacher assessment. The idea apparently also has support from the American Association of Colleges for Teacher Education and the National Education Association, which commissioned the paper.

Posted by Stephen Sawchuk at 4:42 PM | Permalink | Leave a comment | Recommend

Michele "High Bar" McNeil has been doggedly following the latest Race to the Top news, this debate over "side deals" in some Florida districts. The big question is whether these side agreements essentially compromise the "buy in" of local district and unions who signed the state memorandum of understanding.

More from Eduwonk here, and Sherman Dorn at his blog thinks we're all off in left field.

Dorn makes some important points, explaining that some of the apparent redundancy in these local "side agreements" has to do with the fact that the scope of bargaining in Florida differs from that in other states, like New York. But he thinks this is largely a process issue and that the agreements don't substantively differ from the state MOU.

By my read, though, at least one of these local agreements does, in fact, change the contours of the local support. The state MOU, for instance, assumes that locals will negotiate all terms of the state reform plan into their bargaining agreements, or forfeit their share of the grants. When the grant expires or funding runs out, all that happens is that this MOU or agreement to work together also runs out. Nothing happens to reforms that have been successfully negotiated .

But under the terms of the local agreement, any new contract language actively reverts back to its old structure. In other words, the reforms would cease to exist in legal terms.

The word on the street is that local unions felt this was necessary to guard against new programs that aren't funded. Historically, unions have good reasons for being worried about the sustainability of things like performance pay. But it also means that a program that is indeed sustainable would disappear nevertheless.

These agreements—the Hernando one at least—could put a stopper on the duration of any new reforms, and that's a problem when the state is going to be getting extra points for supposed buy-in of local districts and unions on its application.

Posted by Stephen Sawchuk at 10:37 AM | Permalink | Leave a comment | Recommend

The National Institute for Excellence in Teaching, which oversees the popular Teacher Advancement Program school-reform model, is commissioning its own study to figure out why its Chicago site had disappointing results, compared to some of its other sites, according to this release.

Interactive, Inc., an Ashland, Va.-based education program evaluator, will try to determine which variables in Chicago might have led to the results, the NIET group said.

The study is being hailed by some in the field as the death knell for performance pay, but that's probably a bit premature for a couple of reasons. As I noted in my story, TAP has lots of other pieces aside from bonus pay, including job-embedded professional development and a career ladder and added responsibilities for "master" and "mentor" teachers.

And unlike other TAP sites, Chicago only implemented grade and subject-level pay, rather than differentiation based on individual classroom performance, with which more districts are now beginning to experiment.

Those factors may—or may not—have affected matters here. I, for one, hope Interactive provides some additional insights into this most complicated of reform areas.

Posted by Stephen Sawchuk at 4:25 PM | Permalink | Leave a comment | Recommend

After getting clobbered in the first round of the Race to the Top, Oklahoma has managed to pass some aggressive pieces of legislation in preparation for round two. Among them is a bill that makes major changes to the state's teacher evaluation and tenure systems.

It's similar to legislation that passed Colorado, in that teachers who score at the "ineffective" level on the new instrument for two years running could be dismissed.

The state also is poised to join the expanding group of states basing 50 percent or more of a teacher's evaluation on student academic progress. (Like many of those states, test scores will only make up part of that 50 percent; the other part will be based on some other academic measure.)

But what's really interesting is how little press this seems to have gotten. The state teachers' unions apparently supported its application for the Race to the Top, though not all local affiliates did. Still, this bill seems to have gone through with little of the public hand-wringing from unions that characterized such efforts in Tennessee and Rhode Island and Maryland, to name but a few.

Posted by Stephen Sawchuk at 4:05 PM | Permalink | Leave a comment | Recommend

New York City Mayor Michael Bloomberg plans to rescind layoff notices and save over 4,000 teacher jobs, but it's going to come at a cost: The elimination of planned raises, The New York Times reports.

The head of the city's teachers' union, Michael Mulgrew, put out a statement that says, emphatically, that the United Federation of Teachers has not agreed to this proposal, and intimates that the move might not pass legal muster.

The mayor "does NOT have the power to unilaterally decide on the teachers' contract, and we have reached NO agreement on his proposal to freeze teacher pay," Mulgrew said (emphasis his). "If the Mayor has concrete ideas on the next contract, he and his representatives should bring them to the bargaining table at the Public Employment Relations Board, where our contract is currently in mediation."

Bloomberg has, according to the NYT, steadily increased public employees' salaries over his tenure.

Sounds like Mulgrew would like to see both the layoffs rescinded and such raises preserved. Hard to say whether that's a budgetarily sound idea, but in his statement, Mulgrew suggests freeing up more cash by using an early-retirement option for teachers and dipping into a surplus that's been earmarked to cover a loss of state education revenue.

Despite this snafu, Mulgrew and Bloomberg are scheduled to head to Albany together to lobby for more education spending. You've got to love New York politics.

Posted by Stephen Sawchuk at 12:36 PM | Permalink | Leave a comment | Recommend

From guest blogger Dakarai I. Aarons:

District of Columbia teachers could approve the much-watched tentative agreement their union signed with Chancellor Michelle A. Rhee tomorrow. The 15-day voting period ends tomorrow morning, and the results will be released late tomorrow or early Thursday.

Head over to the District Dossier blog for more details and keep your eyes on edweek.org—we'll bring you details as soon as we have them.

Posted by Stephen Sawchuk at 10:55 AM | Permalink | Leave a comment | Recommend