Professional Development

It’s Notoriously Hard to Evaluate PD. A New System Aims to Change That

By Catherine Gewertz — September 22, 2020 5 min read
  • Save to favorites
  • Print

A new process for judging the quality of professional development has made its debut, with the aim of answering a notoriously difficult question: Which PD is high-quality and which isn’t?

The Professional Learning Partner Guide got started with little fanfare earlier this month. Its first round of reviews evaluates 16 providers who offer PD in math, English/language arts, and science. Some are curriculum publishers, such as Zearn and Great Minds. Others, such as TNTP, offer PD on other organizations’ curricula.

Organizations that want to be evaluated had to submit packages of evidence, which typically ranged from 200 to 400 pages, according to PLPG officials. Teams of three evaluators examined each submission, using a 49-page scoring guide to determine if there was sufficient evidence on a range of questions, from the clarity of the program’s learning goals to whether the provider is adequately staffed, and has sufficient expertise, to offer the professional learning it specified.

No organization that publishes curricula is included on the new website unless its materials have been “green-lighted"—met all expectations—for high-quality curriculum across all grade levels in a given grade band (K-5, 6-8 or 9-12) by the rating program Edreports.org. And no PD provider is included for support on curriculum that has not been similarly green-lighted.

Unlike EdReports.org, which selects publishers to review, the PLPG depends on submissions for reviews. Companies choose to submit materials and services for evaluation.

Jennifer K. Rice, the dean of the University of Maryland’s College of Education, and an expert on teacher professional development, said an evaluation system like the PLPG has the potential to provide important clarity in a notoriously opaque field.

“There are so many providers out there. It’s become a cottage industry,” she said. “Some of it is good, and some of it isn’t good. So having the tools to help filter the kind of PD that adheres to best practice could be very important.”

A Natural Partner for High-Quality Curriculum

The Professional Learning Partner Guide emerged from a renewed focus in K-12 circles in recent years on the importance of high-quality instructional materials. Many organizations, including the Council of Chief State School Officers and Curriculum Matters, a network of district leaders committed to high-quality curricula, have worked to define and publicize that idea. EdReports.org grew out of that movement, and has become a widely used way to judge curriculum quality.

But there wasn’t a similar, national system for evaluating the quality of professional learning, said Annie Morrison, the co-founder of Rivet Education, the K-12 consulting organization that designed the Professional Learning Partner Guide. Morrison and Rivet’s other co-founder, Litsy Witkowski, both worked at the Louisiana Department of Education, which developed its own criteria for judging PD vendors.

(EdReports.org has come in for its share of criticism. Early on, after it found that most major math curricula didn’t align well to the common core, math groups attacked EdReports’ methodology, saying it produced inaccurate and misleading reviews. EdReports later revised its review process.)

The Professional Learning Partner Guide encourages district leaders to think of PD in four phases. Users of the site can check boxes to find lists of providers who’ve been approved to offer PD on adopting a new curriculum (eight so far), launching a new curriculum (13), ongoing support for teachers (13), or ongoing support for building or district leaders (13).

Janise Lane, the executive director of teaching and learning in the Baltimore city schools, and a member of the advisory panel for the PLPG, thinks that the phase division is one of the great strengths of the new system.

Most district leaders “live from moment to moment, just getting the materials [in a new curriculum] out of the box and making sure everyone knows what’s what,” she said.

But thinking about how a district’s PD needs will change in years two and three—and how teachers, principals, and central-office staffers will each need different things from PD—is important and useful, and “the marketplace [of PD] doesn’t illuminate that for you,” she said.

The Role of Curriculum Publishers

Organizers of the new website consider it highly unlikely that curriculum publishers would be approved to provide PD in the first of the four stages: adopting a curriculum, Morrison said.

“It would be a hard argument to make that publishers could be objective about adoption advice,” she said. But they’ve been approved to offer PD for the subsequent stages: curriculum implementation and support.

Some providers weren’t approved for all phases they applied for, based on the evaluators’ review of the evidence they submitted, Morrison said. In other cases, curriculum publishers are listed as approved to provide PD only on some of their products because not all have been reviewed by EdReports.org, or perhaps because they received green lights only in specific grades, Morrison said.

The first round of applications for review on the PLPG came from informal outreach, Morrison said. Rivet Education invited publishers it knew would be eligible, based on the EdReports.org ratings of their products, and asked district leaders and K-12 education groups about providers that offer good, curriculum-aligned PD, she said.

The site doesn’t currently give providers a numeric or color-coded rating, as its curriculum-evaluation role model, EdReports.org, does. But Morrison said it will add the ratings of its expert evaluators next year, along with user reviews.

The PLPG recruited evaluators nationally, ultimately choosing nine chief academic officers, principals and charter school leaders through an evaluation process that included interviews and performance tasks designed to elicit their experience leading curriculum-aligned PD, Morrison said.

The PLPG will conduct its second review cycle this winter. It remains to be seen how many big-name publishers will submit their materials. EdWeek asked the Association of American Publishers for thoughts on the new venture, but the organization declined to comment.

Photo: An educator reflects on her group’s ideas during a professional development day held in Long Beach, Calif., in 2019.

Related Tags:

A version of this news article first appeared in the Teaching Now blog.