Opinion
School & District Management Opinion

“Take Aways” from Two Months of What Works Clearinghouse Reviews

By Marc Dean Millot — September 15, 2007 5 min read
  • Save to favorites
  • Print

For the most part public schools do not develop their own curriculum or pedagogy - they buy them, primarily from private enterprise, and mostly from for-profits. I find it ironic that: 1) outsourcing the most central of public education’s core teaching and learning functions is never part of the “privatization” debate; and 2) the quality of those products services and programs didn’t become a subject of serious interest until No Child Left Behind. Yet “god forbid” we allow private enterprise to operate public schools under charter or contract, or manage teachers’ professional development. Given the discrepancy, it’s hard to believe that debate has much to do with what kids need rather than what adults want.

Speaking of quality, and the Reading First fiasco notwithstanding, program efficacy is supposed to matter today. Educators, indeed taxpayers, have a desperate need for independent reviews of program efficacy. In their absence, marketing dominates the sales race.
After a very slow start, the What Works Clearinghouse (WWC), a Department of Education Institute for Education Sciences’ activity (contracted out to a consortium of evaluation organizations) to examine the evidence providers use to claim their programs are grounded in “Scientifically Based Research” (SBR), is starting to pump out reviews across a range of subjects, grades and providers.

The sixteen reviews covered below happens to coincide with September’s two-month catch-up issue of School Improvement Industry Research and Evaluation Announcements http://www.k-12network.com(we vacation in August). There’s something to be learned from the list:

August 13, 2007:

Earobics® - The WWC found Earobics® to have positive effects on alphabetics and no discernible effects on fluency.

Ladders to Literacy for Kindergarten Student… .potentially positive effects on alphabetics and fluency and mixed effects on comprehension.

Success for All…. potentially positive effects on alphabetics and general reading achievement and mixed effects on comprehension.

Voyager Universal Literacy System…. potentially positive effects on alphabetics and potentially negative effects on comprehension.

July 30, 2007:

Doors to Discovery…. no discernible effects on oral language, print knowledge, and phonological processing.

Let’s Begin with the Letter People®…. no discernible effects on oral language and potentially positive effects on print knowledge and phonological processing.

Pre-K Mathematics - The WWC found Pre-K Mathematics combined with DLM Early Childhood Express software to have positive effects on mathematics achievement.

Project GRAD…. no discernible effects on progressing in school or on completing school.

Waterford Early Reading Level One…. no discernible effects on oral language or on print knowledge.

July 23, 2007:

SRA Real Math Building Blocks PreK…. positive effects on mathematics achievement.

July 16, 2007:

Cooperative Integrated Reading and Composition® (CIRC)…. no discernible effects for comprehension.

Peer-Assisted Learning Strategies (PALS)©…. potentially positive effects on alphabetics, fluency, and comprehension.

Beginning Reading: Read Naturally….- no discernible effects on fluency and reading comprehension.

Beginning Reading: Waterford Early Reading Program…. potentially positive effects on alphabetics and no discernible effects on comprehension.

Talent Development High Schools…. no discernible effects on progressing in school or completing school.

Early Childhood Education: Literacy Express…. positive effects on print knowledge and phonological processing, potentially positive effects on oral language and math, and no discernible effects on cognition.

Observations

1. If we take this list and divide it into a crude buyers guide figure according to programs that have some positive evidence and those that don’t, and providers who are nonprofit or for profit, here’s what we get:

Some Positive Finding

For-Profit
• Earobics
• Voyager Literacy
• Pre-K Mathematics
• Waterford Early Reading Program™

Nonprofit
• Success for All
• Ladders to Literacy
• PALS
• Literacy Express

No Positive Finding

For-Profit
• Doors to Discovery
• Let’s Begin with the Letter People
• SRA Real Math
• Read Naturally
• Waterford Early Reading Level One

Nonprofit
• Cooperative Integrated Reading and Composition
• Project GRAD
• Talent Development High Schools

There’s no great quality advantage tied to the label “nonprofit” or any great black mark to “for profit.” Both groups have programs with and without evidence of efficacy. The same for-profit provider can have programs with and without evidence – take Waterford, So can the same nonproit - Cooperative Integrated Reading and Composition is offered by the Success for All Foundation..

2. Our measures of program efficacy are crude and imperfect, but they are still a far better guide than nothing - or marketing brochures. The question now is whether Congress should allow states to use federal funds to buy programs that can’t jump this bar, when there does seem to be a reasonable range of choice. In short, if you can’t pass WWC review with at least some positive finding, can your program really be said to rest on scientifically based research, and should taxpayers be subsidizing a school’s data-free decision to buy it?

3. There’s some kind of cost-benefit analysis that needs to be done by educators before any for any purchase. Still, at any given price what matters to kids is outcomes, not whether the provider has a cash surplus after paying out expenses to deliver the program, or whether that surplus is retained by a nonprofit or passed along to a company’s owners.

4. Jim Ketelsen of Project GRAD and Jim McPartland of Talent Development were no less celebrated a few years ago than John Schnur of New Leaders for New Schools is today, and given the unfairness of scheduling could just as easily taken his place at the recent Gang of Forty-Four hearings on NCLB. And, as capable a manager as John is (and I admire him greatly), he is even less able to muster enough evidence of the efficacy of New Leaders for New Schools to meet WWC’s low bar. If Congress is really serious about the “supply side” of school reform, it has to turn its focus away from philanthropy and its never-ending array of grant-driven pilots unsullied by evaluation. It has to get serious about SBR and about the fee-driven for- and nonprofit providers offering the vast bulk of products and services aimed school improvement who are in this business for the long haul – and the vast amount of potential private sector capital that isn’t flowing to them. Congress has got to start thinking about how to stimulate this industry around “what works”, and only what works..

Related Tags:

The opinions expressed in edbizbuzz are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.

Events

Ed-Tech Policy Webinar Artificial Intelligence in Practice: Building a Roadmap for AI Use in Schools
AI in education: game-changer or classroom chaos? Join our webinar & learn how to navigate this evolving tech responsibly.
Education Webinar Developing and Executing Impactful Research Campaigns to Fuel Your Ed Marketing Strategy 
Develop impactful research campaigns to fuel your marketing. Join the EdWeek Research Center for a webinar with actionable take-aways for companies who sell to K-12 districts.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Privacy & Security Webinar
Navigating Cybersecurity: Securing District Documents and Data
Learn how K-12 districts are addressing the challenges of maintaining a secure tech environment, managing documents and data, automating critical processes, and doing it all with limited resources.
Content provided by Softdocs

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School & District Management Opinion A Good Principal Knows When It's Time to Leave
I didn’t leave my job because of burnout; I stepped away from being a school leader because it was in everybody’s best interest.
Matthew Ebert
4 min read
Conceptual illustration of someone handing off a baton to someone else over a completed puzzle.
Vanessa Solis/Education Week via Canva
School & District Management Principals Tell Politicians on Capitol Hill: We’re Burning Out
Students' mental health top principals' growing list of concerns.
6 min read
People walk outside the U.S Capitol building in Washington, June 9, 2022.
Visitors walk outside the U.S Capitol building in Washington on June 9, 2022.
Patrick Semansky/AP
School & District Management Women Superintendents Experience Bias on the Climb to Leadership
Interpersonal slights and inequities make it hard for women to land the job and stay in it.
3 min read
Woman stands in front of a staircase in different colors. She is about to walk up the stairs. Concept of standing in front of a challenge and finding the right solution and courage to move on.
mikkelwilliam/E+
School & District Management Fewer of Today's Superintendents Are at Retirement Age
A new survey of superintendents adds to what we know about the people who lead the nation's school districts.
4 min read
Conceptual illustration of money, salaries and data.
iStock/Getty