The Letter From: Getting SES Providers Past the Tough Times Ahead
To appreciate where Supplementary Educational Services (SES) need to go, I believe we must proceed from first principles.
In market democracies, the economy exists to enrich the whole society. It is not a perfect system, indeed it is seriously flawed – except, to borrow from Winston Churchill, when compared to every other approach to economic organization mankind has tried to date. Profit is the means by which our society encourages individuals and firms to employ their capital for the general good – remember Adam Smith’s invisible hand. Regulation is the means by which society places limits on competitive impulses, replacing "buyer beware" with some sense of fairness in economic transactions.
For the most part, the idea that regulation must address a dual criterion of profit and fairness is transparent across the American economy. No one objects to the need to regulate pollution in industrial processes, assure product safely in toys or food, or police unscrupulous lenders in the home mortgage business. For the most part, we are only reminded of the need for such rules by reports of egregious violation.
So it really should have been no surprise that in creating a market for school improvement, No Child Left Behind (NCLB) would employ standards for both school performance in the form of AYP and school improvement provider performance in the form of Scientifically Based Reading Research (SBBR), Scientifically Based Research (SBR), and Research Based (RB) products, services and programs. For the common good, the law imposed a meaningful basis for accountability on state and local governments’ monopoly over public education. Accountability to Adequate Yearly Progress (AYP) opened up possibilities for the private sector to provide public education services directly to public school students. It also incentivized public schools managed by state and local government to purchase a better class of products and services for teaching and learning. To protect society from worthless offerings, and attract new private sector providers and capital into a preexisting market overshadowed by the major publishers, NCLB set some guidelines for the Department of Education to regulate in the area of educational efficacy.
The federal government pursued the AYP regulation with great vigor. It essentially dismissed regulation around SBBR, SBR and RB. As I pointed out last week, if the Department of Education had been serious in the development of reasonable standards for program evaluation, Supplemental Educational Service (SES) providers could not have more or less ignored it for so many years, and states would not have learned over the last year how to manipulate standards to dispense with their school districts’ competitors in the near future. If student performance truly matters, providers will deliver it. If it is not a priority, objectives like growth and profitability will move to the fore. If dominant players like school districts are allowed to use their market power to squelch competition by whatever means they can, they will. The Department’s job was to pursue SBBR/SBR/RB regulations to channel the natural impulses of providers and school systems towards ends that profit society. This Administration didn’t do its job, and the result is an SES program that may not survive the next election.
It has been incredibly difficult for this Administration to recognize how important regulation is to nurture a nascent school improvement industry that is vulnerable not only to a local government monopolies, but also to the k-12 education industry’s dominant providers. (For more on this go here.) Much of the resistance is ideological and experiential – for the most part we are talking about people who have spent their lives trying to deregulate a government monopoly. A lack of experience in the regulation of markets outside of education left this group unprepared to concede the need to set floors on quality as a means of attracting investment to goods and services aimed at replacing those offered by established providers. There was the Post Office-like legacy of political spoils in the Department - replacing what was in place with a new set of party loyalists. There was the impact of what appeared to be victory in the reading wars, favoring the broad concept of phonemic awareness, rather than the demonstrated effectiveness of specific products and services. In short, “the market” was given pride of place in political discourse, but - in an unfortunate but real sense - the left was uncomfortably close to the mark calling it simply a code word for privatization. Conservatives have had a vision for “tearing down this wall” of public school monopoly, but not for the development of the school improvement market required to leave no child behind. Simply put, NCLB I was far ahead of the Bush Administration’s learning curve.
If there is to be a meaningful future for SES, a reauthorized NCLB must address two shortcomings of his Administration’s policy on evidentiary standards. First, if there is to be a real market in school improvement, NCLB II must make up for years of lost time by assuring that the next Administration gives practical meaning to the terms Scientifically Based Research in regulations governing the purchase of school improvement products, services and programs. Second, and probably more important for SES providers, given the damage that has been done to the SES concept by governments’ and providers’ failure to take evaluation seriously, the program needs to proceed on an entirely new basis. I’ve addressed both in edbizbuzz and my podcast, School Improvement Industry Week Online. Portions of my postings are excerpted and hot linked below.
A useful standard of program efficacy from the October 24 School Improvement Industry Week Online:
Senators Jeff Bingaman (D-NM) and Richard Lugar (R-IN) have introduced S. 2118, establishing basic requirements for demonstrating the efficacy of products, services and programs purchased under No Child Left Behind. School improvement providers should endorse their efforts.
NCLB I was intended to hold the private sector accountable for its role in student outcomes by restricting schools’ use of federal funds to the purchase of products, services and programs whose efficacy has been demonstrated by “scientifically based research” (SBR). A plain reading of the definition in Section 9101(37) reveals that Congress wanted to end business as usual:
The term ‘scientifically based research’... includes research that— (i) employs systematic, empirical methods ... (ii) involves rigorous data analyses that are adequate to test the stated hypotheses ... (iii) relies on measurements or observational methods that provide reliable and valid data across evaluators and observers, across multiple measurements and observations, and across studies by the same or different investigators; (iv) is evaluated using experimental or quasiexperimental designs in which individuals, entities, programs, or activities are assigned to different conditions and with appropriate controls... with a preference for random-assignment experiments, or other designs to the extent that those designs contain within-condition or across-condition controls; (v) ensures that experimental studies are presented in sufficient detail and clarity to allow for replication or... the opportunity to build systematically on their findings; and (vi) has been accepted by a peer-reviewed journal or approved by a panel of independent experts....
When it comes to SBR, the Department has not only lacked vigilance, it has essentially ignored the law. The effects of this negligence have been hard on organizations whose business strategy rested on a belief that, at least when it came to purchases under NCLB, results will beat established marketing channels and brands. S. 2118 merely spells out standards for evaluation that define what legitimate school improvement providers have always done, will always do, and reasonably assumed NCLB mandated for everyone else.
SEC. 9101(37). RESEARCH PROVEN PROGRAM....
(A) IN GENERAL. The term “research proven program” means a program that is determined to be a qualified program pursuant to to subparagraph (B), and that is evaluated in not less than two studies, both of which meet the following minimum criteria:
(i) The program was compared to a control group using alternative or traditional methods.
(ii) The study duration was not less than 12 weeks.
(iii) Program and control schools were equivalent at pretest in achievement (within 0.5 standard deviation). Analyses of posttest differences are adjusted for pretest differences.
(iv) The post-test measures used to compare program and control groups is a valid standardized or criterion-referenced test, such a State accountability test, and is not inherent to the program. For example, tests made by program authors, or tests of content not studied by control students, do not qualify.
(v) The sample size of each study is not less than 5 classes or 125 students per treatment (10 classes or 250 students overall). Multiple smaller studies may be combined to reach this sample size collectively.
(vi) The median difference between program and control students across all qualifying studies is not less than 20 percent of student-level standard deviation, in favor or the program students....
Absent provisions that end research-free business practices, real school improvement firms will stay on the margins of supply. Expert debate and discussion may adjust S. 2118’s standards. Still, if school improvement is your business, you can back them in principle. If you can’t, what business are you in?
A new basis for SES from the May 6, 2007 edbizbuzz:
If the future of SES in NCLB is decided on "the merits" - demonstrated improvements in student performance to date, thus endeth the program. And with it much of an industry that depends on a single federal funding stream. Alternatively, if it is decided purely on politics, adversaries of SES and NCLB will never stop pointing out the “double standard” of holding schools accountable for student performance, but then taking money from those schools and giving it to private providers in general and SES providers in particular who need not show that they can make edcationally significant improvements in student test scores. Hillary Clinton’s equating of SES providers and Halliburton will become the cry of any Democratic contender for the presidency.
Research and evaluation to date suggests that tutoring is most likely to have an impact on student performance is when it is tied to the classroom closely. Moreover, to be honest, many providers would rather work for the district as a contractor than compete against it under SES. Indeed, this “pull out” educational model was the business model adopted by Sylvan Learning Systems' Contract Services Division for targeted Title I programs in the top 50 school districts long before SES was thought up. It was pretty successful against business and academic criteria.
A new SES program might have some of the features of the old Comprehensive School Reform Demonstration Program. Grant funds would be "fenced off" from Title I for districts with students not demonstrating proficiency in schools on the verge of “in need of improvement.” Districts might propose to use grant funds to involve one SES provider to maximize the strength of ties between student, classroom teacher and tutor, or multiple tutors to get more of the benefits of parental choice. The competition for grant funds presumably would improve the likelihood of "supportive operating environments," allowing the tutoring firms to focus on product development and refinement, including a better understanding of the kinds of programs best suited to different kinds of students.
Such a program would be smaller than the potential spending under today’s SES provisions in NCLB. But, as a practical matter, the market might be just as big and better concentrated - and perhaps more profitable, if for no other reason than marketing/customer acquisition costs would drop significantly. It would also be accompanied by ongoing research and evaluation at the provider and school levels. Rather than killing off an idea that doesn’t seem to be ready for prime time, it would allow the most promising SES programs an opportunity to survive and grow.
The downsides of this option are several. 1) SES providers will have to admit they are not ready for prime time, and they may not be able to muster sufficient political support for a transition to the new approach in a reauthorized NCLB. 2) When "the client" moves from many parents to one district, the market becomes a "winner take all" game and many providers will perish. 3) The ongoing research requirement will kill off more because districts will still be looking for the provider "best able/most likely" to improve student test performance. 4) Potential losers will try to prevent potential winners from making the move to the R&D program option, splitting the SES community.
Still, something like this approach is probably the best hope for SES providers and the SES segment of the school improvement industry overall. Current course and speed basically leads to a demonstration of failure. Putting the program in the "R&D" category moves it into a category where failures of performance are more tolerable as a means of long-term program improvement. Making the funds for SES available to districts puts them in control of SES providers, where they seem more comfortable and maybe even supportive of the SES idea.
For any of this to be implemented, we’ll need a cadre of activists in Congress, the White House, and above all the Department of Education – that favors a well-ordered market over any provider, program, philanthropic trend, or political connection. In short, we’ll need to change federal public education policy from campaign spoils to professional service.