edbizbuzz

For several weeks' the Letter has explored the proposition that accountability in public education requires standards, consequences and due process.

On paper, No Child Left Behind holds providers accountable for the value added by their offerings to student performance. Most providers must demonstrate evidence in the form of program evaluation under the law’s Scientifically Based Research (SBR) provisions; Supplementary Educational Service (SES) providers are held to its lower Research Based (RB) standard. Offerings that do not meet these standards are not eligible for purchase with federal fund allocated by NCLB to states, districts and schools. There is at least the inference of due process, in that the U.S. Department of Education and its state counterparts are required to develop rules, regulations, processes and administrative procedures for implementing the law’s provisions.

On the ground, it is hard to say that what the federal and state education agencies have implemented meets the definition of accountability I've proposed. The Department of Education has failed to convene the evaluation community for the purpose of recommending definitions, methods and measures on which the private sector can rely. It is unsurprising then, that as a matter of administrative practice, SBR covers everything from offerings subjected to third party evaluations, following the scientific method and state of the art statistical tools, covering hundreds of schools over many years, to those with evidence amounting to an essay claiming a relationship to a body of ostensibly relevant research.

It is hard to argue that NCLB intended something so broad that it has had no impact on anything sold to schools for perhaps a hundred years. Imagine a Food and Drug Administration formed to get tainted meat off the market, whose regulations couldn't close one meat packer.

Absent real standards, the consequences for providers with or without products, services or programs whose proof of efficacy is demonstrated by direct evaluation have been arbitrary and capricious. In the case of SES, many states seem to have gone from no review of program efficacy, straight to a much more rigorous take on SBR than the feds, and in the process arguably violating the law by leapfrogging the lower RB requirement. In the implementation of NCLB’s Reading First program, SBR amounted to the proclamation of a third- or fourth-level U.S Department of Education official. And as that fiasco demonstrated, the lack of due process has only been redressed by the Department’s Inspector General, the House and Senate Education Committees, and the Justice Department.

On the educator/Adequate Yearly Progress (AYP) side of NCLB, thousand of schools feel a real impact from an Administration enforcement policy that amounted to "just say no" to any state agency requests for the slightest modification through the era of Rod Paige. Secretary Spellings has been more open, but is negotiating down from a very high bar. The result has been a potential demand for effective products, services and programs from the private sector.

On the demand side of the equation however, the Administration has hardly managed SBR and providers to incentivize the highest quality of supply. Indeed, both Paige and Spellings have made it difficult for providers with relevant capacities to survive, let alone thrive, let alone blow away the academically ineffective competition.

Who benefited from this approach to policy? Clearly not the firms who believed that NCLB’s SBR provisions would be implemented, nor the investors who bought into those firms because they believed the law would give them a competitive advantage over the entrenched players reviewed last week – the multinational publishers, grant-based nonprofit technical assistance providers, and independent consultants. In effect, NCLB did very little to change the k-12 market – except to create the new, unstable SES segment, and a new Reading First funding stream captured by the old line publishers. Firms committed to demonstrating efficacy through rigorous evaluation exist, but while their success might have something to do with the spirit of NCLB, it also happened despite the law’s administration.

This may also be heard as a podcast here.

Posted by Dean Millot at 10:42 AM | Permalink | Comments (0) | TrackBacks (0)

In 2001 a new President with great momentum offered the Democrats a grand compromise in federal policy towards public education - demanding standards for a lot more money. Once the Democrats lost control of the Congress, the standards were in place, but the money didn't quite show up.

Eight years later a lame duck with little bargaining power, quite likely to be replaced by a Democrat, asks for approximately the same deal. Democratic constituencies that care about No Child Left Behind (NCLB) - and have the power to influence votes, know they will get a far more advantageous deal if any Democrat wins the White House, and nothing worse from any Republican.

The 2008 national election exerts enough of a draw on legislators' energies and the Congress has enough on its plate to push NCLB to the end of its "to do" list. From the perspective of providers and investors in the school improvement industry NCLB I will last to at least the end of the Bush Administration, and NCLB II will be more than marginally less advantageous. Between now and the inaugural, everything NCLB is so much trivia for eduwonks. The question for the industry is how to cope with the next new uncertainty.

Posted by Dean Millot at 10:23 AM | Permalink | Comments (0) | TrackBacks (0)

Contracts to Turn Around New Orleans' Failing Schools

From the January 28 issue of K-12Leads and Youth Service Markets Report

Announcement: Nationwide search for groups interested in partnering with the department to reform public schools Due February 15 (Jan 24) Louisiana Department of Education

Their Description: The Louisiana Department of Education (LDOE) is interested in identifying potential companies... that may be interested in operating persistently failing Louisiana public schools....

8. Operate all aspects of the school.

My Thoughts: EMOs and CMOs have a tough business as it is, but the sector’s growth lies in the nation’s worst-performing schools. This bunch may be the toughest of the tough challenges, yet there is a ready, willing and able buyer. Here is “where the fish are.” Go for it. ••••

Posted by Dean Millot at 10:00 AM | Permalink | Comments (0) | TrackBacks (0)

Federal Safe Schools/Healthy Students Program

From the January 28 issue of K-12Leads and Youth Service Markets Report

Announcement: Safe Schools/Healthy Students Program Due March 14 (Jan 24) U.S. Department of Education

Their Description: The Safe Schools/Healthy Students program (SS/HS) supports the implementation and enhancement of integrated, comprehensive community-wide plans that create safe and drug-free schools and promote healthy childhood development....

Additionally, the preliminary MOA must.... (d) Describe how multiple and diverse sectors of the community, including parents and students, have been and will continue to be involved in the design, implementation, and continuous improvement of the project; and (e) Include, as an attachment, a logic model (a graphic representation of the project in chart format) that identifies needs or gaps and connects those needs or gaps with corresponding project goals, objectives, activities, partners’ roles, outcomes, and outcome measures for each of the SS/HS elements.

My Thoughts: The nonacademic, social aspects of public education agencies’ in loco parentis function are beginning to multiply, coalesce and receive greater funding. The areas of student life run from the immediate life-threatening risks of violence, drug use, and disaster to life choices around exercise, eating habits and sexual activity.

Teaching and learning in the academic sense is the core function of public schools. In these other arenas schools have special responsibilities but no unique claim to expertise or capacity. However, youth are congregated in schools and a good portion of what students need to know and be able to do here can not only be taught, but integrated with traditional academics.

These programs will become more important. Over time they hold some prospect of bringing the social services, health care, police, juvenile justice, emergency planning, and recreational agencies budget to bear on youth in a less haphazard way than today. School improvement providers should be thinking about gaining a toehold in this emerging market segment.

Posted by Dean Millot at 9:49 AM | Permalink | Comments (0) | TrackBacks (0)

As an advocate for a market in school improvement products, services and programs, Iembrace marketing. I have criticized a view of marketing that deemphasizes the merits of whatever is being offered and "fit" with the buyer and instead focuses on pictures of happy children and vague but exciting copy.

Marketing materials include everything from postcards to "white papers." On the latter end of the spectrum, we should see a reasonably balanced case for the broad product or service category, important considerations for buyers, and some sense of what's for sale. Granted, these are not third-party reviews, but they should add value to the consumers decision making process. The smartest providers know that in the long run, bad matches lead to poor results and a questionable reputation does not add to the bottom line.

The ideal white paper is easier to specify than write. I recently took my hand to the task for the Center for Digital Education, MPR and Gateway on the subject of one-to-one computing. (See here.)

Given that I feel free to critique others, it's only fair to give others a shot at me. Comments are welcome.

Posted by Dean Millot at 8:31 AM | Permalink | Comments (0) | TrackBacks (0)

Prompted by ongoing stories of pharmaceutical companies that declined to publicize studies suggesting their profitable drug lines might be less than helpful to users, This Week in Education’s uberblogger Alexander Russo asked whether foundations suppressed unfavorable research on the effectiveness of their grantee's educational programs. (See January 18.)

Since then, there have been a few technical comments on methodology. I think Alexander hoped to spark a more expansive discussion.

In that spirit, I offer some observations based on personal experience.

Perhaps the most extensive and expensive review of education programs funded by philanthropy was RAND’s eight-year roughly $10 million evaluation of New American Schools (NAS) and its $100+ million acquaintance with comprehensive school reform design grantees. I experienced the ongoing review as a member of the RAND team, the lead grants officer and later Chief Operating Officer of NAS, and the President of NAS’s lending and equity investment arm, the Education Entrepreneurs Fund. Since that time, I have made a habit of following and summarizing program evaluation and evaluation issues and tracking relevant grant RFPs for subscribers to my firm's web-based publications School Improvement Industry Week and K-12Leads and Youth Service Markets Report.

1. Serious evaluation of products, services and programs to improve student learning is a recent practice and remains unusual - whether developed with grants from philanthropy or private investment. Third-party evaluation has been especially unusual as an activity built into the terms of philanthropic grants. Government grant programs now invariably call for an evaluation plan, but not philanthropy – nor private equity.

2. To the extent program evaluation has been built into grants, it has been focused on initial development and small-scale implementation. The challenge, of course, is taking the program from a field lab experiment to a scaleable system or even a replicable model. (As I see it, replication answers the question "can this model be reproduced?" Scale addresses far more demanding question "can it be replicated in large numbers simultaneously?" See Chapter 16 here, here, and also listen starting here.) So many additional factors come to bear on scale and even replication, that the field evaluation is barely relevant to the larger “what works” question.

3. When a foundation is deciding whether to commit itself to a program developer, it should be trying to determine potential efficacy. When it makes a grant, it should commit the grantee to certain benchmarks, including some related to efficacy. I have never heard of the former taking place, and doubt it has ever been seriously attempted. Most philanthropic due diligence is laughable. I have seen the latter only in NAS contracts with its design teams – and they proved very hard to enforce. Most grantees and most foundation program officers know that the name of the game is over-promising and then stretching out the funding. Once the foundation has made the grant, it is not a disinterested third party – it and the program officers who recommended the grant have a vested interest in success, and very little incentive to point out bad decisions. (Moreover, foundation staff are rarely held accountable for their grant decisions or recommendations. Tom Vander Ark's departure from Gates may or may not have been related to the disappointing return on his "investment thesis" - the sometimes popular "small schools" "theory of change.")

4. It is important to understand the market in which nonprofit educational program developers operate. The nonprofit revenue model is based on grants, not fees. The business is not really developing educational programs to take to scale, it is addressing the interests of grant-making customers in philanthropy. So it is not correct to characterize the role of philanthropy as analogous to venture capital - providing seed money that will lead the grantee to a larger market and a new source of revenues. For most nonprofits, philanthropy is the market. As such, the "sale" of grants is not based much on a nonprofits' records of program efficacy or scaleabilty, but on topical trends in education philanthropy, personality and salesmanship of grant seekers, and pack movement in grant making. The most successful educational nonprofits are managed by leaders who know when one education trend is about to play out, what the next trend will be, and how to craft and deliver conceptually compelling proposals.

5. Philanthropies interests are not pragmatic, except to the extent that they must give 5 percent of their total value away every year - they are ideological. All foundation giving is informed by its social philosophy, policy and politics. This is not an accusation of wrongdoing, it's a recognition of philanthropy's social purpose. So it is no surprise that there is almost certainly more post-hoc evaluation funded for political purposes, than prior evaluation for due diligence, or contemporaneous evaluation for consumer awareness. The studies vary in quality, but in the end, no reasonable observer can take the findings at face value. All must be deconstructed to identify their bias.

6. Finally A) developers have a poor understanding of how and why their programs work in one school, let alone do or do not work in hundreds, and B) the state of the evaluation art is still primitive relative to the demands of demonstrating efficacy at scale. As a consequence, the net result of prominent evaluators and tiny oufits, biased and unbiased, small and large scale, old and current program reviews amounts to the Scottish legal finding “not proved” and the evaluator's old standby “promising.”

The main reason why even “not proved” is good enough to keep going with program development is that we know the current system isn’t meeting society’s needs. But until something like the requirements implied in the Scientifically Based Research provisions of No Child Left Behind legislation are given meaning in administrative law and Department of Education regulation and practice - and enforced as law, I don’t expect much to change in philanthropy’s approach to evaluation.

Postscript:

The problem of method. Even high-quality program evaluations are problematic. All studies are constrained by budget, time, and the cooperation of the schools selected for review. Longitudinal studies are bit like committing to a heading determined by dead reckoning. Once the research plan is set, changing circumstances of budget, school support, and the like are difficult to accommodate. (See, for example MDRC's recent study of the Institute for Learning's district wide strategy.) At least as important is that the "model" being tested is not a set combination of inputs, the fidelity of implementation matters, and few studies attempt to accommodate the real problem of results accountable to differences in developers' "client selection," teacher buy-in, the varying expertise and experience of the designer's staff, or less well-understood factors like students' learning styles. All of these features are absolutely vital to the question of scale, because the goal of reaching large numbers of students is not synonymous with "one size fits all." (For a recent effort to consider fidelity of implementation see here.)

Moreover, models do not remain fixed. They evolve as experience in client schools identifies problems and opportunities. The model evaluated at the start of the study is not the model implemented at the end of the study, nor any year in between. Indeed, "models" are not really what's being evaluated, unless we incorporate the dynamic of organizational capacity and management strategy into the definition. What's really being tested is the ability of an organization to address all the factors that determine student performance in the schools where it is engaged.

Models are not "pills" taken by schools, they are a set of routines designed to maximize outcomes - from the very first identification of a potential partner school, to the very last moment before the provider interacts with teachers and students about to take the high stakes test. They include a provider's relationships with everyone from the superintendent to parents. Evaluations simply do not account for all these factors and their role in student success. The typical study treats all school engagements as equivalent, when we know that every implementation has its own strengths and weaknesses. Today's evaluators are looking for go/no go answers to the "what works?" question, when what we really need to know are the circumstances in which the program works well or not at all.

Negotiating the final report. Finally, the final report of every third party review involves some degree of negotiation between the sponsor and the evaluator. This negotiation involves many parties in each organization. There are judgment calls on method and analysis, report structure, and even word choice. Some issues are clearly legitimate, some are obviously improper, most lie in a vague ground in between.

My experience suggests a fairly clear negotiation calculus. If the researchers want more work from the client, they tend to go easy - if that's what the sponsor wants. If they don't need the business, they call the analysis as they see it, but the more mature project leaders will accommodate wording of the bottom line to put the program in the best reasonable light given the analysis. (If they are fed up with the sponsor, the developer will tend to lose the benefit of the doubt.) RAND's reputation for objectivity is far more important to the organization's viability than any one client's happiness - even the Defense Department finds it hard to rein them in. This is true of all the major evaluators - Westat, Mathematica, SRI, AIR, etc.

In this respect, consider RAND's studies of New American Schools design teams, Edison and NCLB's SES program sponsored by NAS, Edison and the Department of Education respectively. A careful review of each yields reasons to give potential buyer/users of those programs some pause. Yet each was discussed in the press with a more or less positive initial spin (and to the provider's folks in external relations, the initial spin is the one that counts). Based on the the numbers, each report could just as easily been written in a somewhat disparaging tone and released with a modestly negative spin.

Smaller research organizations, whose financial sustainability depends on going from contract study to contract study, have a much harder time resisting sponsor pressure. They too have to protect some reputation for objectivity, but their narrow margin of viability also requires their managers to be more "flexible." The price of real independence can be very high, and word gets around.

More important, evaluation sponsors understandably tend to pick research organizations they believe they can "work with" up front. Post-hoc debates are actually pretty rare. And to most readers a positive study is a positive study, whether it comes from RAND or a firm consisting of two people with doctorates.

Whatever the reason an evaluator accommodates a sponsor, the "promising" label now means much less than it might as an assessment of efficacy. It is generally possible to discover something positive in any evaluation, find hope therein, and latch onto it as a basis for judgment.

What to rely on. Research and evaluation into program efficacy remains vitally important if we are to move school improvement products, services and programs from the era of snake oil to something closer to medicine. But for the next decade at least, school improvement programs cannot be simply purchased "off the shelf." They need to be fit to the circumstances with some care. Buyers will have to put the burden of proof on sellers in the purchasing process and then do the due diligence required before making a selection.

In my view, given the vast uncertainties of program evaluation, I would rely less on intermittent third party reviews, and look more to the capacity the developer has for data driven decision making to support program improvement. My advice to investors, philanthropists and educators (start here) has always been to bet on providers who build program evaluation into their product.

Some relevant podcasts:

• Real School Improvement Providers Back S. 2118

• See also: Department of Education Technology Study Says What Every Major Study of Broad Reform Initiatives Says: It Depends and We Don't Know

• The School Improvement Industry's "Evaluation Shoals"

• Notice to Mariners: Logs, Charts and Navigators for the Industry's "Evaluation Shoals"

• See also: Program Evaluation 101: What AIR’s Study of CSR Can Teach All School Reform Providers

• About That "Gold Standard" (did it happen?)

• Coping With The New Philanthropy

Posted by Dean Millot at 1:43 AM | Permalink | Comments (1) | TrackBacks (0)

Jim Snyder is the Director of the NW Education Cluster

In August 2003 about 25 people gathered at a local university to share ideas about forming an Education Cluster in the Pacific Northwest to discuss issues surrounding education. In the room were CEO’s of over 10 education companies that created products and services for the education industry from pre-kindergarten to adult learners. Fast forward to 2008 and the NW Education Cluster is now over 260 people strong and represents over 40 education companies. The cluster is still a fledging organization but it is attempting to coalesce around a few key priorities.

With help from the Oregon Business Council, key representatives from the cluster met in September 2007 to set a road map on the direction of the cluster. hat meeting served as a dry run for the Oregon Leadership Summit tin early December 2007. At the meeting the cluster held a roundtable where the Oregon State Superintendent of Schools (Susan Castillo), the Oregon Senate Majority Leader (Senator Richard Devlin) and others from the teacher’s union and Governors’ Office all sat together to talk about working towards building 21st Century schools. The businesses themselves could not have had these people at the same table except for the leverage of the cluster. Over 70 attended the meeting and a few ideas percolated.

The cluster started by providing a key statistic that over 80% of the cluster member companies derive less than 10% of their business revenue from within Oregon. How can business and government work together to increase this (this is hardly unique to the Northwest!)?

The two ideas shared by the cluster to these government officials were:

1) An Oregon X-Prize for education in the area of technology. Education Week just this month came out with report ranking Oregon low in technology and staff development. How can we all work together to focus the energy in Oregon to support specific goals using local companies’ expertise?

2) A summer Education Summit where educational companies could share their ideas and technology with state education leaders.

After this meeting there was a question of, where can the cluster go from here? The cluster is a volunteer organization with no outside funding. It is, however, unique – there are no other clusters in education anywhere else in the US to my knowledge. How can the cluster work with the local government to raise money for a prize or a summit? How can they be taken seriously? These are the questions to which we seek answers.
The cluster is working with others in the educational market.

As a follow up to the roundtable summit, the cluster last week had its quarterly meeting where the Portland Schools Foundation and the Chalkboard Project presented that they were up to and how we could all work together. The turnout was 50% higher than normal – the word is getting out!

In the next month the cluster will be sending out a survey to gage interest among members around the summit or X-Prize. We are even getting some pro-bono work from a local PR firm that is interested in what the cluster is doing. Some of the details for a prize or summit that we are exploring are:

1. How can companies help in implementing the mission of the new HS diploma requirements?

2. How can the cluster work to develop virtual school curriculum and ancillary education materials?

3. How can the cluster support the Governor’s workforce strategy? One idea is to connect with very established manufacturing cluster here in Oregon to push for an educational/ workforce reform agenda?

This is still a work in progress but you can see that we are well started on the journey. I would welcome any ideas others had.

Posted by Dean Millot at 8:55 AM | Permalink | Comments (1) | TrackBacks (0)

While for-profit publishers have come to dominate the content employed in teaching and learning, I would argue that nonprofits have owned technical assistance and sole practitioners define the market structure of general consulting in public education.

Until quite recently, nonprofit organizations have had a strong grip on the delivery of professional development services to raise the educational productivity of teachers and administrators. Unlike textbook publishing, this “market” is not a century old. My sense is that philanthropy did not really begin to make a national effort to improve poor and urban schools until the late 1970s, and the evolution here has not involved merger and acquisition. The history is more of a movement than an industry.

I would argue that the single most important factor in the success of nonprofit technical assistance providers has been price. Up until the mid-1990’s, had these services been offered for a fee, there would be no technical assistance in public schools. Frankly, administrators selected these providers only in part to actually improve teaching and learning. At least as important to the decision to bring outsiders into the classroom was the need to respond to political pressures to address local academic shortcomings, and the value of providing outlets for dissatisfied parents and teachers.

Given that the services were offered gratis, had little if any objective proof of efficacy, and rarely any evaluation while on site, the sign of a quality partner sought by educators was the local respect accorded the foundations and corporate sponsors backing the nonprofit provider. As with textbooks educators relied on brand.

If customers become used to getting something for free, are not themselves accountable for results, and have no objective means of assessing their partners value-added to student performance, for profit firms will not be attracted, and so the technical assistance market remains held by small local nonprofits and highly fragmented nationally.

In any large organization there are tasks that cannot be met by materials or technical assistance. Typically they involve management’s need to augment its capacity for specific short-term tasks. Public education is full of an ever-changing pool of consultants, sole practitioners working for federal, state and local education agencies.

When organizations are not accountable for performance, and can’t measure the value added by any input to the process, but need help with management, quality is a matter of personal comfort. Decision makers tend to rely on people they know. Advisors are taken on because of their relationships – with the decision makers, the subject matter, the partners.

Because public education has lacked objective measures of success, it is hard to compete for assignments "on the merits." Because each engagement is highly personal, few consultants remain in business longer than their client. Because decision makers change frequently, it is hard to build a practice. And because it is hard to build a practice, there have been few sizable consulting firms in k-12.

When No Child Left Behind was passed in 2001, the relationship of educators to textbook publishers, nonprofit technical assistance organizations, and individual consultants had nothing to do with providers objective, demonstrated contribution to student achievement. Decades of market success left these providers with no particular capacity to add value to educator’s AYP responsibilities under NCLB.

So when NCLB also sought to hold providers accountable for their contribution to student performance, one might have expected educators to pass the pressure they felt down to publishers, technical assistance providers, and consultants. One might have expected them to embrace a new set of market entrants whose business success depended on their clients’ educational success.

Nothing of the sort happened.

Next: What went wrong with Scientifically Based Research?

Posted by Dean Millot at 1:52 PM | Permalink | Comments (0) | TrackBacks (0)

Last week I argued that in the context of public education policy, accountability consists of standards, consequences and due process. Moreover, whatever its other faults, No Child Left Behind’s (NCLB) Adequate Yearly Progress (AYP) provisions meet that definition.

If schools are held accountable for their role in student performance, what about those who supply schools with the products, services and programs employed in teaching and learning? In my view, NCLB made at least as good a start here – on paper, but the law has been implemented in an arbitrary and capricious manner.

To make this case, I need to provide a background of concepts and history.

The common law of product liability law incorporates the idea that sellers implicitly warrant that what they sell is fit for the purpose it is advertised to meet. A new car that doesn’t run is not fit for its purpose as transportation. The rule is not firm; a car that breaks down daily is not fit – but monthly, every six months? Not so clear. Still, there is a basic principle, and reasonable people generally come to the same judgment on the facts.

Prior to the passage of NCLB, the concept of fitness for purpose did not extend very far in the case of products, services and programs sold for teaching and learning. Textbooks had to hold together, pages had to be in order, text had to conform with generally accepted forms of spelling and grammar – in short, they had to have the attributes we expect of books.

What they did not have to do, and no provider would warrant on their own volition, is play a value-added role in the teaching and learning process. To be specific, they did not have to demonstrate a contribution to student learning. For many years education science did not possess the capacity to distinguish the role played by products services and programs, and even today the state of the evaluation art is not entirely satisfactory. Moreover, to the extent that providers were held accountable for their offerings, they focused more on specific subject area content – in a word the textbook adoption process. Finally, one can argue that schools were not accountable for student outcomes until NCLB, so there wasn’t much reason to inquire into the contribution of providers’ offerings.

When results are not an explicit part of school accountability, how do schools make decisions about the products, services and programs they will use for teaching and learning? Even more important, once patterns of use have been established, what are the incentives for change? And how do those incentives shape the market?

The primary product, service and program employed for teaching and learning is the textbook. It is not terribly useful to explain the century of market developments that led us to a handful of multinational textbook publishers and hundreds of small companies providing supplementary materials. What is important to understand is that in the absence of objective standards of quality accounting for impact on teaching and learning, “brand” became the surrogate for quality, and the gross margins on sales generated from the attraction to brand allowed the major publishers to create and reinforce barriers to entry by competitors. One hundred years of growth and mergers leave the publishers in a market protected by Everest-sized barriers to competition. That power extends so far as to slow the adoption of materials other than textbooks through the lobbying process, move them to the supplemental realm, and so keep most of the dollars spent for classroom teaching and learning in the print market they dominate, and perpetuate their political power.

Next: Nonprofit technical assistance and independent consultants.

Posted by Dean Millot at 1:45 PM | Permalink | Comments (0) | TrackBacks (0)

Digital Humanities Grant as Staff/Product Development Funding

FYI: The humanities include the classics, history, languages and literature, law, performing arts, philosophy, religion, visual arts.

From the January 21 issue of K-12Leads and Youth Service Markets Report

Announcement: Digital Humanities Start-Up Grants Due April 2 (Jan 15) National Endowment for the Humanities

Their Description: This program is designed to encourage innovations in the digital humanities. By awarding relatively low-dollar grants during the planning stages, the goal is to identify projects that are particularly innovative and have the potential to make a positive impact on the humanities....

• projects that mainly involve digitization, unless the applicant is proposing an innovative new method for digitization;
• the implementation or assessment of existing digital applications in the humanities (however, exploration of or planning for a new direction or tool for an established project is allowed);
• recurring or established conferences or professional meetings...
• creative or performing arts;
• empirical social scientific research;
• work undertaken in the pursuit of an academic degree;
• the preparation or publication of textbooks; or
• projects devoted to political, religious, commercial, or social advocacy.

My Thoughts: For-profit firms in the school improvement industry should make the modest investment in legal advice required to form an affiliated nonprofit. With access to this kind of funding, providers can offer staff meaningful professional development opportunities, create a source of new product development and basic research and, in times when revenues are likely to be a bit more scarce than in the recent past, a way to maintain core capacity.

Posted by Dean Millot at 9:31 AM | Permalink | Comments (0) | TrackBacks (0)

Summer School is an Obvious Start at Revenue Diversification

From the January 21 issue of K-12Leads and Youth Service Markets Report

Announcement: Summer School Grant Program Due March 3 Colorado Department of Education

Their Description: This Request for Proposal (RFP) is designed to distribute funds to a school district, consortium of school districts, and/or Institute Charter School pursuant to the requirements of the Summer School Grant Program.... $970,000 is available to applicants during the summer of 2008....

The proposed curriculum and program(s) of instruction must be research-based and intensive enough to demonstrate significant improvement in a student’s performance in a short period of time. Evidence of instructional effectiveness can come from any of the following sources: (1) demonstrated student achievement in formal testing situations implemented by the teacher, school district, or state; (2) published findings of research-based evidence that the instructional methods being used by teachers lead to student achievement or; (3) proof of reason-based practice that converges with a research-based consensus in the scientific literature. This type of justification of educational practice becomes important when direct evidence is not available (a direct test of the instructional efficacy of a particular method is absent), but there is a theoretical link to research-based evidence that can be traced....

My Thoughts: No school improvement provider looking at state budgets and the future of NCLB’s accountability provisions can feel confident about selling their offerings in the same format tapping into the same funding streams. It’s time to repurpose content and diversify revenues. A “sustainability premium” attaches to new sales for new purposes drawing on new funding programs.

Posted by Dean Millot at 9:23 AM | Permalink | Comments (0) | TrackBacks (0)

As an advocate for a market in school improvement services, I welcomed Denis Doyle's commentary in the January 16 issue of Ed Week (Why Markets Are Important (And What They Could Do for Public Education)). It hit all the right points, but I had this curious, discouraging sense of deja vu.

I could swear I read something like this by Dr. Doyle in Ed Week or a similar publication when I began my transition at RAND from national security to public education with the end of the Cold War in 1989. If not by him, then someone equally well-known at the time.

The commentary led me to reflect on how little impact market ideas have had on public education going on 20 years. Yes, we got charters with a few percent market share of public school students. Yes, we have the decidedly mixed advantages of a small highly-committed band of voucher advocates. Yes, post-NCLB I, schools are buying a wider range of services addressing core teaching and learning functions. Yes, there's more "biz-" and "market-speak" coming from superintendents and chief state school officers. Yes we have a cottage industry of eduwonks examining and prostelitizing market based solutions funded by philanthropy.

But, really, so what?

Having spent the first half of my career to date working with another vast bureaucracy exercizing monopoly power over a key social function - the Pentagon, I did learn to measure success in millimeters, and so in part I should be happy about our progress, but I am struck by one crucial difference.

A series of education reforms built on market principles reforms over 20 years seems to have accomplished successive gatherings of the institution's openly discontented, following by a splintering of these marginal groups, rather than attracting the high flyers to create an emerging potential mainstream.

In U.S. defense policy, the reformers gradually grew in strength to take over the system. Don Rumsfeld was a necessary but hardly sufficient cause for the changes that occurred in US military structure in the last decade. He was simply an enabler for a school of thought that believed in a leaner military, mobility, and a more precise application of force; one that ultimately replaced those who favored a massive standing army, sheer weight of effort, and dumb bombs. Absent allied sources of support from within the officer corps, Rumsfeld would have accomplished precisely no changes in our force structure.

In effect, in the first Gulf War the best and the brightest colonels learned lessons that they implemented as Major Generals at the start of the second. There might have been a bunch of "defense intellectuals" on the outside helping to catalog lessons learned by soldiers and refine them into a coherent strategy, but the change came from those "up and comers" on the inside who formulated and internalized the implications of their experience in the field. Forget for a moment how Iraq has turned out, and simply consider the systemic change that has occurred within our military institutions.

Surely an education system based on market principles would have its shortcomings, wrong turns, cases of bad judgment, and disasters. But while the reformers inside the Department of Defense and military services gathered in strength over 20 years to win their "opportunities to excel," the community of market-based education reformers grew as fragmented cults. Looking at them as a collective force for markets, their vulnerablity to destruction in detail is painfully obvious. The charter movement is financially propped up by a few foundations and artificially unified. Vouchers are the market movement's political albatross. NCLB II threatens the stability of an emerging but marginal school improvement industry. The school improvement industry itself is highly fragmented on several dimensions I have discussed many times.

When all is said and done, there's been no expanding movement of folks who were high school principals and small city superintendents in 1990 now running big urban districts - or any districts - who are implementing reform lessons learned from their experience in the front lines of urban education. This is at least one reason why, despite NCLB I, Margaret Spellings and Rod Paige are no Don Rumsfeld, and why - you name your Chief State School Officer, or district Superintendent from the past 20 years with a great reform plan hasn't been either.

Similarly, the relationship between the eduwonk and the educator isn't close to that between the defense intellectual and the warrior. If a defense intellectual recycled an argument about the need for a strike force based on small, highly mobile forces armed with precision munitions, and it somehow got past the editor of Army Times last week, the readers' response would be "that argument is so 1990's. We've got the right warfighting structure. The problem is that neither this Army nor the one it replaced was designed for occupation, insurgency or civil war."

When Ed Week feels comfortable trotting out Doyle's solid, but very old well-worn, argument as somehow, something new for its readers to think about - that tells us something about the editors, the audience. Specifically - how far our market reform ideas haven't come, how little purchase they have gained within the "educator corps," how much what has happened in the way of market reform has beennimposed on nthe system from without rather than evolved from within.

I predict there will be no wave of letters to the editor of Ed Week from superintendents and chief state school officers protesting that for the last ten years, they've been doing what Doyle advocates.

Bottom line - even after agreeing to celebrate success in millimeters, market reform concepts aren't making adequate headway within the public education system.

Those of us who believe in the value of markets, need to stop worrying so much about restatements of, or refinements to, market reform proposals, and start thinking about how we are supposed to get these reforms in place. Of critical importance is how we are supposed to get young high flyer educators on the leadership track to internalize market strategies. Maybe we start with a close examination of why public education's institutional insiders have not advanced reform like their counterparts in the uniformed military, why education policy wonks are so much less relevant to educators and real education operations, and whether there's a strategy by which this might be changed.

Reply to Marc below:

Thanks for the comment - and let your colleagues know you made it!

Let's set Blackwater aside for a moment, and recognize that the debate within the Pentagon over our military structure was not about moving combat missions to the private sector, but about the kinds of systems the military would provide it soldiers to be used according to doctrine developed by the military.

Similarly, the educator need not be interested in handing the education of our children over to private firms to care about the products and services purchased to be used according to what educators see as best practice. Charter school law does not require charter schools to be handed over to private companies, it requires districts to think differently about the relationship of central to the school. Educators already purchase billions of dollars of "educationall systems" from publishers. They've already conceded the privatization of supply - the question is whether there might be some better business partners with better offerings than the multinational publishers.

In short, when it comes to the matter of private sector involvement in public education, educators are already a little bit pregnant. I don't think they are hesitant about privatization so much as they don't realize that the decision to privatize the supply of teaching and learning tools was made decades ago.

For more on this last point, see or hear....

Posted by Dean Millot at 9:52 AM | Permalink | Comments (1) | TrackBacks (0)

John Thompson, is a Teacher in Centennial High School, Oklahoma City Public Schools

In much of life, when we want something done right, we build redundancy into the budget and options into the plan. In education, we buy "silver bullets" that will work for every school, every teacher, every student - as long as everyone does their job flawlessly. In sports, we deride "one man teams." In education, we celebrate heroes who defied the odds of systemic uniformity - and then try to replicate their unique accomplishments.

The school improvement industry has provided a vast toolbox of educational technologies, along with expertise regarding their proper mix and appropriate mix. Society has invested enough in public education to allow teachers to pick and choose from the array of tools. In other words, redundancy. Yet, No Child Left Behind and the technological imperative of American business have combined to increase school districts’ intraprenuerial energy, while reinforcing their top-down tendency to settle on one "quick fix."

At the district level, NCLB’s timeline and benchmarks towards 100 percent student proficiency by 2014 create little interest in reforms that would produce real improvements for smaller groups of students. District leaders are looking for something that works for every student. Reformers within the system are incentivized to a) settle on the one best approach, b) promote its use, b) discourage or prohibit alternatives (especially those favored by bureaucratic rivals), and d) press for taking the reform to scale as quickly as possible.

Periodically, my district’s central office administrators mandate the use of tutoring technology. I’m not sure when this cycle was initiated, but the latest round was touched off when someone discovered stacks of unopened tutorials, dusted off a pile of expensive hardware, and calculated the financial waste in the hundreds of thousands of dollars. No one stops to ask why teachers don’t find the technology useful. Maybe the source of the waste lies more with the decision to buy rather than the lack of use.

Congress mandated a scientific evaluation of computerized programs for reading instruction. A massive, multi-year, $10 million, state-of-the-art study of 15 reading and math products concluded, "products did not increase or decrease test scores by amounts that were statistically different than zero." Researchers concluded, "In all of the software groupings, students on the average spent only about 10% of the time," translating into less than 30 hours a year. Moreover, "When we observed the classrooms through the year and interviewed the teachers, we feel pretty confident that 10% of use reflects the sound judgment of the teacher about how often and for what kinds of instructional modules they want to use technology."

So why not aim for 15% usage in the first place, and budget accordingly?
My school is literally in the middle of a gang war. Many times, from the moment the students walk through the metal detectors they are juicing themselves up for violence. Any rational adult knows what Job #1 should be. But several times in those situations, a fifth of our staff or more have been pulled out for online tutorial training. Online tutorials might be the greatest innovation of all time, but many teachers are likely to associate the program with their perception of a central office that is out of touch with reality, hardly the best way to encourage program adoption.

If I had a magic wand, I would require many teachers to use tutorials. (If I had the power and could find replacements, I would take more drastic actions.) But to win bureaucratic battles, administrators must promote their agenda as cost effective, i.e. the lowest per student cost and transformative enough to meet NCLB goals. The "penny foolish" way to do that is to require one approach to teaching for all teachers.

There is a value in textbooks and the expensive materials that accompany them, especially for young teachers and those who do not understand standards. But our texts have always been written over the reading levels of my students, and today we are required to use books that require even higher skills. Pre-NCLB texts that are much more compelling are stored in warehouses. Redundancy would cost little, in other words - redundancy.

We invest millions in computers but ration chalk and consumables. Often, a teacher must request them, and in doing so they are made to feel unworthy. The message is that effective teachers do not want 20th century materials. But it would cost very little to flood our schools with disposable, high interest, appropriately skilled books.

The savings accrued from retaining time-tested and inexpensive materials could fund expensive digital investments, I would prefer the National Academy of Science recommendation that a much more aggressive effort to introduce digital gaming for classroom instruction. But it would be silly to mandate universal usage. Instead, we should recruit young talent who want to pioneer those tools and the level of our digital capital investments should be determined by our success in building human capital. Let all sorts of flowers bloom.

In our dynamic society, the digital world is a force for decentralization, creativity, and choice. The American way is to reject social engineering and rejoice in the multiple ways to skin a cat. Yet American public education has implemented technology in a manner reminiscent of Soviet Five Year Plans. Isn’t it time to education to adopt the value of redundancy?

Relevant podcasts on School Improvement Industry Week Online. Podcasts run 6-10 minutes. They can be streamed on a computer or downloaded to an mp3 player.

• A Public School Marketplace: What's in it for Teachers?

• What if the Teachers Unions Bought the School Improvement Industry?: Parts I, II, III

• Rules for Educators Purchasing School Improvement Programs

• Organizing the District: Parts I, II, III

Posted by Dean Millot at 9:40 AM | Permalink | Comments (2) | TrackBacks (0)

In the context of public education policy, accountability consists of standards, consequences and due process. If the purpose of accountability is to advance a social goal, all three elements must be present. Standards without consequences and due process amounts to reporting. In the absence of standards and due process, the implementation of consequences amounts to tyranny. Due process without standards and consequences defines a discussion group.

Accountability so defined is a problem throughout the public education market. In the case of public schools, we’ve put a nation-wide system in place. All schools are treated equally. NCLB accountability doesn’t work perfectly, but it’s a very good start. The challenge is making sure that it is improved as a means of accountability when many would “improve” the law by gutting its substance.

In the case of the organizations providing products, services and programs, accountability under NCLB looks good on paper, but hasn’t worked in fact. All providers are not treated the same. Those already in business enjoy de facto presumptions of efficacy, new firms face relatively high hurdles of proof; nonprofits enjoy similar advantages over for profits; rules are applied arbitrarily; and providers injured by government action have no means of appeal.

In the case of labor relations, we've had a system where process has favored the incompetent. The District of Columbia has pointed the way towards one that encourages tyranny.

In this election year, when every policy option is up in the air, it might be useful to talk about these issues in terms of first principles.

School Accountability Under No Child Left Behind.

• In deference to the Tenth Amendment states set standards for what children need to know and be able to do in 4th, 8th and 10th grade, choose the tests that students will take to demonstrate proficiency, and created benchmarks that schools, disaggregated by significant subgroups groups of students, must meet to achieve Adequate Yearly Progress (AYP) towards the law’s requirement for 100% proficiency by 2014.

• If schools or districts do not achieve AYP, they become eligible for improvement grants, students receive rights to seek space in other public schools and/or tutoring, and based on state decisions the school eventually becomes subject to some form of restructuring.

• The standards and consequences are applied equally to all schools, and schools and districts have an administrative process and criteria by which they can challenge decisions that they have not met standards or are not implementing the law’s consequent actions.

We can argue about the validity and utility of the standards, tests and consequences. We can bemoan the fact that some states are in a way penalized for adopting higher standards than others. We can argue about whether NCLB is the best way to meet the nation’s public education needs. We can argue that schools are not resourced equally to meet NCLB’s goals.

We can’t argue that NCLB offers a sham system of accountability. In each state, there are specific standards developed by educators, the tests for meeting those standards are known and uniformly administered, every school is subject to the same consequences for failure, every school has access to the same administrative appeals process. No school can fail to give some students the test, be excused from the consequences of missing a benchmark, or denied the appeals process.

Recognizing that there may be some exceptions, I do not agree with the general proposition that schools lack the financial capacity to meet NCLB. I think it is more often a problem of resource allocation. But whatever its legitimacy in the case against NCLB, the funding argument is not one directed at the accountability system per se. Most opposition to NCLB follows from the almost certainly accurate assessment of the leaders of public education’s traditional institutional interests that whether the accountability system is good, bad or neutral, their institutions cannot survive it without radical transformations of the internal power structures.

Regardless of this political reality, some opposition to today’s NCLB accepts the basic structure of accountability, but seeks to refine it. Changing the law’s Supplementary Educational Service provisions so that tutoring is made available only to student subgroups or students who fail to achieve proficiency; offering schools credit for improvements in student performance that expand upon the existing safe haven provision through value-added (growth model) schemes; moving the 100 percent proficient end point to 2020; or lowering the end point to 80 percent, may be favored by those who oppose any private involvement in public education and surely will shrink the market for school improvement services. Regardless of these interests, each proposal is legitimate on its merits. Not one of these ideas does violence to the principles of accountability as I noted at the start of this essay; arguably each might improves the system.

Reasonable people can argue about whether NCLB is the best way to give every child the best possible chance of succeeding as adults – enable each to reach his/her highest level of intellectual potential, contribute to our economy, make society a better place, and leave this world personally fulfilled. Maybe it could be better resourced. It probably could be improved against the criteria defining a good system of accountability. But at least NCLB offers the advantages of predictability. All schools in any state share one standard of success, one set of consequences for failure, and the same appeals process.

Next: Provider Accountability. Efficiency is important to individual and organizational success. (Let's hope that the right set of objectives have been selected.) Predictability is an important feature of efficiency. Without the stability offered by predictability there is no willingness to make the investments of time, money and people required to improve performance.

Assuming that NCLB’s definition of success (100 percent proficiency in key subjects by 2014) is a positive goal, public schools are the only actors in the public education market that enjoy predictability. For providers, NCLB’s research based and scientifically based research (SBR) provisions establish the accountability analog to AYP. But while the system of provider accountability looks good on paper, the Bush Administration failed to implement the law.

NCLB's SBR standards have no firm meaning. Government disregards them for established providers but applies them to selected new providers, and offers similar preferences to nonprofits over for-profits. There is no process for appeal.

Listen to this as a podcast at School Improvement Industry Week Online.

Other relevant Podcasts at School Improvement Industry Week Online. Podcasts run 6-10 minutes and can be streamed on your computer or downloaded to an mp3 player.

• AYP Regulation and Adaptive Management

• Department of Education Management of Adequate Yearly Progress (Parts I-II)

Posted by Dean Millot at 1:19 PM | Permalink | Comments (0) | TrackBacks (0)

Trace A. Urdan is a Managing Director of Signal Hill.

In May of 2001 in the fading flush of the dot-com boom, my then firm, WR Hambrecht + Co., with my enthusiastic support, helped PLATO Learning sell 2.4 million secondary shares at a $21.00 share price. The basis for the enthusiasm expressed by myself and others had to do with the perceived inevitability of digital content – personalized, interactive, up-to-date, engaging (you get the idea) – to ultimately dominate the education landscape, even at the expense of the printed word. I knew, of course, about the distribution power of the entrenched publishers; but in my view, their need to jump on the Ed-tech bandwagon would make PLATO an asset over which they would all (there were still four back then) compete auction-style. Based in some small measure on my purple prose, (I wasn’t the only one!) the stock reach an all-time high of $26.19 in July of that same year… Last Friday PLATO shares closed at $3.50.

This story has been a defining one for my continued coverage of the K-12 supplemental market. To every budding entrepreneur with a promising product that finds his way to my office, or buttonholes me at a conference, I generally offer gentle discouragement. Now PLATO’s decline is more complex than simply its being outmaneuvered in the m