My E-Mail Exchange With "Socrates" on CMOs and the CMO Study
The blog audience can learn as much through debate as by reading one person's view of the issues, maybe more. The medium permits the audience to respond to a blogger's prognostications in real time or at their leisure. It's an incredible opportunity for good arguments to rise to the top rather than well connected people. Time consuming as this can be, I think it's important to participate in and facilitate these debates.
Below, I've posted a discussion with the anonymous Socrates, prompted by his posting on my quotes in Scott Cech's article in EdWeek on the Mathematica/CRPE study of CMOs sponsored by Gates through New Schools. I don't know if Socrates has an audience of one or one million, but that really didn't matter. In my view he or she raised points worth discussing. Once I think we reached a point where viewers could come to their own conclusions, I left the field. I'm not trying to convince my counterparts to change their minds, only to get the arguments fleshed for others..
For me, the discussion starts with Socrates' comment on my April 2 “Letter From.”
The foundations haven't bet the whole ranch on CMOs - Walton in particular is still funding one-offs. Gates et al have sunk some money into CMOs, but it's hardly the only funding out there.
Your statement that CMOs aren't a good way to scale charters is curious to me. What's the rationale? As long as they maintain the small schools and some sort of quality control, I'd rather have 10 Achievement First schools in my city than 10 random charter schools. You know that with Achievement First they'll all be good, with minimal quality variance, whereas the mom-and-pops will have some great schools and some that are terrible. Of course, best-case scenario is 10 AF schools and 10 mom-and-pops...
This led me to Socrates site, The Socratic Method, and this post of March 28:
Despite Study of Charter Organizations, Grumpy Old Bloggers Can't Get No Satisfaction
Great news that Mathematica and Paul Hill are going to study the impact of Charter Management Organizations (CMOs) like Green Dot and Achievement First. Having seen schools in both networks first-hand, I have no doubt that the evidence will show that they are adding tremendous value. But in the "let no good deed go unpunished" category, Grumpy Old Blogger Marc Dean Millot made sure to get in a nonsensical jab:
“This kind of research should have been going on from day one,” Mr. Millot said. “There is no excuse from a social-purpose standpoint that, ‘Here’s all this money, but we’re not going to open the stove until 10 years into the cooking process.’”
It's just incredible how easy it is to become an expert these days. Mr. Millot, what you apparently don't know, despite all your blustering about how smart you are in the area of school finance, is that organizations like the New Schools Venture Fund already do collect incredible amounts of data - more data than have probably ever been demanded of, or provided by, any school district. More data than are required by NCLB's supposedly-onerous reporting strictures. Yet in the case of charter funding, you don't think this is enough? More likely is that you didn't do your homework to find out that a lot of data are already being collected. Some charter finance expert. Thankfully, Joanne Weiss sets him straight:
"Joanne Weiss, a NewSchools Venture Fund partner and its chief operating officer, said her organization, which will oversee the study, has been gathering achievement, cost-effectiveness, and other data all along."
CMOs are all about data and transparency, in part because they rely on philanthropy, which can go away if they can't prove their effectiveness. School districts, on the other hand, have nothing but incentives to obscure or distort their data, as school districts are typically run by someone with a political interest.
I worry that when the results of this are published and they prove that charter management organizations are highly effective, the grumpy old bloggers will find some way nonetheless to shoot them down, claim they are fudged, or pretend the information doesn't exist. Nothing frightens defenders of the status quo like data, so they are already getting in their shots so when the data prove them wrong, they can cry foul.
Our exchange follows:
Millot (March 31):
For the record, I'm not skeptical of the CMO on ideological grounds, I just don't see the case for it as a cost-effective means of achieving quality at scale.
That assessment is based on every piece of data I have been able to get my hands on. That's been somewhat limited, even after at least one request to New Schools and several of its CMO investees - especially on finances.
I had more to say about the work than ended up in the story, but I stand by the statement that serious evaluation work should have been built into philanthropy's grants program at the start, rather tan tacked on now. It’s a critique based on lessons learned from many other experiences in school reform.
I have at least offered readers some evidence and analysis to back up the reasons for my skepticism - you flagged it as "blustering."
However, you've offered literally no evidence of your assertion that "CMOs are all about data and transparency," let alone that they are a cost effective approach to quality at scale, beyond you observation of a few schools. I admire your faith, but as a doubting Thomas, I'd like to see the proof.
There's an old saying - something beats nothing. Until you've got something, your opinion is closer to the "empty" part of the definition definition of bluster.
If I am a Grumpy Old Blogger, maybe you are a Naive Young Blogger. Every critique of the latest trend in education reform is not an attack on the reform itself or reform or a defense of the blob or even school districts. Smart advocates first take criticism at face value to understand their idea's weaknesses. Mindless defenses don't strengthen your case, they just suggest to fence sitters that you're not be trusted either.
Let’s have a reasoned discussion.
Socrates (April 3):
I appreciate your comments, and wish that I were as young as you imply.
As for your opposition to CMOs being practical and not ideological, this quote from your blog belies that assertion:
"The heart of charter schooling is not technical innovation in management or pedagogy, but a social movement – driven by the idea that public schools should be of, by and for the communities they serve. Centralized management contradicts that idea, which is why I’ve been against Charter Management Organizations as philanthropy’s primary investment strategy for scale."
Sounds pretty ideological to me.
As for CMOs not being an effective way of scaling charters; I actually believe the opposite is true. One-off stand-alone mom-and-pops are much less efficient than CMOs that can take advantage of scale. It's Business 101: bigger organizations have efficiencies that smaller ones don't. As long as you keep the basic educational unit - the school - small, the CMO can provide back-office much more specialized and effective support than the multi-hat-wearing business and operations managers in most single schools can.
As for charters being all about data and transparency, see my next blog post about Mathematica studying KIPP. Or check out the website of New Schools Venture Fund: http://www.newschools.org/portfo...a-business- plan
Or Charter Schools Growth Fund:
Both require clear, measurable data in order to receive funding. That NSVF is now facilitating an even deeper dig into their CMOs should be lauded, not slammed for being too late. What they were already doing is way more than most other foundations, or any districts, were doing, but now they're taking it another step further.
Millot (April 3):
You have a point about my views on centralization, but the line here between ideological and practical is blurry. Centralization has great difficulty with quality - the macro level example would be the East German industry under Communism. The micro-level would be Starbucks. Once you move past the simplest of products - say fast food, it’s just too hard to do well consistently. I've written about all of this in years past.
As for what you believe regarding scale, I've written a lot on this too. You might read my edbizbuzz series "Whatever Happened to the Charter Idea" or the CRPE report on CMOs I co-wrote "Quantity Counts."
As for transparency and access. The Mathematica study of KIPP was announced within the last year, Fisher began funding KIPP years earlier. As for New Schools, I know more than a bit about their internal due diligence first hand, having come within hours of co-investing with them in EMO Learn Now before it was purchased by Edison.
My expertise in these issues was not gained through casual acquaintance. I ran a social investment fund that took many education nonprofits and a few for profits to scale. I ran a fund that made loans and equity investments and needed to be repaid to repay its investors. I've been through the budgets and business plans of these kinds of organizations in excruciating detail, to say nothing of their educational programs and demonstrations of efficacy. I've examined small independent charters and big multi-unit operations.
Again, I may be wrong. I'd like to be proven wrong. But what you need to do is offer some analysis rather than point me to places I've been to for over a decade, or assertions that everyone enamored with the latest trend in reform makes before they get burned.
Please, come to me with your own analysis, based on facts you've put together.
Forgot to respond to two other points, made by you (in your March 28 comment in) edbizbuzz:
I have reviewed Walton's IRS Form 990's for the last several years - some of that analysis is contained in my series on the social keiretsu. As best I can tell from that, the Foundation has committed the overwhelming majority of its charter school investment to CMOs. They do fund some of the charter association, but only a tiny sum is going to individual schools and most of that appears to me anyway to be older charters rather than new starts.
As for Gates and Walton being hardly the only funding out there. Add Broad, Fisher and Bradley, and you've got all the big charter funders with large discretionary budgets. Again, look at my series on the social kerietsu. There are other funders that can and do hand out $50-100K here and there, but they truly marginal in the context of the national charter movement.
Finally, re "You know that with Achievement First they'll all be good, with minimal quality variance." First, that's not known, which is why we need the CMO study. Second, so far, what we do know suggests that these schools are of mixed quality across and within CMOs.
I can't claim the same charter-sector experience as you, clearly. I certainly respect your history within that movement and your knowledge of it. My central point on transparency and assessment is that charters (and many of the foundations that support them) are doing it so much better than the districts are that it's tough to stomach criticism of their practices in this area. Still, just because they're better than the horrendous bureaucracy doesn't mean they're good at it - that I'll acknowledge.
I think the key on assessment is that the charter organizations and foundations are voluntarily undergoing deep analysis of their outcomes. Let's hope that this spurs similar analysis of public district schools.
As for the effectiveness of scaling up - big does not always equal bad. GE got better as they got bigger once Jack Welch took over. Toyota started growing too fast and stopped producing the highest-quality cars, so they slowed down, made adjustments, and now are back at or near the top. Specialization is good for quality, and growth can drive innovation as well.
Finally, Re: Big and Bad
Big does not always mean bad - very true. However, it's worth pointing out that GE makes its vast profits pretty much from one part of the company - GE Finance, and that product is highly scaleable. Automobiles can be as well - the problem there is generally not manufacturing, but bad decisions about product mix versus actual demand.
Quality at scale within reasonable costs is enormously difficult in services. That's why law firms have a hard time achieving Toyota like scale, and why private health care is such a mess here in the U.S.
In my view, the process of education is closer to the provision of legal advice or health care, than decisions about credit scores or the manufacturing of cars. In my view Teachers should be considered closer to lawyers and doctors in terms of how we expect them to work, than lending officers or assembly line workers.
If this is true, than the extent to which economies of scale can be realized is limited. It might work for the provision of high quality facilities, pension plans, communications and other common service purchasing. It probably won’t work on assuring quality; indeed it will probably burden the total enterprise with unnecessary costs and actually hamper the ability of teachers on site to meet local student's needs.
You might want to read "Why Quality Control is a Fools Errand." http://archive.edbizbuzz.com/blo.../7/ 3144053.html and "KIPP Drops 7 Schools" http://archive.edbizbuzz.com/blo...26/ 2906981.html
Socrates (April 3)
It does appear that you're right that the focus has shifted to CMOs for the most part at these foundations. Again, though, I have no problem with this. I'd prefer to have the philanthropists funding the organizations with demonstrated results than shots in the dark. Part of the problem with the charter movement has been that charter authorizers have been notoriously poor at choosing the applicants that will truly develop successful schools. Schools within CMOs may vary in quality, but you have a much higher degree of assurance than an AF school will be good than a randomly-chosen charter school. The charter school "movement," as a whole, has not been successful in raising student outcomes. Where it has been successful is in allowing high-performing charters to emerge, and now, to expand.
I still want to see more flowers bloom, for sure, and I'd love to see charter funders fund high-promise mom-and-pops, but until we have studies that tell us why schools like those run by Achievement First are successful, it's a bit difficult to figure out what works and what doesn't.
Your law firm example is a good one. If I were in big trouble and I had to choose a lawyer from a gigantic firm like Skadden & Arps, or a single-shingle firm, I'd go with the big boys. The same isn't true in charter schools - costs should be lower at the CMO level as back office services are run by specialists rather than generalists.
I agree with you in that service industries have to scale more slowly than manufacturing industries do. However, I'll point out that Google is, though not in a service industry, certainly in a knowledge industry that requires high-capacity professionals at every level except perhaps the most basic, and they've scaled up very quickly, without a diminution of quality inasmuch as I can see, anyway.
One scalable service industry is the hotel industry. There are plenty of brands that have successfully scaled, from low-level hotels like Motel 6 to any of a number of luxury brands. The luxury brands at least prove that high-quality service at scale is not impossible to achieve.
By the way, the other "proof", if proof is possible, is that CMOs have been able to scale without evident drops in quality.
I should add that I appreciate this dialogue.
You've decided that CMOs like Achievement First "have demonstrated results" and want the CMO study to tell you "why?'
Knowing Paul and Brian, I'd say that they intend to find out if they do have such results and, if so, under what circumstances.
Saying "you have a much higher degree of assurance that an AF school will be good than a randomly-chosen charter school" doesn't pass the "so what" test. The real question is whether we can come up with a set of factors that tell us about any charter applicant's chances for success. Frankly, that's not rocket science, you can have a look at the guidebook for charter authorizers I wrote for the U.S. Department of Education in the mid 1990s, also at the CRPE website.
There's tons of good work on all these problems. It's accessible to even the simplest efforts at Googling. Rather than make the same arguments as everyone else, you might immerse yourself in the research that's been done since the early 1990's. Take your conclusions from that, add them to your personal experience, and come to some conclusions you can defend with some assurance. What you are doing now is parroting the unsubstantiated claims of zealots, and then backpedaling in the face of arguments you can't refute head on. You appear to be a smart person who wants to add to the debate, why not take the high road?
As for hotels, that's a much more complicated business - my dad was a hotel and restaurant turn-around specialist. The main point there that most hotels are franchises. Hyatt owns few Hyatts. Hotel owners pretty much buy the name, the marketing campaign and the reservations system.
The issue of maintaining quality in franchise organizations was handled very well by Harvard's Jeff Bradach - but the name of the book escapes me. As for quality, I don't think you can compare checking room cleanliness and food appearance and staff/client ratio to the value-added by teachers products programs to student performance. I've seen both. The former is honored in the breach.
I've gone into which charter school functions scale well, and which don't. Instead or responding off the cuff, you need to read those materials.
Then we can talk seriously.
I've stayed in hotels from the Ritz to the Motel 6, and there are real differences that have nothing to do with marketing. A Ritz is a Ritz, and I'm pretty sure there's no Ritz that's run like a Motel 6. The point stands: there are some brands that spell "high-quality", even at scale in the service industry. Obviously no other industry is exactly like education, but I see nothing unique about education that would indicate that a CMO can't preserve quality while scaling up.
Yes, such a study should tell you both the "whether" and, hopefully, the "why." KIPP schools vary enough in circumstances and practices that this study should be able to determine the efficacy of various practices.
If your guidebook is so helpful, why isn't it being widely used? I imagine you're right that your books and possibly others' as well will tell us something important about which charter schools will work and which won't. If that's the case, charter authorizers clearly are not using your books, because charter schools nationwide are not dramatically outperforming district schools. Certain CMOs, however, are dramatically outperforming districts, despite some inevitable variance.
Saying that the top CMOs are unquestionably successful is not the work of a zealot. Pretending they're not in the face of the existing, if incomplete, evidence, may be.
As for your "so what" test, I agree that determining what makes a great school would be helpful, in that charter authorizers could use it to weed out the schools that aren't so great. The problem is, as you say, we know what makes a good charter school - it's in your book - and charter authorizers still aren't using that information. That means that they either can't read or they don't care or they are bad at executing on the lessons in your book - or all of the above. This is a problem that won't be solved by finding out what it is that AF or KIPP does. As such, it may really be that finding out what works doesn't pass the "so what" test. If it's not going to be used (well), it's not relevant.
So maybe the foundations, if they're doing as you're saying, are right in funding the proliferation of CMOs. When you approve an Achievement First school, you have much more of an assurance that the school will be good. Even charter authorizers who don't understand your book can understand how to approve a brand name charter.
I'm sorry I didn't post a link to AF's scores; I assumed your Google search was working as well as you assume mine is. I've seen their results (looked them up just now in fact); they're good - much better than the average charter that's being approved.
By the way, Achievement First is not a franchise model. Looks like we both have some reading to do.
Re: Backpedaling: You made the service vs. manufacturing scalability distinction. I provided two examples of non-manufacturing companies that scaled well - one was a service company - and you said they're too complex to be relevant to teaching. Yet teaching is too complex to be comparable to manufacturing. So manufacturing and really complex industries can be scaled, but mid-range complexity is prohibitive?
I read the first article, and I found myself agreeing with it. I also found that it bolstered my argument in favor of CMOs. I think I've found the conclusion of yours that I believe is incorrect: you don't think that service industries are scalable because they rely on good people, not a good product design. In other words, school design (read: inputs) is not the secret sauce, having good principals and teachers is the sauce. I couldn't agree more, but that doesn't mean CMOs aren't scalable.
CMOs rely heavily on finding and hiring the best people. Achievement First lists as key elements of their model "aggressive teacher recruitment" and "principals with power to lead" - they are focused not on how they're making the widget, but on who is making the widget. (though to be fair, they do have some basic required inputs - "how to's" - like data analysis, that should be part of any good school) That's how law firms and hospitals scale up, too - by controlling the quality of people through good selection practices, and by evaluating their results to see who they should keep and who they should get rid of - not by prescribing a method.
Indeed, it's how you suggest scaling up, as you quoted: "Again, the article highlighted the importance of initial selection, and the human difficulty of conducting triage on the portfolio post facto."
So maybe back-end quality control doesn't work, but that's not what the high-performing CMOs do; they bet on people and front-end principal-and-teacher selection, then they give them "power to lead," as AF (and KIPP, I believe) puts it. Exactly what your article suggests.
Where CMOs are able to save money through economies of scale is where the task is more akin to making widgets: operations, finance, etc. For the knowledge-related jobs that are at times more art than science - teaching, principalling, etc. - they hire the best and they let them do their thing.
I also completely agree with all but one of your suggestions for making a successful CMO:
1. Compelling vision
2. Narrowly-defined geographic and demographic space
3. Large network of like-minded colleagues to select staff
4. Mentoring and staff development
I don't see why this depends on geographic narrowness, especially if you can build regional teams with the other 3 characteristics in place (which the CMOs do, to my understanding).
So since we agree, then I don't see what your practical objection to CMOs is.
My intent in debate is to give readers enough content to decide for come to their own conclusion, not to convince you. I think we've take this discussion about to the point of diminishing returns. But I will bite on the issue of human resources.
The principal problem faced scaling up a business that depends on high quality people, is the people. I've been through this with many K-12 providers offering services. Once management runs out of people they know, staff selection becomes highly problematic. I've covered the problem here(http://www.rand.org/pubs/monographs/MG248/) and in Quantity Counts which I referenced earlier.
In a nutshell, if quality is hard to quantify as an input, managements needs to both select the right people and mentor them in the common approach. The first is problematic, the second takes time. On top of the problem of finding people qualified to operate work in a charter, they have to fit your profile. The scarcity of qualified people, the right match and the time required to mentor sit on one side of the equation.
On the other is the financial model and grant capitalization. Start with the idea of a central management that will support enough charter schools to justify its cost (the flip side of scale economies) and lead to a financially self-sustaining enterprise, at least on an operating basis. This generally involves a very costly tier of senior managers (check the 990s), and a second tier of managers and some staff, plus headquarters etc. Its the nature of these central functions that they need to be built to support something like the number of schools projected to reach sustainability before the enterprise actually reaches that scale.
The managers get a pile of money from philanthropy to cover the gap between total school unit revenues and total enterprise costs. That pile is based on a growth projection. If there is a delay, the enterprise needs more money, to cut costs or accelerate growth.
I call this "the tyranny of the business plan" and it holds for CMOs as much as it held for the far, far better financed for-profit EMOs. Inevitably management falls behind the timeline they proposed to their investors. Most are forced to compromise on where they site their schools, who they hire, how long they mentor, the kind of central support offered.
The result is uniform. What looked in business plan to be a virtuous spiral up becomes a death spiral down. The result on the for-profit EMO side (and I know most the current and former managers) was failure, merger, and a very unattractive business to investors - after piling in far more money than what's going into the CMOs. And these managers were on the whole far more experienced business people and educators than their CMO counterparts.
Some CMOs will survive, one or two will thrive, most will be strangled.
There is a better way to scale, offer the back-office support to qualified operators without forcing them to buy into a preferred model; consolidate the suboptimal back office operations of some 20 plus CMOs into three or four. Focus on quality assurance up front rather than quality control afterwards; leave the post-facto policing of quality to the chartering agencies (how many CMOs will shut down their own poor performers?. This increases economies of scale vastly and shortens the time to break even dramatically; permits more qualified operators to run charters; and moves school creation from the bottom up top the top down, where operators are best able to judge local needs. In short the key problem with CMOs is the decision to tie the nonacademic back office functions - where opportunities for scale are real, to educational programs - where the market is limited and staff requirements limiting.