Gerald Coles: KIPP Schools: Power Over Evidence
Guest post by Gerald Coles.
In the debate over charter schools, KIPP (Knowledge is Power Program) schools are hailed by charter advocates as illustrative of what these alternatives to public schools can produce. With KIPP, poverty need not impede academic success. Enroll students from economically impoverished backgrounds in a "no excuses" school like KIPP and their chances of attaining academic success would soar markedly. There, neither hunger, poor health, relentless stress, lack of access to the material sustenance and cultural experiences available to students from more affluent homes, nor other adverse effects of poverty are impediments to learning and the attainment of good test scores. If only poor youngsters were not in the nothing-but-excuses public schools where they are taught by nothing-but-excuses teachers.
So the story goes and so it was conveyed to me by a KIPP schools manager who, in an oped exchange, presented what the chain considers its best supporting evidence. Whether this evidence actually makes the case for KIPP I will discuss below, but I first need to provide a brief summary of the background leading to the exchange. In a newspaper opinion piece, "Help Rochester Kids Triumph Over" published in the Democrat-Chronicle, the local Gannett paper in Rochester, NY, Dan Drmacich, a retired principal and member of the Rochester-area "Coalition for Justice in Education" (an activist group of which I'm a member), outlined poverty conditions adversely affecting poor urban students and mitigating against closing the achievement gap.
Replying to the oped, a member of the paper's editorial board, Bill Carpenter, whose occupational background is described as a "25-year career in the health care industry," argued that "Poverty Does Not Preclude Academic Success." He acknowledged that poor children "have a steeper hill to climb than their more affluent peers," but "poverty," he insisted can't "serve as an "excuse" for poor performance in school. To support this conclusion he cited several "no excuses" schools that have produced academic success, among which were the KIPP schools. ("No excuses" schools is the ideological concoction of the right-wing Heritage Foundation. Heritage has an aversion to transferring any money from the "1%" to the poor and attacks anyone who suggests poverty influences academic achievement. For an analysis of Heritage's initial group of so-called "no excuses" schools, see Gerald Bracey's critique.)
Because the newspaper eschews a sequence of oped exchanges by two contributors, I responded to Carpenter by focusing on KIPP, arguably the major poster child of "no excuses" schools, and criticized (1) its "open enrollment" process that self-selects certain students over others, as is evident in the well-documented finding that KIPP schools enroll fewer students with disabilities (particularly those with moderate or severe disabilities) and fewer English language learners; (2) how the disparity in student populations puts public schools at a disadvantage in academic outcome comparisons; (3) how the dropouts in KIPP schools leaves a higher proportion of high achievers, which also affects comparisons with public schools; (4) the high dropout rate of African-American males students; and (5) KIPP's relatively higher per-pupil spending.
KIPP headquarters appears to have a NORAD system constantly monitoring incoming attacks because a response to my oped came quickly from a senior director in KIPP's Philadelphia schools, who proffered what presumably was the best evidence demonstrating that KIPP schools are "really making a difference for inner-city kids."
This best KIPP evidence was a 2010 study produced by Mathematica Policy Research company. Echoing a claim on KIPP's website, he described the study as "independent," an astonishing descriptor, given that the three primary funders of this study also are major financial contributors to promoting and expanding KIPP. One of the study's funders is the Laura and John Arnold Family Foundation, which has given KIPP between one and five million dollars; a second funder, the Atlantic Philanthropies, has given KIPP between ten and twenty-five million dollars; and the third funder, the Hewlett Foundation, has contributed between $100,000 and $500,000.
Can there be any bias in research bankrolled by the corporate contributors of the very company whose product the researchers were expected to validate? We are all familiar with the long history of industry-supported research, such as that of tobacco, drug, auto, and coal companies, all conducted by credentialed researchers, all of whom invariably produced findings that supposedly confirmed the value and safety of the products they were paid to investigate. This research on KIPP schools can be described in various ways, but "independent" surely has to take at least second place to "KIPP-funders funded research."
Even putting aside for the moment the issue of independence, the question remains: how well do the findings of this research actually support KIPP education? The answer: "Not well at all."
Schools Studied & Omitted
Let's start with the number of schools used in the study. The researchers explain that they selected 22 KIPP middle schools that "were established by the 2005-06 year or earlier." Unexplained is why, given that there were 46 KIPP schools open by that time, almost all middle schools, the researchers selected fewer than half for their study. Supposedly this relatively small number was used because of the investigation's "rigorous methodological approach," but there is no evidence -- nor could there have been before the researchers began their study -- that these 22 were representative of the others omitted from the study. I don't know if any "cherry picking" occurred as part of the school selection process, but certainly the Report offers minimal explanation of the selection criteria and process.
Student Groups In KIPP
The study did confirm what other studies have confirmed: yes, KIPP's so-called "open enrollment" policy didn't produce an equal playing field with the public schools (the authors didn't put it that way): the KIPP schools in the study had a "lower concentration of special education and limited English proficiency students than the public schools from which they draw." Why didn't the KIPP schools match the public schools by including comparable proportions of students with the greatest learning challenges? What steps did KIPP take to remedy this disparity? The "independent" investigation contained no research on or minimal explanations of these critical questions.
While the study did not find high attrition for African-American students (contrary to findings in other research, to which I will return), with respect to overall attrition, the rates for KIPP and local public schools were about the same. KIPP attrition rates were higher than that of one-third of the district schools, lower than another third, and about equal for the remaining third. However, for some KIPP schools the attrition rate was very high and at extremes was higher than that of the public schools: overall attrition rates for KIPP middle school students ranged from "10 percent to a high of 76 percent," while the highest attrition rate for a public school was 56 percent. Overall, the study did not provide evidence for KIPP's superiority over public schools in retaining students.
What did this KIPP-funders funded study conclude about academic outcomes? The KIPP website claims that it found that "within two years after entering KIPP, students were experiencing significant, positive math gains in 82 percent of the KIPP schools." KIPP fails to say that the study also reports that by year four, when the number of KIPP schools for which there is academic outcome information declined from the original 22 to 17, the percentage of students showing positive math gains dropped to 59%. For reading, after four years in KIPP schools, approximately 60 percent had insignificantly better or significantly worse reading scores compared with public school results.
The study quickly slides over the issue of the schools "remaining" in the study. For example, lightly mentioned (or buried) at the end of the report (p. 32), is a brief comment on the affiliation loss and closure of two KIPP schools (one in Atlanta, another in Chicago) that had been included in the original research sample. The researchers acknowledge that the schools' academic impacts "were noticeably weaker than those of the majority of KIPP schools that remain open. It is clear that these schools were not performing as well as most of the other KIPP schools." O.k, but shouldn't these failed schools have been included in the comparison of academic outcomes? In a supposedly independent study of KIPP schools, if at least two of the schools (nearly 10% of the original sample) were doing badly enough to lose their KIPP affiliation, doesn't that provide significant outcome information on KIPP's academic impact that must be included in the academic outcome calculations? On what reasonable research basis could these schools be excluded?
Hypothetically reversing this outcome, if two public schools were in a comparison group with KIPP schools and were forced to close because their academic outcomes were extremely poor, wouldn't these academic results count as important data for judging the educational value of public schools? Would anyone accept that these academic outcomes could cavalierly be excluded from research on comparative academic achievement?
Black Student Attrition
Alongside acclaiming its own "independent" research, KIPP has been relentless in dismissing other studies. One done at Western Michigan University (WMU), for example, found high attrition rates for African-American students, but, says the KIPP website, was "immediately discredited," especially by the Brookings Institute. The KIPP manager (erroneously) added that the finding has "not borne out by other independent research."
Regarding the discrediting, according to a KIPP statement, a chief repudiation came from Grover Whitehurst, director of the Brookings Institute Center on Education Policy, formerly director of education research under George W. Bush, a major player in promoting the failed instruction of No Child Left Behind, and currently an education advisor to Mitt Romney. How did Whitehurst "discredit" the research? He merely expressed several objections to the study, mostly through a New York Times article, but offered no substantive analysis of the investigation that reasonably could be described as "discredited" it.
Moreover, as for the claim that the WMU findings have not been "borne out by other independent research" that happens not to be true. A recent study by University of Texas at Austin professor Julian Vasques Heilig, found that "despite the claims that 88-90% of the children attending KIPP charters go on to college, their attrition rate for Black secondary students in Texas surpasses that of their peer urban districts."
KIPP's criticism of this study was that its results ran counter to the findings of the KIPP-sponsors funded research which found that KIPP schools lost fewer black male students than neighboring district schools. However, Heilig pointed out that while that study (the KIPP-funder funded one) focused only on middle schools, his included "secondary students-- middle and high schools in grades 7-12 ."
Greater Per-Pupil Spending?
The Heilig study, finding that KIPP spends "30-60% more per pupil than comparable urban districts" in Texas, supported the greater per-pupil spending reported in the WMU research, but KIPP (and the KIPP manager) replied that great spending conclusions come from faulty data and calculations.
Heilig replied that the spending data, like the attrition data, are publicly available (required by Texas law) and reveal the spending differences for anyone who looks. For example, KIPP Austin's per pupil spending was $17,286 compared with $10,667 for the Austin public schools. For Houston the respective figures were $13,488 vs. $10,127. Heilig notes that given the state's level of educational funding, these disparities are not surprising; recently, e.g., Texas cut school spending by nearly $6 billion.
For a more extensive analysis refuting KIPP's claims about its levels of per pupil spending, see Bruce D. Baker's reply to KIPP's criticism of a study on charter spending that Baker co-authored. For a "no excuses" school, Baker notes, KIPP used "every possible explanation they could - every possible "excuse" - conjured every possible out of context - or different context estimate or "fact" to make their case that they in fact spend equal or less than schools in New York City and Houston."
KIPP defense defies its own bookkeeping on corporate foundation and similar financial contributions that have poured in and continue to pour into KIPP (contributions are listed on KIPP's webpage). Put it this way: If, for example, the more than 60 million dollars donated by both the Walton Family Foundation and the Doris and Donald Fisher Fund, as well as the many millions more from the Bill and Melinda Gates Foundation, the Citi foundation, the Charles and Helen Schwab Foundation, etc., etc. are not going towards the education of KIPP students, it is reasonable to wonder exactly where in the KIPP organization the money is going.
Once Again, Power Over Evidence
The paucity of the "best" research KIPP can hurl on its behalf and its rejection of all other research documenting its faults and failures are similar to the history and arguments over so-called "scientific reading instruction," the stepwise teaching that begins with phonological awareness, continues to phonics, them moves relentlessly onward as these pieces of reading supposedly accumulate in the student and eventually combine to create reading competence.
Sadly for the millions of young readers taught by this nationally mandated method, named "Reading First" in the No Child Left Behind legislation, success stories and purported supportive research never translated into widespread reading success. The method's failure came as no surprise to anyone who bothered to look, from the beginning, at the research that supposedly confirmed its use. The research simply didn't stand up! Nonetheless, the pushers of Reading First, especially many who gained financially and professionally from its use, kept hailing the research...the research.
Every criticism of this research was quickly identified and counterattacked by the "NORAD" system of the Child Development and Behavior Branch of the National Institute of Child Health and Human Development (NICHD), the chief federal office pushing this form of instruction. NICHD and the researchers associated with it produced a stream of defenses and counter-attacks that culminated in a supposedly "gold standard" and "independent" research document, vacuous though it was, that nonetheless served to ensure that Reading First rolled on and over students. A decade had to pass before the National Assessment of Educational Progress reading results revealed the failure of this instruction. Yet even that has done nothing because this failed pedagogy continues on in Common Core Standards instruction.
If educational issues were data driven, policy and solutions for poor children's lives and education would be much different from what they now are. However, current policy and purported "solutions" embodying the class war from above are not about data, but about power. Thus, for poor children the charter "solution" and KIPP roll on and on....and over and over.
What do you think of the evidence Gerald Coles discusses here?
Gerald Coles is a full-time researcher, writer, and lecturer on the psychology and politics of literacy and education. Before devoting himself to full-time research and writing, he was on the faculties of the Department of Psychiatry at Robert Wood Johnson Medical School, University of Medicine and Dentistry of New Jersey, and the Warner Graduate School of Education and Human Development at the University of Rochester. He is an active member of the Coalition for Justice in Education in Rochester, NY. He can be reached at email@example.com