« The Full Page Ad That Won't Be in the Washington Post Tomorrow | Main | NYC's Trojan Horse »

Full Page Ad in the NY Times: $178,633. The Center for Education Reform's Newsblast on DC Charters: Priceless!

| 17 Comments
mrbogus.jpg
This is too precious not to comment on: the Center for Education Reform, the organization that sponsored that full page ad slamming the AFT charter study and the Times in 2004, threw this celebratory paragraph into their newsblast today (see background here):
WHAT'S WORKING. D.C. charter schools are succeeding, according to The Washington Post, and "have opened a solid academic lead over those in [the city's] traditional public schools." An analysis of test results for economically disadvantaged students shows that "D.C. middle-school charters scored 19 points higher than the regular public students in reading and 20 points higher in math." D.C. charter school students outscored their conventional public school counterparts in other areas as well. In an online chat, writers discussed some of the reasons charters are thriving in the District: "A culture where the grown-ups trust each makes it a lot easier to teach kids. The experts doing research say high expectations and standards of behavior have to be applied consistently across classrooms. That's a lot more evident at charters than most DCPS schools I've visited. Charter directors were very consistent in saying that they will not hesitate to get rid of teachers who they feel are not performing. On the other hand, they are also very eager to keep the teachers they like, and provide support and encouragement and training to keep them happy."
This is a far cry from the fire-breathing condemnation of that ad, isn't it?: "The study in question does not meet current professional research standards. As a result, it tells us nothing about whether charter schools are succeeding."

You can't make this stuff up.
17 Comments

The way you describe this, there's really not any drama here.

- In the first situation, they pointed out that a study they don't think reports reliable information, based on methodology.
- In the second situation, they say that charter schools are doing well, based on a different study.

Not sure what you're aiming to convince me. If the problem is their tone in the first or second situations, I feel like it's kind of pot-meet-kettle ("too precious", "newsblast", "You can't make this stuff up", "Mr. Bogus" ...not really neutral tone).

I tend to think it's a good thing when people and groups show that they have the ability to change.

Hi Dave,
The methodology - a raw comparison of charter versus public average scores - is identical in both cases. The only difference is the finding.

Put simply, I find it troubling that interest groups mount massive campaigns against research findings that don't agree with their viewpoint, but do a 180 when an analysis with the same flaws does agree with their viewpoint. I don't think this is positive - or bodes well - for the public education debate.

Oh my, dear eduwonkette! We are so delighted by your coverage of the most important issues of the day. Alas, you broke the first cardinal rule of wonk-dom -- KNOW THY FACTS. We wonks and wonkettes must simply stick together, you know.

The extraordinary work of dozens of prize-winning researchers that signed the famous New York Times ad (including Nobel Laureates) spelled out the problem with bad journalism clearly. The New York Times had used an analysis of government data by a known opponent of charter schools to claim that charter schools were lagging other schools. It was akin to Philip Morris saying that tobacco is safe!

Besides the source, my dear, there is the sticky little issue of what the data really said. In this case, it was sample data of students in 4th grade compared across communities and states! In the case of the Washington Post's journalism this week, reporters -- not unions -- analyzed the results of test scores from ONE community, and amidst those test scores are data for both (EVERY) conventional and charter public school, not just a sample, allowing for apples to apples comparisons.

If you are saying you don't believe the Post, that's okay. Many others have felt that way for years. However, if you are saying we are biased or have a double standard, you are simply wrong, lovely little wonkette. We respectfully beg to differ.

In the coming days, in between other important elf duties, we will show you more. For now, we bid you a fond farewell and a wish that in addition to world peace, we all embrace the notion of digging deeper than just what seems to meet the eye.

Please join us over on Edspresso.

Hi Jeanne, Thanks for stopping by. That this is one community does not resolve one of the central critiques made in your ad and in that debate - that "The study in question does not meet current professional research standards" - because it does nothing to address the issue of selection into charters.

More concretely - your logic above about the methodological differences in the two studies (comparing within one city and sampling) suggests that you would be okay with comparing the average scores of charter and non-charter schools within DC, not just the scores of disadvantaged kids. In that case, DC charter schools are either no different or behind in 4th grade math and reading. It would be a mistake to make a facile comparison like this for all of the reasons that I discussed in the earlier post, and I suspect that charter advocates would be up in arms if the article instead foregrounded this point.

Unfortunately, Jeanne Allen is dead wrong on this one.

There are virtually no differences between the methods used by the AFT (and publicized by the Times) and the methods used by the Post reporters.

For one, both studies use NAEP. The NYT ad spends considerable space pointing out NAEP's limitations for these kinds of statistical inferences. (They are only "snapshots," with limited family background information attached, etc).

Second, both studies are straight comparisons of mean scores. The NYT ad criticized the "unsophisticated analysis" used by AFT, that "considers differences in only one family background characteristic at a time." The exact same thing can be said for the Post: they control only for economic disadvantage. Jeanne is correct in pointing out that this comparison is within one district (DC), which is more sensible than a nationwide comparison. But the advantages end there.

I can't speak for my economist colleagues who signed the NYT ad. But I am confident that all of them recognize that the Post analysis is NOT an "apples to apples comparison," by any stretch of the imagination.

This is why these very same researchers strive in their own work to base their causal claims on randomized experiments, or quasi-experimental evidence that approaches randomization. Only in these cases can we be confident of an "apples to apples" comparison.

Jeanne Allen and the NYT signers were correct in pointing out that we need higher quality research than the AFT study to make causal claims about charter schools. The same standard should be applied here.

Sean Corcoran
New York University

I am new to the discussion and have much to learn. I do have a thought about the point that has been made about comparing the success of students in charter schools versus the students in non charter schools.

It seems valid to say that students and parents that make the step of choosing to attend a charter school have more initiative, care more about their education, and are more likely to succeed wherever they attend school. The argument that comparing students that attend charters and students that don't is not a valid research experiment makes sense in that light.

However, doesn't the fact that people have accepted as fact the idea that an engaged, diligent, responsible parent that wants the best for their children will choose a charter school basically mean that we believe that charters are the better option? Or at least that the state of the traditional schools is so poor that responsible parents are right to want their children to have a better alternative?

Doesn't that logic make sense?

Chad: I think you're right that it would be reasonable for well-informed parents in DC to consider charters as an alternative for their children that may be better than the traditional DC public schools. My criticism of many articles on charter schools is that they falsely make the assumption that the charter schools themselves are somehow better schools (by dint of teacher quality or other variables) without taking into consideration this selection bias. I think it's quite possible that a school that accepts only motivated students with a demonstrated desire to improve their educational attainment may be a good place for its students. But I haven't seen any evidence that this is improvement is the result of anything the charter schools do over and above the effects of accepting only motivated students and holding them to higher levels of effort and behavior than traditional public schools are allowed by law to do.

Pretend the data were trouble free, the students were randomly assigned, the data were from elementary and high schools, not just middle schools, and that the test scores were the sole criteria of success.
Now, group the KIPP schools-- which are part of their own system, not independents --together with the single original DCPS school,enrollment 600, that went charter long ago to sustain its high achievement, and still serving about the same mostly black nad not well of demographic in the same building. (And here's a shout-out to Paul JHS principal Barbara Nophlin, (formerly with DCPS)!!)
Findings:
1)The students in KIPP and Paul JHS PCS outperform the rest of the charters
2) the Paul students are indistinguishable from the KIPP students; and
3)there is no statistical difference between the rest of the charters and the DCPS schools.

3 groups, 2df, & 1 statistically significant difference. Schools out.

I press the plunger to stop time running on my clock. No apology for excessive intimacy with the data. Over to anyone else to now talk about demonstrated charter school "effects" on achievement in DC

One of the most surprising things I learned when I became a teacher in 1964 was the close correlation between a parent's interest in his or her child's education and the level of achievement of the child. So even in the worst schools in Cleveland, a child with a very involved parent would do fairly well. Often that parent would look for ways to get the child into a "better" school. In 1964 that meant a parochial school or a school on the "west side." (more affluent part of town)

When I had children of my own I became acutely aware of the behavior of the other children. My top priority became the safety of my child. When things got too rowdy at the local public school I put my sons in a Catholic school. I had no knowledge of the teachers but I knew that the students were well-behaved; that's what I was after. My sons went on to the best universities in the country and are successful professionals today.

I shared this story because there are just so many variables involved in a child's education that it is close to impossible to say why one school is "better." Did the Catholic school I sent my sons to have better test scores because of better teachers or did they have a select student body? I'm almost certain it was the latter because all parents chose the school and paid tuition. Almost any school chosen by parents will appear superior to a school that accepts children who are assigned to it.

I have a question for Jeanne Allen. Do you think most citizens realize how powerful non-school factors are? I didn't have any idea until I started teaching. Thanks.

One of the most surprising things I learned when I became a teacher in 1964 was the close correlation between a parent's interest in his or her child's education and the level of achievement of the child. So even in the worst schools in Cleveland, a child with a very involved parent would do fairly well. Often that parent would look for ways to get the child into a "better" school. In 1964 that meant a parochial school or a school on the "west side." (more affluent part of town)

When I had children of my own I became acutely aware of the behavior of the other children. My top priority became the safety of my child. When things got too rowdy at the local public school I put my sons in a Catholic school. I had no knowledge of the teachers but I knew that the students were well-behaved; that's what I was after. My sons went on to the best universities in the country and are successful professionals today.

I shared this story because there are just so many variables involved in a child's education that it is close to impossible to say why one school is "better." Did the Catholic school I sent my sons to have better test scores because of better teachers or did they have a select student body? I'm almost certain it was the latter because all parents chose the school and paid tuition. Almost any school chosen by parents will appear superior to a school that accepts children who are assigned to it.

I have a question for Jeanne Allen. Do you think most citizens realize how powerful non-school factors are? I didn't have any idea until I started teaching. Thanks.

Almost any school chosen by parents will appear superior to a school that accepts children who are assigned to it.

A corollary to this -- that benefits charters when compared to district schools -- is that almost any impediment to enrollment, even having to sign up for a lottery, will select for motivated, involved parents, and change the student body in subtle ways that a simple comparison of income or parental education wouldn't illuminate.

The harder issue is that these school may actually provide a better educational environment simply by selecting for motivated parents. Where does that leave the kids of uninvolved, unmotivated parents? Should education be tracked that way, and if so, to what extent?

What is it reasonable to expect from parents? Signing up for a lottery? Checking homework every night? Coming in for a weekly conference? Volunteering 4 hours a month? Volunteering 4 hours a week?

What sort of financial commitment is consistent with public education? Buying uniforms? Voluntary purchases of wrapping paper and magazines? Voluntary $10 PTA dues? Voluntary $100 PTA dues? Voluntary $1000 PTA dues?

My experience supports Linda's view that an engaged, informed, and interested parent will find a way to make sure their child is in position to receive the education and training needed to be a responsible, contributing member of society.

Is there a consensus that increasing the amount of parents that fit this description is the best thing we can do to improve the education of as many children as possible?

If so, are there any success stories where such an initiative has taken hold?

My experience supports Linda's view that an engaged, informed, and interested parent will find a way to make sure their child is in position to receive the education and training needed to be a responsible, contributing member of society.

Is there a consensus that increasing the amount of parents that fit this description is the best thing we can do to improve the education of as many children as possible?

If so, are there any success stories where such an initiative has taken hold?

Chad -- I think parent outreach and education is a priority for many districts (it certainly is for ours) but I haven't seen much national-level of discussion about what is particularly effective.

"Almost any school chosen by parents will appear superior to a school that accepts children who are assigned to it.

A corollary to this -- that benefits charters when compared to district schools -- is that almost any impediment to enrollment, even having to sign up for a lottery, will select for motivated, involved parents, and change the student body in subtle ways that a simple comparison of income or parental education wouldn't illuminate."

Following this same logic, could we then conclude that adoptive parents--who have lots of hoops to jump through in order to become parents--are better, more involved and highly motivated parents than the biological kind?

Margo: I don't think that's the point of these comparisons. Obviously, many adoptive parents are very involved and caring, just as many biological parents can be involved and caring. The point is that articles comparing the academic achievement of charter schools vs. public schools do not always account for the selection bias that goes into the charter schools' student populations.

Articles often state that the SES characteristics of public vs. charter school students are roughly equivalent. They then conclude that the two groups of students are basically the same going into the school year - and further conclude that all differences in performance by the end of the school term are due to some innate characteristic of the charter schools themselves. Some of the posters on this site have pointed out that there is a selection bias in charter and private schools: Only families who are tuned into their kids' education to a certain degree will apply. Therefore, the student populations are NOT equal.

An appropriate analogy might be comparing students who try out for the school soccer team with those who don't attend tryouts - having them play in an end-of-year soccer tournment. The kids who take the time to try out for the team (even if they don't make it) will probably be, on average, better players than the students who don't try out for the team. Assuming the soccer team kids beat the non-soccer team kids in an end of year match-up, assigning all the credit to the soccer coach is not appropriate b/c the student groups are not the same at the beginning. Even if they are they have the same general athletic ability, there is obviously a level of motivation that the "tryout" kids possess.

Atty:

I get the point, and I was being a bit tongue in cheek with my suggestion--but not totally. Kids are pretty randomly assigned to bio parents, and generally stay with that randomly assigned family unless the parents are really profoundly deficient, over time.

Adoptive families have to go through a selection process that makes applying to a charter school like like a walk in the park on a sunny day. They have to be motivated to even go through all that--not to mention that fact that nobody ever becomes an adoptive parent by accident. They are frequently expected to be not only open to the demands of parenting, but frequently expected to be open to the demands of parenting the neediest of kids (those whose bio parents were deemed profoundly deficient, over time).

This comes to mind because adoptive parents are so frequently blamed by various school and other authorities for "having" children with so many problems--but also because it seems to be one of the least important questions to be asking. I understand the importance of taking into account selection bias in looking at school comparisons--but I would prefer to be looking at something more substantial. Are there critical differences in teaching methodology? In class size? In experience? In credentials? In per pupil expenditure? In administrative cost vs instructional cost? In any of the things that can be controlled within the school? We are so enamored of clumping together unlike school entities with a single commonality (charter, public, private, etc). Underlying the concern for parent selection bias, it seems to me, is a belief that somehow kids arrive at the school door with innately determined capabilities that schools can do little to impact.

Yes, families have an impact. Knowing that, and 4 bucks, will get you a cup of coffee from your local barrista. I think that the more critical question is what can we do within the schools that we have right now, and the students that we have right now to get closer to a universal level of basic literacy and numeracy?

Comments are now closed for this post.

Advertisement

Recent Comments

  • Margo/Mom: Atty: I get the point, and I was being a read more
  • Attorney DC: Margo: I don't think that's the point of these comparisons. read more
  • Margo/Mom: "Almost any school chosen by parents will appear superior to read more
  • Rachel: Chad -- I think parent outreach and education is a read more
  • Chad: My experience supports Linda's view that an engaged, informed, and read more

Archives

Categories

Technorati

Technorati search

» Blogs that link here

Tags

8th grade retention
Fordham Foundation
The New Teacher Project
Tim Daly
absent teacher reserve
absent teacher reserve

accountability
accountability in Texas
accountability systems in education
achievement gap
achievement gap in New York City
acting white
admissions
AERA
AERA annual meetings
AERA conference
AERJ
Alexander Russo
Algebra II
American Association of University Women
American Education Research Associatio
American Education Research Association
American Educational Research Journal
American Federation of Teachers
Andrew Ho
Art Siebens
ATR
Baltimore City Public Schools
Barack Obama
Bill Ayers
black-white achievement gap
books
books on educational research
boy crisis
brain-based education
Brian Jacob
bubble kids
Building on the Basics
Cambridge Education
carnival of education
Caroline Hoxby
Caroline Hoxby charter schools
cell phone plan
charter schools
Checker Finn
Chicago
Chicago shooting
Chicago violence
Chris Cerf
class size
Coby Loup
college access
cool people you should know
credit recovery
curriculum narrowing
D3M
Dan Willingham
data driven
data-driven decision making
data-driven decision-making
David Cantor
DC
Dean Millot
demographics of schoolchildren
Department of Assessment and Accountability
Department of Education budget
desegregation
Diplomas Count
disadvantages of elite education
do schools matter
Doug Ready
Doug Staiger
dropout factories
dropout rate
dropouts
education books
education policy
education policy thinktanks
educational equity
educational research
educational triage
effects of neighborhoods on education
effects of No Child Left Behind
effects of schools
effects of Teach for America
elite education
ETS
Everyday Antiracism
excessed teachers
exit exams
experienced teachers
Fordham and Ogbu
Fordham Foundation
Frederick Douglass High School
Gates Foundation
gender
gender and education
gender and math
gender and science and mathematics
gifted and talented
gifted and talented admissions
gifted and talented program
gifted and talented programs in New York City
girls and math
good schools
graduate student union
graduation rate
graduation rates
guns in Chicago
health benefits for teachers
High Achievers
high school
high school dropouts
high school exit exams
high school graduates
high school graduation rate
high-stakes testing
high-stakes tests and science
higher ed
higher education
highly effective teachers
Houston Independent School District
how to choose a school
IES
incentives in education
Institute for Education Sciences
is teaching a profession?
is the No Child Left Behind Act working
Jay Greene
Jim Liebman
Joel Klein
John Merrow
Jonah Rockoff
Kevin Carey
KIPP
KIPP and boys
KIPP and gender
Lake Woebegon
Lars Lefgren
leaving teaching
Leonard Sax
Liam Julian

Marcus Winters
math achievement for girls
McGraw-Hill
meaning of high school diploma
Mica Pollock
Michael Bloomberg
Michelle Rhee
Michelle Rhee teacher contract
Mike Bloomberg
Mike Klonsky
Mike Petrilli
narrowing the curriculum
National Center for Education Statistics Condition of Education
NCLB
neuroscience
new teachers
New York City
New York City bonuses for principals
New York City budget
New York City budget cuts
New York City Budget cuts
New York City Department of Education
New York City Department of Education Truth Squad
New York City ELA and Math Results 2008
New York City gifted and talented
New York City Progress Report
New York City Quality Review
New York City school budget cuts
New York City school closing
New York City schools
New York City small schools
New York City social promotion
New York City teacher experiment
New York City teacher salaries
New York City teacher tenure
New York City Test scores 2008
New York City value-added
New York State ELA and Math 2008
New York State ELA and Math Results 2008
New York State ELA and Math Scores 2008
New York State ELA Exam
New York state ELA test
New York State Test scores
No Child Left Behind
No Child Left Behind Act
passing rates
Pearson
picking a school
press office
principal bonuses
proficiency scores
push outs
pushouts
qualitative educational research
qualitative research in education
quitting teaching
race and education
racial segregation in schools
Randall Reback
Randi Weingarten
Randy Reback
recovering credits in high school
Rick Hess
Robert Balfanz
Robert Pondiscio
Roland Fryer
Russ Whitehurst
Sarah Reckhow
school budget cuts in New York City
school choice
school effects
school integration
single sex education
skoolboy
small schools
small schools in New York City
social justice teaching
Sol Stern
SREE
Stefanie DeLuca
stereotype threat
talented and gifted
talking about race
talking about race in schools
Teach for America
teacher effectiveness
teacher effects
teacher quailty
teacher quality
teacher tenure
teachers
teachers and obesity
Teachers College
teachers versus doctors
teaching as career
teaching for social justice
teaching profession
test score inflation
test scores
test scores in New York City
testing
testing and accountability
Texas accountability
TFA
The No Child Left Behind Act
The Persistence of Teacher-Induced Learning Gains
thinktanks in educational research
Thomas B. Fordham Foundation
Tom Kane
Tweed
University of Iowa
Urban Institute study of Teach for America
Urban Institute Teach for America
value-addded
value-added
value-added assessment
Washington
Wendy Kopp
women and graduate school science and engineering
women and science
women in math and science
Woodrow Wilson High School