« | Main | Why School? Rethinking Essentials »

The NCLB Paradox Enters the Twilight Zone

| 38 Comments

Dear Deborah,

Over the past week, you and I have each weighed in on the defects of testing. You have been arguing for many years that standardized testing is replete with flaws. I have only recently recognized the ways in which pressure to raise scores, mainly prompted by NCLB, has corrupted testing and accountability.

Our policymakers have fallen in love with the idea that incentives and sanctions can "drive" educational improvement. They believe that if we promise rewards when test scores go up, we will see test scores go up. So they commit hundreds of millions of dollars to give "merit pay" or "performance pay" to teachers and principals, even to students—if the scores rise. Simultaneously, they threaten to inflict serious sanctions on those schools, principals, and teachers if their students' test scores do not go up. They don't dock their pay, but do something worse: They threaten to close their schools, fire the staff, and tarnish the reputation of anyone who taught there.

Behind these promises and threats lies a simple theory: scores are not high enough, because teachers are either lazy, don't work hard, or aren't motivated enough to do a good job. So teachers will work harder and be more successful if they can get more money, and they will work harder and be more successful if their livelihoods and reputations are on the line.

The problem with the incentives and sanctions approach is that it works. It does produce higher scores. We see scores going up in many states, sometimes at rates that defy belief. Some states may actually reach that dreamy goal of 100 percent "proficiency" by 2014.

So what's the problem? The problem is that schools, principals, teachers, and students will reach the goal by hook or by crook. Some states, like New York and Illinois, will play statistical games (like dropping the cut point, or creating conversion tables to change low scores into high scores). Some states will dumb down their tests, carefully field-testing the tests and removing any questions that are too difficult. Some districts will scrub their scores to remove low-scoring students (yes, this has happened). Some districts will find other ways to exclude the low-scoring students, such as by giving accommodations to students who are not usually entitled to them. Some schools will reclassify students to put their lowest scoring students in a group that doesn't "count" because its numbers are so small. Some will even cheat.

Ultimately, we will have what I call the NCLB Paradox, wherein scores go up, but actual educational improvement does not occur. We will see districts where the reading and math scores are through the roof, and where graduation rates have climbed, but where the rate of college-ready students is unchanged. I expect there are many such districts. The one I know best is New York City, which won the Broad award in 2007 for its excellence in improving urban education. Test scores have soared, based on dumbed-down tests; graduation rates are up across the board. Yet when graduates of the New York City public school system enter the community colleges of New York City, 74 percent of them require remediation in basic skills! These are students who passed five state Regents examinations, yet they need to be remediated in reading, writing, and mathematics! This suggests, does it not, that there is something amiss with those impressive test scores and graduation rates?

This raises the question: With scores so often rigged and fraudulent, how can we use them to pay bonuses or to close schools? New York City's last round of phony test scores (noticed as phony even by the august New York Times) triggered a payout of $33 million in bonuses to teachers; the union is laughing all the way to the bank! So millions are awarded in fraudulent bonuses at the same time that school budgets are cut to the bone. Is this the way that big business operates? If so, it is no wonder that we had a financial meltdown.

I fear that American education has now entered into a twilight zone, where nothing is what it appears to be, where numbers are meaningless, where public relations and spin take the place of honest reporting, where fraud is called progress.

Diane

38 Comments

If we agree that the emphasis on testing has led to what amounts to false advertizing -- a twilight zone -- may we disagree about this being a new thing?
The pendulum has swung this way before. In fact, that golden age when assessment truly reflected how education was going seems a mythic thing. Was that ever even the intent? Wasn't testing, by its very nature, rigged towards the interests of school-as-business?

Meanwhile, I think it's important to remember that learning goes on: not always where we expect it or how we imagine it should go -- and certainly not in a way that tests are very good at documenting. But in our sadness and anger over how school works these days, we can't afford to forget that for a bunch of kids inside the system, it's not twilight at all but dawn. And they are finding ways to learn both with and without adults, classrooms, and tests.

I agree with the writer of this blog when saying that these state test are a bit fraudulant. We want each student excel in all their work but its not reality for every single student in the United State Public School System to get 100% on all their state test.

The simple truth in my district is this: teachers took their jobs because they wanted to help kids, but the environment all too soon twisted them. Teachers who would have willingly flung themselves into the fray armed with rubrics, authentic assessment, and meaningful parent contact are crushed into a small cube, pressed in on all sides by redundant paperwork, meaningless data analysis (when meaningFUL data analysis is quite possible but not worshipped by the school systems), enforced testing regimens, and curriculum-based rather than community-based schooling.

Combine that with diminishing funds and a dazzingly wide array of new initiatives, each requiring a little more unpaid work from the teachers, and it is absolutely no wonder that kids are not coming out of the school system with minimal levels of competence.

Accountability measures must be rethought in a transparent way. There must be an open and organized discussion. Real experts must be called in from across the field of education: teachers, psychologists, administrators, even politicians. Leave the businessmen out of it, though. Those same children who couldn't make it in college were perfect fodder for the industrial machine that drives more wealth into hands of a very few at the expense of a great many.

We may have passed through the twilight zone into something even more surreal.

When "proficiency" is defined in terms of cut scores on a test, whether the test is state or national, it's loony toons time.

When a school district is awarded the Broad Prize when it's test scores are no higher than the State averages and it's SES classification is above state averages, it's bonkers.

When IES studies that show "no significant differences" in mandated nostrums are buried on the one hand, while calling for data-driven decision making on the other hand, it's Alice in Wonderland.

When the "What Works Clearinghouse" doesn't, the clock is striking 13.

Etc., Etc.

And there is every sign that conditions will get worse.

The "Common Core State Standards"
http://www.corestandards.org/Standards/index.htm
are conceived in naivety that is exceeded only by that displayed in the
"Technological Literacy Framework
for the 2012 National Assessment of
Educational Progress."
http://www.edgateway.net/cs/naepsci/download/lib/234/090821%20NAEP%20Tech%20Lit%20Framework%20Discussion%20Draft.pdf?x-r=pcfile_d

The only basis for optimism I can see is that the condition of the el-hi enterprise is not as dire as it's been painted. Kids have adequate assets and teachers are adequately trained to reliably achieve the academic accomplishments to which the citizenry aspires. Parents are reasonably satisfied with their own kids' school, even when it's crumby.

The public school system, with all its flaws is a robust institution and it does many things well. Ironically, it's only serious weakness is in reliably delivering basic instructional capabilities--its primary function.

Yes, we need change we can believe in, but the changes are in instruction--which is receiving (all but)no attention in the focus on standards and testing.

Kids try to learn what they are taught. Many of them do, despite unintended mal-instruction. But loose appeals to "formative evaluation" and "response to intervention" will not provide the feedback necessary
to address the situation.

There is methodology to provide instructional intelligence (what's going on) analogous to business intelligence. Planned variations can also be unobtrusively conducted. But such change points to the top of the EdChain, not the bottom. And the top is being well served by the status quo and by current initiatives.

As a former public school teacher and now, professor of education preparing new teachers, we always gave standardized tests to our students. The difference then was that the tests were used to inform our teaching for the next year. We did it to improve on what we were doing, not for money, hence no reason to cheat or lie about results. Standardized testing has become the gorilla on the back of all dedicated teachers who entered the field to better the lives of students. Ah...I do long for those good old days when teachers were trusted to do their jobs and improve themselves where needed.

I want to know more about the passage of five Regents exams. Are their scores on Regents above the minimal pass that are correlated with not needing remediation? Do the Regents have a high enough ceiling to be relevant to real college readiness and college success.


Diane Ravitch's description of the "twilight zone" of education is accurate, leading to the conclusion that we are approaching a "night" that may not end.

The fact is that most of the people making educational decisions today are not qualified to make them. At the local level bad decisions can be reversed, they are resisted, or they die of their own toxicity . At the national level the harm is wide-reaching, deep, and long-lasting. I fear that this generation of students will not survive the destruction of their intellects begun by "A Nation at Risk," intensified by "No Child Left Behind, and now institutionalized by "The Race to the Top."


Diane Ravitch's description of the "twilight zone" of education is accurate, leading to the conclusion that we are approaching a "night" that may not end.

The fact is that most of the people making educational decisions today are not qualified to make them. At the local level bad decisions can be reversed, they are resisted, or they die of their own toxicity . At the national level the harm is wide-reaching, deep, and long-lasting. I fear that this generation of students will not survive the destruction of their intellects begun by "A Nation at Risk," intensified by "No Child Left Behind, and now institutionalized by "The Race to the Top."

Mike Kirst,
The Regents have been dumbed down by the NY State Education Department, along with the 3-8 tests. The state uses a conversion chart, so that a student can answer less than half the questions correctly and the score is converted magically to a 65.
Watch for the new State Commissioner of Education David Steiner to try to bring transparency to the Regents exams. Let's hope he succeeds. It is a scandal that so many kids get a diploma that signifies nothing.

Diane

Hi All... hope this finds you well.

Things are moving very quickly!

Update Common Core Standards:

The Common Core State Standards is a joint effort by the National Governors Association Center for Best Practices (NGA Center) and the Council of Chief State School Officers (CCSSO) in partnership with Achieve, ACT and the College Board.

Governors and state commissioners of education from across the country committed to joining a state-led process to develop a common core of state standards in English-language arts and mathematics for grades K-12.

These standards will be research and evidence-based, internationally benchmarked, aligned with college and work expectations and include rigorous content and skills. The NGA Center and CCSSO are coordinating the process to develop these standards and have created an expert validation committee to provide an independent review of the common core state standards, as well as the grade-by-grade standards.

The NGA Center and CCSSO released the first official draft of the college- and career-readiness standards. Feedback on the draft is being accepted until October 21, 2009.

If you are interested in seeing the standards and providing feedback...your opportunity to do so can be found here: http://www.corestandards.org/

Now is the time for your feedback!!!

On another note: Diane... the use of testing at the college level to determine remedial class requirements is another area of the current testing movement that needs to be looked at very carefully!!! If you dig into that world a bit.... i think you would find very similar concerns about testing and its uses on kids!!

be well.... mike

Diane writes, "I fear that American education has now entered into a twilight zone, where nothing is what it appears to be, where numbers are meaningless, where public relations and spin take the place of honest reporting, where fraud is called progress." I suggest you run and get yourself a copy of Chris Hedges' new book, Empire of Illusion. The End of Literacy and the Triumph of Spectacle. What you fear was put in place long before NCLB became law...

I have two questions that I hope someone answers. I wrote to a researcher recently and he admitted to me that these test scores of New York and many other cities are probably fraudulent; but he said "it's all we have." My questions are: If "everyone" knows that these scores are probably invalid, why are university researchers using them? Why don't they insist on doing their own testing? If the Stanford research is based on incorrect information, look at the damage it could do in the sense of affecting the direction that educational reform takes.

Daniel Wolff writes:
If we agree that the emphasis on testing has led to what amounts to false advertizing -- a twilight zone -- may we disagree about this being a new thing?
The pendulum has swung this way before.

It seems to me that the testing "tail" is wagging the dog more strongly than it ever has before, and that however much policy makers give lip service to "the whole child," what comes out of Washington (and out of many state capitals) is about testing.

Over the past week, you and I have each weighed in on the defects of testing. You have been arguing for many years that standardized testing is replete with flaws. I have only recently recognized the ways in which pressure to raise scores, mainly prompted by NCLB, has corrupted testing and accountability.

Our policymakers have fallen in love with the idea that incentives and sanctions can "drive" educational improvement. They believe that if we promise rewards when test scores go up, we will see test scores go up. So they commit hundreds of millions of dollars to give "merit pay" or "performance pay" to teachers and principals, even to students—if the scores rise. Simultaneously, they threaten to inflict serious sanctions on those schools, principals, and teachers if their students' test scores do not go up. They don't dock their pay, but do something worse: They threaten to close their schools, fire the staff, and tarnish the reputation of anyone who taught there.

Behind these promises and threats lies a simple theory: scores are not high enough, because teachers are either lazy, don't work hard, or aren't motivated enough to do a good job. So teachers will work harder and be more successful if they can get more money, and they will work harder and be more successful if their livelihoods and reputations are on the line.

The problem with the incentives and sanctions approach is that it works. It does produce higher scores. We see scores going up in many states, sometimes at rates that defy belief. Some states may actually reach that dreamy goal of 100 percent "proficiency" by 2014.
Diane writes:
"So what's the problem? The problem is that schools, principals, teachers, and students will reach the goal by hook or by crook."....She says "Some will even cheat." I personally fight the cheating came everyday because I know cheating corrupts educationa and corrupts learning. Just today I picked up the notebooks of a student whom I though was showing real improvement. But what I found instead was a cheat sheet with all the answers including wrong answers. I stapled the cheet sheet to his test which i marked SEE ME and NC (no credit). When I suspect cheating (I don't catch everyone) I give them a double F. If students do poorly on quizzes I let them take them over again for full credt. But above all my tests and I repeat MY tests are authentic instuments. No bubble tests at all. Students on the easiest portion do matching (filling in the Blank and writing name or answer good practice.) Then they must identify or explain (people or terms ) and write in short sentences and the last part of the test they have to actually compare and contrast. For lower level students they have to write a short essay on a them I told them to study; for higher level students they have to write a series of essays on, for example, why the printing press was so influential in creating advances in education, indivudal rights and also the Protestant Reformation. All my students have to learn that the Renaissance is a precursor to the Enlightenment and when and what the Enlightement is. When students have to read and explain things they demonstrate what they know. I also put famous quotations and have them interpret them (even for the state tests they have to interpret primary documents). All I can say is what my students tell me. They appreciate how much they learn with my testing regime. They also say they are bored with multiple choice tests and they tell me cheating is much more rampant when all teachers use the same test and the same scantron form. One never knows what these tests mean. One has to always consider that they are faked. Over and over I give a "common formative assessment"and oddly students who have done nothing get 5 our of 5 when good sthdents get 4 out of 5. But on the written portions I give the student with the 'perfect' scores leaves the advanced portions of the test entirely blank!!!
Diane says
"Ultimately, we will have what I call the NCLB Paradox, wherein scores go up, but actual educational improvement does not occur."

I once served in the military. The Marines themselves used surveys but they did not judge units on those things alone or on time tables but by a whole variety of instruments which included peer reviews and reports by observers -both officers and NCO's-on the real life performance of units. The Marines were not interested in great inspections if they were faked. They were intereseted in teaching good habits, good hygiene and saving lives while being prepared to do whatever they could to meet the mission. And above all INTEGRITY was the watchword. If the fitness reports or inspections did not reflect the integrity of the officer or men they were considered invalid. Period.

When Diane notes, wisely that rates of college ready students are unchanged (or SAT I, SAT II or AP scores are flat), then there is something rotten in Denmark. One has to ask: "Do these tests have integrity?" Once again, it seems to me that the CAHSEE (high school exit exam; administered by ETS) has integrity and so do AP tests. But local benchmarks and state tests (all multiple choice) do not have the same sort of validity.

Once again, I can only speak for myself but if students of mine who are not D students graduate from HS and pass the CAHSEE (exit exam) and then need extensive remediation then I have failed and the school has failed. But over and over my students who go to college tell me my classes perpared them for college. Our JROTC students say the JROTC program gave them the edge for boot camp.
Diane wrote
"Yet when graduates of the New York City public school system enter the community colleges of New York City, 74 percent of them require remediation in basic skills! These are students who passed five state Regents examinations, yet they need to be remediated in reading, writing, and mathematics! This suggests, does it not, that there is something amiss with those impressive test scores and graduation rates?"
TO THIS I say: yes absolutely. There must be something amiss and even worse than that I think. Schools are giving out hollow credits and degraded diplomas. And to take money and bonuses for that is dishonest and corrupt.
I taught AP classes for 12 years and never got a penny for teaching Satudays or hundreds of passing exams. I only had the satisfaction that I was able to help at risk kids show they could be competitive with the best students nationwide on authentic and truly difficult tests.

One thing I like about Diane is that she listens to people on the ground like a good batallion commander or a good division commander. That's why she writes
"I fear that American education has now entered into a twilight zone, where nothing is what it appears to be, where numbers are meaningless, where public relations and spin take the place of honest reporting, where fraud is called progress."

I love accurate tests. I love to learn from tests which is why i personally grade every test and essay I give. I will say again teacher who only rely on scantron bubble tests remove themselves a step from the peformance of their students. They do not learn as much about the test and about their students as I do. And as hard as I work my tests have another benefit. Sometimes they are hilarious (by accident), and sometimes they are very wise. Ten students can get 100% but when I read an unusual and most excellent test I see great talent, extraordinary talent. And these same students have won top academic prizes such as the Gates Millenium Scholarship or full rides, to Yale, Bryn Mayr, Berkeley etc. Or they have become interpretors with the courts or the military. Only the teacher in the classroom really knows his or her students. Only the teacher knows if they have progressed to excellence or independence. My grades are a very accurate predictor of high school and college performance because I use a variety of instruments. I don't rely on one narrow test. If narrow tests give postive results and I think the students know nothing I begin to suspect the test isn't secure. So I try something else. When everything is mandated one is just a cog and there is nothing else. It is just idolatry and scientism not education. Just more worshipping at the foot of the Scantron God. And Sophia, wisdom, weeps.

Dear Diane: I posted on Bridging Differences and I think I see in your comments
real concerns by real classroom teachers (not just me but others). Your
observations were right on target. Our test taking regime has become like a
devoring Monster from the Id (remember Forbidden Planet?). Dr. Morbius wanted
all knowledge but he forgot MODERATION reinforces other virtues and lack of
moderation (temperance) WEAKENS THEM. I love learning -as you know. I love
kids -as you know. I even love authentic tests -as you know. I am a good
soldier and try to carry out without insubordination what I am told to do. I
have never disliked or opposed the CAHSEE (exit exam). I have never dislike or
opposed the AP tests (but I have opposed the idiocy that anyone can sign up for
these clases without teacher permission and without GPA's. When I taught AP
classes I poured on the work the first six weeks (like boot camp) to wash out
all the wimps. I never in 12 years gave a single F or D. I didn't have to .
The D daredevils were long gone.

But NCLB is an all consuming monster. Our Edusoft is an all consuming monster.
It is for me, with the soul of a poet, a journey of the cross. I never thought
anything could kill my love for education and teaching but now I have found my
nemesis. I used to tell students I would be the last teacher in America to
succomb to the Scantron God. And yet, without my wanting it, 40 days a year are
spent administering and proctoring bubble tests. Gone are the lovely maps and
art projects. Gone are the oral reports. Gone are the posters of famous
quotations. Gone are the book reports. Gone is much of the enchantment of
learning. For the first time in my life I am closer to the end of my career
than the beginning and I am glad.

That should be for you the most damning evidence of NCLB and its ID Monster
Bubble Tests. But I will not stop teaching but I think with joy when i am
tutoring catechists and teaching in a Catholic school part time for only $1000 a
month like when I was young. And I know then teaching will be service and joy
again. When I am in that educational paradise and among my books and disciples
I will not miss the Murder Machines that are America's schools.

RICARDO

First, I am not sure how to treat long posts that seem to provide a lot of experience-based judgment, but then they say something like:

"When Diane notes, wisely that rates of college ready students are unchanged (or SAT I, SAT II or AP scores are flat), then there is something rotten in Denmark. One has to ask: "Do these tests have integrity?" Once again, it seems to me that the CAHSEE (high school exit exam; administered by ETS) has integrity and so do AP tests. But local benchmarks and state tests (all multiple choice) do not have the same sort of validity"

Well, it so happens that CAHSEE (Calif. High School Exit Exam) is a state test, and consists solely of multiple choice items, except for a writing prompt, yet Richard likes it. But he doesn't like "local benchmarks and state tests (all multiple choice)." I can't say much about local benchmarks (except possibly they they seem to be overused) but the "state tests" in Calif. are the CSTs (Calif. Standards Tests) and they are ... all multiple choice, except for the writing prompt. So much for the "difference."

Moving on to more substantive issues, Diane identifies the deficiencies of current high stakes (for teachers) testing regimes. I agree with most of them, although I think that at this point in time Diane exaggerates the deficiencies and belittles the benefits. For example, while NY (and NYC in particular) may have monkeyed with its tests, Calif. did not, and the data is comparable since 2001.

Be it as it may, what I miss is her suggested solution. I think I know what Deborah Meier suggests--abolish the abomination of student testing and trust the teachers. What I am not sure about is Diane's proposed solution. Abolish regular individual student testing? Provide national standards and national student testing? Trust the teachers? Leave teachers salary as today based solely on longevity and provably meaningless certification? Remove accountability aspects from schools? Don't collect or report achievement data except for NAEP?

Accountability and accountability-related testing came into wide use for a reason. What's your suggestion for replacement?

Let's take one statement and look at it. "Some states will dumb down their tests, carefully field-testing the tests and removing any questions that are too difficult."
With 37 years in the classroom and 10 years authoring and reviewing questions for Cisco Systems I have found that field testing validates the question. Any question that scores low across all demographics could, very possibly, be a bad question. Removing that question or reautoring it is not dumbing down. It is the only way to get valid data about the student's learning.

Hi All... hope this finds everyone well.

The real issue for me isn't about the test results.....

the real issue to me isn't about the standards being "good" or "bad".....

the issue is "STANDARDIZATION" period.

My take, we fail to see the "industrial-factory" thinking and its assumptions that follow the development of standards. It runs always in a very predictable manner...regardless of the "quality of the standards" or the "quality of the tests".

Think....train track or assembly line and go down the line a bit.....it always goes to the same place!!!

Here is the track and the almost automatic thinking that follows:

Step 1: KNOWLEDGE: All the current knowledge to date...now doubling extremely quickly.

Step 2: Gather "experts" and they decide what is important to know and to learn. These experts now gathered nationally... will "define" the correct body of knowledge. Notice the questions that are missing: for instance..... Do All people need algebra 1 and 2? How about Geometry and Chemistry.... what is missing for the discussion...who's voices are not present?

Step 3: Curriculum: The core content from step 2 is broken down into smaller "chuncks". These chuncks tell you what "every" student should "Know" from kindergarten and quickly moving toward college ( see the P-21 initiatives ) The go grade by grade in a nice, neat, straight line. Peek inside these grade level benchmarks and you will see progamatic maps...day by day...week by week.

Step 4: Purchase progams and materials...aligned with the curriculum maps.

Step 5: Train ( key word train ) teachers to "deliver" the program.

Step 6: LAST BUT NOT LEAST---- TESTS---lots of them to check the quality of the line.

Funny....America has really lost its industrial base...its factories are all but history...yet its thinking is still deeply embedded in industrial age thinking.... Taylorism is alive and well!

Any one out there seeing this go in a different direction????

Wonder if most people see the above assembly line as "school"?

be well... mike

I am not opposed to tests, not even to multiple-choice standardized tests. But we place too much emphasis on them, with the result that we cultivate multiple-choice thinking.

Multiple-choice thinking is strategic but not thorough. You can usually eliminate one or two of the options right off the bat. Of the remaining two options, you may be able to choose the correct one with only a hunch or good sense. Choosing the correct answer may not involve much understanding of the subject. And it turns into a bad habit.

It turns into the habit of not thinking problems through--just thinking about them enough to select an answer. It turns in the habit of trying to get around a problem rather than work on it. It is to the test-taker's advantage not to think too much about a problem--because you can arrive at the right answer with less thought, and if you think too much, you might have doubts.

All of this is fairly obvious--but when we emphasize multiple-choice questions more and more, when they become part of the daily life of students, we encourage a certain trickery on the one hand and apathy on the other. The goal is to figure out quickly which answer is right and move on.

When I was in high school, it was considered somewhat nerdy to talk about class after class. Class was over, and you were done with it. (I wasn't done, and my reputation at times reflected this.) But now this switching off of thought has entered into the subjects themselves. Got the answer? Good. Next.

Diana Senechal

Mike says, "America has really lost its industrial base...its factories are all but history...yet its thinking is still deeply embedded in industrial age thinking.... Any one out there seeing this go in a different direction????"

Absolutely! Remember, the world and we still produce things...more things than ever! So, production has not gone away.

What he have done is radically improved the efficiency of the processes. That means many things! (Bear with me!)

OK, to make a ton of iron once took an entire forest. We'd strip a mountainside (say above the Shenandoah valley, or above the Ohio), pile the timbers, burn them into charcoal, smelt the pig iron, and hope for the best. (The Johnstown flood was one result!)

By the time we had gas and electric fired furnaces, huge numbers of colliers were not needed. But we used them elsewhere. Later we repeated that efficiency ramp with the steelworkers themselves, who effectively said, "we can get better jobs in other industries. To keep us, you'd have to pay more than the market can bear."

In the past 20 years, those young men--and now women!-- who might have been on the floor of the factory moved to the brainy jobs. And the the ranks of knowledge workers not only swelled, but soon...were surpassed by a new class--meta knowledge workers!!

But that's not what happened in teaching and education.

In education in the 80's, 90's, and 00's, no budget shifted from the ranks of the old workers to the ranks of the new workers. The budget for the old workers doubled. The ranks of the meta-knowledge workers grew a small bit, but not much. So, all we got was bubble tests.

The whole point of the "reform" process is to unclog that Luddite mentality, and to make education much more effective. We can do it!

Ed,
I'm not so sure about your analogy. Yes, certain industries had large increases productivity due to technological innovation. But other industries didn't. I'm not sure how much more productive the average truck driver is today than he was 25 years ago, (or plumbers, or heavy equipments operators, or welders.) So, yes, we can manufacture LCD TV's vastly more efficiently today then we could 10 years ago, but I'm not sure that I can get the oil changed in my car much faster than I could 20 years ago. Why? Because technological innovation helps some fields and industries more than others.

Also, simple economics tells us that, ceteris paribus, if there are only two jobs in the world--oil changers and TV makers--then if a technological innovation makes TV makers produce more TVs then wages for TV makers should rise. (Good old marginal revenue product of labor). But even if technology has no effect on oil changing, wages for oil changers will rise anyway. (Supply of oil changers falls, as they take jobs as TV makers, but demand for oil changers stays the same.) You seem to be believe that education is the kind of enterprise where technological innovation ought to have produced large productivity gains. You might be right, but what is the evidence? It is not enough to simply point out that some fields have experienced rapid growth in productivity from technological innovation, and then complain that education (or truck driving, welding, and oil changing) has not.
I'm certainly not arguing that what you are suggesting is impossible, but it is not obvious either. It is not at all clear that smelting iron ore is the best analogy for educating students.

Also I think you play a bit fast and loose with the consequences of creative destruction (not to mention the sustainability of gas and (coal-fired) electric furnaces. Not all those who used to work on factory floors moved on to "brainy" jobs. Some are mopping floors and stocking Wal-Mart shelves at a fraction of their previous wages. Take a look at "And the Wolf Finally Came: The Decline and Fall of the American Steel Industry" by John Hoerr for a different version of what happened to steelworkers.

As for Diane's vision of the NCLB twilight zone, how about this for an analogy: Ed, what would you think if the government passed a law that required all engineers to be evaluated on the basis of how much iron ore they helped industry use each year. The more the better. If our engineers are really doing there jobs, it seems like we could really use a lot of iron. What, you say--you are a software engineer?--stop making excuses for your poor performance. Wouldn't it better to use less iron--Hmph! Mere rationalizations for a job poorly done. An overly simplistic measure of performance you say--Well, it must be better than nothing. The teachers I know aren't afraid of being evaluated on the basis of data, they're afraid to be evaluated on the basis of badly used and interpreted data, which is pretty much the only way they see their managers use data.

Ed, forgive my facetiousness, but it must be my clogged up Luddite mentality. In the end, I hope you are right . . . I hope technology and innovation can profoundly improve education--we certainly should try. But, alas, I'm a bit less optimistic.

RE: the validity of the CAHSEE

I am very aware of what the CAHSEE is. I just spent the entire day grading Freshman SCAP essays (which are a diagnostic for the CAHSEE). We also got our API scores today. What they reveal is very interesting. 45% of tested students are proficient via the CAHSEE but those same students were only 25 % proficient according to the CST's or State Tests. That is a clearn indicuation to me that the students try harder on a test that that COUNTS for them. Also the CAHSEE is multiple choice but it tests a whole array of skills, punctuation, literary analaysis, writing strategies etc. PLUS a writing sample. A few years ago we had only a writing and math proficiency. What is required now is harder because students must should knoweldge of literary terms and grammar plus write a competentent essay with (I believe ) harder prompts.

The CAHSEE is not perfect but it is closer to being an authentic tests because of the basic categories tested (more than required before ) AND the all important written portion. It is no easy matter to get a 4 or 3 on the writting portion so the CAHSEE essay helps movtivate students to do their best.

RE: the validity of the CAHSEE

I am very aware of what the CAHSEE is. I just spent the entire day grading Freshman SCAP essays (which are a diagnostic for the CAHSEE). We also got our API scores today. What they reveal is very interesting. 45% of tested students are proficient via the CAHSEE but those same students were only 25 % proficient according to the CST's or State Tests. That is a clearn indicuation to me that the students try harder on a test that that COUNTS for them. Also the CAHSEE is multiple choice but it tests a whole array of skills, punctuation, literary analaysis, writing strategies etc. PLUS a writing sample. A few years ago we had only a writing and math proficiency. What is required now is harder because students must should knoweldge of literary terms and grammar plus write a competentent essay with (I believe ) harder prompts.

The CAHSEE is not perfect but it is closer to being an authentic tests because of the basic categories tested (more than required before ) AND the all important written portion. It is no easy matter to get a 4 or 3 on the writting portion so the CAHSEE essay helps movtivate students to do their best.

Richard,

Originally you seemed to imply that multiple choice is the issue. Now that you argue validity based on content rather than on format, let's examine that.

CAHSEE has a collection of items based on 9th and 10th grade standards in Language, and items based mostly on 7th (with a small fraction of 6th and 8th) grade level standards in mathematics. Consequently, it is unsurprising that the success rate for high schoolers in grades 10 and above is higher on CAHSEE than on the grade level CSTs. However, based on these tests blueprints, it is also quite obvious that CSTs have a much higher validity. In other words, CAHSEE is constructed to be below level and hence create artificially inflated passing rates, exactly what Diane is complaining about. The CSTs are not.

Keith, Thank you for a pretty thoughtful response!! Let's put on our thinking caps and tease through this.

First, Huzzah!! I have before used here and elsewhere When the Wolf as precisely the type of cautionary tale teacher labor leaders should be reading! Moreover, I live and was raised and educated in the midst of that very story.

What happened to Big Steel was this: Labor and Management spent all their time fighting each other, not working with each other. The price was that the industry fell behind in not just technology, but in methods of organization and communication, in simple inter- and intra- departmental procedures, in the way they communicated and dealt with customers, the way they collaborated with suppliers, their relationship with investors and financiers, the whole gamut of life in a big organization.

More than that, Steel simply refused to be humble about its role in a changing economy. As late as 1998, they were still seeking national political solutions to their problems, rather than taking a hard look at the business they were in. You know the result, very little steel is produced here in the Pittsburgh/Cleveland area. They do a little better at NUCOR, a company which early on took a different path.


OK, Lets skip to a more intellectual industry for an additional example/analogy: The SCO group is only one company, but it stands as an example of how not to withstand competition. SCO was a software company, especially making its coin off Unix. But in the '90's, Linux began to take off as the dominant industrial strength operating system. SCO's business was substantially threatened.

Think now of all those State ed orgs filing suits, running press operations, pressuring lawmakers, and even spending their dime on partisan political elections. To what end? To stop the tiniest of competition brewing out there, mostly in the form of charter schools and vouchers to Catholic and other private schools. This latter approach is how SCO sought to advance its business. It did not drive the company on to some great new and successful innovation. It instead hired lawyers.

SCO, too, is bankrupt.


Can technology and innovation improve education: You took on mostly technology, and I do see much to be improved via tech. Classroom 2.0 I hold up as an example that teachers believe so too! Yet tech is not the only means of innovation.

Organizational innovations, too, have been rampant in industry and nonprofit alike! Schools,...not so much! Too frequently, they still look pretty much the same as 1950!

What would organizational change look like? Its not clear to me how much intermural org can change; yet there is stuff we can do with bringing in more teaching assistants and with dividing up responsibilities for the customer (student) interface jobs. Why should a great teacher be doing cafeteria or homeroom duty? Should they really all be coaching for extra pay? Should the math department perhaps be contracted out? Or intensive reading instruction for a school producing 60% dropouts? Is Project Based Learning a better curriculum path for bottom-end urban schools? I don't know; I do know the freedom to innovate in these matters was much less in 1990 than now. For all its shortcomings, NCLB took power from the everything-the-same group and in a few places let more experiments happen. I believe we could get more.


I'd encourage you to read some of the Entrepreneurs in Education papers from AEI to explore some more dimensions, and how regulations and protesting Luddites thwart experimentation.


Ed,
Only time now for quick reply . . .
I think one of reasons that some educational innovators, (like Deb, lest we forget) have become suspicious of NCLB reforms, RTT, and charters, is that they perceive that they squelch just the kind of organizational innovation that you think they might encourage. Sure, charters lead us away from encrusted school bureaucracies (Chubb and Moe's line I think), but in the current environment it seems only certain kinds of charters are likely to be encouraged (the work more, do more, raise test score kind). Imagine trying to start a charter school in the current environment (I have) that doesn't have raising standardized test scores as primary purpose. How much easier to start a new variation of something like KIPP, rather than something like the the MET? Or even something really radical like Sudbury?
I think we agree that schools should have more experimentation and risk taking, (so does Deb, I think) but disagree about which policies are likely to encourage such risk-taking. NCLB has made many talented people in education absolutely test-score paranoid, which doesn't exactly encourage experimentation. At my (public)school I have struggled (unsuccessfully)for years just to get my colleagues to consider that the word "data" could refer to something other than accountability standardized test score data.
As for the Hoerr book, he certainly has criticisms enough for both management and labor, but one of the lessons I took from it was management's inability to get past viewing labor as "labor." So as usual, while you think of the book as cautionary tale for labor (and it is) I see it also as something that managers (think Duncan, Rhee, Klien) should be reading to warn them off trying so hard to break teacher's unions that they contribute to crashing the whole system.
Finally, I think I percieve in your writing more than little bit of a libertarian streak. Please Ed explain how you think that a law that centralizes more power over education into the hands of the federal government than at any time in our history is likely to encourage innovation, experimentation, and risk-taking in the long run. Surely you must see that Feds are NEVER giving this power up, and that in the long run the only kinds of schools that will be permitted are the kinds that a very small number of government elites say are OK. I fear some us may live long enough to look back fondly at the good old days when the worst encrusted bureaucracies were local school boards and teacher's unions rather then the Octopus that the Department of Education might become.

Keith, I skipped a couple of your questions, and they deserve response as the answers really illuminate the issue!

Truck drivers: A truck driver may appear all that more efficient than 25 years ago. But lets look deeper!

For example, say I need to have a load of fence brought to a customer here in Ohio. The best deal on fence is in Texas. That's a lot of time and diesel to get down there and back!

Today, however, I can go to DirectFreight.com. There, 513 outbound loads are wighting for a truck to leave Ohio in the next week. If I check Pittsburgh and West Virginia, I'll find more. Thus, I load up someone else's outbound shipment here, drop it off in Texarkana, and bring my fence back along with payment for the dropped load. Technology has just turned an expensive or unreasonable trip into a money-making opportunity for me the driver, as well as the customer.

It's also helped the planet save oil and CO2.


But of course the real impact of innovation is not with one driver. Its with a large group of drivers! Take UPS! UPS has so changed its business with the help of technology that it is now as much a Supply Chain Management organization as a shipping firm. Need your Chinese computer repaired? UPS won't ship it to China, they'll repair it at their site! Now that's innovation!

UPS didn't just jump into this all at once, though. Nor could anyone have predicted it 25 years ago. Yet someone, somewhere at UPS embraced the PROCESSES of innovation and ingenuity. To be specific, I'll bet they were quite forceful about Total Quality Management.

In fact, in 1984, UPS had a real problem. Their truckers relied on aircraft, and those aircraft had just been deregulated, resulting in a loss of many of the flights the company depended on. They turned this "problem" into a platform for improvement: UPS airlines was created.

The revolutionary computer system used to plan the aircraft schedules and loads gave them more than fuel efficiency: it gave them the professional memes for the broad businesses to follow!

By 1993, another technology revolutionized the company: Every truck driver got a handheld scanning and signature device. Suddenly UPS computers , managers, customers, and even consumers at home could know exactly where each package was.

This didn't just change UPS--it changed all business. Suddenly, businesses could see their production past and future much more clearly. They could begin to plan their own arrivals more intelligently. Many didn't even need to maintain supply-chain expertise in house; that means their own employees can spend more time, talent, and resources focusing on the customer--what the customer needs now, and how he might be better served in the future.

In 1984, a lost truck driver would have had to search for a pay phone. Today, one driver can save ah hour here and there using a cell phone. But a fleet of tech-enabled drivers...can change the world!


Sigh. Ed, you say, that's business. That stuff doesn't apply. We work for the people, the taxpayers.

Well, just for one starter, so do the truckers of the US Army. And in 2003, those truckers and their brethren tank and humvee drivers performed one of the most miraculous operations in the history of war and transportation. No one, anywhere, had ever dreamed an Army of such size could move so quickly and cover so many miles. This wasn't merely a remarkable military accomplishment; it was an astouding tour de force of the logisticians who planned it, the computer, software, communications, waveform, and logistical logistical engineers who designed the enabling tech, and, yes, the agile young truck drivers who ran it all!

Yep, I believe educators have been missing out. More importantly, the really good opportunities are just arriving!


(While I've been writing, so have you. More good questions. But its past bedtime!!)

If being held accountable for student achievement is truly the way to go then shouldn't society as a whole be held accountable for allowing millions of children to grow up in poverty. Poverty which has been found to be the single most prevalent "root cause" related to "low performing" schools. Should we also punish doctors and nurses who work in public hospitals for having too many patients that come in the door with histories of abuse, malnutrition, and/or ignorance regarding the functioning of their bodies?

On a somewhat different topic, if merit pay and charter schools have seemingly scant or no evidence of success why do such reputable papers as the Boston Globe and the New York Times continue to press for their inclusion in education reform?

Read Monday morning's NYT editorial, "Mr. Duncan and That $4.3 Billion" piece.

While it's probably safe to presume the folks on the editorial boards of these two papers all send their kids to only the best private schools, they sure seem convinced for some reason that Obama and Duncan are on the right track with both of these Race To The Top proposals.

Come on Diane. Let's hear it.

Paul,
You can't be serious in suggesting that we should look to the editorial boards of the NY Times, the Boston Globe (owned by the Times), or any other newspaper to know what to do in education policy.
There is no research and exactly zero experience to demonstrate that paying teachers more improves the education of their students. It may lead to higher test scores, and there is a growing body of evidence that the test scores (as I have written in previous columns) are rigged, sometimes by teachers eager to make more money, sometimes by the district, sometimes by the state education department, all of whom are "incentivized" to get those scores up!
Tennessee has been using Bill Sanders value-added methodology for nearly 20 years. Would you like Massachusetts to look like Tennessee? For the answer, go look at NAEP scores, which have been tracking state performance since 1992.
Why do the editorial boards support the Klein, Rhee, Duncan, Obama, Sharpton, Gingrich agenda? If you were a historian, taking the long view, you probably would conclude that the editorialists (who are not education experts) are simply reflecting the conventional wisdom.
It's our job to know more than the conventional wisdom.
Diane

The Times and the Globe are historically not irresponsible. It's not like the Boston Herald or the New York Post taking these positions to satisfy their readerships. That's what makes me wonder.

So that's a reputable President of the United States, his appointed Secretary of Education and two of the most respected newspapers in the country all in favor of RttT. What is it exactly they're attempting to tell the educational establishment - the same educational establishment responsible for the deplorable condition of our public schools prior to education reform?

That's at least part of the reason I like its direction.

And no, I would not wish Tennessee's schools on Massachusetts kids, Bill Sanders notwithstanding. While I respect his efforts, the jury appears to still be out on value-added measures.

Paul,
I wrote a book to answer your questions, and it is too long to summarize here! It will be published in March.
What I can tell you is that Obama and Duncan are not education experts, neither are the editorial writers at the Times and the Globe. You have more teaching experience than all of them put together.
Forget Arkansas. Would you be happy to see Boston perform on par with Chicago schools after seven years of Arne Duncan? Check the NAEP scores. I don't think so.
Diane

Guess I'll just have to wait to read your book. I'm sure it'll be interesting. Too bad it won't be out till March. By then RttT will probably be up and running. Obama could have used your expertise before making the leap.

I too am looking forward to reading Diane's new book. But Secretary Duncan and his band of Reformers could have read Diane's previous books. They could have looked at the experience in TN and other states. They should have known that "The Race to the Top" is an emptier slogan than "No Child Left Behind." And so on. And so on.

There is no research and no experience to support any of the four "pillars" of the Education Stimulus or the RttT.

While waiting for Diane's book to appear, try Chris Hedge' new book "Empire of Illusion: The End of Literacy and the Triumph of the Spectacle." It's not a happy story and I don't want to give away his pitch. But one set of "bad guys" he holds responsible is the major research universities. His argument is difficult to refute.

Diane writes to Paul: "Obama and Duncan are not education experts, neither are the editorial writers at the Times and the Globe. You have more teaching experience than all of them put together."

Thank goodness the President is not weighing in on any other issues where he has not had years of in service experience!!

Imagine if, say, he had to decide between the on-the-ground counter-insurgency strategy recommended by his military commanders or the hands-off, from the air and sea, so-called counter-terrorism strategy recommended by his Vice President.

...A situation where he must decide in fields where he has far far less experience than Urban poverty, community building, and education!!


Seriously, time spent teaching shows you have spent more time in the school building than the rest of us (maybe good, maybe bad), and less time out in the world you're preparing students for.

We expect out military leaders to defer to civilian control, and not to participate in partisan politics. There are reasons we elect leaders.

Although the buck stops with the President, the accountability is really with the Secretary of Education and his advisers. Secretary Duncan has proved by his actions to date that he is not an education expert, and he has appointed staff who aren't expert either.

The President may be capable enough to identify the flaws in RttT early enough to do a "policy review," as he's doing with the Afghanistan War. Or the Secretary may make spectacular gains in his "Listening and Learning" phase. But right now the Ed Stim and RttT have all the characteristics of implosion, rather than reform.

I would like to piggy back on Viera's comment: Has anyone in the government ever considered that the entire problem does not stem with the schools and the teachers. What about the students and families? What about home life and society? What about the media, drugs, gangs, apathy, get rich quick mentalities? And yet, schools and teachers are to defy all of these things and raise test scores.

I am a recently retired teacher, who toiled in public and private classrooms for nearly 20 years. I oppose standardized testing because it just doesn't show us how it 'fits.' All tests show what the students know at the time they take the test. This is true for the famous 'end of unit test,' as well as the infamous 'Finals,' over which student stress so much.
In a sumer school class a number of years ago, the district was testing a test, using the summer school to assess if the test was even the right instrument. It was a test designed by a group of teachers in the area. It was designed to ascertain if the students were learning the 'right' information. (This was in the years before the standards movements which began in the late 1990's.) Problem was the test contained so many factual inaccuracies, that they district decided it could not be used, as the summer final. Using the test and some of its questions (a,b,c,d) as a guide, I rewrote much of what was written. I coul;d se where the test was going, not quite why it was going where it was going. I probably ended up using about 30 percent of that test as a guide, supplementing my own body of knowledge for the rest of the Final.
The designed instrument never made it off of some shelf in the district office. It gathered lots of dust. (I am not in that district any longer, so I don't know of its eventual fate.)
By the time students reacherd me in social studies, they should have been able to make an argument using their critical thinking skills. But I was aghast that many students did not have crkitical thining skills, which led me in later years to do formal lessons on developing those skills and using them eveytime I could. Whether students knew it or not, I was helping them to develop their CT skills by questioning something as simple as asking them why they wrote such and such as they did. (Students automatically think that if there are comments in red, 'they did bad,' though I tried over the years to explain to them that that was not the case.)
I was condemned in a formal evaluation once said, "...they always seem to be answering worksheets when I come in..." What he failed to realize and probably was not capable of asking the right question: From where do these worksheets come? The truth, which he would have found out that had he asked, the students would write questions and make statements whic other students would respond, sometimes in the form of an ansnwer, other times in the form of a statment. I was much more interested in how they came up with their answer or statement, rather than whether it was right or not.

Comments are now closed for this post.

The opinions expressed in Bridging Differences are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.

Follow This Blog

Advertisement

Most Viewed on Education Week

Categories

Archives

Recent Comments

  • hertfordshire security installers: Greetings. Great content. Have you got an rss I could read more
  • http://blog.outsystems.com/aboutagility/2009/04/challenges-of-scoping-and-sizing-agile-projects.html: I would like to thank you for the efforts you've read more
  • http://acousticwood.net/mash/2008/03/yeah_off_to_the_uk.html: Between me and my husband we've owned more MP3 players read more
  • buy cheap metin2 yang: When you play the game, you really think you equipment read more
  • Nev: Anne Clark - If a Dr. instructs a patient that read more