« In Defense of Politics—Sort of | Main | The Fallout from Testing »

The Grand Illusion of Proficiency

| 11 Comments

Dear Deb,

As usual, you raise lots of interesting questions and you sharpen our clear differences. Yes, I do think we should have national testing. This idea that fifty states should each have their own standards and their own tests is nutty. We are not getting higher standards; we may even be getting lower ones.

How did we get to this point? President Clinton's Goals 2000 pushed the states to create their own standards and tests (Clinton, to his credit, actually preferred national tests, but he couldn't persuade the Republican Congress to go along with his proposal for such tests). Then along came NCLB, and President Bush wanted a bigger emphasis on standards and testing, but knew that his own party would never accept national testing. So he built on the idea that each state should set its own standards, develop its own tests, grade its own progress towards the goal of having every student "proficient" in reading and mathematics by the year 2013-2014. Since the bill passed Congress in the fall of 2001, I assume that the goal of 2013-14 was based on the idea that this was the amount of time (12 years, starting in 2002) necessary to raise the achievement of children who were then in kindergarten.

As we both know, and as everyone knows who thinks about the matter, we will not reach the goal of having universal proficiency by 2014, unless we define "proficiency" to mean low-level, basic literacy.

Writing this goal, no matter how impossible and absurd, into federal law put pressure on the states to come up with plans to demonstrate that they intended to do it. How crazy was that? So every state has a year-by-year plan in which they will raise "proficiency" and "achievement" towards that elusive goal. This in turn guarantees that the states will dumb down their tests and focus relentlessly on test prep so that they can at least try to fulfill their promises to the feds.

We all know the results of this grand illusion.

First, we know from studies such as the one by the Center on Education Policy, that the curriculum has been narrowed in a majority of schools; many children are not having any chance to study the arts, history, or anything else that is not going to have an immediate impact on their reading and math scores. Though I would argue that children will get much higher reading scores if they spend more time learning history and engaging in the arts, school officials aren't willing to take the risk. The scores must rise! And they must rise by constantly drilling the kids in how to take tests and in practicing the kinds of test items that they are likely to encounter on the state tests. As a result of this idiocy, we may be losing a generation of young people who associate schooling with the worst kind of drudgery and test grind.

Second, we know from studies such as "The Proficiency Illusion" by the Thomas B. Fordham Institute (where I am a trustee) that many states are using a very low-level definition of proficiency in order to inflate their scores and claim progress. There are states that tell the public that a large majority of the students are "proficient," and getting ever more proficient. Yet the proportion of students in these states who are proficient or even basic on NAEP remains nowhere near what the states are claiming.

We have a fundamental problem of honesty here. The public is being misled in most states about academic gains. NAEP is the only measure that is maintaining a consistent standard across the fifty states.

Yes, I believe we should have a scheme of national testing. Yes, you are right, I do not think that there should be "stakes" associated with national testing. I think that we should have national tests so that we have better information. Having watched the misuse of test results in NYC these past few years—where tests are being used to reward, punish, and grade students, teachers, principals, and schools, I would hate to see national tests corrupted in these ways. I see two healthy uses for national testing (I don't expect you to agree): First, the tests should provide information that is reliable and consistent. Second, the tests should provide information with a diagnostic value, so that educational shortcomings can be addressed in a timely manner.

The irony is that we already have national standards. They aren't exactly written out in a single document or series of documents. But if you want to know what they are, look at the handful of standardized tests given in states and districts; look at the textbooks used in the great majority of classrooms. These are not the standards that you or I would want; but there they are. I recall that about ten years ago, I gave a lecture in Wyoming on the subject of national standards and national tests. Several people in the audience, not surprisingly, objected, saying that they preferred their standards to be written in Wyoming. I pointed out that the textbooks most widely used in the state came from a New York City-based publisher and that the tests used by the state came from a testing company in New Hampshire. No one could point out what was especially local or Wyomingesque about their standards and tests.

I agree with your concern for democracy in our society. I don't think the problem is uniquely limited to what happens in schools. The changes in the mass media have made all of us feel like spectators and consumers of other people's decisions. This requires another discussion.

Diane

11 Comments

I believe that you have put a finger where it should be: right on the problem of honesty. The problem is that while the American energies have been going into LOOKING as if we are succeeding, the energies of other countries have been plied on ACTUALLY succeeding in improving education.

In the quick first glance at the PISA scores, it is clear that the best that can be said of the US performance is that it didn't get any worse. Meanwhile, other countries, even Mexico, Slovenia, Estonia, have been on the upswing. We are bottom heavy--having more kids at the bottom and fewer kids at the top than the international average and the nations that are in line ahead of us. We are significantly below average.

Meanwhile we diddle about with such hare-brained schemes as counting students who fall into more than one AYP category as only half (or less) of a student in each one. Michigan just got caught telling its secondary special education teachers that they could be "highly qualified" if they could pass an elementary ed exam. Come on people. These are our children we are talking about. No wonder they don't understand the first thing about scientific thinking. We are all about appearance thinking here.

Diane,

GGood piece today on national standards aand national tests. When you talked about Republicans not backing Clinton's Goals 2000 or Bush's NCLB you failed to mention why. From everything I've read on the topic the GOP didn't want any part of federalizing our schools becuase they thought they would then have to come up with the corresponding monies for the legislation. They, of course, attempted to rationalize their posture by claiming the original founders thought education should be a state and local responsibility. Baloney!

You're right about fifty states going in fifty directions being a ludicrous concept. It is. Unfortunately it was a concession/compromise Bush had to make to his party to get NCLB passed. When you look at how states like Mississippi, Alabama, Tennessee, West Virgina, Oklahoma, Texas, etc., mutilate their definition of proficiency, it essentialy eliminates all credibility from their efforts. Their state tests don't even come close to being reliable when compared to NAEP results, commonly referred to as the "nation's report card."

The development of national standards and tests need not be left to the federal DOE. Represntatives from the fifty state DOE's, with the NAEP as their guide, should be able to come to some degree of consensus as to what could be the rich body of knowledge they would want all US students to be exposed to for a world class education.

Diane,

Good piece today on national standards and national tests.

However, when you talked about Republicans not backing Clinton's Goals 2000 or Bush's NCLB you failed to mention why. From everything I've read on the topic the GOP didn't want any part of federalizing our schools because they thought they would then have to come up with the corresponding monies for the legislation.

They, of course, attempted to rationalize their posture by claiming the original founders thought education should be a state and local responsibility. Baloney! Based on Tom Friedman's book (The World Is Flat) and other contemporary literature, education in this country has become a national priority, some would argue a question of national security.

You're right about fifty states going in fifty directions being a ludicrous concept. It is. Unfortunately it was a concession/compromise Bush had to make to the then Republican controlled Congress to get NCLB passed. When you look at how states like Mississippi, Alabama, Tennessee, West Virginia, Oklahoma, Texas, etc., malign their definition of proficiency, it essentially eliminates all credibility from their ed reform efforts. Their state tests don't even come close to being reliable when compared to NAEP results, which we know as the "nation's report card."

The development of national standards and tests need not be left to the federal DOE. Representatives from the fifty state DOE's, with the NAEP standards and tests as their guide, should be able to come to consensus as to what should be the rich body of knowledge they would want all US students to be exposed to for a world class education.

This all works for me ONLY IF PUBLIC SCHHOLS STOP (pardon the expression, I have a tremendous degree of difficulty with it) TEACHING TO THESE BLOODY TESTS. Teachers are responsible for all children in their charge. High-average and high students deserve an appropriate education too and teachers cannot continue to ignore them. The duplicitous concept of teaching only one lesson to the whole class every day in an attempt to get low and average kids over the state NCLB hurdle must end. It’s clearly more work but it’s also the duty of professional educators to ensure this gets done. Otherwise the public’s perception of many public school teachers will remain in doubt.

Just one retired teacher's thoughts.

First, the tests should provide information that is reliable and consistent. Second, the tests should provide information with a diagnostic value, so that educational shortcomings can be addressed in a timely manner.

Building on Deb's "Consumer Reports" idea, perhaps we should think of national standards/tests as the equivalent of government safety testing for cars. Part of the reason CR works is that they put the various products they review through the same paces -- so even though there are not many mandated safety features (though there are some) -- there is still a way to compare the crash-worthiness of a Ford and a Toyota in a way that's fairly easy to interpret.

Yes, I agree 100%. Thanks for putting it in such a succinct manner:

"I see two healthy uses for national testing (I don't expect you to agree): First, the tests should provide information that is reliable and consistent. Second, the tests should provide information with a diagnostic value, so that educational shortcomings can be addressed in a timely manner."

In addition to establishing a national test to better account for student progress, full reform of our public schools should also include:

1. Raising the compensation levels for high-quality teachers, especially in the areas of science, math and Special Education;
2. Providing better information on school performance to parents;
3. Eliminating district boundaries between schools allowing parents to choose the right school for their children, regardless of where they live;
4. Allowing greater flexibility for schools to hire and fire teachers;
5. Making school funding more transparent by attaching education dollars to the student instead of the school district.

For more information on school reform, visit www.paths2choice.com.

Andreas Schleicher (OECD Education Directorate) made an interesting observation at the National Press Club's PISA 2006 briefing yesterday. After noting that the U.S. was at one end of the extreme among nations in terms of its reliance on multiple choice tests, he said that our students are actually pretty good at multiple choice tests. However, he also noted that our students "have trouble with open ended tasks."

Unfortunately, life is an "open ended task."

"Teaching to the test" does not worry me--if it is the right test. You want to teach to the PISA science, math, or especially the "problem solving" tests? Go ahead--knock yourself out!!

Not that PISA or NAEP are perfect tests! But it would be hard for kids to do well on PISA without knowing how to think scientifically, how to analyze data, use evidence to reason to a conclusion (or find a flaw in an erroneous conclusion), use data from multiple sources, solve multi-step problems, etc.

Am I afraid of teachers teaching to this kind of test? No! I am more afraid that most teachers are not.

NPR reported tonight that the National Intelligence Estimate was previously a document aimed at consensus, with dissents at the bottom. The process for this year's NIE was that the various perspectives were described up front.

I'd see an educational Consumer Report as following the upfront approach. In fact, that's something in the routine of academic writing that should be institutionalized in all educational policy reports.

Speaking of academics, Thoedore Rabb's Education Week Commentary is a must read. He describes how much better it would be to have a 1/2 dozen different approaches to US History in the same middle school, each bringing the passion and perspective of that teacher.

Now, that sort of idea sounds like heresy. But a few years ago, most educators would have agreed that history must be taught from multiple perspectives.

If my district is any indication, we have moved way beyond teaching "to" the test to teaching "for" the test. The very first sentence out of my principal's mouth after her welcome back from summer speech this fall was, "Today we are going to talk about how to raise our test scores". No mention about how are we going to educate our students in a way that inspires a lifelong love of learning. No interest in sparking the future writers, scientists, and inventors. Certainly no interest in turning out socially aware and involved citizens. Indeed, the term "Citizenship" no longer even appears on the report cards.

A few calls to fellow educators at other public schools in the area revealed the same focus. Only when I spoke with friends who taught at private schools did I find even a consideration given, let alone an emphasis on, providing a quality education.

These days, I have trouble referring to myself as an educator when what I'm really hired to do at my Reading First School is coach little test takers and bubblers. Just how much need is this society going to have for workers who can do little more than fill in the blanks?

If my district is any indication, we have moved way beyond teaching "to" the test to teaching "for" the test. The very first sentence out of my principal's mouth after her welcome back from summer speech this fall was, "Today we are going to talk about how to raise our test scores". No mention about how are we going to educate our students in a way that inspires a lifelong love of learning. No interest in sparking the future writers, scientists, and inventors. Certainly no interest in turning out socially aware and involved citizens. Indeed, the term "Citizenship" no longer even appears on the report cards.

A few calls to fellow educators at other public schools in the area revealed the same focus. Only when I spoke with friends who taught at private schools did I find even a consideration given, let alone an emphasis on, providing a quality education.

These days, I have trouble referring to myself as an educator when what I'm really hired to do at my Reading First School is coach little test takers and bubblers. Just how much need is this society going to have for workers who can do little more than fill in the blanks?

Dear Diane and Deborah,

Do you believe there is a material distinction between standardized tests and specifically multiple choice tests? Do you know of any analysis of the problems inherent in multiple choice testing?

I know that in many countries, testing is standardized, but it often takes the form of questions and answers, not a choice of A, B, C, or D.

The language of multiple choice tests is a world unto itself. Often the questions and the options are intentionally made dense and ambiguous.

Test prep in other countries may involve review of material. Here it involves drilling of multiple-choice questions. I realize that I am stating the obvious; but the ramifications are deadly. Schools put their energies into teaching a warped language that has little application or value outside of the tests themselves.

Even the rubric applied to the "constructed response" questions is highly problematic. Very little weight is given to the substance of the response; instead, the rubric awards points of "fulfilling the charge of the task," "using two or more details," and so forth.

Perhaps we should have national tests, but of a very different sort. Or even if we changed the tests at the state level, the nature of preparation would change for the better.

Comments are now closed for this post.

Advertisement

Most Viewed on Education Week

Categories

Archives

Recent Comments