What's the Appropriate Role of Profit-Making Companies in Student/School Assessment?
Today Deborah Meier and Joe Nathan discuss the role of for-profit companies in assessing students. Deborah begins, and Joe responds.
"Pearson should pay for state-wide testing process," you write. Of course. How could anyone have signed a contract that didn't require that the product works?
Of course, even worse, is the fact that we've bought--for years now--a fraudulent product. Pearson is "just" the latest version. And like Pearson, past versions also come with textbooks and test prep to match. At one time they also came with technical explanations, covering measurement error et al. But they've dropped that years and years ago.
The 8th grade version of the recent Pearson ELA test also gave a "slight" advantage to 8th graders who had been exposed to the Pearson 8th grade textbook. It included, word for word, one of the reading passages used in the real test. A good selling point.
But a more interesting point (after all profit-making firms have long abided by a simple principle: "buyer beware") is why so many "well-educated" experts have allowed an instrument with such a fundamentally biased design to be defining what works. They've made it very hard for laymen to recognize that they are being bamboozled.
That includes me. My wake-up call came when my bookworm 8-year-old son "failed" a 3rd grade reading test (eons ago). It's a great story which I love telling. But would I have ever questioned the whole scheme otherwise? Maybe not. So, thanks Joe for raising these questions again--with all the added bells and whistles that test-making companies have added to the original flawed instrument.
Can any standardized instrument ever be unbiased?? I'm not sure how--each attempt to do so has different trade-offs and different biases. The idea that bubble-tests, tests scored by machines, are less biased forgets that human beings designed the language of the test and the options--including which one was "right." Human beings have that fundamentally treasured but essential element--they make judgments, they construct their own version of reality, the respond to the phenomenon differently, they have different pasts, futures, and therefore self-interests, and on and on. We're stuck with that kind of human being and it makes any rank-order an act of a particular set of biases. I wouldn't have it otherwise.
The system we've bought into is essentially tied to the rank order of the society we live in. It cannot help itself. The choice of answer "c" is right because the right kids answered it right. If the "wrong" kids do, that item is useless and is tossed out. By the time our students take these tests the only thing undecided--in the latest versions--is where to place the benchmarks. That's a political decision dependent on what kind of impact we want the results to have.
But how else can we do it? We can't avoid "bias"--try as we might, some creeps in because we are humans. But the less mysterious, the more transparent the exam system, and the more we acknowledge and document the process, the more likely we can take note of bias and even decide whether it's a bias we want to embed in the test!
More than 30 N.Y. state high schools have for 30 years been pioneering such an exam system. It has been applauded by a panel set up by an opponent--superintendent of N.Y. state schools Mills. They didn't conclude that it was a perfect system or without flaws, they just concluded that it seemed to identify students who went on to do good work and was no more or less reliable or credible than the N.Y. State Regents exams that they were intended to replace.
Someone suggests a new "reliable" and "credible" test for colleges: Check back five years after they graduate and see whose students make more money. It's bias-free in one sense, and fundamentally racist and classist. It's a test for maintaining the status quo--only more ferociously.
Democracy is not another word for the free marketplace. The marketplace works for its purposes--deciding who will make a profit and who will not. Democracy is after an entirely different goal. Whether it's compatible, or even essentially partnered with democracy is arguable. But we need to take the profit motive out of judging our fellow citizens--starting at birth.
Deb, several things. First, we agree about the value of the performance assessments that the N.Y. high schools have developed. I think schools should have the option to use well-developed, applied assessments such as they have created.
Second, we agree that Pearson should pay. Perhaps Columbia Teachers College President Susan Fuhrman, a national education leader, can help. She was on the Pearson board for 9 years but retired in 2013.
Finally, I don't think the sole purpose of the marketplace is "deciding who will make a profit and who will not." We agree that democracy and the marketplace are not the same. But the marketplace is not just about profit. I think it's also about allowing people to decide which products and services are worthy, and which are not. The Edsel was not a good product. Neither was the Corvair. People all over the world have decided that the Apple iPhone is a good product. (I agree.)
I'm not saying that the free market is perfect, or that it works well for people who have low incomes. Unquestionably people with few financial resources often are not well-served.
Some years ago students at the St Paul Open School, where I worked, found that the quality of produce, for example, offered by grocery stores in low-income communities sometimes is not as good, and sometimes is more expensive, than produce in grocery stores in more affluent neighborhoods.
But I think a market economy is better than a centrally controlled economy (such as existed in the Soviet Union for some years). I've talked with many immigrants from Russia who have settled here and much prefer both our democracy and our economy.
Joe, I agree that there's a place for markets but I also believe that for good or bad the bottom line in a market economy is profit. We need to have alternatives where such goals are inappropriate. Test scores are the reformers' answer! But accountability is much more complex and that's the far-from-perfect purpose of democracy.