skoolboy says: Some of My Best Friends are Psychometricians, But...
She's absolutely right, in my view. Preparing children and youth to be citizens in a democracy is a critical purpose of schooling. eduwonkette has written that there's a lot to schooling that can't possibly be measured by standardized tests – I think my favorite line is from the title of a post in January riffing on New York City's "Thank a Teacher" nomination process, "They Never Say, 'Thanks for Improving My Test Scores!'" – but it's easy to fall into the trap of treating the current testing regime as the natural order of things.
We need to be mindful that public schooling is now what institutional analysts such as Pat Burch call an organizational field, with lots of actors influencing our definitions of schooling and its outcomes, including textbook publishers, testing firms, test-prep firms, and a variety of other commercial entities. Lots of commercial enterprises and non-profits owe their livelihood to public education, and are engaged in an ongoing project to shape our definitions of "real school."
Testing is big business in the U.S. Non-profits such as the Educational Testing Service and ACT have annual gross revenues approaching $900 million and $400 million, respectively. ETS's K-12 testing operation had gross revenues of $172 million in their 2006 IRS filings. On the for-profit side, Pearson Education had gross revenues worldwide of $4.6 billion in 2006, with $600 million in adjusted operating profit. Their annual report crowed of a "healthy outlook in school testing underpinned by 2005 contract wins with a lifetime value of $700m (including Texas, Virginia, Michigan and Minnesota)." McGraw-Hill Education had revenues of $2.7 billion in 2007, with operating profit of $400 million.
With this much money, and more, at stake, you can bet that there are ongoing projects to define tests and testing as the appropriate way of defining what counts as good education. They tap into a logic that defines the modern world as increasingly rational, and society as a collection of individuals with increasingly differentiated roles, identities and personal preferences.
I'm not sure what the right approach is to counter all of this. At one point, I thought that giving politicians, educators and parents vivid representations of good teaching and good learning –e.g., videos, or portfolios--would be sufficient to persuade them that test scores don't come close to capturing what we aspire to in public education. But I haven't seen that strategy be successful. Preaching to the choir isn't going to do it – we need to find a way to put people in the pews. Readers, do you have ideas?
Comments
This is an interesting question, and helps get past the debate over the pro's and con's of testing. I'm willing to believe that tests measure something but not that they measure everything that's valuable in schooling.
I agree with Deborah Meier that "habits of mind" are key. It's fairly easy to imagine being able identify school situations that make developing those habits possible -- and endless test prep isn't one of them. I think the task of identifying which students have developed them consistently is harder (or at least more labor intensive).
The standardized testing mindset is that every student should be tested by an outside monitor every year. Its a mindset that values a large quantity of information over a high quality of information, and confuses the questions of whether the purpose is to evaluate school, or to evaluate students.
As a start we might look at ways of evaluating schools that allow us to look more deeply than multiple choice test scores. Even if testing is involved, perhaps instead of giving every grade multiple choice tests, we gave a few grades more probing assessments?
Posted by: Rachel | April 28, 2008 11:38 AM
The tests should be made fully public soon after their administration. That way we can analyze them and propose improvements.
The current secrecy and tight security around each test impede public scrutiny. We need access to the tests in order to consider at least three questions:
1. What is being tested, and why? How does the material being tested fit into the overall subject matter?
2. Do the questions test this material effectively? How so, or how not?
3. What could be done to (a) improve the curriculum and (b) ensure that the test reflects it?
A website with past tests and comments would be a start.
Posted by: Diana | April 28, 2008 12:20 PM
Jettison the tests...
http://www.youtube.com/watch?v=45HIW3jgEPg&eurl=http://www.educatorroundtable.net/frontPage.do
Posted by: philipkovacs | April 28, 2008 12:47 PM
I'm glad Rachel brought up our collective confusion about the purpose of an accountability system - are we evaluating students or schools? If we want to evaluate schools, why not administer more complex assessments to a random sample of students in addition to giving these basic skills tests? (By the same token, do we need to test everyone on basic skills tests every year?) That would keep costs in check, and allow us to evaluate schools on a broader set of skills.
And I agree with Diana that sample tests should be public. It is an important first step in educating parents and citizens about what these tests can and can't do.
Skoolboy, well done on the testing revenues!
Posted by: eduwonkette | April 28, 2008 2:34 PM
You said, "If we want to evaluate schools, why not administer more complex assessments to a random sample of students..."
I thought, "Once again, people don't think about rural schools." Most education writers think only of urban and suburban districts, and national policy is set with those schools in mind.
Very small schools, located in areas with small populations (and often large distances), have to live with those decisions. Regarding your current suggestion, they don't have enough students per grade level to determine school effectiveness now....how could they possibly sample?
If larger schools could save money by sampling, and small schools could not, then small schools would pay a disproportionately large proportion of their meager budgets on tests and testing. This would exacerbate a situation that's already bad. When was the last time you walked into a school of elementary children where the lights weren't on and the kids wore their coats to stay warmer? For us, the answer is, "last winter."
So...this is way off topic! I appreciate the introduction of financial interests -- big financial interests! -- into the discussion. It's an important issue that doesn't get enough coverage.
Posted by: Kate | April 28, 2008 4:39 PM
If larger schools could save money by sampling, and small schools could not, then small schools would pay a disproportionately large proportion of their meager budgets on tests and testing. This would exacerbate a situation that's already bad.
You could give schools enough money to cover the cost of testing the number of students they actually were asked to test. Radical idea, though, actually funding mandates...
Posted by: Rachel | April 28, 2008 5:23 PM