« Name Fordham's Boy Band! | Main | What is Social Justice Teaching, Anyway? »

skoolboy says: Some of My Best Friends are Psychometricians, But...

Deborah Meier added a comment to the end of the value-added thread from last week. (Thanks for stopping by eduwonkette's blog, Deb!) Her point is too important to overlook. She writes that standardized tests of reading proficiency are only loosely correlated with good reading habits—i.e., that a student can score well on a test of reading proficiency without demonstrating the habits of mind that could enable him or her to engage in a critical discussion of a text. Meier also writes that we do not have tests that measure "the more significant intellectually sound habits of heart and mind fundamental to being a well-educated member of society. The capacity to confront a phenomenon of interest in ways that help one best understand it, and then to make use of the knowledge acquired, is surely more important than being able to guess the one out of four 'best answer.'"

She's absolutely right, in my view. Preparing children and youth to be citizens in a democracy is a critical purpose of schooling. eduwonkette has written that there's a lot to schooling that can't possibly be measured by standardized tests – I think my favorite line is from the title of a post in January riffing on New York City's "Thank a Teacher" nomination process, "They Never Say, 'Thanks for Improving My Test Scores!'" – but it's easy to fall into the trap of treating the current testing regime as the natural order of things.

We need to be mindful that public schooling is now what institutional analysts such as Pat Burch call an organizational field, with lots of actors influencing our definitions of schooling and its outcomes, including textbook publishers, testing firms, test-prep firms, and a variety of other commercial entities. Lots of commercial enterprises and non-profits owe their livelihood to public education, and are engaged in an ongoing project to shape our definitions of "real school."

Testing is big business in the U.S. Non-profits such as the Educational Testing Service and ACT have annual gross revenues approaching $900 million and $400 million, respectively. ETS's K-12 testing operation had gross revenues of $172 million in their 2006 IRS filings. On the for-profit side, Pearson Education had gross revenues worldwide of $4.6 billion in 2006, with $600 million in adjusted operating profit. Their annual report crowed of a "healthy outlook in school testing underpinned by 2005 contract wins with a lifetime value of $700m (including Texas, Virginia, Michigan and Minnesota)." McGraw-Hill Education had revenues of $2.7 billion in 2007, with operating profit of $400 million.

With this much money, and more, at stake, you can bet that there are ongoing projects to define tests and testing as the appropriate way of defining what counts as good education. They tap into a logic that defines the modern world as increasingly rational, and society as a collection of individuals with increasingly differentiated roles, identities and personal preferences.

I'm not sure what the right approach is to counter all of this. At one point, I thought that giving politicians, educators and parents vivid representations of good teaching and good learning –e.g., videos, or portfolios--would be sufficient to persuade them that test scores don't come close to capturing what we aspire to in public education. But I haven't seen that strategy be successful. Preaching to the choir isn't going to do it – we need to find a way to put people in the pews. Readers, do you have ideas?

This is an interesting question, and helps get past the debate over the pro's and con's of testing. I'm willing to believe that tests measure something but not that they measure everything that's valuable in schooling.

I agree with Deborah Meier that "habits of mind" are key. It's fairly easy to imagine being able identify school situations that make developing those habits possible -- and endless test prep isn't one of them. I think the task of identifying which students have developed them consistently is harder (or at least more labor intensive).

The standardized testing mindset is that every student should be tested by an outside monitor every year. Its a mindset that values a large quantity of information over a high quality of information, and confuses the questions of whether the purpose is to evaluate school, or to evaluate students.

As a start we might look at ways of evaluating schools that allow us to look more deeply than multiple choice test scores. Even if testing is involved, perhaps instead of giving every grade multiple choice tests, we gave a few grades more probing assessments?

The tests should be made fully public soon after their administration. That way we can analyze them and propose improvements.

The current secrecy and tight security around each test impede public scrutiny. We need access to the tests in order to consider at least three questions:

1. What is being tested, and why? How does the material being tested fit into the overall subject matter?
2. Do the questions test this material effectively? How so, or how not?
3. What could be done to (a) improve the curriculum and (b) ensure that the test reflects it?

A website with past tests and comments would be a start.

I'm glad Rachel brought up our collective confusion about the purpose of an accountability system - are we evaluating students or schools? If we want to evaluate schools, why not administer more complex assessments to a random sample of students in addition to giving these basic skills tests? (By the same token, do we need to test everyone on basic skills tests every year?) That would keep costs in check, and allow us to evaluate schools on a broader set of skills.

And I agree with Diana that sample tests should be public. It is an important first step in educating parents and citizens about what these tests can and can't do.

Skoolboy, well done on the testing revenues!

You said, "If we want to evaluate schools, why not administer more complex assessments to a random sample of students..."

I thought, "Once again, people don't think about rural schools." Most education writers think only of urban and suburban districts, and national policy is set with those schools in mind.

Very small schools, located in areas with small populations (and often large distances), have to live with those decisions. Regarding your current suggestion, they don't have enough students per grade level to determine school effectiveness now....how could they possibly sample?

If larger schools could save money by sampling, and small schools could not, then small schools would pay a disproportionately large proportion of their meager budgets on tests and testing. This would exacerbate a situation that's already bad. When was the last time you walked into a school of elementary children where the lights weren't on and the kids wore their coats to stay warmer? For us, the answer is, "last winter."

So...this is way off topic! I appreciate the introduction of financial interests -- big financial interests! -- into the discussion. It's an important issue that doesn't get enough coverage.

If larger schools could save money by sampling, and small schools could not, then small schools would pay a disproportionately large proportion of their meager budgets on tests and testing. This would exacerbate a situation that's already bad.

You could give schools enough money to cover the cost of testing the number of students they actually were asked to test. Radical idea, though, actually funding mandates...

Comments are now closed for this post.


Recent Comments

  • Rachel: If larger schools could save money by sampling, and small read more
  • Kate: You said, "If we want to evaluate schools, why not read more
  • eduwonkette: I'm glad Rachel brought up our collective confusion about the read more
  • philipkovacs: Jettison the tests... http://www.youtube.com/watch?v=45HIW3jgEPg&eurl=http://www.educatorroundtable.net/frontPage.do read more
  • Diana: The tests should be made fully public soon after their read more




Technorati search

» Blogs that link here


8th grade retention
Fordham Foundation
The New Teacher Project
Tim Daly
absent teacher reserve
absent teacher reserve

accountability in Texas
accountability systems in education
achievement gap
achievement gap in New York City
acting white
AERA annual meetings
AERA conference
Alexander Russo
Algebra II
American Association of University Women
American Education Research Associatio
American Education Research Association
American Educational Research Journal
American Federation of Teachers
Andrew Ho
Art Siebens
Baltimore City Public Schools
Barack Obama
Bill Ayers
black-white achievement gap
books on educational research
boy crisis
brain-based education
Brian Jacob
bubble kids
Building on the Basics
Cambridge Education
carnival of education
Caroline Hoxby
Caroline Hoxby charter schools
cell phone plan
charter schools
Checker Finn
Chicago shooting
Chicago violence
Chris Cerf
class size
Coby Loup
college access
cool people you should know
credit recovery
curriculum narrowing
Dan Willingham
data driven
data-driven decision making
data-driven decision-making
David Cantor
Dean Millot
demographics of schoolchildren
Department of Assessment and Accountability
Department of Education budget
Diplomas Count
disadvantages of elite education
do schools matter
Doug Ready
Doug Staiger
dropout factories
dropout rate
education books
education policy
education policy thinktanks
educational equity
educational research
educational triage
effects of neighborhoods on education
effects of No Child Left Behind
effects of schools
effects of Teach for America
elite education
Everyday Antiracism
excessed teachers
exit exams
experienced teachers
Fordham and Ogbu
Fordham Foundation
Frederick Douglass High School
Gates Foundation
gender and education
gender and math
gender and science and mathematics
gifted and talented
gifted and talented admissions
gifted and talented program
gifted and talented programs in New York City
girls and math
good schools
graduate student union
graduation rate
graduation rates
guns in Chicago
health benefits for teachers
High Achievers
high school
high school dropouts
high school exit exams
high school graduates
high school graduation rate
high-stakes testing
high-stakes tests and science
higher ed
higher education
highly effective teachers
Houston Independent School District
how to choose a school
incentives in education
Institute for Education Sciences
is teaching a profession?
is the No Child Left Behind Act working
Jay Greene
Jim Liebman
Joel Klein
John Merrow
Jonah Rockoff
Kevin Carey
KIPP and boys
KIPP and gender
Lake Woebegon
Lars Lefgren
leaving teaching
Leonard Sax
Liam Julian

Marcus Winters
math achievement for girls
meaning of high school diploma
Mica Pollock
Michael Bloomberg
Michelle Rhee
Michelle Rhee teacher contract
Mike Bloomberg
Mike Klonsky
Mike Petrilli
narrowing the curriculum
National Center for Education Statistics Condition of Education
new teachers
New York City
New York City bonuses for principals
New York City budget
New York City Budget cuts
New York City budget cuts
New York City Department of Education
New York City Department of Education Truth Squad
New York City ELA and Math Results 2008
New York City gifted and talented
New York City Progress Report
New York City Quality Review
New York City school budget cuts
New York City school closing
New York City schools
New York City small schools
New York City social promotion
New York City teacher experiment
New York City teacher salaries
New York City teacher tenure
New York City Test scores 2008
New York City value-added
New York State ELA and Math 2008
New York State ELA and Math Results 2008
New York State ELA and Math Scores 2008
New York State ELA Exam
New York state ELA test
New York State Test scores
No Child Left Behind
No Child Left Behind Act
passing rates
picking a school
press office
principal bonuses
proficiency scores
push outs
qualitative educational research
qualitative research in education
quitting teaching
race and education
racial segregation in schools
Randall Reback
Randi Weingarten
Randy Reback
recovering credits in high school
Rick Hess
Robert Balfanz
Robert Pondiscio
Roland Fryer
Russ Whitehurst
Sarah Reckhow
school budget cuts in New York City
school choice
school effects
school integration
single sex education
small schools
small schools in New York City
social justice teaching
Sol Stern
Stefanie DeLuca
stereotype threat
talented and gifted
talking about race
talking about race in schools
Teach for America
teacher effectiveness
teacher effects
teacher quailty
teacher quality
teacher tenure
teachers and obesity
Teachers College
teachers versus doctors
teaching as career
teaching for social justice
teaching profession
test score inflation
test scores
test scores in New York City
testing and accountability
Texas accountability
The No Child Left Behind Act
The Persistence of Teacher-Induced Learning Gains
thinktanks in educational research
Thomas B. Fordham Foundation
Tom Kane
University of Iowa
Urban Institute study of Teach for America
Urban Institute Teach for America
value-added assessment
Wendy Kopp
women and graduate school science and engineering
women and science
women in math and science
Woodrow Wilson High School