« Job Opening In Dallas | Main | Best Of The Week (July 30-August 3) »

What Testing Guru Bill Sanders Really Meant About Multiple Measures

| 6 Comments

Once in a while, I actually do some reporting, and today I happened to talk Prof. William Sanders, the testing guru whose recent letter to Congressman Miller was leaked to the press and seemed (according to an Ed Daily story) to put Sanders squarely against Miller's proposed use of multiple measures in AYP.

Well, it turns out that Sanders is against the use of portfolios and classroom observations that are often called multiple measures, but not against end of course tests, college entrance tests, and the like that he thinks Miller is talking about. "Those things have a place," says Sanders, who points out that they are already part of the growth model projections that he has developed and are being used in some pilot states.

1080.jpg
To those who are concerned about the complexity and transparency of both the current AYP and proposed changes, Sanders says such intricacies are the price of a nuanced and reliable rating system. "A simple system could be developed," he says, noting that some states are going that direction, "but it would be less reliable and more biased [than a more complex one]."

His main accountability concern, however, is not so much that the current AYP relies on "a single test" (a description he says irks him and ignores the fact that there are three years of tests and hundreds if not thousands of test item responses that go into each year's AYP calculations), but rather that it encourages too much focus on lower-performing kids rather than "early high-achieving kids" who get ignored. He proposes a rating system that evaluates schools not only on reducing the achievement gap but also on helping already-proficient kids do even better -- apparently a part of the Tennessee pilot and perhaps what Nevada is trying to do here.

6 Comments

A correction - Sanders' work in Tennessee is well past the pilot stage. He pioneered value-added achievement in the Knox County schools there in the late 80s, and the resulting statewide system - the Tennessee Value-Added Assessment System, or TVAAS - was in place 10 years prior to NCLB.

Value-Added, or growth, data does do what you imply - it allows you to not only see whether low-achieving kids are making progress towards grade-level proficiency, but also to see whether high-achieving kids, who are already at or above proficiency levels on state tests, are continuing to achieve or are resting on their laurels.

Sanders is right. Portfolios can be easily compromised by myriad parties and classroom observations are a subjective joke - every teacher receives the standard hearts and flowers about how 'terrific' they are. Why doesn't Miller just craft a new bill that will return us to the good old days of social promotions and high school diplomas for everyone merely for showing up (or not) for four years. Has the NEA really bought out this California Democrat? It unfortunately looks that way. You have got to be kidding Miller! Multiple measures of what? Graft? Bribery? Coercion? Idiocy?

Alexander, my story for Ed Daily said that Sanders “reiterated concerns” about several of the “multiple measures” some organizations support as part of the reauthorization story. Indeed, the letter expresses concerns about adding formative assessments and non-academic measures into the accountability system. But the story did not say that Mr. Sanders opposes the concept of multiple measures outright. For example, I quoted a part of the letter where Sanders notes that projection growth models that use multiple years of test data to calculate growth trajectories for students would essentially be systems based on multiple measures.

I'm with Sanders on his reaction to the "single test" verbiage that has become codified. It doesn't fit with the reality--either of accumulated long-term results for schools and districts, or multiple testing opportunities for students on the few tests (like graduation tests) that are truly "high stakes."

It is right to question exactly what is meant by "multiple measures." In fact various states have instituted measures for various other things: growth, averageing and other "safe harbor" provisions; value added measures; climate-related issues; parent involvement. Nothing in NCLB prohibits states from measuring and reporting these things, or rating schools and districts based on them--or from setting standards and measuring progress in the disciplines outside of NCLB.

Yet when certain voices start to advocate for multiple measures, I begin to suspect a subtraction of expectations, replacing standardized testing with something more easily manipulated.

sorry if i got something wrong, steve --

the story i saw included only a graph or two from sanders, which made it seem like he was pretty much against the whole thing:

"Prominent statistician William Sanders, whose research has been used as the basis of Tennessee’s and Ohio’s growth models under the Education Department’s pilot program, reiterated several of the civil right organiza- tions’ concerns in a July 17 letter to Miller and McKeon.

“Most of the measures usually advocated under the banner of ‘multiple measures’ have so little reliability that any attempt to use them in summative assessment is certain to provide results so untrustworthy that es- sentially no distinction among schools can be made,” he wrote.

Sanders promoted growth models that incor- porate multiple years of student test data as the “greatest use of ‘multiple measures’ in a reliable, robust way.”

If there's another story, or if you have the whole letter, I'd be happy to update and correct this.

Prominent statistician William Sanders,
whose research has been used as the basis of
Tennessee’s and Ohio’s growth models under
the Education Department’s pilot program,
reiterated several of the civil right organiza-
tions’ concerns in a July 17 letter to Millerand McKeon.
“Most of the measures usually advocated
under the banner of ‘multiple measures’ have
so little reliability that any attempt to use
them in summative assessment is certain
to provide results so untrustworthy that es-
sentially no distinction among schools can be
made,” he wrote.
Sanders promoted growth models that incor-
porate multiple years of student test data as the
“greatest use of ‘multiple measures’ in a reliable,
robust way.”

Whenever I read that we wish to have "multiple measures," I assume it is because we want to see how well kids think, not simply find out what they know. Do we really care how well kids think? The general public gives little credit for it. Even highly respected shows like "Jeopardy" test speed and knowledge. On one TV "Millionaire" show the question was "What number is a perfect square and is the sum of two perfect squares?" This question was for $16,000. The contestant couldn't figure it out, and asked the audience for help. The possible answers were 16, 25, 36 and 49. The highest choice was 16, which was wrong. Yet, this is a very simple Pythagorean Theorem question which I would expect any sophomore to answer. Furthermore, on this particular show the contestant has unlimited time to "figure it out."

I simply don't believe we really want our children to be good thinkers. Even Shakespeare said, "Yon Cassius is dangerous, he thinks too much." Furthermore, reasoning and thinking is far more difficult to measure than simple knowledge. How do you measure someone's thought process when he/she "thinks outside the box?" As a teacher I tried to encourage kids to go beyond the textbook, but damn, it was hard to assess. One of the brightest students I ever taught defined a geometric object as an "equilateral equiangular quadrilateral." What a beautiful use of our language. Yet, because this was a talented kid among other talented kids, I challenged the definition as being "minimal" in a mathematical sense. The entire discussion took 15-20 minutes of class time, but that was necessary for the kids to grasp the nuances of language when studying something like mathematics.

When Lewis Carrol wrote the "Alice" books, he played with language, logic and even parodied the English aristocracy. Clearly, simply reading the books would not address the type of reasoning that went into these stories. Teaching kids to reason is not a simple task that can be measured with norm referenced assessments.

I recommend a rereading of the "30 School/Eight Year Experiment" of the Progressive Education Association which was carried out in the 1930's. It was an attempt to teach kids to think, not simply know. It worked well. It was reported in a five-volume series, "Adventure in American Education." (1942, I think)

Thoreau once said, "Schools was to make a straight ditch cut out of a meandering brook." Isn't that was all of this NCLB stuff is about, getting everyone to follow the same path and disregard the individual differences among us?

Comments are now closed for this post.

Advertisement

Recent Comments

  • Patrick: A very fitting farewell. So does this mean that David read more

Archives

Categories

Technorati

Technorati search

» Blogs that link here