Putting More Human Judgment Into Student Assessment
I'm trying to get at the heart of our agreement and disagreement.
Example: I think we are interpreting the word "measure" differently. I'm using judgment, in this case, to distinguish from measurement, which has a connotation of numerical precision and objectivity that judgment lacks--for good reasons. Judgment, as I see it, has all the defects of its blessings. It doesn't claim to be able to do more than it claims and for that very reason is at its best when it is required to work in collaboration with other factors to reach a consensus. It doesn't, also, claim to be righter than what it represents. It's at the heart of democracy, and that means it is sometimes wrong!!!!
For the purpose of graduation requirements judgment therefore operates best when its members or contributors include a majority who have no special interest or conflict of interest with the task set before it. I liked including a member of the faculty and an adult chosen by the candidate for graduation and the student assistant. Generally we kept the membership as constant for all meetings and defenses as possible, adding--where needed--someone expert in the field/domain under review to add expert disciplinary knowledge if needed.
Our system never ranked candidates. It was a pass/fail system. There was no way to "scientifically" prove its reliability, although the psychometricians who the state appointed to review it claimed it did as well in this regard as most current tests (like New York Regents' exams) do.
We sought evidence as close to the real thing as we could. In art, for example, it was in coordination with an art exhibit, or concert--depending on the field. At Mission Hill we insisted that every graduate perform in some way before an audience--reading a poem, doing a small drama skit, singing, playing an instrument, etc. The visual arts was on a separate studio night. In math and history we added a short pre-presentation activity--a research activity and a time-line exercise for history and a short standardized math test for mathematics. The portfolio of work put together by the student and his or her advisor with a presentation of one of the items included was all that was required in science and literature. The "outside experience" included affirmation letters regarding real-world experiences. Yes, it was a scheduling mess, and time-consuming, but it was extraordinarily valuable as a motivator and as a learning experience. It counted as educational time, not testing time.
At the high school we did add a way to get honors, but it required an additional project and review that was optional for graduation purposes. Many students had incompletes, most of which could be judged for completion by just the faculty advisor. Some required a second (or even third) graduation committee meeting.
Honoring human judgment becomes more, not less, important as modern technology sometimes fools us into thinking we can be replaced by technologically implemented equations. The troubles inherent in resting our case on that kind of process lie in the direction of supporting democracy, with its undue respect for ordinary human judgment. To paraphrase Jefferson, democracy faults require more education of ordinary people, not less respect for their judgment.
For many reasons, it's hard to put in black-and-white what constitutes a sound basis for judgment--although I can recognize that it's worth a try. Different academic disciplines even use rather different criteria for what constitutes good evidence. So, probably, do good democrats. Thus it should be short and sweet, and leave the details to each community. I keep remembering that when I was born, we had 160,000 school boards. We now have less than 16,000, and the remaining are often appointed and members removable--especially where low-income minority families constitute the school system's majority. And all this as the population has probably nearly doubled!! It is a shift that hasn't improved human judgment or the vitality of democracy.
We need to return a public to public education--whether charter or not.
Joe Nathan replies:
Deb, you and I agree about the value of performance assessment as part of high school graduation. I think the driver's license exam offers a useful metaphor for high school graduation. I also think we should be learning from the classic "Eight Year-30 School Study" when we consider organization and graduation from high schools.
Let's start with the driver's license exam. In most states, there are two parts: the written and the actual driving demonstration. Today the written exam often is given on a computer. It's essentially the same as when people wrote their answers. People are asked to demonstrate knowledge about certain facts. This does not guarantee that they will be good drivers. But it does measure whether they know at least some of the things that will be required for safe driving. People are asked, for example, which lane they should turn into, or how to safely park on a hill, or why it's important to not just look in the mirror, but actually check before moving into a different lane.
The second part of the driver's exam involves actually driving. Again, no one expects that people will be perfect. But the performance exam is a measure of whether a driver has at least some minimum knowledge of how to operate a car safely.
No one suggests that either these two parts is enough. We wisely expect people to do both before being licensed to drive.
I think a combination of some form of statewide test and real-world performance makes great sense in graduation. You mentioned that schools can learn from each other about what those standards and assessments could be. Again, we agree.
The St. Paul Open School, where I worked, used this approach. Faculty, community members, families, and students developed a combination of performance assessments, as well as requiring students to demonstrate certain math and reading skills prior to graduation.
As mentioned earlier this week, we produced a report that described how this process works in a number of high schools.
We also learned from the classic "Eight-Year Study." My mentor and friend Dr. Wayne Jennings introduced me to this report. It was done in the 1940's and examined how well students did in college after graduating from high schools that used approaches. As one reviewer explained, a key finding was that in college, "the farther high schools departed from the traditional college preparation program, the better was the record of their graduates."
High school students who set goals, carried out projects, often interdisciplinary projects, and produced real-world products did well in college.
Recently we've been working with and learning from Harding, a large traditional St. Paul district high school that has developed a portfolio process as part of high school graduation. Before graduating, each student must present information about her or his high school years. This includes community service and work experience, grades, extra-curricular experiences and post high school plans. People reviewing the portfolio include both faculty and community members. After participating in the process, I wrote, "students review their accomplishments, plan for the future and consider what they have accomplished. It's a great gift, making graduation much more meaningful."
What should be in the exam or exams that all students must pass in order to graduate, as they must pass to earn a driver's license? Those are important questions that I'd prefer to discuss another day.
The Carnegie Unit, and the rigid three-years-of-this, four-years-of-that approach to high school don't make sense, at least for many students. A combination of traditional tests and performance assessments are a better approach to graduation.
Deborah Meier responds:
I have always loved the driver's test analogy but I seriously doubt anyone does more than cram for the written test. Surely I did. And do I remember how many feet to stop, to park by a hydrant or corner, et al? No, I count on other ways to figure these out.
But I could imagine states, in a cost-savings move, attempting to standardize and make more objective the whole process by eliminating the performance part of exam (the it depends a lot on the judgment of the man you're driving with), making the written part "more rigorous". That's the path we've taken with schools. The connection between knowing how to drive and answering tough questions (including better understanding of how the engine works?) is dubious, but perhaps there is a rough connection that could be created by pre-testing the pool of answers. Why not? Probably one advantage of an easyish test that rests mostly on performance is that more people drive cars, which is good for the auto industry and the oil industry. (In my day you even got a booklet to prep with for the written test. Do they still do that?)
I'm surprised that you advise a statewide standardized test (plus performance assessment)--given their incurable bias as well as the tendency to "prep" for them for useless hours. If we can't be completely weaned away from standardized, let states do sampling. But we need to use exams we respect and that can over time change as we learn better. These are the types of assessments that should play a role in real-life high-stakes decisions. Schools can add on school-written exams that they believe in for factual knowledge, saving the portfolio presentation before an audience for the more important stuff.
The "Eight-Year Study" unfortunately got buried in World War II and then hit a kind of anti-progressive era in American schooling. (Remember Johnny who couldn't read?) But yes, it should be read by one and all. Some will say, "but those kids came from middle class families and..." etc. True. But we say, what the rich need the poor need even more of.
The feature that's most critical in the assessment or judgment process, more so for kids understandably alienated from what they often see as a scornful and often dangerous society--and that's not paranoia--are a community of adults that includes their families and their teachers. They need good reasons to trust them, especially when they must be annoying nags, as allies on behalf of their futures. That means they must get to know each other--teachers, families and student--over time. It takes a different kind of schedule and a different definition of the job to make this not merely feasible, but a pleasure.