« If we're so smart, why are we broke? | Main | Organizing for Success in Hands-On Science »

Doug Christensen: He Fought the Law...

| 13 Comments

For the past six years, Nebraska educators, led by Commissioner Doug Christensen, have waged a lonely battle to preserve the integrity of assessment in their schools. Their system survived challenge after challenge, but now, even as NCLB may be on the ropes, Nebraska is implementing standardized tests. As a result, Christensen has resigned his post as state Commissioner of Education. He recently responded to a series of questions I posed to him, and as you will see, he has some potent lessons for us as we weigh the alternatives facing us related to teacher empowerment, the Federal role in education and No Child Left Behind.

Question: What were the important accomplishments of the STAR system (Student-based Teacher-led Assessment and Reporting System) in Nebraska?

STARS created a system of education where the classroom was the center of the system not the bottom of a hierarchy. It centered the work of the system on what happens in the classroom and clarified that the work of the system was teaching and learning. It placed students at the center of what schools do and placed the work of teachers in a leadership role.

Under STARS, teachers became instructional leaders, principals became leaders of learning, superintendents became "chief education officers," and local boards became policy leaders. All of the roles are leadership roles and all designed to support the work of teaching and learning and the classroom.

STARS put the tools in place to cause instruction and teaching to improve and put the responsibility for making changes in instruction and teaching in the hands of teachers and their principals. It put the data in the hands of teachers and principals so that changes could be made as learning occurred, or did not occur, rather than waiting until the results from an end of the year test are returned and students have moved on in the curriculum.

Question: What difference does it make what role teachers play in designing assessments?

Without a system like STARS, the sequence of strategies is something like standards---assessment----curriculum----accountability. A STARS-like system requires that assessment comes out of instruction that is aligned to standards (but not limited by the standards) and the only professionals that can bring assessment out of instruction and teaching are the teachers. Without every teacher having a role in designing assessments for their own classroom, teachers really cannot effectively teach. They need to have a clear notion of what the learning they are after looks like and that means what they are going to assess and how they are going to assess it.

And, without leadership and control over assessment, teachers will never be regarded as professionals. Professionals, in every field including education, are those workers who have control over the "metrics of determining good practice" and control over the "metrics of what determines successful work." Imagine for a moment, auditors, lawyers, medical professionals and even morticians allowing someone other that those that are "trained" in the field to determine what is good practice and to decide what will be used as the measure of the work.

Question: How did the STARS system impact student learning?

Using our state writing assessment as a benchmark, our proficiencies in reading, mathematics and writing were all an average of 86% for grades 4, 8, and 11. Using ACT scores as a benchmark, STARS supported the continuing increases in our statewide averages for ACT scores. Using our standardized test scores as a benchmark, STARS supported the increase in the statewide averages.

Question: What do you think the appropriate role is for standardized tests?

I do not think there is any appropriate role for standardized tests. We used them as benchmarks only because we had to have something that people could look to to make sense of the proficiencies and determine if our STARS scores were "for real." In a perfect world, the only way to build an assessment system for measuring student achievement and having additional measures to validate the results is to begin building the assessments from the classroom up with the assessments given in the classroom to be the major assessments and the comprehensive assessments which are then "validated" with assessments at the school/district and state levels that are samples of the assessments given at the level below.

Question: How do you respond when people suggest the schools and teachers are dodging accountability if they do not embrace standardized test scores as a valid measurement of their success?

How can one be dodging accountability when standardized measures do not align to standards? When they do not align to intellectual skills like problem solving, thinking skills, application skills, .........creativity, analysis, synthesis.......?

Standardized tests are based on metrics of getting the same score if given over and over again. The only way to get such reliability scores is to test the lowest levels of knowledge for which there is one and only one right answer. Is the knowledge that we value most the kind for which there is only one response? Is the selection of a response from a list of responses a representation of learning the content of the question, being able to read, being able to discriminate, guessing, all of the above, none of the above?

Question: What do you think of the idea that the problem with NCLB was that there were not national standards, so we lacked a "level playing field"?

National standards are not the answer to anything. If national standards are the answer, as is being proposed by many, what is the real question? If the real question is "what do we want our students to know and be able to do," let us realize once and for all that there is no universal answer to this question. The only answer is the one that considers each student, each context, each set of goals and aptitudes.....

Because national standards will "standardize" the system when assessments are tied to them, we will reduce what is taught and what students learn to what can be captured in the statements of the standards and what can be measured in the design of the assessments. What we need is more variation in the ways schools are run and variation in the ways teachers teach, not making the process "teacher proof."

If the question is "can we trust teachers/educators to do the right things and to push students to the highest levels of learning" and if the answer to the question is "no," then national standards and standardized assessments is the answer. However, I do not accept that this is the right question and I certainly do not accept that the answer is that educators can not be trusted. We should try a system that trusts educators before we decide that we cannot (trust them).

Questions: How do you see the landscape shifting as we anticipate a change in administration?

I see the landscape shifting in ways that will open doors to other kinds of assessments. I hope the landscape will change so that we articulate the issues and questions more clearly so that whatever we construct addresses a common understanding of our issues, our goals, and the work to be done. I hope the landscape embraces a "theory of action" that is more focused on local leadership, building capacity at the local level, finding "common" measures of learning (not same measures), builds accountability from the inside out not the outside in and builds the accountability on improvement in results, not status, and I hope that we begin to describe what we want in terms of results from schools to be more than academic achievement such as engagement, meaningful relationships, etc.

Nothing new in NCLB will happen until we recognize that we need to re-build not re-authorize. Nothing will happen to change federal education policy until there is a "space" for conversation and engagement and a change in administration could create the "space" needed to engage a national conversation about education, about education policy and practice and about accountability.

Question: What has brought about your decision to leave your position as Commissioner of Education for the state of Nebraska?

I had notified the State Board over a year ago that I was nearing the time that I would step down and was given the opportunity by the Board of setting my own timetable and afforded the opportunity to stay as long as I wanted to stay. In January, I indicated to the Board that I felt that the end of this calendar year would be the time for me to (1) move on to the next phase of my life, (2) give time back to my family that they sacrificed for me to do this work, (3) write the book that I believe is in me, (4) spoil my grandchildren and (5) teach students who are entering the leadership dimension of their careers, e.g. principals and superintendents.

Then, in April, the Legislature passed legislation that called for transition to a state test in 09-10. My values of what I believe is right for students and for teachers will not allow me to lead our state into a state test. I believe that state testing is wrong and is not in the best interests of students, teachers and other educators, and schools. I cannot uphold the constitutional responsibility of being a Commissioner who is to uphold the "law of the land" and put in place something that I believe is so dreadfully wrong as education policy and so destructive as public policy about education.

So readers, what do you think about what Doug Christensen has shared? Should we seek variation rather than standardization? Can teachers be trusted in the ways Christensen suggests?


13 Comments

How do we clone Doug Christensen? He talked the right talk--against daunting opposition. And then he walked the talk.
Plus the guy thinks! That's nearly a lost art in EdLand. I'd argue a bit with a few of his positions, but the positions are logical rather than ideological. I'll look forward to reading his book.

Doug is right on the money. Or rather, right on the place where the money isn't -- thoughtful education. Thoughtful (democratic) education is really a no-brainer: teachers with responsibility sharing that responsibility with the rest of their schools, including the students, who share responsibility with everyone else. If there really were "right" answers -- like we learn from our multiple-choice standard assessments -- then choosing our President would be a much simpler task. So my question is: why don't educators act like educators? Why is it that so many violets shrink into the shadows and do what they're told, when what they're told is so anathema to what our country, and this world needs for our children? I applaud Doug -- and can only shake my head in dismay over the thousands of teachers, students, parents in Nebraska who hear him and can only lower their heads waiting for the next round of idiocy from bureaucrats who have difficulty even spelling "algebra" (or pronouncing "nuclear" correctly). Unfortunately, American education will not appreciably vary from a century-old script "designed" for a pre-WWI population in a burgeoning relationship with mass production and psycho-metrics -- until its people realizes that education is not about giving everyone the same sets of facts, but about helping persons to learn how to learn, and to learn to act like democratic actors in a global civilization.

In reality, STARS has been used to cover up the serious educational neglect of poor and minority students. While STARS testing showed that 75% of Nebraska's African American students were reading on grade level in 4th grade, NAEP tests showed that only 9% of African American students were actually reading on grade level. Under STARS, the scores of African American students have actually dropped, rather than improved. The long term consequences of this neglect is reflected in the fact that in Nebraska only 39% of African American students graduate from high school on time as opposed to 83% of white students. NCLB has serious defects, but STARS does not present a positive alternative.

RE: neglect of poor and minority students in Nebraska-- Only about 4% of Nebraska's population is African American (compared to about 13% nationally, and, of course, vastly higher percentages in most large urban districts, where minorities and poverty are concentrated). You need to look at the context for minority students in Nebraska to understand the statistics and apply causality.

WHY is the graduation rate of such a tiny minority different from the much larger and wealthier white student cohort? What is it like to be the one black child in a class of 25 white kids--and might that impact achievement far more than who designs and administers the assessments?

I attended a seminar at the National Academy last November and saw a presentation on STARS. One well-received point was that a program like STARS depends on building "assessment literacy" and expertise in teachers. If teachers are designing the assessments, they need to know how to build valid and reliable tests and performance measures, a skill that's been overlooked in many teacher training models. Any state program that supports and increases teachers' skill in aligning required content to assessments is aiming for the right goals.

To turn the responsibility for measuring student learning over to professional, for-profit test-makers is irresponsible, even if required by law.

Nancy:

I didn't understand Judy's comments to be inferring any causality with regard to the low achievement of low-income and African American students. But if your indicators are developed in such a way as to mask that reality, one never even gets to ask that question (regarding causality).

I would agree that there is a profound lack of ability in the classroom with regard to developing valid and reliable measures of student achievement, particularly aligned to an standard. This is why grades tell us so little about what students know. I don't know that Nebraska was on the right track to improving this through using STARS as its statewide testing system (and as I recall, they were only able to pull this off by having districts determine their own standards). I am also not sure that I see the irresponsibility of having professionals write the tests--even if they are for-profit companies. Certainly we have relied for many years on for-profit companies to produce text-books, with very little question. As you point out, we lack skill in the field and if we are to have fifty testing systems we cannot be too choosy about where we hire from. Another option would be to develop a US Office of Assessment. The government would then hire from the same limited pool of experts. I rather suspect that there might be some issues raised there also.

Margo/Mom,
I am struck by your use of the words "used to mask"--as if the state of Nebraska conspired to hide the low achievement of minorities through a well-developed program of assessments tied to benchmarks set by the state. The fact that 39% of African American students graduate HS in NE, with the white student rate more than twice that, can be attributed to many factors--but it's doubtful that it can be linked reliably to STARS. After all, both African American and white students are learning and being assessed under STARS. If STARS is problematic, it would be problematic for both black and white students.

We basically have two choices in assessment: give teachers more training in developing valid and reliable measures of achievement, including performance assessments--or rely on standardized tests created by for-profit organizations and denigrate the assessing done by teachers as being tainted by their desire to "hide" something. I would certainly be in favor of having "professional" psychometricians work with educators in designing better assessments. That's not what has happened, however.

And as for saying that for-profit test-making is OK because for-profit textbook creation is the also the norm is accepting something questionable because there are other questionable practices. The control that profit-making textbook publishers have over state and district curriculum is not only inflexible but also unethical.

Nancy:

You keep trying prove that STARS has not caused failure of African American and low income students--a charge that no one is making. While there may never have been a closed door meeting of men in suits who discussed choosing between STARS and a state-wide test and voting for STARS because the state-wide test would show that African American and low income students are less well-educated at the end of the day, one of the most substantial "push backs" from school systems nationally has been that their test scores for disaggregated groups makes them "look bad."

Valid and reliable teacher-made tests have a place in education. They provide immediate formative and diagnostic information to guide instruction. One of the shortcomings of state-wide standardized testing is that it does not do a very good job of this--nor should it be expected to. Statewide testing is a very good summative tool used for accountability. We need both. But the two should not be confused.

I am not familiar with the demographics of Nebraska--but I rather suspect that the small percentage of African American students are not evenly distributed at a rate of one per classroom, as you suggest. Based on my experience in other states, there is a greater likelihood that both ethnic minority and low-income students are concentrated in a smallish number of schools/districts. If the state chooses to rely on teacher-made tests, developed at the school level (knowing that this a weakness in the field), with the bias problems that this presents, that meeting of men in suits doesn't need to take place. The state is choosing a methodology very likely to overlook areas of concentrated weakness. The fact that this has been the outcome is supported by the difference in STARS and NAEP outcomes.

Teachers need quality formative assessments in the classroom--this should be supported. This does not, howerver, negate the need of policy makers to have quality summative assessment at the building, district and state level.

[Judy] In reality, STARS has been used to cover up the serious educational neglect of poor and minority students. Under STARS, the scores of African American students have actually dropped, rather than improved.

[Margo/Mom] You keep trying prove that STARS has not caused failure of African American and low income students--a charge that no one is making.

[Nancy] Well, someone is making that charge, Margo.

Standardized tests are only "summative" when they are aligned to clear content standards and benchmarks. One of the huge problems with standardized tests is that they're usually not tied to curriculum tailored for identified student needs. Teacher-created assessments can measure whether students have learned what teachers taught--and serve both a formative and summative purpose. I have actually heard policy-makers argue that formative assessments don't "count" (so, as you suggest, we let the teachers design them)--but formative assessments often provide the most critical and useful information to both teacher and *student* about growth, diagnosis and next steps. They count, and count a great deal.

Standardized tests are good for comparing groups of students--as you noted, building to building, state to state. The "problem" in NE was that schools and teachers there had the temerity to value their own judgment over tests created by psychometricians.

We can set national content standards and benchmarks and align them with standardized tests--producing a massive, seductive data base, something that researchers have been dreaming of for years. And then, we will definitively know who is best at taking standardized tests. And it will be the same kids who were high achievers before we invested billions in value-added data analysis. And the kids who most need our formative and summative feedback will still be at the bottom of the heap.

Of course STARS testing does not cause minority group underachievement. The problem with STARS testing is that it does not accurately reflect reading achievement. STARS testing suggests that 75% of African American students are reading on grade level, while NAEP tests show that only 9% of African American students in Nebraska are actually reading on grade level. The situation is almost as dismal for Latino students with STARS testing showing 80% of Latino students reading at grade level while NAEP tests show only 12% of these students reading on grade level. By obscuring the achievement gap, STARS discourages honest discourse about the causes of this gap and potential remedies.

I am a classroom teacher in a struggling school myself, and I understand the value of teacher made formative assessment. But it is natural for a teacher to assume that what is average for the students in her class reflects what is actually grade level achievement. Objective standards are needed to alert teachers and policy makers to serious educational problems.

Our goal is to help all students achieve better, and beyond that, to grow into healthy, productive and happy people. The question is how do we make sure that our schools have the best chance of making this happen. Doug Christensen has described a process where local districts define their standards and empower their teachers to assess how their students are doing relative to those standards. There are some significant advantages to this approach. It is fundamentally democratic, in that the decisions are made at the community level, where citizens, parents, teachers and students have the most influence. It is inherently engaging, in that the teachers are active participants in crafting the means by which their students' achievement is measured. Tying this to classroom practice encourages more authentic assessment.

This is being criticized as "not objective." I think there are some big assumptions being made here, and we are at a big disadvantage discussing a program with which we have no firsthand knowledge. If anyone who actually teaches in Nebraska and has experienced the STAR system would like to weigh in, that would be most welcome. But I think we need too be very careful about the "objectivity" we ascribe to tests such as NAEP. Recently significant flaws have been pointed out in the reliability of NAEP. From this op-ed in the Washington Post: "The National Academy of Sciences said, 'NAEP's current achievement level-setting procedures remain fundamentally flawed, appropriate validity evidence for the cut scores is lacking, and the process has produced unreasonable results.'" The point is that tests are imperfect means of assigning value to achievement. I think it is useful to have a variety of sources of data, and to have teachers actively reflect on the meaning of different indicators. But I do not believe standardized tests should be the last word on achievement.

The achievement of our black students remains a major area of concern and in some ways, regardless of herculean efforts to address them, remain unchanged and intractable. No one gives up on any one in Nebraska. And, we are not giving up on any of our students, let alone our students of color.

STARS is intended to show the growth of achievement not the static "trapping" of a score at a point in time and by a methodology that is negatively affected by poverty, mobility, disability and learning the English language. STARS does not "mask" achievement of any group. If it did, we would not know that our black students are not doing as well. However, I am not willing to take any other SINGLE measure such as NAEP and make global generalizations like the ones that we are masking performances and that our black students do not do as well.
]

The fact is this. On NAEP our black students do not score well. On STARS they show considerable growth.

Before condeming STARS on the basis on conflict with one other indicator (NAEP), review the improvement made by our other subgroups particularly our students who are learning the English Language, who are hispanic\Latino, who are mobile and who are individuals with learning disabilities. Their trend lines are up and the gaps between them and our overall achievement is narrowing.

STARS does not purport to just show growth. It claims that 75% of African American students and 80% of Hispanic/Latino students actually read on grade level. Even if we ignore NAEP testing, it is hard to see why students who read so well, have such abysmal graduation rates. As far as trend lines go, the 2003-2005 data show that Latino achievement has remained stagnant while African American achievement has actually decreased. If STARS testing accurately reflected reading ability,the Black and Latino students of Nebraska would have among the highest rates of reading proficiency of students of any ethnic background in even the wealthiest states. Imagine what would be achieved if the citizens of Nebraska could confront their lawmakers with more realistic data. Reading proficiency rates of 9 to 12% are appalling. Perhaps they could have been used to shame the powerful into adequately funding minority education. STARS testing may temporarily make people feel good, but it allows politicians to suggest that there are no problems so that more education money is not needed.

I really didn't like Doug Christensen's act, and think he did a lot of damage to our academics here in Nebraska. I was glad when he abruptly quit to go and pout because he didn't get his way. Sheesh. He walked out on me once when I was trying to testify to the State Board of Education, and it was really, really disappointing, to be treated like that when I was only trying to be a good citizen and speak out on something I know a lot about. It's so inconvenient for many educational administrators to be faced with subject-matter experts who aren't buying what they're trying to sell. I was asked as an "outsider," or non-educator, to come in and score writing assessments by some students in late grade school. By that age, they should have been spelling and composing fairly well. But, oh, brother. Their work was practically unintelligible, and these were well-off, suburban kids. I saw first-hand how poorly they write, and yet I was "fired" as a (volunteer) scorer because my scores were so much lower (more reality-based) than those of the scorers hired by the State Ed Department, who think the kids are writing just ducky-fine, because they're PAID to score that way. That's why the numbers are so disparate between what Nebraska "tests" show, and what nationally-standardized "tests" show. It's all a shell game, and it's because of the slippery ability of people like Doug C. to escape reality-based accountability. People like Doug are why our schools have slipped so far in quality over the past generation. I have high hopes for the incoming ed commissioner in Nebraska, who I think is a far less B.S.-happy kind of guy. But this has really, really hurt the low-income and minority kids, and it's a crying shame.

Comments are now closed for this post.

Advertisement

Most Viewed On Teacher

Categories

Archives

Recent Comments