« Students Defend Public Education -- How about Teachers? | Main | Early Achievement Gaps Must Not be Ignored! »

Where is Data Driving Us?

| 12 Comments

Many of our schools these days are guided by a business school practice known as Data-Driven Decision-Making (DDDM). This approach means that we do not base decisions on whim or convenience, but rather rely on actual student achievement outcomes to guide us. The first step in this process is determining which data we actually care about. That key decision has been made for us in the public schools by the mandates of NCLB. The data that matter most are student test scores in language arts and math. Data-Driven Decision-Making means we then must make choices that will increase those scores.

The term "Data-Driven Decision-Making" has a sort of value-neutral, rational sound to it. It means we are basing our choices on facts, that we are willing to make tough choices in the interest of student achievement. That should be good news, right?

But the choices that are made actually do carry value judgments, and I am not sure that we are considering all the relevant data when we make these decisions.

A few weeks ago, a San Diego area teacher named Ellen posted the following comment to this blog. She wrote:

I can see the disparity on a daily basis, as the tight economy and the effects of NCLB with its relentless pursuit of annual "progress" narrow the scope of my students' education. Where once students had the opportunity to express themselves in art, music or organized sports, they are now forced into the straightjacket of language arts and math.
I am required to have a daily 2 1/2 hour language arts block (using a scripted program, no less) and a 1 1/2 hour math block. Science was recently added to this limited curriculum because it is now tested on the CSTs, but there are no hands-on experiments because of the time constraints and lack of equipment. Science consists of students reading from a textbook and answering multiple choice comprehension questions in a workbook.
Recently our school celebrated an increase in test scores, but teachers were castigated because the English Learner subgroup did not pass.

It appears that the program Ellen describes has, in fact, resulted in increased test scores in Language Arts and Math. Thus, according to the rules of Data-Driven Decision-Making, it should be considered a success. But Ellen's description has me wondering about the nature of the data we are relying upon, and what questions we might ask to uncover additional data.

1. The CST data we are using measures primarily language arts and math performance. What information might we be missing as a result of being driven by this narrow set of data?
2. What academic, cultural and physical activities were cut to make room for the daily regime of 150 minutes of Language Arts and 90 minutes of Math? Has this data been sought?
3. What might the impact of the elimination or reduction of hands-on science, history, art, music and PE be on the long-term success of our students? Has success in math and language arts come at the expense of future success in other subjects? Is this question being asked?

Scientists know that the data one collects depends on the questions we ask.
It is possible to go a long way down the wrong path if we focus on the narrow set of questions posed by NCLB.

The underlying issues behind my questions are ones of equity. These intensive language arts and math programs are motivated by concerns about inequitable outcomes for the mostly Latino and African American students attending the affected schools. But there are also equity concerns raised by the time taken that must come at the expense of other subject areas. It is the higher-performing schools that have more time for science, history, art, etc. because they are not obliged to spend 90 minutes a day on math and two and a half hours a day on language arts.

This raises a bigger question about Data-Driven Decision-Making in general. This term is used in a way that implies an objective, value-neutral focus on results. In fact, the data is so limited that the decisions one makes are constrained within a narrow range of options. The data gathered carries a set of values that have been determined, in this case by priorities set by NCLB.

Another business school term is "opportunity cost."
There is no free lunch. If our school communities wish to make student achievement on math and language arts our highest priority - over science, history, art, PE, and music, then at the very least we need to be aware of what we are giving up in the bargain. I have a problem with policymakers and school leaders making this decision without acknowledging that there are in fact choices being made, and that there are tradeoffs, and that these are decisions that have values embedded within them. I have a problem when we do not even measure the impact these choices are making in other areas, and then we act as if our decisions are driven by pure data.

What do you think? What data is missing from the decisions being made at our schools?

12 Comments

What data is missing? The human factor. The absolute fact that all students are human beings and individuals. This is the crisis we are moving toward in education - have you seen 'Gattaca'? It's not always about academics...

You're absolutely right that this is an equity issue. The myopic focus on test scores has been exacerbated by budget cuts, so that the imperative of test scores remains, but the slim opportunities to enhance the tested-curriculum are disappearing from our schools that lack local resources. So, in some districts, wealthy and privileged children still get music, art, science, field trips. These children are stimulated by school, see a purpose in their education, and learn reading and math with some sense that these skills are part of an integrated body of knowledge that will help them get an education and a job like Mom and Dad. Meanwhile, in schools and districts that can't rely on supplemental donations from well-heeled foundations, students see a narrow curriculum, severely lacking a broad, interdisciplinary scope and opportunities to see the value of learning reading and math. All that matters is that they raise their school and subgroup average scores. They burn out and drop out, or, if they're lucky, go on to universities where they have the great opportunity to take on huge debts for a shot at jobs that will barely allow them to make ends meet while they pay off those debts.

I agree there is a cost to this narrow-approach to education. We have educated a generation of students who can read to perform on a test, but cannot comprehend a college level text, where they will required to read, analyze, synthesize, and reflect on their own. I don't think it is a coincidence that the push has been on Language Arts and Math, either. These are the scores that colleges look at on SAT tests. Writing is rarely taken into consideration; personal opinions which are substantiated are not on the agenda. The message being sent is clear-if you can look at information or a piece of text and spit it back, then you are a successful reader. That is what we have equipped our students to do. Somewhere along the line this has to stop. We are doing future generations of learners a huge disservice by continuing this trend....looking at the data should mean reviewing what is really happening in our classrooms, not just on test-day(s). We must ask ourselves, "How well equipped to handle real-world reading are our students?" You need only look at the number of students who come out of high school, and place into developmental/remedial Reading classes at local community colleges. There's a number that should really bring about change....

Laura makes several important points. We have been completely sold on the idea that the purpose of K-12 education is preparation for admission to college. And since the gates to college open with the right combination of grades and test scores, that has become our dominant focus.

But college is rapidly rising out of reach of even the middle class. Students who took out $100,000 loans a few years ago are finding it impossible to stay afloat.

Once we thought a four-year degree was a ticket to the good life. Now it seems we need to re-think that approach. The middle class is on the ropes, and it will take a lot more than school improvement -- whatever that means -- to get our society back on track.

I couldn't agree more. In the real world, my friends who are mechanics, firefighters, and electricians are currently earning sky-high incomes while I watch increasing numbers of college-educated friends be laid off.

A narrow curriculum that rewards a finite set of skills will glut the job market in certain fields. The people who excel will be creative thinkers with good problem-solving and interpersonal skills--exactly the capabilities we can't quantify by standardized testing.

The data that is missing from the decisions being made in our schools is this: No one is asking the question "Is this what is right for kids?"

I am VERY fortunate to work in a district where we DO ask that question. We are NOT a wealthy district, and are struggling like everyone else to reduce our expenditures in the face of California's 'moving-target-budget-cuts'. Yet, we will not cut music, art, field trips or athletics from our schools.
Our teachers...our classified staff...our administration...our community...we all look at the test score data, and then make our decisions based on what we feel is right, guided by our core values and beliefs about what is vital to the education of the whole child.

"Scientists know that the data one collects depends on the questions we ask."

But the data also depends on how we ask the questions.

NCLB mostly uses multiple-choice tests scored by just counting right marks. This rewards the lowest levels of thinking: rote and guessing.

When multiple-choice tests are scored for judgment, as well as for knowledge, students are rewarded for using all levels of thinking. They are rewarded for taking the responsibility for reporting what they know rather than mark and trust to luck.

Schooling is directed at producing self-correcting, self-motivated achievers rather than negative passive pupils.

I have worked with over 3000 students turning under-prepared college freshmen into successful students. This change from external to internal motivation and correction should take place before high school. Knowledge and Judgment Scoring made the change in classes of 12 to 120.

Free software is available for any teacher to do this in their classroom (http://www.nine-patch.com). When both quantity and quality are assessed, high quality students can trust the foundation they have as a basis for future learning. The information teachers need is available from knowledge and judgment scoring.

The emphasis of NCLB testing on multiple-guess testing is self-defeating. It robs students of quality instructional time and restricts the curriculum and student development. It is the wrong way to question.

Congratulations to LisaMarie for having a school that puts students first. I am curious, however, about how your school has this freedom? In my district, many of our schools are in their fourth or fifth year of low-performing status, and face possible closure if scores do not rise significantly. That creates a powerful pressure to implement intensive programs focused on boosting those scores -- with the narrowing effects I described. How can your school avoid these consequences?

I teach kindergarten and daily see the 'abuse' the NCLB standards are inflicting on the youngest of victims. With already high test scores, my school district has gone to the extreme in focusing on reading. All kindergartners are expected to be proficient readers by the end of kindergarten (DRA 4-6). Just last year, I had a student who read at a level 14 but would have a 'melt-down' everytime she tried to put her coat on by herself. She had NO self-help skills and often became very frustrated. The day I witnessed the first melt-down, I went home and cried, too.....for MY PART in no-longer teaching the whole child.

This is such a large issue worldwide. In Australia we have been doing exactly the same thing. There is a slight difference, perhaps. English and Mathematics are being used as 'vehicles' to improve teaching and learning practices and the data is used to evaluate the effectiveness of professional development and changes in practice. Still, at the end of the day, for schools to demonstrate improvement, they have needed to push money into these areas for physical and personnel resources at the expense of other areas, experiences and opportunities for students. We are about trying to grow the whole child - physical, social, emotional and academic... I too wonder if, in some cases, the right questions are being asked at the right times. Anyone can manipulate numbers, but there is a lot more at stake than just performance-based funding.
Unfortunately, wherever you may be, education is largely a political game. The rules are set and changed at whim by those in power who don't always understand our job or the nature of the students we work with. They too, want data for the sake of impressing their voting constituents to show how they've made a difference in the short term; forgetting the long term implications of today's child becoming effective contributing members of society tomorrow.

Kudos for discussing uses of empirical data in making schooling decisions. I agree with your apparent inference that these data formalize part of observations teachers have made informally for eons as we monitor and adjust instruction in order to increase student learning. That's good; formal data makes our job easier, so we can attend more closely to what these data do not monitor. Thanks for bringing up the topic.

An aside: states, not the Federal government, establish and implement tests in order to demonstrate that they use Federal (NCLB) funds appropriately.

"Data-driven" decision making is not driving us to a place that is very professional. I know of no profession that kicks the knowledge, experience, and disposition of the professional and says that these things don't matter and only data matters. Professionals make judgments and make decisions informed by data but not "driven." There is no leadership in enterprises "driven" by anything other than their core mission and core work and creating data are not our core work and mission, teaching and learning is.

Comments are now closed for this post.

Advertisement

Most Viewed On Teacher

Categories

Archives

Recent Comments