Multiple Measures: The Story Data Doesn't Always Show
Learning is so nuanced and often hard to measure. We try to simplify how we do it for the sake of efficiency and consistency, but in doing so, we often subtract the moments where student learning is evident but less easy to codify. When we make these sweeping choices, the students who need us to notice most, lose.
The students who know how to play school will figure out how to game the equation. What are teachers looking for? Where are the measures that "count"? How can they maximize the assessments without having to exert any real effort to lifelong learning?
Unfortunately, we have too many students who don't know the game we are playing, have more pressing issues than learning the game, or have no vested interested in conforming to the game. These kids lose regardless of how smart or curious they are in most traditional school settings.
Although data is supposed to be the objective indicator that helps us measure what kids know and can do, that objective indicator is just as flawed as all of the other depending on the context.
Exams, teacher-made or standardized, only measure student output in a controlled setting for one moment in time. Although it may measure what a kid knows, it may also measure how much effort a child decided he or she wanted to show. If a child has parents who are going through a divorce, and walks out of the house to get on a bus after witnessing a nasty fight between those parents, the likelihood of that child being able to concentrate or even care about a test first period is low.
And how many other stories can you think of that would skew the data in any circumstance? Life happens and we don't often account for that in our measures.
So what are we actually measuring in this instance?
Adam Dyche, Social Studies Department Chair of Waubonsie Valley High School, said, "To paraphrase @rickwormeli2, it's really about the common evidence, not a common method. So it bears true that multiple measures bring validity to that evidence. And exposing students to various types of measures supports self-discovery of those methods that work & those that need work."
As we consider the best ways to measure student learning and continue to ask teachers to gather data, it's important to also discuss what kinds of data to collect and how. Data does NOT just mean test scores.
Data can encompass any or all of the following (and more):
- status of the class collection during class activities
- drafts of a writing assignment done over time
- a project completed for one class period or for many
- the formative conversation being had during a class lesson when we provide feedback on the spot
- the conversation we have with a student when we notice something isn't right
- tests presented by the teacher
- tests that are state or nationally normed
- observations made during the day both in and out of our classes
- conference conversations around student portfolios
- student reflections and self-assessments
- participation in small group work
- participation in whole class discussion
- questions students ask for further clarification
- any kind of assessment that provides us with the understanding of what a student knows that is done in front of us
- conversations with our colleagues who share the students with us and their performance in more than one space
As we continue to talk about data and the necessity of it while we are making choices about programs and students, we need to remember that each child's story is what makes the data valuable.
Andy Hatton, a director of academic affairs, said, "Evidence of learning should be visible; ideally able to be duplicated by the learner over time; clearly depends on the purpose of assessment but application to new situations is nice."
Teacher Adam Yankay said, "Using data from standards-based grading, I get more direct feedback on what my students can and cannot do after discovery and practice. I use this data to form spiraled homework, re-assessment, and class remediation."
Different educators use data differently, and therefore trying to find one measure that does an adequate job of tracking or showing learning of any particular child is challenging and gray at best. Especially when each child is equally unique as well.
Systematically, we need to think about our desire for "magic bullets" that don't exist. Instead, we have to remember that we are in the business of developing lifelong learners and that for each one of them, the path is different.
What kinds of multiple measures to do you use to ensure that the whole child is being assessed and provided with the opportunities or scaffolds needed to achieve the highest level of success? Please share.