What Are the Best Ways to Measure Student Learning?

Bill Farmer

Learning is an infinitely complex process, yet as a society we seem determined to relegate learning to a single letter grade or percentage score aimed at sorting and ranking students. There can be some useful information captured by a well-written high stakes exam. It provides a snapshot of isolated and specific aspects of a student's learning at a given point in time.

Unfortunately, the elevated emphasis placed on these types of assessments, several of which are poorly constructed, oversimplifies the intricacies involved in truly documenting and understanding a student's learning process. It is difficult to ignore the impact that this has on a student's perception about learning.

In the high school setting where I teach, students appear to be conditioned to focus on the quantified end result. At the beginning of the year, I find myself having to counteract this tendency of students to only pay attention to the total number of points earned on an assessment. I accomplish this by using a detailed rubric void of any point values to "grade" their first formal lab report. Initially, students are extremely uncomfortable with the fact that the rubric doesn't have a summative grade attached to it. Eventually I am able to direct their attention to the important information that the rubric does provide, which is how they can make their lab report better.

We cannot lose sight of the essential reasons why measuring student learning is so important. Not only does it supply the teacher with valuable, personalized data to inform instruction, but assessments also should be constructed in such a way as to provide students with the critical feedback necessary to guide their continued learning. With all of this in mind, educators should be taking the lead in the development of effective tools to assess and promote student learning.

Posted by Bill Farmer at 11:53 AM | Permalink | Leave a comment | Recommend

Jessica Hahn

I believe that reading testing is extremely problematic. For example, kindergarten through 2nd grade students in my school have historically read on or above grade level, as measured by the Fountas and Pinnell benchmark assessment. This assessment relies on running records and miscue analysis. And yet, when these same students get to 3rd grade, their reading scores on the state standardized test are much lower.

Are we to assume that these students can't read? The government sure does. And then schools do. They assume that they are doing something wrong. They assume that their teachers aren't teaching literacy properly.

Why don't we question the assessment itself? Is it measuring what we value about reading? Do our kids just not know the genre of test taking? Or are the scores indicative of a learning problem?

It's time that educators stick up for ourselves. It's time we say: "We know we have work to do. We know we can better serve our students. BUT, we are teaching them to read and we do have the data that show they are learning."

In this light, one of the comments on Rebecca's post really stands out. Karl Wheatley from Cleveland State University suggests that standardized tests only have the power and privilege we give them.

Therefore, I believe we have the ability to TAKE AWAY their power and privilege.

Which lends itself to a greater conversation. Since the 1950's our government has committed itself to maintaining excellence and equity in our public schools. In its effort to do so it has chosen standardized testing as the measure. I believe it has done so because it sees schooling and teaching as a science, much in the sense that Karl, in his post, compares measuring learning to measuring coffee or lumber. I also think the government has chosen standardized testing because standardized tests are easier to administer, collect, and grade.

Is the government ready to accept the complexity of measuring student learning and the variety of tools that come along with it?

Posted by Jessica Hahn at 10:20 AM | Permalink | Leave a comment | Recommend

Sarah Henchey

Commenter Megan Allen asked, "How did your PLC decide on these particular steps? What advice do you have for other PLCs to 'get their legs' and get started on this journey?"

The evolution of my PLC has been a process. This is only our second year working together, but each year has strengthened our instruction.

Last year, we created essential learning outcomes for our content based on our state's curriculum. To establish these, we asked ourselves, "What does a 6th grader need to know and be able to do by the end of the year?" We started out with quite a wish list but finally decided on 10 skills and abilities students needed to take away from our classes.

We referred back to these outcomes as we designed daily learning goals and created lessons. This was our first step in determining how we would assess student success.

Through our reflections, we identified obstacles that prevented students from achieving mastery. Key trends included lack of proficiency in academic vocabulary, limited exposure to texts, and lack of opportunities to practice skills. We then addressed these challenges through logical, authentic means. For example, we adapted and implemented Donalyn Miller's 40 Book Challenge to increase students' familiarity with and exposure to a variety of texts.

As we reflected on students' needs, we discovered how to better plan for and assess student learning.

I recommend PLCs start to have reflective conversations about what they want students to take away from their course—what's truly essential? Then, move those conversations toward mini-steps: What skill(s) can be integrated into this current unit? What will students need to know and be able to do? How will you assess mastery?

As you build a shared vision, you will be forced to become more vulnerable and transparent about your practice. My PLC found our dialogue naturally shifted, with student learning becoming the true focus of every conversation.

My PLC knows we're taking a risk with our approach. We anticipate that we'll "cover" less material this year. As a result, students may encounter a less familiar phrase or skill on the end-of-year tests. But we've devoted ourselves to assessing students based off what we know is essential to their success. We're confident in our choices and will continue to prioritize the data we gain from students every day, not just at the end of the year.

Posted by Sarah Henchey at 9:42 AM | Permalink | Leave a comment | Recommend

Marsha Ratzel

Summative assessments at the end of the unit are important too, providing me with a reliable picture of how well students have mastered the unit's objectives.
Frankly, I think that multiple-choice questions are insufficient as end-of-unit measurement tools. I do use multiple-choice tests to make sure students know the basic facts, but these are only a precursor to more authentic summative assessments. In my science classroom, I learn the most about my students' learning when they demonstrate their understanding of an idea by sharing a model they've created or by presenting their lab results to the classroom.

One of the biggest challenges facing teachers is how to create assessments that resemble how professionals are performing relevant tasks in their work today. As a science teacher, I am continually on the prowl to learn more about how scientists work and communicate their findings to colleagues. Then I try to mimic that work in a scaled-down version in my classroom. For example, instead of typewritten lab reports, I now require lab groups to share their data via a Google Docs spreadsheet and a presentation that is run like scientists' lab meetings: Students respond to comments and moderate the discussion about their findings. Going beyond a lab report lets students demonstrate their abilities to communicate and defend their ideas to many people (not just the teacher). This kind of assessment also models how many people now do their work: creating a shared document that is authored by many people and that is not bound to any one machine or any one user.

Collaborating with colleagues helps me become a better teacher—and I think it could help me build even better assessments. If schools and districts built communities of practice, teachers could have the time and opportunity to dialogue with colleagues about our effectiveness in creating realistic assessments, matching the way we assess to the learning goal, and in giving feedback that helps students know where they stand

Posted by Marsha Ratzel at 9:25 AM | Permalink | Leave a comment | Recommend

Dan Brown

The structure is great; the opportunity to talk with a horizontal cohort of educators about students and what's going on in class is invaluable.

The problem is the data. It didn't jive with the achievement I'm seeing in class.

Many of my students who have surged in recent weeks absolutely tanked the test. Their multiple-choice scores are far below the caliber of achievement I've seen from them in class discussions, projects, and homework assignments. On the essay section, many were confused by the prompt and their writing veered way off-topic. A lot of these students have built their self-advocacy skills and know how to reach out to find clarification—but that's not allowed on tests the way it is in real life.

Timed tests are only a tiny part of success in college and beyond. The ability to work within deadlines is certainly a vital skill, but the highly pressurized write-on-demand Testing Conditions environment created in schools does not build or assess that. Taking high-stakes exams is a game with its own rules that barely—if at all—transfers any useful skills to adult life.

On data days, the test scores can be used only as a jumping off point for discussion; they are limited, even misleading measurements of student achievement.

Looking carefully at visible learning is important. Looking at multiple-choice scores produced under high-stress conditions is not.

Posted by Dan Brown at 9:00 AM | Permalink | Leave a comment | Recommend

Rebecca Schmidt

However, I have to disagree with the position that we discard assessment altogether. I view my students' reading and math abilities as their human rights—and my job is to empower them to be productive changemakers in their communities by accessing these rights. Without assessment, I would not know if my teaching strategies were effective. I want to be clear that I am not dismissing reading assessments altogether—nor do I place much value in most of them. Instead, I would like meaningful assessments for monitoring students' learning through the year to formally indicate progress in the eyes of D.C. Public Schools and the rest of the country.

For example, at the beginning of the year we test our students' reading ability using DIBELS (fluency; how many words a student can read in one minute) and TRC (a combination of fluency and comprehension), which results in a reading level based on Fountas and Pinnell's system. We continue to collect these data throughout the school year—more frequently for the students below grade level, less so for those who are at or above grade level—and use this information to plan our teaching with guided reading groups and lessons based on the Continuum of Literacy Learning. My students celebrate their progress—and they make great progress—but unfortunately none of it "counts" outside of our classroom.

A far more helpful reading-assessment method would be a measure of progress from beginning to end of year, such as the TRC tests, or a test I've heard about (but never seen in person) called the MAP: Measures of Academic Progress. Similar to the GRE, the MAP is computerized and adapts to a student's answers as they take the test by giving harder questions as the student answers correctly and easier ones if the student answers incorrectly.

If our children are to be considered leaders in their community, country, and throughout the rest of the world, we educators have a critical job to ensure they are developing literacy and numeracy skills and knowledge. We can't waste our time (or our students') with meaningless assessment.

Posted by Rebecca Schmidt at 8:05 AM | Permalink | Leave a comment | Recommend

Ryan Kinser

I suspect thousands of teachers feel like Bill Ferriter, who in his recent blog entry, "Is Real Formative Assessment Even Possible?" describes the utter exhaustion that comes from constantly assessing hundreds of students and then improving his teaching strategies. Add in the learner traits that are difficult to quantify like those three C's I mentioned in my previous post (creativity, communication, collaboration), and many of our current tests start to look like an ugly fourth C: convenience.

So, how do we eschew the convenience of superficial, data-driven tests for efficient, yet purposeful assessment and feedback?

We make the process transparent and co-owned by students.

Spending time early in the year deconstructing rubrics into student-friendly language, asking students to propose other important benchmarks, and modeling exemplary performance, gains precious instructional time in the long run. Add prewritten feedback opposite those benchmarks whenever possible, and you have transparent, reusable data chat templates for students. Anticipating your comments helps students predict common mistakes and assess themselves and each other during the process.

Taking this a step further, why not invite students to focus on a few specific benchmarks within a larger set of objectives? In my student-led conferences, I ask students to select one or two benchmarks they want to discuss. Then I do the same. For those youngsters who need more guidance, offer conference starter sentences for each objective. Students come to the table aware they'll be graded on all the objectives but they'll get my detailed personal feedback on the area they need the most. We compare notes, and then students are required to write summaries of my comments in comparison to their own. You might even consider recording narrative summaries of these discussions or of your feedback rather than handing back lots of written comments. These digital files are great for parents to gain perspective on the interactive process in your classroom.

By focusing on creating clearly defined expectations, replicating detailed rubrics, and eliciting student self-assessment when appropriate, teachers can hone the art of assessing the smaller daily interactions between our students and us. This is where the real learning takes place.

Posted by Ryan Kinser at 7:02 AM | Permalink | Leave a comment | Recommend

Rebecca Schmidt

Assessing student learning and progress each year is important: All students should have the same opportunities to learn and succeed and we educators need to know if our teaching is helping our students develop needed knowledge and skills. Standardized assessment is one (albeit imperfect) way to help us fine-tune our practice, and hold schools and teachers accountable for student learning. However, DC-CAS (and four benchmark tests before April) have my students reading, calculating, and bubbling circles for over 16 days each year—almost a month of school.

With so much time devoted to these tests, and so much riding on them, it's a shame they paint a grossly incomplete picture of student success in our classrooms. The DC-CAS and benchmark tests are helpful and illustrative in math, yet are not true measures of student achievement or teacher effectiveness in reading. Math—being a collection of discrete skills and explicit processes used to arrive at specific answers—lends itself to measurement using the DC-CAS. I can tell if my students struggle with geometry but not measurement, for example. Reading is more holistic. More helpful to examine is our students' reading progress through the year—doing so gives a clearer picture of student learning and teacher effectiveness.

Like many in DC, my 3rd grade class entered with a range of reading abilities. Students read on level A (pre-readers) to above T (5th grade). Even with 1.5 years of growth within a year, a student entering at level C (end of Kindergarten) would only be level K (mid 2nd grade) by the end of the year. That's tons of progress, but not close enough to the end of 3rd grade reading level (P) required for even a chance of success on the DC-CAS.

It's difficult for teachers to celebrate success or respond to failure on DC-CAS or benchmark reading tests. Unlike the math tests, the standardized reading tests don't tell us what reading skills our students have mastered or not. They merely confirm what we already know about our students: Those who can read on a 3rd grade level do well, those that can't, don't. I already know that information, without DC-CAS, benchmark tests, or missing recess.

Our time in school is precious. Student assessment is crucial to improving teaching and learning, but only when the results are relevant and useful to teachers and learners.

Rebecca Schmidt is in her fifth year of teaching in D.C. Public Schools.

Posted by Rebecca Schmidt at 3:20 PM | Permalink | Leave a comment | Recommend

Dan Brown

In fact, Hattie's exhaustive research into what quality teaching and learning looks like is the best road map I've ever seen for how to get the most out of students. I hope my two-year-old daughter's future teachers have read it; if not I just might be foisting copies on them at Back to School Night.

Standardized tests aren't nearly good enough to measure student learning. Sadly, the outsized emphasis on them in many schools is a destructive force. So what does real learning look like and how can we measure it?

In a powerful conclusion to his chapter on "The Argument [for Visible Teachng and Visible Learning]," Hattie writes: "The teacher needs to provide direction and re-direction in terms of the content being understood and thus maximize the power of feedback ... It [visible learning] also requires a commitment to seeking further challenges (for the teacher and the student)—and herein lies a major link between challenge and feedback, two of the essential ingredients of learning. The greater the challenge, the higher the probability that one seeks and needs feedback ..."

In my high school English classroom in Washington, D.C., I try to bring to life worthwhile challenges and quality feedback in long-term student-directed projects. For example, each year my 12th grade students read a Shakespeare play (currently we're exploring Much Ado About Nothing) and then work in groups to develop an alternate design concept for a new staging of the play. They need to transplant the script of Much Ado About Nothing from old Messina to a new time and place—one that emphasizes or dramatizes key themes in the play. Aided by research, they design appropriate new costumes, sets, and sound cues. Then they organize and offer their visions in written and oral presentations. The class chooses one group's design concept to use for a whole-class production of Much Ado. Each student takes on roles in the cast and crew. They rehearse, become experts on their characters, compose reviews of professional performances, write analytical essays on the text, maintain a reflective journal of their participation, and ultimately perform in public. The curriculum, titled Text Alive! and developed by the Shakespeare Theatre Company, builds in great challenges and constant opportunities for feedback, nudging students to "become their own teachers"—a crucial goal that Hattie identifies.

The multifaceted performance-based Shakespeare project leaves behind a trail of visible learning. It's messier than a bubble test, but true learning should be. As we prepare students for a knowledge-based professional sector, these are the types of tasks that an expert like John Hattie—and common sense—would celebrate.

Dan Brown teaches high school English in Washington, D.C.

Posted by Dan Brown at 2:30 PM | Permalink | Leave a comment | Recommend

Bill Farmer

The discussion of student growth by mainstream media outlets tends to give the impression that this area needs greater emphasis in our public education system. States across the country, fueled by Race to the Top money, have been initiating a strong push towards the systematization of assessment tools for the purpose of efficiently measuring student growth and linking it to teacher accountability through their evaluation.

From the perspective of those of us who reside within the classroom, the purpose envisioned by legislators for measuring student learning seems slightly misguided. As teachers, we monitor learning by applying various diagnostic tools to inform and guide our instruction. Our end goal is developing an understanding of the individualized needs of our students to enable us to make professional judgments that will further their learning.

Measuring learning is an immensely complex objective that likely cannot be accomplished by a single state test or even a well-designed series of traditional standardized assessments. Learning is multifaceted in its composition and is very personalized. Therefore it is extremely difficult to develop a one-size-fits-all approach to measure it.

Learning often also varies by discipline. Science education, for example, is in the midst of a transformation to adjust to the 21st century digital learner. Immediate access to the library of science knowledge enabled by technology has eliminated the need for a curriculum largely centered on the memorization of facts. Instead courses are now focusing on a student's ability to utilize their knowledge to practice science and think critically about scientific issues.

In my biology classroom, multiple and frequent forms of assessment are required to get a comprehensive idea of the learning that is occurring. Essentially every interaction I have with a student can provide informal or formal data that provides valuable information. Several instructional strategies have proven to be particularly useful. Each unit begins with critical thinking questions requiring students to access and apply prior knowledge. Their responses on these pre-assessments provide me with a baseline of their initial understanding. Various informal dip-stick assessments, such as remote clicker responders and opener questions, are utilized to gauge student growth, identify misconceptions, and plan future instruction.

In addition, performance based assessments or inquiry lab reports are effective at providing an in-depth analysis of student learning. By using a consistent detailed rubric for these types of assessments, I am able to provide valuable feedback to the student. I am also able to monitor how their performance on these rubrics improves over time which provides clear evidence of the complex learning that is taking place.

Bill Farmer has been teaching biology and chemistry for nine years at Evanston Township High School in Evanston, IL.

Posted by Bill Farmer at 2:30 PM | Permalink | Leave a comment | Recommend

Ryan Kinser

Now I'm a little older, a smidge more veteran, and my values haven't changed. But I have a confession: Now I think we should teach to the tests. We just need to construct more useful ones—curriculum-embedded performance assessments that are valid, reliable, and accurate measures of what and how students learned. I envision these tests as both formative and summative at the same time. They show student mastery, while helping teachers tailor future instruction. I can still remember my Advanced Placement American History teacher constantly refining his pedagogy despite the daunting breadth of knowledge we seniors needed for the AP exam. He returned to what I now know were Socratic Seminars discussing the Missouri Compromise even after we protested that we could explain the event.

"Yes, but few of you can argue whether or not it was a good decision," he snapped, "if it was truly a compromise, or what alternative solutions might have existed."

In order to reflect 21st Century college- and career-readiness skills, I think we should teach to assessments that capture authentic student voice through what my colleague Barnett Berry refers to as the 3 Cs: creativity, communication, and collaboration. All three of these traits require teachers to assess relevant processes, not just products. I offer students choices not on the content but the method of many assessments. Portfolios, student reflections, conferences, and multimedia project logs help me drive achievement much more than chapter tests. I evaluate my kids on their metacognitive steps, their thinking about my teaching strategies and the students' own learning. I want to know, how did students prioritize and delegate responsibilities? What obstacles did they encounter as teams and as individuals? What further questions did they discover?

The millennium my own teachers dangled as a not-so-distant threat is here. Success in today's world demands students know so much more than content. Students must now demonstrate that they know how to learn and build upon that content to solve problems. They must develop versatile communication skills, work collaboratively and competitively, and be comfortable reinventing themselves over multiple careers. Give me tests that are flexible and transparent, and I'll teach to them.

Ryan Kinser is a 6th grade English teacher at Walker Middle Magnet School for International Studies in Tampa.

Posted by Ryan Kinser at 2:10 PM | Permalink | Leave a comment | Recommend

Jessica Hahn

I actually like our district-wide benchmark math tests for 1st grade. I think they are an example of an effective measurement tool. These tests measure student learning in a very complex, deep way. There are no multiple-choice questions. Students must explain their thinking. And most importantly, these tests ask students to show what they have learned in a variety of ways. The complexity of the benchmark tests holds me accountable for making sure that I am teaching my students in a rigorous way.

While my school does support me in and encourage me to measure student learning in all the ways I have described, they ultimately measure student learning in a very narrow way: the state's standardized test. Currently, our government values standardized testing as showing mastery and therefore so do our schools. I see the tension between what kids know and how they are asked to show it in my own 1st grade classroom. I believe that when my kids read authentic literature, and can summarize, infer, and draw conclusions about the text and its characters that they are showing growth. And I believe that measure of reading comprehension is more meaningful and complex than a series of questions about a decontextualized passage.

Do we need citizens who can answer questions about a passage or citizens who can think and talk critically about text and events? If we do value the latter—if that's the kind of thinking that the 21st century begs—then we are prioritizing tests that don't align to that vision. If we want schools to change the way they measure learning, we as a government and a society need to change what we value about what students know and how they show it.

Jessica Hahn has taught elementary grade children for six years in Phoenix and New York City.

Posted by Jessica Hahn at 12:15 PM | Permalink | Leave a comment | Recommend

Marsha Ratzel

Similarly, each student has a complex set of individual needs—emotional and intellectual—that I must understand to build the trust that will make learning possible. Knowing a student's unique characteristics helps me design instructional strategies that will motivate and engage them. And I have to connect the pieces. In a classroom of 31 kids, when one-on-one teaching just isn't possible, I must create small groups that let me deliver targeted mini-lessons based on students' needs.

When I start a unit, I create a clear picture of what students should be able to know and able to do by the end of the unit. I keep that picture at the forefront of my mind. It drives every instructional decision I make, helping me use time wisely and show students what is expected of them. This helps students—and me—track our individual and aggregate processes. It gives us specific things to talk about and guides my coaching of them throughout their learning journey.

An initial assessment tells me where my students are. That way, I can tailor how I teach, expediting the pacing if they come with lots of background knowledge. This initial assessment establishes the partnership I build with students. They have to know that we'll do this together and we're a team. It isn't graded. In my classroom, learning is not about me pouring knowledge into students' heads—it's about a journey we'll take together.

Throughout the unit, I use formative tests, quick measurements of discrete knowledge or skills.

For example, I recently asked students to draw a diagram of how energy is transmitted from the sun, and hand it to me as they exited. I thumbed through their index cards to see if they were visualizing the big ideas of heat transfer. The cards offered immediate information that I needed to design the next day's lesson. Half my students understood the basic concepts, so the next day, they used temperature probes, a globe and some graphing software to measure differences in how the sun heats different parts of the Earth. Meanwhile, I worked with the other half of the class, using a flashlight and globe and asking students to verbalize what they saw. The students and I coached one another: If someone stumbled on vocabulary, we interjected the proper word or offered hints. Using formative assessments this way helps me deliver learning possibilities that match what students are ready to learn.

Of course, summative assessments (at the end of a unit) are also important. In my next post, I'll explain how I handle these.

Marsha Ratzel is a National Board-certified teacher in the Blue Valley School District in Kansas, where she teaches middle school math and science.

Posted by Marsha Ratzel at 11:46 AM | Permalink | Leave a comment | Recommend

Sarah Henchey

Somehow, in my first year as an English language arts teacher, I lost sight of the role assessment plays and thought my "grading" could wait until the weekend. This led to many moments of frustration as I could not accurately gauge my students' needs.

Thankfully, as I gained experience, I finally understood what my college professors had tried to impart—assessment must be a priority in teaching.

Like all teachers, I use a variety of tools to assess what students know and are able to do. I gain the most valuable feedback during my daily lessons and interactions with students. I learn a tremendous amount from puzzled expressions, stop-and-jots, and post-it notes passed to me as I circulate around the room. I frequently use "cold calls" to check for understanding and find exit slips invaluable as I modify and adjust to students' needs.

When planning summative assessments, I've learned to be purposeful and proactive. This year, my professional learning community decided to begin the year with three mini-units focused around the skills students would need in order to be successful.

We used the following steps to plan for each unit:
1. Identify key vocabulary, skills, and goals.
2. Create an assessment that would demonstrate mastery.
3. Use the assessment to construct targeted lessons.
4. Assess students on their ability to apply their learning.
5. Analyze results and determine interventions.
6. Allot an additional week for intervention and enrichment lessons.

We've found the last step to be crucial. During this week, we provide small-group restructuring based on students' needs and then reassess. Meanwhile, students who demonstrated mastery participate in authentic enrichment activities.

As I've come to understand and appreciate the role assessment plays in student learning, I've been forced to re-prioritize. To truly understand the needs of my students, it's essential that I remain focused on their responses and questions when I'm teaching. I also have to make time each day to regularly evaluate more formal assignments (homework, quizzes, etc.) and determine the degree to which students understand and can apply what we're learning.

Some days, it's tempting to return to my old ways. But I know the difference it makes in my teaching when I understand where my students currently are and where we're going. For this reason, assessment has to be my priority.

Sarah Henchey is a 6th grade language arts teacher in Orange County, NC.

Posted by Sarah Henchey at 9:37 AM | Permalink | Leave a comment | Recommend