This is the last Sputnik blog post to appear on Education Week. Beginning next week, I'll bring my discussion of evidence-based reform in education to the Huffington Post. I hope you'll continue to take part in the conversation.

The original idea of Sputnik was to spark interest in the idea that progress in education must come from improving and augmenting proven, replicable programs and practices, and then putting proven programs into policy. This simple and obvious notion, "use what works," has been a long time coming, and still has a long way to go. In Sputnik I've commented on some of the most important developments in evidence-based reform: i3, the OMB embrace of evidence-based reform, proposals for ARPA-Ed, evidence-based reform in fields other than education (including dairy cows!), and many developments in research, development, and methodology. This is an exciting time for evidence-based reform, and if you have enjoyed reading Sputnik as much as I've enjoyed writing it, I'm sure you'll equally enjoy reading and reacting to my blog at Huffington Post. I look forward to hearing from you.

I'd like to thank Lauren Gibbs, Beth Comstock, Robert Johnston, Debra Carroll, and the editors at Education Week for their help with Sputnik.

Posted by Robert Slavin at 8:05 AM | Permalink | Leave a comment | Recommend

In a wonderful article in the Washington Post on December 7, Dylan Matthews makes the altogether rational argument that Congress should routinely authorize evaluations of programs it is thinking of signing into law:

"In a perfect world, (reporting on policy) would be a kind of science reporting. Just as my colleagues at the health desk often explain which medicines are effective and which are a bust, I'd ideally be able to describe what sociologists, economists, and political scientists have discovered about which policies work."

Followers of Sputnik will not be surprised to hear that I strongly support this position, and am delighted to see it expressed in the Post, especially after a recent Op Ed by Jon Baron in the New York Times made similar arguments. But I did want to build a bit on Matthews' argument. In fact, there is often evidence on whether a given program is likely to work, but Congress routinely ignores it. It may be that the evidence is mixed or difficult to understand. In other cases, the evidence is clear but for political or ideological reasons it does not matter.

Further, the vast majority of government funding goes into programs that already exist, and are so politically popular that they are sure to exist for a long time. In education, the classic example is Title I, a $15 billion program that has provided funding to disadvantaged schools since the Johnson administration.

So here is my friendly amendment to Matthews' article: In addition to testing out initiatives that Congress is considering, let's also proactively test out innovative ways of using the money that government already spends. For example, let's fund the creation, evaluation, and dissemination of programs to be used under funding from Title I, or IDEA, or other long-established programs certain to continue in some form. This is a goal of the federal Investing in Innovation (i3) program, of course, which is funding development, evaluation, and scale-up of proven programs, but at $150 million annually, i3 is only 1% of Title I funding and a trivial proportion of all education funding.

Building up a strong, well-validated set of solutions to the enduring problems of education is the best investment Congress could make in education, and the current Congress and administration have gone further in this direction than any before them. Yet there remains a lot of work to do to ensure that the best programs and practices become available to all children, especially those in need.

Sputnik is moving! Follow me to the Huffington Post, where I'll be continuing to write about evidence-based reform in education. See you there! And don't forget to follow me on Facebook and Twitter.

Posted by Robert Slavin at 8:00 AM | Permalink | Leave a comment | Recommend

Note: This is a guest post by Steven M. Ross, Professor in the Center for Research and Reform in Education at Johns Hopkins University. The post originally appeared on the Bush Center Blog.

Evaluating the effectiveness of school principals is in everyone's best interests--students, teachers, parents, and arguably principals most of all. But what are fair and valid measures of success? Unfortunately, past practices in defining and evaluating principals raise concerns about consistency and fairness. A few months ago, the National Association of Elementary School Principals and the National Association of Secondary School Principals released a report, "Rethinking Principal Evaluation: A New Paradigm Informed by Research and Practice," which I co-authored with Mathew Clifford. We hope that this report stimulates thinking at the federal, state, and district levels about using principal evaluations to support two essential functions: (a) judging principals' effectiveness and (b) helping principals to increase their effectiveness. Below are some of the main points we raised:

• For schools to be effective in raising student achievement, it is imperative that prospective principals with strong leadership potential be identified, recruited, trained, and supported. This goal needs to be a key focus of every district and state.
• Principals must be held accountable for ensuring high student achievement. Yet expectations for success need to be tempered by realities. First, principals affect student learning only indirectly through the school environments they create, educational programs they bring in, and the teachers they recruit and develop. Second, improving conditions for instruction and developing faculty take time. Therefore, evaluations of progress and feedback for improvement are needed regularly and frequently. Schools and school districts also differ considerably in the types of students and communities they serve. Principals who are placed in more challenging contexts may need more time and resources to raise achievement to the high levels desired. Evaluation criteria that that put too many eggs in one assessment basket, expect positive change to happen immediately, and ignore contextual variables seem likely to misclassify many principals, boosting some who are ineffective and downgrading others who are doing good things that simply need more time to work.

To increase the validity, utility, and fairness of evaluations, multiple indicators should be used.

• Student Growth and Achievement: The degree to which the principal succeeds in fostering school-wide gains and high performance in student achievement, behavior, and personal growth and development.
• Professional Growth and Learning: The degree to which the principal has followed through on professional development or learning plans to improve personal skills and practices.
• School Planning and Progress: The principal's success at managing the school planning process to achieve school improvement goals and increase student learning.
• School Culture: The principal's development of a positive school culture that promotes safety, collaboration, high expectations for teachers and students, and connectedness with the community.
• Professional Qualities and Instructional Leadership: The principal's leadership knowledge, skills, and competencies.
• Stakeholder Support and Engagement: The principal's ability to build strong support and involvement by teachers, parents, and the community.

None of these domains is difficult or expensive to include in a district or state evaluation. Of course, the risks of basing judgments of principals and feedback to them on insufficient or invalid information would likely be far greater.

Sputnik is moving! Follow Robert Slavin to the Huffington Post, where he'll continue to write about evidence-based reform in education. See you there! And don't forget to follow Dr. Slavin on Facebook and Twitter.

Posted by Sputnik Contributor at 8:20 AM | Permalink | Leave a comment | Recommend

Readers are often - and understandably - frustrated when it comes to reports of educational experiments and effect sizes. Let's say a given program had an effect size of +0.30 (or 30% of a standard deviation). Is that large? Small? Is the program worth doing or worth forgetting? There is no simple answer, because it depends on the quality of the study.

The Institute of Education Sciences recently issued a report by Mark Lipsey and colleagues focusing on how to interpret effect sizes. It is very useful for researchers, as intended, but still not so useful for readers of research. I wanted to point out some key conclusions of this report and a few additional thoughts.

First, the Lipsey et al. report dismisses the old Cohen characterization of effect sizes of +0.20 as "small," +0.50 as "moderate," and +0.80 as "large." Those numbers applied to small, highly controlled lab studies. In real-life educational experiments with broad measures of achievement and random assignment to treatments, effect sizes as large as +0.50, much less +0.80, are hardly ever seen, except on occasion in studies of one-to-one tutoring.

The larger issue is that studies vary in quality, and many features of studies give hugely inflated estimates of effect sizes. In order of likely importance, here are some factors to watch for:

• Use of measures made by the researchers.
• Very brief studies (often, one hour or less)
• Studies with small sample sizes

Studies that incorporate any of these elements can easily produce effect sizes of +1.00 or more. Such studies should be disregarded by readers serious about knowing what works in real classrooms and what does not.

The Lipsey et al. review notes that in randomized studies using "broad measures" (such as state test scores or standardized measures), average effect sizes across elementary and middle school studies averaged only +0.08. Across all types of measures, average effect sizes were +0.40 for one-to-one tutoring, +0.26 for small-group interventions, +0.18 for whole-classroom treatments, and +0.10 for whole-school treatments. Perhaps we should call effect sizes from high-quality studies equivalent to those of one-to-one tutoring "high," and then work backward from there.

The real point of the Lipsey et al. report is that the quality and nature of studies has to be taken into account in interpreting effect sizes. I wish it were simpler, but it is impossible to be simple without being misleading. A good start would be to stop paying attention to outcomes on researcher-made measures and very small or brief studies, so that effect sizes can at least be understood as representing outcomes educators care about.

Sputnik is moving! Follow me to the Huffington Post, where I'll be continuing to write about evidence-based reform in education. See you there! And don't forget to follow me on Facebook and Twitter.

Posted by Robert Slavin at 9:40 AM | Permalink | Leave a comment | Recommend

Recently, Public Agenda released a report on nine Ohio elementary and secondary schools that were "beating the odds," which means demonstrating outstanding student achievement in high-poverty schools. Of course, many other organizations have also identified such schools in the past, notably the Education Trust.

What was different in the Public Agenda report is that, at least in the case of the elementary schools, two of the three schools highlighted used a comprehensive school reform model--Success for All*. In other words, those schools were using a program that others could directly adopt, instead of trying on their own to figure out what all successful schools have to figure out: how to cultivate effective leadership, set high expectations, implement strong professional development, effectively use data, and so on.

The difference between a set of principles and a replicable program is night and day. A replicable program implements similar principles, but does so on purpose, and knows how to do it again and again. In one long-ago baseball game, Babe Ruth famously pointed at the right field wall and then hit the next pitch over it. Lots of baseball players hit home runs, but what made Babe's home run memorable is that he said he was going to do it and then he did it-on purpose.

There are lots of schools that do find a way to "beat the odds" on their own, and I am not arguing that the only path to success is any particular form of comprehensive school reform. But using a proven approach removes the trial and error and false starts; the investment in innovation has already been made by someone else. Implemented with fidelity, a proven program removes much of the risk that typically lies on the path to success. We need a lot more programs capable of helping struggling schools beat the odds--on purpose.

*Note: Robert Slavin is cofounder and chairman of the board for the Success for All Foundation.

Sputnik is moving! Follow me to the Huffington Post, where I'll be continuing to write about evidence-based reform in education. See you there!

Posted by Robert Slavin at 8:30 AM | Permalink | Leave a comment | Recommend

Note: This is a guest post by Monica Beglau, Ed.D., Executive Director, and Lorie Kaplan, Ph.D. , eMINTS Program Director for the eMINTS National Center at the University of Missouri.

Does this sound familiar? "Our school just purchased the latest mobile technology tablets for all of the students in our elementary school. Does anyone know where we could get some training about how to use them and what apps we should buy?" We've heard variations on this theme across our state and nationally for several years. Too often, as others have noted, the allure of the device outweighs practical planning for the implementation. Appropriate high-quality professional development and ongoing support for teachers is essential to success. Just as having sweet fluffy cotton candy for breakfast hardly fits the bill for a nutritious breakfast, short-term "summer boot camps" or a few hours of professional development after school leave educators hungry for more and without the necessary "nutrients" for effective instructional practices.

When we help schools and districts successfully implement technology initiatives, we turn to the evidence that has guided our work since 1999:

Leadership - leadership at all levels is essential and the principal is one of the most important variables in large-scale technology implementations.

• A clear vision and goals connect the technology implementation to identified instructional priorities agreed upon by all stakeholders.
  
• Ongoing professional development support provides principals with the knowledge and skills needed to achieve teacher buy-in and to understand best practices that support technology-transformed learning.

• A plan is in place to provide technology staff with the resources needed to support the devices, the network, and the maintenance issues that impact implementations.

• Active learning - participants must be engaged in interactive learning, not just listening to a lecture or presentation.
• Coherence - participants must see an explicit connection between the professional development and their classroom practice or leadership.
• Duration and intensity - if professional development contact time is less than 49 hours, it will produce little effect on student achievement.
• Personalization - professional development must take into account the varied learning styles and preferences of educators.
• Coaching - in-classroom or on-site coaching and mentoring is required to help educators "translate" what they learn in professional development sessions to their own classrooms or schools.

Posted by Sputnik Contributor at 7:59 AM | Permalink | Leave a comment | Recommend

Sputnik will be on "winter break" until the new year. While we're away, here are a few of Sputnik's top stories from the fall and a seasonal cartoon for you to enjoy.

• What Constitutes Strong Evidence of Program Effectiveness?
• Won't Back Down: Union-Bashing Goes Hollywood
• Better Teachers or Better Programs?
• Clicking Our Way to Great Teaching
• Innovative Technology Doesn't Mean Instructionally Innovative



Illustration: Slavin, R.E. (2007). Educational research in the age of accountability. Boston: Allyn & Bacon. Reprinted with permission of the author.

Posted by Robert Slavin at 7:30 AM | Permalink | Leave a comment | Recommend

Here's an astonishing statistic. Apparently, dairy cows today each produce six times as much milk as they did in 1950. Consumption of dairy products per person is about the same as it was then, so if milk per cow were the same as in 1950, we'd need six times as many cows per person; vastly greater acreage and other resources would be needed. The same pattern is true for almost any area of agriculture.

Yet do you recall any breakthroughs in agriculture in the past 60 years? I don't. Instead, the steady gains in agricultural productivity are due to hundreds or thousands of small advances. In the case of dairy cows, it's advances in breeding, feed, veterinary care, milking technology, and so on.

In education, we often act as though we're waiting for breakthroughs: New technologies, new assessments, radically new teaching methods, and so on. When breakthroughs do not materialize, we lose faith in research and development as a path to reform.

Yet in medicine, technology, agriculture, and other fields that base progress on evidence, progress is constant and cumulative. Breakthroughs may take place, but more often it's small, step-by-step improvements with evidence of effectiveness that move the field forward. When education finally embraces R&D as a basis for adoption of innovation, progress in each subject and grade level will probably also be steady rather than remarkable. Programs and practices found to make a modest but meaningful difference in student learning outcomes will accumulate over time, as took place in dairy farming and so many other fields that have seen substantial progress over the years.

As my colleague Jon Baron recently wrote in a New York Times article, "Scientifically rigorous studies - particularly, the "gold standard" of randomized controlled trials - are a mainstay of medicine, providing conclusive evidence of effectiveness for most major medical advances in recent history. In social spending, by contrast, such studies have only a toehold. Where they have been used, however, they have demonstrated the same ability to produce important, credible evidence about what works - and illuminated a path to major progress."

Precisely because genuine progress in educational programs and practice is likely to be gradual, it is especially critical that support for the R&D process be sustained and steady over time, since exciting headlines will be rare. If someone comes up with a "smart pill" or a new technology that doubles learning rates, all the better; the same R&D process that supports evolutionary change could also produce revolutionary change. But don't count on it.

Let's be clear. Reading scores in the U. S. have been virtually unchanged since 1980. Achievement gaps by social class and race have been about the same for 30 years. We should be outraged by this, but we need to turn that outrage into a commitment both to use the proven programs and practices available now and to engage in research and development that leads over time to truly transformative innovation.

Don't forget to follow me on Facebook and Twitter.

Posted by Robert Slavin at 7:30 AM | Permalink | Leave a comment | Recommend

One of the most serious problems in American education is the persistently low-achieving school, one that despite substantial attention and efforts over the years fails to make significant progress on test scores and other indicators. My colleague Robert Balfanz calls high schools like this "dropout factories," but there are persistently low-achieving elementary and middle schools as well.

For many years, localities, states, and the federal government have tried a variety of carrots and sticks to try to improve these schools. Most recently, persistently low-achieving schools have been eligible for substantial School Improvement Grants (SIG), but to get them they have to choose among four draconian alternatives, including school closure, turning the school over to a charter operator, or replacing the principal and at least half of the staff. Most SIG schools choose a "transformation" alternative in which the principal is replaced and the school receives extensive professional development. Yet a recent analysis of SIG data from the U.S. Department of Education shows success is spotty and elusive for these schools, especially considering the billions of dollars spent on them.

Designing replicable "transformation" programs to help persistently low-achieving schools would seem to be essential, yet it has not happened. A few years ago, the Institute for Education Sciences put out a request for applications to create and evaluate whole-school designs for turning around persistently low-achieving schools, but inexplicably, they did not fund any of the proposals they got and never issued another RFA on the topic. Whole-school reform models developed and evaluated in the 1990s are almost all gone, due to opposition by the Bush administration. Some of these had excellent evidence of effectiveness, but this did not matter. These models were not even mentioned in an IES-produced practice guide on turnaround programs, for example. As a result, schools now receiving SIG funding are mostly making up their own strategies, often with the help of consultants. Some of these home-grown strategies may work, but we won't know which ones, or why.

The recent funding of Investing in Innovation (i3) might offer a model for identifying and expanding school turnaround practices. Some of the i3 funded programs that have already been proven effective were designed as transformation or turnaround models, especially our own Success for All model and our colleagues' Diplomas Now high school program. Despite their considerable evidence of effectiveness, neither of these are used in many SIG schools. Existing proven models, whether or not they are funded by i3, certainly need to be used in turnarounds. Yet more proven programs are needed. The Department of Education should set aside funds for fast-track development, evaluation, and scale-up of proven transformation models, patterned on i3 but restricted to turnaround strategies. This problem is far too important for there to be so few proven programs available for these desperate schools to use.

Don't forget to follow me on Facebook and Twitter.

NOTE: Robert Slavin is co-founder of the Success for All Foundation, a recipient of i3 grants.

Posted by Robert Slavin at 10:05 AM | Permalink | Leave a comment | Recommend

Note: This is a guest post by Frank LaBanca, EdD, Director of the Center for 21st Century Skills at EDUCATION CONNECTION.

Have you seen the latest mini tablet computer? It can shoot video, take photos, play music, send and receive email, and browse the web. Therefore, we must immediately buy one for every student and figure out how to use it in the classroom later.

Isn't this too often the paradigm in education? We jump on to what's trendy without stopping to consider how the tool can effectively be used in the classroom to promote high-quality learning. Moreover, the quick-to-adapt device bandwagon often neglects evidence-based best practices.

I often suggest to educators that before they adopt a new technology tool, they should determine what they want students to learn. I often guide the discussion by referencing 21st century skills including:

• information literacy

• collaboration
• communication
• innovation/creativity
• problem solving
• responsible citizenship

These skills, coupled with high-quality, standards-based content are the foundation for learning. Once foundational decisions are made, then efforts can shift to determine what digital tools work best to promote that learning.

Just because the technology is innovative doesn't mean the instructional approach to using it is. As my colleague Jonathan Costa argues, when we retrofit with technology, we rarely change the paradigm. Take, for instance, the high school teacher who converts from a "chalk and talk" to a PowerPoint or interactive whiteboard presentation. There's no real difference in pedagogy--it's still direct instruction. Watching a video on YouTube may not be very different from a VHS or DVD, or for that matter, a filmstrip with cassette. If we replace the inferior textbook with the just-as-expensive online digital version, we still have the same lousy product that may not harness the power of authentic primary-source resources or evidence-based practice. Device-agnostic technology that provides access to the Internet and appropriate Learning Management Systems, coupled with a committed teacher, is often all that is needed to help students become powerful consumers and producers of knowledge.

Instructional technology must be transformative to be innovative. If we are utilizing technology in a meaningful way, its instructional value must offer options that couldn't exist without the tool. Online collaboration tools are such an example. A forum allows a written conversation between students that can take place asynchronously. Internet telephony (VoIP) services, such as Skype, allow students to communicate synchronously with experts in faraway cities, states, or abroad. And certainly the creation of digital media products including animations, videos, and podcasts provide a voice for students to communicate, tell their story, and share their novel ideas and learning. When their peers can provide online comments and feedback, the power of the technology becomes even more apparent. The recent boom in smartphones and tablets has lead to the development of millions of apps. Many apps have great educational value, but perhaps there is even more learning potential when students develop, market, and showcase their own.

The real question, ultimately, is, "Does technology help our students become better independent, self-directed learners?" That's the game-changer. It's not about the latest fancy device, hot off the shelf. That device is just a tool-- it's not knowledge and it's not a skill. Just because we haphazardly give students technology tools doesn't mean they are going to learn better--the evidence definitely supports that. Learners purposefully interacting with the tool and using it for production, facilitated by thoughtful, forward-thinking educators, is the way to get to a student-centered learning environment that improves engagement and achievement.

Center for 21st Century Skills at EDUCATION CONNECTION provides students and teachers with innovative, quality, timely, and evidence-based programs and services that increase learning achievement and engagement. EDUCATION CONNECION is a non-profit regional educational resource center in Connecticut dedicated to promoting the success of school districts and their communities.

Posted by Sputnik Contributor at 8:10 AM | Permalink | Leave a comment | Recommend