edbizbuzz

If you think the education evaluators' debate over value-added models is arcane, this argument between two lawyers is about the legal meaning of two words - "based on" as in "based on scientifically based reading research."

Fellow blogger and GW Law '94 classmate Ken DeRosa of D-Ed Reckoning maintains that rules of statutory construction imply Congress intended that Reading First providers needed only to show phonics and claim their program had a relationship to a body of evaluation.

Edbizbuzz argues that some involved in drafting the Reading First provisions may have wanted that interpretation, but rules of statutory construction lead to the legally "better" interpretation of Congressional intent as requiring specific program evaluations.

If you agree with D-Ed Reckoning's reasoning, Chris Dougherty was doing his job. If you go with the edbizbuzz position, he was off the reservation (and maybe the fall guy).

If you enjoy truly arcane wonk arguments, have some interest in the problem of statutory interpretation and the interaction of k-12 policymaking and law, or know something about the inside story of Reading First, you might find this interesting.

Moreover, you might have evidence to share and help resolve our disagreement.

We both want your input.

Continue reading "Help D-Ed Reckoning and Edbizbuzz De/Re-Construct NCLB's SBRR Provisions" »

Posted by Dean Millot at 7:18 PM | Permalink | Comments (0) | TrackBacks (0)

For several weeks' the Letter has explored the proposition that accountability in public education requires standards, consequences and due process.

On paper, No Child Left Behind holds providers accountable for the value added by their offerings to student performance. Most providers must demonstrate evidence in the form of program evaluation under the law’s Scientifically Based Research (SBR) provisions; Supplementary Educational Service (SES) providers are held to its lower Research Based (RB) standard. Offerings that do not meet these standards are not eligible for purchase with federal fund allocated by NCLB to states, districts and schools. There is at least the inference of due process, in that the U.S. Department of Education and its state counterparts are required to develop rules, regulations, processes and administrative procedures for implementing the law’s provisions.

Continue reading "The Letter From: Where Provider Accountability Went Wrong" »

Posted by Dean Millot at 10:42 AM | Permalink | Comments (0) | TrackBacks (0)

Today, I was struck by how easy it is for states to run with No Child Left Behind’s standards of effectiveness for its school districts' new business competitors – Supplementary Educational Services (SES) providers, and how hard it seems to be for them to implement comparable rules for districts' traditional business partners. The traditional structure of public education isn't opposed to business involvement in teaching and learning, but to new businesses entering the field.

Continue reading "The "Program Evaluation" Bar - Why Just for SES Providers?" »

Posted by Dean Millot at 12:36 PM | Permalink | Comments (0) | TrackBacks (0)

(Readers' please note: The December 20 posting generated a great deal of email. A comment worth making is one that should be posted on the blog. Emails to me that are not prefaced with "not for publication" are subject to posting.)

On December 20 I posted a piece on Edvance's review of the Texas Early Education Model. The bottom line of that work, which covered only the first two years of a four-year effort, was equivocal:

There was considerable variation both between and within communities with regards to student performance and teacher outcomes. For about half of the communities, students in the treatment groups (with TEEM) improved more than students in the control groups (without TEEM), and for the other half of the communities students in the control groups improved more than the students in the treatment groups on the student outcome measures. TEEM did lead to overall improvement for teachers, although there was considerable variation, with teachers in both control and treatment groups obtaining both positive and negative difference scores on the teacher outcome measure.

Staci Hupp of the Dallas Morning News translated this into "no proof that most children fared better in TEEM than in conventional preschool programs." How should policymakers and taxpayers read the results? Like Hupp's headline - "Landmark preschool program isn't paying off"? And how should we think about school improvement program evaluation?

Continue reading "Making Sense of School Improvement Program Evaluations (II): The Case of TEEM" »

Posted by Dean Millot at 8:12 AM | Permalink | Comments (0) | TrackBacks (0)

Staci Hupp of the Dallas Morning News writes that the Texas Education Agency has released a third party review of the Texas Early Education Model (TEEM) managed by the Texas Health Science Center (THSC) in Houston. The study, conducted by Edvance Research, finds that TEEM does no better preparing kids for school than other preschool programs. (The report is available with the article.)

In the 2006-07 school year TEEM served 27,000 children. Ultimately, the program is to be offered statewide via an interesting dissemination model. Head Start and private childcare centers are offered substantial financial incentives to adopt the model. The initiative has cost Texas taxpayers $45 million since 2003. It will cost a great deal more if implemented across the state. So the evaluation matters.

Hupp reports the usual reactions from those who would like to kill TEEM off, and the developers who want to keep going. What’s a policymaker – or taxpayer - to do?

Continue reading "Making Sense of School Improvement Program Evaluations: The Case of TEEM" »

Posted by Dean Millot at 10:35 AM | Permalink | Comments (3) | TrackBacks (0)

There's a lot being made of the inequities in school staffing.

New teachers are assigned to inner city schools that administrators find hard to staff. If they don't leave the profession, when they have enough seniority to get first dibs on an opening, many move to schools in less demanding neighborhoods. The argument is that schools in wealthier, or less disadvantaged, neighborhoods get more experienced, and therefore better, teachers.

Intuitively obvious, but does evaluation show that teacher experience per se is educationally significant?

Continue reading "Random Research Question" »

Posted by Dean Millot at 9:13 PM | Permalink | Comments (0) | TrackBacks (0)

The U.S. Department of Education's What Work's Clearinghouse (WWC) is the closest thing the nation has to a National Highway Traffic Safety Administration (NHTSA) for school improvement programs.

The WWC's Intervention Reports serve something like the function of the NHTSA's crash test ratings. You can buy almost anything you want in our free market, but if you look at the government's reports first at least you'll know something objective about what you're getting into.

Continue reading "Let the What Works Clearinghouse Know What's Important" »

Posted by Dean Millot at 3:01 PM | Permalink | Comments (0) | TrackBacks (0)

Some blogs managed by k-12 providers are obvious extensions of marketing. Some take a more enlightened perspective. For-profit evaluation firm Empirical Education's monthly Latest Evidence falls in the latter category.

Now that the school improvement industry has been given a two year's reprieve on NCLB reauthorization, it has a second chance to get serious about a credible, but practical definition of the evidence required for programs to qualify for federal funding under Title I. Reasonable people can start from different positions, but should want to arrive at a consensus of all stakeholder groups. That includes providers and educators - not just practitioners of the evaluation arts. And that requires proposals and analysis intelligent laypersons can understand.

Here's where Latest Evidence adds to the dialog.

Continue reading "Business Blogs: Empirical Education" »

Posted by Dean Millot at 9:01 AM | Permalink | Comments (0) | TrackBacks (0)

Today, most k-12 program evaluation is, at best. a single snapshot of what worked - or not - where and when. At worst it's a political weapon employed in the charter, voucher, privatization, math or reading wars. Typically it's a less than completely informative marketing tool. Rarely is it employed for the kind practical evaluation described earlier in this series of blog posts.

Continue reading "Experience With Practical Evaluation (III): Today" »

Posted by Dean Millot at 5:55 PM | Permalink | Comments (0) | TrackBacks (0)

“Practical evaluation” involves more than grading the efficacy of a school improvement program, it informs operating decisions. Yesterday’s posting on the subject reminded me of some relevant experience.

Continue reading "Experience With “Practical Evaluation”(I)" »

Posted by Dean Millot at 10:04 AM | Permalink | Comments (0) | TrackBacks (0)

Pittsburgh's decision to hire for-profit research firm Mathematica Policy Research Inc. to evaluate the efficacy of Harcourt's Everyday Math in the city’s public schools is happening for one reason. Program results matter. The reason they matter is the present No Child Left Behind Act’s requirement that schools make Adequate Yearly Progress (AYP) towards 100% student proficiency in key subjects by 2014 or their students will be offered a range of help – and, more important, the adults in those schools will face a series of unpleasant consequences.

Continue reading "Thank NCLB I for the (Interrupted?) Birth of Practical Evaluation" »

Posted by Dean Millot at 11:02 AM | Permalink | Comments (0) | TrackBacks (0)

Today I received a notice from the Society for Research on Education Effectiveness (SREE). The Society was founded at the instigation of the Department of Education's Institute for Education Sciences in 2005. Like many academic activities, his one has taken a long time to get organized. SREE’s first conference was held in 2006. The first issue of its Journal of Research on Educational Effectiveness (JREE) has yet to be published. According to the notice explained that the second conference will be held in March of 2008.

The mission of the Society for Research on Educational Effectiveness (SREE) is to advance and disseminate research on the causal effects of education interventions, practices, programs, and policies. As support for researchers who are focused on questions related to educational effectiveness, the Society aims to: 1) increase the capacity to design and conduct investigations that have a strong base for causal inference, 2) bring together people investigating cause-and-effect relations in education, and 3) promote the understanding and use of scientific evidence to improve education decisions and outcomes.

Continue reading "The Society for Research on Education Effectiveness: What Direction?" »

Posted by Dean Millot at 8:40 PM | Permalink | Comments (1) | TrackBacks (0)

For the most part public schools do not develop their own curriculum or pedagogy - they buy them, primarily from private enterprise, and mostly from for-profits. I find it ironic that: 1) outsourcing the most central of public education’s core teaching and learning functions is never part of the “privatization” debate; and 2) the quality of those products services and programs didn’t become a subject of serious interest until No Child Left Behind. Yet “god forbid” we allow private enterprise to operate public schools under charter or contract, or manage teachers’ professional development. Given the discrepancy, it’s hard to believe that debate has much to do with what kids need rather than what adults want.

Speaking of quality, and the Reading First fiasco notwithstanding, program efficacy is supposed to matter today. Educators, indeed taxpayers, have a desperate need for independent reviews of program efficacy. In their absence, marketing dominates the sales race.

Continue reading "“Take Aways” from Two Months of What Works Clearinghouse Reviews" »

Posted by Dean Millot at 2:38 PM | Permalink | Comments (1) | TrackBacks (0)