Early Learning Challenge: Potential Peer Review Problems?
I've already voiced a considerable amount of skepticism and concern this week as regards the new Early Learning Challenge program under Race to the Top, but my biggest concern about this program isn't about the program itself--it's about the peer reviewers.
In the wake of Race to the Top and i3, there's been enough [digital] ink spilled on the challenges of ensuring quality, unbiased review in competitive federal grant programs that I don't need to say any more about that now.
But if you thought that Race to the Top and i3 posed scoring challenges, you ain't seen nothing yet. The legislative language defining the Early Learning Challenge program is a lot sparser than that for the original RTT, laying out only three very broad parameters:
"(A) increase the number and percentage of low-income and disadvantaged children in each age group of infants, toddlers, and preschoolers who are enrolled in high-quality early learning programs;6201
"(B) design and implement an integrated system of high-quality early learning programs and services; and
"(C) ensure that any use of assessments conforms with the recommendations of the National Research Council's reports on early childhood.
Take a look at the third one. Do you know what it means? I've read the National Research Council's reports on early childhood, and I can't tell you what it means, because I don't know if it's talking primarily about this report or a wider body of NRC reports, and in any case those reports are notoriously reticent to give the type of specific guidance needed to evaluate whether a state's approach to early childhood assessment complies with them. The use of assessment in general is a hugely ideologically charged issue in the early childhood community, and if we get a dozen early childhood folks in a room we'll get nearly that many competing views on what is and is not developmentally appropriate when it comes to assessing young children.
I don't think it's possible for the administration to avoid outright the kind of problems that came up with RTT and i3, but I do think that three steps can help to at least avoid even greater problems than came up in those competitions:
- Define exceptionally clear criteria in the first place. As the administration crafts the application criteria for this program, designing criteria with clarity and precision to minimize opportunities for reviewer bias to sway judgments should be a paramount concern.
- Provide very precise rubrics and training for reviewers (obviously, this is easier if the criteria are clear to begin with).
- Use reviewers from outside the early childhood field: Obviously, reviewers need to have some knowledge of early childhood, but the administration should also use reviewers who are from outside the field and therefore not party to the various ideological camps within it. Because the focus of the competition is on systems-building, there are many issues in the design of systems that do not require in-depth knowledge of early childhood development or pedagogy, and the administration should seek out reviewers who have expertise in issues like data, and design of effective systems of public service delivery, workforce development/adult learning, and monitoring quality of diverse providers.