Ed. Dept. Made Key Scoring Change in Latest 'i3' Round
The 23 finalists from the Investing in Innovation grant program are probably already counting their winnings (so long as they've secured their required private-sector match), but it's important to note the U.S. Department of Education made a possibly crucial change in scoring this year that may have altered who made it to the winners' circle.
The department did not standardize scores to determne this year's winners, a change from last year's first round of i3.
Wonky, yes. But an important change.
According to a department spokesman, feedback from the public indicated there was confusion over how the applications were scored last year, which, among other things, prompted the change. Count Politics K-12 as among those confused over the standardization process, which EdWeek blogger Sara Mead also called "incomprehensible".
The reason for standardization during the first round, according to the department, was to mitigate what can sometimes be big variation in the judging process between easy graders and hard graders. Standardization is a highly statistical process that makes sure some applicants aren't improperly penalized just because they got a bunch of hard graders (or vice versa). Standardization was not used in the department's other high-profile competition—Race to the Top, which garnered its own share of scoring criticism.
An analysis by the Rural and Community Trust indicated standardization wasn't a big factor in determining the fates of applicants in the first round, though it did have some effect.
Nonetheless, the department got rid of standardization. A spokesman told me the department then took additional steps to address concerns over the naturally human and imperfect process of judging, and how to balance hard and easy graders. Among those steps: Judges were given clearer numeric ranges to help them decide on a score, and the department limited the number of applications each peer reviewer judged.