The U.S. Department of Education’s Race to the Top program director, Joanne Weiss, has a big job on her hands now that hundreds of comments have been filed on the 19 criteria that the department proposes to use in awarding $4 billion in competitive grants. I’ve read most of the comments, and folks raise tons of good questions that showcase just how difficult it is try to apply one set of criteria to 50 different states. The comments also foreshadow how difficult it will be to fairly judge these states, which have different constitutions and governance structures, different politicians, and operate in different contexts.

There are the predictable comments, such as from states without charter schools that object to that being used as one of the criteria on which states will be judged. Plenty of others are objecting to the components of the criteria that seek to improve teacher and principal effectiveness. And the National Conference of State Legislatures, predictably, wants to be recognized in the criteria for the role its members have in education policy.

Weiss has said that the department will take time to read each comment and make any changes to the proposed criteria, with the goal being to have the final regulations done by October or early November.

Consider this a brief summary of other comments, organized around common themes.

Paperwork burden: Several state officials say that the documentation required for the application could be overly burdensome. They want more clarification on the department’s requirement that states obtain memoranda-of-understanding from participating school districts (as in all of them?) And many think that requiring a state’s attorney general to sign off on the interpretation of state laws that are used as evidence for meeting a criteria will take too long. Education chiefs in Massachusetts and Florida suggested that each education department’s chief legal counsel could do that job. The National Governors Association thinks the attorney-general requirement should be removed altogether. (And the NGA also says that some governors object to the requirement that the state board of education president must sign on to the Race to the Top Fund application).

“Participating LEAs": States and education groups are curious as to whether they can award the second half of their Race to the Top funds to a select group of school districts, or if they have to dole out money to all of them. (The first half of the money is doled out per the Title I formula.) States are making it clear that they may want to direct their Race to the Top money to a select group of schools where the most good can be done.

IEPs: Many states, special education advocacy groups, and even U.S. Rep. George Miller, the California Democrat and chairman of the House education committee, object to language about how states should measure achievement for special education students. The criteria call for using students’ Individualized Education Programs (IEPs) as a gauge for measuring their achievement in non-tested subjects. All point out that IEPs are meant to set goals, not to measure student achievement.

Transparency and accountability: Many states (such as Washington, Colorado, Texas, Kentucky), want to know, going in, what the scoring rubric will be on meeting the criteria, and whether failing to meet a single criterion will knock a state out of the running. They want to know which criteria will be given the most weight, and how the peer reviewers will be picked. The Coalition for Student Achievement wants the Education Department to post all of the state applications online, even before they’re approved. And the Bill & Melinda Gates Foundation wants the education department to hold back some of a winning state’s grant money as leverage to make sure the state delivers on its reform plan.

Common standards: The NGA and the Council of Chief State School Officers, which are partnering in the common standards effort, point out that the time line for states to adopt common standards is far more aggressive in the Education Department’s criteria than in the agreement reached by states. (The states agreed to adopt standards within three years; the criteria call for them to be adopted in just one year.) In addition, the department’s criteria call for all standards to be “identical” across states, whereas the states have agreed that 85 percent of their standards should match.

Early learning: State officials from Colorado and Kentucky, along with foundations such as The Pew Charitable Trusts, are encouraging the Education Department to ask states to integrate early learning and pre-K programs into their Race to the Top applications.