Studies of STEM-Focused Schools Yield Mixed Results
For those who are concerned that the nation is starving for students who specialize in science, technology, engineering and math, STEM-focused schools are a popular solution.
In 2010, the President's Council of Advisors on Science and Technology called for 1,000 such such schools to open in the next decade. But the results of several new studies raise questions about the overall efficacy of this approach.
In an article published online in March by the peer-refereed Journal of Educational Research, Michael Hansen, a principal researcher at the American Institutes for Research, found that students in STEM schools in North Carolina were significantly more likely to take core, advanced, and vocational-technical STEM courses than were their peers in other types of schools. However, in Florida, STEM students participated in vocational-technical STEM courses at higher rates but were about as likely as students at other types of schools to take core and advanced STEM courses.
Hansen also found that students in STEM-focused schools were no more likely than students in other school types to perform well on state math exams in Florida and North Carolina between 2006 and 2008. Results from 8th grade science tests in North Carolina likewise showed no association between attending a STEM school and student achievement on the exam.
Hansen's analysis used so-called "value-added" methods that accounted for prior student achievement, demographics, and other factors. It included five different samples of elementary and middle grades in both states, the largest of which included more than 3,200 schools.
The goal of that study was to tease out characteristics of schools that were higher and lower-performing in science and in math. It did not focus on STEM schools per se.
But another article published online in February by that same journal focuses specifically on STEM schools. That study was unusual in that it examined STEM-focused elementary and middle schools. Studies tend to address STEM schools at the high school level, where they are generally more common. For instance, Hansen's study found that 28 percent of STEM-focused schools were high schools, as compared to 10 percent of non-STEM schools in that state. In North Carolina, nearly half of STEM-focused schools were high schools, as compared to 15 percent of non-STEM schools.
In his article on STEM elementary and middle grades, Eugene Judson, an associate professor of science education at Arizona State University in Mesa, compared the TerraNova math, language arts and reading results of 289 students who transferred into 11 STEM-focused magnet and charter elementaries in his state in 2005 and 2006 and then remained there for at least three years. (Magnets are regular public schools that seek to attract students with specialized offerings while charters are independently operated, publicly funded schools.) The results were mixed. Transferring to STEM magnets did not change the achievement trajectories of students, who performed at about the same levels as peers who transferred to non-STEM schools in the same district. Transferring to STEM-focused charters in 2005 (but not in 2006) did appear to help students make significantly more academic growth in math and language arts than they had made before, although not in reading. But those increases were similar to the increases experienced by children who transferred to non-STEM schools during that same period.
"These results do provide mild indication that the STEM-focused charter schools contributed to growth in the traditional content areas, but overall the evidence is insubstantial," Judson concluded. "Further, attention is drawn to the finding that the students to whom the charter school students were compared also made noteworthy gains during each of the four-year periods. This may possibly be due to students, who transfer to another school system, simply becoming more established after three years and thus posting such gains. If that is the case, then the same argument can be made for the students who transferred to the STEM-focused charter schools; this would wash away the conjecture that it is the STEM-focused charter school itself that led to the gains."
Judson noted that his analysis was limited in that he did not have data on science, technology or engineering achievement—just math.
A richer set of STEM-related student achievement data was available for another article published in the June issue of the peer-reviewed Economics of Education Review. The first author of that article was Matthew Wiswall an assistant professor of economics at Arizona State. Wiswall and his co-authors examined math, biology, chemistry, and physics course-taking and exam results for more than 70,000 students attending both selective and nonselective public STEM high schools in New York City. (Although STEM schools in many areas of the country were historically exam schools for high-achieving students , newer versions tend to eschew selective admissions in favor of open/lottery-based policies that aim to attract a wider array of students.)
The NYC STEM schools appeared at first glance to be doing a better job because they had higher scores and STEM-course-taking rates than other local public high schools. But once the researchers accounted for demographics and prior test scores, most of the STEM-focused schools' advantages disappeared, suggesting that the schools were disproportionately attracting higher-achieving students who were interested in STEM.
That said, there was also a bright spot: Three groups of students (females, Hispanics, and blacks) who are underrepresented in many STEM fields performed better in STEM than non-STEM high schools, after accounting for prior student achievement. For females, the results were somewhat mixed. Females at STEM schools earned higher scores in biology and math than their counterparts in non-STEM schools. But so did males. Also, the gender gap within STEM schools was larger than the gender gap within non-STEM schools, with males outperforming females in both school types.
For minority groups that are underrepresented in STEM fields, the results were more positive.
"STEM schools boost the likelihood that blacks take Math A, Biology and Chemistry by 5.4, 6.2 and 16.4 percentage points relative to whites," the authors write. "Similarly, Hispanics in STEMs receive an additional benefit relative to whites of around 7 percentage points in the probability of taking Math A and 9 and 13 percentage points for the probabilities of taking Biology and Chemistry, respectively. The black-white and Hispanic-white gaps are also smaller for Math B [test scores]."
The authors concluded by suggesting that, in the future, researchers peek inside the black box of STEM high schools to see what, if anything, STEM schools do differently, especially when it comes to females, Hispanics, and blacks.
"While the literature might suggest that STEMs are characterized by a 'chilly environment,' where minorities and females can feel unwelcomed, our results suggest that, in contrast, these schools are doing something right for them," they write.