A Math Remediation Effort Boosted Students' College Credits. But Did Learning Improve?
College remediation is a big, expensive deal.
More than two-thirds of students in two-year colleges take a remedial class at some point after enrolling, and about 40 percent of those in four-year courses do, too. The courses are costly for students who pay for them, especially since they don't get credits for taking them. And finally, advocates fiercely debate whether the classes do anything to better prepare students—or whether they're just a big roadblock to a degree.
A few years back, Tennessee began trying out a novel solution to some of these problems: a transition course in senior year, in which high school students could master the math skills colleges require—and then directly enroll into credit-bearing classes, rather than remedial ones.
The idea of transition courses has since caught on among states eager to save kids (and taxpayers) cash. Now, the first large-scale study of Tennessee's initiative finds some good news for the program, but also raises questions about its underlying purpose. On the one hand, the study finds that the Tennessee initiative did help participating students enroll directly into college math, and to earn a few more credits compared to those students who didn't take the class. But the new course did not seem to boost students' actual math knowledge.
In all, the findings led the researchers leading the project to ponder whether higher education's entire approach to college remediation needs a serious rethink. High school transition classes may be a good first step, but perhaps remediation needs to begin earlier in students' high school trajectories, or coupled with a more intensive menu of services, they concluded.
"Whether it's in school or in college, we need to identify a more effective model of remediation," said Thomas Kane, the Harvard researcher who co-led the research team. "We need to commit to piloting and testing different models, perhaps a more intensive model, perhaps moving remediation earlier in high school."
Context of Tennessee's SAILS Reform
The study was conducted jointly by the Center for Education Policy Research at Harvard Unviersity and the Peabody College of Education and Human Development at Vanderbilt University, in Tennessee.
Since its debut, more than 57,000 students have enrolled in Tennessee's math transition class, called Seamless Alignment and Integrated Learning Support, or SAILS. Begun in 2011 at Chattanooga State Community College, it has since expanded to high schools throughout the state.
SAILS math uses a blended-education format, in which students proceed at their own pace through computer-based modules, including homework, assignments, and quizzes. Nearly 90 percent of students enrolled in SAILS completed it by the end of the 2017-18 school year, according to the Tennessee Higher Education Commission. The program's evolution has also paralleld a decline in the number of entering community-college students needing remediation.
Let's step back a minute here to review why the new Harvard-Vanderbilt research matters.
For one, there is not much research on what students learn in developmental education in general. Most studies instead focus on those courses' relationship to enrollment and completion rates. Second, transition courses like SAILS are only now starting to get a good look from education researchers. The few studies of them so far find generally small effects, some positive and some negative, noted Elisabeth Barnett, a senior research scholar at the Community College Research Center at Teachers College, Columbia University.
"The transition courses make total sense. Why wouldn't you want students to graduate ready for college? But of the research results we've seen so far, the outcomes haven't been headline-making," Barnett said. "There are still a lot more people doing transition courses than there were even a few years ago, and there's a lot of momentum about the idea of using 12th grade to do this kind of thing. The question is, how can we do it well?"
The idea has indeed taken off among states, as Education Week reported a few years back. California, Florida, Illinois, New Jersey, New York, Tennessee, and West Virginia all offer variations on the theme. It has also spawned efforts to develop some shared curricula. The Southern Regional Education Board has crafted model math and literacy transition courses for states and districts to adopt.
More Credits, More Learning?
The Harvard and Vanderbilt team used several methods to gauge the effect of SAILS.
First, because the program expanded each year from 2012-13 through 2015-16, the researchers were able to compare outcomes for students in SAILS schools with those in schools that hadn't yet implemented it. To do that, they took advantage of the program's cutoff point.
Students who received a score on the ACT college-entrance test below 19 were recommended to enroll in SAILS. To make up a sort of natural control group, the researchers compared them to students who just passed the ACT threshold. (This is the next best thing to having a random-assignment study.)
Finally, for a subset of high school seniors, the researchers gave a modified version of the ACT math exam to compare how students who enrolled in SAILS did in relation to those who took some other math course in their senior year.
Here's a look at the most important findings:
- In their first year in community college, participants' enrollment in college math did increase by 29 percentage points, and roughly half of those students passed that course. By their second year, the students had taken 4.5 additional credits (about a course and a half) compared to students in high schools without SAILS.
- That said, the increases weren't enough to change the proportion of students completing an associate degree or certificate within two years.
- SAILS also improved students' attitudes about math's usefulness and their preparation in math.
- SAILS did not noticeably boost performance on the math test.
A shift in policy might have affected these findings, too. By 2014, Tennessee moved to allow students to enroll in college remediation alongside credit-bearing courses—what's known as "co-requisite classes" in higher education lingo—and that effectively undercut some of the appeal of SAILS.
The researchers didn't study the blended-learning nature of SAILS. They did note, though, that teachers took different approaches to the online course. Some teachers provided more guidance and some whole-class teaching after finding that students struggled with one topic in particular; others were more inclined to simply let students work at their own pace. (All students took the class on school computers, not from home.)
SAILS has won numerous awards since its rollout, and it wasn't immediately clear whether the findings would prompt changes to the initiative.
"We appreciate the analysis undertaken on the program, and as we conduct our review of the study, it's important to remember this was a targeted solution developed and implemented for Tennessee, and SAILS has truly surpassed all expectations since its inception," said Mike Krause, the executive director of the Tennessee Higher Education Commission in a statement. "The return on investment for our students has been seismic, saving both credit hours and tuition dollars by avoiding math remediation."
Funding for the research came from the Bill & Melinda Gates Foundation.