Writing in Google Docs Doesn't Affect Student Test Scores, Early Research Finds
But enthusiastic usage of Google Docs has not impacted student scores on standardized writing and reading exams, according to researchers set to publish a new study based on a relatively small sample of Colorado middle schoolers.
"This study could provide suggestions for K-12 classroom teachers who have adopted Google Docs," wrote Binbin Zheng, an assistant professor of education at Michigan State University and the lead author of the new study, in an email.
"To maximize the educational benefits of Google Docs," Zheng said, "teachers could give more co-authored assignments and provide students with clear guidelines for [giving] appropriate and helpful feedback" to their peers.
The researchers' paper, "Middle School Students' Writing and Feedback in a Cloud-Based Classroom Environment," is currently in press with the academic journal Technology, Knowledge, and Learning. Zheng and her co-authors will be presenting the work this week at the annual meeting of the American Educational Research Association, being held in Chicago. (Follow Education Week's reporting from the conference using the AERA2015 tag at the bottom of this story.)
The study examines how 257 6th grade students at a single middle school in suburban Colorado used Google Docs to write papers and exchange feedback with their teachers and with each other during the 2011-12 academic year. That was the school's first year using the online word-processing tool, which allows for multiple users to write, edit, comment, and communicate together on a single document, either simultaneously or at separate times.
Google Docs is provided for free to schools as part of the Google Apps for Education tool suite, which now has more than 35 million users worldwide. It is also central to the popularity of the inexpensive Chromebook, a cloud-based laptop computer that has been making major inroads in the K-12 market.
As part of their study, Zheng and her colleagues collected 3,537 student writing samples, then developed an analytical tool that allowed them to extract electronic records of each individual editing session done on every paper—18,146 sessions in all.
Their study relies heavily on quantitative analyses, including how many words were added and subtracted to a given document during each editing session; a survey of student attitudes and behaviors as relates to Google Docs; and a statistical analysis to determine whether those factors influenced student test scores.
The researchers also randomly selected 40 students and conducted a qualitative content analysis of every document written by each student, focusing on the nature and scope of the feedback that students received about their writing.
No impact on test scores
The study's big finding: Using Google Docs did not improve (or worsen) student scores on standardized writing or reading standardized tests.
That was determined by comparing students' results on the 2010-11 Colorado Student Assessment Program reading and writing tests, administered before Google Docs was introduced, to their results in 2011-12, after Google Docs had been in use for a year. Neither the extent of a student's revisions, measured by the number of words added to and deleted from an original document, nor the number of times a student edited his or her document had any statistically significant impact.
The researchers also found that the amount and nature of feedback that students received from both teachers and their peers within Google Docs did not have a statistically significant effect on students' test scores.
The study's test-score findings should be treated "cautiously," the researchers wrote, noting the relatively small sample size in their study; the fact that they did not know how much paper-based writing students were also doing; and the study's comparatively heavy emphasis on counting the number of words added/deleted, rather than focusing deeply on document quality.
Among the study's other findings:
- The students surveyed (89 percent of whom were white, and 7 percent of whom were Hispanic) preferred Google Docs to pencil-and-paper and more traditional word-processing software programs. They reported using Google Docs between 1-2 hours per day in their English/language arts classrooms and several times per week at home.
- Students reported using Google docs to make extensive revisions to their work, a finding the researchers described as consistent with previous findings about other types of computer-based writing. "These results suggest that Google Docs provides support for a process-writing perspective that encourages students to draft, edit, revise, and share documents over longer stretches of time," according to the study. The immediacy and convenience of feedback and communication among teachers and students within a given document was seen as a major advantage of Google Docs.
- Nearly three-quarters of the documents analyzed were edited by a single author. Just 5 percent were edited by three or more contributors.
- Most of the documents analyzed consisted of a single true author, with others providing feedback. More advanced levels of collaboration, including joint or parallel writing, were rare.
- Teachers using Google Docs in the classroom typically offered students heavy doses of "direct feedback," in which they corrected errors related to grammar and writing mechanics (e.g., spelling, capitalization, and punctuation.) Edits and comments related to content, organization, and word choice represented only about one-fourth of the feedback students received, and that came almost entirely from teachers. Peer feedback from other students was almost exclusively related to grammar and mechanics or general comments, such as "Excellent essay!"
- Documents with multiple authors were typically edited more times and ended up longer than documents with single authors. They also took longer to get started and longer to complete, a finding that led the researchers to suggest that "teachers should not expect rapid development of co-authored documents" and might instead focus on letting the collaborative process play out.
Overall, the researchers expressed enthusiasm about Google Docs' potential to improve student writing, noting that the process of reading each other's work and offering feedback, if done correctly, enhances students' active participation on the writing process and encourages students to think more deeply about content and structure.
The fact that teachers at the school being studied were at the beginning stages of using Google Docs and exploring ways to integrate it into their writing instruction was likely a factor in why its classroom practices did not reflect that lofty goal, the researchers said.
"Most of the writing activities teachers guided during the studied school year were targeted towards students' individual writing with feedback from others, since it seems to be the easiest type of collaboration for teachers to master at this early implementation stage," the study reads.
"Teachers could, however, explore other forms of collaborative writing with Google Docs in the future."
Photo: Chromebooks in use during an advanced 6th grade reading class at Ridgeview Middle School in Gaithersburg, Md. --T.J. Kirkpatrick for Education Week
Sign up here to get news alerts in your email inbox when stories are published on Digital Education.
- Chromebooks Gaining Popularity in School Districts
- Google Under Fire for Data-Mining Student Email Messages