A Meta-Analysis of Single-Subject Design Writing Intervention Research

In this meta-analysis of single-subject design writing intervention studies, 88 studies in which it was possible to calculate an effect size were located. Nine writing treatments were identified as effective. These were strategy instruction for planning/drafting, teaching grammar and usage, goal setting for productivity, strategy instruction for editing, writing with a word processor, reinforcing specific writing outcomes, prewriting activities, teaching sentence construction skills, and strategy instruction for paragraph writing.

Author: Leslie Ann Rogers & Steve Graham

Source: Rogers, L.A. & Graham, S. (2008). A meta-analysis of single-subject design writing intervention research. Journal of Educational Psychology, 100(4), 879-906. DOI: 10.1037/0022-0663.100.4.879

  • Many children do not learn to write well enough to meet classroom writing demands.
  • Concerns about writing are not limited to elementary and secondary schools; college instructors estimate that 50% of high school graduates are unprepared for college-level writing demands.
  • At school, weaker writers are less likely than their more skilled peers to use writing to support and extend learning in content classrooms.
  • Their chances of attending college are reduced, as writing is used to evaluate many applicants’ qualifications.
  • At work, writing has become a gateway for employment and promotion.
  • Why do so many students not write well enough to meet grade-level demands?
  • One possible reason is that schools do not do an adequate job of teaching this complex skill.
  • The National Commission on Writing (2003) offered the following recommendations: double time students spend writing, assess their writing progress, use technology to advance the learning and teaching of writing, and better prepare teachers to teach writing.
  • The impact of these recommendations is likely to be reduced if teachers do not use effective instructional practices.

Need for a meta-analysis of writing interventions tested via single-subject design studies

  • A practical approach for identifying effective writing practices is to conduct systematic reviews of writing intervention research.
  • With meta-analysis, an effect size is computed for each empirical study investigating a specific treatment. It then is averaged across studies to provide a summary statistic on the intervention’s effectiveness.
  • However, only two meta-analyses have computed effects sizes for treatments tested via single-subject designs, and they only examined a single treatment: teaching strategies for planning/revising.
  • Thus, the primary purpose of this study was to conduct a more extensive meta-analysis of single-subject design writing interventions to identify effective writing practices for students in grades 1–12.

What is a single-subject design study?

  • In single-subject design studies, each participant serves as their own control, with performance before and during or after intervention repeatedly measured to establish performance patterns before treatment and compare performance patterns across experimental phases (e.g., baseline versus treatment).
  • One manipulation for establishing experimental control involves the introduction and withdrawal of treatment.
  • A second manipulation involves the staggered introduction of the treatment. For example, treatment is implemented with one student to determine whether it influences their performance predictably. Then the treatment is executed with another student to determine whether the pattern is replicated. This systematic delay in introducing the treatment continues until all students receive instruction.
  • Experimental control is established only if performance on the dependent measures is stable during each testing phase. There is no trend in the pattern of baseline performance in the direction predicted by the intervention.
  • In addition, experimental control is not established until at least three demonstrations show that the manipulation had the predicted impact.
  • Single-subject design examines the effectiveness of treatment at the individual level.
  • External validity is established by systematically replicating effects across multiple participants, locations, and researchers.

Why a meta-analysis of single-subject design studies is important

  • First, the meta-analyses of true- and quasi-experimental investigations of writing interventions have identified only 12 interventions that improve the writing of elementary and secondary students.
  • A meta-analysis of single-subject design writing interventions has the potential to broaden current evidence-based recommendations.
  • Second, a meta-analysis of single-subject design studies also has the potential to strengthen, undermine, or nuance the trust that can be placed in one or more of the 12 writing treatments identified as effective earlier.
  • Third, most of the true- and quasi-experimental writing intervention research has been conducted with students representing the full range of writing ability in a typical classroom. In contrast, single-subject design studies often involve students’ experiencing difficulty.

The study

This meta-analysis draws on but dramatically extends the two previous meta-analyses of single-subject strategy instruction research in writing. The primary research question of this review was, which writing practices tested via single-subject design procedures are effective with students in grades 1–12?

Method

Studies were included if they involved grades 1–12 students and provided data to calculate the effect size. Overall, 119 documents were found, from which 88 were suitable. Studies were categorised based on treatments used, and summary statistics were calculated only to those categories that included at least four studies. The ten treatment categories were: strategy instruction (planning/drafting), teaching grammar/usage, goal setting for productivity, strategy instruction (editing), word processing, reinforcement, prewriting activities, sentence construction, strategy instruction (paragraph construction), and self-monitoring.

Results

Strategy instruction: planning/drafting

  • Twenty-five studies examined the effectiveness of teaching strategies for planning/drafting specific types of text.
  • Typically, students use specific features of the target genre to help them generate and organise possible writing ideas.
  • Of these 25 studies, 21 had elements as a common outcome measure.
  • Teaching students a planning/drafting strategy greatly impacted the number of essential genre elements in their writing, maintained over time. It also had a moderate impact on enhancing the generalisation of elements from an instructed genre to an uninstructed one.
  • Teaching students a planning/drafting strategy greatly impacted productivity and quality during or immediately following instruction. In addition, students generally maintained productivity gains.
  • Strategy instruction effectively enhanced the number of elements, written output, and quality of students’ writing; the effects for elements and productivity were maintained over time.

Teaching grammar/usage

  • Four studies evaluated the effectiveness of teaching grammar/usage.
  • Teaching grammar/usage included peer directly teaching capitalisation skills to classmates to teachers instructing on adverbial phrases and possessives to correct capitalisation, subject/verb agreements, conjunctions, incomplete sentences, and run-on sentences.
  • Outcome measures on these studies focused on the correct use of grammar.
  • Directly teaching grammar/usage had a moderate effect on improving grammar skills.

Goal setting for productivity

  • Seven studies examined the impact of setting goals.
  • Goal setting ranged from teachers encouraging students to exceed their previous writing performance and receiving immediate feedback on their success to teachers setting a goal for how much students would write. Students placed a star on a public chart if the goal was met.
  • Productivity was the standard outcome measure for these studies.
  • Goal setting for productivity had a large to moderate effect on increasing writing productivity.

Strategy instruction: editing

  • Five studies examined the effectiveness of strategy instruction for editing.
  • Errors corrected was the typical outcome of these studies.
  • Teaching an editing strategy had a large to moderate effect on correcting errors in writing.

Word processing

  • Five studies evaluated the effectiveness of word processing.
  • Four of the five studies used productivity as the outcome measure.
  • Word processing had a moderate effect on increasing students’ productivity.

Reinforcement

  • Seven studies examined the effectiveness of using reinforcement to enhance writing performance.
  • Four of the studies included productivity as a standard outcome measure.
  • Reinforcement had a significant effect on students’ writing productivity. However, the overall quality of the studies was not strong.

Prewriting activities

  • Four studies examined the effectiveness of prewriting activities.
  • Prewriting activities included using a computer prewriting outline to generate and organise information, using a graphic organiser to create ideas before persuasive writing, and learning to use a story web to generate ideas prior to writing
  • Three studies included writing quality as a standard outcome measure.
  • Prewriting had a negligible effect on improving writing quality.

Sentence construction

  • Five studies examined the effectiveness of teaching sentence construction skills.
  • The studies used complete sentences as a standard outcome measure.
  • Sentence construction was an effective practice in increasing the percentage of complete sentences produced by students. However, the quality of the studies was poor.

Strategy instruction: paragraph construction

  • Five studies examined the effectiveness of teaching students strategies for constructing paragraphs.
  • Writing elements was used as an outcome measure on four of the studies, and it involved determining whether the basic parts of a paragraph were evident and correctly used.
  • Teaching strategies for writing paragraphs had a significant and positive impact on the schematic structure (i.e., elements) of students’ paragraphs. However, the studies were of poor quality.

Self-monitoring

  • Eight studies examined the effects of self-monitoring.
  • Productivity was used as an outcome measure in seven studies.
  • Self-monitoring had only a small effect or no effect. However, the quality of the studies was not good.

Conclusions

  • Writing is a critical skill in an advanced technological society.
  • Ensuring that students become skilled writers involves teachers’ use of effective writing practices.
  • Recommendations:
    • Teach students strategies for planning/drafting both narrative and expository text. This is effective with struggling writers in grades 2–8 and typical writers in grades 4–8.
    • Teach grammar skills to struggling writers directly.
    • Set clear and specific goals to increase students’ writing productivity.
    • Teach students strategies for editing their compositions.
    • Make it possible for students to use word processing as a primary tool for writing.
    • Reinforce students for their writing productivity.
    • Engage students in prewriting activities for gathering and organising ideas in advance of writing.
    • Teach students how to form complex sentences.
    • Teach students strategies for writing different types of paragraphs.
  • When implementing the recommendations, it is helpful to continually monitor the treatment’s effects to see whether it is effective under new conditions.