Designing for Scale: Reflections on Rolling Out Reading Improvement in Kenya and Liberia

In this study, the Liberian and Kenyan experiences of implementing large-scale reading programmes are examined, documenting the critical components and conditions of the programme designs that affected the likelihood of successfully transitioning from pilot to scale. The study also reviews the design, deployment, and effectiveness of each pilot programme and the scale, design, duration, enabling conditions, and initial effectiveness results of the scaled programmes in each country.

Authors: Amber Gove, Medina Korda Poole, & Benjamin Piper

Source: Gove, A., Korda Poole, M., & Piper, B. (2017). Designing for scale: Reflections on rolling out reading improvement in Kenya and Liberia. In A. Gove, A. Mora, & P. McCardle (Eds.), Progress toward a literate world: Early reading interventions in low-income countries, New Directions for Child and Adolescent Development, 155, 77–95.

In this study, the Liberian and Kenyan experiences of implementing large-scale reading programmes are examined, documenting the critical components and conditions of the programme designs that affected the likelihood of successfully transitioning from pilot to scale. The study also reviews the design, deployment, and effectiveness of each pilot programme and the scale, design, duration, enabling conditions, and initial effectiveness results of the scaled programmes in each country.

  • UNESCO estimates that 250 million children globally are not learning basic reading and maths skills.
  • Both Liberia and Kenya began pilot programmes to evaluate various approaches to improving reading outcomes, specifically the Early Grade Reading Assessment (EGRA) Plus programme in Liberia and the Primary Math and Reading (PRIMR) Initiative in Kenya, with noteworthy results.
  • The resulting rigorous evidence encouraged both countries’ governments to scale up the programmes to a larger number of schools.

Liberia

  • The country has struggled to provide basic education services to its population.
  • Net primary enrolment is less than 40%, and more than 400,000 children are out of school.

Kenya

  • The country is not as poor as Liberia.
  • The quality of early primary school education remains low.

The study

This study examined the particular characteristics of scaling up reading programmes in Liberia and Kenya and documented essential technical components and conditions of the programme design that increased the likelihood of a successful pilot-to-scale transition.

Research questions:

  1. Do pilot interventions in Liberia and Kenya improve learning outcomes?
  2. How do the pilot findings inform the scale-up design in Liberia and Kenya?
  3. Do the scaled-up interventions improve learning outcomes in Liberia and Kenya?

Liberia’s EGRA Plus programme was designed with two treatment groups and a control group that received the intervention after the endline assessment. Each of the three groups included 60 schools. The ‘light’ treatment included support for classroom-based assessment and the dissemination of student results to parents and community members. In the full-treatment schools, teachers received instructional materials and a manual with 110 scripted reading lesson plans. They were trained by instructional coaches, and were given a small library. Liberia began implementing an approach in 2011 designed to be similar to the EGRA Plus full-treatment intervention in 1,200 schools.

Kenya’s PRIMR initiative was implemented in 1,384 schools. Students received both English and Kiswahili textbooks. Teachers received a corresponding teachers’ guide. Teachers were trained 7 days per year, and they observed and then practiced a model mini-lesson. There were also tutors who focused on improving teaching and providing in-classroom instructional support.

Findings

Liberia Pilot Results

  • Full treatment increased student achievement for every EGRA subtask, with an overall effect size of 0.79 SD, ranging from 0.39 SD to 1.23 SD.
  • Full treatment increased letter-naming fluency by an additional 1.2 years compared to the effect of being in school for 1 year. This was the smallest effect size.
  • The light treatment group had a statistically significant effect on letter-naming fluency and phonemic awareness but no effects on the other areas of literacy. This suggests that accountability alone cannot rapidly improve results.

Liberia Large-Scale Intervention Results

  • Letter-naming fluency and phonemic awareness were the only skills to show significant improvements relative to the control group, with effect sizes 0.56 and 1.03 SD, respectively.

Kenya Pilot Results

  • Kenya’s PRIMR programme produced effect sizes between 0.3 to 0.8 SD in reading.
  • The mathematics outcomes were somewhat smaller in size but were still statistically significant.
  • The most important factor to learning outcomes was the higher-quality student book.
  • The PRIMR resulted in moderate-to-large effects for all subjects, including English, Kiswahili, mathematics, and mother tongue.

Kenya Large-Scale Intervention Results

  • Tusome (a large-scale PRIMR programme) was implemented in more than 23,500 schools in 2015 and currently serves more than 2.5 million pupils annually.
  • Results are not yet available for this programme; however, large-scale implementation seems promising.

Implications

Both pilot interventions—the EGRA Plus in Liberia and the PRIMR in Kenya—have been found to improve reading. However, the final results from large-scale interventions are not yet available because the Ebola crisis affected the large-scale intervention in Liberia, and in Kenya they are waiting to determine the effects of the intervention.