Effectiveness of Teachers’ Guides in the Global South: Scripting, Learning Outcomes, and Classroom Utilization

This report presents the results of RTI International Education’s study on teachers' guides across 13 countries and 19 projects. Using quantitative and qualitative methods, we examine how teachers’ guides across the projects differ and find substantial variation in the design and structure of the documents. We develop a scripting index so that the scripting levels of the guides can be compared across projects. The impact results of the programs that use teachers’ guides show significant impacts on learning outcomes, associated with approximately an additional half year of learning, showing that structured teachers’ guides contribute to improved learning outcomes. During observations, we find that teachers make a variety of changes in their classroom instruction from how the guides are written, showing that the utilization of structured teachers’ guides do not create robotic teachers unable to use their own professional skills to teach children. Unfortunately, many changes that teachers make reduce the amount of group work and interactivity that was described in the guides, suggesting that programs should encourage teachers to more heavily utilize the instructional routines designed in the guide. The report includes a set of research-based guidelines that material developers can use to develop teachers’ guides that will support effective instructional practices and help improve learning outcomes. The key takeaway from the report is that structured teachers' guides improve learning outcomes, but that overly scripted teachers' guides are somewhat less effective than simplified teachers' guides that give specific guidance to the teacher but are not written word for word for each lesson in the guide.

Persistence of pre-primary programs in Kenya: Evaluating how Tayari's program impacts change over time and into primary school- CIES 2018 Presentation

CIES 2018 Presentation, given by Ben Piper. While there has been recent evidence pointing to the efficacy and cost-effectiveness of pre-primary interventions, there remains little known about how those interventions might work at medium or national scale, nor whether the theoretical impacts of the programs increase, decrease or remain steady over time, nor whether the gains persist into primary school. The endline results from two years of Tayari’s intervention provide longitudinal data on nearly 2500 children who have been evaluated across a range of early learning skills (literacy, numeracy, socio-emotional) at three data points, including their first year of Grade 1. The results presented will explain how Tayari program impacts change over these three data points and determine whether Tayari’s impact persists into Grade 1.

National level classroom monitoring: the impact of Kenya Tusome on learning outcomes and accountability: CIES 2018 Presentation

CIES 2018 Presentation, given by Ben Piper. Tusome is supporting the Ministry’s systems to increase the utilization of monitoring data, and to expand the accountability structures of the government using the data provided by the national literacy program. Of particular interest is the classroom observation data collected by government-paid coaches, in Kenya these officers are called Curriculum Support Officers (CSOs). The data presented in this panel shows how the instructional support structures in Kenya have worked over time, with particular interest in the availability and usage of classroom support data.

The role of language instruction in schools as a tool among marginalized groups in Kenya- CIES 2018 Presentation

CIES 2018 Presentation, given by Ben Piper. This paper examines the influence of Mother Tongue (MT) as a medium of instruction in lower primary schools in 4 zones of Machakos County based on the PRIMR program. The relationship between MT implementation at the classroom level and its impact on pupil’s literacy outcomes are explored.

Linking literacy and numeracy in Tayari- CIES 2018 Presentation

CIES 2018 Presentation, given by Ben Piper. To improve the quality of preprimary education in Kenya, the Ministry of implemented the Tayari program in collaboration with four county governments. The program works with government officers to train teachers and support them in the classroom on a daily basis. Teachers rely on Teachers’ Guides developed to guide them in their lessons and are encouraged to expand children’s learning through two core strategies: the use of materials and extended learning. This presentation details one way in which reading and mathematics can be intertwined on a project through consistent instructional messages in trainings, materials for teachers, and support visits.

Understanding whether and how the Tusome program worked: Evidence from the national scale-up of a tested literacy program in Kenya- CIES 2018 Presentation

CIES 2018 Presentation, given by Ben Piper. The Tusome national literacy program has been implemented in each of Kenya’s more than 22,000 public primary schools and 1500 low cost private schools in the slums since 2015. In addition to showing the comparisons between before and after Tusome’s implementation, this presentation will examine Tusome implementation data to reveal key characteristics of the scale-up framework that Tusome was designed to respond to. In particular, we share the extent of classroom utilization of the Tusome materials, the size of the classroom observational structures that Tusome tried to revitalize, and the responses of the community to the Tusome intervention. The findings suggest that Tusome had a substantial impact on literacy outcomes in both Kiswahili and English, in both Grade 1 and 2, in both public and low cost private settings, and in both urban and rural settings. This means that the program’s effect meant that nearly 1 million more children were able to be considered readers by the midterm intervention.

Tayari's Longitudinal Endline Results: Program Impact and Implementation Decisions- CIES 2018 Presentation

CIES 2018 Presentation, given by Ben Piper. The Kenyan Ministry of Education (MoE) in collaboration with RTI International and the county Governments of Laikipia, Nairobi, Siaya and Uasin Gishu have been piloting the Tayari Early Childhood Development and Education (ECDE) model since 2014. Built on a Randomized Controlled Trial (RCT), RTI has been tracking a cohort of learners from both treatment 2 and control schools over the last two years: The baseline data was collected in January 2016; midline in October 2016, at the end of the academic school year; and endline a year later in October 2017. The overall effect of Tayari showed a larger effect at midline – after ten months of implementation – than was expected at end line, i.e. after two years of implementation. The gains in the Tayari group was about 50% higher than in the control group.

Is ‘summer’ reading loss universal? Using ongoing literacy assessment in Malawi to estimate the loss from grade-transition breaks

Published abstract: "Summer learning loss – decreased academic performance following an extended school break, typically during the period after one grade ends and before another grade starts – is a well-documented phenomenon in North America, but poorly described in sub-Saharan African contexts. In this article, we use the term ‘grade-transition break’ loss in lieu of ‘summer’ loss to refer to the period after one grade ends and before another grade starts. This study analyses data from early grade reading assessments in Malawi, estimating statistically significant average reductions of 0.38 standard deviations (SD) across several measures of reading and pre-reading skills during two grade-transition breaks. The data show the loss in reading skills during the extended breaks between grades 1 and 2 and between grades 2 and 3 in two consecutive years. The study found no gender-based differences in loss. The findings suggest a need for early grade reading interventions to develop and evaluate mitigation strategies lest significant proportions of within-year performance gains be lost over the break between academic years."

Implementing large-scale instructional technology in Kenya: Changing instructional practice and developing accountability in a National Education System

Article published in the IJEDICT, Vol. 13, No. 3 (2017). Published Abstract: "Previous large-scale education technology interventions have shown only modest impacts on student achievement. Building on results from an earlier randomized controlled trial of three different applications of information and communication technologies (ICTs) on primary education in Kenya, the Tusome Early Grade Reading Activity developed the National Tablets Program. The National Tablets Program is integrated into the Tusome activity by providing tablets to each of more than 1,200 instructional coaches in the country to use when they visit teachers. This enables a national database of classroom instructional quality, which is used by the education system to monitor overall education quality. The tools provided on the tablets are designed to help coaches increase the quality of their instructional support to teachers, and deepen the shallow accountability structures in Kenya’s education system. Using results of a national survey, we investigated the ability of the National Tablets Program to increase the number of classroom observations done by coaches and to improve student learning outcomes. Survey results showed high levels of tablet program utilization, increased accountability, and improvements in learning outcomes. We share recommendations regarding large-scale ICT interventions and literacy programs.

Examining the secondary effects of mother-tongue literacy instruction in Kenya: Impacts on student learning in English, Kiswahili, and mathematics

Limited rigorous evidence is available from sub-Saharan Africa regarding whether children who learn to read in their mother tongue will have higher learning outcomes in other subjects. A randomised controlled trial of mother-tongue literacy instruction, the Primary Math and Reading (PRIMR) Initiative, was implemented in Kenya from 2013 to 2014. We compared the impacts of the PRIMR mother-tongue treatment group in two languages with those of another group that did not use mother tongue, but utilised the same instructional components. Results showed that assignment to the mother-tongue group had no additional benefits for English or Kiswahili learning outcomes beyond the non-mother-tongue group, and that the mother-tongue group had somewhat lower mathematics outcomes. Classroom observational analysis showed that assignment to the mother-tongue group had only small impacts on the usage of mother tongue in other subjects. Advocates for mother-tongue programmes must consider such results alongside local implementation resistance in programme design.