Tusome Endline Performance Evaluation

The USAID/Kenya and East Africa (KEA) Office of Education and Youth, in partnership with the Kenya Ministry of Education, Science and Technology (MoE), is implementing an $81.8 million basic education initiative to improve the reading skills of approximately 6.7 million Kenyan children who began primary school during the 2014-2021 school years. In 2019, USAID contracted NORC to conduct an independent endline evaluation in order to address descriptive and normative questions about Tusome following the activity’s national rollout.

Tusome seeks to improve children’s reading skills on a nationwide scale through evidence-based programming. In line with USAID policy, the testing of innovative activities is built into the Tusome design and the previous Primary Math and Reading (PRIMR) initiative, which developed and tested the methodology that lies at the heart of Tusome. The overarching purpose of the endline evaluation was to highlight the achievements of Tusome project goals, highlight the program’s impact on Kenya’s education policies, and document lessons learned during its implementation. This will inform USAID and the Government of Kenya on designing and implementing sustainable education programming in support of Kenya’s Journey to Self-Reliance.

Given the diverse goals of the evaluation, several methodologies were used including: (1) pre-post outcomes assessment, which focused on outcomes at the school-level using quantitative data collected at baseline, midline, and endline; (2) summative evaluation using a cross-section of quantitative and qualitative data to retrospectively assess program effectiveness, stakeholder beliefs and practices, and lessons learned from a variety of perspectives; and (3) formative evaluation to prospectively identify procedures, policies, and guidelines that can enhance the sustainability of Tusome-related activities over the long term.

Primary data collection included interviews with Head Teachers, Curriculum Support Officers, teachers, and pupils at a nationally representative sample of 204 schools that were selected randomly at baseline. Within each school, 24 pupils (approximately 5,000 total) in grades 1 and 2 were randomly selected to complete an Early Grade Reading Assessment (EGRA) which measures phonological awareness, understanding and application of the alphabetic principle, and reading fluency and comprehension. In addition, NORC conducted key informant interviews and focus group discussions with approximately 40 stakeholders including representatives from USAID and its implementing partner as well as Kenyan education officials at the national- and county-levels.

Overall, findings provide strong evidence for the efficacy of the Tusome model of instruction. From baseline to endline, pupils have shown statistically significant improvements on all EGRA subtasks in both English and Kiswahili. Of particular note, grade 2 English reading gains observed from baseline to endline are roughly equal to gains from an additional full year of schooling. Furthermore, over the life of the USAID activity exposure to Tusome has consistently corresponded with reading performance: increased exposure to Tusome between baseline and midline/endline corresponded with an increase in pupil reading performance. Similarly, reduced exposure to Tusome between midline and endline under the revised Competency-Based Curriculum (CBC) timetable corresponded with a decrease in reading performance.

Statistical models at endline also show that better implementation of Tusome in the classroom positively correlates with reading fluency. Tusome lesson plan progress is positively associated with English reading performance: each 10-unit advance in the Tusome teacher’s guide is associated with an increase of one correct word per minute in English reading fluency. Similarly, teacher-reported frequency of pupils sounding out unfamiliar words is positively associated with English reading performance. When teachers use both English and Kiswahili while teaching Kiswahili—i.e., use “code switching"—pupils score 3.19 words per minute higher in Kiswahili.

We also find that teachers generally demonstrate moderate to high levels of support for Tusome, yet faced a number of implementation challenges. While teachers broadly support Tusome, they have struggled to keep pace with the instructional approach from the midpoint of the program onward. Insufficient time was the main implementation challenge teachers reported at endline, with 39 percent of teachers who face challenges saying they lack sufficient time to cover the content and 23 percent claiming that Tusome lesson pacing is too fast, likely owing to the reduction in instructional time from five periods to three periods per week for English and five periods to three periods per week for Kiswahili under the 2019 CBC timetable.

Finally, we find that Tusome was largely successful in securing ownership and buy-in from the government at different levels of government and at different points in the implementation process, but more work is needed to ensure long-term sustainability. Most government respondents were personally supportive of the methods and materials developed under Tusome and hoped to see them continue in the future. However, long-term sustainability of Tusome will depend on upholding existing procedures, policies, and guidelines—particularly the centralized book procurement procedure, with its facilitation of the 1:1 textbook to student ratio, and school-based coaching and teacher mentoring.

Tusome ranks among USAID’s first experiences partnering with government to take a piloted literacy program to national scale. As such, this evaluation offers an important and timely case study for translating USAID-funded pilot programs into large-scale national educational reforms. More broadly, results of this evaluation can offer key insights on strategies for transitioning donor-supported educational activities to partner governments, with the ultimate goal of ensuring program sustainability and reducing dependence on foreign assistance.

 Content Editor

Infographics and Visualizations


 

 

View All