One of STS’s goal for activities in Tanzania in 2023 is to generate evidence of the effectiveness of the Whole Child Model—our claim that if children receive adequate education, health, and community support, they will thrive in school. To make this claim, we need to know two things:
- How well are students learning?
- How do we know if our efforts are helping them learn?
Teachers typically answer these questions by observing their students during instruction, giving them tasks to carry out, or tests to complete in order to gauge knowledge levels. Teachers then (hopefully) adjust their approaches so all students can learn. When organizations like STS work across several schools, however, we answer these questions a bit differently. Instead of gathering “real-time” information in a classroom, we assess children’s reading and math abilities and we observe conditions in their schools and communities over time. STS is doing a baseline survey and hopes to follow up with at least two more assessments in the next two years. By gathering information about multiple time points, STS can identify changes in students’ learning. By triangulating those findings with information on activities and programs, STS can identify factors that may be contributing to these changes. Factors such as improved teachers’ skills, better parental support, availability of instructional materials, and healthy school conditions—all components of the Whole Child Model.
But even if we detect change over time, how do we know these changes are due to WCM interventions? It may be possible, for example, that if STS assessed learning in non-intervention schools, we’d find similar changes. In order to know if changes are due to STS’ interventions, we need to do two things. First, we need to compare results in schools STS support to results in schools that are not receiving WCM interventions. Second, we need to be sure we are not inadvertently selecting schools for interventions that have more favorable conditions, such as better teachers or stronger students. This of course would lead to faulty comparisons (‘of course our schools are performing better—they were stronger to begin with’).
To control for this possibility, researchers use an “experimental design” where they randomly assign participating subjects (in this case, schools) to two groups, treatment and control, then intervene in the treatment group while measuring change over time in both groups. This solves the problem of selecting schools with inherent attributes that might favor the intervention. One problem with experimental designs, though, is that they can be expensive. This year, the WCM will provide professional development for teachers; we bring them together from different schools so they can learn from one another. If we randomly assign schools to treatment and control groups, treatment schools might be large distances apart. This would mean higher travel costs to bring teachers together.
So instead of using an experimental design, STS will use a “quasi-experimental design” where we will select intervention schools based on their similarity and proximity to each other, then identify similar schools to compare results. While we will not be able to claim causality using this design, we can claim correlations wherever we find better learning happening together with WCM interventions. These correlations will provide evidence that suggests the WCM approach is working, and, along with other kinds of data—such as observations and interviews with participating teachers, students, and parents—we can make a strong case for government officials and funders to consider expanding the model to other schools or financing a larger study. As STS continues to do this work, we look forward to sharing the continued efforts and what we’re learning. And most importantly, this evidence will help us understand what is working and allow us to continuously refine our approach so that all children can thrive.