In evaluation reports, the voices of students can seem secondary to charts of learning gains or tables of project indicators. In the final evaluation of a project targeting adolescent girls in Mali, STS and its partners sought to amplify these often-overlooked voices by using the most significant change (MSC) approach. The resulting evaluation report reinforced the approach’s effectiveness and revealed important lessons for its future use.
Using the Most Significant Change (MSC) Approach in Program Evaluation
The project: Winrock International led a consortium of partners to implement the USAID-funded Girls Leadership and Empowerment through Education (GLEE) project in Mali. The project aimed to decrease adolescent girls’ barriers to accessing quality education, improve their safety, and increase their knowledge and adoption of positive health behaviors.
The approach: At Winrock International’s request, STS partnered with EdIntersect and a research firm in Mali—Centre d’Étude et de Recherche sur l’Information en Population et Santé (CERIPS)—to conduct a mixed-methods performance evaluation of GLEE. One of the evaluation’s primary goals was to listen to and engage with girls as key informants on GLEE’s outcomes.
STS selected the MSC approach because it granted an opportunity for girls themselves to describe how the project had made a difference in their lives.
Focus group discussions included six girls, a facilitator, and a notetaker. First, each girl shared a story detailing the most significant change in her life that resulted from the project, and then the group as a whole selected one story that captured the most significant change out of all the stories shared.
The results: Five of the twelve groups selected a story related to early marriage. Interestingly, GLEE did not implement any activities related to addressing the practice. However, as an indirect result of interventions that encouraged girls to go to school, stay safe, and learn about sexual and reproductive health, girls also were empowered to speak up against early marriage.
In one of the most poignant stories, a 14-year-old ninth grader shared how she intervened in her family. “I made my father sit and told him, ‘Dad, you must not agree to give my sister away in marriage, because she might get sick’,” shared the girl, who discussed the dangers of early pregnancy with her father. “He said he understood.” Later, she shared how she was grateful for the awareness that GLEE raised in her community. “What struck me the most is that if I had not listened carefully to what the project said, my sister was going to make a mistake.”
Lessons learned
MSC creates space for “bottom-up” discussions by listening to girls and uncovers unexpected outcomes. Projects’ results frameworks often guide key evaluation questions and do not leave room for discovering unforeseen impacts, but an approach like MSC can uncover those, as noted in the stories about early marriage.
MSC is time-intensive, but it’s worth the investment. Although a 3- to 4-hour MSC focus group discussion lasted twice as long as a typical 90-minute focus group discussion, the stories that were gathered during the process were worth the effort.
Plan for MSC and these types of approaches from the outset. Fortunately, STS and its partners did not have to convince Winrock International about the efficacy of the MSC approach. Winrock was on board from the beginning about using an approach to highlight girls’ stories.
Learn more about STS’s work with Winrock International, EdIntersect, and CERIPS on the USAID-funded Girls Leadership and Empowerment through Education (GLEE) project in Mali.
