When Evaluating EdTech Interventions

Our field knows that girls across the globe have greater difficulty than boys in accessing quality education. Today’s International Day of the Girl—commemorated with the theme Digital Generation. Our Generation.—highlights the technological gender gap.

Girls have more difficulty accessing the internet, using and owning fewer devices, and are less likely to pursue careers in tech.¹ Many education groups have made efforts to engage girls in tech from an early age, incorporating tech and apps into interventions. But what do we know about how effective such interventions are in closing the digital gender gap?

To mark this year’s #DayoftheGirl, we want to share some considerations STS uses for evaluating EdTech interventions—and some tech we think is beneficial for assessing girls’ education outcomes.

When evaluating EdTech interventions, we consider the following questions with a lens of gender equality and social inclusion (GESI).

• What are gender norms around girls and technology in the context that we will be evaluating? How might this have affected the intervention?
• To what extent did the intervention incorporate the minimum gender standards?²
• Did the intervention address barriers to girls’ access to, control over, and benefits from tech resources?
• Did the intervention increase girls’ understanding of their rights and their ability to determine their life outcomes? Did it also influence their decision-making?

We are also making efforts to incorporate tech innovations that improve our research. Some promising new approaches for evaluating interventions and programs that target girls include chatbots, participatory technology, and data collection led by girls.

• Interventions have mainly used chatbots to disseminate information because they harness the reach of social media. Chatbots also have great potential as survey tools. Girls may feel more comfortable sharing information about sensitive topics and behaviors with a chatbot than with an enumerator.
• Participatory technology could be useful in evaluating girls’ education interventions. Girls may feel more empowered by using these techniques to provide data. In turn, the findings may be richer and more nuanced because they capture girls’ experiences in their own voices.
• Finally, data collection led by girls provides a unique way for girls to participate in evaluations. When girls collect data from their peers, they can gather real-time insights into their counterparts’ lives and shape evaluations’ priorities.

Asking questions about EdTech’s impact on girls enables us (as external evaluators) to do our job more effectively. It also creates opportunities that can build girls’ confidence, increase their exposure to technology, and ultimately reduce the digital gender gap.

Beneficial Tech Innovations