Afterschool Programs: Better Data, Better Outcomes

With millions of children attending hundreds of afterschool programs, the opportunities to make a positive impact are enormous. But with so many program options, state decision makers need high-quality information on what works.  As a co-author of the Afterschool Programs: A Review of Evidence Under the Every Student Succeeds Act with Ruth Neild and Wendy McClanahan, I had the opportunity to review virtually all of the literature on what works in afterschool programs.  We defined “what works” using Tiers I-IV of the Every Student Succeeds Act’s (ESSA) standards for evidence, and identified more than 60 programs that met the standards set by the top three tiers. This tells us that the standards are workable, but we also learned that we have a ways to go before this still-new approach is applied to—and validates, where appropriate—the many afterschool programs that are active around the country.

As we work to further refine the program assessments and identify new programs that can provide successful outcomes for kids, I’d like to share some of the advice we came up with while conducting this study for state policymakers.

Don’t Let Perfect Be the Enemy of the Good … Yet
Despite my observations above, we concluded that states are justified in relying on evidence from all four tiers (e.g., Tiers I and IV) until more programs have provided evidence that meet Tiers I-II standards. ESSA allows states to determine what level of evidence is required for funding. We cited the following example in the our report: “[A]lthough 20 afterschool programs meet Tier I-II research design requirements and improved at least one student outcome, only five programs meet these tiers when all the Department’s recommendations, such as minimum study sample size, are applied.” Rather than wait until more Tier I and Tier II programs are identified, we suggest moving forward with programs that have demonstrated at least promising results.

Local Solutions to Local Problems
The above is a temporary workaround. How do we build an evidence base with the larger sample sizes—and across multiple sites—that the Department of Education recommends? Adopt a local solution to solve a local problem. We recommend that state departments of education work with school districts to identify their priorities and then develop evaluations that are tailored to those priorities. Working across the districts, states could study multiple variations of different programs with larger samples, garnering the recommended breadth and depth.

Supporting the Effort
Finally, states should work to ensure these multiple evaluations across districts meet Tier I and Tier II standards and encompass the Department of Education’s recommendations. To help meet that goal, funding needs to be adequate, but I believe that investment will be rewarded. The promising results we have thus far tell us we can provide better programs with better outcomes for students if we support the pursuit of better data.

While this post offers insights for states, the full report includes recommendations for afterschool program providers, federal agencies and evaluators; click here.