Comprehensive Evaluation of Roca Inc.
Highlights
- What influences program implementation and impact after adaptation and scale-up?
- We are conducting a comprehensive evaluation of Roca, Inc.’s program.
- Roca’s model is adaptable and helps the people it serves.
Roca, Inc., a non-profit organization, aims to foster behavior change and improve outcomes for high-risk young adults. Recently, Roca has expanded its program throughout Massachusetts and to Baltimore, MD, and introduced a behavior change curriculum to its model that is grounded in cognitive behavioral theory (CBT). Roca contracted with Abt to conduct two implementation and outcome evaluations: one of the CBT curriculum and one of the intervention model it uses.
For these evaluations, Abt used a combination of program, qualitative, and survey data to evaluate Roca model implementation at each of the five Roca sites; whether participant outcomes vary across and within Roca sites; and how outcomes are related to program implementation. For CBT in particular, Abt assessed CBT’s impact on intended outcomes, such as changes in program engagement, employment and arrests. Abt also assessed adherence to the curriculum as designed.
Roca embeds the core aspects of its model — relentless outreach, transformational relationships, stage-based programming — in the culture and front-line operations at each site. We did observe variation within and across sites in the implementation of those core elements, demonstrating the flexibility of the model to adapt to fit the needs of local contexts.
Roca participants experienced improved outcomes: reduced recidivism for violent offenses and increased likelihood of unsubsidized employment. More CBT training in the first three months led to greater participation in the program, and CBT made it more likely participants would land a job and have fewer arrests.
Learn More: