Learning Together: Building Stronger Practitioner-Researcher Partnerships

This blog was written by David Fein, a former Principal Associate at Abt.

Promoters of evidence-based policies and practices are seeking to engage practitioners more fully in developing and carrying out technically challenging evaluations — notably randomized controlled trials (RCTs).
 
Evaluation sponsors for some time have recognized that providing supports for logistical aspects of such studies can increase practitioners’ willingness to open up their programs to scrutiny. Lately, the emphasis has broadened — from supporting logistics to encouraging more involvement in research design, interpretation, and dissemination. Advocates see genuine evaluation partnerships as more likely to focus on issues that matter to practitioners and more likely to influence policies and programs.
 
The six years-and-counting collaboration between Abt Global and Year Up – a leading non-profit youth development organization – offers clues for how to build successful partnerships.

A Promising Partnership: Weaving Research Into an Evolving Agenda

The partnership began in 2010, when Year Up agreed to participate in a randomized controlled trial as part of the multi-site Pathways to Advancing Careers and Education (PACE) evaluation. PACE’s primary sponsor, the federal Administration for Children and Families (ACF), put a strong emphasis on engaging participating career pathways programs as partners in this study.
 
Year Up, the largest PACE site, operates its stand-alone core program from its offices in eight major urban areas and each year serves about 2,000 low-income 18-24 year-olds with a high school diploma or equivalent. The program is intensive, providing:

• Six months of training in IT, financial, and other high-demand occupations;
• A six-month internship with a major national firm;
• Extensive support services; and
• Weekly stipends conditioned on adherence to standards for professional behavior. 

Employer payments cover three-fifths of the $27,000 cost for each participant.

Staff from Year Up and Abt forged a close working relationship to implement the PACE experiment. We successfully increased recruitment by 50 percent in all local offices, created and maintained a randomly assigned control group, and arranged for extensive data collection via surveys, administrative records, and field visits. As the relationship deepened, staff from both organizations discussed a wide range of research questions of interest, with each bringing new ideas and questions to the table.
 
These discussions evolved as Year Up turned to next-generation programs capable of reaching low-income youth on a larger scale. In a report titled "Scaling Up to Close the Opportunity Gap¸" we chronicled this shift, distilling lessons from Year Up’s core program and expansion toward college settings. The expansion resulted in the new Professional Training Corps (PTC) model, with pilots underway or planned at more than a dozen colleges nationally.
 
The PTC initiative raises a host of important questions about program design, implementation, and effectiveness. Researchers at Year Up, Abt, and the University of Pennsylvania obtained funding from the Institute for Educational Sciences (IES) and Social Innovation Fund (SIF) for early research on implementation and efficacy of the new model — to first “improve” the programs and then “prove” that they can work, in the words of Year Up’s founder Gerald Chertavian. These ongoing research projects, which include an RCT of overall impact in each site, utilize focused “mini-studies” to address implementation challenges identified by program stakeholders. One such mini-study uses a random assignment design to test improved responses to student academic difficulties in several sites, exemplifying how experiments can be used to both develop improvements and assess overall efficacy at different sites in the same program.
 
Our experience attests to the potential for more significant learning when partnerships extend across multiple generations of promising programs. For Year Up, the research platform will support valuable comparisons of varying implementation challenges, as well as of impacts and cost-effectiveness. Taken together, knowledge from the PACE and IES studies will exceed the sum of the findings on each program.

Factors Contributing to Strong Partnerships

Strong partnerships should engage each partner’s interests and expertise as fully as possible. Four ingredients in the chemistry of the Abt-Year Up partnership are noteworthy:
 
1. Emphasis on partnership at the outset: ACF designed PACE to create a sturdy foundation for extended collaboration, encouraging Abt researchers to engage sites as genuine partners, and providing substantial pass-through funds to support partners’ involvement. This emphasis has paid off in many ways. For example, as ACF support for an evaluation coordinator at Year Up wound down, Year Up institutionalized its internal research capacity by establishing a permanent evaluation department.
 
2. Establishing a foundation for long-term collaboration: Now in its tenth year, PACE has allowed time for practitioners and researchers to get to know each other, share expertise, cultures, and ideas, and build trust and respect. Relationships between the evaluator and the partner were allowed to develop in an incremental and comfortable fashion. While a new program may not immediately be ready to engage in research, exploratory discussions can help both sides begin thinking about possible approaches. The PTC’s start-up challenges left little capacity for Year Up to take on new research projects, but once early challenges were met research planning quickly got underway.
 
3. Qualitative research: Field interviews provided important opportunities for researchers and program staff to get to know one another. Trust builds as practitioners see researchers adhering to strict standards of privacy and confidentiality and mastering the workings of the program. Early feedback on program design and operations from fieldwork can be a valuable benefit for practitioners during the sometimes lengthy wait for major reports.
 
4. External funding: Shared external funding has helped to engage both partners in this joint research enterprise. Well-designed research funding programs can provide valuable structure and help discipline the work that partners undertake together. By insisting on high standards, funders can establish critical parameters for ensuring quality, transparency, and objectivity in research design and reporting. At the other extreme, funders also should consider building capacity to explore new ideas and questions. Such flexibility will help both to stimulate creativity and sustain interest in practitioner-researcher partnerships.

These core elements have led to a true collaboration between Abt and Year Up, one which we believe will yield important benefits for both of us, our agency partners, and the field for years to come.