Urban Institute
Distinguished Fellow
Urban Institute
Research Assistant

Learning how the PFS intervention affects demographic subgroups

Series: Using Your Pay for Success Project to Capture Additional Lessons
January 11, 2018 - 11:26am

PFS projects provide lessons for government that extend beyond the project itself. The knowledge gained can have substantial value even if the intervention is falls short of its outcome targets. This blog is the second of a three-part series that explores such opportunities to learn:

  1. about other service qualities of importance to clients,
  2. how the intervention affects various client demographic subgroups, and
  3. from changes in service delivery procedures during implementation.

This blog describes the second opportunity. 

In most pay for success (PFS) projects to date, the contract already identifies specific demographic groups as the target population for the project. However, not every individual will react to an intervention in the same way. Outcomes can differ, for example, for service recipients of different genders, ages, races/ethnicities, and educational levels. Different subgroups of a target population have different needs and will present providers with varying levels of difficulty in achieving targeted outcomes.

Disaggregating outcomes based on these differences can uncover potential demographic patterns and may be done in addition to collecting aggregate findings on outcomes that trigger success payments. Although no PFS projects to date, to our knowledge, have done this, success payment rates could also be correlated to characteristics like case difficulty: higher payment rates could be established for more disadvantaged subgroups or individuals.

A hypothetical example is provided in Table 1 below. It compares data for a fictional PFS project incorporating an employment training program for individuals recently released from prison. Disaggregating the outcomes data reveals that the outcome (percent gaining employment) varied substantially based on education level at program entry. These data suggest that individuals with less education may need additional training, which can inform future iterations of the program. 

Table 1

Employment Outcomes: Percent Employed by Educational Level at Program Entry and Program Type

Information on the relationship between service recipient characteristics and project outcomes is likely to be very useful to providers and government. Collecting this information

  • is basic to improving service delivery for population subgroups,
  • can protect against service providers focusing efforts on the easiest-to-help service recipients in order to achieve aggregate outcome targets,
  • is an important way to assess the fairness of the service being provided, and
  • provides in-depth information to service delivery units on who their service recipients are.

Obtaining this demographic information would require added effort but little actual cost. Much of the information on each service recipient might already be contained in program records. Alternatively, it is possible that government agencies have collected administrative data on service recipients in the past. In cases where there is no existing demographic information, providers would need to develop a data collection process. This aspect of the project would have to be incorporated into the PFS contract, along with any added expenses. Such a process would need to be conducted in consultation with the PFS project evaluator, who would then use the data to disaggregate outcomes.

The project evaluator can link each recipient’s outcome information to her or his demographic characteristics so that the outcome data for each subgroup can be separately tabulated and examined.  Modern statistical software enables such tabulations to be made both quickly and inexpensively. These additional insights can then assist the service provider in better serving specific recipients. 

Have a Pay for Success question? Ask our experts here!

As an organization, the Urban Institute does not take positions on issues. Scholars are independent and empowered to share their evidence-based views and recommendations shaped by research. Photo via Shutterstock.