Understanding key SIPPRA components: Costs, savings, and evaluations

Welcome back to SIPPRA week! If you are new to this series, be sure to check out Monday’s blog post with FAQs from the NOFA and yesterday’s blog post with a tool for assessing projects.

A shift is underway in how governments make funding decisions and track results. This transition, years in the making, reflects a growing appreciation that outcomes and impact–not inputs and outputs–are most valuable to governments, service providers, and communities at large. This outcomes orientation is a direct result of pioneering efforts of elected officials, committed public servants, community partners, and the emergence of new tools and models that help track and incentivize results.

This shift manifested at the federal level in the passage of the 2018 Social Impact Partnerships to Pay for Results Act (SIPPRA), which provides federal funding to support feasibility studies and project evaluations for outcomes-based or pay for success (PFS) projects.  

After last week’s release of the SIPPRA Notice of Funding Availability (NOFA), we now know that the US Department of Treasury has allocated about $66 million to support outcome payments, with a separate NOFA for feasibility funding to be released, likely later this year. Now, state and local governments less familiar with results-based models need a firm understanding of core elements a project must have to qualify for SIPPRA funding. These elements include:

• Projected federal, state, and local government costs and other costs to conduct the project;
• Projected federal, state, and local government savings and other savings, including a budget impact analysis* to estimate the public-sector savings (decrease in federal outlays) and federal tax receipts (potential increase in revenue) if the project is implemented and the outcomes are achieved as a result of the intervention;
• An evaluation design that must consist of a randomized controlled trial (RCT), or if applicants can make the case that an RCT is not feasible, a quasi-experimental design. Additionally, the evaluator must include an assessment of any changes in federal revenue or spending caused by the SIPPRA intervention.

These elements reflect two important components of most outcomes-focused approaches: a strong understanding of a project’s cost-savings potential and a method to verify the outcomes achieved.

Understanding and estimating project costs and savings

PFS’s salient selling point is its potential to save the government and society money. Though the PFS field has generally shifted away from requiring new projects to meet a net cost savings standard, it is still important for projects to accurately estimate current costs and those of the proposed intervention. This is crucial to accurately scoping a project and determining its feasibility. Understanding potential cost savings can facilitate accurate budgeting, encourage well-designed projects, and secure political support. If you apply for SIPPRA funding, undertaking this step is a requirement.

One key difference between SIPPRA funded projects and typical PFS projects is that ultimately, programs will have to deliver on their outcome goals and demonstrate federal savings as a result of the project in order to have access to repayment via SIPPRA funds. SIPPRA applicants should know that costs and savings can accrue across multiple systems and several funding streams. For example, investments in a housing program (e.g., implementing a program to help low-income residents access stable, affordable housing) can result in marked improvements in downstream health outcomes. But, to qualify for SIPPRA outcome repayment, projects must specifically identify the total value to the federal government through changes in federal revenue and spending.

Making sound cost decisions is equally critical to determining which interventions are feasible, ensuring the project is priced correctly during transaction structuring, assessing whether costs are under control during implementation, and reflecting on the realized costs and benefits after project completion to help determine whether the project should continue.

At the outset, projects should strongly consider undertaking a full cost-benefit analysis to estimate expected federal, state, and local government costs and savings. Cost-benefit analyses include not just monetary costs, but often also monetize the value of nonmonetary outcomes such as improved quality of life or access to additional services beyond the intervention’s objectives. This analysis has the added benefit of being a participatory process, as it requires different project partners to share data and encourages collaboration by bringing more stakeholders to the project planning table.

Evaluation and SIPPRA

SIPPRA’s provision requiring applicants to identify an evaluation design mirrors the PFS model, which builds impact evaluation into the project design and contract. An evaluation determines whether the project has met its outcomes and helps build broader evidence for an intervention model. In PFS, evaluations also trigger outcome payments (if a certain outcome target threshold is achieved).

The Urban Institute has developed a toolkit of resources to help project partners understand the role of evaluations, their options, and elements such as comparison groups and data collection. SIPPRA’s requirement for evaluators to implement an RCT if possible, and if not an RCT, then a quasi-experimental design, aligns with Urban’s recommendations that evaluators should start by considering the most rigorous evaluation design option available and then tailor their strategy based on their unique context and program features.

Bringing these concepts into your SIPPRA application

SIPPRA requires applicants to outline anticipated project costs, potential government cost savings, and plans for evaluating outcomes–among other factors. But this isn’t simply a “check-the-box” exercise. Questions related to costs and evaluation are not only critical to understanding whether a project is feasible, they also get to two potential benefits of social impact projects: they lead to better use of scarce public funds and they generate rigorous evidence on effective programs. Assembling this information and using it to inform decision-making also sets a precedent for other, non-SIPPRA funded, projects.

Tomorrow, the SIPPRA week blog series will feature our early education toolkit, a helpful tool especially for any SIPPRA projects focused on outcomes for children. Friday’s blog post will describe the role of the US Commission on Social Impact Partnerships and the Interagency Council, and finally, the blog series will wrap up next week with an explanation of how the Urban Institute can help sites looking to apply to SIPPRA. Also, join us for a webinar on Thursday, February 28, to learn more about these new federal funding opportunities for PFS projects. Our team will review key takeaways from the SIPPRA NOFA and answer your questions about PFS.

* A tool to assist SIPPRA applicants in their calculations will be available on Treasury’s SIPPRA website.

Have a Pay for Success question? Ask our experts here!

As an organization, the Urban Institute does not take positions on issues. Scholars are independent and empowered to share their evidence-based views and recommendations shaped by research. Photo via Infogram.