The PFS Project Assessment Tool (PAT) helps people answer a fundamental question: What makes for a strong PFS project? It describes core elements of PFS projects, explains why those elements are important, provides a scoring system to help distinguish the strengths and weaknesses of a proposed project, and generates recommendations for improving those weak areas. The PAT is designed for individuals, governments, and organizations working through PFS projects or, even earlier on, simply considering engagement with PFS.
This brief provides a basic overview of evaluation designs to assist pay for success (PFS) stakeholders engaged in deal development. It focuses on comparison and its relation to various designs, and it presents key questions that PFS planners should address as they participate in evaluation design discussions. In PFS projects, strong evaluations are tasked with determining what happened, if the program caused these outcomes, and if outcome payments are triggered.
Pay for success (PFS) projects offer governments opportunities to invest in outcomes and employ new capital to meet the needs of their communities. But PFS projects also carry risks. For investors, the risks relate to the project failing to meet its outcomes or the government reneging on its commitment to pay. Investors’ perceptions of risk matter. Projects with high or unclear risk may discourage investors and prevent the project from launching.
Evaluations are a key feature of pay for success (PFS) projects, and rigorous evaluation designs are important for building the evidence base of effective programs by determining whether a project’s outcomes can be attributed to the program. Randomized controlled trials (RCTs) are considered the most rigorous evaluation design and give us the best approximation for what would have happened without the program. However, PFS stakeholders often don’t know about RCTs or consider them too expensive, difficult, or controversial.
The question at the heart of all PFS projects is whether a social program can measurably improve outcomes for a specific group of people. If the program works—as measured by a rigorous evaluation—investors get their money back, the government realizes cost savings, families and society benefit from better outcomes, and social service providers strengthen the case for funding their model.