Meg Massey
Urban Institute
Outreach Manager, Pay for Success
Urban Institute
Principal Policy Associate, Research to Action Lab and Justice Policy Center; Managing Director, Pay for Success Initiative
Justin Milner
Urban Institute
Associate Vice President, Research to Action Lab; Director, Pay for Success Initiative and Evidence-Based Policymaking Collaborative
Urban Institute
Senior Research Associate

PFS + ECE: Evaluation Design

Pay for Success Early Childhood Education Toolkit Report #6
December 6, 2016 - 6:23pm

Evaluation is perhaps the most critical element in a pay for success (PFS) project's design. Evaluation measures the impact of a program on the people it serves. This is important, not just to determine whether the project met the outcome targets that form the basis for repayment, but also to help estimate whether the program itself caused those outcomes, building the underlying evidence base of the intervention. Regardless of whether the intervention achieves its outcome targets, the evaluation gives us the knowledge we need to further understand what works and what does not.

This report describes evaluation design in PFS projects, with a focus on early childhood education (ECE). It is part of a larger toolkit for states, localities, and investors considering early childhood PFS projects and is based in part on stakeholders’ experiences with ongoing PFS projects.

In the first part of this report, we outline the role of evaluation in a PFS project and the importance of considering evaluation design early. The second part of the paper provides a high-level overview of three categories of evaluation design, including randomized controlled trials (RCTs), which are generally considered the most rigorous evaluation design. In the final section, we review considerations for planning an evaluation in the context of ECE programs.

PDF icon Download Report (377.03 KB)