Urban Institute
Research Associate II
Lisette Vegas
Urban Institute
Project Associate

Future of PFS: What does it mean to be "data ready"?

September 7, 2017 - 12:58pm

Pay for success (PFS) projects rely on data to understand the issues facing a community and measure the impact of the chosen intervention. But what does “data ready” look like for a city or a county?

Case Western Reserve University maintains the Childhood Integrated Longitudinal Data (CHILD) System, an integrated data system (IDS) used to support the Cuyahoga County pay for success (PFS) projects, including the Partnering for Family Success project designed to help reunite children in foster care with their parents by addressing parental challenges. The CHILD system gathers administrative data on experiences and outcomes of families living in Cuyahoga County.

To learn more, we spoke to Meghan Salas Atwell, a senior research associate at CWRU’s Center on Urban Poverty and Community Development, about her experiences with CHILD, data sharing, and Cuyahoga County’s pay for success projects.  


PFSI: What are some key components of an IDS like the CHILD system?

Salas Atwell: The Childhood Integrated Longitudinal Data (CHILD) System incorporates administrative data from over 25 different governmental and nonprofit sources to provide information on the experiences and outcomes of children born or living in Cuyahoga County since 1989. Used together, these data inform planning, monitoring, evaluation, and action to improve services to families in Cuyahoga County, Ohio.

PFSI: How is the data in CHILD managed?

Salas Atwell: Strict data use agreements are executed between Case Western Reserve University  and all data providers that explicitly state the expectations of confidentiality and security. Through the specifications in these agreements, the data providers govern the use of their data.

The Institutional Review Board at Case Western Reserve University assures that all data with personally identifiable information (PII) are handled according to the highest standards for data security and protection. They also assure that research conducted using data from the CHILD System comply with all federal protections of human subjects, including those related to privacy and risk.

Additionally, the CHILD System has an Advisory Group that meets on a quarterly basis to represent the collective interests of public agencies and the community.

PFSI: How does the CHILD System support or improve the development and implementation of the Cuyahoga PFS projects?

Salas Atwell: PFS projects are all about innovation, and this means that they often involve the interaction of agencies and systems that may have never interacted before.  Thus, the data from these agencies might exist separately from one another, making PFS project planning and evaluation challenging. For this reason, an IDS has been a key asset for our community.

In the early stages, the IDS is helpful because disparate administrative data sources are matched and linked, which is key to understanding the extent of the problem and the target population. The IDS also is instrumental in measuring an array of outcomes across administrative data sets.  It’s important to find out if any entities in your community have an IDS. Actionable Intelligence for Social Policy is a great resource to learn more about IDS across the nation.

PFSI: What obstacles have you encountered when using the CHILD System in this PFS project and how were they overcome?

Salas Atwell: Administrative data systems were largely designed for case management purposes.  They weren’t designed with the idea that they may, at some point, be integrated into a data system where matching and linking needs to occur. As a result, the administrative data sets that make up an IDS can, at times, be difficult to match to one another: each system may use different unique identifiers (e.g. name, date of birth, or social security number), have a different unit of analysis (e.g. child-level, parent-level, or household-level), and/or might not capture the exact data points relevant to the PFS project. As a result, building a proper IDS requires a good deal of expertise and time.

Our PFS projects required that we integrate new data sources into our existing IDS. Data sharing agreements can be challenging to secure for obvious reasons. Data providers may be reluctant to share data with a third party in the interests of protecting the privacy of their clientele and reducing the burdens placed on their staff tasked with managing their data systems. Building trusting and reciprocal relationships are key. Having patience is also important as these things take time.

PFSI: How did this work in practice for Cuyahoga’s first pay for success project?

Salas Atwell: For our Partnering for Family Success project, it was necessary to identify homeless families with children. The data provided by the Office of Homeless Services through the Homeless Management Information System (HMIS) tracks only those individuals who enter the shelter. Therefore, if a mother enters shelter, but her children do not accompany her, the children are not logged in the system. Thus, we needed to use child-level data from the Department of Child and Family Services (DCFS) to identify children in the foster care system whose parents we could locate in the HMIS data system.

PFSI: What are some recommendations you have for a community that is looking to share data?

Salas Atwell: First, I would canvas your community to get a sense of what kinds of data are being collected and if any of it is already being integrated. If you’ll be the first to integrate data in your community, get a sense of what kinds of data you’ll need for your project and start small. By starting small and getting access to limited data (maybe only for a year or two) from two different relevant providers, you can begin to do some analyses that add value.

Other important details to keep in mind are:

  • Relationships. Nurturing relationships that are reciprocal and trusting with data providers is an essential element in successful data sharing.  Balancing data sharing burdens and concerns with privacy with the value-added to the community from shared data is challenging to achieve but is essential.
  • Reputation. One should also build a reputation as a good data steward so that gathering data in the future is easy and less time consuming because the data provider will know that the data is protected and privacy won’t be breached. By proving that you are a good data steward and the data analysis provides new and important insights, you can start to build on that momentum to strategically enhance your data capabilities.
  • Patience. On average, data sharing agreements can take up to 6 months to establish (some can take years). And, that doesn’t include having the data in-hand which can often take several more months.
  • Strategy. People involved in the project should be strategic. For our PFS preschool feasibility study, it was important for us to have already established a strong data-sharing agreement with the largest school district in the county in order to prove our legitimacy when approaching other, smaller, school districts to share data.

This is the eighth blog in our Future of PFS blog series.

On June 22nd and 23rd, 2017, the Pay for Success Initiative hosted a National Symposium on the Future of Pay for Success in Washington, D.C. The invite-only Symposium brought together leaders from government, nonprofits, and organizations active in pay for success to consider the big questions facing the field, as well as highlight lessons for engaging in PFS efforts. More information on the Symposium can be found here.

Over the next several months, the Initiative will be releasing a series of blogs highlighting important conversations, themes, and questions that arose during the Symposium. To join the conversation, visit pfs.urban.orgfollow @UrbanPFSI and #FutureofPFS on Twitter, and subscribe to our monthly newsletter.

Have a Pay for Success question? Ask our experts here!

As an organization, the Urban Institute does not take positions on issues. Scholars are independent and empowered to share their evidence-based views and recommendations shaped by research. Photo via Shutterstock.