By Mindy Hightower King, Ph.D., Limelight Analytics

Most nonprofit organizations are experienced at measuring and reporting who they serve and the types of services they deliver. These indicators are commonly referred to as outputs. Nonprofits are also encouraged to report outcomes, which can often be more challenging to measure. With limited time and resources to devote to evaluation, it may be tempting to focus more on one of these types of measures than the other. However, interpreting outcome data requires that organizations also measure and understand their outputs. Here’s an example of why both of these measures are necessary, but not individually sufficient, to understand the impact of a program.

Outputs v. Outcomes

Outputs represent critical measures of service delivery for nonprofit organizations. They can include the number and characteristics of those served, how often they participate, and the types of support that are delivered.

Outcomes represent changes in knowledge, attitudes, behaviors, or conditions that are thought to be a result of the program. They describe how individuals and/or families benefit from the service/support provided.

Suppose your organization runs an early childhood education center that offers childcare to single parents looking for stable employment. A strategic priority of your organization might be for children who attend this childcare center to make progress toward developmental milestones. Making “progress toward developmental milestones” is an outcome. It represents a benefit of attending the childcare center, and it can be measured using a number of validated measurement tools.

At the end of the fiscal year, your Director of Programs analyzes the data on developmental milestones, and the results show very little progress has been made. Is the program a failure? Does the childcare director need to hire new staff or reassess the curriculum? The answer is – not yet. In order to thoroughly understand and interpret these outcome data, you need to consider them within the context of relevant outputs. In this case, the most important outputs likely include the length of time children have been attending the childcare center and how frequently they attend. It’s very reasonable to expect that, in order to benefit from a high-quality early learning center, young children need to consistently attend over a sustained period of time. Children who may have attended for only a month or two are not likely to show the same level of developmental progress compared to those who have been attending for a year or more. In this way, it’s important to track and understand children’s attendance patterns—which are outputs.

Measurement of these outputs could hypothetically show that only one-third of the children attending the childcare center have been there for six months or more. It’s reasonable to expect that children who have been attending consistently for at least six months will show developmental progress. Therefore, outcome data for those children who meet this attendance threshold (six months) could be analyzed to better understand the benefits of consistent and ongoing attendance.

In this example, I present a scenario that demonstrates how reporting outcomes for some but not all program participants can more accurately represent the impact of a program than reporting outcomes for all participants. Some will surely ask, “Isn’t this cherry picking?” Is it fair to just consider some of the participants when we are trying to understand the overall benefit of a program? I would argue that it’s the ONLY way to effectively understand outcomes.

We all know that participants often have very different levels of engagement with programs and nonprofit organizations. If this is the case, then why should we expect them all to experience the same benefits? Instead, program directors and staff are well served by thinking critically and defining what constitutes a meaningful level of engagement with their program in order to benefit from it. Using the childcare example above, it may be that children who attend the childcare center at least three times a week for at least six consecutive months can be expected to benefit from the program. If this is the case, then the organization can most accurately represent its impact by reporting outcomes for this group of participants.

Does this mean an organization should ignore data from children who don’t meet that threshold of engagement with their program? Not at all. It’s very important to understand relevant outputs related to program engagement. In fact, for many programs, consistent participation or engagement is necessary for participants to benefit (e.g., experience the expected outcomes). For these programs, it’s imperative that staff find ways to encourage more consistent and long-term participation in order to maximize benefits for participants. Measurement of and reflection on key outputs related to program engagement are critical to setting expectations and then understanding outcome data.

Mindy Hightower King, Ph.D., is the owner of Limelight Analytics, an evaluation and measurement firm in Bloomington, Indiana. She has conducted evaluation and performance measurement for schools, nonprofit organizations, and foundations for the past 22 years. Dr. King holds adjunct teaching appointments at Indiana University, Bloomington in the O’Neill School of Public and Environmental Affairs and the Hamilton-Lugar School of Global and International Studies.