Don't Go Down That Rabbit Hole, Define Your Outcomes First

“Would you tell me, please, which way I ought to go from here?”

“That depends a good deal on where you want to go,” said the Cat.

“I don’t much care where,” said Alice.

“Then it doesn’t matter which way you go," said the Cat.

—Lewis Carroll, Alice in Wonderland

As the Cheshire Cat tells us, if you don’t know where you are going, any path will get you there. This suggests that if you are not clear about what you, your agency and your team are to “produce”—what the clear outcomes of your work are to be—then it is OK for staff to spend their time on whatever their current practices or preferences of work might be. This leads to the perspective that work—any work— is good, whether it is making a needed difference or not. What we need in our government is a focus on outcomes—the value delivered to the American people by a high-performing federal workforce.

If we consider the importance of outcomes, our search engine steers us quickly toward the field of health care. Myriad results discuss global health, clinical trials, infections, epidemics and emergency services. In medical parlance, measuring outcomes is critical to the effective clinical treatment of disease, which leads to healthy patients. Applying this theory to an organization, rather than an organism, doesn’t change the theory—healthy organizations measure outcomes.

Defining Outcomes

It is important to start with an understanding of the difference between outputs and outcomes. Outputs are the goods or services produced by programs or agencies, whereas outcomes can be defined as the impact that those outputs have on social, economic or other indicators. Outputs are often easy to measure and track—you can count them up—but they simply reflect the fact that somebody is working. To determine the value of a program or project, and to improve the efficiency and effectiveness with which it is delivered, it is essential to measure outcomes.

Outcomes are not activity-based, such as “conduct five tax workshops” or “implement a paperless admissions process at 20 hospitals” or “develop a new pest-testing protocol.” None of these statistics reflect results achieved and therefore cannot inform funding and budget decisions, which should be based on the value of the project, not the activities or products of work delivered.

To develop outcomes, program personnel must first define success—the specific, measurable and meaningful results expected of the program. Outcomes can be something the project wants either to maximize, such as evidence of “increased learning by workshop participants” or minimize, such as “reduce pest damage to fruit.” Some outcomes can be measured with financial indicators. For example, by implementing paperless admissions, the personnel and material costs associated with hospital admissions will be reduced. The outcome, however, isn’t just a cost-savings, it is a more efficient and effective admissions process that is faster for patients, results in fewer errors and improves the patient experience.

Choosing the Right Outcomes

As critical as it is to shift focus from measuring metrics to measuring outcomes, it is even more important to choose the right outcomes. Choosing the right outcomes requires an understanding of the overall agency mission, how the program supports the mission, and what success looks like. What happens when you choose the wrong outcomes? A law enforcement agency had, as part of its duties, the responsibility of moving prisoners between jail and the courthouse. The cost of maintaining the bus fleet, providing drivers and guards, and shuffling the prisoners around was high. The agency found that by hiring an outside security firm, it could manage prisoner transport at a much lower cost. The outcome was measured in cost-savings. A few years later, the agency discovered that the ability to carry out its primary mission of fugitive apprehension had been compromised. It turned out prisoner transport was one way law enforcement personnel kept their skills current. Measuring the right outcome–increasing the skills and abilities of law enforcement personnel to apprehend fugitives—would have resulted in a different set of decisions around prisoner transport. Outsourcing the activity reduced costs, but resulted in lost skills needed to execute the mission.

Metrics Versus Outcomes

To ensure you measure what matters:

1. Transition from measuring activity to also measuring value

2. Measure contribution, not attribution

3. Use metrics focusing on business and development outcomes, the value of the project and effective implementation.

Measurement in government is often an act of compliance—counting up the work plan activities completed and monitoring how funds are spent. Ensuring that adequate progress is made on the project and funds are accounted for and expended responsibly is an important aspect of performance reporting, but to demonstrate impact and value of a project requires another type of measurement. Agencies have a responsibility to track funds and activities, but they also have a responsibility to communicate value through outcome measurement.

Moving to a performance-based culture that measures outcomes rather than an accountability-based culture that measures metrics is critical. An accountability-oriented measurement system asks, “Did it happen?” It tracks activities and dollars. Data is collected for transparency and rarely revisited.

Performance-oriented measurement attempts to prove theories of change, asking, “Did it work?” and “How well is it working?” It is a value-oriented approach that captures outcomes and progress toward long-term and systemic change and uses data to improve results and demonstrate value delivered.

Outcomes must be specific, measurable and meaningful. They should be the foundation upon which dollars are prioritized and allocated within our federal budget. Otherwise, like poor Alice, you’ve just spent a lot of money to fall down another rabbit hole.

Lyn McGee is vice president at the Center for Organizational Excellence Inc., directing client solutions, business development and marketing functions. 

(Image via Evoken/Shutterstock.com)