It’s Time to Improve Government’s Use of Metrics and Analytics

The final report by the Commission on Evidence-Based Policymaking promises to refocus attention on the use of performance data. This is not new of course; the use of metrics has evolved over more than two decades across all levels of government. Even with extensive experience, the Commission was created by Congress in early 2016 to “fulfill the promise of evidence-based decision making” “to facilitate program evaluation, continuous improvement, policy-relevant research and cost-benefit analyses.”

The value of metrics and more recently analytics has been proven repeatedly. When used effectively in managing performance, the gains can be impressive. In 2011, the Partnership for Public Service worked with the IBM Center for the Business of Government to develop three excellent reports highlighting success stories, “From Data to Decisions.” Everyone interested in this subject should read the reports.

John Kamensky highlighted a key reason why the commission was needed in a recent column, “Proponents of Evidence-Based Policy Face a Critical Challenge.” He commended the Commission for taking an important step to create “a better supply of data for researchers and policymakers.” That’s clearly important, but it’s only the first step. As he argues, “the next step is huge—getting people to use the data.”  

That’s been a continuing problem; research shows that many public sector organizations collect and report performance data, but then leaders are unwilling to grant decision making authority to lower level staff. The reluctance to devolve decision authority and empower employees is not limited to the use of metrics.

An added problem is the appropriateness of the data for decision making. People cannot be expected to use data effectively if it’s not presented in a straightforward, easy-to-use manner. The many automated tools now available make identifying and tracking possible measures almost too easy. But, the best software will not help to ensure that the right metrics and the right number of metrics have been chosen. To have value, metrics need to be meaningful and central to accomplishing the mission and strategic goals.

There are several potential problems managers should avoid:

• Holding employees accountable for metrics beyond their control
• Relying on performance standards or goals that fail to take into account local operational differences
• Tracking more metrics than can be measured with limited resources  
• Focusing on what’s easy to measure versus what’s important to measure
• Tracking metrics that are peripheral to achieving desired results
• Fostering culture that pressures employees to skew results
• Developing metrics that don’t follow the SMART (Specific, Measurable, Achievable, Relevant and Time-bound) approach, or some similar model
• Using results to blame groups or individuals, instead of helping them find ways to improve
• Failing to follow through in making changes after results have been analyzed
• Failing to measure the impact of the metrics and their use.  

Metrics and Performance

The importance of establishing metrics goes beyond decision making. Appropriate metrics can define performance standards for employees. There are situations in government where metrics determine rewards and can trigger adverse personnel actions. In theory that is a sound practice but it doesn’t work when the data suffer from the problems above. Employees need to understand how metrics are determined; ideally they should be involved in defining them. They also need to believe the performance expectations are fair. Continuing to rely on poorly conceived and managed metrics over time will destroy employee morale.

Another key challenge is that many people lack an aptitude for working with data and various analytical methods. Data analytics can be a powerful tool in understanding trends or measuring performance, but too few senior leaders have the requisite understanding of statistics.

It’s the same problem Brad Pitt had in the movie “Moneyball” when he tried to convince Oakland A’s Manager Phillip Hoffman to rely on Jonah Hill’s statistics. Now metrics are used in all pro sports.

One of the earliest applications of what is now analytics was in the 1980s when demands for pay equity first surfaced. Multiple regression analyses were applied to employment data to measure the impact of discrimination. The practice gained acceptance and was adopted by large employers as a tool for managing salaries but often when a new compensation director did not understand the statistics, it was quickly dropped. Today, human capital analyses are common.

Central to gaining acceptance for analytics is developing an effective approach to communicating the logic and the conclusions to senior management and, when appropriate, employees. Transparency is essential.  If analytics are the purview of only a specialized office, the tools will never be widely used.

This is not the first time a commission was created to promote the collection of data to improve policy-making and performance. In 2010, the National Performance Management Advisory Commission developed a performance management framework for state and local government. The report captured perfectly the purpose in using metrics:

“Performance management comprises the concerted actions an organization takes to apply objective information to management and policy making in order to improve results. Performance management uses evidence from measurement to support governmental planning, funding, and operations.”

Measurement alone accomplishes nothing. The emphasis should be on management and improvement.  The data should be used to identify, assess, build consensus and address problems that impede progress and results. That would benefit all levels of government.

Marc D. Berson is the chair-elect of the American Society for Quality’s Government Division and president of Practical Management Envisioneering LLC. He is a certified Six Sigma Black Belt and Project Management Professional. Mr. Berson has over 30 years of experience supporting all levels of government in addressing performance issues.