Fed Confession: I Have No Idea How to Measure Performance

Ask EIG is your chance to seek answers to public sector management challenges and conundrums. Submit your questions here.

I hear so much talk about performance metrics. The reality is, I’m not really sure what we’re talking about in these conversations. I do my job and try to ensure my team does theirs, but when it comes to measuring performance, I don’t know where to begin. What have you found to be the best, and most telling, metrics when trying to measure and help improve federal performance?

--Anonymous

Much confusion exists around where and when to use performance metrics.  All too often, performance metrics are used inappropriately to reward or punish, decreasing organizational performance instead of improving it.  To help unpack the confusion and complexity surrounding performance metrics, please consider four types of jobs:

• A-Jobs: involve tasks where all aspects of the job’s output can be measured in an accurate, precise, and objective way.  An example of an A-Job might be operators for the U.S. Postal service who manually key-in postal addresses for letters they read (most letters now are optically scanned).  The job’s output is well defined and can be measured in an accurate, precise, and objective way if letters processed per hour and error rates can be tracked.  Operators need not interact with anyone to perform their tasks.
• B-Jobs: Other tasks, like some of those found in B-Jobs, may not be so easy to measure accurately and precisely.  For instance, analysts at the SEC may be required to review and assess the 10-K reports for public companies.  These reports vary considerably, with some more complex and difficult to analyze than others.  With such variety, managers will face great difficulty in assessing the quality of each analysis.  Moreover, if a requirement of ten analyses-a-day is specified, it will be quite natural for analysts to focus on producing what is easily measured (reports per day) and focus less on those tasks that are inaccurately, imprecisely, and costly to measure, if at all, like the quality of the analysis.
• C-Jobs: Involve interactions with other individuals in ways that, while the team’s output is measureable in an accurate, precise, and objective way, individual contributions are difficult to measure.  For instance, measuring the productivity of any specific TSA agent at airport security may be difficult but the number of passengers, error rate, and complaint rate can be measured for the entire team.
• D-Jobs: The most complicated of the four types.  These jobs have tasks that involve interactions with others but in ways that cannot be measured in an accurate, precise or objective way.  Many types of projects across the federal government satisfy these conditions.

My hope from this four-part classification scheme is that you will realize that appropriate metrics will depend on the specific job and on what the jobholder does to produce a valuable output.  No single metric or set of metrics can be used for all jobs. 

Perhaps less clear is what these four types of jobs can tell us about how to use metrics to improve federal government performance.  For instance, except in those instances where the utmost integrity is needed, research shows that A-Job positions, ones where performance can be completely specified, should be outsourced to non-governmental vendors employing their own workers.  For such jobs, outsourcing typically leads to lower costs as well as greater adaptability because contractors that do not perform well can be readily dropped and others hired. 

The same can be said for C-Jobs.  Outsourcing the entire team typically leads to lower costs because of the ability to adapt to the changing circumstances of time and place.  To understand why, imagine that the federal government hires a team of workers where the output of the team is accurately, precisely and objectively measured.  If they perform below expectations, what can managers do to improve performance?  Dismissing employees from the federal government requires much effort and takes a long time.  Moving workers who do not fit well to new positions may simply pass the poor fit on to another manager.  Such poor fits benefit neither the worker, who is unhappy and unlikely to excel, nor the organization, which suffers low performance and complaints that may contaminate the organization’s culture.  By contrast, if the team is outsourced the government can much more easily adapt by switching vendors or the vendors can adapt by finding workers who are a better fit.

B- and D-Jobs offer a different story.  Outsourcing these positions can lower performance, sometimes dramatically, as workers shift efforts to deliver what is measured and away from tasks that are poorly or not measured.  Switching to another vendor fails to solve this problem.  With employees, managers can provide oversight by assessing the effort (instead of outcome) workers put forward to deliver on both the measureable and difficult to measure tasks.  Alternatively, ongoing education, establishing professional norms of behavior, and developing communities of practice, and other organizational and leadership levers can be used to encourage both high productivity and difficult to measure high quality.

Herein lies the critical rub.  Those jobs (absent accuracy concerns) that should be within government (B and D-Jobs) are precisely the ones where managers, using performance metrics to drive productivity, can cause disastrous results.  It is in these positions that the old saw “what gets measured gets done” lowers performance; focusing on performance metrics can cause workers to not provide the difficult to measure outcomes of high quality, collaboration, or innovation.  As a result, overall performance for these situations will be lower, sometimes dramatically lower, than it could be. 

If you don’t know where to begin developing performance metrics for you and your team then there may be a good reason for it.  You and your team may have B- or D-Jobs.  If so, use your leadership skills to focus on establishing high professional norms, use ongoing education to establish higher expectations of performance, and develop a community of practice that recognizes the importance of measurable as well as non-measurable dimensions of performance.

Duce a mente (may you lead by thinking),

Jackson Nickerson