What Mid-Size Cities Can Teach Feds About Performance

Early lessons have begun to emerge from the three-year, $42 million effort launched last year by Bloomberg Philanthropies to help 100 mid-sized cities improve their performance through open data, performance management, evaluation, and results-driven contracting.

Cities are front line implementers for numerous federal programs, including those in transportation, education, health and social services. Now at the halfway point, the project – called What Works Cities – has begun working with 55 cities across the United States representing a combined population of 19 million.

What have they learned? Perhaps one of the most important lessons learned so far has been to start small with a handful of issues that are vitally important to the community and its local leaders.

“Data isn't urgent in and of itself,” said Simone Brody, executive director of the initiative at Results for America. “To make it relevant, it needs to address an actual program or policy in the city.”

Success also requires buy in. “Commitment really matters. If the mayor or senior team of the city isn't really on board, it gets lost. People can move onto other issues,” she said.

Progress across the 55 cities has varied, with most focusing on just one or two of the following:

Open Data

Making a city's inner workings more publicly transparent can make it more accountable for performance. Despite the significant potential of open data, however, cities that launch such initiatives sometimes struggle.

The key to success is citizen engagement, says Stephen Larrick, the project’s open data director at Sunlight Foundation. Simply putting data online is not enough, he said. Without significant engagement, monthly visits to a city’s open data portal can be low to non-existent.

Larrick, whose team has worked with 40 of the project’s cities, says that better results come from talking to local policymakers to determine their highest priorities and then ensuring that the most relevant data sets are prioritized. Citizen interest can also be gauged by talking to local groups and by analyzing public information requests.

Starting small with a limited number of high value data sets can create a constituency and build momentum, he said.  Some of the most commonly chosen data sets include those covering local land use development, brownfields, transportation, and public safety.

Performance Management

Data-driven management can exist side by side with, and often complements, open data. Both require a similar investment in capacity, according to Beth Blauer, executive director at the Johns Hopkins University’s Center for Government Excellence, which is overseeing the project's performance management efforts.

Unlike larger cities like New York or Los Angeles, she says, most mid-sized cities lack the needed capacity. Early stage work usually requires face-to-face conversations about existing IT systems, data inventories, governance, and management practices.

While capacity is an important part of the work, such systems are of little value unless they are used. "We ask what problems do you want us to solve?" said Blauer.

"Managing from data is the most difficult part of the work," she said. "If you don't have leadership like a mayor or city manager reinforcing that this is how things get done, then it becomes really hard. People revert back to decision making based on bureaucracy and anecdotes."

Evaluation

Achieving better outcomes becomes easier when it is accompanied by a better understanding of what produces those outcomes. The Behavioral Insights Team is helping six cities adopt a more rigorous approach to continuous improvement by helping them test improvements against current practice through randomized A/B testing.

The cities have tested a wide variety of innovations, including improvements in hiring practices, bill collection, vehicle registrations, inspections, and utilization of local services. About 75 percent of the tested innovations have proven to be more effective than previous business-as-usual practices, according to Elspeth Kirkman, who is overseeing the effort.

What explains the high success rate? Unlike full scale evaluations that try to determine whether a program is working overall, these evaluations are focused on a large number of incremental changes that can add up over time. "We are testing the small levers that make or break success," she said.

Results-Driven Contracting

Much of what local governments do falls to outside contractors. The Harvard Kennedy School’s Government Performance Lab is helping cities optimize their contracting practices to drive better results.

Common contracting challenges are often quite basic, including such issues as improved vendor selection, procurement reviews, and overall portfolio management, according to Jeffrey Liebman, the Lab's director.  But about one-third of the cities have adopted some form of results-driven contracting.

One success story is in Seattle, which is moving toward results-driven contracting for its homeless services. With assistance from the Lab, the city has set up a pilot for monitoring and tracking results across a subset of its homeless service providers.

Over the first year, the pilot has already identified several key metrics for assessing provider results, including homelessness prevention, housing stability, and moving individuals and families into permanent housing. The pilot has also incorporated risk metrics to ensure that providers are not punished for working with populations that are harder to serve, according to Hanna Azemati, assistant director at the Lab.

The city is using the data to actively manage its providers through monthly progress reports and check-ins. It is not yet using the data to pay providers based on their performance, but it may do so in the future after assessing the potential benefits and challenges, including possible perverse incentives.

The Federal Link

What do these early lessons mean for federal programs? While better data could foster improved local implementation, limited capacity is a common concern.

“We need to help cities build a bench,” said Brody. “Cities need help with professional development.” She cited the Obama administration’s recently launched Police Data Initiative as a possible model for capacity building and shared data.

Standardizing federal reporting requirements could promote greater use of data for management purposes, not just reporting for compliance purposes, said Blauer.

Much of the progress, however, will need to come from the cities themselves. “Mid-sized cities are not Los Angeles, Chicago, or New York,” said Larrick. “But there are a lot of mid-sized cities that actually have incredible capacity and have produced major outcomes. We’ve been able to elevate a whole new fresh set of champions for the work.”

Patrick Lester in the director of the Social Innovation Research Center.