Building a Culture of Informed Decision Making

A recent article by Ed O’Brien, an associate professor at the University of Chicago’s Booth School of Business, observes that “people assume they can and will use more information to make their decisions than they actually do, according to the research.”

While this is distressing news to those who promote the use of evidence in decision making, it doesn’t have to be that way. There is an effort to actually change the culture within federal agencies to become more evidence-based in decision making. The recently signed Foundations for Evidence-Based Policymaking Act of 2018 creates new momentum for this culture change.

This new law has three distinct sections:

• The first section creates a new capacity for, and governance of, evidence building activities and the use of data. For example, it requires agencies to designate an agency evaluation officer and create a career path or professional occupational job series for evaluators. It also requires agencies to develop evidence-building plans (sometimes called learning agendas) to address key questions policymakers want answered.

• The second section focuses on increasing access to government data. It requires agencies to designate chief data officers, conduct inventories of the data they have, and generally make their data open by default.

• The third section strengthens privacy and confidentiality of agency data by codifying several existing administrative practices to protect against inappropriate disclosure of personal information. But it also encourages data sharing by shifting assumptions from “no access without explicit authorization” to a “presumption of access unless prohibited.”

Shortly after the law was signed, the National Academy of Public Administration hosted a panel to discuss the new law’s implementation. What follows are highlights of that conversation.

Key Implementation Challenges

Of course, the primary locus of implementation of the new law will be at the agency and program levels, but having the top institutions aligned will be important to sustain implementation over a period of years. Experts highlight two key challenges:

• The first is identifying the big policy questions that need to be asked by agencies.  Agencies have to define what they want to know and where the data is located that can answer these questions. The existing annual reviews of agency strategic objectives could be one venue for identifying questions, but some of the questions will reach across agency boundaries and will need cross-agency advocates to raise and answer them.

• A second challenge is coordinating implementation across the various cross-agency management councils (such as the President’s Management Council, the Program Improvement Council, the Chief Information Officers Council, etc.) and program-level executives in the White House Office of Management and Budget as well as within the agencies. Typically, each has different perspectives and portfolios as to what the priorities should be for a common research agenda. Historically, there has been too much infighting over the control of data and turf to craft such agendas, and a new governance structure is needed at multiple levels to provide necessary coordination.

Parallel Centers of Influence

To align the top-level institutions of government around a common agenda and to create an evidence-based decision-making culture, there are three centers of influence that should be leveraged to ensure the law actually changes the culture and doesn’t just become another compliance exercise:

• OMB must play a key role to connect people within agencies across their traditional silos and help ensure the agencies ask questions that matter. OMB is best positioned to spur a new learning culture and establish the necessary supporting systems required to develop and bring evidence to bear at all stages of policy development and implementation. For example, OMB could use the currently ongoing development of the Federal Data Strategy to help bridge institutional silos. It has done this successfully in the past: the Bush Administration’s Program Assessment Rating Tool was a good example of how to bridge the internal OMB silos.
• Congress should reinforce the importance of the law through appropriations committee report language and other means. Authorizing committees could ask agencies how they are identifying the most important questions for which the agencies are seeking answers. Congress also should address overlapping committee jurisdictions to reduce redundancy. This has precedent. A similar coordination effort was undertaken in the mid-1990s in the House via a set of special cross-committee task forces that assessed agencies’ first strategic plans developed under the Government Performance and Results Act.
• Outside stakeholders could help agencies focus on asking the right questions and develop the best evaluation techniques. These include state and local governments; academics and the research community, and private industry. Advocacy groups have a role as well, providing outside pressure to help sustain momentum (this happened with the implementation of the DATA Act).  

Anchoring Change Within Agencies

The agencies are where the work must be done to ensure the law does not become just another set of requirements to implement. If OMB and Congress truly want to institutionalize an evidence-driven culture in agencies, efforts will be needed to connect evidence and data to the management of actual programs, including at the state and local levels. A key piece of this will be the development of agency-level data governance that defines and leverages the roles of the newly-designated agency evaluation officers and chief data officers.

One approach might be for a third party to create a public scorecard for each agency to assess progress and allow agencies to compare themselves with their peers.  The right metrics would capture not just procedural changes but also the successful application of evidence to policy. A possible example is a scorecard developed by the nonprofit Results for America. In that scorecard, the Labor Department is one of the pioneering agencies that has made significant progress in embedding the use of evaluation and evidence into its decision-making process. Scorecards have been used in other program areas to draw attention to an initiative and create a sense of urgency for continued implementation momentum.