The Opportunity Agencies Should Seize

Learning agendas are a hot topic these days among federal agencies’ evidence and evaluation staffs. The reason: They’re now required of most large agencies by the Foundations for Evidence-Based Policymaking Act, which was signed into law in January. To get the ball rolling, recent guidance from the White House Office of Management and Budget instructs those agencies to document their progress in developing learning agendas by next month.

What are learning agendas? They’re documents that identify an organization’s priority research questions so that staff can focus evidence-building resources, such as evaluation funding, on their most important issues and challenges. Think of them as strategic plans for research. Examples of research questions might include: Is Program X effective? Which version of Program Y is most cost-efficient? Could Initiative Z be better targeted to those in need?

Once those key research questions have been identified, the idea is to use the most rigorous methods possible to address them, whether through program evaluation, implementation studies, data analysis or other methods. Some questions might be answered quickly using data analysis or rapid experimentation; others might take months or years to answer.

Some federal agencies have already implemented learning agendas, including the Labor Department, the Millennium Challenge Corporation and the Small Business Administration. They have found the agendas to be useful for prioritizing evidence activities and, more broadly, for strengthening a culture of learning and improvement.

For agencies just starting down the path of developing learning agendas, here are several suggestions based on my experience working with agencies that use them.

Clarify the Purpose

As is always good advice in public management, start with purpose. That means being clear about why your agency is developing a learning agenda (beyond OMB’s requirement) and how it will be used. As former Labor Chief Evaluation Officer Demetra Nightingale and her colleagues have noted, important reasons for developing and using a learning agenda include identifying gaps in an agency’s knowledge and conducting evaluation and research to fill them; aligning evidence-building work with the agency’s mission; coordinating that work across the agency; and catalyzing a culture that encourages staff to use their programs’ data and evidence effectively.

Settle on the Process

Next, decide whether the process of developing the learning agenda will be top-down or bottom-up. Top-down means that one entity, such as a chief evaluation office, leads the process, seeking input on research priorities from sub-agencies along the way. A bottom-up approach means requiring sub-agencies to develop their own learning agendas, then rolling those up into a departmentwide document. Each has advantages: A top-down approach is likely to be quicker, while a bottom-up one is more likely to embed learning agendas into the culture of the organization and create broader ownership of them.

Customize Learning Agendas

Be creative in structuring learning agendas to fit the needs of leadership and program staff. For example, the SBA aligns its learning agenda to its strategic goals. Labor requires each of its sub-agencies to “own” their learning agendas. And the Millennium Challenge Corporation has a unique approach. It has separate learning agendas for each of its six priority sectors, such as road investments, agriculture and education. But it also has a learning agenda focused specifically on operational issues, meaning those that relate to how MCC does its work across sectors.

Consider an Emphasis on Innovation

Finally, find ways to spark leaders’ interest in using learning agendas. You could, for example, frame your agency’s broader evidence-based decision-making work, including learning agendas, as a catalyst for innovation within the agency. That turns a common criticism of evaluation and evidence—that it is mainly backward-looking—on its head. It also underscores how evidence-building can support creative new ways to tackle agency challenges by, for example, clarifying where problems exist, encouraging and supporting ongoing experimentation, and providing credible evidence about whether new approaches actually work. At Labor, an informal tagline for the Chief Evaluation Office is “cultivating innovation in government.”

Because of the new evidence law, many more federal agencies now have learning agendas on their radars—and on their to-do lists. The law’s learning-agenda requirement is both a challenge and an opportunity. It is a challenge because anything required by Congress and the White House can easily become a mere compliance exercise. It is an important opportunity, though, because agencies can adapt a very useful tool to fit their own needs and use it to help them tackle high-priority challenges in far more effective ways.

Andrew Feldman is a director in the public sector practice at Grant Thornton and also hosts the Gov Innovator podcast. He served as a special adviser on the evidence team at the White House Office of Management and Budget in the Obama administration.