​Designing a Test Worth Teaching To

Today, across the globe, school children are tested—often annually, sometimes more frequently. And one problem with this approach to learning what the children are learning is that the teachers teach to the test: How can we know how well the teachers are really teaching, how can we know how much the students are really learning, if the teachers are always teaching to the test?

But wait! Isn’t that what we want the teachers to do? Isn’t that what we told them to do? Certainly, there is no law or rule commanding: “The teachers shall teach to this test.” Still, the message is quite unambiguous:

“Hey teacher: Pay attention. This test is important. We want every student to learn the concepts on this test. So make sure that all of your kids learn this stuff. If they pass, we know you are a good teacher. If they don’t, we know you aren’t.”

From society, this message to the teacher may be only implicit. From the school principal, however, the message may be quite explicit: “I will evaluate you based on how many of your students pass the test.”

So what will the intelligent teacher do? Indeed, what would you do? Answer: You (and if you were a teacher, you would be a very intelligent one) would teach to the test. This is predictable—very predictable. After all, you, like all teachers, are human.

Moreover, this is precisely the type of classroom behavior we want from teachers. We have told teachers—quite clearly—that they should focus on the ideas, concepts, principles and lessons that will be on the test.

As a society, we have decided (through the political process) that we want our children to learn some important things—be they long division; or Archimedes Principle; or the social, political, and economic causes of World War I; or the insights of Aldous Huxley’s Brave New World. That’s why we put them on the test. Other things—important, but not as important—are not on the test.

So what do we want the teachers to do? We want the teachers to concentrate on what will be on the test—whether these are scientific theories, or historical principles, or literary concepts. And, we want them to teach their students to think analytically and to explain an idea coherently, succinctly and persuasively.

This message is quite explicit. So is another message: Spend less time teaching those things that although still important—are not on the test.

In response, what will the very intelligent teachers do? They will do precisely what we told them to do. They will teach to the test.

From this observation about human behavior, comes the test-design mantra: “The challenge in educational testing is designing a test worth teaching to.”

This challenge does not, however, apply exclusively to education. It applies to all efforts to improve performance. Whenever public officials establish an output measure, or an operational standard, or a performance target, they first need to ask: “Is this a measure, standard, or target worth teaching to?”

Not literally, of course, but metaphorically: Does this capture the purpose that we are trying to achieve? Is this something on which we want everyone to focus? Will this help motivate our desired human behavior?

In addition, they need to ask: Are we willing to accept that everyone will spend less time on other things that, while important, aren’t that important? Because if they aren’t, these officials might want to create a different measure, standard or target.

Unfortunately, no measure, standard or target is perfect. That’s why teachers bristle at the educational tests: “This test does not capture everything that I [and society] want students to learn.” True. Always true.

No test, measure, standard, or target captures everything that we want an organization or individual to do. And, as a corollary to an old management law states: “What doesn’t get measured doesn’t get done.”

Any test, measure, standard, or target concentrates everyone’s attention on some things. In the process, it convinces everyone to ignore other things. No test, measure, standard, or target is perfect. Get over it.

Without, however, some test, measure, standard or target, individuals and organizations will be left to choose for themselves. This might not be bad. It might be disastrous.

It might mean that different sub-units (e.g. different schools or different child welfare offices) will choose to focus on different things—some of which are important and others of which are irrelevant. Or, even worse, it might mean that different sub-units might focus on nothing at all.

People and organizations need “tests.” Not sit-down, pencil-and-paper, fill-in-the-little-ovals tests. They need tests that tell them and others how well they are doing. They need tests to focus their attention.

That is why all public officials—not just educators—need to design tests worth teaching to.

Robert D. Behn, a lecturer at Harvard University's John F. Kennedy School of Government, chairs the executive education program Driving Government Performance: Leadership Strategies that Produce Results. His book, The PerformanceStat Potential, has just been published by Brookings.

Copyright 2014 Robert D. Behn

(Image via archerix/Shutterstock.com)