Better Data Doesn’t Always Mean Better Decisions, Decision Analysis Does

From the archives of Government Executive...

November/December 1984—There are many types of decision support systems available today with new ones being developed at an astounding rate as executive and managerial decision-making assumes more and more importance. While the advent of the personal computer has made decision support software affordable and approachable, the fact remains that most decision-makers, particularly senior managers and top executives, need something other than more or better data. What they need is greater efficiency in using this newly available data to improve the quality and timeliness of decision-making. With that in mind, managers and executives have turned to a special subset of decision support known as a decision analysis.

Decision analysis itself is not a type of software system, but a philosophy of management decision-making. Its basic assumptions are that whatever data may be available, there will always be room to collect more data, and some necessary information may be unavailable in objective form. Nonetheless, one cannot wait forever to collect perfect information, but must make the best decision possible with data available at the time and with selected criteria data that must be collected within a planned time frame. The inevitable gap in information must then be filled by using the best personal, or subjective, information available from experts—including the decision-maker and his/her staff.

Organizing for Analysis—How does management review policy, planning and budget problems? How are facts presented, analyzed and incorporated into the decision-making process? Are tools available to continue and improve the performance of an office or department? Observation of the situation indicates there is no clear set of tools being used to directly assist management today.

A Decision Analysis Center (DAC), properly configured, solving specific functional requirements, problems and needs is one answer. It may well represent the first true amalgamation of all office automation history in a single, effective tool for the manager to use in decision-making.

The Office of Program Planning and Evaluation (OPPE) in the U.S. Department of Commerce in Washington has the first operational Decision Analysis Center in the federal government. The center has been used by several other departments and is in almost constant use by offices and agencies within Commerce. The center utilizes a Columbia Data Products controlling computer with hard disks, plus twelve other computers—a mix of Compaqs, Columbia Data VPs and IMB PC microprocessors. These are connected in a simple star network with three Compucorp word processors. The network functions to collect and move information from outside agencies, internal offices and outside information sources to produce management assistance information so that Decision Analysis Conferences may be convened.

Existing software packages have been used as much as possible. These include Lotus 1-2-3, PerfectWriter, MicroGANTT, PC-Talk, CONDOR and the DDI Program Tracking Program. By using mostly off-the-shelf software and simple hardware, the center has been able to hold costs to a minimum. An added advantage: training and performance are simple and quickly produce high levels of quality work.

Defining the Problem—As a professional discipline, decision analysis incorporates methods in mathematical modeling, computer science, individual and social psychology, and operations research into a complete framework with a well-established academic basis.

How, philosophically and practically, does the decision analysis process work? The first step is to define the problem with the decision-maker(s). Using a combination of “brainstorming” techniques and preliminary analyses, a trained decision analyst can determine an appropriate mathematical model for the problem, and can recommend a strategy that uses this model to focus attention on the key information needed to make the decision. Some of the important questions to be considered include: what options are available, what considerations would favor one option over another, what are the timing factors involved, and what key risks should cause concern?

In the second step, the actual analysis is performed. Hard data, subjective judgments, and occasionally pure guesses—on a provisional basis when no other information is available—will be plugged into the model. Based on this tentative model, the need for critical data will be determined. At this point, some judgments may be confirmed by checking with available data, while other judgments may be designed for immediate research priorities. All judgments are annotated to indicated their source, their support in fact or personal judgment, and rationale for any otherwise unsupported entries. A computer-based model is constructed as this step proceeds and the results of that model are compared with the participants intuitive ideas and predictions. The result of the computer model is a preliminary decision which, after any required checking of facts, can form a tentative recommendation.  

Usually, unless time is extremely short, decision analysis then proceeds to the third and most important phase. In this phase, the results of the model are subjected to various challenges. These may be registered by participants in the decision process who disagree with one or more judgments, or who simply want to play “devil’s advocate” to test the model’s assumptions. Sometimes they may be one or more unknown data items, which can only be guessed at present. A sensitivity analysis re-computes the model’s results under a variety of alternative data assumptions to determine which data could conceivably vary enough from their assumed values to significantly alter the indicated decision. Based on this analysis, additional options may be identified, or additional factors added to the mode, but eventually the decision-maker(s) can be satisfied that everything has been considered as carefully as possible, and a final decision is made.

What distinguishes decision analysis and related decision analysis software systems from other forms of decision support, including databases, statistical analyses, spreadsheets, etc.,  is the functional, goal-directed nature of the process. Rather than simply providing additional data to the decision-maker, decision analysis provides a definite guide to the steps necessary to arrive at a well thought-out decision based on whatever data is available. Thus, decision analytic software does not compete with other decision support systems; instead, it makes it easier for the manager or executive to use the additional information they provide more effectively.

System Integration—The question for managers or executives in the near future will therefore be “How may I used decision analytic technologies and software with my organization.” Over the years, many organizations have relied on large mainframe computers with intricate databases and highly sophisticated decision support software to aid in the decision-making process. In today’s environment, some decision support systems, such as statistical analyses and spreadsheets, are available at a relatively inexpensive cost to be run on microcomputers. The decision analytic software, which allows the manager to more efficiently use the appropriate data, has not been readily available to date on micros. There are few true decision analytic packages that can be purchased.

The entire involvement in the analytic decision-making process requires an integration of the system, the software and some support which is referred to as facilitation. Generally, a facilitator is a third party, trained decision analyst who conducts the “brainstorming” referred to earlier, which is really a decision analysis conference. During this conference, the facilitator works with the decision-makers to develop the models and parameters. It is the facilitator who then ensures that the appropriate data is fed into the analytic software so that the model can perform as efficiently as possible. The result of such a conference is the recommendation, which the decision-makers can then further refine with additional information.

In organizations where decision analysis conference facilities have been implemented, managers and executives have found numerous applications for this analytic process. Good decision analysis software can assist in such areas as goal and strategic operational planning, policy planning, facility planning, resource forecasting, program plans and objectives, budgeting, policy objectives and issues, as well as performance tracking. The software, further integrated with decision support spreadsheet and database access, allows an organization to more efficiently manage all reports and tracking objectives such as financial management, project tracking, personnel tracking and performance. It provides a more effective tool for organizational planning which covers operational objectives, program or milestone plans, operating budgets, personnel performance, organizational performance plans and reorganizations, as well as any other resource planning activities such as space, travel, and other considerations.

In order to be effectively used by an organization, the decision analytic process must be implemented at the highest levels. The entire process is based on management style and organizational objectives. Organizations which have implemented analytic decision-making concepts have found that a specific facility known as a decision analysis conference room should be installed. Such facilities generally are equipped with the necessary microcomputers and disk storage connected to projection devices to allow the activity of the computer (modeling, spreadsheeting, tracking, etc.) to be viewed by all participants of the conference. In this manner, all members of the organization involved in the decision-making process have an opportunity to view all data simultaneously.

Commerce DAC Operations—At the Decision Analysis Center run by the Department of Commerce, the emphasis is on a two-day conference, with as comfortable and non-distracting surroundings as possible. An important feature of the center is the design of a complete room with soundproofing, white magnetic boards for conference facilitation, electronic overhead projection for computer screen display, staging areas for incoming participants, refreshments on site and a small room to house the microprocessor head and printer to avoid any distraction. No telephones are available at the center.

To put on such a conference, special management, budget or policy information is requested or delivered in advance and files are created to be used during the conference. The facilitator then controls the conference to produce some desired output. While the product is not always the expected return, it is meaningful in relation to the participants and the original goals of the conference. Hard copy documentation is provided upon conclusion of each conference. As each step proceeds, it is documented and provides briefing officers with needed detail for the next level.

A definite advantage of the system is that at each level of conference, a manager stands before his or her immediate supervisor with peer managers equal and in a negotiating situation. This direct confrontation with skillful facilitation and real arbitration is providing the government with a more effective management tool plus a sound, organized, structured policy, planning and budget capability.

The success of a decision analysis conference depends heavily on the interchange and exchange of ideas and concepts among individual decision-makers. Even when hard data is available, people must evaluate and interpret that data. If there is a single individual responsible for making or approving a decision, it is important that this individual be present during the conference. This allows the supporting managers to input their thoughts, ideas and data. Conflicting views can be presented and analyzed, but ultimately the decision must be made about which data goes into the model.

The use of decision analytic technologies and software is going to become more and more prevalent in the coming years. Organizations must recognize that more data does not necessarily mean better decisions. They must also recognize that better supported data does not necessarily result in better decisions. Better decisions are achieved through use of available data and human interpretation of that data, which is then processed by an analytic model. And the result is the most feasible recommendations, well supported by facts, executive inputs and custom computer analysis.

In 1984, William A. Maidens was a manager in the IS/EDP group in Washington, D.C. He had been directly involved in data automation since 1961 and designed and developed major communications software packages.