Which of the following is an effectiveness objective of the analytics process?

Advances in computer-based information technology in recent years have led to a wide variety of systems that managers are now using to make and implement decisions. By and large, these systems have been developed from scratch for specific purposes and differ significantly from standard electronic data processing systems. Too often, unfortunately, managers have little say in the development of these decision support sysems; at the same time, non-managers who do develop them have a limited view of how they can be used. In spite of these drawbacks, the author found that a number of the 56 systems he studied are successful. And the difference between success and failure is the extent to which managers can use the system to increase their effectiveness within their organizations. Thus, the author suggests that this is the criterion designers and managers should jointly ascribe to in exploiting the capabilities of today’s technologies.

What can managers realistically expect from computers other than a pile of reports a foot deep dumped on their desks every other week?

Everyone knows, for instance, that computers are great at listing receivables. But what about all the promises and all the speculations over the past few decades about the role of the computer in management? While there have been advances in basic information retrieval, processing, and display technologies, my recent study of 56 computerized decision support systems confirms the common wisdom that very few management functions have actually been automated to date and all indications are that most cannot be.

Instead, my findings show what other researchers have reported: applications are being developed and used to support the manager responsible for making and implementing decisions, rather than to replace him. In other words, people in a growing number of organizations are using what are often called decision support systems to improve their managerial effectiveness.1

Unfortunately my research also bore out the fact that while more and more practical applications are being developed for the use of decision makers, three sizable stumbling blocks still stand in the way of others who might benefit from them.

First, managers and computer users in many organizations are familiar with only a few of the types of systems now in use. As a result, different types of innovative systems have often been conceived and nurtured by internal or external “entrepreneurs,” not by the system users or their superiors.

Second, and closely related to my first finding, these entrepreneurs tend to concentrate on technical characteristics. Too often, this myopia means that they fail to anticipate the ways in which such systems can be used to increase the effectiveness of individuals in organizations.

Finally, highly innovative systems—the very ones management should find most useful—run a high risk of never being implemented, especially when the impetus for change comes from a source other than the potential user.

Quite simply, my purpose in this article is to discuss, without getting into the technology involved, the high potential of a variety of decision support systems, the challenges and risks they pose to managers and implementers, and a wide range of strategies to meet these challenges and risks.

Types of Decision Support Systems

While there are many ways to categorize computer systems, a practical one is to compare them in terms of what the user does with them:

• Retrieves isolated data items.

• Uses as a mechanism for ad hoc analysis of data files.

• Obtains prespecified aggregations of data in the form of standard reports.

• Estimates the consequences of proposed decisions.

As Exhibit I indicates, EDP reporting systems usually perform only the third function in this list of operations, which I have organized along a dimension from “data-orientation” to “model-orientation.” Hence, unlike the EDP user who receives standard reports on a periodic basis, the decision support system user typically initiates each instance of system usage, either directly or through a staff intermediary.

Exhibit I Comparison of Uses, Purposes, and Characteristics of EDP Systems vs. Decision Support Systems

Although decision-oriented reporting systems often grow out of standard EDP systems, I will concentrate on seven distinct types, briefly describing one example of each type.

Incidently, it is interesting to note that external consultants developed the systems cited in my second, fifth, and seventh examples, while those of the first, third, and sixth were the creations of people acting as internal entrepreneurs through staff roles; only the fourth system was developed on direct assignment by the user. This same pattern of initiation of innovative systems by people other than the users was present in many of the 56 systems.

1. Retrieval only—a shop floor information system.

In order to help production foremen improve the percentage yield on a newly developed 50—stage process for manufacturing micro-circuits, the management of one company has installed an on-line, shop floor information system. Operators submit daily piecework reports, which include yield, release date, identification of the person who does the work, and so on. The foremen then juggle this information to obtain productivity data by operation, operator, machine, and lot.

Thus they are able to use the system in a number of ways. They can monitor work flow, pinpoint yield problems, and settle day-to-day questions such as who worked on which lot when, and which operators are ahead of or behind schedule, or below standards. The foremen have 13 standard commands by which they can retrieve the data stored in the system and display them on a cathode ray tube terminal. The commands permit them to tailor reports to their needs.

2. Retrieval and analysis—a portfolio analysis system.

Before advising clients or making authorized trading decisions, the portfolio managers at a bank I studied use an on-line system to analyze individual portfolios. The managers can bypass time-consuming manual methods and obtain up-to-date and clearly organized portfolio information in either graphic or tabular form.

Depending on the situation, a manager can inspect both individual portfolios and groups of portfolios from different viewpoints—for example, rank them in different ways, obtain breakdowns by industry or risk level, and so on. With this kind of flexibility, the bank’s portfolio managers make more effective use of a vast amount of information, most of which had existed prior to the system, but had been accessible only through tedious manual analysis.

3. Multiple data bases plus analysis—sales information systems.

Greater flexibility was also the reason that two consumer products companies and one manufacturing company I looked at developed sales information systems which are quite similar. Standard EDP functions were too inflexible to produce ad hoc sales analysis reports in a timely and cost-effective manner for those in the companies’ marketing and planning areas. In each case, information extracted from the EDP systems is now maintained separately in order to have it handy and, in two instances, to be able to analyze it in conjunction with externally purchased proprietary data bases and models.

Basically, each system is a vehicle by which a staff man or group tries to help decision makers. Their modus operandi is incremental: identify a problem; bring the current system and existing expertise to bear on it; develop a solution in the form of an analysis or additional system module; and incorporate the results into an expanded version of the system.

4. Evaluating decisions using an accounting model—a source-and-application-of-funds budget.

To expedite operational decision making and financial planning over a two-year horizon, an insurance company is using an on-line, source-and-application-of-funds budget system. Inputs are projections of future business levels in various lines of insurance and investment areas, plus assumptions concerning important numbers such as future money-market rates. The output is a projected overall cash flow by month.

An investment committee uses the model to allocate funds across investment areas and to minimize the amount of cash left idle in banks. The committee compares projected cash flows based on different allocation decisions; the decisions that it actually adopts are those that produce adequate projected cash flows and that are acceptable to the various groups in the company.

Actually, the system is an accounting definition of the company. There is no question about the accuracy of the relationships in the model, so the only way projected results can be in error is if estimates of business activity levels or money market rates are incorrect.

5. Evaluating decisions using a simulation model—a marketing decision system.

In order to provide a more rational basis for repetitive marketing decisions, a consumer products company uses a model that relates levels of advertising, promotions, and pricing to levels of sales for a particular brand. The model was developed in a team setting by reconciling an analysis of historical brand information with an individual’s subjective feelings concerning the effects on sales of various levels and types of advertising and other marketing actions.

The model was validated by tracking its accuracy in predicting sales based on the competitive actions that were taken. Unlike the accounting model I just mentioned, this is a simulation model in which some of the most important relationships are estimates at best. For instance there simply is no rule by which it is possible to predict sales with certainty based on advertising levels. In fact, this was the heart of the issue in developing the model.

Even though it has turned out to be useful for prediction, much of the value of the model lies in the company’s improved understanding of the market environment.

6. Proposing decisions—optimization of raw materials usage.

Another consumer products company, faced with short-run supply problems for many of its raw materials, has developed an optimization model to solve the mathematical puzzle of choosing and balancing among various product recipes.

The inputs to the model include a series of different recipes for many products, short-run supply levels for raw materials, and production requirements for finished products. The output is the choice of recipes that maximizes production using existing supplies. When the short-run supply situation shifts, the model can be revised and a new set of recipes chosen.

The system has had a major impact on the way managers view allocation policy. Initially, they considered allocating scarce raw materials to products by setting priorities among products. The model showed that it was more advantageous to start with production requirements and then allocate scarce resources by optimizing the mix of product recipes.

7. Making decisions—an insurance renewal rate system.

As an outgrowth of an overhaul of its group insurance information system, an insurance company has developed a system to eliminate part of the clerical burden associated with renewal underwriting and to help assure that rate calculations are consistent and accurate.

Instead of calculating renewal rates by hand, underwriters fill out coded input sheets for the system, which calculates a renewal rate based on a series of standard statistical and actuarial assumptions. Since these assumptions might or might not apply to a particular policy, the underwriters review documentation accompanying the policies and decide whether the standard calculations are applicable. If they are not, the coding sheet is modified in an appropriate manner and resubmitted.

In effect, the system makes the decision in completely standard situations, while the underwriter decides whether the situation is standard and, if not, what adjustments are required. As a result, the underwriters can concentrate on the substance of their jobs rather than the related clerical chores.

Spectrum of possibilities

These seven systems represent a wide range of approaches in supporting decisions. The first one helps production foremen by simply providing rapid access to historical information such as who worked on what lot, and when the work was done. But the foremen must decide what should be done once they have the information. At the other extreme, the system supporting the underwriters virtually makes the decision in some cases. Between the two extremes, analysis systems and model-oriented systems help people organize information and also facilitate and formalize the evaluation of proposed decisions.

Although managers in most large companies have used budgeting or planning systems similar to the source-and-application-of-funds model I mentioned, the spectrum of possibilities for other kinds of decision support systems is surprisingly wide. Obviously, some of these systems are of no particular use in many settings. Still, their variety suggests that most companies should have a number of genuine opportunities for applying the concept of computerbased support for decision making.

Motives of Managers

What do decision support systems do that actually helps their users? What is their real impact? In my survey, answers to these questions proved elusive in many cases since the users valued the systems for reasons that were completely different from initial ideas of what the systems were to accomplish. In fact, a wide range of purposes exists for these systems. While many decision support systems share the goals of standard EDP systems, they go further and address other managerial concerns such as improving interpersonal communication, facilitating problem solving, fostering individual learning, and increasing organizational control.

Such systems can affect interpersonal communication in two ways: by providing individuals with tools for persuasion and by providing organizations with a vocabulary and a discipline which facilitates negotiations across subunit boundaries.

Tools of persuasion

Standard texts on systems analysis totally overlook the personal use of decision support systems as tools of persuasion. But consider the following “offensive” (persuading someone else to do something) uses to which various companies have put these systems:

• The manager of a chemical plant was attempting to meet output goals (quantities by product) that were being set by a marketing group. Unfortunately, the group was setting goals without much consideration of raw material shortages under which the plant was operating. The plant had been using a model to calculate production mixes.

At one point, it occurred to the plant manager that he could use this model to investigate whether marketing was setting goals that resulted in poor plant utilization and made him appear inefficient. As he ran the model under a series of different production mix goals, it became clear that this was the case, and he used the results to persuade marketing to change his plant’s production mix.

• A data retrieval and manipulation system first received wide exposure in a transportation company when a number of the company’s top executives used it to develop a good quantitative rationale for a proposed merger. With the system, it was possible to explore and manipulate a large data base of information on the industry.

Although the merger was not approved, management thought that the system helped it put up a good fight.

• The management of a shipping company found that a system it used in consolidating and fine-tuning strategic investment plans also helped it negotiate with banks. The banks and other sources of financing seemed to be uniformly impressed by the careful computer-based analysis on which management based its financing requests. The resulting edge in credibility was small, but, in management’s opinion, noticeable.

Now that we have seen illustrations of the offensive tools of persuasion, let us turn to examples of the “defensive” (persuading someone that the user has done a good job) uses of these systems:

• When asked whether he ever made direct use of a case tracking system, the head of an adjudication group in a government regulatory agency said that he remembered only one instance. This was when he spent a lunch hour generating a report to make his group’s recent performance appear as favorable as possible in spite of some unfortunate delays and problems that made the standard report look bad.

• The new president of a large conglomerate used a one-year budgeting model to leam about the budgeting choices that existed, as well as to help him discount what people in various areas claimed concerning their own budgetary needs.

• The class scheduler of a training school for a company’s service personnel had found his job frustrating because it was always difficult to justify the budget on explicit grounds.

With a model that generated optimal training schedules, the scheduler could protect himself very easily by saying: “Using these assumptions concerning attrition, acceptable peak-time shortfalls, and other considerations, this is the best budget. If you (the budget cutter) would like me to change these assumptions, I would be glad to generate a new budget. What level of shortfall do you suggest?” Thus the system not only helped the scheduler make decisions, but also helped him defend them.

• Many people suspected that a new product venture in a consumer company might not be worthwhile, but no one knew exactly why. When a risk analysis was carried out with a model, the reason became clear: the venture had a very substantial downside risk. In addition to sealing the decision, the analysis provided an understandable response to the people who had proposed the venture.

A cynic might contend that the people in these situations were taking advantage of or abusing the systems. A more practical conclusion is that these systems simply serve to improve managers’ effectiveness in their organizations by helping them communicate with other people. My point is that much of the benefit of many of the decision support systems in my sample was of this sort.

Aids to communication

Decision support systems also help managers negotiate across organizational units by standardizing the mechanics of the process and by providing a common conceptual basis for decision making.

During my survey, managers frequently commented that consistent definitions and formats are important aids to communication, especially between people in different organizational units such as divisions or departments. In a number of instances, the development of these definitions and formats was a lengthy and sometimes arduous task that was accomplished gradually over the course of several years, but which was also considered one of the main contributions of the systems.

For example, one of the purposes of some of the model-oriented systems in my sample was to estimate beforehand the overall result of decisions various people were considering separately, by filtering these decisions through a single model. In these cases, the system became an implicit arbiter between differing goals of various departments. Instead of arguing from their own divergent viewpoints, marketing, production, and financial people could use the model to demonstrate the effect of one group’s proposals on another group’s actions and on the total outcome. As a result, issues were clarified and the negotiation process expedited.

The production foremen I mentioned earlier noted the same kind of facilitation. It helped them in work-scheduling discussions and problem investigations by providing immediate access to “objective” information about “who did what, when, and how well on any production lot in the shop.”

Value to user

Although the implementers of a number of the successful systems I studied found it necessary to go through the motions of presenting a cost/benefit rationale which attributed a dollar value to personal effectiveness, they didn’t believe these numbers any more than anyone else did. Management usually decided to proceed on the basis that the proposed system seemed to make sense and would likely have a beneficial impact on the way people interacted and/ or made decisions.

Monetary savings are obviously a very important and worthwhile rationale for developing computer systems, but it should be clear at this point that the EDP-style assumption that systems should always be justified in these terms does not suffice in the area of decision support systems.

Equally obvious, there is a definite danger in developing a system simply because someone thinks it makes sense, especially if that someone is not the direct user of the system. In fact, the systems I cited as my first, second, and fifth examples began this way and encountered resistance until they were repositioned as something that users would want in order to become more effective.

Again, the general problem here is a common tendency for technical people to concentrate on the “technical beauty” of a system or idea and to assume that nontechnical people will somehow see the light and will be able to figure out how to use the system in solving business problems. This sort of overoptimism was present in the history of almost every unsuccessful system in the sample.

The message is clear: try to take advantage of the creativity of technical experts, but be sure that it is channeled toward real problems. The challenge, of course, is how to accomplish both of these goals. There are a number of ways, which I shall now discuss.

Patterns of Development

Despite the common wisdom that the needs of users must be considered in developing systems and that users should participate actively in implementing them, the users did not initiate 31 of the 56 systems I studied and did not participate actively in the development of 38 of the 56.

The results, illustrated in Exhibit II, are not surprising. Intended users neither initiated nor played an active role in implementing 11 of the 15 systems that suffered significant implementation problems. Conversely, there were relatively few such problems in 27 of the 31 systems in which the users had a hand in initiating and/or played an active role in implementing.

Exhibit II Systems Resisted by Users

But it would be wrong to infer from these findings that systems should be avoided totally, if intended users neither initiate them nor play an active role in their implementation. For one thing, 14 of the 25 systems I studied in which this was the pattern were ultimately successful. More important, many of the genuinely innovative systems in my sample, including 5 of the 7 that I described earlier, exhibited this pattern.

On the other hand, many of the systems initiated by users do little more than mechanize existing practices. While such mechanization can be very beneficial, and while I’m certainly not suggesting that major innovations must come from outside sources, the real challenge is to be able to use insights regardless of their source.

One way to do this is to devise an implementation strategy to encourage user involvement and participation throughout the development of the systems regardless of who originated the concept. Examples of successful strategies follow.

Impose gracefully: Marketing and production managers in a decentralized company did not relish the extra work (format changes and data submission requirements) needed for a yearly budgeting system, which top management was installing. Initially, they were especially unenthusiastic because they thought the system would not really help them.

So at every stage the designers made a point of developing subsystems to provide these middle managers with sales and materials usage information that had never been available. This quid pro quo worked well; instead of seeing the system as a total imposition, the manager saw it as an opportunity for them to take part in something which would be beneficial to them.

Run a dog and pony show: Central planning personnel in two companies designed systems for budgeting and financial analysis. In one company, the system never caught on despite lengthy training demonstrations for divisional staff and other potential users. These individuals seemed enthusiastic about the system’s possibilities, but never really used it unless corporate planning people did all the work for them.

In contrast, the training program for the system in the other company fostered immediate and active involvement. In order to attend the workshops, people were required to bring their own financial analysis problems. They learned to use the system by working on these problems. When the workshops ended, many users were enthusiastic: not only did they know how to use the system, but they had also proved to themselves that it could help them.

Use a prototype: Two ever-present dangers in developing a system are creating a large, expensive one that solves the wrong problem or creating one that some people in the organization cannot live with.

Either can happen, not only when the system is designed without consulting the user and affected parties, but also when there is no one having enough experience with the particular kind of system under consideration to clearly visualize its strengths and weaknesses before it is built.

Implementers of a number of systems in my sample avoided these traps by building small prototypes, which gave the users something specific to react to. As a result, the large-scale version could be developed with a realistic notion of both what was needed and what would fly in the organization. A similar approach, also successful, was simply to build systems in small pieces that could be used, changed, or discarded easily.

Hook the user with the responsibility: Each new module or application developed as an outgrowth of one of the three sales information systems I mentioned earlier goes through three stages. The first stage consists of general, uncommitted discussions of any current problem areas with which user groups are concerned. Following research by the management science staff, the second stage is a brief formal problem statement written in conjunction with the user group. In addition to describing the problem, this statement goes over the methodology and resources that will be required to respond to it. The third stage is a formal request for authorization of out-of-pocket expenses.

Sell the system: In one of the companies I studied, a marketing analysis group used a direct selling procedure to convince people of the merits of a sales forecasting system. The pitch was very simple: they compared manual monthly forecasts for one year with the system’s forecasts. The system’s forecasts proved to be more accurate in ten months out of twelve, with less error overall than the manual ones. The system was adopted.

In another company, management had a real-time system installed for monitoring the largely automatic production of an inexpensive consumer item in order to minimize material loss due to creeping maladjustments in machine settings. During the initial installation, the implementation team discovered suspected, but previously unsubstantiated, cheating by piecework employees; more pieces were leaving many machines than were entering them. Discreet hints were dropped that the monitor had to be checked because it was registering “impossible” results. The employees were sold on the new system: they knew very well that it worked.

Fundamental Changes

Despite extensive experience with EDP, many organizations have used no more than one or two of the seven types of decision support systems I have illustrated here.

One reason for this is that justifying such systems can be difficult: quantifying the impact of replacing ten clerks with one computer is one thing, while quantifying the impact of improved individual effectiveness of line personnel is quite a different thing. Another reason is that implementation can be tricky: many of the ideas come from people other than the users.

Nevertheless, developing a decision support system makes sense when it becomes clear that a fundamental change may be needed in the way decisions are reached and implemented. Often, the process of defining the system is every bit as valuable as the system produced.

My final point is that the concept of decision support systems itself can help managers in understanding the role of computers in their organizations. As the name implies, data processing systems systematize and expedite the mechanics of carrying on business activities by processing masses of data automatically. On the other hand, the decision-oriented extensions of these systems help people make and communicate decisions concerning administrative and/or competitive tactics and strategy.

The decision support systems I have discussed go one step further. Instead of starting as extensions of existing data processing systems, many decision support systems are built from scratch for the sole purpose of improving or expediting a decision making process. The underlying philosophy is that the use of computers to help people make and communicate decisions is every bit as legitimate and worthwhile as the use of computers to process masses of data.

There is evidence that this viewpoint has caught on to a certain degree and is becoming more widely accepted. The implication is not that all organizations should get on the bandwagon, but rather that managers should be aware of the opportunities and challenges in this area and should attempt to assess whether their organizations should move in this direction.

1. Steven Alter, “A Study of Computer Aided Decision Making in Organizations,” Ph.D. thesis, Sloan School of Management, MIT, 1978.

Which of the following is one predictive analytics technique which measures the effect of a set of variables on another variable?

Which of the following statements is true regarding data mining?

What type of analytics seeks to make decisions to achieve the best performance possible?

What makes analytics part of a business process?