[Reprinted with permission from: Intelligent Software Strategies, Cutter Information Corp., Arlington, MA, September 1992]
In our Fall 1990 survey of intelligent software at the help desk, we identified this activity as one of the fastest moving and most innovative areas of corporate automation. That analysis has been validated. There have been more than a dozen new software products introduced here in the last year and a half, and more are coming every month. Case-based reasoning, for example, which had just appeared on the scene, is now in use at several sites. Existing products continue to evolve, resulting in converging functionality and cross-platform compatibility. Vendors are cooperating to integrate and/or embed their products to offer the multifaceted functionality that the market now demands.
And the market is growing. The Help Desk Institute, a consortium whose goal is to help raise the level of professional awareness at corporate help desks, has doubled in membership over the past two years. However, the help desk market is not gigantic as automation markets go, and, historically, it has been a relatively impoverished one. This market is important beyond its size and budget. A new kind of software is evolving here. Its function involves the automatic distribution of knowledge in an organization. The help desk is the first example: Just as record-keeping databases eventually found utility beyond the payroll department, knowledge distribution is a fundamental organizational need, and its automation is an important development.
In this article, we will define and describe the various technologies that make up help desk software, review current activity, and indicate what is hot, product-wise. We'll also bring up some caveats for those who are planning automation products, and take a look at the short- and long-term trends in this exciting software market.
But the factor that makes these support organizations so important to the evolution of intelligent software has to do with a unique quality of this work environment, and with groupware. The help desk, as corporate activities go, is a uniquely cooperative endeavor. Here the incentives are right for sharing knowledge and experience -- if you don't explain, people continue to ask you the same questions. Sadly, nowhere else in corporate life is it so often in one's best interest to let other people know what you know. And since, thanks to networks and client-server architecture, groupware is now technically within reach, those organizations who can use it to share their experience (and not just to schedule their meetings) will be the leading edge in a major change in the way we work. Some organizations are even building home-grown help desk solutions with groupware tool kits like Lotus Notes. (For more on groupware, see the books in our readings list.)
The first set of problems to be addressed is classical data processing, automation of record keeping: What information do we have about the caller, their equipment, the versions of the software they are running, and our service agreement with them? Have they called about this or other problems before? Who took the calls, when? Was the call "escalated" (referred to an expert)? Was it resolved? How and when? What kinds of calls are we getting? Why? How are we at the help desk performing?
Specialized, modern database products, customized for the help desk, are in common use today. They are often integrated with the telecommunications system, so that calls are answered in the appropriate order and escalation is just a matter of forwarding the call (and later making sure it got taken care of). True, a big segment of the information center help desk market is tied to a mainframe/DP solution and will continue to modify their existing "asset management" database to try meet the requirements of a modern support organization. But clearly the trend, especially among the more affluent customer support operations (which also tend to be more willing to leave the mainframe behind for a networked solution), is toward a customized call-tracking database product. (Of course, if you already have useful data, whatever system you buy should be able to use that data in its current form.)
Answer's Apriori product, which includes a CBR-like component, is certainly one of the best selling call-tracking tools, but the market is getting very crowded. Interesting recent entries in this category include Support Advantage from ProActive Software (formerly Information Workbench), Utopia from Hammersly Technology Partners, Clientele from AnswerSet Corp., Action Request System from Remedy Corp. and SupportWise from BusinessWise, Inc.
We review now the basic knowledge technologies that have been put to use at the help desk. Many products on the market combine these elementary technologies, for example, offering a call-tracking database along with a text-retrieval capability for on-line documentation. The result is a new kind of automation which will eventually affect the many corporate departments that produce documentation, policy and procedures manuals, product bulletins, regulatory advisories and training. In these departments, as at the help desk, knowledge distribution is the essence of the job. Keep in mind that knowledge distribution systems is a new class of software, with different functionality and value to the organization -- not just a new set of programming tools.
Note that client-server, CASE, multimedia and object-oriented programming, for example, aren't covered here. They are fundamental technologies that are changing all software development, not just knowledge distribution. (We do, however, know of two state-of-the-art, home-grown help desk systems, at Novell and at Hughes, built from scratch with OOP tools.) This discussion is intended to help you deal knowledgeably with help desk software vendors or with your systems development department. With each technology, we will try to convey the state of the art, again, to inform you only. Your decision on what to buy should be based only on what you need or will need.
Intelligent Text Retrieval. Text retrieval is relevant to the help desk because so often there is a great deal of information availability in on-line documentation, technical notes, bulletins, e-mail logs and, increasingly, gigantic CD-ROM libraries supplied (sold) by the vendors. Text retrieval has been around a long time and has been the focus of a great deal of clever invention over the years. If you haven't checked into this technology recently, you will be surprised at current capabilities: Parallel searching of pre-indexed text databases (distributed across your network) and of real-time text data streams using advanced algorithms that match against a sophisticated representation of the meanings of the search keys. The result is like having a staff of librarians skimming over all the documents for you.
Verity's TOPIC text retrieval system is still the state of the art in this technology. TOPIC allows keyword and Boolean searches, but also implements a way for users to combine basic words into concepts, called "topics". (Some words or phrases may be indicated as more "relevant" than others.) Topics can in turn be combined to form other topics, and the entire knowledge base of retrieval concepts can be shared among users and further customized. TOPIC then displays all the matching text base elements in order of "relevance." Recently, Lotus announced that TOPIC was licensed for inclusion in Notes, the leading groupware development tool kit.
Text retrieval technology is impressive and it is a must if you have volumes of documentation on-line. But it does have it's shortcomings. Even if the retrieval is perfect, so that all and only those documents of high relevance are selected (the theoretical ideal), the user still has to read them, find what he or she needs, and understand what it says. This on-line information generally can't help a naive user through the steps of systematic troubleshooting, for instance.
Hypertext. Another text-related technology that is relevant to the help desk is hypertext, a user-interface design that facilitates browsing of text/graphic documents. Hypertext is really a new way of writing -- the document's creator is responsible for "linking" relevant sections so that the reader can browse through the modules with just a mouse click. This allows, for instance, both expert and novice users to read the same material, but at different depths. Although some cross-indexing can be automated, the impact of hypertext at the help desk must await the enlightenment of documentation writers. The on-line help for some major software products is already hypertext browsable.
Case-Based Reasoning. The rising star at last year's AAAI, case-based reasoning is another retrieval technology that now has begun to show its value at the help desk. (See this year's Innovative Applications of AI paper by Acorn and Walden about Compaq's CBR-Express-based SMART system.) Unlike text retrieval, which is a general technique for retrieving text files, CBR is tuned to retrieving information about past situations that might be similar to the current situation.
Many call-tracking products have simulated this functionality. Verity's TOPIC has been used in some installations to retrieve the highest-ranking match from a database of text descriptions of previous cases. Similarly, Answer's Apriori product, also one of the premier help desk products, produces CBR-like functionality using only text-retrieval technology. (Answer, however, embeds this case-retrieval technology within one of the slickest client-server call management products on the market.) Some of the new call-tracking entries (e.g., Support Advantage and Utopia) talk about automatically (no extra work) cataloging product defects and customer problems, which can be scanned later.
However, as you accumulate more cases, provide service with less-experienced technicians or directly to end users, and deal with more far-fetched problem descriptions, CBR technology has added value over simple text retrieval in accuracy, speed, and interactive retrieval. Since it stores the information about past cases not as text files, but as structured data describing events in the domain (previous service calls), CBR can not only be more accurate in the retrieval, but also can indicate what questions might be asked to clarify the situation (and determine which partially-matching case is most relevant). There are five CBR "products" on the market at this time, and they are all quite different, technically. Some systems use the sentence structure of the problem description to index into past cases (this was the original AI technology). Some use specialized text retrieval technology. One even claims to use fuzzy logic to identify the most relevant cases. The current leader in CBR, Inference Corp., has also announced a run-time tool, CasePoint, for distributing to end users case-bases developed with CBR Express. And Esteem, originally developed in KappaPC, has been ported to Level 5 Object.
Several call-tracking product vendors, including Answer and BusinessWise, have been talking to CBR companies about incorporating this technology into their help desk products. Aion has also explored integrating Cognitive Systems' CBR technology into ADS, but the recent merger with AI Corp. seems to have put CBR on the back burner for a while.
In its simplest form, a CBR product should take the description of this event or call, say from the original data entry screen, sometimes called a "trouble ticket," and answer the question "has anybody seen anything like this before?" More advanced systems, as we said, would pose some clarifying questions. This is the essence of knowledge sharing, and this technology has some important advantages:
As we said, some organizations are now putting vast amounts of text-based knowledge (documentation, troubleshooting guides, technical notes) on CD-ROM's for public distribution. Some publishers, like Ziff Communications, are also in the business of selling text-based information on CD-ROM's. Soon we will see similar ventures distributing case bases. But both of these technologies have a serious drawback. The user has to describe the problem "correctly," and then sift through, read and understand the text, which so often seems written without a reader in mind, much less a naive reader. What you really want is to have the expert by their side, asking relevant questions and clearly explaining what should be done. This, of course, will lead us to expert systems.
Decision-Tree Programming. But first, a warning about a simple, good idea. Once you decide to "engineer" your department's troubleshooting expertise into software, you might be tempted to have your experts engineer themselves. The most common approach is to have each expert describe their troubleshooting expertise as a tree of questions and answers, a decision tree. The user is then directed to the appropriate tree by a dispatching tree.
Programming tools exist which allow experts to easily develop these trees themselves. Most tools offer a compelling, graphic tree-building environment, a tree debugging environment and user-interface building tools. They should also be able to retrieve data from existing databases, so that the user is not asked inappropriate or unnecessary questions.
This approach should work. In our personal experience, however, it was no easy task. The trees get cumbersome very quickly and no one can maintain them except the experts, who are typically busy with other things. However, the tools will improve. Like it or not, end-user systems are a part of the communications technology we are creating. Decision trees are a natural way to communicate small diagnostic procedures. They fit into the picture somewhere. My recommendations, if you choose this route, is that you have the system designed by professionals, that you focus only on slowly-changing, frequently-demanded knowledge, and that your experts be made to understand the long-term nature of their commitment.
A novel product announced recently is the Edify Information Agent. The Information Agent isn't a decision-tree builder, per se, but it is an end-user programming tool and it doesn't fit in anywhere else, so we mentioned it here. It allows a non-programmer to specify the performance of a "software agent" that might, for instance, answer the phone, as a few questions, fax back an on-line document, and update a tickler database. The development environment, called Agent Maker, looks like a spreadsheet: You place icons with names like "ask, speak, fax, or update" into the cells to define the particular agent you need. Built in interfaces to LAN and host database and text files make the process pretty simple.
Expert Systems. Expert systems is a programming technology for embedding the knowledge of an expert into a program that then behaves like an expert in a box (asks questions, draws conclusions, and makes recommendations). There are two major types of expert systems: rule-based and model-based. Rule-based or "heuristic" systems operate much like a doctor or repairman, associating symptoms with hypotheses which are then tested further to reach a conclusion. Model-based systems perform the kind of diagnosis that requires a deeper knowledge of the broken mechanism. They work by simulating in what way the thing could have broken, so as to produce the observed symptoms. Model-based systems are only appropriate in the most complex troubleshooting situations (modern weapons systems) or when there is no troubleshooting expertise (the space station).
Think of it this way: When your computer repair person shows up, she runs some tests, tightens a cable or replaces a board. If these simple procedures don't fix the problem, however, she must pull out the schematic diagrams to figure out what's wrong. She has transitioned from heuristic to model-based reasoning. And your repair bill will skyrocket.
Expert systems take the knowledge stored about the diagnostic situation, whether heuristic or model-based, and ask questions. In other words, they generate a decision tree on the fly. They are more flexible than decision trees, they can "bridge the gap" to naive users by asking clarifying questions if the user can't respond, and are easier to maintain for large amounts of knowledge. But they are maintained by programmers, not by experts.
Expert systems technology has evolved rapidly and taken new forms at help desks and customer support operations. Some systems are small, decision-tree-like programs that guide callers through the first few questions. Some use model-based reasoning to help experienced people do failure-mode analysis on very complex equipment. Some even help field-service dispatchers figure out enough of what's gone wrong so that they can decide whom to send out on the job and what equipment they will need to get it fixed on the first visit. (See the Whirlpool system described by Danilewitz and Freiheit at this year's IAAI.)
One major impact of expert systems on support is in the creation of "customized" documentation and trouble-shooting guidelines which use knowledge about the user and his situation to offer specific assistance. You will be buying these advisors in the near future, perhaps first as expert "librarians" for those voluminous CD-ROM's. Eventually, this kind of expertise will be embedded into the software products you buy and the machines and networks you use. And some day, you too may also be distributing your department's expertise this way.
Expert systems is the ultimate technology for codification of knowledge to be embedded and distributed in software. They allow fusion of knowledge from different experts. They dramatically reduce training time and support a higher level of consistency and thoroughness of problem resolution. And they can be debugged -- you can ask an expert system why it followed a certain line of reasoning.
But this technology has several important contra-indications:
Formally analyze your situation and your needs. Again, all help desks are different. But there are some common problems, like being too busy to think, dealing with unhappy callers all day, and being a group that management doesn't want to think about, much less budget for. A good analysis of your operation will uncover the underlying causes of your help desk problems and determine whether they are likely to be treatable by automation. This "systems analysis" should be done before you spend your money on a computer system of any sort. The process should not take more than three months and the results should make sense to you.
Automation may not be necessary. Consider alternatives to automation that will meet your projected needs: Hiring more experienced staff, getting more training for your current staff, writing new procedures manuals, documentation, troubleshooting guides, periodic bulletins and bulletin boards, having more frequent group meetings and informal communication, or using the old Rolodex.
Get your call-management act together first. Though knowledge technologies offer great promise, they are tricky. You will need to have the call-tracking system in place, both hardware and software. This will improve your operations so much, maybe you won't need anything fancy in the problem-resolution area for a while.
Automation requires an investment. Automation is not something to get involved in on a shoe string. The initial costs of buying and installing the system, and training everyone to use it, may be equaled by the annual cost of keeping it up to date. Although it is tempting to get involved with slick new technology, talk to someone who has done it before you decide to. If you can't completely justify an adequate investment to your management, you are not ready.
Think across departmental lines. Try to involve documentation, training, marketing, sales and engineering departments in the design of your system. And especially consider whether your new software maintenance effort will be duplicating, and improving on, an existing paper-based activity in another organization. Don't just automate, re-create. Think of all of these organizations as conduits of knowledge. Solutions might involve embedding expertise in your software or creating auxiliary software products.
Automation may not work. Customer support begins with product design, quality, and an understanding of the market. If you have messed up here, no amount of support will fix the problem. And if you are having morale problems at the help desk, automation may even make things worse. The system may make people feel they are doing more clerical activity without helping them do their jobs better. It will not lift people's spirits.
Knowledge for Sale. The trend most apparent at the Help Desk Institute's Knowledge Technologies Conference in Boston was the emergence of a secondary market for help desk knowledge. Vendors like Microsoft, Apple and Novell are already distributing CD-ROM's into this market. Vendor consortia like Compaq's Third Party Support Alliance are trying to address the issue of support in a multi-vendor environment. Intermediaries like Ziff Communications are signing software and hardware vendors up for the rights to their on-line documentation and technical notes. By the way, a division called Ziff Technologies Group has recently acquired the Help Desk Institute and Bendata Management Systems' HEAT product. Another intermediary, M'aidez, which had signed up several software vendors for inclusion in its CD-ROM juke box for technical support documentation, has closed its doors due to financing difficulties.
But text-based information is not knowledge until it is integrated into people's behavior. Knowledge systems and case bases reflect actual experience in a way that documentation rarely does. And these new technologies are interactive -- text bases, no matter how big they become, never ask clarifying questions of the user.
Three systems which appeared this year indicate an increased value placed on the embedded knowledge itself. NYNEX is now selling an expert-system-based product called ALLINK which monitors and troubleshoots problems in a corporate wide telecommunications network. A similar idea was pursued by Network General's SNIFFER for local area network troubleshooting. And, at the other extreme of the size scale, Teknosys sells an expert-system-based tool called Help! that loads into a Macintosh and looks around for potential hardware and software problems and conflicts. The current release has some 3000 troubleshooting rules (up from 2300 in the original release, and growing all the time). We even met a spare parts supplier, Computer Service Supply Corporation, who plan to offer an RS6000-based diagnostic aid that automatically orders spare parts once the problem is identified.
Clearly, the next step is to sell the knowledge itself, to be embedded into other peoples operations. In our 1990 article, we mentioned that AdvantageKBS, which sold a specialized expert system shell, was also selling "knowledge modules" for common help desk problems. This product line has been re-written (it was originally done in Aion's ADS) and sold to Legent Corp. (Legent's forthcoming product, Ph.D. Advisor, is discussed below.) A new vendor appeared at the HDI's Knowledge Technologies conference that foreshadows things to comes in this arena.
Knowledge Brokers, Inc. of Los Angeles is in the help desk knowledge publishing business. Their goal is to maintain and sell case bases and knowledge bases that embody help desk expertise. Some of these products they will manufacture, based on experience in their help desk outsourcing operation (see below). Most, however, will be knowledge bases developed by third parties, which are then licensed, edited, distributed and maintained by KBI. The idea of selling the knowledge and not just the knowledge engineering tools is not a new one. KBI is situated as the leader in the knowledge distribution business in the first major corporate market that is actually coming to fruition -- the information center help desk. Expect to see the various knowledge systems tool vendors and CD-ROM publishers following KBI's lead into the knowledge distribution channel.
Outsourcing. A major trend in all of IS these days is the search for savings by outsourcing certain common, non-competitive activities, like running the data center or writing new database applications. The help desk seems ripe for outsourcing. It is very expensive to develop an adequate level of competence amongst the staff to cover the range of problems. And after all, most companies now own similar kinds of equipment and software and could share some of this help desk expertise. There are now local operations in many cities that will link into your phone system and serve as the front-line support organization, managing the calls and escalating back into your organization those that are specific to your environment. They are then responsible for maintaining adequate staff to deal with load peaks and with specialized expertise, like UNIX, Novell networks or Windows.
Outsourcing can be threatening, especially to help desk personnel who have developed over the years a great deal of expertise about all kinds of equipment and systems, before they were commonplace business tools. And this move toward outsourcing can be carried too far, eliminating strategic strengths. But, as is the case in other kinds of IS outsourcing, times change: Outsourcing redefines the boundaries between your work and your tools.
Integrated Help Desk Products. The movement toward integrating multiple functionality into the products by incorporating several technologies will continue, resulting in a few major products with a great deal of functionality. Vendors of different types of products are cooperating to form integrated solutions for specific clients, and sometimes to enhance their products by combining technologies.
For instance, Answer's Apriori now incorporates the Fulcrum text retrieval technology. Ultimate has integrated Carnegie Group's Testbench model-based diagnostic product. Several call-tracking DB vendors are negotiating to incorporate one of the CBR tools. New players are introducing new products which combine technologies like call-tracking, CBR and text retrieval for on-line documents. The trick will be to keep these products easy and natural for the user, and to package them modularly so that people don't have to buy more than they need. This trend toward modular products will temporarily mask a long term trend, the fragmentation of the help desk market.
The customer support market is splitting into three categories: software vendors, hardware vendors and all other customer service. In software and hardware support, the problems are getting more complex due the complexity of the systems themselves and the multi-vendor, open computing environments. At the same time, many of these vendors are expanding less rapidly than they are accustomed to, and can no longer hide service costs in new revenue. This, of course, will lead immediately to more aggressive attempts to charge for service.
Some customer support operations involve field service scheduling, which is a different market entirely, one already viewed as a money-making enterprise. Finally, there is the classic market fragmentation between small and large, and between rich and poor. Product differentiation toward the low-end and high-end markets is already apparent.
Intelligent Products. Customer support begins with product design. The products your callers ask about may soon have help, configuration or troubleshooting advisor subsystems sold with them, or even built in. Major vendors, including Amdahl, Boole and Babbage, and Legent, have already embedded diagnostic expert systems into their software and hardware products.
Knowledge Asset Management. Companies that spend millions capturing and protecting data in the most expensive computer systems in the world, still keep their knowledge assets in three-ring binders. You will be hearing the term "knowledge asset management" a lot in the next year. Both the AI industry and the "repository" movement have decided that this is a good way to describe the benefits of knowledge engineering. (See Thomas Stewart's 1991 article in Fortune.)
Open Knowledge Architecture. There is no inherent reason why the different technologies and products cannot work together. Market forces will eventually lead to the creation of standards and protocols. A help desk knowledge repository will be possible, with knowledge entered on the fly by help desk technicians or engineered in by the successor to the current "documentation" group, and accessible on-line, by fax, or over the phone.
Rethinking the Help Desk. Once it was the "complaint department", located in some windowless basement room. Now it is the "support center", reporting to IS or to marketing, but still windowless. Someday it will be viewed as the central function of the organization, with all other departments supporting the help desk. As technology expands the capabilities of the help desk, it will evolve into an important conduit of knowledge, reflect the service orientation of the organization and simultaneously putting it in direct contact with the customers.
For technology to have a global impact on the corporation, it must automate something so essential that it is already being done manually by just about everybody. The help desk is to automated knowledge distribution what payroll was to the automation of record keeping . . . a universal application that fits the new technology like a glove. We believe that the automation of these activities is a major new area of business computing, and a major challenge for overworked and shrinking IS departments. Before long, automated, corporate-wide sharing of experience will be viewed as a competitive necessity. And the political implications of a technology that automates the asynchronous sharing of experience, a fundamental human need, are interesting as well. One could view the entire planet as a big help desk -- certainly everybody who shows up here needs help.
Robert Johansen, Groupware: Computer Support for Business Teams, Free Press, NY, 1988.
Susanna Opper and Henry Fersko-Weiss, Technology for Teams: Enhancing Productivity in Networked Organizations, Van Nostrand Reinhold, 1991.
Jeff Pepper, We're Off to Seize the Wizard: The Revolution in Service Automation, ServiceWare Inc., Verona, PA, 1991.
Michael Schrage, Shared Minds: The New Technologies of Collaboration, Random House, 1990.
Innovative Applications of Artificial Intelligence, 1992. Edited by A. Carlisle Scott and Philip Klahr, AAAI Press, Menlo Park, CA. See the three customer support papers.
Thomas A. Stewart. Brain Power: How Intellectual Capital is Becoming America's Most Valuable Asset. Fortune, June 3, 1991.
