Keynote: Revolutionizing KM with AI & Document Understanding #KMWorld


Speaker: Paul Nelson, Innovation Lead, Search & Content Analytics, Accenture

Session Description: Intelligent document understanding is drastically changing the search and knowledge management landscape thanks to AI technologies. As 80% of all enterprise data is unstructured, document understanding delivers tangible benefits across industries and business functions saving time, money and resources. Paul highlights how one of the world’s leading pharmaceutical companies leveraged this solution, along with innovative AI technologies and assets, to automate KM for the purpose of detecting sensitive IP within unstructured enterprise content. He shares how this solution is helping other clients, including Accenture, to improve compliance and risk management, increase operational efficiencies, and enhance business processes.

[These are my notes from the KMWorld Connect 2020 Conference. Since I’m publishing them as soon as possible after the end of a session, they may contain the occasional typographical or grammatical error. Please excuse those. To the extent I’ve made any editorial comments, I’ve shown those in brackets.]


  • He will focus on the three practical KM scenarios to make abstract AI real
  • Scenario #1: Customer Support — we started with individual customer support representatives who use their knowledge to respond to customer request, then we documented the usual queries and responses (FAQs), then we added some search for efficiency. But as the system scaled, we needed a document management system to organize all then content. Then, we superimposed AI to monitor data streams, summarize the patterns, and determine what topics should be addressed by technical writers and included in the document management system. But, with the advent of a new neural network (GPT-3), which can recognize new tokens, now we can replace a technical writer by a robot writer to create a fully automated customer support system.
  • Scenario #2: Research Reports — initially we relied on individual researchers to run tests and produce research reports. But sometimes, this resulted in duplication. So we interposed a gatekeeper who received research requests, checked to see if the work has already been done, and then either distributed the earlier report or commissioned a new report. Now, a robot can replace the gatekeeper. It can understand the incoming request, search the database, understand the content of the research reports, and send the appropriate responsive report — even if it contains words that are different from those in the initial request.
  • Scenario #3: Proposal Writing — how do you respond to RFPs or RFIs? No one person knows enough to respond to the entire proposal request by themselves. Currently, various experts complete the section of the proposal related to their expertise. To increase efficiency, we have tried to create templates and snippets that can be used as necessary. Now, the proposal writing is happening in collaborative sites. And key sections are tagged with relevant data. Going further, you can slice and dice the sections into Semantic pieces that can be sent into a semantic search of a neural network to find relevant information that can be reused automatically. Further, the search can identify cross-correlations, which will find the pieces that usually go together for a specific kind of proposal. After the proposal has been assembled and sent to the potential customer, the system can use cross-correlation between the proposals and the CRM system to determine which proposals were successful and which were not.
  • Closing Remarks
    • AI is more than just classification and entity extraction
      • It can find new topics
      • It can connect content by meaning — Semantic Search
      • It can evaluate content quality
      • It can summarize and rewrite
    • Successful projects will improve existing knowledge flow
      • [This is similar to the old KM adage: pave the cow paths]
      • Don’t fight the existing process. Instead, work within it.
    • Successful projects will include targeted AI
      • Focus on solving the problems you can solve today via AI
      • Gather the data you need to solve the problems of the future
    • You still need a great search engine
      • The results of AI feed and enhance the search engine
      • Search engines are the only way to scale

Coalition of the Willing

Lawyers in most firms are given a lot of freedom to decide how to manage their own knowledge. In fact, it’s a rare law firm that can demand that its lawyers handle their knowledge in a particular way. For many, the battle began and ended with the document management system. At this point, most firms with document management systems have persuaded their lawyers to create and store documents primarily within the DMS. This has the signal benefit of ensuring that the firm’s work product is located in one place.  The problem, of course, is that while you can require that documents be created within the DMS, it’s much harder to get lawyers do anything more than the most rudimentary profiling of their documents.  As a result, it has until recently been extremely difficult to capture much metadata regarding a document. What’s changed? In part, it’s that lawyers are beginning to learn the value of metadata to assist in the document searches they do every day.  In addition, new document management systems are more intelligently designed and allow simpler filing of documents, coupled with the ability to let new documents “inherit” metadata from the folder in which they are placed.  Couple this with the metadata extraction capabilities of some work product retrieval systems, and the burden on the individual lawyer to create metadata is lightened considerably.

So the good news is that after nearly 20 years of document management systems, we’re finally getting to a point where the technology allows them to work more seamlessly and intuitively for lawyers.  This should encourage greater use (and more rewarding use) of the DMS by lawyers. The bad news is that relatively little of a firm’s knowledge in contained in its work product. What’s your strategy for dealing with that problem?

Unless your firm is run by Attila the Hun, you won’t be able to compel lawyers to share their knowledge via a central repository or medium.  Further, you will run into the problem observed by Steve Denning (see The Economic Imperative to Manage Knowledge) regarding the behavior of “experts” with respect to their knowledge:

As preliminary efforts to establish what the organization knew were launched, it started becoming apparent – to the surprise of many – that the organization did not know what it knew. Inquiries as to the cause of the hesitancy revealed that even the experts were not sure of what they knew. The experts even contested whether they were responsible for sharing their knowledge. They often contended that their job was to meet with their clients and deal with their needs, not sit in an office in headquarters and assemble best practice manuals.

What’s the solution? If you can’t compel sharing, you’ll need to coax sharing.  The best way to do this is to work individually with your experts to identify their personal knowledge management challenges and then find ways to address those needs in a manner  that results in a solution that is satisfactory for that expert AND yields rich material in a selectively shared content repository. Notice, that I used the words “selectively shared.”  Unless you can promise some measure of control over the knowledge, you’ll have a hard time winning the cooperation of your experts.  They will undoubtedly want the freedom to gather and organize the content as they see fit — not as necessarily as the IT department dictates. The key here for technologists and knowledge managers alike is to provide very lightweight systems that provide the individual flexibility cherished by experts. One obvious choice is the range of Enterprise 2.0 tools now available, but I could imagine implementing some firm-wide systems in a way that encourage personalization, sensible organization and sharing rather than the unmanageable wilderness currently found in everyone’s favorite content repository — Outlook.

One challenge is that your work with these individual experts will result in information silos.  However, you can go some distance in managing these new silos by ensuring that the content can be shared easily. Then, see the good that happens when your intelligently-designed system interacts with what Dave Snowden observed as our basic tendency to help in times of true need.

The bottom line is that you have to build a coalition of the willing — willing experts, that is.  Once you’ve helped them organize and find what they know, they’ll be better equipped to share that with others.

[h/t to John Tropea for pointing out the Steve Denning piece]

[Photo Credit: lumaxart]


Bad Habits

Kudos to Jordan Furlong at Law21 for his summary of the document management presentation by Steve Best and Debbie Foster at the ABA Techshow and his thoughtful observations on the depressing state into which lawyers and their document management systems have sunk:

The speakers emphasized that the only truly effective DM system is one that makes compliance involuntary. Human nature and office culture are both such that staffers will always look for a way to get around the new system of naming, filing and locating documents in order to use their own. This reminded me of what I’ve been hearing more often in knowledge management circles, that the most reliable way to harness lawyers’ knowledge is to automate the process, extracting the information from lawyers without them knowing it or participating in the process: many law firms have not found ways to sufficiently motivate lawyers to freely share what they consider their stock in trade. It makes me wonder about the bad habits we’ve developed in the legal profession regarding the information we use every day, that we’re at the point of needing to circumvent choice and remove human activity to guarantee success. That’s not good.

Where did we go wrong? To begin with, earlier generations of document management systems set up enormous barriers to entry in the form of over-reaching profile pages. These pages collected information that lawyers didn’t care about or didn’t care to share. Truly useful information (e.g., client matter numbers) was mixed up with much less helpful information (e.g., randomly-selected document types). At bottom, there was a huge disconnect between the source of the information — the lawyer — and the DMS. Unfortunately, there were few incentives or cogent explanations to bridge the gap.

And then, there are the lawyers. What is it about lawyers that makes us so uncooperative on these issues? First, the pressure of the billable hour pushes us to move as quickly as possible through our work. When you are playing “beat the clock,” who has time to fill out an extensive profile page? Second, we are tightly focused on client demands. This necessarily makes the non-billable needs of firm administration and systems much less compelling. Third, it is the nature of big law firm practice that very few first-year associates will actually stay in their firm long enough to become partners. So what’s the incentive to contribute to the institution and its systems?

The other problem with lawyers is that we’re only human. And most human beings don’t like to do what they don’t like to do. In fact, some will go to great lengths to launch workaround wars on systems they haven’t specifically endorsed. (To be honest, when was the last time the managers of your DMS sought user feedback much less endorsement?)

And what are the “bad habits” lawyers have developed regarding the types of information that allows us to marshal and manage lawyer work product efficiently? Lawyers have grown accustomed to noncompliance with impunity. Law firms have not made compliance a priority. If they had, they would have found more successful ways of convincing us that sharing this information is useful not only to the firm at large, but to the individual lawyer as well. And, they would have enforced compliance more effectively. Firms have also allowed lawyers to focus on very short term goals (e.g., meeting the client requests of the day no matter what the cost) at the expense of long term goals (e.g., building a knowledge infrastructure that allows tomorrow’s client requests to be met with much less effort).

Jordan Furlong’s observation applied to the full range of KM systems leads to a disquieting conclusion: if we are not able to elicit the voluntary participation of lawyers in the creation and sharing of knowledge, then we will be compelled to build KM systems that “circumvent choice and remove human activity to guarantee success.” But that is success at a high price.