KM Tools Lawyers Love #ILTAG128 #ILTACON

Session Description:

Knowledge management (KM) professionals often design and implement tools they are certain their lawyers will love, only to have them fall flat and quickly slip into oblivion. Sometimes KM and IT launch a tool expecting lawyer pushback or disinterest and are pleasantly surprised by immediate adoption. Let’s focus on the KM tools lawyers love as we learn about some of the KM tools practicing lawyers have found most helpful and easy to incorporate into their practices. Whether you are just starting out with KM, looking to refresh a long-standing KM initiative or operating with a tight budget and limited resources, come learn which projects will be the quickest, be the easiest and win big points with your lawyers.


  • Develop a better understanding of the practicing lawyer’s priorities and concerns
  • Gain insight into how lawyers think about their practice and work with their clients and each other
  • Leave with a short list of winning projects to take on when starting out with KM, refreshing KM or performing KM on a tight budget
  • Establish a check list of things to consider when deciding which KM or legal IT projects to pursue and which to postpone or even ditch

Speakers: Patrick DiDomenicoPatrick DundasSally GonzalezMeredith Green

[These are my notes from the International Legal Technology Association’s 2017 Conference. I’m publishing them as soon as possible after the end of a session, so they may contain the occasional typographical or grammatical error. Please excuse those. To the extent I’ve made any editorial comments, I’ve shown those in brackets.]


  • What is Knowledge Management?
    • KM 1.0 = Improving client service delivery (plus, in the UK: current awareness, professional development, sharing knowledge with clients)
    • KM 2.0 = Winning more business — experience management, knowing the client, business development activities
    • KM 3.0 = Improving processes
    • KM 4.0 = Leveraging AI
  • How do you measure success? A tool is successful if
    • it is used by the lawyers themselves — they don’t delegate its use to others
    • lawyers call you immediately when the tool is down
  • Tools that help GENERATE WORK PRODUCT.
    • Schulte’s Forms Project
      • Know-how: forms stored in iManage folders
      • WARNING: a forms project is extremely time-intensive and effort-intensive. So do not begin the project unless you know for sure that there will be sufficient use of the form. In addition, creating it is less useful if you can’t/won’t keep it up-to-date.
      • Whee there is a business need, these forms will help make the practice of law more efficient.
      • At Schulte, the put drafting instructions in footnotes. They use MS Word’s commenting function to explain the reason why certain language is being used.
    • McGuire Woods uses an external provider such as the Practical Law Company (PLC)
      • PLC provides forms and practice notes
      • For junior associates, this was a godsend — especially since the firm did not have a vast bank of current forms
    • Ogletree has model/form docs, augmented with Lexis Practice Adviser
      • they have homegrown “cream of the crop” model/form documents that their practice support lawyers maintain
      • these materials are collected and available in their Knowledge Resource Center in their intranet, in the document management system (DMS) AND via enterprise search.
      • they use Lexis Practice Adviser to fill in the gaps
  • Document Automation.
    • See the article by Patrick Dundas in the recent ILTA KM white paper
      • Documents drafted using the Schulte document assembly platform (HotDocs) results in substantial reductions in cost and effort
    • At Ogletree the have both internal-facing and client-facing document automation. They did a large-scale document automation process for a multinational client. It resulted in substantial savings of time and costs. And the work product was more consistent. The client was so delighted that they awarded a bonus for this work.
  • Finding what you need.
    • Ogletree uses Recommind. (They are transitioning to Handshake search.) Enterprise search was the most important tool they have implemented to help lawyers find what they need: content, people, matters.
    • McGuire Woods is just starting their KM program so enterprise search is a bit too ambitious for them right now. They have deployed Lexis Search Advantage (LSA) to help lawyers find content in the DMS and in the Lexis collection. They can also tag content with established tags or new tags you create yourself. In their experience, LSA is more intuitive than their DMS native search.
    • LSA does a great job of classifying content. At Ogletree, they have discovered that some of the LSA filters are better than the filters in their DMS.
  • Client/Matter Pages.
    • Lawyers love these types of pages because they give easy access to critical information.
    • These pages provide on demand, instant access to information that was previously buried in PDF reports generated by underlying systems like the time and billing system. Therefore, they do not have to step away from their work to ask someone to generate a special report for them. They can stay “in the flow” of their work.
  • Cara by Casetext. This tool is a great asset for litigators. It allows you to drag a brief into their web interface. Then the tool identifies what cases are relevant to that brief but were not cited by the brief. You can use it to check your own briefs; you can use it to identify the holes in an opponent’s brief.
  • Harvesting PTI. “Pardon the interruption” emails are commonplace in law firms. While they may surface in-the-moment assistance, the content is usually buried in private email threads.
    • In response, Schulte has set up a shared iManage folder to harvest these emails and permit later search and retrieval. As a result, it saves time and money, and causes fewer interruptions. Plus, it costs nothing to implement.
    • Ogletree also has an iManage folder. In addition, because it is stored in the DMS, it is retrievable via enterprise search. Therefore, the lawyers do not need to check the shared folder; all they have to do is run a search in the enterprise search system.
  • CRM Systems. Valuable client information is stored in the client relationship (CRM) system. However, while the Marketing Department might be able to manage the CRM interface, it may not be as easy for lawyers.
    • If you make the data stored in the CRM system accessible via the enterprise search system, then the lawyers can find it without having to tangle with the CRM interface.
  • Experience Management. There are some focused experience management tools.
    • An alternative approach is a simple grid in an MS Word document that details the breadth of legal experience/competencies across the lawyers of the firm.
    • You can also do this via a simple database. However, chances are that the lawyers won’t like the interface of that database.
    • With enterprise search, you can search time entries and matter profiles to infer experience
  • Document proofreading tools. Wordrake and EagleEye are very popular with attorneys.
  • What tools have been less successful in their experience?
    • Task Management System
      • lawyers like a resource that helps them organize themselves, but the lawyers are unwilling/unable to dedicate the time necessary to maintain the system.
      • Baker Donelson is using K2 to provide task management support

You’ve Created an Innovative Product/Service – Now What? #ILTAG122 #ILTACON

Session Description: As more law firms experiment with innovation by delivering their services in new and interesting ways, many encounter logistical challenges. How do you price the products/services in a market bereft of competitors? How should you go about positioning and selling these innovations? How will you maintain and scale operations to keep the product/service updated? What’s the best way to deliver support for the new product or service? We will answer these questions and more as we present real-world examples of innovative products and services that have been attempted and actualized.


[These are my notes from the International Legal Technology Association’s 2017 Conference. I’m publishing them as soon as possible after the end of a session, so they may contain the occasional typographical or grammatical error. Please excuse those. To the extent I’ve made any editorial comments, I’ve shown those in brackets.]


  • Background.  The firms of these speakers are at different stages of innovative product development.
  • First Things First.
    • Making the Business Case
      • Define the value to the firm
        • how does this move the needle for the firm?
        • Will it be a value-add, a revenue-generator?
        • Does the value warrant the level of effort the project is likely to require?
        • How do you balance benefit to the firm and benefit to a smaller group within the firm (e.g., a partner’s vanity project)
      • Include the client point of view — this may be uncomfortable because it likely will require changes in the way your firm has always delivered legal services.
  • Governance.
    • Define governance and the decision-making process
    • Identify key stakeholders who should be involved — be prepared to include a wider group of stakeholders than you would normally involve.
      • it is better to have them in the tent early than have them torpedo the project later
    • Determine Go/No Go
      • a strong business case is critical for buy-in
      • be prepared for naysayers — sometimes they signal their lack of support early in the project so be alert and address their concerns early
      • collaborate early and often
      • be willing to experiment, which means being willing to fail
        • sometimes this means “going rogue,” operating under the radar until you have useful project data to share with the fim
        • this requires having some buy-in from firm leadership to provide the latitude for action and resources
  • Adoption/Change Management
    • any time you create a product or service that will disrupt current products and services, you should expect significant pushback
    • develop a change plan prior to launch
    • identify and secure commitments from internal champions
    • consider engaging a third party change management consultant — especially if the disruption you are planning will have a major impact internally
  • Create strong internal marketing and communications
    • be transparent from the beginning
    • be sure to explain “what’s in for me” — from the perspective of everyone who is likely to be affected by the change
      • this is especially important if the product/service is likely to reduce billable hours
        • in this case, focus on the longer term benefits to the firm
        • bring evidence to the audience — market research and peer evidence (from other partners or even other firms)
    • attorneys are on the front line — they need to have enough information to market it to each other and to their clients
    • keys to success
      • know your audience
      • have a strong messaging/communication plan
      • focus on education
      • use multiple resources — videos, case studies, talking points, pricing, contact info for the dedicated sales team, etc.
        • create an internal, central place where attorneys can find all the resources they need in single place
    • Don’t rely solely on email — in many firms people are already inundated with email. Therefore, travel to your offices and schedule in-person meetings.
  • External Marketing and Communications.
    • use experience data to drive strategy — this means that you REALLY have to get to know your buyer and their sales cycle, and then align your marketing/sales plan to that cycle
    • create a realistic action plan
      • test the waters before a major launch — even asking clients for their thoughts on the planned product/service
      • don’t forget to use indirect marketing as well — put clients in touch with other clients who have experienced success with this product/services
    • Keep it simple
      • be direct but impactful — attorneys, especially, may need coaching to stick to the key powerful facts
      • appeal to your audience
    • Adjust and try again
  • Pricing.
    • Don’t start by telling your client what they need and what you are going to charge them!
    • Rather, start by ASKING what your client what the need and what their comfort level is with respect to pricing. It isn’t just about cheap, it might be about increased predictability/reliability in pricing.
    • Understand the worth of your product/service
      • what is the true value add — TO THE CLIENT
      • is it a revenue generator
      • is it a brand builder
    • Understand your client
      • consider customized pricing
      • understand how they are using internal legal or consulting alternative resources
    • What will the market bear?
      • what is your firm’s experience data on this?
      • what are other firms charging?
        • this information doesn’t usually come via online research
        • you can ask clients — they are usually willing to tell you
        • you can ask colleagues in other firms
  • Ownership: How to keep it going.
    • These projects can require substantial investment by the firm — consider having a partner lead the effort (on the
    • You can be a victim of your own success — you now have a product/service you have to maintain for your clients
      • keep the produce up-to-date regarding design, technology, process
      • technology products often require external support
      • keep content fresh
      • monitor the market on an ongoing basis
  • Evolving the Product.
    • Keep track of client satisfaction
      • you can use surveys, face-to-face feedback, third-party consultants
      • gather input that drives change
    • plan for resources to maintain and improve the product/service in planned phases
    • usage data
      • you may need to refine your collection/analysis as your product matures

Beyond the Hype: Putting AI to Work at Liberty Mutual #ILTASS18 #ILTACON

Session Description: Neota Logic and Liberty Mutual will share details of the design and implementation of Neota Logic’s AI-driven expert system platform to automate document creation and drive internal efficiencies at Liberty Mutual.


[These are my notes from the International Legal Technology Association’s 2017 Conference. I’m publishing them as soon as possible after the end of a session, so they may contain the occasional typographical or grammatical error. Please excuse those. To the extent I’ve made any editorial comments, I’ve shown those in brackets.]


  • It’s just software.  According to Kreutzer, “It’s only AI when you don’t know how it works; once it works, it’s just software.”
  • Why did Liberty Mutual even consider using AI?
    • As a life long geek, Jeff thought that AI was a cool thing to explore. The purpose of this exploration was to try to reduce some of the cognitive load, some of the lower-value tasks that slow the legal department’s lawyers down. Ultimately, it was intended to allow the lawyers to focus on higher-value tasks.
  • Forget the pixie dust. With all the hype about AI, it is easy to be dazzled by the “pixie dust” aspects of the technology. However, in Marple’s view, 90% of the work in an AI deployment project involves capturing, organizing and cleaning the relevant data. Without this essential work, you cannot get to the pixie dust.
  • Start with small, low-risk projects. To prove the value of the technology, they started with their internal non-disclosure agreement (NDA) process. (As with many companies, the NDA process was more onerous and time-intensive than the corporate legal department — and their internal clients — would like.)
  • Their timeline to deployment. The bulk of the time was spent exploring the technology options and identifying the right use case for their pilot. The actual process of creating and deploying their instance of Neota Logic took two to three weeks.
  • Their new toolbox. They are creating a toolbox that includes Neota Logic to be deployed throughout the organization. Because Neota is a business-facing tool, the business folks can use Neota to improve processes without involving a single developer.
  • What skills and competencies best support AI deployments?
    • Legal Engineers are perfectly placed to support AI deployments. They can be lawyers or business analysts. However, they need an understanding of the law and they must be willing to “lean in to technology.” Plus, they must have a healthy curiosity about the technology.
    • Some law schools are training their students in legal technology by using Neota Logic in their courses. Faster than we know it, most law schools will be training students in legal technology. And, when they enter law firms, they will simply practice law in a tech-enabled way.

Keynote: Transforming the Business of Law with Cognitive Computing – Watson Legal #ILTAKEY2 #ILTACON

Session Description: “Artificial intelligence” (AI) is arguably an overused label for a vast array of technologies that vary in features, complexity and benefits. What is AI, and how do you know if you have a problem it can solve? Brian Kuhn (founder of IBM Watson Legal) and Shawnna Hoffman ( IBM Global Co-Leader, Global CoC) will address key misconceptions around artificial intelligence and cognitive computing and share insights from business-of-law use case workshops IBM conducted with corporate legal departments and law firms over the last two years. Brian and Shawnna will identify patterns of customer interest and success related to the application of artificial intelligence and cognitive computing to the legal domain, and he’ll discuss the top facts everyone needs to know about what AI can — and can’t — do.


  • Brian Kuhn, Esq., Global Leader and Co-Creator of IBM Watson Legal
  • Shawnna Hoffman, Global Cognitive Co-Leader, Global CoC

[These are my notes from the International Legal Technology Association’s 2017 Conference. I’m publishing them as soon as possible after the end of a session, so they may contain the occasional typographical or grammatical error. Please excuse those. To the extent I’ve made any editorial comments, I’ve shown those in brackets.]


  • What is AI.  A type of tool to extend our cognitive abilities. While computers have extended our cognitive abilities, 88% of the data we have created is “invisible” to us. It is rendered invisible because it is unstructured — narrative creative for consumption by humans rather than computers. The language is context heavy and may be rife with ambiguity. Historically, computers have been unable to parse unstructured data because they did not understand human language.
  • AI is an umbrella term. There is no single agreed definition. It is a constellation of capabilities (e.g., machine learning, neural networks, etc.)
  • Cognitive Legal. All cognitive systems understand human language, they can reason to extact ideas, they can learn from past results, and then they can interact in a natural way. They can read 800 million pages per second and, in the background, derive meaning and form hypotheses based on this work.
  • Cognitive systems democratize knowledge. They have the capacity to read and understand more content than humans could do in a lifetime. And the cognitive systems can do this in a matter of days or weeks. Better still, while human intelligence is focused on discrete bodies of knowledge that leads to deep expertise, cognitive systems can consume and comprehend multiple domains of knowledge and derive new insights that humans could never reach by themselves. This ability to synthesize information from across domains of knowledge will profoundly alter professions.
  • How do you know you have a problem AI can solve? IBM’s Cognitive Legal practice focuses on the business of law rather than the practice of law. This was the choice of their clients. They have been identifying use cases that:
    • provide tangible business value
    • identifiable key performance indicators
  • Identifying good use cases.
    • Wha are the painpoints?
    • What is hte business value of this use case?
    • Has the content that will fuel the solution been identified?
    • The best use cases provide
      • organizational beef
      • end-user benefit
      • strategic alignment
      • speed to implemention
  • Examples of Use Cases.
    • Outside Counsel Insight — designed to reduce spend on outside counsel by
      • automating manual invoice review processe to detect billing anomalies,
      • producing insights into trends and patterns that demonstrate the quality of outside attorney performance, resulting in outcome and price consistency,
      • producing brenchmarking insights at the case and line-item level that establish a reasonable level of effort for repeatable legal tasks, which then helps justify fixed-fee pricing arrangements versus hourly billing.
    • Early Case Insights
      • Consuming historical work product by outside counsel to provide appropriate advice for repeatable work and new insights to address current needs — without reinventing the wheel or pay again for the work to be done
    • Expertise Finder
      • using internal and external data
      • it could be used across a network of organizations or law firms. The underlying data would reside in public filings or even in a private blockchain.
  • Cognitive Legal “Cartridge”. We will be training “cartridges” in what and how the people of your firm think. Others could then purchase these cartridges and point them at their own data sources so that they can derive new insights by using what and how the people who trained the cartridges think. IBM Watson has done already by having three oncologists at Sloan-Kettering train a cartridge, and then applying that cartridge to the medical data collected by the largest hospital in Thailand.
  • Closing Insights
    • “Cognitive is a mirror you hold up to yourself.”
    • Use AI internally first. The power of AI lies in turning it onto your proprietary data. The richest data source is not located on the internet. In fact, 80% of data resides inside organizations and behind firewalls.
    • “We think about Watson in the context of AI — not artificial intelligence but augmented intelligence.”
    • Access to justice — 80% of people involved in domestic violence disputes represent themselves pro se in court. If there were a cartridge trained by domestic violence legal experts, that cartridge could help make sense of the data for each of those cases.
    • Legal Blockchain +AI:
      • blockchain contains enormous stores of data
      • AI can parse that data and make sense of it
      • blockchain and AI could be combined to create trusted and transparent self-executing contracts

Innovative Law Department of the Year #ILTAG20 #ILTACON

Session Description: In this session, short-list nominees for ILTA’s 2017 Distinguished Peer Award for Innovative Law Department of the Year present their innovations. Attendees at this session will vote for their favorite, and the winner will be announced on Tuesday evening at the Awards Dinner.

The nominees are:

  • NetApp, Inc.
  • American Electric Power
  • Telstra

[These are my notes from the International Legal Technology Association’s 2017 Conference. I’m publishing them as soon as possible after the end of a session, so they may contain the occasional typographical or grammatical error. Please excuse those. To the extent I’ve made any editorial comments, I’ve shown those in brackets.]


  • NetApp.
    • Connie Brenton (Chief of Staff/ Director of Operations at NetApp) and Justin Hectus (CIO/CISO at Keesal, Young & Logan) co-presented. They credited ILTA and CLOC for bringing them together and giving them the opportunity to create a tool intended to be of benefit for the entire industry.
    • Working collaboratively, they used ThinkSmart to create automated workflow for eDiscovery document production.
    • NetApp designed the initial workflow, which included components for outside counsel to complete.
    • Keesal, Young & Logan provided time and resources to build the application. They made this contribution as a “gift to the legal community.”
    • ThinkSmart proved to be a flexible, easy-to-use tool. No matter how complex the workflow was, this team found that they could design the workflow on the fly using a simple drag-and-drop process.
    • An unexpected benefit of using ThinkSmart was that they discovered that they could fold into their work some prior work done at Keesal, Young & Logan to create an automated workflow for eDiscovery document collection. This meant that they had an end-to-end automated approach without having to reinvent the wheel.
    • They collectively invested about 20 hours of dedicated work on the tool.This was significantly dwarfed by the massive efficiency gains the workflow tool generated and will generate.
  • American Electric Power.
    • AEP was founded in 1906 and is one of the largest electric utilities in the US. AEP has just announced that they will be building the largest wind farm in the country.
    • This project involved AEP Legal moving to the cloud with iManage.
    • The project started as a primarily technical improvement:
      • They had been using iManage on premises since 2011. But they had concerns with stability, upgrades, and capability to handle disaster recovery.
      • When they started speaking to the lawyers, however, they learned that the problems went beyond merely technological. They discovered that the lawyers did not trust the system and, in fact, did not use even close to the full capabilities of the tool. Further, some of the things they were requesting were actually already available in the version implemented on premises, but the lawyers did not realize this.
    • Once they identified what the lawyers needed, and what would work better for them, they realized that they needed to move to the cloud to meet their new requirements.
    • Before proposing the move to the general counsel, they had AEPs security experts verify the security of the proposed iManage cloud.
    • One added benefit of their move is that they are now able to collaborate more easily with law firms and government agencies via the cloud.
  • Telstra.
    • Telstra is a leading telecommunications company headquartered in Australia.
    • Since the global financial crisis, their legal team has faced year-on-year budget cuts of 10%. So they operate under pressure to be more efficient while remaining effective.
    • Between 2013 and 2016 their team transformed into an award-winning legal department This involved adopting an Innovation Culture and an Innovation Agenda.
    • The Legal Innovation Forum
      • This forum is a project created collaboratively between Telstra’s legal department and Herbert Smith Freehills (HSF). The forum convened 15 Telstra lawyers of varying seniority who worked with HSF design-thinking experts to tackle specific efficiency issues such as reducing time-wasting meetings, eliminating pointless internal reporting, eliminating unnecessary legal review. Then they prototyped and tested their solutions in eight-week “sprints” within the Telstra legal department. Here are their results:
      • Reduce time-wasting meetings — a few years ago, their legal team collectively spent 60,000 hours in internal information-sharing meetings each year.
        • As a first step to tackling this problem, the Legal Innovation Forum characterized all internal meetings as either information-sharing or decision-making.
        • Then they ran a trial: they were allowed only 2.5 hours per week of information-sharing meetings but permitted internal lawyers to participate in unlimited decision-making meetings, provided the meeting organizers informed them in advance of the decision they needed to make AND the organizers invited to the meeting only those lawyers whose participation was critical.
        • According to an article by Zach Warren, these new procedures yielded impressive results: “a 52 percent reduction of time spent in both kinds of meetings, which represented more than 2,000 total hours saved across the Telstra legal department.” Plus lawyers surveyed after the fact reported a striking improvement: “92 percent of attorneys surveyed after the trial felt they weren’t missing anything by not attending those meetings, and 90 percent said they felt even more productive.”
      • Eliminate pointless internal reporting — an internal report to senior managers that evolved to over 40 pages each week, involving over 75 hours collectively per week on the part of Telstra’s lawyers. By determining which critical issues really required regular reporting, and then designing a streamlined new template for the report, they were able to save a considerable portion of those 75 hours.
      • Eliminate unnecessary legal review — over time, they had developed the practice of having legal department lawyers review internal communications. However, this ate up significant amounts of lawyer time. Instead, the Legal Innovation Forum identified the types of internal communications that were high-risk and really required legal, and then instituted a no-review policy for all other internal communications. This resulted in a 29% reduction in hours spent reviewing.
      • Automate common workflows — a review by the Legal Innovation Forum revealed that the practice of drafting/reviewing confidentiality agreements ate up too much time — the equivalent of two FTE lawyers. In response, the Legal Innovation Forum created an automated process to produce and track NDAs. Now lawyers do not have to participate in routine NDA drafting/review.
    • Lessons from their experience:
      • You need good data –otherwise you cannot measure your progress. Once you have your data, you need to be able to explain and defend your data.
      • Look for early, small, easy wins that can provide momentum for your innovation efforts. Innovation always involves some failure so it is good to have a few wins under your belt before those failures arise.
      • Don’t be paralyzed in a hunt for the perfect. It is better to act than to wait until you have figured it all out.
      • People resist loss not change. Always think about what people need to give up. And then think about how to address this.
      • Don’t reinvent the wheel. Look at others inside and outside your company who may have tackled similar problems. Then, learn from them.

The Future of Search #ILTAG31 #ILTACON

Session Description:

What does the future hold for search in law firms? How far will legal knowledge management push the search envelope beyond documents, matters and expertise? Further than you think! Let’s explore the future of search, including integrating search-enabled applications, broadening the search scope available to the mobile professional, incorporating artificial intelligence, enhanced visualization and the use of predictive analytics, and the use of machine-generated metadata  to improve search results. See how search can fulfill its promise of making your lawyers more effective and firm-client relationships more collaborative.

Identify possible search functions
Visualize the future of search in your law firm
Learn how you can prepare for the future of search
Hear Case Studies from two law firms to improve search


  • Doug Freeman, Knowledge Systems Manager, White & Case
  • Peter Wallqvist, VP of Strategy, iManage/RAVN
  • Todd J. Friedlich, Senior Manager of KM Technology and Innovation, Ropes & Gray
  • Glenn LaForce, VP of Knowledge Management, Aderant

[These are my notes from the International Legal Technology Association’s 2017 Conference. I’m publishing them as soon as possible after the end of a session, so they may contain the occasional typographical or grammatical error. Please excuse those. To the extent I’ve made any editorial comments, I’ve shown those in brackets.]


  • Drive User Adoption.
    • Maximize points of contact — Don’t put search under a separate tab. Instead, put a search box on every page of the intranet, on the laptop, on their desktop, on their Citrix desktop
    • Reduce user “click” – Enable type ahead so users don’t have to type too much
    • Understand Adoption Stats — look at content usage (are the lawyers looking for legal documents or specific tools?). Plus tie in other data sources. For example, combine financial stats with your user stats to see if there is a correlation between high users and profitability.
    • User adoption — is the “document” the primary container of knowledge? Lawyers and professionals don’t think only in “documents”. There are other containers of knowledge that are natural for users. So deconstruct documents into their constituent parts, such as clauses. Focus first on the business problem they are trying to solve in their search.
  • Provide an Intuitive Search Experience.
    • On the web, search seems intuitive. Unfortunately, it is too often a completely different experience inside the enterprise.
    • To improve the search experience, move from keyword counts to contextual understanding to determine relevance. Google did this by using hyperlinks to validate relevance. Because you may not have hyperlinks within the firm, you will need to bring other internal and external data to bear. (White & Case uses internal and external (from vendors) data to help determine relevance.)
    • Make it easier to find your content by increasing the refinement options beyond those provided by the system that is the source of your data. Ropes & Gray merges data from a variety of sources to provide more ways to refine/filter their search results.
    • Leverage search intent to provide customized refinement options/weighted results. For example, increase the weight of rankings based on the identity of the searcher.
  • Provide a More Complete Set of Information.
    • How to bring in more external sources to augment search results? Earlier, the only way was to use federated search. Then some external vendors (e.g., Practical Law Company) allowed firms to index PLC content, which drove much higher levels of PLC use. Now some external data sources are allowing firms to index on premises subsets of their vendor content (e.g., treatises).
    • When bringing in vendor data, also bring in snippets that allow the user to preview the content.
    • These techniques really drive higher levels of usage of vendor content.
  • Lowering the Cost of Custom Search-Based Applications.
    • A search platform is in the unique position of being a universal portal/repository of information. It can show data from disparate systems in a single interface.
    • New search systems should have a flexible API to enable creative uses of their search tool.
    • Repurpose existing system: For example, can you repurpose the custom macros or javascript you have already developed?
    • As vendors provide more flexible APIs, it will be possible to combine/recombine those APIs without a developer
    • Ropes & Gray is prototyping tools for non-developers/power users:
      • Data Dictionary and Custom Aggregation Tool to allow joining of data by end/power users
      • RAD tools such as SharePoint/ Nintext so power users can create workflow-enabled, dynamic forms to power search.
    • The more you can lowering these barriers, the more you lower the cost of development by enabling work by non-developers.
  • Trends for the Next Five Years.
    • Chat bots — for example, Do Not Pay is a chat bot for fighting tickets. It guides the user through a process online. Can we extend this to other operations? It allows the computer to figure out what you are looking for and then deliver it to you. It allows you to “go on a journey” just like you do when you speak with a person.
    • Cloud — if you have applications in the cloud, how can you access and index them efficiently in your on prem system? But going beyond this, should search be a cloud service? (Where the document is on prem but its text is “represented” and indexed/searched in the cloud.)
      • Search will have to “follow the data” where it lives. Then we will need appropriate policies/governance so clients are confident about security and confidentiality.
      • One day, clients may ask to search their documents on your system by using your index.
    • Time and Billing — use AI to determine key characteristics of time records, then use those results to enrich search. In the next five years, there will be a convergence between AI and search.
  • Security and Rights Management.
    • This has been a perennial challenge for search.
    • A proper enterprise search engine should respect your security rules. But the humans in charge need to specify those security rules for the search engine.
    • GDPR is an incredibly onerous data protection law that is coming into force in Europe. Now it is a criminal offense to expose data that should not be accessible. On the other hand, a properly-implemented search engine that respects security rules should help address this problem.
    • When a firm is required to lock content down on a need-to-know basis, you can still derive value from search because sometimes the metadata exposed by the search engine is as important (or more important) that the underlying hidden document.

What Uber Can Teach Us About Feedback #ILTAG21 #ILTACON

Session Description:

Contemporaneous customer feedback has become an integral component of the economic model for many service industries. From restaurants to ride shares, a wide array of companies leverage this feedback to improve their offerings, connecting them more meaningfully to what their customers value most. Why has law not implemented a similar feedback loop between firms and clients? Are there structural impediments to real time feedback? How might we go about constructing a system where such feedback is encouraged and used to improve legal services? Come hear a law department leader, an AmLaw executive, and an itinerant futurist share their thoughts on the availability of feedback in law today, the type of contemporaneous information we should be collecting, how it presents opportunities to firms and clients alike, and how organizations like ILTA can play a pivotal role in fostering the collaboration needed to develop a meaningful feedback system.


  • Understand the benefits of meaningful feedback for firms and clients
  • Generate ideas regarding the type of feedback valuable to improving the services firms offer
  • Identify how organizations such as ILTA can be pivotal in creating feedback standards


  • John Alber, ILTA Futurist
  • Michael Haven, Senior Director, AGC, and Head of Legal Operations at Gap Inc.
  • Jeffrey C. Schwartz, Head of Legal Operations Innovation at Hinshaw & Culbertson

[These are my notes from the International Legal Technology Association’s 2017 Conference. I’m publishing them as soon as possible after the end of a session, so they may contain the occasional typographical or grammatical error. Please excuse those. To the extent I’ve made any editorial comments, I’ve shown those in brackets.]


  • Make sure you are requesting feedback for the right reasons. Initially, some firms undertook client feedback programs because they heard it “made the client relationship stickier.” This is not the best motivation. The better motivation is to elicit information that helps you provide better client service.
  • Make sure you are asking for the right kind of feedback.  It is tempting to ask for qualitative feedback: How responsive do you find us? How well do you think we understand the critical area of law? Rather, ask for quantitative feedback regarding efficiency. Clients invariably are more interested in the feedback that has a clear positive impact on service delivery.
  • Think about how clients prefer to evaluate. Some clients are developing internal tools to measure the service of their various external lawyers. Ask early what they are tracking, what is important to them.
  • Frequency. Don’t wait until the end of a matter before requesting feedback. Clients would prefer at least quarterly check-ins.
    • At Uber, every single touch with a customer gets rated.
    • No firm represented in the audience seeks feedback after every client interaction. There were very few in the audience who did it more frequently than annually.
    • As tempting as it might be to ramp up to requesting evaluations after every interaction, keep in mind that this may be more than the client wants or can handle. Ask for the client’s preference.
  • Granularity.
    • How granular should the feedback request be?
    • You need to calibrate this depending on (1) what’s important to the client and (2) the quality of service delivery.
    • The net effect of fairly granular and regular feedback it that it causes greater accountability and engagement on the part of the lawyers.
  • Metrics.
    • There are data science consequences and human behavior consequences to the type of metrics you track. So check the science as you are designing your system.
    • See the comments above about qualitative and quantitative metrics — and what clients care about.
  • Barriers.
    • Unlike your interaction with an Uber driver, this is an ongoing relationship rather than a transactional relationship. Therefore, if you give your lawyer a bad rating, it can be uncomfortable to work with that lawyer again. Does this lead to grade inflation. (Think about grade inflation for personal assistants in law firms.)
    • How do you create enough anonymity that allows people to provide bad feedback.
    • We need to learn how to have the “courageous conversation” in a professional, constructive way. A key to this is to make obvious that you are invested in the relationship and really want it to work well.
    • Increasing the frequency of feedback helps focus on small problems early rather than huge deferred problems. These small conversations are understood as “tweaking” a good relationship rather than upending it.
  • What if every member of the team was rated? Bryan Cave had experience with a client feedback system that was connected with a bonus structure.
  • Where to begin in your firm?
    • Michael Haven — Start by talking to your client. They probably already have an operations group that is measuring various things about your firm. So ask them at the beginning of an engagement what is important for them.
    • John Alber — Mine Your Own Data:
      • Begin by asking how sticky a client relationship is.
      • Then, ask why.
      • Then, discern the data that appear to correlate with high and low stickiness.
      • Then present these data to your firm in a way that intrigues them.
    • Jeffrey Schwartz — Increase transparency within the firm: Give the firm data that helps the lawyers ask productive questions about existing client relationships.
      • are we noticing a difference in the rate of bill payments?
      • are we noticing an increase/decrease in demand?
    • Collaborate with the Marketing group of your firm. They are voracious producers/consumers of data.
    • Pay attention to patterns of behavior in your document management system. What does this indicate about efficiency? About approaches by specific client teams?

Pablos Holman Keynote: Innovate or Die Trying #ILTAKEY1 #ILTACON

Session Title and Description: Innovate or Die Trying: From the Mind of a World-Renowned Hacker

“If you’ve been to an ILTACON keynote presentation before, you know we’ve featured futurists. To start off our 40th ILTACON, we’re turning up the dial with a futurist, inventor and notorious hacker who has a unique view into breaking and building new technologies, which leads to new approaches to solving world problems. Pablos works on projects that assimilate new technologies and make wild ideas more practical. What are computers yet to accomplish? What advancements will legal technology experience? ILTA members are concerned with disruption by legal startups and the need for innovation. Pablos will help us think differently about how we could solve some of our legal technology challenges. Katie DeBord, Chief Innovation Officer at Bryan Cave, LLP will join Pablos at the end of his talk to ask him some questions — her own and those the audience posts in the mobile app during the session. Then we’ll open the floor for live audience questions.”

[These are my notes from the International Legal Technology Association’s 2017 Conference. I’m publishing them as soon as possible after the end of a session, so they may contain the occasional typographical or grammatical error. Please excuse those. To the extent I’ve made any editorial comments, I’ve shown those in brackets.]


  • Today’s objective.  To help us start thinking like hackers. In Holman’s view, “no one ever invented a technology by following directions.” What it takes is not just thinking outside the box, but actually taking the box AND its contents apart to figure out how they work and how they might work differently or better.
  • Computers are the key. The key to prescience is to look around the environment to find functions/areas that do not yet leverage computers. Then, find a way to apply computing logic/power to that. This is a great way to come up with fabulous new interventions.
  • Malaria is a great opportunity for hackers. Malaria is one of the greatest causes of death of children. And it is entirely preventable. The prior methods for dealing with malaria (e.g., DDT) are neither scalable or sustainable. So he hired hackers to hack the mosquito/malaria system. They “bought some junk” on eBay and then six weeks later, they were able to track bugs and show that tracking on a computer. Next, they developed a laser that can identify the characteristics of each mosquito tracked. Finally, they use a lethal laser to kill the malaria-carrying mosquito. This is an example of using hacking mindset + computing power + imagination to tackle a wicked problem.
  • Imagination Constrained. Holman says that for years we were technology constrained. (Just think about the processing power of early computers.) Now we have tremendous processing power in smartphones. Yet we use these phones to play games, make rude noises, exchange photos, etc. In his words, “we used to be resource constrained. Now we are imagination constrained.”
  • The Nuclear Challenge. Current nuclear reactors were designed in the era of the sliderule. They are only 7% efficient. The rest of the uranium is waste and generates tons of nuclear waste that is simply being stockpiled until our children can figure out what to do with it. Holman and his team ran Monte Carlo simulations to figure out every possible reaction inside a nuclear reactor. Then they redesigned the reactor to be 100% efficient. Plus, the genius part is that they use all that stockpiled nuclear waste as the fuel.
  • Rapid Iteration. Software is eating the world today because of rapid iteration. The tech sector continuously A/B tests critical aspects of the software and then upgrades on the fly. This continuous deployment is far faster than the 18-month production cycles in old-school manufacturing.
  • It’s a New Era. Biological evolution allowed the human species to survive and win. The name of the game was basic survival. Now, however, if we do not have to worry about mere survival, what should we do? We need to use our brains, our imaginations, our technology to evolve to the next level — beyond Maslow’s hierarchy of needs (at least with respect to the lower parts of the pyramid, the quantity of life issues). But, is technology really helping us with the higher levels of they pyramid? Social, Self-Esteem, Self-Actualization? Technology is not solving the problems related to these higher levels, the quality of life issues.
  • Humans must make choices. We have to make better choices about how we use our tools. And we need to make better choices about how we interact with those tools. Robots do only what we train them to do. Unfortunately, humans are very bad role models.
  • “Vision without action is a daydream. Action without vision is a nightmare.”

Guiding the Goats in Your Firm

Continuing with my recent menagerie theme, today let’s think about goats. Goats? Yes, Moroccan domestic goats, to be specific. As you can see from the picture in this post, these goats are quite extraordinary; they have the ability to forage for food — in treetops. Why is this interesting? Because by nature goats know how to climb hilly terrain but they do not know how to climb trees. So what gives?

Here’s how Nicholas Bakalar, writing in the New York Times, tells their story:

These domestic goats live in southwestern Morocco, where the climate is dry and in some seasons the only available forage is in the trees. So the goats climb up to get it.

Goats are good climbers — some sure-footed species live happily on mountains, leaping from ledge to ledge. But these domestic goats are not born with an ability to climb trees. They learn the technique as kids.

Their keepers help them climb, and they trim the trees to make it easier for the kids. The goats eventually learn to do it themselves. In the autumn, when there is little food on the ground, they spend most of their time grazing the treetops.

Because the readers of this blog tend to be smarter than the average bear (sorry — I am obsessed with animals this summer!), you will probably have figured out exactly where I am headed. This story has some great lessons for knowledge management personnel:

  • The goats in your firm — and you can define who is a goat in your firm! — are not born with the natural ability to do most of the things you and your knowledge management colleagues know how to do.
  • To train goats properly, you must start by teaching them when they are kids — grab them when they are summer associates and help them learn how to work efficiently and effectively. Above all, teach them early to question “the way we’ve always done it around here.”
  • You will need to provide support for the kids until they master the necessary skills. This support is especially critical because some older billy goats will be dismissive of the value of the knowledge you have to impart. And, some of those billy goats will be worried that these new skills will reduce billable hours. So you will have to help the kids withstand the negative pressure from the goat gerontocracy.
  • You will need to trim the trees to make it easier for the goats.  This means creating sensible, frictionless systems and then removing any unforeseen roadblocks that might arise.
  • The goats must eventually learn to do things for themselves. You cannot hold their hooves forever.
  • This is a matter of survival — it will help them be productive in the lean seasons and the busy seasons.

So here are a couple of question for you: Can the goats in your firm graze in treetops yet? If not, when will you start guiding the goats in your firm so they can learn to do things they would otherwise never be able to do?


For more critical knowledge on goats and goatherds, see this extract from the greatest movie ever made 😉

[Photo Credit: Arnaud 25, Wikimedia]


Managing Your Monkey and Monster

“Summertime, and the livin’ is easy…”

If only. In fact, over the last few years, my summers have become increasingly busy. This is partly due to a terrific course I’m teaching at Columbia University called Collaboration at Scale. It’s also due to some speaking engagements on interesting and challenging topics like Thriving in Change (at the American Association of Law Libraries PLLIP Summit) and Sustainable KM (at ILTACON 2017). And it’s also because of some marvelous consulting clients who keep me busy and on my toes.

With this much happening, there’s a big premium on getting and staying productive. But high productivity does not happen by accident. It takes thought and planning. And, it takes careful management of your monkey and monster.

What monkey and monster?

Tim Urban (Wait But Why) brilliantly described the Instant Gratification Monkey and the Panic Monster in his hilarious TED talk: “Inside the Mind of a Master Procrastinator.” (See below) Along with this monkey and monster, he also introduced the revered figure we wish actually ran our brains: the Rational Decision-Maker. However, for the average procrastinator, their Instant Gratification Monkey has an uncanny ability to hijack the Rational Decision-Maker’s careful plans. And then you drift until an imminent deadline wakes up your slumbering Panic Monster who puts you into a state of high anxiety. This allows your Rational Decision-Maker to take the reigns of control away from the Instant Gratification Monkey — at least until the immediate crisis has passed.

And so it goes. Until you exhaust yourself or encounter a deadline that you simply cannot meet using this pattern of behavior.

If this sounds at all familiar, you will know that you have to find ways to stay focused on the things that are more important than the things that interest your Instant Gratification Monkey. The problem is that many of those important things may actually not come due for quite a while, if at all. For example, studying now for a future career or laying the groundwork now for an interesting opportunity later.  Consequently, these important things for a hard-to-see future almost never cause your Panic Monster to wake up and scare away your Instant Gratification Monkey.  As a result, you almost never achieve these important things. So what to do?


According to the Merriam-Webster dictionary, to parse a sentence is to divide it into its parts and then identify those parts and their relationship to each other. Similarly, when you have a pile of things that must be done, break them down into their smallest meaningful parts, figure out which of these parts cause dependencies, and then tackle those first. The point of this exercise is to make the tasks so small that they pass under the radar of the Instant Gratification Monkey. Better still, if you can make those tasks fun (or mindless) the Instant Gratification Monkey might be fooled into thinking those tasks were its idea after all.


Stephen Covey once famously said “Don’t prioritize your schedule, schedule your priorities.” But that can be easier said than done — especially in a law firm environment where everything seems to be urgent and everyone feels overworked. S0 how do we manage this? Theoretically, your supervisor should be helping you prioritize. But there are times when your supervisor cannot see the forest for the trees and, let’s face it, there are time when you simply do not want to let your supervisor know exactly how lost you are. To help with these situations, consider developing a productivity partnership with a trusted colleague. To do this, make the following deal with each other: that you will act as each other’s lifeline in moments of stress to help the other identify which next action that would be most productive. You’ll soon discover that there will be many times where you don’t actually need someone else to tell you what to do next, you just need someone willing to listen to you for five minutes as you figure it out aloud.


Scrum software development has reminded us of the value of short sprints of work to get things done. You can use this principle to tackle the tasks you have identified through the parsing and prioritizing steps above. Take the small task you have identified and commit to working on it for a very short sprint. This is not a commitment to complete it — that might freak out your Instant Gratification Monkey, your Panic Monster, AND your Rational Decision-Maker. Rather, it is a commitment to move things forward in a meaningful way. The trick to keeping your Instant Gratification Monkey on hiatus is to make the time period of this commitment short, say 20 or 25 minutes. After all, this is such a short period of time that it’s hardly worth it for the Instant Gratification Monkey to get out of its hammock. What that monkey does not realize is that if you can build up some rhythm to these pulses, you will be able to create momentum to complete the larger tasks as well.


To create that desired rhythm to your pulses, consider using the Pomodoro technique. This gives you a structured way of doing a short sprint followed by a guaranteed rest period. Provided you honor the time commitments for both the sprint AND the rest period, you should be able focus and get a great deal done. (Admittedly, that proviso is a big one but it does get easier with practice.)


While all of this seems perfectly rational and even doable, you should be aware that this disruption of the natural order is likely to send your Instant Gratification Monkey into a tailspin. And your Panic Monster may feel a little underemployed. So you need to find a way to placate them so they are not motivated to disturb your productivity. You can help placate your Instant Gratification Monkey by ensuring that your Pomodoro breaks are spent on truly fun things or things that really make you feel better about life. And then, because you’ve had an insanely productive day, walk away at the end of the allotted time and really enjoy the leisure you have earned. As far as your Panic Monster goes, it is actually to useful to keep the monster around but on a highly reduced schedule. The monster can help keep the monkey at bay so that you can focus on getting to the end of each sprint. So while you won’t have a full-on panic attack, you will have that thrill of urgency that keeps you on your toes and focused.


If you’d love a fun and educational break from your productive streak, here are links to Tim Urban’s TED talk, as well as some terrific blog posts he wrote on the topic. Enjoy!

Why Procrastinators Procrastinate

How to Beat Procrastination

The Procrastination Matrix

[Photo Credit: Tim Urban]