Above and Beyond KM A discussion of knowledge management that goes above and beyond technology.

Awards & Recognition

Subscribe to Above and Beyond KM

Subscribe in a reader

Enter your email address:

Delivered by FeedBurner

Facebook

Disclaimer

This publication contains my personal views and not necessarily those of my clients. Since I am a lawyer, I do need to tell you that this publication is not intended as legal advice or as an advertisement for legal services.
  • The Metrics Mess

    I recently saw the perfect illustration of how we can get ourselves completely tangled up in unproductive activity by measuring the wrong thing. In this case, it was someone on Twitter who thought they had hit the jackpot because they had hundreds of followers. Further, this person was offering advice on how to increase the number of followers his readers had. This struck me as misguided at best. To be honest, there are folks I follow whom I’m sure don’t realize I exist. Equally, there are folks who follow me, but I’m largely oblivious to them because our paths don’t cross very often. So the numbers alone don’t tell the whole story and may, in fact, tell a misleading story.

    The real issue isn’t size of following as much as it is scope of impact. How many of these folks are really paying attention to you? How many do you actually affect? Unless you know this, you don’t have a good understanding of your interaction with Twitter. Admittedly, there are Twitter stars whom everyone likes to follow. And, assuming we follow because of their established reputations, we’re more likely to pay attention to what those Twitter stars say. For the rest of us in the Twitter mob, however, the number of our followers is a poor (and possibly inaccurate) proxy for our impact.

    Coming back to law firm knowledge management, take a moment to consider whether your efforts to measure the wrong thing are leading you into unproductive activity. Don’t focus on bulk — focus on impact. For example, counting how many times a particular document is opened via your portal or document management system may be interesting but not helpful. What you really want to know is how many times was it opened and actually used? And, how often was it exactly the thing the user was searching for? In the latter two cases, you learn much more about the quality of your content and the quality of your search engine.

    Consider the following: a document was opened 10 times and used each time, but then opened 20 times and discarded because it was not on point. For someone looking at bulk alone, they’d say, the document was opened 30 times, declare victory and go home. However, someone measuring impact would say it was used 10 times not 3o, and then would ask why. When you ask that question you create the possibility of learning and insight. That’s when you know you’re on the path to using metrics intelligently.


    [permission to use granted under a creative commons license]

    Published on November 20, 2008 · Filed under: KM, knowledge management, Metrics, Twitter;
    2 Comments
  • Mary Abraham

    Oz Benamram e-mailed me the following observation:

    “To measure the real impact you also have to check who are the people who used it and how many times they won/lost the case when reusing that document… The story is, as often, in the meta-meta-data.”

    - Mary

  • Mary Abraham

    As usual, Oz is absolutely right.

    If you’re serious about achieving true success and representing your performance accurately, you’ve often got to do a great deal more than merely report raw usage numbers. And, above all else, you have to start by tracking the right things.

    - Mary