Thanks very much for digging deeper into the quality of metrics. Many glibly
talk about the need for metrics, but few speak as frequently about the
inherent problems with metrics. Metrics, like all numbers, are
approximations. Our first job is to ensure that the metrics we choose to
collect are close enough approximations that they answer the questions we
need to answer. You're absolutely right that poor approximations, or
substitute metrics as you call them, can cause more problems than they
solve. A brilliant interpretation of the wrong data doesn't help anyone.

- Mary

Thanks very much for digging deeper into the quality of metrics. Many glibly
talk about the need for metrics, but few speak as frequently about the
inherent problems with metrics. Metrics, like all numbers, are
approximations. Our first job is to ensure that the metrics we choose to
collect are close enough approximations that they answer the questions we
need to answer. You're absolutely right that poor approximations, or
substitute metrics as you call them, can cause more problems than they
solve. A brilliant interpretation of the wrong data doesn't help anyone.

- Mary

This post was mentioned on Twitter by VMaryAbraham: New Post: Metrics Need Context http://bit.ly/d6ELVq (Getting the numbers is easy. Understanding them is harder.) #KM #e20…

Temperature is a substitute metric for what really matters in this case, “How warmly should I dress.” Providing context is a start, but it really requires considerable context to make this metric meaningful. How sunny is it? How windy is it? What will the weather likely be in two hours? And of course, as you note, is it Miami or NYC? Assemble all that context, and you still have a complex system that doesn't directly address the question you meant to ask.

Substitute metrics are as evil as bad, incomplete, or mismeasured metrics. They pretend to information they don't actually convey.

As H.L. Mencken said, For every complex problem there is an answer that is clear, simple, and wrong. I love metrics, but they matter only when at least as much work is done on assuring that they directly measure what you need to change as is done on taking the measurements themselves.

— Steven B. Levy
Author, “Legal Project Management: Control Costs, Meet Schedules, Manage Risks, and Maintain Sanity”

Thanks very much for pushing the analysis further. You're right that there
are loads of benefits to be gained from socialized content. The key is
putting the content in a place where it can be read and reacted to. AND,
you need a culture that encourages people to interact with content this way.

- Mary

Now translate that situation to the enterprise. For example, a proposal is in the works for a major client. The proposal is about to be finalized, when someone looks at it and notices there’s a missing piece of crucial data — one that the author had not thought to include. If the proposal is stored in a “social knowledge management” system (as opposed to a traditional CMS), the astute employee can leave a comment about the missing data. (Here’s where the “social” comes in.) Other people see this comment, support it, comment back, tag it, and the proposal is now stronger than the original.

Just as the metrics by themselves don’t tell the complete story, neither does most enterprise content. Through the wisdom of the community (or as you state, “faithful, honest interpreters”), socialized content provides context where the “good stuff happens.”

Mike Cassettari
http://blog.inmagic.com