Measuring E2.0 Success & Business Value – Metrics & Analysis

Here are my notes from the third session of the Enterprise 2.o Black Belt Workshop:  Measuring Success and Business Value – Metrics and Analysis

Speakers:

  • Ted Hopton, Wiki Community Manager, United Business Media
  • Donna Cuomo, Chief Information Architect, the MITRE Corporation

Background:

[These are my quick notes, complete with  (what I hope is no more than) the occasional typo and grammatical error.  Please excuse those. Thanks!

From time to time, I’ll insert my own editorial comments – exercising the prerogatives of the blogger.  I’ll show those in brackets. ]

Notes:

Ted Hopton:

  • His company organized the Enterprise 2.0 conference
  • They use Jive software for their enterprise 2.0 platform
  • Focus first on participation
    • Use the analytics module of your enterprise 2.0 tool to see who is visiting the site, where the activity is taking place, who is creating and viewing content, etc.
    • Analyze active members by level of activity
  • Problem: Make sure the metrics tie back to your project goals
  • Use qualitative measures to improve your understanding
    • Use a survey – ask how and how often people use the community
    • List possible positive outcomes and ask users which of these outcomes they have experienced
    • Ask why they don’t use the tool more
      • Use blunt, negative statements
      • Encourage them to tell you exactly how they feel
    • Use this information to benchmark (and draw out the venom – otherwise it festers)
    • Net Promoter Score – give users a scale of 1-10 and ask them how likely they are to promote your work.  Scores above 6 indicate that they will promote rather than detract your promoter. Subtract your scores below 6  from your scores of 6 and above. This yields your “Net Promoter Score.”  Obviously, the higher the better.
    • Track your success stories and share them
  • Lessons Learned
    • While it’s good to have consistent metrics, be aware that metrics evolve and your methods should evolve too
    • Beware of benchmarks (e.g., the 90-9-1 standard of participation). Make sure the benchmark you are using really applies to E2.0 projects.

Donna Cuomo:

  • The Mitre Company runs four differently federally funded programs (including for the Dept. of Homeland Security and the Dept. of Defense)
  • Use Case 1:  Improve MITRE’s Research Program Selection Process
    • They used Spiggot to be their “innovation management tool”
    • They wanted to codify their research competition process
    • They wanted to stop people further down the food chain from weeding out ideas too early
    • They wanted to encourage broader participation (from a review perspective)
    • They created an “Idea Market”based on a SharePoint wiki
    • Their first-year metrics indicated broad participation
    • They were able to create widespread transparency
    • They used surveys to compare the new tools (and user satisfaction) against the old tools/methodologies
  • Use Case 2: Social Bookmarking
    • Hypothesized that social bookmarking would inmprove resource sharing, leveraging the research of others across teams and the corporation
    • They also thought the tagging would help identify experts within the organization
    • They used a tool similar to Delicious
    • Bookmarks helped create a lightweight newsletter (this was an unexpected benefit)
    • You don’t need many participants in order to provide real value to the entire organization
  • Use Case 3: Babson SNA Study
    • They identified super users of their internal social networks and social media (brokers) and then interviewed their colleagues
    • They discovered that these super users tended to be innovative and provide huge value to their networks
    • Frequency of interactions was not as important as the number of unique connections each broker had (indicative of their ability to have an impact on a wider range of people).

Exercise:

  • What are the most important things you are NOW measuring?
    • Number of communities
    • Number of community members
    • Percentage of contributors versus consumers
    • Usage across geographies, business units, etc.
    • Number of visits
    • Dwell time (how long is each visit)
    • Number of concurrent users at any one time
    • Number of people editing (indicates collaboration)
    • Number (and identity of ) lurkers
    • Measuring conversion of lurkers to active participants
    • Participation in community activities (who is sharing, who is editing, who is tagging, etc.)
    • Utilization of the various social tools
    • Success stories
  • What are the most important things you should be measuring?
    • Abandonment rate – when do visits/activity drop off
    • Tracking against business goals
    • Net Promoter Score
    • Day/time of highest activity
    • first and last page viewed
    • business improvement metrics
      • = correlation of usage to operating metrics
      • = correlation of usage to improved business process
    • Measuring cross-fertilization (the extent to which people choose to go outside their community for information)
    • Number of new ideas/ rate of innovation
    • What’s the reduction in other forms of overhead activities (e.g., now that the subject matter expert is posting answers on a social platform, what is the resulting decline in repetitive e-mail requests?)
    • Percentage of profile completion
    • Rating content
    • Ability to determine a dollar value to participation
    • Where was the content reused, how was it reused, and what were the results of the reuse (e.g., cost savings, process improvement, etc.)
  • Presentations:  www.e2conf.com/boston/2010/presentations/workshop
    • User name: Workshop
    • Password: Boston
  • Presentations also on Slideshare: http://slideshare.net/20adoption
Share

2 thoughts on “Measuring E2.0 Success & Business Value – Metrics & Analysis

  • June 14, 2010 at 7:14 pm
    Permalink

    Great notes! Thanks for posting 🙂

    Minor correction: NPS scale runs from 0-10, rather than 1-10 (just so you'll be consistent with the way it's used elsewhere and the research behind it).

  • June 15, 2010 at 3:23 am
    Permalink

    Thanks very much for the clarification, Ted. And thanks for a great
    session today.

    – Mary

    VMaryAbraham
    AboveandBeyondKM.com

Comments are closed.