Here are my notes from the third session of the Enterprise 2.o Black Belt Workshop:  Measuring Success and Business Value – Metrics and Analysis

Speakers:

• Ted Hopton, Wiki Community Manager, United Business Media
• Donna Cuomo, Chief Information Architect, the MITRE Corporation

Background:

[These are my quick notes, complete with  (what I hope is no more than) the occasional typo and grammatical error.  Please excuse those. Thanks!

From time to time, I'll insert my own editorial comments - exercising the prerogatives of the blogger.  I'll show those in brackets. ]

Notes:

Ted Hopton:

• His company organized the Enterprise 2.0 conference
• They use Jive software for their enterprise 2.0 platform
• Focus first on participation
  • Use the analytics module of your enterprise 2.0 tool to see who is visiting the site, where the activity is taking place, who is creating and viewing content, etc.
  • Analyze active members by level of activity
• Problem: Make sure the metrics tie back to your project goals
• Use qualitative measures to improve your understanding
  • Use a survey – ask how and how often people use the community
  • List possible positive outcomes and ask users which of these outcomes they have experienced
  • Ask why they don’t use the tool more
    • Use blunt, negative statements
    • Encourage them to tell you exactly how they feel
  • Use this information to benchmark (and draw out the venom – otherwise it festers)
  • Net Promoter Score – give users a scale of 1-10 and ask them how likely they are to promote your work.  Scores above 6 indicate that they will promote rather than detract your promoter. Subtract your scores below 6  from your scores of 6 and above. This yields your “Net Promoter Score.”  Obviously, the higher the better.
  • Track your success stories and share them
• Lessons Learned
  • While it’s good to have consistent metrics, be aware that metrics evolve and your methods should evolve too
  • Beware of benchmarks (e.g., the 90-9-1 standard of participation). Make sure the benchmark you are using really applies to E2.0 projects.

Donna Cuomo:

• The Mitre Company runs four differently federally funded programs (including for the Dept. of Homeland Security and the Dept. of Defense)
• Use Case 1:  Improve MITRE’s Research Program Selection Process
  • They used Spiggot to be their “innovation management tool”
  • They wanted to codify their research competition process
  • They wanted to stop people further down the food chain from weeding out ideas too early
  • They wanted to encourage broader participation (from a review perspective)
  • They created an “Idea Market”based on a SharePoint wiki
  • Their first-year metrics indicated broad participation
  • They were able to create widespread transparency
  • They used surveys to compare the new tools (and user satisfaction) against the old tools/methodologies
• Use Case 2: Social Bookmarking
  • Hypothesized that social bookmarking would inmprove resource sharing, leveraging the research of others across teams and the corporation
  • They also thought the tagging would help identify experts within the organization
  • They used a tool similar to Delicious
  • Bookmarks helped create a lightweight newsletter (this was an unexpected benefit)
  • You don’t need many participants in order to provide real value to the entire organization
• Use Case 3: Babson SNA Study
  • They identified super users of their internal social networks and social media (brokers) and then interviewed their colleagues
  • They discovered that these super users tended to be innovative and provide huge value to their networks
  • Frequency of interactions was not as important as the number of unique connections each broker had (indicative of their ability to have an impact on a wider range of people).

Exercise:

• What are the most important things you are NOW measuring?
  • Number of communities
  • Number of community members
  • Percentage of contributors versus consumers
  • Usage across geographies, business units, etc.
  • Number of visits
  • Dwell time (how long is each visit)
  • Number of concurrent users at any one time
  • Number of people editing (indicates collaboration)
  • Number (and identity of ) lurkers
  • Measuring conversion of lurkers to active participants
  • Participation in community activities (who is sharing, who is editing, who is tagging, etc.)
  • Utilization of the various social tools
  • Success stories
• What are the most important things you should be measuring?
  • Abandonment rate – when do visits/activity drop off
  • Tracking against business goals
  • Net Promoter Score
  • Day/time of highest activity
  • first and last page viewed
  • business improvement metrics
    • = correlation of usage to operating metrics
    • = correlation of usage to improved business process
  • Measuring cross-fertilization (the extent to which people choose to go outside their community for information)
  • Number of new ideas/ rate of innovation
  • What’s the reduction in other forms of overhead activities (e.g., now that the subject matter expert is posting answers on a social platform, what is the resulting decline in repetitive e-mail requests?)
  • Percentage of profile completion
  • Rating content
  • Ability to determine a dollar value to participation
  • Where was the content reused, how was it reused, and what were the results of the reuse (e.g., cost savings, process improvement, etc.)

• Presentations:  www.e2conf.com/boston/2010/presentations/workshop
  • User name: Workshop
  • Password: Boston
• Presentations also on Slideshare: http://slideshare.net/20adoption