How Do You Measure That?

Rawn Shah (Business Transformation Consultant, IBM) and Hardik Dave (Senior Business Analyst, IBM) talk about how to create a strategy to collect the metrics that are best for your social business implementation.

[These are my notes from the Enterprise 2.0 Conference 2011 in Boston.  Since I’m publishing them as soon as possible after the end of a session, they may contain the occasional typographical or grammatical error.  Please excuse those. To the extent I’ve made any editorial comments, I’ve shown those in brackets.]

NOTES:

• Why do you need metrics?Metrics help demonstrate value — particularly value to customers. Metrics also can help demonstrate operational excellence. Finally, metrics can shed light on what makes the people and culture of your organization special.
• Who will use the Metrics?The intended users and uses help identify the necessary data points that you should be collecting. For example, Marketing and HR have very different business goals and may need completely metrics. That said, they both may want similar metrics regarding how their content is consumed.
• What Types of Metrics Do You Need? Metrics reflect qualitative, quantitive, attitudinal, or behavioral characteristics. All are related to demographics. The business value comes from providing the right metric to the right person. The metrics chosen often depend on the role of the recipient. For example, one person may be interested in content consumption rates, another interested in brand management or sentiment. Both may be tracking activity, but for different reasons.
• How do you collect metrics?Collection methods depend on the type of metric. For example, activity or traffic data may be collected passively by your social software. By contrast, attitudinal data would be collected by surveys, anecdotes, day-in-the-life studies, interviews and focus groups. However, the type of survey you use to assess group vitality (attitudinal) will be quite different from the survey used to assess interactions.
• When do you collect Metrics?The timing is a function of the maturity of your social business. Start with basic satisfaction, social activity and content activity data. As the program matures, add social graph and group vitality data. Further along, you can measure brand reputation and individual reputation.
• Surveys.As long as you ask the right questions, surveys can help identify gaps. Later, you can use surveys get feedback from early adopters to obtain guidance on technical issues. For example, the IT department at IBM has a workplace effectiveness survey with respect to social business. They asked which social activities were most important to employees and how satisfied they were with the available tools. The resulting data helped the IT department understand which social business tools were most important and which needed to be improved first.
• Interviews and Focus Groups.Interviews and focus groups are an effective way to collect anecdotes. As you are doing this, always be on the lookout for success stories, as well as early warning signs of potential problems.
• Day in the Life Visioning.This a great tool for understanding business process in real life. By random sampling across people in similar jobs, you can reveal patterns that are firm wide.
• Activity Logging.How and how often are people using the social tool? How are they interacting with the content, with each other?
• Content and Sentiment Analysis.This data helps provide aggregate understanding of brand, reputation and the content. What are people talking about?
• It’s not the size (of the data) that matters. It’s what you do with it.
• Making a Social Analytics Strategy Map. Start by understanding what business issue you need to address and then consider what data will help illuminate the problem and solution. Once you have the data, you can starting making some interesting connections. For example, how does group vitality affect content sharing? What is the impact of sentiment on collaboration levels? Make sure you understand properly the direct impacts, the correlations. At IBM, they have done a study of high performing sales people and the extent to which they are also high performing collaborators. A study like this requires data across multiple years. (Rawn released this data on SlideShare in 2008.)
• What’s the Dollar Impact?Don’t always assume that the data will lead directly to a dollar impact. However, it should help the business leaders understand their business better and make better decisions.