Made to measure


I’ve heard lots of talk recently around metrics and ROI on social business tools. Many of the Intranet, IT, HR and Marketing people I speak to are looking for metric packages that will provide some of the traditional measurements around online sites.

In terms of ‘measurement’ here are some of the criteria I use to generally benchmark knowledge and collaboration communities and programmes. There are some basic metrics to measure participation and activity within the communities.

                     Total number of users

                     Number of new users per week

                     Number of new posts, threads (plus response), ideas, blog and other content

                     What are members doing in the community?

                     What are the popular trends in posts? Blogs? Forums?

                     What resources are being used?

However, these basic metrics won’t provide a true understanding of the value around the collaboration that a platform could provide to individuals and the community as a whole. For the measurement of value around collaborative communities I would generally look at analytics that articulate:




                     Social Knowledge – this can be defined in many ways such as assets being shared around a community (and beyond) and related practices emerge.

                     Relationship development – the ability to create new relationships and networks that previously didn’t exists

Some of the questions I ask to evaluate these items would include:

                     % of members / users which make a contribution

                     Members active within the past 30 days

                     Contributions per active member and the value of these contributions related to the purpose of the community

                     Content popularity

                     Number of relationships created by individuals – look at followers and participation in threads

                     Discovery of communities – have members joined communities outside their ‘physical’ or existing network

The various social business tool reports (i.e. Community Manager Reports in Jive) will not provide this type of information and much of it will be antidotal evidence. Social analytics are poor within most social tools (it will be a major revenue stream for a vendor that can start to provide some of the softer metrics that articulate quality and not just quantity).

Over the years of working within communities I’ve compiled a list from various sources that are useful in articulating some measure of value or return on the community. Top 30 are:


1.             What % of newcomers remain members for more than a month.

2.             Speed of replies to discussions. How quickly are discussions receiving a reply? The faster the responses, the higher the level of social presence within the community and the greater the level of participation.

3.             The % of newcomers which initiate a discussion. This highlights whether newcomers may be unmotivated or intimidated to start discussions.

4.             Language and tone of voice. What language do members adopt when they address each other? Is it formal and polite? Is there friendly banter? Is there a sense of familiarity? This will let you know what stage the community is in.

5.             Do members’ interactions have continuity and depth? (Are members engaged in productive, on-going, interactions?)

6.             What collaborative activities are emerging?

7.             What documents, tools, resources, or other artefacts are created and utilized. (How are these useful to the members?)

8.             Is the community providing value for its sponsors

9.             Is participants’ involvement in the community affecting their professional practices and learning.

10.          What are the on-going practices and processes that contribute to the “life” of the community and keep members engaged?

11.          How is knowledge being shared within the community? Beyond the community?

12.          Are leaders or roles emerging in the community? In what ways? How are they being cultivated?

13.          How are members being supported in the community?

14.          How are members contributing? Posting? Replying? (When? How often?)

15.          What are the prevalent patterns of interactions?

16.          What is the proportion of new topics that get 5+ replies? The percentage of new blogs that get 5+ comments? The percentage of (video, audio, lesson plan, etc.) uploads that get 50+ downloads or 5+ comments?

17.          What proportions of new topics or new blog posts are un-responded to or uncommented on (an important measure of the responsiveness of the community, which in turn affects key factors like trust)?

18.          What is the average new (topics, replies, blog posts) created per member?

19.          What are the emerging benefits of the community for members?

20.          What is the average number of “followers” that community members list or have collected in/on their member profile pages?

21.          What are the proportions of topics or replies that specifically relate to your practice?

22.          The proportion of replies where links to potentially helpful resources or other referrals are provided

23.          The proportion of replies to a post in which helpful or constructive advice is directly provided

24.          The proportion of replies that build on previous posts (as opposed to just responding to the original poster)

25.          The proportion of replies that contain offers of collaboration or introductions to potential collaborators

26.          The proportion of replies that contain creative, novel, or innovative ideas

27.          The proportions of replies that summarize, distil, or synthesize prior posts/replies

28.          The proportion of posts in which community members show or express vulnerability, such as a lack of domain knowledge

29.          The proportion of posts in which community members share personal stories

30.          The proportion of posts in which community members are (emotionally) supportive or helpful to other members



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s