Online community ROI

Online Community ROI Best Practices Survey / April 2007 Survey Coordination: San Francisco / Washington, DC Online Community Research Series Sponsors Leverage Software http://www.leveragesoftware.com Give your audience a voice and enable peer participation. Fortune, The New York Times, InfoWorld, Microsoft, Salesforce.com benefit from community. LiveWorld http://www.liveworld.com LiveWorld is the leading interactive marketing agency specializing in online customer communities and social networking for Fortune 1000 companies. Omidyar Network http://www.omidyar.net Omidyar Network is committed to creating opportunity for individuals to improve the quality of their lives. SolutionSet http://www.solutionset.com Strategic consulting with “custom” creative and technology solutions to achieve your organization’s Community goals. SolutionSet works with you. Online Community Research Network http://www.onlinecommunityresearch.com A research-based network of online community professionals working to understand key community building issues and best practices. Online Community Metrics 2 3/07 Executive Summary The Online Community ROI survey was initiated in April of 2007. Bill Johnston (Forum One Communications) sent approximately 150 survey invitations to online community professionals in our network of executives and practitioners. We received a 33% response rate (50 completed surveys). The mix of participants was made up of approximately 90% commercial groups, the balance representing non-profits. Commercial participants included large software companies, large community destination sites, niche community sites, platform providers and individual community practitioners. The respondents, most of whom we know professionally, represent a very senior and experienced group. The survey results provide a good deal of qualitative information. Section summaries and charted data have been provided for the majority of questions. Highlights from the full results include: • Current Organizational Attitudes Towards Online Community Investment (pages 7 & 9): Executive and senior management’s dispositions toward investment and return with online community activities; Community Budgets (page 10): online community spends; Headcount (page 11): size of online community staff; Dimensions of Community Value Reported (pages 12 & 13): quantifying the value of community for management and stakeholders; Strategies & Tactics (page 16): Strategies and tactics for reporting value back to management; Advice (page 19): Guidance for community managers concerning quantifying and reporting online community ROI. • • • • • In general, most organizations have faith that their online community initiatives are producing value, but are only able to report on relatively few dimensions of quantifiable value. A top priority for most participating organizations is establishing a model for quantifying and reporting ROI in the near term. Online Community Metrics 3 3/07 Survey Results Questions 1-3: Name, Organization, E-mail Address [responses suppressed] Question 4: What is your role with regard to online community for your organization? I am responsible for all online communities and related budgets I am responsible for some online communities and related budgets I manage some / all online communities, but do not manage budgets Other 28% 33% 21% 18% Online Community Metrics 4 3/07 Question 5: Approximate number of online communities in your organization? Online Community Metrics 5 3/07 Question 6: What kinds of online community activities does your organization engage in? Summary: Not surprisingly, discussion groups were the most popular online community activity, followed closely by blogs, support forums and social networks. Video sharing, wikis, and social bookmarking were deployed less frequently by survey respondents. “Other” Write in Responses: Career advice Chat Profiles Product reviews Events Fan fiction and fan art communities Is there a good term for reader feedback communities, in relation to professional journalists? We have one of them too. Market Research with online communities Online publications created by communities of interest Open source Open source communities enterprise developer communities Online Community Metrics 6 3/07 podcasts Ratings and Reviews (I classified recommendations as customer driven new product recommendations, not a review of existing product), Second Life (although it's not a generated community, we are participating in an existing community) Question 7: In general, what is your current organization's predisposition towards online community investment? Summary: Overall, the survey results indicate a fairly high tolerance for investing in online community activities without clear “hard numbers” ROI. As we will see later in the survey results, dimensions of value other than fiduciary are being accepted as “return” on community investment and involvement. However, the majority of respondents did say they were expected to communicate clear return in the future. Creating a clear ROI model for most organizations is clearly a priority, even those not under immediate pressure to communicate value. “Other” Write in Responses: A combination of two and three. We realize communities are the cost of doing Online Community Metrics 7 3/07 business, but we also continually seek ways to monetize community. As a Foundation our ROI for online communities is to support the work of our grantees. We have had limited success in the online community part of our grant making. Community is measured with customer support and technical support "acceleration" metrics over traditional community metrics, such as posting, lurker ratios, etc. Internal communities are considered corporate communications/interactions. External communities do not have a number associated but are expected to have a return on investment/participation although the return may not be monetary. Online communities are accepted as a "cost of doing business" but we definitely expect a return. Online communities are accepted as a 'cost of doing business', and a clear measure of return is not expected - true to an extent, but there is a strong desire to measure the return. Online communities are critical if you expect to maximize return on involvement for your organization. Online communities are seen as an essential part of our business and a key growth area for the future, and we are actively working on ways to increase our ROI. Our business is built on online communities Subset of overall content strategy, which is monetized via ad impressions Online Community Metrics 8 3/07 Question 8: What is your current attitude towards your organization's online community initiatives? Summary: A relatively small number of respondents were confident in their ability to tie community initiatives back to their corporate goals, and to clearly communicate ROI. The majority of respondents feel their initiatives are adding value, but can’t provide a complete ROI model. A small percentage of respondents feel their initiatives are disconnected from corporate goals, and they currently don’t report on value. This again speaks to the need for most organizations to create an ROI model, and one that includes more than direct financial value. “Other” Write in Responses: We build online communities. It’s our means of creating value Varies based on which community offering. Almost all of the above. Online Community Metrics 9 3/07 Question 9: What is your organization's annual online community budget for communities that you manage, not including headcount? This sum would include development, hosting, management and reporting. Summary: 75% of those that answered indicated a spend of at least $50k, and there were a significant number of that indicated spends of over $100k and over $500k annually, not including headcount. Obviously one would need to understand an organization’s spend in other areas to determine the proportion of overall annual budget, but these budget numbers do indicate significant investment in community by the participating organizations. Online Community Metrics 10 3/07 Question 10: How large is the team that works on the online communities that you manage? Summary: Surprisingly. the majority of the responses fell within the 1-10 person range, which could indicate that internal teams are understaffed, or that some community roles are being outsourced. Online Community Metrics 11 3/07 Question 11: Which of the following dimensions of value do you regularly report back to management and stakeholders? Summary: Not surprisingly, page views, unique visitors and new memberships topped the list of value points reported back, as they are the most accessible metrics. A fair number of organizations are reporting on engagement and loyalty, which speaks to longer-term relationships and the perceived value of customer advocacy via the community. It’s surprising that call avoidance and cost reduction ranked the lowest on the list, as these were points of value that have been articulated for years with regard to support discussion groups. Direct revenue per member was also surprisingly low. Online Community Metrics 12 3/07 Question 12: What other dimensions of value do you regularly report back to management? Summary: The write-in answers concerning other dimensions of value proved to be a rich source of data. Highlights are summarized below. Please note that answers that duplicated selections from the question 11 were not included. Demographic data: particularly member profile data is an under-analyzed and underreported source of value, especially when you consider the cost of obtaining this data by other means, like consumer research. Member vs. Visitor Activity: Monitoring and reporting on member activity vs. nonmember activity can help benchmark and track the relative value of community members. Comments / Ratings: User comments and content creation are valuable in several respects, including steering editorial direction, getting product feedback, and engaging the community ecosystem (offsite) in a broader conversation. Lead generation: Lead generation is an established metric for most marketing activities. This is an easy metric to compare cost to other lead generation activities, like trade shows and print advertising. Member Content Creation: Member-generated content value can usually be compared to and valued by the cost of internally produced content. Detailed Responses: % of replies posted by employees demographics podcast downloads top contributors or 'thought leaders' *we are somewhat limited to what we can report because of the platform we are currently using and how members are set up in our entitlement system. Anecdotal evidence of meeting mission goals (ie. impact). Anecdotal information regarding usage and engagement. Comments, articles, community creation activity Lead activity generated, BANT (ed. budget, authority, need, and timing) criteria, # of downloads Online Community Metrics 13 3/07 Lead generation Member activity Member engagement vs. site guest engagement -time spent -pages per visit -repeat visits Post numbers and Buzz outside our community Same dimensions by geographic/country, by community service type (blog, forum, etc.) Specific forum projects that solve editorial or marketing goals Subscription and/or ad revenue. Total Revenue/Margin on product links from our site (however, inability to capture all revenue as customer may not click on link but open new window, etc - no cookie tracking for product purchase) but not revenue by member Number of posts (but are an increase/decrease in posts good or bad? We want an increase in posts in regards to general discussions but a decrease in support posts) Unique posters Total number of posts Average posts per poster Ratio of posters to lurkers Time spent per visit of community members PVs / activity by community (to track topic popularity and interest) Online Community Metrics 14 3/07 Question 13: How often do you report back to management on ROI? Summary: The majority of respondents indicated they were reporting back to management monthly given that the majority of respondents also indicated they were reporting back on relatively few points of value. There were also a surprising number of respondents reporting back weekly, especially given the overhead of manually gathering and compiling ROI data. Other Responses: Clear ROI in dollar terms has not been established yet. Using lead gen and lead maturation Just starting out with reporting so not set yet Not ROI, but reports on traffic, search key words used Report metrics weekly, true ROI more like quarterly Online Community Metrics 15 3/07 Roi is not a useful measurement Question 14: From your experience, what are examples of effective strategies and tactics for making the case to management for investment in online community activities? Summary: Key strategies and tactics for making the case to management for community investment were: Report Direct Revenue: Report on direct revenue from Community Members. Establish Comparative Costs: Compare cost of the same result or activity in other media, like blog post comments vs. the cost of hosting a focus group. Communicate Cost Savings: Cost savings by hosting community, including support call deflection. Word of mouth vs. traditional marketing vehicles. Compare value of Member vs. Non-Member: Track and compare activity of subset of community member vs. non-member activity, including activity outside of community. Highlight the Direct Connection: Communicate the disintermediation opportunity for businesses, especially if your business is not directly selling to customers. Their online community creates a connection with the customer during and after the sale being handled by channel. The 24/7 Focus Group: A subset of your community will be giving real-time feedback on initiatives, products and company activities. “Human voice compliments the numbers.” Evangelist Creation and Mentoring: Communities attract your biggest fans and give them a place to connect with others, in the process creating new evangelists. Communicate the Cost of Not Being Engaged: Including increased marketing spend, and the threat of competitors engaging prospects and taking existing customers. “Value for all Departments” Communicate the broad value back to the business, including: • • • • “HR - Recruiting talent through fostering relationships with engineering interns and other prospects Customer Service - Tie in to the KnowledgeBase and help users help themselves and others Sales - pre/post sales, online EBC (executive briefing center), lead gen Online Community Metrics 16 3/07 • • Marcom/Marketing - brand awareness and loyalty Engineering - product development/enhancements” Detailed Responses: * * * * Call center deflection A direct two-way connection with customers for the company Customer-driven innovation Enhance product/brand loyalty 1) Traffic volume (visitors, visits, views) 2) ad impressions and sponsorships 3) differential visitor behavior over site as a whole -- length of stay, sense of engagement 4) positive feedback anecdotes A tightly knit combination of metrics and anecdotes. I firmly believe that stats mean little without human voices supporting them. Brand Loyalty Engagement with the Brand Leveraging Consumer insight Influencing purchasing behavior Dialogue Community as a competitive differentiator Growing the membership base to use for our business benefits including focus groups, referral programs, usability studies, brand excitement, loyalty, self-support channel Increased customer success through intelligent community content New product ideas and feedback Connection with post-sales customer (retention). Growth potential for participation by customer base (small percentage actually participating). Create Awareness, Nurturing Activities Customer Engagement, Product Education, Customer Satisfaction & Retention (positive brand association), Lowered Support Costs, etc. Have both quantitative and qualitative data to show off both numbers and community member interviews. Tie feedback to critical company initiatives. Compare community activity to traditional corporate activities: example blog readers/comments vs. product web page or direct mail responses. In our case community is one of the products we offer. The page views it drives are key to maintaining our leadership position. Showing how rapidly communities grow, the value of content they create, the level of satisfaction among participants and how evangelical they are about our site has helped. We do this with traditional metrics and regular surveying of the audience. We've also had success showing how insights from community members drive growth throughout the rest of our business. It's a 24/7 focus group. You just need the right folks listening. We've also constantly working on finding Online Community Metrics 17 3/07 new ways to generate revenue from community. We're getting better and better. Lead gen, revenue, community activity New page visits as a result of community applications -page consumption of members vs. site guests Online communities build volume of people with a positive predisposition toward the organization. All of these people are potential customers Page view to post ratios, loyalty, page views, advertising opportunities, links to specific areas where execs can "see for themselves" what customers are saying. Proven success stories with case studies of similar organizations. Identifying a clear between use of new tools and organizational goals. Reduced cost of product innovation, user testing and feedback. Retention, acceleration and activation over acquisition ROI, either through income produced or savings in other areas. Some of it is permitted to be grey. There is a clear cost to NOT having certain community elements. Similar to the eBay Germany study - take a sample of your active community members (however you define active) and look at their purchase history over a set time period (12-24 months) and compare revenue and margin spend against non-community members. Data around word of mouth marketing surpassing traditional media vehicles. Understand your business, broaden how communities are used - not just for 'support' anymore: HR - Recruiting talent through fostering relationships with engineering interns and other prospects Customer Service - Tie in to the KnowledgeBase and help users help themselves and others Sales - pre/post sales, online EBC (executive briefing center), lead gen Marcom/Marketing - brand awareness and loyalty Engineering - product development/enhancements (just a few examples of how communities are used for more than just support) You don't... start small, have small successes. Then grow your activities, and thus your successes ever larger over time. Don't talk to management if you don’t' have to. Online Community Metrics 18 3/07 Question 15: What advice would you have for a colleague that needs to quantify and report on the return on investment of their online communities? Summary: Common threads ran through responses to several of the survey questions, and these themes emerge again in this question about advice for a colleague. Of particular note were suggestions to: Establish an ROI Model: Establish an ROI model, ideally by understanding costs, revenue, indirect revenue and value unique to the community experience. If you can’t put together the “big picture”, establish some dollar value to the quantitative metrics you do have, like the value of page views on your community vs. the same page views on your main site (if applicable). Additional metric suggestions include: • Value of Member to organization “measured in number of posts, number of friends, or some other contribution made” • Value of member-to-member activity, based on overall content and satisfaction ratings. Evangelize Your Community Efforts: Provide metrics, clear examples of ways community helped business goals, engage executives with active links to examples of community at its best, circulate industry reports and best practices, use community tools (don't just talk about them). Know the Metrics you Need: Know what community metrics you want, based on business goals. Understand how you will use those metrics to communicate value. Ensure You Can Get Your Metrics: Ensure that you have the proper infrastructure to get the metrics you need. Community platforms and service providers rarely have the exact set of metrics “out of the box” your business requires. Cookie-ing all visitors to the community will help establish value of member vs. non-member. Additional Advice includes: • • Check membership data against key wins and big customer data (tie in) Cost reduction, especially with regard to research. Detailed Responses: 1) Understand your costs 2) Understand your direct revenue, if any 3) Understand how to value indirect revenue. For example, PR professionals routinely value the space a story picked up by a newspaper occupies by the ad value of the same space. Online Community Metrics 19 3/07 Similarly, one can use marketing metrics established for site visitors to value the traffic generated by communities. Similarly, use organic search traffic to community vs. cost to drive same traffic via paid search ads for another value comparison. Check regular membership data against new wins and customer data Community when tied to commerce and content is a triple threat strategy for engaging the consumer with your brand and creatively harnessing the power of the consumer advocate to serve as grass roots brand ambassadors in an ongoing personal dialogue. Cookie each site visitor to determine if guest or member and separately engagement metrics into the 2 buckets Cost savings is a big one if you typically do a lot of expensive research in other ways. Doing regular surveys (even as simple as survey monkey) to measure community members' satisfaction with your products, how often they recommend you to others, etc. can help get at the marketing value. Tying community insights to actions that have improved your products or business. Depends on the business model. If you don't have hard cash, perhaps numbers of interactions or satisfied people is the best starting approach. Grab on to what you are able to measure quantitatively and try to a fix some value to it in terms of money. This might be value per member (as measured in number of posts, number of friends, or some other contribution made). Also try to quantify the value that community members offer each other. If you can show your community efforts are generating value for the community members, this can be couched in terms of ROI even if it is not direct revenue. Make sure the service provider you use can provide robust metrics - our community is hosted outside of our IT environment which has many benefits but we lost a lot of tracking capabilities – click-through to purchase path, leakage, conversion, referring URL, next click, etc Know what information you want to know about your users but more importantly, know how you want to use it and how you can measure it (ie: wants to grow certain segments - can we identify what segments our community members fall into and show that the community is contributing to the companies efforts to grow/target that segment) Educate and get high level buy-in. Communities have a P&L impact through viral sales, increased brand awareness, building trust, strengthening relationships, etc but these are difficult to measure (especially against other brand building initiatives - TV ads, direct mail, investment in CE) Measure results over time, look for trends, bring the voice of your audience front and center, get your readers' attention by providing an executive summary at the beginning. Provide metrics, clear examples of ways community helped business goals, engage Online Community Metrics 20 3/07 executives with active links to examples of community at its best, circulate industry reports and best practices, use community tools (don't just talk about them). Read Matthew Lees "Measuring the Success of Online Communities - A CustomerCentric Approach to ROI". When looking to use a platform, understand what analytics are offered. There is a high chance you will either need to pay more from them to build in the reporting you require or will need to look at another reporting tool all together. It is one of those hidden "gotchas". Set a limited number of clear strategic objectives and measure against them Online Community Metrics 21 3/07 Final Comment We appreciate the survey participants taking the time to respond to a survey this detailed. Thank you! We continue to be encouraged by participation in these surveys and feedback to the reports. We plan to continue to regularly explore the areas of online community strategy, development, management, and marketing as part of this ongoing research series. If you have suggestions for future studies, please let us know. Best regards, Bill Johnston bjohnston@forumone.com About the Online Community Research Network The Online Community Research Network (OCRN) is a collaborative effort of online community professionals to better understand the principal challenges of building and managing online communities. Despite the enormous growth and importance of online communities, there is relatively little useful information available based on actual practice. The best source of information (by far) is other professionals. The Research Network coordinates the collection, analysis and dissemination of useful information among online community professionals. We offer free and premium services. For more information, please see: http://www.onlinecommunityresearch.com About Forum One Communications & Forum One Networks Forum One Communications http://www.ForumOne.com is a web strategy and communications firm which assists organizations in using new technologies to enhance communications and collaboration. Forum One Networks is a division of Forum One Communications. Forum One Networks focuses on the Online Community space and its practitioners through events, ongoing research, newsletters and blogs. Please see www.onlinecommunityreport.com for more information. For further information, or for sponsorship opportunities, see our website or drop Bill Johnston a note bjohnston@forumone.com Online Community Metrics 22 3/07