10 Reasons Why Satisfaction Surveys Don't Fail

10 Reasons Why Satisfaction Surveys Don’t Fail Author: Michael WhitehouseiPerceptions Survey-driven attitudinal analytics have exposed the hearts and minds of real website visitors, and the findings have started a whole new wave of conversations about the voice of the customer in corporate boardrooms. Gone are the days when customer relationship management meant only periodic telephone satisfaction surveys and bi-yearly focus groups. With online attitudinal reporting at their disposal, progressive decision makers are continuously attentive to the voice of their customers. This flourishing industry is not without its critics, however. There are influential voices in the field of market research that have lobbied for a more minimalist and simplistic approach. In his seminal work on customer loyalty and brand evangelism, The Ultimate Question, consultant Fred Reichheld argues that satisfaction surveys are poor barometers when it comes to the true salient issues with which companies are grappling. Further, he dismisses the notion that high customer satisfaction, within the frame of feedback surveys, has any causal link to corporate performance and growth. Reichheld’s work is not to be taken lightly. Indeed, he is incontrovertibly a pioneer in the movement to monetize customer loyalty. Yet, in many ways, he overstretches himself in this book and attempts to impugn the value of customer satisfaction surveys in ways that are, at best, misguided and, at worst, misinformed. All of his analysis is based on offline observations; indeed, despite being written within the last eighteen months, his book has a decidedly dated feel to it. Nowhere does he grapple with the nascent phenomenon of survey-based web analytics and the way in which this type of analysis can yield insights that his monochromatic approach never could. Yet given his stature and the reputability of the company he works for, Reichheld’s arguments have often been cited as reasons why online customer satisfaction surveys are not the way to go. Therefore, this paper will do more than just meet and discredit Reichheld’s objections—it will propose 10 reasons why satisfaction surveys absolutely do not fail. People like knowing that their opinions are valued, but they are always conscious of the demands being placed on their time. Reichheld contends that when “surveys grow to thirty or forty questions or more… response rates drop, and the sample size shrinks,” a pattern that, according to him, “introduces sample bias and makes scores volatile and unreliable.”1 Gratuitously long telephone surveys can certainly become tedious very quickly, especially if there are young children needing attention in the background or dinner boiling on the stovetop. Responding to more than 10 questions in a situation like that might be an extreme exercise in patience. Yet with an unobtrusive online survey, delivered after the site visit, there are surprisingly low leakage rates, even as the number of data points collected exceeds 30. Consider the following example, taken from data collection for a major electronics OEM. Structurally, our survey allows for drop-off only after the first 10. Online surveys are getting more and more unobtrusive Introduction 1Fred Reichheld, The Ultimate Question, 79.3 iPerceptions two sections, which, in this case, comprise 5 data points. As the graphic below demonstrates, even as the number of data points collected rose to 30 and then 35, the abandonment rate never exceeded 8% of the total sample during this five-month collection period. A robust enough sample base, devoid of bias, was maintained at all times. A similar pattern emerges across industry verticals, such that low samples sizes are most frequently attributable to low site traffic, rather than wholesale survey abandonment. While response rates are not dissimilar to those observed in offline market research, completion rates regularly trend above 90%. People like keeping their word and seeing a survey through to its end. By soliciting responses without obstructing the site visit or taxing the visitors’ time, it is that much easier for them to do so. Further, by employing a 3-month cookie, visitors who have already taken or refused the survey are not then solicited again. Experience has borne out the fact that indiscriminate surveying only yields respondents upset at being queried for the third, fourth, or fifth time. Minimizing response fatigue and making sure not to over sample—these are two small but significant ways in which the survey experience is made that much more visitor-friendly. Reichheld raises a good point when he argues that, unlike in political polling, where representative samples are essential to make statistically significant generalizations about the broad population, in ROI driven surveying, it is essential only to speak to the right people. What he fails to account for, however, is the fact that the “right people” in an online medium are not simply the largest spenders. Instead, the right people to talk to are what Malcolm Gladwell has called “connectors:” people who are willing and enthusiastic to propagate their opinions 9. Listening to the right people Respondents through first sections ( 5 data pts) Jan-07 2203 >>>>> Feb-07 1481 Mar-07 1478 Apr-07 1416 May-07 1580 Section 3 (30 data pts) Section 3 (35 data pts) Jan-07 Feb-07 Mar-07 Apr-07 May-07 6% 6% 7% 7% 8% 5% 6% 5% 6% 5% Abandonment Rate >>>>>iPerceptions and consensuses over the internet. These are the type of people who respond to online surveys; they are also the type of people who maintain blogs, make frequent posts on popular message boards, and in general exert an influence over online opinion that is probably far disproportionate to the actual amount of money they spend on any one company’s products. It is now possible for these opinions to spread very quickly through viral distribution networks. It is therefore imperative for companies to solicit and take direction from the feedback generated by these influencers. Reichheld charges that survey data, because of their complexity, do not empower employees with the necessary tools to take corrective action. Yet the aspersion that survey data are somehow recondite or overly obscure is very quickly being discredited. Rather, what employees have at their disposal now are ASP solutions that offer a portfolio of sophisticated online tools. These deliver nearly realtiim information in an intelligible format and they also allow for myriad drill-downs into more tactical and granular tidbits. Reporting should never be restricted to only top-line analysis. As Reichheld correctly opines, “customer feedback needs to relate specific problems to specific groups of customers.”1 We could not agree more, which is why iPerceptions works with its clients and partners to undertake tactical segmentation, based on demographic or psychographic criteria, that allows employees to look at response patterns and draw out problematic issues for high risk or high value clients. By integrating attitudinal data gleaned from surveys with behavioral data pulled from click stream analysis, companies can analyze not only where leakage is occurring in the purchase process, but also what aspects of the online experience are mitigating the desire to purchase and affecting conversion. Reichheld charges that many market research surveys are executed for purely promotional purposes. According to him, this process has helped destroy the credibility of satisfaction data. He cites the triviality of some of the categories in which J.D. Power awards are distributed, such as airlines that offer flights over 500 miles. It is our belief that there is nothing wrong with touting high grades in customer satisfaction. Having an industry-leading iPerceptions Satisfaction Index score is the result of a concerted effort on the part of many divisions within a company to heed the voice of the customer, and it is certainly deserving of recognition and approbation. Our industry verticals comprise many sector leaders. Therefore, if a company is outperforming its industry in terms of 8. With good data, employees are empowered to take action 7. Surveys establish objective industry standards 1Fred Reichheld, The Ultimate Question, 81.5 iPerceptions customer satisfaction, it is doing so relative to other well-established and reputable firms. Coming back to Reichheld’s original point, paging through our questionnaire, one realizes very quickly that it is not in any way a branding exercise. Participation in the survey is never incentivized either in the form of a cash honorarium or in the form of a product subvention. No promises of reward of any kind are made. We merely call on the visitors’ goodwill and desire to make the site better. Finally, if by drawing attention to an industry-leading iPSI score, a company experiences a short-term increase in sales, so be it. Only a longitudinal commitment to visitor satisfaction will drive the company’s online business forward in the long term (see below). In attempting to discredit customer satisfaction surveys, Reichheld parrots a commonly repeated fallacy: that because a company’s customer satisfaction scores are asynchronous with its earnings or stock performance, the satisfaction data can be considered dubious. Yes, it is reasonable to expect that a company that maintains high levels of customer satisfaction will outperform industry benchmarks over the long term. In the short term, however, there are far too many circumstantial factors in play. Companies should strive to grow satisfaction because the windfall is long-term, and not something that will necessarily be borne out by short-term stock or fiscal analysis. There are at least a dozen environmental factors that could impinge on the earnings/stock performance of a company whose satisfaction ratings are nonetheless high: labor costs, development costs, macroeconomic balance of trade (import/export fluctuations), monetary policy (currency issues), market climate (bear/bull), credit crunches (the subprrim mortgage fiasco leaps immediately to mind), political events, seasonality, corporate malfeasance, media coverage or reviews, large share dispositions, takeover bids. This is far from being an exhaustive list, and a financial analyst would probably chastise me for missing some very obvious ones. More pragmatically, a manufacturer with a deep commitment to customer satisfaction might still see its stock value eroded by a court order to pay out higher retirement benefits to its former employees. Alternately, an interest rate hike by the Federal Deposit Reserve would trigger both an increase in the cost of borrowing, with an attendant effect on corporate bottom lines, and a movement away from stocks into fixed income, which would drive share prices down. This would be the case across sectors, irrespective of customer satisfaction levels and CRM best practices. The point to be conveyed is that customer satisfaction should always be pursued as an end in itself, and progressive decision makers should never forget its intrinsic value. A satisfied customer base will always translate into long-term sustainability, even if their rosy brand outlook is not always mirrored in the last quarter’s dividends. 6. Customer satisfaction links to long-term sustainability6 iPerceptions To borrow from Tolstoy: every satisfied customer is alike, but every dissatisfied customer is unique in his or her own way. For this reason, companies need to have research methodologies at their disposal that will provide more than just directional insight. Brand, product and web presence stakeholders have consistently voiced their need for research that is targeted and permits for deep dives into crucial visitor segments. Concurrently, all brands need to be able to evaluate their customer satisfaction relative to competitors, which is why there is a marked need for stock categories of data, which allow for easy industry benchmarking. To that end, our automotive industry comparative site satisfaction report comprises six categories that are consistent across the brands for which data is collected: relationship to the company, purpose of visit, purchase horizon, age group, visit frequency, and overall site satisfaction. Many insights have been gleaned from these categories. For example, understanding website visitor age distribution relative to the industry can help a company identify whether or not it is attracting its targeted visitors. The key is to strike a balance between data collection that can easily be syndicated within an industry sector and data collection that is agile enough to allow for a high level of customizability. We firmly believe in deploying research questionnaires that contain both types. Thus, we avoid the “one-size-fits-all”1 research that Reichheld impugns, all the while making allowance for the fact that companies will always have a healthy appetite for industrywiid baselines. Web analytics, both behavioral and attitudinal, are still a nascent form of conducting market research. It is to be expected, therefore, that many satisfaction metrics have been, and still are, vying for primacy and ubiquity. The iPerceptions Satisfaction Index (iPSI), however, leverages several key factors that are just not present in other metrics. The iPSI’s 11-point scale allows for a healthy spectrum of opinion and encompasses a sufficient amount of response diversity. An analysis of the relationship between ordinal iPSI scores and verbal site experience ratings yields tangible proof of the iPSI’s credibility. As the graphic below demonstrates, site satisfaction for visitors to a leading software manufacturer’s website increased in direct correlation to their selfrepoorte level of online satisfaction. Further, the iPSI has a robust research pedigree. Its genesis was in the work of the late Dr. Max Garfinkle, who devised a cutting-edge perceptual framework that was later adapted to analyze the totality of the web experience. In essence, the model adumbrates the web experience as an amalgam of five distinct Dimensions: Navigation, Content, Interactivity, Motivation, and Adoption. . The iPSI is the emerging standard for knowing customer satisfaction 5. Targeted solutions bring immense value to companies 1Fred Reichheld, The Ultimate Question, 87.7 iPerceptions Each Dimension comprises multivariate identifiers of the online experience. Scores are assigned to each of these Attributes and all Attribute scores are rolled up and weighted equally within the iPSI. While we applaud Reichheld’s call for a “standard, intuitive system” to measure experiential data, no company or index can possess a monopoly on understanding visitor satisfaction. At times, multiple sources of business intelligence must be leveraged to understand the customer more fully. Yet the iPerceptions Satisfaction Index (iPSI) is increasingly becoming the gold standard for online voice of the customer (VoC) research. iPSI scores have now been outputted for over three iPSI Predicts Overall Satisfaction .0 0.0 . .8 3. 3.6 .0 . .8 5. 5.6 6.0 6. 6.8 7. 7.6 8.0 8. 9. 9.6 Fair 5. Good 6.8 8. Outstanding 9.3 . 3.6 Very good Very bad8 iPerceptions million actual website visitors. Most critically, the iPSI is, to our knowledge, the only metric conceived exclusively for online data collection. Thus, it does not suffer from the same portability issues that have to date plagued metrics based on offline collection methods. Many companies are content only to understand the purchase process, with the goal of identifying leakage and boosting conversion. While this is certainly a fruitful exercise on many levels, transactional and behavioral studies of this nature fail to account for the totality of the web experience. By restricting their scope of analysis to only the purchase process, decision makers who employ these tools exclusively blind themselves to many causal insights. Only attitudinal analytics can unearth the “why” behind the “what,” and not only in terms of transactional behavior or pathing. To his credit, Reichheld is dead on when he calls for surveying that includes every aspect of the online experience, including emotional and branding issues. Our research consistently demonstrates that relationship building is incipient well before the transactional stage of the site visit. This is why our perceptual framework, while taking into account elements that speak to commerce and conversion, pushes well beyond that narrow ambit to include all front end and back end aspects of the site experience. More specifically, our perceptual framework recaast the web experience as the convergence of two conceptual axes. One axis is temporal, and it weighs the creation of immediate versus long-term value; the other axis is based around orientation and measures front-end user desirability versus backeen site usefulness. Understanding the website in these terms allows us to evaluate how effectively it helps to construct and maintain crucial emotive links with the visitor. These are the links that either make or break the company-customer relationship; time and again, data sets show that these links begin to form well before the visitor clicks “buy now.” Research that is inimical to enthusiastic participation by volunteers is bound to fail. In many ways, online surveying is a natural evolution of the telephone surveying method, but without its greatest flaws. Our surveys never intrude on the website visit. Instead, a polite solicitation is presented on arrival to the site. If declined, visitors are thanked and sent on their way. If accepted, a second, minimized window is launched and visitors can then proceed without having any part of the site occluded by the surveying interface. The focus should be on the site, not the survey. Therefore, after respondents have completed their visit and turned their attention toward the survey, they are again reminded to not begin the survey until they have thoroughly completed their onsite task. Layered invitations are also coming into . Online surveys provide a safe forum for expression 3. Understanding the totality of the web experience is key9 iPerceptions use, and they undoubtedly point the way forward in terms of user-friendliness. There is another important factor to consider here. During telephone surveys, respondents are routinely asked questions (for example, related to age, income, or religion) that they may be unwilling to answer while connected with another party. Their discomfort can lead them to dissemble and furnish incorrect or inaccurate information. Online surveys are not plagued by these shortcomings. Instead, they provide a safe and secure forum for reasoned, deliberative, and, above all, honest feedback. This way, concern for tact and personal embarrassment never impede on the sincere expression of truth. Reichheld contends that employees manipulate satisfaction scores to gain accolades; but the opinions of real customers in real situations are objective and immune to manipulation. While the currency of customer satisfaction is certainly tremendously valuable, progressive organizations understand that gauging the pulse of online visitors is a subtle and iterative procedure, one that must necessarily draw upon the collective business 1. Respect for the customer improves communication within a company Immediate Usefulness Desirability Long Term Adoption interactivity content navigation motivation10 iPerceptions acumen of many divisions within a company. All of our clients are dealing with a vast wealth of experiential data, some of which do not fit easily into established paradigms about how shoppers, browsers, and researchers behave. This is precisely why it is so important that customer insights get a wide readership and substantial upward mobility within a company. We always stress to our clients the importance of engaging as many stakeholders as possible in a conversation about customers. A satisfied visitor base should be the concern of everyone. Working towards solutions with clients, we have found that a sincere and conscientious effort to bring the voice of the customer to the decision making table is healthy for an organization. Rather than inciting internecine conflict between divisions over who gets the highest loyalty scores, genuine consumer advocacy heightens interoperability and helps to build lateral bridges between the various components and departments of a company, all of which have an unequivocal stake in making sure visitors to the website leave feeling that their time was well spent. Often, this dispels faulty assumptions about the customer and engenders a desire to innovate. As Jim Sterne so aptly notes, “Bringing the voice of the customer to senior management is one of the most powerful tools for internal change.”1 I would add that it is also one of the most powerful tools for boosting inter-departmental collaboration and giving everyone a clear goal towards which to strive. The evolution and maturation of the entire field of web analytics have radically altered the way in which a company’s online presence is evaluated and have irrevocably placed the customer at the epicenter of all tactical and strategic website decision making. Going further, it is a universally recognized fact among industry leaders that survey-based attitudinal analytics have furnished an invaluable trove of business intelligence for website decision makers. At the same time, the expectations being placed on websites are maturing and becoming more exacting. As noted by a leading mind in the field, “ (now) there is a deep demand for the web channel to be held just as accountable as the other channels… companies are demanding that the web leadership justify investments being poured into the channel. This is an amazingly transformative experience for the channel as its leadership is looking in all places to prove results.” One number can never be powerful enough to provide all the insights that website decision makers need to meet these new challenges. At the same time, one number can never allow for the full spectrum of visitor commentary and opinion. A far more holistic picture is needed. This picture can only Conclusion 1 Jim Sterne, We Know What You Did Last Summer, 9. 2Avinash Kaushik, Web Analytics: An Hour a Day, 6.11 iPerceptions be drawn by survey-driven attitudinal analytics. The time is now for a different approach, one driven only by a devout respect for the voice of the customer. About iPerceptions iPerceptions is one of North America’s leading web-focused attitudinal analytics providers. Its webValidator Continuous Listening solution and its Proprietary Satisfaction Index (iPSI), turn thousands of data points into easy-to-understand strategic and tactical decision support for website marketers. iPerceptions’ clients include such well known brands as InterContinental Hotels, General Motors, Dell Computers, Hyundai, CompUSA, LG Electronics, Toshiba, Choice Hotels International, BMW and Hilton Hotels. iPerceptions has offices in New York, Toronto and Montreal. For further information please contact: Jonathan Levitt Vice President Marketing 877.796.3600 ext 50 jlevitt@iperceptions.com www.iperceptions.com Contact Us USA 575 Madison Avenue Suite 1006 New York, NY 100 -511 USA Tel: 877-796-3600 Fax: 866-8 -600 Canada 999 St-Catherine St. West Suite 500 Montreal, Quebec H3Z 1T3 Canada Tel: (51 ) 88-3600 Fax: (51 ) 8 -600 info@iperceptions.com www.iperceptions.comKnow SatisfactionTM