10 REASONS WHY SATISFACTION SURVEYS DONT FAIL

Document Sample
10 REASONS WHY SATISFACTION SURVEYS DONT FAIL Powered By Docstoc
					10 Reasons
Why Satisfaction Surveys
Don’t Fail




Author: Michael Whitehouse
Introduction
Survey-driven attitudinal analytics have exposed        can yield insights that his monochromatic approach
the hearts and minds of real website visitors, and      never could.
the findings have started a whole new wave of
conversations about the voice of the customer           Yet given his stature and the reputability of the
in corporate boardrooms. Gone are the days              company he works for, Reichheld’s arguments have
when customer relationship management meant             often been cited as reasons why online customer
only periodic telephone satisfaction surveys and        satisfaction surveys are not the way to go. Therefore,
bi-yearly focus groups. With online attitudinal         this paper will do more than just meet and discredit
reporting at their disposal, progressive decision       Reichheld’s objections—it will propose 10 reasons
makers are continuously attentive to the voice of       why satisfaction surveys absolutely do not fail.
their customers.

This flourishing industry is not without its critics,      10. Online surveys are getting more and
however. There are influential voices in the field         more unobtrusive
of market research that have lobbied for a more
minimalist and simplistic approach. In his seminal
work on customer loyalty and brand evangelism,
The Ultimate Question, consultant Fred Reichheld        People like knowing that their opinions are valued,
argues that satisfaction surveys are poor barometers    but they are always conscious of the demands being
when it comes to the true salient issues with which     placed on their time. Reichheld contends that when
companies are grappling. Further, he dismisses the      “surveys grow to thirty or forty questions or more…
notion that high customer satisfaction, within the      response rates drop, and the sample size shrinks,”
frame of feedback surveys, has any causal link to       a pattern that, according to him, “introduces sample
corporate performance and growth.                       bias and makes scores volatile and unreliable.”1
                                                        Gratuitously long telephone surveys can certainly
Reichheld’s work is not to be taken lightly. Indeed,    become tedious very quickly, especially if there are
he is incontrovertibly a pioneer in the movement to     young children needing attention in the background
monetize customer loyalty. Yet, in many ways, he        or dinner boiling on the stovetop. Responding to
overstretches himself in this book and attempts to      more than 10 questions in a situation like that might
impugn the value of customer satisfaction surveys       be an extreme exercise in patience. Yet with an
in ways that are, at best, misguided and, at worst,     unobtrusive online survey, delivered after the site
misinformed. All of his analysis is based on offline    visit, there are surprisingly low leakage rates, even
observations; indeed, despite being written within      as the number of data points collected exceeds 30.
the last eighteen months, his book has a decidedly
dated feel to it. Nowhere does he grapple with          Consider the following example, taken from data
the nascent phenomenon of survey-based web              collection for a major electronics OEM. Structurally,
analytics and the way in which this type of analysis    our survey allows for drop-off only after the first



Fred Reichheld, The Ultimate Question, 79.
1




                                                                                             iPerceptions
two sections, which, in this case, comprise 5 data       A similar pattern emerges across industry verticals,
points. As the graphic below demonstrates, even as        such that low samples sizes are most frequently
the number of data points collected rose to 30 and        attributable to low site traffic, rather than wholesale
then 35, the abandonment rate never exceeded 8%           survey abandonment. While response rates are
of the total sample during this five-month collection     not dissimilar to those observed in offline market
period. A robust enough sample base, devoid of            research, completion rates regularly trend above
bias, was maintained at all times.                        90%. People like keeping their word and seeing a
                                                          survey through to its end. By soliciting responses
                                                          without obstructing the site visit or taxing the visitors’
                                                          time, it is that much easier for them to do so. Further,
     Respondents through first  sections                 by employing a 3-month cookie, visitors who have
     (5 data pts)
                                                          already taken or refused the survey are not then
                                                          solicited again. Experience has borne out the fact
     Jan-07          >   2203                             that indiscriminate surveying only yields respondents
                                                          upset at being queried for the third, fourth, or fifth
     Feb-07          >   1481
                                                          time. Minimizing response fatigue and making
     Mar-07          >   1478                             sure not to over sample—these are two small but
                                                          significant ways in which the survey experience is
     Apr-07          >   1416                             made that much more visitor-friendly.

     May-07          >   1580

                                                              9. Listening to the right people


     Abandonment          Section 3       Section 3
     Rate                 (30 data pts)   (35 data pts)
                                                          Reichheld raises a good point when he argues
                                                          that, unlike in political polling, where representative
     Jan-07          >    5%              6%
                                                          samples are essential to make statistically significant
     Feb-07          >    6%              6%              generalizations about the broad population, in ROI
                                                          driven surveying, it is essential only to speak to the
     Mar-07          >    5%              7%              right people. What he fails to account for, however,
                                                          is the fact that the “right people” in an online medium
     Apr-07          >    6%              7%              are not simply the largest spenders.

     May-07          >    5%              8%              Instead, the right people to talk to are what Malcolm
                                                          Gladwell has called “connectors:” people who are
                                                          willing and enthusiastic to propagate their opinions




  iPerceptions                                                                                                 3
and consensuses over the internet. These are the         partners to undertake tactical segmentation, based
type of people who respond to online surveys;            on demographic or psychographic criteria, that
they are also the type of people who maintain            allows employees to look at response patterns and
blogs, make frequent posts on popular message            draw out problematic issues for high risk or high
boards, and in general exert an influence over           value clients.
online opinion that is probably far disproportionate     By integrating attitudinal data gleaned from surveys
to the actual amount of money they spend on any          with behavioral data pulled from click stream
one company’s products. It is now possible for           analysis, companies can analyze not only where
these opinions to spread very quickly through viral      leakage is occurring in the purchase process, but
distribution networks. It is therefore imperative for    also what aspects of the online experience are
companies to solicit and take direction from the         mitigating the desire to purchase and affecting
feedback generated by these influencers.                 conversion.




        8. With good data, employees are                    7. Surveys establish objective industry
        empowered to take action                            standards




Reichheld charges that survey data, because of           Reichheld charges that many market research
their complexity, do not empower employees with          surveys are executed for purely promotional
the necessary tools to take corrective action. Yet the   purposes. According to him, this process has
aspersion that survey data are somehow recondite         helped destroy the credibility of satisfaction data. He
or overly obscure is very quickly being discredited.     cites the triviality of some of the categories in which
Rather, what employees have at their disposal            J.D. Power awards are distributed, such as airlines
now are ASP solutions that offer a portfolio of          that offer flights over 500 miles.
sophisticated online tools. These deliver nearly real-
time information in an intelligible format and they      It is our belief that there is nothing wrong with touting
also allow for myriad drill-downs into more tactical     high grades in customer satisfaction. Having an
and granular tidbits.                                    industry-leading iPerceptions Satisfaction Index
                                                         score is the result of a concerted effort on the part
Reporting should never be restricted to only top-line    of many divisions within a company to heed the
analysis. As Reichheld correctly opines, “customer       voice of the customer, and it is certainly deserving of
feedback needs to relate specific problems to specific   recognition and approbation. Our industry verticals
groups of customers.”1 We could not agree more,          comprise many sector leaders. Therefore, if a
which is why iPerceptions works with its clients and     company is outperforming its industry in terms of



Fred Reichheld, The Ultimate Question, 81.
1




                                                                                                iPerceptions
customer satisfaction, it is doing so relative to other    not something that will necessarily be borne out by
well-established and reputable firms.                      short-term stock or fiscal analysis.

Coming back to Reichheld’s original point, paging          There are at least a dozen environmental factors that
through our questionnaire, one realizes very               could impinge on the earnings/stock performance
quickly that it is not in any way a branding exercise.     of a company whose satisfaction ratings are
Participation in the survey is never incentivized either   nonetheless high: labor costs, development costs,
in the form of a cash honorarium or in the form of         macroeconomic balance of trade (import/export
a product subvention. No promises of reward of             fluctuations), monetary policy (currency issues),
any kind are made. We merely call on the visitors’         market climate (bear/bull), credit crunches (the sub-
goodwill and desire to make the site better.               prime mortgage fiasco leaps immediately to mind),
                                                           political events, seasonality, corporate malfeasance,
Finally, if by drawing attention to an industry-leading    media coverage or reviews, large share dispositions,
iPSI score, a company experiences a short-term             takeover bids. This is far from being an exhaustive
increase in sales, so be it. Only a longitudinal           list, and a financial analyst would probably
commitment to visitor satisfaction will drive the          chastise me for missing some very obvious ones.
company’s online business forward in the long term
(see below).                                               More pragmatically, a manufacturer with a deep
                                                           commitment to customer satisfaction might still see
                                                           its stock value eroded by a court order to pay out
   6. Customer satisfaction links to long-term             higher retirement benefits to its former employees.
   sustainability                                          Alternately, an interest rate hike by the Federal
                                                           Deposit Reserve would trigger both an increase in
                                                           the cost of borrowing, with an attendant effect on
                                                           corporate bottom lines, and a movement away from
In attempting to discredit customer satisfaction
                                                           stocks into fixed income, which would drive share
surveys, Reichheld parrots a commonly repeated
                                                           prices down. This would be the case across sectors,
fallacy: that because a company’s customer
                                                           irrespective of customer satisfaction levels and CRM
satisfaction scores are asynchronous with its
                                                           best practices.
earnings or stock performance, the satisfaction data
can be considered dubious. Yes, it is reasonable           The point to be conveyed is that customer satisfaction
to expect that a company that maintains high levels        should always be pursued as an end in itself, and
of customer satisfaction will outperform industry          progressive decision makers should never forget
benchmarks over the long term. In the short term,          its intrinsic value. A satisfied customer base will
however, there are far too many circumstantial             always translate into long-term sustainability, even
factors in play. Companies should strive to grow           if their rosy brand outlook is not always mirrored in
satisfaction because the windfall is long-term, and        the last quarter’s dividends.




  iPerceptions                                                                                               5
                                                          contain both types. Thus, we avoid the “one-size-
        5. Targeted solutions bring immense value         fits-all”1 research that Reichheld impugns, all the
        to companies                                      while making allowance for the fact that companies
                                                          will always have a healthy appetite for industry-
                                                          wide baselines.
To borrow from Tolstoy: every satisfied customer is
alike, but every dissatisfied customer is unique in his
or her own way. For this reason, companies need              . The iPSI is the emerging standard for
to have research methodologies at their disposal             knowing customer satisfaction
that will provide more than just directional insight.
Brand, product and web presence stakeholders
have consistently voiced their need for research that
                                                          Web analytics, both behavioral and attitudinal, are
is targeted and permits for deep dives into crucial       still a nascent form of conducting market research.
visitor segments.                                         It is to be expected, therefore, that many satisfaction
                                                          metrics have been, and still are, vying for primacy
Concurrently, all brands need to be able to evaluate
                                                          and ubiquity. The iPerceptions Satisfaction Index
their customer satisfaction relative to competitors,
                                                          (iPSI), however, leverages several key factors that
which is why there is a marked need for stock
                                                          are just not present in other metrics. The iPSI’s 11-
categories of data, which allow for easy industry
                                                          point scale allows for a healthy spectrum of opinion
benchmarking. To that end, our automotive industry
                                                          and encompasses a sufficient amount of response
comparative site satisfaction report comprises six
                                                          diversity. An analysis of the relationship between
categories that are consistent across the brands for
                                                          ordinal iPSI scores and verbal site experience
which data is collected: relationship to the company,
                                                          ratings yields tangible proof of the iPSI’s credibility.
purpose of visit, purchase horizon, age group,
                                                          As the graphic below demonstrates, site satisfaction
visit frequency, and overall site satisfaction. Many
                                                          for visitors to a leading software manufacturer’s
insights have been gleaned from these categories.         website increased in direct correlation to their self-
For example, understanding website visitor age            reported level of online satisfaction.
distribution relative to the industry can help a
company identify whether or not it is attracting its      Further, the iPSI has a robust research pedigree.
targeted visitors.                                        Its genesis was in the work of the late Dr. Max
                                                          Garfinkle, who devised a cutting-edge perceptual
The key is to strike a balance between data collection    framework that was later adapted to analyze
that can easily be syndicated within an industry          the totality of the web experience. In essence, the
sector and data collection that is agile enough to        model adumbrates the web experience as an
allow for a high level of customizability. We firmly      amalgam of five distinct Dimensions: Navigation,
believe in deploying research questionnaires that         Content, Interactivity, Motivation, and Adoption.



Fred Reichheld, The Ultimate Question, 87.
1




    6                                                                                            iPerceptions
                                     iPSI Predicts Overall Satisfaction

                 9.6
                 9.                                                                                9.3
                 8.
                 8.0
                                                                                       8.
                 7.6
                 7.
                 6.8
                                                                         6.8
                 6.
                 6.0
                 5.6
                 5.
                                                          5.
                 .8
                 .
                 .0
                 3.6
                                             3.6
                 3.
                 .8
                 .
                 .0           .
                 0.0
                             Very bad                     Fair          Good       Very good    Outstanding




Each Dimension comprises multivariate identifiers of             understanding visitor satisfaction. At times, multiple
the online experience. Scores are assigned to each               sources of business intelligence must be leveraged
of these Attributes and all Attribute scores are rolled          to understand the customer more fully.
up and weighted equally within the iPSI.
                                                                 Yet the iPerceptions Satisfaction        Index (iPSI) is
While we applaud Reichheld’s call for a “standard,               increasingly becoming the gold             standard for
intuitive system” to measure experiential data, no               online voice of the customer (VoC)       research. iPSI
company or index can possess a monopoly on                       scores have now been outputted            for over three




  iPerceptions                                                                                                       7
million actual website visitors. Most critically, the     More specifically, our perceptual framework re-
iPSI is, to our knowledge, the only metric conceived      casts the web experience as the convergence of
exclusively for online data collection. Thus, it does     two conceptual axes. One axis is temporal, and it
not suffer from the same portability issues that have     weighs the creation of immediate versus long-term
to date plagued metrics based on offline collection       value; the other axis is based around orientation and
methods.                                                  measures front-end user desirability versus back-
                                                          end site usefulness. Understanding the website in
                                                          these terms allows us to evaluate how effectively it
      3. Understanding the totality of the web            helps to construct and maintain crucial emotive links
      experience is key                                   with the visitor. These are the links that either make
                                                          or break the company-customer relationship; time
                                                          and again, data sets show that these links begin to
                                                          form well before the visitor clicks “buy now.”
Many companies are content only to understand
the purchase process, with the goal of identifying
leakage and boosting conversion. While this is
certainly a fruitful exercise on many levels,                . Online surveys provide a safe forum for
transactional and behavioral studies of this nature          expression
fail to account for the totality of the web experience.
By restricting their scope of analysis to only the
purchase process, decision makers who employ
these tools exclusively blind themselves to many          Research that is inimical to enthusiastic participation
causal insights. Only attitudinal analytics can           by volunteers is bound to fail. In many ways, online
unearth the “why” behind the “what,” and not only in      surveying is a natural evolution of the telephone
terms of transactional behavior or pathing.               surveying method, but without its greatest flaws.
                                                          Our surveys never intrude on the website visit.
To his credit, Reichheld is dead on when he calls for     Instead, a polite solicitation is presented on arrival
surveying that includes every aspect of the online        to the site. If declined, visitors are thanked and sent
experience, including emotional and branding              on their way. If accepted, a second, minimized
issues. Our research consistently demonstrates            window is launched and visitors can then proceed
that relationship building is incipient well before the   without having any part of the site occluded by the
transactional stage of the site visit. This is why our    surveying interface. The focus should be on the site,
perceptual framework, while taking into account           not the survey. Therefore, after respondents have
elements that speak to commerce and conversion,           completed their visit and turned their attention toward
pushes well beyond that narrow ambit to include           the survey, they are again reminded to not begin the
all front end and back end aspects of the site            survey until they have thoroughly completed their
experience.                                               onsite task. Layered invitations are also coming into




  8                                                                                              iPerceptions
                                                   Immediate




                                                                      con
                                                        ion
                                                   at                    t




                                                                              en
                                             ig
                                            na v




                                                                                   t
                 Desirability                                 Adoption                         Usefulness




                                                                              vit y
                                             mo


                                                   ati



                                                                         c ti
                                                                              ra
                                               tiv


                                                         on
                                                                     i nt e




                                                    Long Term




use, and they undoubtedly point the way forward in
terms of user-friendliness.                                                    1. Respect for the customer improves
There is another important factor to consider                                  communication within a company
here. During telephone surveys, respondents are
routinely asked questions (for example, related to
age, income, or religion) that they may be unwilling                 Reichheld contends that employees manipulate
to answer while connected with another party. Their                  satisfaction scores to gain accolades; but the
discomfort can lead them to dissemble and furnish                    opinions of real customers in real situations are
incorrect or inaccurate information. Online surveys                  objective and immune to manipulation. While
are not plagued by these shortcomings. Instead,                      the currency of customer satisfaction is certainly
they provide a safe and secure forum for reasoned,                   tremendously valuable, progressive organizations
deliberative, and, above all, honest feedback. This                  understand that gauging the pulse of online visitors
way, concern for tact and personal embarrassment                     is a subtle and iterative procedure, one that must
never impede on the sincere expression of truth.                     necessarily draw upon the collective business




  iPerceptions                                                                                                        9
acumen of many divisions within a company. All
of our clients are dealing with a vast wealth of
                                                               Conclusion
experiential data, some of which do not fit easily
into established paradigms about how shoppers,
browsers, and researchers behave. This is precisely
why it is so important that customer insights get a         The evolution and maturation of the entire field of
wide readership and substantial upward mobility             web analytics have radically altered the way in
within a company. We always stress to our clients           which a company’s online presence is evaluated
the importance of engaging as many stakeholders             and have irrevocably placed the customer at the
as possible in a conversation about customers.              epicenter of all tactical and strategic website decision
A satisfied visitor base should be the concern of           making. Going further, it is a universally recognized
everyone.                                                   fact among industry leaders that survey-based
                                                            attitudinal analytics have furnished an invaluable
Working towards solutions with clients, we have
                                                            trove of business intelligence for website decision
found that a sincere and conscientious effort to
                                                            makers.
bring the voice of the customer to the decision
making table is healthy for an organization. Rather         At the same time, the expectations being placed
than inciting internecine conflict between divisions        on websites are maturing and becoming more
over who gets the highest loyalty scores, genuine           exacting. As noted by a leading mind in the field,
consumer advocacy heightens interoperability and            “ (now) there is a deep demand for the web channel to
helps to build lateral bridges between the various          be held just as accountable as the other channels…
components and departments of a company, all of             companies are demanding that the web leadership
which have an unequivocal stake in making sure              justify investments being poured into the channel.
visitors to the website leave feeling that their time was   This is an amazingly transformative experience for
well spent. Often, this dispels faulty assumptions          the channel as its leadership is looking in all places
about the customer and engenders a desire to                to prove results.”
innovate. As Jim Sterne so aptly notes, “Bringing
the voice of the customer to senior management is           One number can never be powerful enough to
one of the most powerful tools for internal change.”1       provide all the insights that website decision makers
I would add that it is also one of the most powerful        need to meet these new challenges. At the same
tools for boosting inter-departmental collaboration         time, one number can never allow for the full
and giving everyone a clear goal towards which              spectrum of visitor commentary and opinion. A far
to strive.                                                  more holistic picture is needed. This picture can only




1
    Jim Sterne, We Know What You Did Last Summer, 9.
2
    Avinash Kaushik, Web Analytics: An Hour a Day, 6.



     10                                                                                            iPerceptions
be drawn by survey-driven attitudinal analytics. The
time is now for a different approach, one driven only
by a devout respect for the voice of the customer.      Contact Us

About iPerceptions                                      USA
iPerceptions is one of North America’s leading          575 Madison Avenue
web-focused attitudinal analytics providers. Its        Suite 1006
webValidator Continuous Listening solution and          New York, NY
its Proprietary Satisfaction Index (iPSI), turn         100-511
thousands of data points into easy-to-understand        USA
strategic and tactical decision support for website
marketers. iPerceptions’ clients include such well      Tel: 877-796-3600
known brands as InterContinental Hotels, General        Fax: 866-8-600
Motors, Dell Computers, Hyundai, CompUSA, LG
Electronics, Toshiba, Choice Hotels International,      Canada
BMW and Hilton Hotels. iPerceptions has offices in      999 St-Catherine St. West
New York, Toronto and Montreal.                         Suite 500
                                                        Montreal, Quebec
                                                        H3Z 1T3
                                                        Canada

                                                        Tel: (51) 88-3600
                                                        Fax: (51) 8-600



   For further information please contact:              info@iperceptions.com
                                                        www.iperceptions.com
   Jonathan Levitt
   Vice President Marketing
   877.796.3600 ext 50
   jlevitt@iperceptions.com
   www.iperceptions.com




  iPerceptions                                                                       11
Know
Satisfaction
           TM

				
DOCUMENT INFO
Shared By:
Categories:
Stats:
views:337
posted:2/21/2008
language:English
pages:12
Description: 10 REASONS WHY SATISFACTION SURVEYS DONT FAIL