Get documents about "07 The art and science of marketing measurement
Document Sample
Delivery
The art and science
of marketing
measurement
Jacquelyn s. Thomas
Associate Professor of integrated Marketing
Communications, Northwestern university
sage wagner wodarz
Vice President, Marketing,
HSBC Card and Retail Services
reigh robitaille
Associate Marketing intelligence Manager, Clorox
Abstract
With millions of dollars directed towards funding an infinite number
of marketing initiatives companies must be able to quickly evaluate
and determine which marketing programs most successfully meet
the company’s business objectives and thus deserve continued finan-
cial support. ROMI is a marketing specific measure with roots derived
from the basic financial and accounting concept of return on invest-
ment (ROI). This research investigates how companies are currently
measuring the effectiveness of their marketing investments. We
discuss several best practices and how companies are implementing
them. We find that the ‘science’ of marketing measurement is just a
prerequisite. It is the ‘art’ of measurement that leads to a competitive
advantage for most firms.
43
The art and science of marketing measurement
With millions of dollars directed towards funding an infinite number state of marketing measurement this research lead to insights
of marketing initiatives, companies must be able to quickly evalu- with respect to what appears to be the next horizon with respect
ate and determine which marketing programs most successfully to marketing measurement — social media and the impact of word-
meet the company’s business objectives and thus deserve contin- of-mouth.
ued financial support. New marketing mediums arise daily, so it is
particularly imperative that companies know which vehicles are At the onset, one thing became amazingly clear: best-in-class mar-
most effective and distribute their marketing budget appropriately. keting measurement extends beyond the marketing department of
Historically, the challenge many marketers face is the ability to most organizations. Specifically, the skill set and efforts required
demonstrate tangible measures of their direct efforts. The ability to to apply best-in-class ROMI analysis goes beyond the day-to-day
quantify and communicate their efforts secures marketers a seat at function of the marketing department, which traditionally tracks
C-level discussions, allowing them to justify their budgets and plan attitudinal (i.e., satisfaction, awareness) and behavioral (i.e., pur-
for more profitable future investments. Thus one could argue that chase frequency, trial rate) measures. Interestingly, we also picked
the ‘holy grail’ of marketing metrics is return on marketing invest- up on another trend — a lack of measurement. Time constraints
ment (ROMI). are often cited as the reason for the lack of focused measurement
of marketing effectiveness. However, others voiced a different
ROMI is a marketing specific measure with roots derived from the explanation. DeAngelis notes, “It is human nature to shy away from
basic financial and accounting concept of return on investment focusing on measures that look sobering. We [at Wachovia] do in
(ROI). In other words, ROMI is a measure of how efficiently market- fact have measures that don’t all have arrows pointing up. But we
ers are spending their marketing dollars. As we will demonstrate use that information to figure out what we can do differently in
later, the math behind ROMI computations is straightforward and order to improve.”
fairly common. Less well established is how companies are using
this metric in practice and for success. This is the central issue we A hard-line retailer (HLR) echoed DeAngelis’ sentiment, pointing
seek to illuminate in this article. out that in addition to validating process effectiveness, measuring
ROMI also provides an understanding of what is not working and
Bob DeAngelis, Senior VP-Managing Director of Customer Analytics, what needs to be changed: “We’ll dial it up, dial it back… if it’s not
Research and Targeting at Wachovia says developing a sophis- meeting threshold, how are we going to cut costs to get the effec-
ticated ROMI analysis requires “technology, commitment, and tiveness up?”
resources.” It is not an effort easily or quickly achieved, nor can
it be adopted with the support of one or even a few employees. Uncovering nuggets of truth and developing an action plan for
Moreover, without the support of senior management, the initiative improvement enhances the marketers’ credibility in the executive
goes nowhere. Comments such as this suggest that measuring the suite and provides invaluable insights that may lead to a competitive
effects of marketing is more than a simple quantitative exercise, as advantage. With this as the goal: “how do companies leverage their
its roots may suggest. What is behind the measurement process? marketing measurement to get there?” Synthesizing all of our data,
Are marketing executives up for the challenge? we discovered the answer: best-in-class marketing measurement is
both an art and a science. The science (i.e., data and models) is a
A seminar hosted by a prominent business school captivated thirty- prerequisite. We found that nearly everyone who actively engaged
two senior level marketing executives struggling daily to justify and in ROMI measurement had some sort of model in place, ranging
quantify returns on their investments. Some primary challenges from the most basic calculations to highly sophisticated marketing
cited with respect to measuring the effects of marketing included a mix models with thousands of variables. Despite the ubiquitous
lack of access to critical data, a lack of ability to attribute incremen- models, the companies we interviewed had vastly different levels of
tal returns to the driving marketing vehicle, and a lack of internal success and satisfaction with their ROMI capabilities.
technology and employee resources allocated to tracking returns.
Fueled by this insight, we set out to get a behind-the-scenes look The companies that felt marketing measurement has contributed to
at the practice of marketing measurement. In addition to leverag- their success attribute that achievement to the art of the measure-
ing secondary research, we conducted a series of interviews with ment process. In their view, moving beyond the science to the art
mid- to senior-level executives in various industries. Examples of leads to a distinctive competitive advantage.
those industries include consumer package goods manufacturing,
apparel retailing, financial services, hard-goods retailing, advertis- science — the prerequisites
ing/communication agencies, technology and chemical manufactur- Science is critical to measurement and no business could produce
ing, technology retailing, automobile rental, third-party logistics successful ROMI measures without proper models and data in place.
providers, and consulting. In addition to learning about the current This is old news, but still a critical aspect of the overall process.
44 – The journal of financial transformation
The art and science of marketing measurement
The tactical elements of the ROMI process include compiling, spent gathering the data necessary for analysis [Adelman (2003)].
cleansing, and storing the appropriate data. A model is only as Capital One Financial Corporation found that by implementing a
valuable as the data put into it, so successful data management centralized data warehouse, data gathering time was reduced to
is critical. Several companies we talked to are investing heavily in 20%, which left more time for analysts to do their intended job of
the resources required to manage the various data sources. We answering the backlog of questions. This suggests the importance
deduced several critical elements for improving the data manage- for key stakeholders and data users to reach a consensus on what
ment system and the analytic capabilities of the organization. information should be collected and stored in the central data
repository.
Data management
Data management element #1 — create access to a centralized Data management element #2 — manage the quality of the data
data repository Once a company has compiled, integrated, and cleansed the data, a
The collection and management of disparate data from both inter- formalized process for data management must also be developed.
nal sources and external (vendor or third-party) supplemental Accurate and ongoing data entry and auditing is a crucial step in
suppliers creates a complex situation. Several of the consumer the data management process. According to one of the consumer
products companies we spoke with admit that relying on channel package goods companies that we interviewed, a formalized data
partners to provide end user transactional data from a recent sell- cleansing and validation process helps analysts focus their time
ing period can lead to data quality issues. Additionally, some critical on leveraging the data to answer critical questions. Data manage-
or high value channel partners (i.e., Walmart) refuse to supply pur- ment can involve anything from data input and audit procedures to
chase data at all or delay the availability of critical time-sensitive handling outliers and creating new variables. In addition to the 100+
information. For example, the attitudinal data and brand metrics of analysts mentioned earlier, Wachovia also has a corporate data
an apparel retailer are stored with a third-party vendor, making it management and governance group whose primary responsibility
difficult to access the data when necessary. is to manage the data warehouses. Data managers are also respon-
sible for working with a cross-functional team to identify the critical
With respect to fragmented data, the most important thing a com- pieces of data that are needed to compute ROI. These teams might
pany can do is create a single centralized database. An executive at include representatives from marketing, sales, finance, media, and
Nimblefish observed, “Marketing is fifth on IT’s list of things to get new products groups.
to and the fragmented requests [for a centralized database] going
up to the CEO won’t make it on his or her to-do list.” To overcome Analytical tools and technologies
this prioritization issue and to make sure the request for a consoli- Analytical element #1 — develop the model
dated database makes it on to the CEO’s radar they sold their case The companies that seemed most satisfied with their marketing
by demonstrating in a series of smaller-scale projects the long- measurement used some form of regression or marketing mix model
term ROI and savings that would be garnered from a central data to optimize resource allocation and measure marketing returns. We
source. While the consolidation was expensive and time-intensive, observed this more sophisticated level of analysis at companies like
Nimblefish affirms that the pay-off has been worth it. Frito Lay, Wachovia, the hard-line goods retailer, and the apparel
retailer. Procter and Gamble has also been reportedly using these
With redundant data accounting for approximately 50% of the types of models to aid in media allocations for some of their brands
data in data warehouses, significant cost savings calculations can [Neff (2006)]. Because the customer intelligence derived from
support a plan for consolidation when approaching senior manage- these models provides such a valuable competitive advantage, the
ment. An organization in Sid Adelman’s (2003) study consolidated customized tools developed by software designers, IT personnel,
their six data marts into a single warehouse for a total savings and econometricians are well-kept secrets. Developing a propri-
(over three years) of U.S.$11.5 million at the cost of U.S.$1.9 million. etary model will take several years and will likely be a multi-million
If a sophisticated data warehouse is used, the overhead should be dollar investment, in terms of time and expense.
allocated across all programs [Adelman (2003)]. The average total
cost of ownership of a data warehouse includes the maintenance, To realize the benefit of this investment, the company must establish
training, and software costs, in addition to the initial purchase. Data a model’s purpose before implementation. Readily available data
warehouse maintenance costs can exceed three times the initial can help guide this decision and aid in determining what data should
purchase cost. be gathered in the data warehouse. Start by defining the ultimate
goal: to drive sales, improve resource allocation, increase share of
A centralized source for data empowers employees and improves wallet or market share, or enhance customer lifetime value. With an
motivation by providing access to resources necessary for success. established goal in mind, the business can better understand which
Consider for a moment that on average 80% of an analyst’s time is variables to include and how to best proceed. Whatever the ultimate
45
The art and science of marketing measurement
goal, it is important that the model captures the relationship between as they spend increasingly more on marketing, they experience a
the marketing efforts and the resulting outcomes. diminishing, albeit profitable, rate of return. In some cases, com-
panies choose to spend more marketing on less profitable items
While some companies had technically sophisticated in-house because they expect enough volume increase to make up for the
people who could develop the model, others sought the advice of difference. Frito Lay optimizes the items they promote based on
outside consultants. An interesting and unique example of this is both manufacturing costs and incremental sales. Marketers must
the partnership between Dell and their advertising agency DDB manage their optimization options and decide whether to maximize
Chicago. Specifically, Dell partnered with DDB Chicago to create a profit or to maximize ROMI.
Marcom ROI model that combines the internal and external sources
of data required for analysis. The purpose of creating this tool was CDW explained that determining which metric to optimize depends
to gain marketing communications efficiencies and improve the on where a customer is in their lifecycle with the company. In CDW’s
speed of information exchange. The tool took several years to cre- case, a newly acquired customer will be measured with ROMI, while
ate and was developed in three phases which can be summarized as an existing customer will be held to a profitability measure.
follows [Bender and Zambianchi (2006)]:
This implies that companies can be flexible in their measurement
Phase 1 — Build a stronger infrastructure for managing the large methods, provided that the metrics are designed to achieve the
volume of transactional data (internal). behavioral objectives corresponding with each customer segment.
This finding also highlights the importance of overlaying customer
Phase 2 — Track and process all Marcom information with the intelligence models when deciding whether to optimize ROI or
involvement of the advertising agency (external). profitability.
Phase 3 — Bridge the gap between orders and marketing communi- Art — the competitive advantage
cation efforts to combine the two data sources. Some have a model, but few know how to effectively use it. As a
result, companies experience a wide range of satisfaction with their
Dell expects to see a 3% gain in efficiency as a result of activating ROMI process. In essence, those that have truly derived value from
this Marcom ROI tool. their marketing measurement efforts approach the challenge with
a more comprehensive perspective: they value and invest in the
Analytical element #2 — leverage the differences between science, but they also recognize and excel at the art. The science
romi and profitability provides a functional necessity, while the art provides a benefit that
Companies acknowledged a difference between their ability and/or cannot easily be replicated. The most successful firms in terms of
desire to optimize profitability versus ROMI. When profit increases ROMI all had something essential: a culture of accountability. When
at a decreasing rate, the highest ROMI falls below the optimal prof- asked how the different constituents in the organization respond
itability point. This is depicted graphically in Figure 1. This figure to the focus on data, models, and ROMI monitoring, the consistent
shows that the highest profits may not be achieved when the ROMI answer was that, ‘it is just expected.’ In our interview with HLR it
is the highest. To understand this, consider a CPG company that was stated, “Whether it’s marketing or merchandising or finance…
I think people are used to it … I’m sure to try to go to that in most
companies would be revolt — ‘you can’t make me do that, it’s gonna
take forever!’, but I think culturally people have just got to become
accustomed to it. There’s a certain way you do things, and you know
you’re going to have to report out on the effectiveness of whatever
ROMi PROFiT
you do, so better to load it in on the front side than deal with the
pain on the back end.”
Optimal ROMi Optimal
profitability
This same culture was expressed at Wachovia as a ‘show me’ cul-
ture. Wachovia’s CMO Jim Garrity spent the last four years estab-
lishing a “culture of measurement” [Mahoney (2005)]. Garrity’s
New customer Mature customer approach requires managers to support their recommendations
(Acquisition) (Retention)
input = and decisions with facts as they transition from “intuition-based
marketing
spend decision making” to “fact-based decision making.” As a result of
consolidating customer behavioral and attitudinal data, educating
Figure 1 – Profits versus ROMI employees, activating marketing-mix modeling, and measuring
4 – The journal of financial transformation
The art and science of marketing measurement
brand value, Garrity has not only improved Wachovia’s shareholder one thing in common: the support of the entire organization. This
value, but has maintained its position on Fortune’s top 100 places is bigger than just cross-functional teams. It hinges on senior
to work annual list. Wachovia’s success indicates that to be a role management. It is the organizational support that makes the data
model in the arena of ROMI, a culture shift needs to take place — in and analytic efforts worthwhile. It is difficult to enumerate how to
a way that encourages and motivates employees. create an organizational culture. Thus we consider this the art of
measurement. However, from the companies that did have a cul-
Similar to Wachovia, Frito Lay can also be noted as having a strong ture of accountability, we found several key commonalities which
culture of accountability. Frito Lay’s direct store delivery model pro- we identify as organizational focuses.
vides almost real-time data, making it possible to realize responsibil-
ity for marketing actions and make adjustments in the near-term as organizational focus #1 — established and agreed
opposed to several months into the future. At Frito, accountability upon definition of romi throughout the organization
is embraced at the highest levels. According to the VP of Marketing Aligning objectives — Senior level executives need to have the
Mix and Pricing at Frito, the CEO and the upper-level executives are same goal in mind. Finance executives tend to focus on short-term
all attuned to marketing measurement. As Frito Lay demonstrates, returns, while marketers and brand managers understand the
senior management must instill a culture of accountability and align long-term benefits of accepting lower returns in the short-term.
resources to allow for ongoing measurement systems. Before embarking on a costly and time-consuming ROMI initiative,
Dell recommends that any organization must first know: “What is
While senior management support is critical, the marketing group this tool [model] intended to do for us? What is it going to help us
can take a lead on initiating a culture of accountability. At CDW, the answer? And will that really convince us to change the way we do
marketing department convinced senior executives of the impor- business?” Gaining agreement on objectives is a necessary exercise
tance of ROMI by showcasing tangible cost savings. With ROI-based in change management for most organizations. Once the objectives
targeting on an acquisition campaign, they demonstrated that a are in place, a clear understanding of how the company defines
spending cut of 75% could yield the same number of customers ROMI helps internal and external teams focus their measurement
as the previous “one size fits all” acquisition campaigns. Given this efforts around collecting the appropriate data for accurate and
success, CDW now has an established measurement process in ongoing measurement.
place that is used on all marketing investments.
Defining romi — Our research participants clearly defined their
Our research uncovered evidence of this culture of accountability company’s perspective on ROMI as the ability to identify the incre-
throughout the marketing process. Senior management at another mental contribution margin associated with an incremental invest-
CPG company has mandated that program measurement plays a ment value. While some companies did not take into account the
key role during the planning and forecasting process. All marketing cost of capital when measuring ROMI, others did discount the future
managers at the apparel retailer must demonstrate a positive ROMI cash flow of both the return and the investment:
at the program-level to receive funding approval for their projects.
[NPV of incremental net marketing contribution ÷ NPV of incremen-
While not quite as far along on this culture dimension, the chemi- tal investment] x 100% = ROMI
cal and technology manufacturer showed signs of moving in this
direction. An internal communications group in this organization More sophisticated ROMI companies used a profit (margin) metric
was engaging in an initiative to educate and encourage managers rather than a revenue or sales measure and allocated overhead
to engage in marketing measurement, specifically ROMI. Due to costs to the model. For example, in order to determine the net
its complex channel structure, some divisions in the organization marketing contribution, all costs related to a specific marketing
had difficulty obtaining the relevant data for measurement. The program were isolated and included in the model. These costs and
organization had invested in a marcom measurement model, but sources within the organization may include those highlighted in
was not leveraging it to its fullest capability. Although there were Figure 2.
some challenges with respect to the prerequisites, their focus on
disseminating knowledge throughout their organization about mar- Developing a map of customer brand contact points will aid in
keting measurement demonstrates their commitment to validating identifying the cost and revenue drivers that should be included in
marketing efforts and emphasis on accountability. This example the model.
highlights how the art and the science must go hand in hand.
Adding the customer — In spite of the growth of CRM and LTV
If culture of accountability is the golden nugget with respect to measurement very few companies we spoke with defined their cur-
marketing measurement, all of our best-in-class companies had rent practice of ROMI as a customer-level return. Most viewed it as
4
The art and science of marketing measurement
cost source of data
more qualitative brand perception and ad effectiveness measure if a
Media buy (TV) Marketing quantifiable ROMI cannot be calculated. One CPG company that we
Agency/production costs Advertising spoke to measures both the short-term ROMI of tactical campaigns
Additional personnel during promotional period Operations
Overhead allocation of operational expenses Finance/operations
and the long-term ROMI of TV advertising and overall brand health.
Cost of goods Finance For the long-term analysis they run a modified version of their exist-
Cost of promotional discount Marketing ing model to account for returns over the different time spans.
Purchase and maintenance of data warehouse Analytics
Development and execution of marcom materials Marketing
organizational focus #3 — put people in place to
Benefit source of data execute the process
Incremental sales (revenue) Sales/operations
Incremental sales (profit) Finance
In addition to senior support, companies need analytically minded
Incremental market share External agencies employees. While clear and frequent communications from senior
Incremental customer satisfaction Customer insights management will reinforce the importance of measuring market-
Incremental customer loyalty Customer retention
ing investments, this top-down approach will fall short if employees
Figure 2 – Inputs to a ROMI calculation cannot deliver on the expectations. As Paul McGhee, Vice President
of Marketing Solutions at Nimblefish Technologies said, “You need
some heavy analytic types. Pulling the data is really hard. Analyzing
an aggregate measure of marketing activity at the program-level, it is even harder.”
and a few consider both. For example, Frito Lay measures their
trade funds at the customer level, which helps them forecast pro- Hiring practices — For marketers who do not have direct experi-
motional sales, while media expenses are measured in terms of the ence measuring ROI, understanding the fundamental drivers of the
program or brand, allowing them to account for cross-brand elas- models will lay the groundwork for future development. Marketers
ticities. We found that some of the practitioners that do not already must be able to grasp the inner-workings of the model’s ‘black
recognize returns at the customer level cite this as something they box’ as well as suggest recommendations for how to evolve the
aspire to in the long run, because of the significant and scalable model and improve its accuracy. Thus, companies must hire mar-
returns that level of customer intelligence would allow. keting managers with strong analytical backgrounds and acumen.
According to CDW and an apparel retailer, some of the strongest
organizational focus #2 — established romi process analytical minds in the marketing department are employees who
To successfully measure marketing returns, an organization must have a consumer packaged goods or direct marketing background.
promote a process supported by senior managers and expected of all Both industries provide an excellent training ground for ROMI
employees. This is true when using the traditional ROMI formula or thought-leaders because of the wealth of readily accessible data.
when supplementing ROMI metrics with program-specific ROI linked
to customer profitability, marketing mix models, and probability of Training — To improve the abilities of veteran employees com-
response models. For example, The Home Depot overlays customer panies must develop and implement training programs that will
satisfaction measures and is able to model the potential lift of a pro- advance the analytical skills of the entire marketing team. The
motional program while accounting for external market factors1. apparel retailer places new hires in training sessions with their
team’s dedicated finance person in order to learn the ROMI meth-
standards for success — Most companies we spoke with estab- odology.
lished a ROMI ‘hurdle rate’ before a marketing communications
project was started. This helped define success as well as provide The cfo — We have already acknowledged the importance of
a mid-campaign metric for benchmark analysis. Study participants cross-functional teams; a particularly important member of these
cited agreed upon ROMI thresholds as a key component to effective cross-functional teams should be a dedicated representative from
measurement. Gaining consensus from key stakeholders, such as the finance department. In many organizations, the CFO is respon-
finance and senior management, allows all members of the cross- sible for setting the return thresholds on capital expenditures. For
functional team to know how they will be measured during the marketing expenditures to be treated as investments, marketers
post-program analysis. In contrast to a single and consistent hurdle should involve the CFO when setting hurdle rates. The apparel
rate set by CFOs and used for all capital expenditures, many ROMI retailer touted the advantages of having a finance team mem-
hurdle rates are more flexible, to align with both short- and long-term ber dedicated to the marketing department. This person is the
business objectives. Some companies acknowledged that branding gatekeeper for all ROI models, processes, and formulas. Before
efforts are held to a different standard than promotional efforts, this position was in place, the marketing team had a difficult
due in part to the short-term versus long-term tracking abilities. For time obtaining budget approvals because the finance department
example, the apparel retailer sometimes substitutes ROMI with a did not see the value of marketing campaigns. Finance’s deeper
1 John Costello, Executive Vice President, Merchandising and Marketing: The Home
4 – The journal of financial transformation Depot, “Top marketers share perspectives on marketing measurement,” http://www.
marketingnpv.com/article.asp?id=1146
The art and science of marketing measurement
understanding of the profit-driving nature of the campaigns place, the one consistent characteristic that was expressed as a
strengthens marketing’s seat at the boardroom table, and in fact measurement nirvana is real-time data which allows for very near-
finance now lobbies for marketing dollars. The CFO also plays an time ROMI measurement, program evaluation, and possible adjust-
integral role in establishing the models (marketing mix or other- ment. Two of the consumer package goods companies expressed
wise) because of his or her access to data. Sources of costs can that they are approaching this nirvana. However, for many com-
be fragmented throughout an organization because of disparate panies real time information and customer insight may not be as
budget structures. The CFO’s responsibility in aggregating the easily addressed with the traditional ROMI science which we have
various data sources into one database is a key component to discussed. Rather, what is emerging, due to the explosion of social
creating an accurate model. media, such as Facebook and MySpace, is a new type of real-time
data. That data, which is now easily trackable, are what people are
organizational focus #4 — measurement is an saying to each other about companies, brands, services, and their
ongoing process experiences with these offerings.
By establishing checkpoints for continuous measurement through-
out a marketing program, the organization creates an ongoing Companies have always acknowledged the impact, either posi-
process. When asked during the interviews, “How often do you tive or negative, that WOM can have upon a business. With the
measure ROMI during a marketing program?” we heard an over- Internet, blogs, chat rooms, news groups, and other forms of elec-
whelming response that pre-, during-, and post-metrics are crucial tronic media, the reach and potential impact of WOM is immense
to establishing an actionable baseline for setting the objectives for and growing. For example, www.howardforums.com logged over
future programs. 279,000 messages just about cellular phones over a three-month
period. All of these messages came from over 3700 unique posters.
Although we do not have the ultimate secret for how to create a While the number of posters may be small relative to any firm’s
culture of accountability, we have deduced four common organiza- total customer base, clearly these people represent a group that is
tional imperatives common to all of the companies who excelled at very engaged in cellular phones. Naturally, there are two burning
ROMI and marketing measurement. In Figure 3 we depict how these questions that a cellular phone provider would want to know about
imperatives may be ordered to help facilitate ROMI measurement this, “What are the people saying?” and “How can I influence what
and the evolution of a culture that places a priority on holding people are saying?”
people accountable for their efforts.
Without a doubt businesses are taking notice of these new media
forums. Some are actively trying to influence the ‘buzz’ that occurs
1. 2. 3. 4. in online forums for their benefit. Others are taking a more pas-
Clear understanding Establish Putting people in Make measurement
and agreed upon the process place to execute an ongoing process sive approach and just observing and tracking the dialogs in these
definition the process
forums2. Their goal is to increase their customer intelligence and
learning in order to better serve their customers. Regardless of
whether firms take an active or passive approach to these new
Figure 3 – Facilitating ROMi in the organization
media platforms, as they continue to emerge and grow, market-
ers will be faced with the option of trying to facilitate venues for
An interesting caveat is that with respect to the third step encouraging buzz. With that come the questions: will allocating
— “Putting people in place to execute the process” — organizations marketing resources to influence and/or study WOM be worth the
must employ change management techniques and clearly com- investment? And, how can WOM be measured?
municate this process to all employees. This may be necessary in
order to mitigate employee resistance. When implementing their Applying quantitative metrics to WOM and determining the ROMI on
reorganization, Wachovia overcame the initial employee resistance WOM investments is where many that we spoke to said that their
through communications that conveyed senior level support and traditional ROMI approaches fell short. Because of this, some com-
fostering a true culture change within the company. Garrity stated panies faced a lack of enthusiasm or slight resistance to investing
that, “Over time the changes were accepted and expected.” marketing dollars in new media platforms and research. However,
Scott Deaver, Executive Vice President of Marketing at the Cendant
The future of the art plus science Car Rental Group warns that, “Unless you’re careful, the tyranny
Although the best-in-class ROMI companies expressed high levels of of measurement dumbs down marketing to focus unduly on just
satisfaction with their measurement approach, they still seemed to the measurable elements3.” Garrity at Wachovia believes that
be striving for some measurement nirvana. What is that nirvana? marketing managers will continue to purchase traditional forms
Interestingly, despite the uniqueness of each business’s market- of media (TV, Print) because they are comfortable with it. “Tried
2 “Is there a reliable way to measure word-of-mouth marketing?” http://www.market-
ingnpv.com/article.asp?ix=1175&key=%233xVwdvSWLWn 49
3 “Top Marketers Share Perspectives on Marketing Measurement,” http://www.market-
ingnpv.com/article.asp?id=1146
The art and science of marketing measurement
and true media is comfortable. The biggest challenge is getting final thought
people [marketing managers] to invest in new media. Online media Best-in-class marketing measurement requires companies to com-
[and blogs] are much easier to measure than traditional media… bine both the art and the science of measurement — data, analysis,
you can really track behaviors in a closed loop system online4.” and a culture of accountability. Without all three working together,
For example, a CPG company tests the effectiveness of new media companies will not reap the greatest benefit from their efforts.
through experimental design. By running test and control scenarios, There is a consensus that a ROMI culture encourages a disciplined
the customer insight and strategy team can revise their model to approach to marketing, however there is also a concern that too
pick up the impact of new message vehicles. Companies should view much emphasis on ROMI stifles creativity and innovation. With
their investments in new media as investments whose purpose is to today’s marketplace exploding with respect to social media, com-
understand and ultimately influence those who influence others. panies are being forced to take notice of a marketplace where it is
difficult to measure a valuable asset — word-of-mouth. The key to
Thus to take marketing measurement into the next decade, along successful ROMI implementation, then, is to establish a culture of
with a culture of accountability, companies need a culture of accountability that recognizes the value of the measurement and
exploration: a willingness to take calculated risks. Now that the to incorporate the knowledge that measurement provides without
new media platforms have provided consumers with a vehicle eliminating intuition altogether. As a hard-line retailer puts it,
to respond, it is possible to measure how much campaign mes- “[ROMI is] not going to replace tactics, [it’s] just used to provide
sages resonate among the public in real time and examine the deeper understanding.”
viral effect of their campaigns. Some specialized consultancies like
MotiveQuest help their clients achieve competitive advantage by references
• Adelman, S., 2003, “Measuring data warehouse return on investment,” Sid Adelman &
deriving insights from WOM. MotiveQuest measures word-of-mouth
Associates
or ‘buzz’ marketing from sources such as blogs, news groups, and • Bender, M., and A. Zambianchi, 2006, “The reality of ROI: Dell’s Approach to
web forums. In essence, MotiveQuest is an example of an online Measurement,” Journal of Integrated Marketing Communications, 16-21
• Mahoney, S., 2005, “Garrity seeks holy grail of ROI,” Point (Advertising Age),
anthropologist. They study communities to build a model of that
November, 8-15
society. From that model they derive what makes the community • Neff, J., 2006, “P&G pumps up print ad spending, trims TV,” Advertising Age,
tick. Then they communicate those insights to their clients to help November 6, 77:45, 1-2
them figure out the best strategies and tactics for the future. Their
clients feel that including what people are saying adds another Appendix
dimension to traditional marketing measurement. While they can vary in complexity, at its most basic level a Marketing Mix
Model can be defined as a regression that provides a baseline of sales
through which incremental sales can be tracked over time and attributed to
Some of MotiveQuest’s analysis involves not only the frequency the promotional levers employed by the marketing communications program.
of brand, product, or specific topic mentions (i.e., the graphics on An example of a linear regression model, with ‘sales’ as the dependent vari-
the latest video game), but also the emotional context in which the able and a mix of promotional marketing vehicles as the independent (input)
consumer writes about a brand, company, or topic. Using propri- variables is shown below.
etary models they provide their clients with insights with respect
Ŷ (sales) = b1*(dollar amount spent on point of sale displays) + b2* (trade
to consumer sentiment which can then been leveraged to develop dollars) + b3*(dollar amount spent on direct mail) + b4*(dollar amount
or modify strategies and to measure the effect of marketing ini- spent on online advertising) + b5*(dollar amount spent on TV advertising)
tiatives. Their clients report that this added insight has proven + b6*(number of PR exposures) + b7*(external market factors)
equally as influential to a campaign’s success as the campaign’s
The values of each of the standardized coefficients (b) indicate the strength
execution.
of that particular marketing vehicle on the sales impact. In the fictitious model
below, ‘point of sale displays’ provides the greatest impact on sales, while
While some marketers tend to shy away from such ‘raw’ feedback, ‘external market factors’ have a negative impact on sales.
it is a valuable gauge of market performance that simultaneously
links customer satisfaction and attitudinal information to financial Ŷ (sales) = 5.1*(dollar amount spent on point of sale displays) + 3.2* (trade
dollars) + 1.5*(dollar amount spent on direct mail) + 2.6*(dollar amount
returns. Based on MotiveQuest’s extremely high client retention
spent on online advertising) + 3.0*(dollar amount spent on TV advertising)
rate, it appears that marketers that supplement their own creativity + 1.1*(number of PR exposure) – 2.1*(external market factors)
with ROMI guidance tend to realize improved business success. As
new technologies become available, companies will have even more Additional drivers of the model may include external market factors such as
options and metrics to leverage their ROMI tools. industry consolidation, seasonality, media expenses, competitive activity, pric-
ing, and distribution factors. An advanced level of analytics and modeling is
required to operate a marketing mix tool.
4 Interview with Jim Garrity, CMO of Wachovia, 12/1/05
50 – The journal of financial transformation
