15
What can we learn from advertising evaluation research?
• Rocket analogy • Decision Analyst
• Http://www.decisionanalyst.com • CopyScreen • CopyCheck
• What are the pros and cons of testing ads and marketing communication pieces at various stages of development?
Discussion Slide
© 2007 by Prentice Hall
15-1
15
Evaluating an Integrated Marketing Program
Chapter Overview
• Matching methods with objectives • Message evaluations • Evaluation criteria • Behavioral evaluations • Evaluating public relations • Evaluating the IMC program
© 2007 by Prentice Hall
15-2
Evaluation Categories
• Message evaluation techniques
• Physical design • Cognitive elements • Affective elements
• Respondent behavior evaluations
• Conative elements • Measurable with numbers • Customer actions
© 2007 by Prentice Hall 15-3
Evaluation and IMC Objectives
• Evaluation should match objectives • Pre- and posttest analysis • Levels of analyses
• • • • • Short-term Long-term Product-specific Corporate level Affective, cognitive, or conative
© 2007 by Prentice Hall
15-4
FIGURE
15.1
Message Evaluation Techniques and When to Use Them
Message Evaluation Method
• • • • • • • • Concept testing Copytesting Recall tests Recognition tests Attitude and opinion tests Emotional reaction tests Physiological tests Persuasion analysis
When the Test Is Normally Used
Prior to ad development Final stages, or finished ad Primarily after ad has been launched After ad has been launched Anytime during or after ad development Anytime during or after ad development Anytime during or after ad development After ad has been launched
© 2007 by Prentice Hall
15-5
Concept Testing
• Prior to ad development • Average cost of 30-second TV ad is $358,000 • Focus groups • Concept testing instruments
• Comprehension tests • Reaction tests
© 2007 by Prentice Hall 15-6
Copytesting
• Used when marketing piece is finished or in final stages • Methods used
• • • • Portfolio test Theater test Focus groups Mall intercept
© 2007 by Prentice Hall 15-7
Copytesting
Copytesting can be used to determine if viewers comprehend this ad and what their reaction to it is.
© 2007 by Prentice Hall
15-8
Copytesting
• Criticisms of copytesting
• Some agencies not using • Stifles creativity • Focus groups not good judge
• Support of copytesting
• Issue of accountability • Majority support because clients want support for ad decision
© 2007 by Prentice Hall
15-9
Recall Tests
• • • • • Day-after recall (DAR) Unaided recall Aided recall Incorrect answers Use primarily after ads launched
© 2007 by Prentice Hall
15-10
FIGURE
15.2
Items Tested for Recall • • • • • • • • • Product name or brand Firm name Company location Theme music Spokesperson Tagline Incentive being offered Product attributes Primary selling point of communication piece
© 2007 by Prentice Hall 15-11
Recall Tests: Do Viewers Remember?
© 2007 by Prentice Hall
15-12
Sample DAR Test
30-Second TV Advertisement for Pet Food
25.0%
20.0%
15.0%
10.0%
5.0%
0.0% Brand name Theme music Spokesperson Tagline Incentive Product Attribute
Test Ad
Competitor A
Competitor B
© 2007 by Prentice Hall 15-13
Sample DAR Test
30-Second TV Advertisement for Pet Food
24.6% 25.0% 21.4% 20.0% 18.3% 16.3% 14.6% 15.0%
Overall Recall 12.9%
10.0%
9.4%
8.5%
5.0%
0.0% Males Females Pet Owners Dog Owners Ages 18- Ages 36- Ages 51+ 35 50
© 2007 by Prentice Hall 15-14
Recall Decay Magazine Ad vs. Television Ad
100% 100% 100% 80% 60% 40% 20% 0% DAR Two days later Magazine Television Eight days later 86% 75% 65% 43%
Source: Magazines Canada’s Research Archive
© 2007 by Prentice Hall 15-15
Recall Tests
Factors That Influence Scores • Attitude towards advertising • Prominence of brand name
• Brand used by respondent • Institutional ads
• Respondent’s age
© 2007 by Prentice Hall
15-16
Recognition Tests
• Respondents shown marketing piece • Good for measuring
• Reaction • Comprehension • Likeability
• Often used with recall tests
© 2007 by Prentice Hall
15-17
Recognition Tests
• Expression of person’s interest
• Ad liked + 75% • Ad interesting + 50% • Brand used + 50%
• Affected by ad size, color, length • Scores do not decline over time
© 2007 by Prentice Hall
15-18
Recognition Tests
Can be used to measure
• Reaction • Comprehension • Likeability
© 2007 by Prentice Hall 15-19
Attitude and Opinion Tests
• Used in conjunction with other tests
• Recall tests • Recognition tests
• Closed-ended questions • Open-ended questions • Roper Start ADD + IMPACT
© 2007 by Prentice Hall
15-20
Emotional Reaction Tests
• Affective advertisements • Used for material designed to solicit emotions • Difficult to measure emotions with questions • Warmth monitor • Emotional reaction tests are self-reported instruments
© 2007 by Prentice Hall
15-21
Sample Graph from a Warmth Meter
30-Second TV Advertisement
Sample Ad Rating
Warmth Meter
Ad section that elicited negative emotions
Target Audience
Total Audience
Start
10 s econds
20 s econds
30 s econds
© 2007 by Prentice Hall
15-22
Physiological Arousal Tests
• Measure fluctuations in a person’s body • Psychogalvanometer – sweat • Pupillometric test – pupils of eyes • Psychophysiology – brain waves and currents • Cannot be faked easily
© 2007 by Prentice Hall
15-23
Persuasion Analysis
• Appraise persuasiveness of marketing item • Requires pre- and posttests • ASI Market Research studies
© 2007 by Prentice Hall
15-24
FIGURE
15.3
Copytesting Principles of PACT
• Testing procedure should be relevant to objectives. • Researchers should agree on how the results will be used in advance. • Multiple measures should be used. • The test should be based on some model or theory of human response to communication. • Testing procedure should allow for more than one exposure. • In selecting alternate ads to include in the test, they should be at the same stage in the process as the test ad. • The test should provide controls to avoid biases. • Sample used for the test should be representative of the target sample. • Testing procedure should demonstrate reliability and validity.
Source: Based on PACT document published in the Journal of Marketing, (1982) ,Vol. 11, No. 4, pp. 4-29. © 2007 by Prentice Hall 15-25
FIGURE
15.4
Behavioral Measures
• • • • •
Sales Response rates Redemption rates Test markets Purchase simulation tests
© 2007 by Prentice Hall 15-26
Sales and Response Rates
• Measuring sales with UPC codes • Scanner data
• Retailers • Manufacturers
• Sales changes can be caused by other factors
© 2007 by Prentice Hall
15-27
Difficulties in Evaluating Advertising
• Influence of other factors on behavior • Delayed impact of advertising • Consumers change their mind in the store • Whether brand is in evoked set • Goal of ad may be to build brand equity, not increase sales
© 2007 by Prentice Hall 15-28
FIGURE
15.5
Responses to Marketing Messages That Can Be Tracked
• • • • • •
Changes in sales Telephone inquiries Response cards Internet inquiries Direct marketing responses Redemption rate of sales promotion offers
• Coupons, premiums, contests, sweepstakes
© 2007 by Prentice Hall
15-29
Test Markets
• Used to assess: • Advertisements • Consumer and trade promotions • Pricing tactics • New products • Cost-effective method of evaluation prior to launch • Resembles actual situation • Design test market to model full marketing plan • Length of test market is a concern • Competitive actions must be considered
© 2007 by Prentice Hall
15-30
Purchase Simulation Tests
• • • • Bias in purchase intention questions Simulated purchase tests Research Systems Corporation Does not rely on opinions and attitudes
© 2007 by Prentice Hall
15-31
Evaluating Public Relations
• • • • Number of clippings Number of impressions Advertising equivalence Comparison to PR objectives
© 2007 by Prentice Hall
15-32
Evaluating the IMC Program
• • • • Greater demand for accountability ROI of advertising and marketing Difficult to measure ROI – 70% Difficult to define ROI – 70%
© 2007 by Prentice Hall
15-33
TAB LE
Definition of ROI
15.2
Percent Using
Definitions of ROI for Marketing
Incremental sales from marketing
Changes in brand awareness Total sales revenue from marketing
66%
57% 55%
Changes in purchase intentions
Changes in market share
55%
49%
Ratio of advertising costs to sales
Reach/frequency achieved
34%
30%
Gross rating points delivered
Post-buy analysis comparing the media plan to its delivery
25%
21%
Source: Paul J. Cough, “Study: Marketers Struggle to Measure Effectiveness,” Shoot, Vol. 45, No. 29 (August 20, 2004), pp. 7-8. © 2007 by Prentice Hall 15-34
FIGURE
15.6
Measures of Overall Health of a Company
• • • • • • • •
Market share Level of innovation Productivity Physical and financial resources Profitability Manager performance and development Employee performance and attitudes Social responsibility
© 2007 by Prentice Hall
Source: Pete Drucker, Management: Tools, Responsibilities, Practices, New York: Harper and Row, 1974. 15-35