Performance Metrics
CPS 181s
April 10, 2003
Should Performance Be
Measured?
Metrics
Assessing the progress and health of on-line businesses
Financial progress: sales, margins, profit, market share
Strategic metrics require entire examination of the company:
perception of value proposition, marketplace offering,
effectiveness of implementation
Branding and implementation metrics focus on
> supply-chain performance
> organizational dynamics
> marketing communication effectiveness – including branding
Seniorand stock market analysts tend to have a bias
toward financial metrics
provide history of the company in the marketplace
do not reflect the strategy of the company
Managers need early warning system – metrics
Should Senior Managers be
Concerned About Metrics?
Organization metrics represent performance targets of
the company
Percent increase in revenue
Percent increase in unique visitors
Length of time visitors remain on site
Cost of customer acquisition
Metrics valuable as employee reward or taking strategic
action
Change processes
Change strategy
Change product strategies
Metrics Drive Behavior in a
Number of Ways
Help define the business model
Concrete goals with precise measurements
Focusing on measurement can increase the precision of the value
proposition
Help communicate strategy
Document performance targets
Communicates strategy to workforce
Creates buy-in to the metric-setting processes
Help track performance
Instantaneous feedback concerning site performance real-time
Usage, visitors, length of time on site, average sales, page views
Metrics Drive Behavior in a
Number of Ways...
Help increase accountability
Metrics must be linked to the performance appraisal system
Individual performance can be appraisals tied to companywide,
team-specific, and individual metrics
Site usability can be tied to the interactive design function
Customer service metrics can be tied to the customer service
department
Help Align Objectives
precise objectives, departmental functional goals,
Align
companywide strategic activities as a whole
Current Challenges to Specifying
Metrics for Online Businesses
Companies Strategies Change Rapidly – Internet time
requires rapid change
Measurement is resource intensive
Setup and maintenance of capital intensive/human resource
intensive systems and procedures
Data capture, data mining, and information use require time and
commitment of senior executives
measurement systems are vulnerable – easy to
Online
manipulate final results
Current Challenges to Specifying
Metrics for Online Businesses...
Soft metrics are not valued by the investment community
Investment community looks at revenues, margins, number of
visitors, length of time on site, customer acquisition costs
But, ease of customer use, best value perception, and other market
research measures are frequently early warning indicators of site
performance
Meaningful metrics change on Internet time – often as quickly as
the time it takes to learn best how to interpret the retained original
data
What Should We Measure?
How Can We Assess the Health
of eCommerce Firms?...
The Balanced Scorecard
Introduced by Kaplan and Norton
Finance balanced by other domains of the business
Includes internal business processes and customer responses
Four categories
> Financial
> Customer
> Internal business systems
> Learning and growth
Kaplan-Norton Balanced Scorecard
FINANCIAL
To succeed financially, how
should we appear to our
shareholders?
CUSTOMER INTERNAL BUSINESS
Vision PROCESS
To achieve our vision, how
should we appear to our
and To satisfy our shareholders and
customers? Strategy customers, at which businesses
must we excel?
LEARNING AND GROWTH
To achieve our vision, how will
we sustain our ability to change
and improve?
Source: Kaplan, Robert S. and David P. Norton. 1996. Using the Balanced Scorecard as a Strategic Management System. Harvard Business Review 74 (January-February): 76.
K-N Financial and Customer
Metrics
Financial Metrics
Revenue, revenue growth, gross margins, operating income,
net margin, earnings per share, cash flow
Reflect recent planning period , to some degree all previous
planning periods
Customer Metrics
Management relationship of customers by the firm
Marketshare
Customer acquisition
Customer satisfaction
Customer profitability
Customized by the target segment
K-N Internal Business Process
Metrics
Innovation includes customer’s perception of
innovativeness of the company
Operations – quality of entire supply
Postsale service – quality being offered to
customer
K-N Learning and Growth
Metrics
Employee
Selection
Training
Retention
Satisfaction
Information Systems
Qualityof infrastructure
Timeliness, accuracy, and utility of data
Motivation
Empowerment
Alignment of company and personal goals
Limitations to the K-N
Balanced Scorecard
No clear definition of strategy or business
models
Unclear location of organizational
capabilities or resources in framework
Unclear where partnerships reside in
framework
The Rayport-Jaworski Performance
Dashboard and Strategy Framework
The Performance Dashboard - intended to reflect the
health of the business
The Strategy Framework Drives the Necessary Metrics
Six critical steps
> opportunity assessment
> business-model metrics
> customer-interface design
> market communications and branding
> implementations
> financial
Introducing the R-J Performance Dashboard
Steps of Strategy:
Market Opportunity Implementation
Business Model Customer Interface Evaluation
Assessment and Branding
Performance
Dashboard:
Implementation Customer
Opportunity Business Model Financial
& Branding Interface &
Metrics Metrics Metrics
Metrics Outcome Metrics
Areas
Addressed:
Size of market Uniqueness of value Branding Customer acquisition Revenue
opportunity proposition
Technology Changes in customer Profit
Attractiveness of Attractiveness of infrastructure behavior
Cost
target segments offering
Internal organization Site experience and
Balance sheet
Competitive Firm capabilities usability
Relationship with
landscape relative to
suppliers and
competition
partners
Sustainability of
Fulfillment capability
competitive position
Components of the R-J
Performance Dashboard
Opportunity Metrics
to discern unfulfilled needs in the market place is critical
Ability
Market-opportunity metrics asses the degree to which the firm
can accurately gauge the market opportunity
Business-model Metrics
Capture subcomponents of the business model: egg diagram,
resource systems, financial metrics
Value Proposition or Cluster Benefits Metrics
> target segments, benefits offered, capabiliti4s that drive benefits
> focus on customer’s perceptions
> performance relative to the competitor’s benefits
R-J Branding and
Implementation Metrics
Focus on the supply-chain performance,
organizational dynamics, marketing
communication effectiveness (including
branding)
Fulfillment of the brand process metrics:
customer awareness of the brand
Innovation metrics: ability to rapidly innovate,
even if in a potentially discontinuous fashion
R-J Customer-Interface
Metrics
Customer-interface and customer-outcome metrics
Customer’s perception of the firm’s performance on
each level of the 7 C’s of the customer interface
Example:customer’s rating of customization compared to
competitors’
Other critical interface metrics
Including value proposition
R-J Customer-Outcome
Metrics
Capture process measures that the firm will produce
favorable customer responses such as satisfaction
and loyalty
Focus on subjective customer-outcome metrics
satisfaction
length of stay at site (stickiness)
overall evaluation of the experience
Focus on objective customer-outcome metrics
customer acquisition costs
average order size
customer profitability
number of visits per month
Financial Metrics
Revenues
Costs
Profits
Balance–sheet metrics
Lise Buyer
Life Cycle of a Company
STARTUP/ CUSTOMER
MONETIZATION MATURITY
BETA ACQUISITION
6 Months- 1 Year 1 Year - 2 Years 2 Years - 5 years > 5 Years
Strategy Develop a Build market Increase Control costs and
platform for rapid share as quickly revenues and optimize
growth by as possible by customer lock-in marketing
building a strong aggressively by developing expenditures to
team and spending on new revenue achieve profitable
creating a flexible partnerships and streams. growth.
site. promotion.
Examples* Business.com mValue.com Yahoo.com Schwab
TNBT.com Paypal.com Amazon.com AOL
Evite.com
* At time of writing (3/2000)
What are the Steps to Implement
the R-J Performance Dashboard?
Step One: articulate business strategy
market opportunity assessment
business model
customer-interface design
branding
implementation
evaluation
Steps Two and Three
Step Two: translate strategy into desired outcomes
not settling target levels but specifying outcomes
Step Three: devise metrics
identifiesspecific metrics that reflect the desired
outcomes
look-to-book ratios
Bill Gurley on the Power of
Conversion Rates
Conversion Rate 2% 4% 8%
Advertising Spend $ 10,000 $ 10,000 $ 10,000
Visitors 5000 5000 5000
Transactions 100 200 400
Cost/Transaction $ 100 $ 50 $ 25
Revenue $ 10,000 $ 20,000 $ 40,000
Marketing/Revenue (%) 100% 50% 25%
Average transaction size = $100 Source: J William Gurley
Source: Gurley, J. William. 2000. The Most Powerful Metric of All. CNET News.com, 21 February.
URL: http://www.news.com/Perspectives/Column/0,176,403,00/html?tag=st.ne.
Steps Four and Five
Step Four: link metrics to leading and lagging
indicators
map entire set of indicators
including leading & lagging indicators
Step Five: calculate current and target performance
current conversion rates
advertising expenditures
degree of positive word of mouth
Blueprint to the R-J Performance Dashboard
Step One: Step Two: Step Three: Step Four: Step Five:
Articulate Translate Devise Link Metrics Calculate
Business Strategy Into Metrics To Leading Current And
Strategy Desired And Lagging Target
Outcomes Indicators Performance
Market Opportunity Market Opportunity
• Opportunity size? • Market size and growth
• Competitive environment? • Average age and income
• Competitor concentration
Business Model Business Model
• Unique value proposition? • Customer perceived benefits
Develop resource system required to deliver the strategy
• Capabilities vs. competition? • Exclusive partnerships • For each metric, • For each metric,
Define goals and value proposition
• $ invested in technology vs determine the metrics calculate current level
competition that it affects and that of performance
affect it • Determine target
Implementation and Implementation • Map the linked set of level required to meet
Branding • Customer brand awareness metrics, indicating outcomes described
• How to develop brand? • System uptime percentage leading and lagging in Step 2
• How to go to market? • Number of IT staff indicators • Ensure that targets
• % inaccurate orders • Ensure that there is a are consistent with
balance between each other
Customer Customer leading and lagging
• How to acquire customers? • Market share indicators
• How will customers change? • Purchases / year
• The customer experience? • Success rate
• Service requests / customer
Financial Financial
• Financial consequences in • Revenue
terms of revenue, profit, cost • Profit
and balance sheet? • Earnings per share
• Debt to Equity ratio
What are the Three Sources of
Metrics Information That Firms Can
Use to Chart Their Progress?
Industry standard metrics
Benchmark values
On-line Information
Online Information
Market research – primary customer data
Analysts reports – blending market data
Financial information – income statement, balance sheet,
statement of cash flow
Use of Bizrate – use of dimensions of service
Use of Forrester Reports – e-commerce reports –
competitors, consumers, technology evolution in a segment
of the industry
Use of Hoover’s Online – financial and market information for
both online and offline companies
U.S. Top 25 Web & Digital Media Properties (March 2000)
Rank Digital Media/Web Unique Visitors (000)
1 AOL Network 59,858
2 Yahoo Sites 48,336
3 Microsoft Sites 46,581
4 Lycos 32,899
5 Excite @ Home 28,571
6 Go Network 23,006
7 NBC Internet 17,169
8 Amazon 15,217
9 Time Warner Online 13,636
10 Real.com Network 13,482
11 Go2Net Network 13,041
12 AltaVista Network 12,557
13 About.com Sites 12,329
14 Ask Jeeves 12,269
15 eBay 11,155
16 LookSmart 10,557
17 ZDNet Sites 10,226
18 CNET Networks 10,023
19 eUniverse Network 9,198
20 JUNO/JUNO.COM 9,177
21 EarthLink 8,526
22 Infospace Impressions 8,305
23 Viacom Online 8,139
24 FortuneCity Network 7,809
25 CitySearch-TicketMaster 7,689
Online
Source: Media Metrix (www.mediametrix.com/usa/press/releases/20000424.jsp) March 31, 2000
Market Research Source - Bizrate.com
Analyst Source - Forrester
Financial Information Source - Hoovers Online
Mapping Internet Research Onto The Performance Dashboard
Financial
Market Research Analyst Information
Media AC BizRate Forrester Gomez Jupiter Creative Hoovers
Metrix Nielsen Good
Market Info
Market
Traffic
Fulfillment
Implementation Implemen-
tation
Privacy
Usability
Content
Customer
Customer
Satisfaction
Customer
Service
Financial
Financial Performance