Docstoc

Dan Olsen: Product Development 101: Designing & Optimizing the DNA of a Killer App

Document Sample
Dan Olsen: Product Development 101: Designing & Optimizing the DNA of a Killer App Powered By Docstoc
					Designing & Optimizing the DNA of a Killer App
Dan Olsen, CEO & Founder, YourVersion
Startonomics · San Francisco · Oct 2, 2008
Copyright © 2008 Olsen Solutions LLC

Building the DNA of a Killer App with Metrics and Optimization
How to integrate metrics & optimization  Using metrics to optimize value creation


Value  Customer Value  UI Design


 Business

Illustrate with real world examples

Copyright © 2008 Olsen Solutions LLC

Adding Metrics and Optimization to your Product Process
Plan
Business Objectives Product Objectives Prioritized Feature List

Site Level

Scoping

Feature Level

Design
Develop Optimize

Requirements & Design Code Test Launch

Metrics & User Feedback
Copyright © 2008 Olsen Solutions LLC

Optimization through Iteration: Continuous Improvement
Measure the metric Analyze the metric Identify top opportunities to improve Design & develop the enhancement

Learning
Gaining knowledge: • Market • Customer • Domain • Usability

Launch the enhancement
Copyright © 2008 Olsen Solutions LLC

Optimizing Business Value Creation with Metrics

Copyright © 2008 Olsen Solutions LLC

Approaching Business as an Optimization Exercise
Given reality as it exists today, optimize our business results subject to our resource constraints.

Copyright © 2008 Olsen Solutions LLC

Define the Equation of your Business “Peeling the Onion”
Profit = Revenue - Cost Unique Visitors x Ad Revenue per Visitor Impressions/Visitor x Effective CPM / 1000 Visits/Visitor x Pageviews/Visit x Impressions/PV New Visitors + Returning Visitors Invited Visitors + Uninvited Visitors # of Users Sending Invites x Invites Sent/User x Invite Click-through Rate

Copyright © 2008 Olsen Solutions LLC

How to Track Your Metrics


Track each metric as daily time series
Date Unique Visitors Page views Ad Revenue New User Sign-ups …

4/24/08
4/27/08 …

10,100
10,500

29,600
27,100

25
24

490
480



Create ratios from primary metrics: X / Y
Example: How good is your registration page?  Okay: # of registered users per day  Better: registration conversion rate = # registered users / # uniques to reg page

Copyright © 2008 Olsen Solutions LLC

Sample Signup Page Yield Data
Daily Signup Page Yield vs. Time New Registered Users divided by Unique Visitors to Signup Page
100% 90% 80% 70% 60% 50% 40% 30% 20% 10%
Changed messaging Added questions to signup page Started requiring registration

Daily Signup Page Yield

0% 1/31 2/14 2/28 3/14 3/28 4/11 4/25 5/9 5/23 6/6 6/20 7/4 7/18 8/1 8/15 8/29 9/12 9/26 10/1 0

Copyright © 2008 Olsen Solutions LLC

Using Metrics to Optimize the Equation of your Business
   

What are the metrics for your business? Where is current value for each metric? How many resources to “move” each metric?


Developer-hours, time, money

Which metrics have highest ROI opportunities?
Metric A Good ROI Return Return Metric B Bad ROI Return Metric C Great ROI

Investment

Investment

Investment
Copyright © 2008 Olsen Solutions LLC

Case Study: Optimizing Viral Loop Metrics
% of users sending = 15% invites Invites per sender = 2.3 Invite click-through rate

Active Users
% of users who are active

Invite

Prospective Users Don’t Click

Click

Registration Process

Fail

Succeed
Conversion = 85% rate

Users

• Multiplied together, these metrics determine your viral ratio

• Which metric has highest ROI opportunity?
Copyright © 2008 Olsen Solutions LLC

Maximum Upside Potential of a Metric
?

100%

100%

85%

15%
0 0 % of users sending invitations

2.3 0 Avg # of invites sent per sender

Registration conversion rate
Max possible improvement

0.15 / 0.85 = 18%

0.85 / 0.15 = 570%

? / 2.3 = ?%

Copyright © 2008 Olsen Solutions LLC

Doubling Number of Invitations Sent per Sender by Adding Address Book Importer

Copyright © 2008 Olsen Solutions LLC

Optimizing Customer Value Creation with Metrics

Copyright © 2008 Olsen Solutions LLC

The Fuzzy Math of Customer Value


You create customer value with a product that
Satisfies customers’ needs  Is easy to use  Has a good price  Is better than other alternatives


  

Customer value = customer satisfaction Not fuzzy: # of users and frequency of use Understanding user needs and satisfaction
Quantitative: site analytics, usage metrics, surveys  Qualitative: support emails, usability sessions (best)

Copyright © 2008 Olsen Solutions LLC

What is Your Value Proposition?
Which user benefits are most important?  How well does your product deliver?

User Benefit
Benefit 1
Benefit 2 Benefit 3 Benefit 4 Benefit 5 Benefit 6
Doesn’t matter

Importance to User
Low High

Current User Satisfaction
–
+

Upside Potential
High High
opportunity

Low ? Med

opportunity

Low ? Low

Copyright © 2008 Olsen Solutions LLC

Analyzing Product Ideas by ROI
Return (Value Created)

4
3
Idea A

? Idea D Idea B Idea C

2
1
Idea F

1 2 3 4 Investment (developer-weeks)
Copyright © 2008 Olsen Solutions LLC

Prioritizing Product Ideas by ROI
Return (Value Created)
Idea C

8 7 6 5 4 3 2 1

Idea B

Idea A

1 2 3 4 5 Investment (developer-weeks)
Copyright © 2008 Olsen Solutions LLC

The UI Design Iceberg
What most people see and react to

Visual Design

Interaction Design

What good PMs and Designers think about

Information Architecture
Conceptual Design
Recommended reading: Jesse James Garrett’s “Elements of User Experience” chart, free at www.jjg.net

Copyright © 2008 Olsen Solutions LLC

Approaching UI Design Analytically


Typical UI design question: “When using web pages, do users scroll down?”
- Yes - No

  

UI questions are never yes/no! (not binary) Should ask: “What percentage of users …?” UI changes impact your metrics
Impact can be positive, negative, small, large  Seek high-ROI UI changes

Copyright © 2008 Olsen Solutions LLC

Put Key Conversion Actions Above The Fold
Landing Page A Landing Page B

Key conversion action is above the fold

The Fold

Key conversion action is below the fold
Copyright © 2008 Olsen Solutions LLC

The Fold Isn’t Binary Either
768 px

% of Users

The chrome steals about 170 pixels

600 px

1024 px

Data courtesy of ClickTale Free trial at www.clicktale.com

Copyright © 2008 Olsen Solutions LLC

Case Study: Account Signup Process Redesign
100%
100%

80%

Biggest drop
62.3% 58.8% 50.9%

% of Users

60%

40%

34.4%

32.7%

20%

0%

Sign in / Account Type Cash vs. Registration Margin

5 Partner Pages

3 Partner Pages

Copyright © 2008 Olsen Solutions LLC

Analysis of Sign In/Registration Flow
Open Account
44% 55% (24% of Total)

Register

Registration Process
45% drop off (20% of total)

64% of Total 36% overall drop off for this step

Account Selection

56%

Sign in

83% (46% of Total)

30% (14% of Total)

Forget Password
17% drop off (10% of total)

70% (32% of Total)

Change Password
20% drop off (6% of total)

80% (26% of Total)

Copyright © 2008 Olsen Solutions LLC

Redesigned User Flow Improved Registration Conversion Rate
Abandonment Rate (7 Day Moving Average)
Steps 1-2

80% 70%
Abandon ment Rate (7 Day Moving Averag e)

60% 50% 40% 30% 20% 10% 0%
1/6/03 1/13/03 10/14/02 10/21/02 10/28/02 11/11/02 11/18/02 11/25/02 12/16/02 12/23/02 12/30/02 1/20/03 10/7/02 11/4/02 12/2/02 12/9/02

Released New Design

37% improvement in conversion rate

Copyright © 2008 Olsen Solutions LLC

Summary: Building the DNA of a Killer App


Define what success means
 Customer

value proposition  Equation of your business

Instrument your site and track key metrics  Identify opportunities and prioritize by ROI  Launch, learn, and iterate


Copyright © 2008 Olsen Solutions LLC

Questions?

www.yourversion.com dan@yourversion.com
Copyright © 2008 Olsen Solutions LLC

Appendix

Copyright © 2008 Olsen Solutions LLC

Taking Optimization to The Next Level
Multivariate testing, aka A/B testing  New challenges emerge

 Are

we at local vs. global optimum?  How do we identify top new ideas to test? (creativity and innovation)  Are there assumptions we should challenge?  Are we missing the forest for the trees?
Copyright © 2008 Olsen Solutions LLC

Olsen’s Hierarchy of Web User Needs (adapted from Maslow)
Customer’s Perspective
Increasing Satisfaction

What does it mean to us? Usability & Design Feature Set

How compelling and easy to use is the functionality? Does the functionality work?

Absence of Bugs
Decreasing Dissatisfaction

Is the site fast enough?

Page Load Time

Is the site up when I want to use it?

Uptime
Copyright © 2008 Olsen Solutions LLC

Kano Model: User Needs & Satisfaction
User Satisfaction Delighter (wow)

Performance (more is better)

Need not met
Must Have Needs & features migrate over time

Need fully met

User Dissatisfaction

Copyright © 2008 Olsen Solutions LLC

Importance vs. Satisfaction Ask Users to Rate for Each Feature
100 95 90 85

Great
84 87

98

Bad
55 70 72 75

79 80 80

86 84

Importance

80 75 70 65 60 55 50 40 50 41

60

70 Satisfaction

80

90

100

Recommended reading: “What Customers Want” by Anthony Ulwick

Copyright © 2008 Olsen Solutions LLC

Scalability for Startups:
How to GROW Up Without BLOWing Up Frank Mashraqi http://Mashraqi.com


				
DOCUMENT INFO
Shared By:
Categories:
Stats:
views:352
posted:10/6/2008
language:English
pages:33
Description: What does it take to plan, research, and develop a great web 2.0 application Learn tips & best practices for setting the right process, goals, and metrics for your startup.