Seven Web Analytics Sins, and How to Avoid Them

VISUAL WEB ANALYTICS Seven Web Analytics Sins, and How to Avoid Them Toll-Free: (877) 773-2249 ClickTracks Analytics, Inc. 101 Cooper Street Santa Cruz, CA 95060 VISUAL WEB ANALYTICS Seven Web Analytics Sins, and How to Avoid Them “People commonly use statistics like a drunk uses a lamp post; for support rather than illumination.” - Mark Twain Mark Twain probably didn’t have web analytics in mind when he made the above statement—but he could have. Far too often, the purpose of web analytics is to produce a graph that goes up and to the right. But good, meaningful web analytics is a deductive business and marketing process, not just an effort to produce a chart. So how do you keep from sinning during your web analysis? By investing the time in truly exploring the data, gaining insight into its cause and effect and understanding the underlying data points while simultaneously filtering out the noise. Seven Web Analytics Sins white paper outlines common errors in web analytics thinking and/or implementation and offers suggestions for a more positive outcome: Sin #1: Simple Visitor Counts Learn the factors that can potentially skew visitor data. Sin #2: Search Term Popularity Understand why marketers must concentrate on the quality of visitors a keyword delivers, rather than the quantity. Sin #3: The Linear Funnel Learn the reasons why traditional sales funnels can lead to dangerous assumptions. Sin #4: Data Overload Know why it’s important to be able to separate interesting information from actionable information. Sin #5: Relying on Absolute Number Understand the reason why it’s more important to concentrate on trends instead of absolute numbers. Sin #6: Relying on Top 10 Lists Learn how getting stuck in your top 10 referrers can cost you long tail opportunities. Sin# 7: Technicolor Report Understand the reason why the way that information is displayed can have a huge impact on ease of use and perception. Toll-Free: (877) 773-2249 ClickTracks Analytics, Inc. 101 Cooper Street Santa Cruz, CA 95060 VISUAL WEB ANALYTICS Sin #1: Simple Visitor Counts All visitors are NOT created equal. Many entities arriving at your web site may appear to be human visitors, but they aren’t. Unfortunately this inhuman traffic is counted along side your human visitors—so you can see why ‘number of visitors’ data can be faulty. A few of the factors that lead to non-human visitors being counted as humans include: Bots Bots are robots that visit your site to extract data from its pages, including search engine crawlers Googlebot and Yahoo! Slurp. You definitely want these bots to visit your site so your pages will appear in search engine results. Legitimate bots follow a particular protocol, identifying themselves to web analytics programs so that they aren’t mistaken for human activity. But some bots don’t play by the rules, especially when they’re engaging in less-than-honorable behaviors like e-mail address harvesting. These ‘spam’ bots prefer the cloak of anonymity that masquerading as a human visitor gives them, and their programmers know if they follow protocol and identify themselves for what they really are, webmasters would block them. So they simulate human behavior and end up skewing your stats. Fraudulent Clicks A very real, rapidly growing problem, click fraud occurs when someone (or some computer script) knowingly clicks on your PPC ad with no interest in your products or services. Whether they’re trying to fraudulently make money from the traffic being sent to your site, or are maliciously attacking your ad budget, one thing is clear—when click fraud occurs, your ROI decreases as your stats increase. Uptime Monitoring Services Uptime monitoring services are ‘good’ traffic in that they provide a useful service. By probing your site as if they’re human visitors, they alert you to problems human visitors might be experiencing. These services are certainly useful ... and will certainly skew your data. 2 Toll-Free: (877) 773-2249 ClickTracks Analytics, Inc. 101 Cooper Street Santa Cruz, CA 95060 VISUAL WEB ANALYTICS Combine Visitor Counts with Other Stats for Better Accuracy Any of the aforementioned examples can cause a spike in a simple visitor count or an upward trend. Getting past this bogus information and to the actual human behavior requires a willingness to examine the data and not just take comfort in the stats, however uncomfortable that may be. Looking at a variety of stats in conjunction with visitor counts helps you put this fallible stat into perspective. Determining Qualified Visitors A metric must be defined to bring into focus only ‘qualified’ visitors. Different sites have different methods of defining this, but a fairly universal method is to include only those visitors that spend more than 10 seconds on the site. This will exclude many bots and instances of click fraud because those entities are likely to visit only a single page for a moment. In web analytics, we can only measure time on page/site at the second click, since it’s the second click that actually defines the boundaries of the first page time period. (For example, the program doesn’t know when someone closes the browser window.) It’s worth noting, however, that the 10-second-or-more label may inadvertently exclude some small number of real human clicks, like those who click through from an ad, don’t like what they see on the single landing page and immediately exit. Note: ClickTracks’ software contains algorithms to help distinguish bot activity from human activity, even when the bot does not follow prescribed protocols. This helps reduce the impact of bots on a simple visitor count, but click fraud and other non-human activity remains a significant volume of traffic for many sites. 3 Toll-Free: (877) 773-2249 ClickTracks Analytics, Inc. 101 Cooper Street Santa Cruz, CA 95060 VISUAL WEB ANALYTICS Sin #2: Search Term Popularity When it comes to choosing search terms, concentrate on quality over quantity. It is the quality of the traffic that matters, not the quantity or number of times it drives traffic to your site. In the world of marketing and web analytics, quality means the likelihood of a prospect becoming a customer. Whether it’s a trial download, sale or lead generation, we want to know what makes customers likely to buy so that we can work to encourage that behavior and replicate it elsewhere. Judging Popularity Is Straightforward Popular search terms are no exception to the rule and are easy to spot when performing site analysis. Note the blue shading in the example below. It helps to distinguish the relative popularity of the same term across different search engines and directories. From this we can begin to understand how various search engine users behave differently. How To Determine a Quality Search Term Instinctively, the first place we look for quality is in the amount of revenue generated by each keyword. However, this metric is a blunt instrument with a too-hard line that delineates success from failure. Two factors reduce its usefulness in determining the visitor’s likelihood to buy: cookie deletion and latent conversions. Setting a cookie is an absolute must for tracking ROI, but they’re only useful while they live on the client’s machine. If the user deletes the cookie, the conversion data become invisible. First-party cookies are the way to go, as they aren’t usually blocked by anti-virus or anti-spyware software. 4 Toll-Free: (877) 773-2249 ClickTracks Analytics, Inc. 101 Cooper Street Santa Cruz, CA 95060 VISUAL WEB ANALYTICS Latency or latent conversions occur when a user comes into your site via search, receives a cookie, looks around for a while, leaves and then returns sometime in the future making a purchase. While the keyword eventually gets the credit for triggering a sale, you’re without that data for 30 or 60 days or more, depending on your sales cycle. In essence, you’re spending valuable PPC dollars while you wait for ROI. Average Time on Site One of the most reliable, insightful metrics is Average Time on Site (ATOS). This metric speaks to the fact that web users just don’t waste their time on a site that doesn’t interest them. There are hundreds--even thousands of other sites just a click away in the search results if the one they click on doesn’t meet their needs. The screen shot below shows ATOS by term and search engine. Looking at the screen shot above, one could deduce that the longer the visit length, the greater the interest in the products or services offered. Please note that ATOS is not meaningful in absolute terms or across different sites—only use this metric within the same site for different search terms or segments. ATOS helps solve ROI problems because it’s not cookie-dependent, and there is no latency; you have data within hours of a campaign going live. ATOS closely correlates to the probability of prospects becoming customers, which is extremely helpful in making smart marketing decisions. When Average Time on Site Shows Short Visits When you find search terms that have a Short Average Time on Site compared to others, your first reaction might be to eliminate that keyword from your PPC campaigns. But before deleting the keyword and the campaign, make sure that you’re meeting the needs of the visitors. Does the landing page correlate to the search term they used? If not, make tweaks to the landing page instead of the campaign, then retest. 5 Toll-Free: (877) 773-2249 ClickTracks Analytics, Inc. 101 Cooper Street Santa Cruz, CA 95060 VISUAL WEB ANALYTICS Sin #3: The Linear Funnel Executives love funnel reports because they look so easy to understand, with their clearly defined exit points. But regrettably they’re often misleading, because they don’t expose the underlying human behavior. The fact is that humans do not traverse web sites in a linear fashion. A sales funnel shows: • • The number of visitors that arrive at your site, and out of those, how many become interested in your offerings? From the group who became interested, how many placed a product (or products) in their cart, or visited a demo, registration or similar conversion page? From that group, how many then actually purchased or completed the conversion process? • What a sales funnel fails to show is why the activity is occurring. A Fatal Funnel Flaw The problem with simple funnels is that they impose a linear flow on the user’s actions. According to a simple funnel, a user either advances to the next stage, or exits to something outside the funnel. The reality is much more complicated. People click around on countless links, surf around on product pages looking for the configuration that best suits their needs, compare products, and then finally place an item into the cart. Then they change their mind. Once the item is in their cart, another flurry of activity occurs. The almostbuyer begins to mull the decision over, decides to check the return policy, shipping charges and how quickly they’ll receive the item. Some of this activity may require them to leave the shopping cart pages. But is that an exit from the funnel? Not necessarily. Non-linearity is the foundation on which the web was built. It allows people to freely click from page to page within a web site—or to actually head over to a completely different web site—in any order they see fit. Reigning in useable data requires a visualization method that acknowledges a basic flow forward through the funnel from stage to stage, while correctly showing that users click where they want to. Exacerbating this flaw is the fact that different groups of visitors behave within the funnel in radically different ways. Visitors from a PPC campaign have no relationship with you yet; those from an e-mail campaign probably do. The behavior here is so polarized that you can’t extract meaningful data from a funnel unless you can view and compare those visitor segments, side by side. 6 Toll-Free: (877) 773-2249 ClickTracks Analytics, Inc. 101 Cooper Street Santa Cruz, CA 95060 VISUAL WEB ANALYTICS A Better Funnel Design An effectively designed funnel report solves traditional funnel flaws in three ways. First, it provides a visualization of which pages encourage forward motion into the funnel, without attempting to show the order of those pages within a particular funnel stage. This is done by calculating which pages are ‘persuasive,’ meaning that when visitors see that particular page, they are more likely to advance to the next stage of the funnel. 7 Toll-Free: (877) 773-2249 ClickTracks Analytics, Inc. 101 Cooper Street Santa Cruz, CA 95060 VISUAL WEB ANALYTICS This visualization more closely matches the underlying behavior: The user clicks around seemingly at random, but when viewing a statistically large enough sample, we can see that some pages are more persuasive than others. The ClickTracks funnel report adds a darker shading to those pages the program finds more persuasive than others, visually cueing the marketer to interesting information. The second part of the solution is to view your site as page groups rather than individual pages, using a metaphor that collapses many individual pages into larger chunks of data that are easier to understand. For example, all individual products in the product catalog might be collapsed into page groups including product overviews, details and specifications groups. Finally, the funnel process needs to be viewed for multiple segments simultaneously, side by side. Achieving this requires the overall visualization technique be simple, so that adding multiple segments doesn’t greatly complicate the data and make it impossible to derive meaningful information and insights. In the above funnel report snippet, the far left and far right columns show the funnel in the ‘traditional’ fashion: you see the number of visitors in the stage, and the number that exit. The stages are assumed to be ordered linearly. The individual page groups are shown in center column. The data reveals the degree to which each page group influences the user to advance to the next stage, i.e. the persuasiveness. Compared to other page groups, is a user more or less likely to become a customer? For example, imagine our site has a coupon page. We want to know if a visitor seeing the coupon page tends to improve conversion—but we don’t really care how people got to it, whether the coupon page was the first page they saw or the 13th page they visited right in the middle of the checkout process. More advanced analysis of the funnel can be performed by applying visitor labeling, just like any other ClickTracks report. The user behavior within the funnel will be very different depending on whether the users are coming from an e-mail campaign, from a paid search ad and so on. 8 Toll-Free: (877) 773-2249 ClickTracks Analytics, Inc. 101 Cooper Street Santa Cruz, CA 95060 VISUAL WEB ANALYTICS Sin #4: Data Overload Sometimes you really can have too much of a good thing. Too much data can lead to more than a slight headache—it can lead to analysis paralysis, a condition that occurs when you have more data than you know what to do with or need. Some web analytics vendors tout the sheer number of reports they have ... and that’s where analysis paralysis begins. Marketers must understand that it’s not the quantity of reports that matters, but the quality of the data they provide. Don’t waste valuable time trying to decide which report to use or looking at data that may be interesting, but doesn’t lead to any particular action. Making Business and Marketing Decisions In the early days of the Web, when ad impressions were the predominant business model, the key metrics of success were traffic and page views. During this time, anybody with an idea for an online offering that generated traffic could get funding. Thankfully, those days of excess are long gone, replaced by strong business models and sound marketing decisions based on data derived from your web site. The key to successful web analytics is the ability to make marketing decisions based on the data you’re able to mine—but the data alone is not enough to make you successful. A tool that provides 100 reports isn’t necessarily inferior to one that provides 200 reports or more. It just means that one tool may allow you to work smarter rather than harder. Determining Which Data Matters—And Taking Action So how do we narrow our focus, letting us closely examine what’s important while ignoring the background noise? First, take each report and map it to a business need by asking yourself the question, “What aspect of my online presence will I change with the information I learn here?” If you can’t answer this question, whether it be because you can’t affect change or you don’t know what to change, don’t be bothered with the data. For example, knowing the time-of-day that your site traffic peaks is rarely useful, because you simply can’t react fast enough to take advantage of the data. Knowing that traffic peaks at 2pm EST assumes you’ll be waiting by your PC each day, waiting to flip a switch that changes something on your site, thereby taking more advantage of that traffic. You’ll find that your time is better spent on bigger problems. Next, try to get a handle on exactly what the report is measuring. For example, seeing referrals from Google could mean more organic search results (usually a good thing) or more paid search results (not as good as organic listings since you paid for them). When you see the number in the report, think how it’s computed and therefore what’s implied. In the Google example above, you’d probably want paid and organic search to be shown distinctly, since it’s possible for one to rise while the other falls. Ironically this means more data, not less—but it is data that can be used for making a decision, so it makes sense to include it. Finally, try and find ways to explore your data dynamically. Even with a starting point of 200 preconfigured reports, you’ll have difficulty finding insights that relate to your specific situation. For example, while site page views aren’t necessarily valuable, knowing site page views broken out by keyword or some other dimension may yield more usable data. If you think measuring page views can lead to useful decisions about your site, start slicing the data and examine that metric from multiple angles. The analysis paralysis will abate, revealing previously unseen data delights. 9 Toll-Free: (877) 773-2249 ClickTracks Analytics, Inc. 101 Cooper Street Santa Cruz, CA 95060 VISUAL WEB ANALYTICS Sin #5: Relying on Absolute Numbers The numbers that indicate success for your site may be a sign of failure for someone else’s site. For example, if your site sells a high-end luxury product online, your conversion percent may be miniscule (.5%) while your time spent on the site may be very lengthy (18 minutes). The same numbers that spell success for you would spell lean times (and something very wrong) for a site that sells $5 products. More important than absolute numbers are the differences in absolute numbers. As those numbers rise and fall, they create trends, and it’s the trends—not the absolute numbers—that hold the key to better understanding how your web site is performing. Refine and Improve Your Online Presence The biggest factor that influences whether your site improves is your ability to refine and improve your online presence. But don’t let the ‘trends’ fool you into thinking that you can control user behavior—that when the line goes up you’re doing well, and when it goes down you’re not. Sadly, the line for most metrics moves up and down seemingly at random. Your site is subject to huge external forces that make the graphs move around erratically. The fact is, if you have a web site, your basic visitor stats are likely to trend up and to the right, no matter what you do. Internet usage grows and inevitably, your site grows with it. What you can’t tell is how much better you could’ve done if you’d taken action to improve the site, campaigns, usability etc. Making such improvements usually results in a noticeable jump in stats, rather than a trend. Trends are Worth Tracking While web analytics software makes it easy to view certain trends, like the number of visitors who came to the site within a certain time period, that data is more interesting than it is useful. Why? 1. Real world events: Time intervals chosen by the software don’t line up with “real world” events. Analyzing the past month of data gives you an idea of how things have been going for the past thirty days, but doesn’t take into consideration the e-mail campaign that was sent mid-month or the site redesign that occurred 60 days ago. Activity disconnect: The software shows numbers, but not necessarily the activities that led to those numbers. If your site experiences a large spike in traffic (due to being featured on a popular blog) or a decrease in visitors (due to technical difficulties on the part of one of your largest affiliates) the software can only show you the absolute numbers—not the reasons why it occurred. Line graph limitations: Something that’s a big deal in a conversion rate (a jump from 2.07% to 2.76%) can’t be accurately depicted as a line graph on a computer screen. These sorts of subtle, yet important changes need to be expressed as numbers in order to have meaning and allow you to take appropriate action. 2. 3. 10 Toll-Free: (877) 773-2249 ClickTracks Analytics, Inc. 101 Cooper Street Santa Cruz, CA 95060 VISUAL WEB ANALYTICS To reiterate: A monthly visitor report doesn’t take into account the activities that occurred during that month that potentially had an impact on the numbers. Simply viewing the month data by day can also be misleading—because of the way traffic trails off at the weekend (or rises, for B2C sites). Set Appropriate Time Parameters A more accurate way to gauge trends over time is to take control of the time periods your software is comparing. Instead of letting the software prescribe a 30-day or 6-month time period, set your own custom dates based on events and activities occurring as part of your marketing plan. For example: If you did a large direct mail campaign on the 18th of June, compare the 20 days before June 18th with the 20 days after. This will give you an idea of the success (or failure) of your efforts in a way that a pre-canned trend could never accurately convey. The best way to determine improvements in ROI, unique visitors or any other of your key metrics is to label two adjacent time periods—one before a new event or initiative is launched, and one after. By doing this, you’ll be able to weed out any ‘wild-goose-chase’ trends and focus on whether tangible improvements are taking place. 11 Toll-Free: (877) 773-2249 ClickTracks Analytics, Inc. 101 Cooper Street Santa Cruz, CA 95060 VISUAL WEB ANALYTICS Sin #6: Relying on Top 10 Lists All web analytics tools provide a variety of ‘top n’ reports such as the top pages, top referrers. For many of us, they’re the first thing we look at—especially the referrers. Since referrers are basically sources of visitors, it’s interesting and somewhat exciting to see which sites are sending you traffic. The problem with these reports is they inevitably display just the most popular few, and not the interesting stuff that lives beyond the head, in the long tail of the data. Moving Past the Top 10 Take a look at your top 10 referrers for your site. They probably haven’t changed for many months, perhaps remaining in the same ranked order. If you compare the content of this report across different segments you can gain more value, but unless you’re really on top of your analysis you won’t do this more than once or twice a month. Of course you can examine much more than the top 10 referrers. A quick visit to the settings or options allows you to expand your referrer list to show the top 10,000 referrers. The key is to find a happy medium where you have enough data to see the big picture, without having so much data that analysis becomes unwieldy. For Example: Your Site Gets Prominent Blog Mention Suppose your site just got mentioned in a blog. You get a small bump in visitors (e.g. referrers) from the blog site, and those visitors are highly qualified. Almost by definition any blog that mentions you will bring highly targeted, qualified traffic. The problem is that it doesn’t bring a large volume of traffic, because it’s so narrowly focused. Furthermore, the traffic peaks shortly after the blog is published and then quickly falls off as other articles replace it. If you look at your top referrers for the month, this blog referrer wouldn’t even make it to the top 100. There was a small number of highly qualified visitors, but they’re unfortunately silent among the larger noise of the data. Ideally, your analytics software will tell you what’s changing from time period to time period. Imagine the analytics software is keeping track of millions of data points, and showing you the top 100 because that’s about all you can keep track of. It’s not good enough to know about changes within this top 100. You may need to know that something has gone from rank 200,000 up to rank 1,000, because that’s a huge jump—but it would be difficult, if not impossible, to work that out manually. Please note: The What’s Changed report in ClickTracks was designed for exactly the situation described above. It calculates statistically significant changes over time and alerts the user to emerging trends and unusual activity. 12 Toll-Free: (877) 773-2249 ClickTracks Analytics, Inc. 101 Cooper Street Santa Cruz, CA 95060 VISUAL WEB ANALYTICS Sin# 7: Technicolor Report Presenting data for the purpose of persuading others within your organization to take action is probably one of the hardest tasks within the world of web analytics. Sure, you can segment, compare and calculate ROI, but when it comes time to persuade the rest of the team to fix that non-performing campaign, go back to an older and better performing version of the web site that nobody internally likes or change the shopping cart process, you’re faced with the daunting task of proving the why and how. It’s What You Say AND How You Say It Your ability to persuade largely depends on the visual presentation of rational data in an easy-to-understand format. But most web analytics tools don’t make your job easy—rather, in a feeble attempt to make the data look slick, they miss the mark by making it unnecessarily hard to understand. It comes down to this: Data must be clear first, attractive second. The most common culprit of unclear data presentation is the 3D pie chart: Can you easily tell which the second largest segment is without having to read the numbers? The red segment seems the biggest because it’s the most prominent color, and more significantly, it has an additional block of color visible on the leading edge because of the 3D effect. To figure out which is the second largest, our brain needs to add up and compare the surface area of each slice of the pie and then conclude which is the second largest slice. We have to work extra hard to subtract the 3-dimensional area—and even then, the chart is STILL hard to interpret because of the distortion introduced by the perspective. For example, segment R6 appears to be half the size of R4, but it isn’t. Let the Numbers Speak for Themselves In the end, using 3D adornments on graphs only muddies the waters. Keep your goal of successfully bringing about change based on the facts. To do this, you need to convey the facts clearly and concisely, not necessarily in a 3D rainbow of colors. Data Display Principles During the design and development process at ClickTracks, we’re constantly generating new ideas that require unique ways to show data. We’ve gradually come up with a set of data display principles: 1. 2. Show segments: All reports should be segmentable, with the same metric visible across all segments simultaneously, on the same screen. Comparing segments is the most important thing. Careful with colors: Using colors to represent a value is a mistake. For example ‘yellow’ is not higher, lower, better or worse than ‘green’. Our brains see different colors as having different meanings (red = warning) but not different values. On the other hand using shades of the same color does convey values. Stop using 3D: These 3D adornments do nothing but distract. 13 3. Toll-Free: (877) 773-2249 ClickTracks Analytics, Inc. 101 Cooper Street Santa Cruz, CA 95060 VISUAL WEB ANALYTICS About ClickTracks Based in Santa Cruz, California, ClickTracks is the maker of award-winning web analytics software. The ClickTracks family of software was designed out of sheer frustration with existing web analytics offerings. Spending hour upon hour trying to understand web site visitors was certainly a motivating experience: there had to be a better way. ClickTracks was voted ‘Best Web Site Analysis Tool’ in ClickZ’s 2005, 2004 and 2003 Marketing Excellence Awards, 2006 Product of The Year by Small Business Technology Magazine and was rated “Positive” in Gartner’s 2006 MarketScope for Web Analytics. The company received a “Very Good” top rating from the InfoWorld Test Center for its 6.1 product suite, and a Computerworld Innovative Technology award in the web site management category. ClickTracks’ software has been reviewed and featured on CNET, ZDNet, Builder.com, TechRepublic, Internet.com, CRM Magazine, Media Magazine, MarketingSherpa and About.com. ClickTracks is an indirect wholly-owned subsidiary of J.L. Halsey Corporation (OTCBB:JLHY). For more information, please visit www.clicktracks.com or send e-mail to info@clicktracks.com. 14 Toll-Free: (877) 773-2249 ClickTracks Analytics, Inc. 101 Cooper Street Santa Cruz, CA 95060