Online Price Comparison Websites A Mystery Shopping Survey Technical report and Management Summary for research conducted by Ipsos MORI for Moneysupermarket.com February/ March 2008 Technical Details Survey Methodology A mystery shopping study was designed in order to replicate the customer experience while shopping online for motor insurance quotes. This enabled independent, objective and consistent quotations to be obtained across the six online comparison websites, using a number of different scenarios. All enquiries were conducted at the end of February 2008. Scenarios 100 scenarios were designed by Ipsos MORI containing all the information required across all six comparison websites being tested in order to obtain a motor insurance quote. The only details not specified were name, address, telephone number and email address - mystery shoppers entered their own personal details in these fields. The majority of variables remained consistent with the key determinants of cost being varied across the scenarios. The make and model of the cars used in the exercise were selected at random, ensuring a spread of manufacturer, age and engine size. Included within the list were several speciality or premium cars and also the top selling eight cars currently in the UK. Five different driver ages were used; 18, 21, 35, 50 and 65 and the number of years licence held and no claims bonus was varied accordingly across these ages, ranging from less than one year, three years, six years, ten years or 20 years for licence held, and none, two, four, six, eight or ten for number of years of no claims bonus. Enquiries were carried out across 11 specified regions and within each region different postcodes were used at random. Occupation and industry sector were also varied and an even number used within each region. Each mystery shopper was given a scenario and was required to obtain one quote per comparison website. In total therefore 600 searches were carried out, 100 to each comparison website. The same scenario details were used for each online website to ensure comparable findings and consistency. For each scenario the six enquiries were carried out one after the other to ensure that any differentiation in the findings was not a result of timing variation in the market. The majority of enquiries were conducted Thursday to Sunday to avoid the busiest days of the week for the websites. Summary of Findings Background and Objectives Online comparison shopping is a fast growing and exciting sector with thousands of people using an expanding number of cost comparison websites each day, to compare and buy a wide range of products such as personal finance, broadband and holidays. Moneysupermarket.com are one of a number of organisations offering such a service in an increasingly complex and competitive marketplace. Research was required to evaluate how moneysupermarket.com perform compared to their key competitors, specifically focussing on online motor insurance quotes. The key objective of the study being to provide comparable findings which may be published by moneysupermarket.com, to determine which online comparison website is more likely to provide the best rates for customers. The study focussed on searching and comparing motor insurance quotes among the following leading comparison websites: Moneysupermarket.com TescoCompare uSwitch Comparethemarket.com Confused.com GoCompare In total 600 searches were carried out across the six websites; 100 enquiries per online comparison site. Key Findings Given the differing nature of quotes returned across each of the comparison websites i.e. many included a vast number of special offers, discounts and additional features such as free breakdown cover, courtesy cars etc, making an assessment on the full offer would be incredibly complex and may open the research up to question. Assessment was therefore made solely on annual premium returned. Overall, moneysupermarket.com performed well, returning the lowest, or equal lowest, annual car insurance quote in 42 enquiries out of 100 completed; this was well ahead of its nearest rival, TescoCompare which found the cheapest, or equal cheapest, quote in 22 out of the 100 tests. Moneysupermarket.com provided the lowest quote across a range of cars, petrol and diesel and both manual and automatic, from the latest models to cars registered in 1989. The findings across all the comparison websites are as follows: Number of scenarios where Comparison Website lowest annual premium returned 1. Moneysupermarket.com 42 2. TescoCompare 22 3. Confused.com 19 4. GoCompare 17 5. Comparethemarket 13 6. uSwitch 8 * please note these figures denote the number of occasions where the lowest or equal lowest number of quotes was returned As can be noted uSwitch only returned the cheapest or equal cheapest annual premium in 8 out of the 100 of the tests, with Confused.com, GoCompare and Comparethemarket all providing the lowest or equal lowest annual car insurance quote in less than 20% of enquiries. In addition to the differences highlighted with the level of annual premium returned, variations were also noted in the ease of use of some of the sites, with moneysupermarket performing well for ease of use and navigation in comparison to its key competitors on several occasions: “The moneysupermarket.com and the confused.com were the quickest and easiest sites to navigate.” 26 February 2008 “Moneysupermarket and Tesco were the easiest to navigate.” 24 February 2008 Summary In summary, moneysupermarket.com performed well, returning the lowest, or equal lowest, annual premium in just over two-fifths of tests and was nearly twice as likely to offer cheaper annual car insurance than other leading online comparison websites in this exercise. Furthermore, the site was perceived well in terms of navigation and comments from those carrying out the enquiries would suggest that moneysupermarket is performing well for ease of use in comparison to some of its competitors.