Insurance Complaints by leader6



      Richard L. Morris, College of Business, Winthrop University, Rock Hill, SC 29733,
                           (803) 323-2684,,
       Glenn L. Wood, College of Business, Winthrop University, Rock Hill, SC 29733,
                            (803) 323-2599,


Evaluating the quality of service from a prospective insurer is a formidable challenge—even for
the most experienced financial advisors. One approach to evaluating “service” is to assess the
insurer’s complaint ratio. Using data that are not readily available, this study describes the
complaint ratios for six basic lines of insurance and analyzes the unique problems inherent in the
complaint ratios for small insurers. In addition, the authors analyze the relationship between
insurer size and complaint ratios, and investigate the question of whether “bad” complaint ratios
tend to be followed by improved ratios.


The purpose of this paper is to perform a “macro” (aggregate) analysis of complaint data for six
lines of business for the years 2004-2006. Unless otherwise noted, all the data for this paper
were provided by material supplied by the National Association of Insurance Commissioners
(NAIC) apart from information available from the Internet website [1]. The lines of business
under scrutiny are:

       1.   Private passenger auto
       2.   Homeowner’s
       3.   Group life
       4.   Individual life
       5.   Individual accident and health, and
       6.   Group accident and health.

More specifically, we will:

       1. describe the complaint ratios for the above six lines of business,
       2. analyze the relationship between insurer size and complaint ratios [2],
       3. investigate the question of whether “bad” complaint ratios tend to be followed by
          improved ratios, and
       4. determine whether there is a relation between complaint ratios of companies
          operating in similar lines of business.

There is no intention of analyzing the statistical problems inherent in the data as this has been
done previously [3], but we will, by necessity, address a few of the major questions that arise in
the interpretation of the information provided.
                             THE SMALL COMPANY PROBLEM

For purposes of this paper complaint ratios are calculated as follows:

                    # of complaints
Complaint Ratio= Premium Volume x1,000,000                                              (1)

In analyzing premium volume data, it becomes evident that large insurers write most of the
business in every line and many, many small companies compete for the remainder. This market
fact raises a very important problem with the interpretation of complaint ratio data for small
companies. Specifically, the complaint ratios for small companies can be extremely misleading;
they can make small companies look much better or much worse than they really are.

Consider the following simple example for two companies in the same line of business:

                                       Company A       Company B
       Expected Number
       Of Complaints                       800               2

       Premium Volume                 $5,000,000,000   $12,500,000

The term “Expected Number of Complaints” is the number of complaints we would expect from
each firm if it is operating as usual. An equivalent definition is that it is the long-run average
number of complaints that each firm would incur if it is operating as usual. Company A’s
expected complaint ratio is then:

Expected Complaint Ratio =                   x1,000,000 = .16

and Company B’s is:

Expected Complaint Ratio =                x1,000,000 = .16

So both companies are operating at a similar level of service as measured by their expected
complaint ratios. However, it is very easy to see that Company B, just by chance variation, could
have zero complaints in a year, resulting in a complaint ratio of 0. Likewise, if it has six
complaints in a year, just by chance, it would have a complaint ratio of .48. Using the Poison
distribution we can model the distribution of complaints for both companies. Using this
approach, the probability of zero complaints for Company B is about 13.5%. The probability of
having six or more complaints in a year is about 1.7%.

We can use a reverse analysis for Company A to get the same probabilities. The probability of
Company A’s having 769 or fewer complaints is about 13.7%. This would yield a complaint
ratio of .154. The probability of Company A’s having 861 or more complaints is about 1.7%,
with a complaint ratio of .172.
Looking at this example in another way, the probability of each company operating in its range
above is about 85% [100 - (13.5 + 1.7) = 84.8% for Company B; 100 - (13.7 + 1.7) = 84.6% for
Company A]. So Company B’s complaint ratio will lie within the range 0 to .48 while Company
A’s will be in the much smaller range of .154 to .172, both with equivalent probabilities.

 The problems of scale in evaluating complaint ratios are very serious. The following table
illustrates average complaint ratios and the variability of the ratios as premium volume increases:


 Private Passenger                            Homeowner’s
 Quintile      x            s                 Quintile    x              s
     1       0.130        0.014                  1     0.125           0.022
     2       0.251        0.048                  2     0.150           0.030
     3       0.332        0.072                  3     0.234           0.098
     4       0.312        0.090                  4     0.250           0.107
     5       0.514        0.338                  5     0.524           0.365

 Group Life                                   Individual Life
 Quintile       x           s                 Quintile      x            s
    1         0.013       0.004                  1       0.032         0.004
    2         0.023       0.003                  2       0.038         0.006
    3         0.027       0.006                  3       0.042         0.009
    4         0.023       0.007                  4       0.076         0.015
    5         0.289       0.254                  5       0.269         0.170

 Individual Accident and Health               Group Accident and Health
 Quintile       x          s                  Quintile     x          s
     1       0.090       0.003                   1      0.081       0.026
     2       0.231       0.154                   2      0.097       0.029
     3       0.226       0.161                   3      0.079       0.023
     4       0.248       0.074                   4      0.117       0.048
     5       0.687       0.507                   5      0.182       0.106

This table shows the mean and standard deviation of the complaint ratio by quintile of premium
volume for each line of business. In the table, the simple arithmetic average of each measure is
calculated for the first through fifth quintile in each line of business. From Table 1 we see, for
every line of business, a very strong tendency for the average complaint ratio to increase as
premium volume decreases. This relationship is clear and almost perfectly consistent. A
reasonable, or at least possible, explanation is that the larger companies do a better job of
providing higher-quality service to their customers.
The greater variability in complaint ratio for smaller companies is likely due to the effects of a
small absolute number of complaints for smaller companies versus larger absolute numbers of
complaints for larger ones. After a fair amount of analysis we concluded that there is no “magic”
cutoff size, i.e., a company size where the number of complaints becomes much more (or less)
meaningful. To address the issue further we determined the number of zero complaint ratios in
each quintile. We found that they were highly concentrated in the fifth quintile for each line of
business with just a smattering of them in the fourth quintile.

In recognition of the above problems some states list only the companies with a number of
complaints that exceed a specified minimum, such as ten. Other states do not list the complaint
ratios of companies that have a premium volume less than a certain minimum. Unfortunately
there is no method that solves the small company statistical problem without introducing
additional problems. There is no “natural” dividing line between “large” and “small” companies
in any line of insurance. This means any classification by size will be somewhat arbitrary and
subject to criticism. Nonetheless, financial advisors (and consumers) who use complaint ratios
for very small companies should understand that these ratios have little or no meaning.


An insurer might be concerned if its complaint ratio increases substantially from year to year or
is “too high” compared to comparable size companies. In other words, a company might be
concerned if the complaint ratio has increased, or the company might be concerned if the level of
complaints is viewed as unacceptable. That is, a “bad” complaint ratio might be viewed as one
that is increasing or it could be defined as one that is higher than some standard set by the
company. Consequently, in examining the question of whether companies tend to improve after
“bad” ratios, we tested both concepts.

To test the idea that a company might take steps to improve its complaint ratio after experiencing
a “bad” ratio, we defined a “bad” ratio as one that had increased by 5% or more from the
previous year. With three years of data, we arbitrarily decided that if a company’s complaint
ratio increased by 5% from 2004 to 2005, then 2005 would be labeled a bad year. Then we
determined whether or not those companies with a bad year in 2005 improved their complaint
ratio in 2006. In the Private Passenger line of business, for example, 67% of the companies
experiencing a bad year in 2005 improved their complaint ratios in 2006, suggesting that there
may be a pattern of improvement after a bad year.

To determine if there was a significantly higher proportion of firms with bad complaint ratios in
2005 that improved versus those that did not have bad complaint ratios in 2005 we performed a
test of two proportions. The hypotheses we tested were:

       Ho: The proportion of companies showing improvement from 2005 to 2006 is the same
           regardless of whether they experienced bad complaint ratios in 2005.
       Ha: The proportion showing improvement is higher for those having bad complaint ratios
           in 2005.
We performed this test on each line of business using a z-test on proportions. Because of the high
variability inherent in very low numbers of complaints, we based the analysis only on companies
whose number of complaints for each year was greater than ten. The results are shown in Table
2. As shown in the table, the z-test statistic for the Private Passenger line of business is 1.785,
with a p-value of .037. The null hypothesis will be rejected if the p-value is less than the
significance level, . If we choose the customary .05 , we would reject the null and can
conclude that there was a significant improvement effect in the Private Passenger line.


                                                 # of
                         #of Bad                 CR's
                         CR's in Proportion        in     Proportion   z-
 Line of Business         2005   Improving       2005     Improving statistic       p-value
 Private Passenger         67      67.2%          249       55.0%    1.785          0.037*
 Homeowner's               25      76.0%          109       53.2%    2.079          0.019*
 Group Life                 3      33.3%            8       50.0%    -0.494          0.689
 Individual Life           35      65.7%           71       47.9%    1.731          0.042*
 Individual Accident
 and Health                 39        56.4%        95       53.7%        0.288      0.387
 Group Accident and
 Health                     64        65.6%       137       59.1%       0.881        0.189
                                                                 Significant at the .05 level
                                                                 Significant at the .01 level

Note that the sample sizes for Group Life insurance are too small to draw any meaningful
conclusions. This is true of the remaining tables in this section also. Therefore, we will eliminate
this line of business from any further consideration.

In Table 2, the second column lists the number of bad complaint ratios for 2005 in each line of
business. The next column is the proportion of those with bad complaint ratios that improve from
2005 to 2006. The next two columns list the number of firms with good complaint ratios in 2005
and the proportion of those that improved from 2005 to 2006. These firms act as a control group
we can compare to those with bad ratios. From this table, it can be seen that while the proportion
improving was higher for those companies having bad complaint ratios in 2005 in five of the
lines of business, only three, Private Passenger, Homeowner’s and Individual Life, had
significantly higher proportions at the 5% level of significance.

Taking the other approach, we reasoned that some companies might look at other similar
companies as a means of deciding whether their complaint ratios need improvement. A very
rough way of doing this is to look at whether a company’s complaint ratio is high relative to the
mean of the quintile it is in. We used the same 5% figure as in the previous analysis. That is, if a
firm’s complaint ratio is 5% higher than its quintile average, then it would be classified as a bad
ratio. We then determined the proportion of those with bad complaint ratios that improved and
compared this to the proportion of those companies not classified as having bad ratios that
improved. Table 3 shows the proportions of companies in both categories that improved from
2004 to 2005 and Table 4 shows the same information for improvements from 2005 to 2006. As
before, we eliminated from consideration those companies with ten or fewer complaints in any of
the three years considered.

                        TABLE 3. IMPROVEMENTS RELATIVE TO
                       SIMILAR SIZE COMPANIES (2004 AND 2005)

                                               # of
                        #of Bad                CR's
                        CR's in   Proportion     in    Proportion   z-
 Line of Business        2004     Improving    2004    Improving statistic       p-value
 Private Passenger        157       86.0%       159      61.0%    5.026          0.000**
 Homeowner's               76       84.2%        58      67.2%    2.308          0.010**
 Group Life                 5       80.0%         6      66.7%    0.494           0.311
 Individual Life           54       64.8%        52      51.9%    1.347           0.089
 Individual Accident
 and Health              113        68.1%       21       52.4%       1.397        0.081
 Group Accident
 and Health              100        69.0%       101      58.4%       1.560        0.059
                                                                Significant at the .05 level
                                                                Significant at the .01 level

                        TABLE 4. IMPROVEMENTS RELATIVE TO
                       SIMILAR SIZE COMPANIES (2005 AND 2006)

                                               # of
                       #of Bad                 CR's
                       CR's in    Proportion     in    Proportion   z-
  Line of Business      2005      Improving    2005    Improving statistic        p-value
 Private Passenger       124        67.7%       192      51.0%    2.933           0.002**
 Homeowner's              58        67.2%        76      50.0%    2.000           0.023*
 Group Life                3        33.3%         8      50.0%    -0.494           0.689
 Individual Life          52        65.4%        54      42.6%    2.353           0.009**
 Accident and
 Health                 112        58.0%        22       36.4%       1.866        0.031*
 Group Accident
 and Health              93        72.0%        108      51.9%       2.929        0.002**
                                                                 Significant at the .05 level
                                                                 Significant at the .01 level
Table 2, as discussed previously, suggests that companies try to improve after a bad year, but the
results are not particularly dramatic. Tables 3 and 4 give stronger results. In Table 3, only two
lines of business yield significant results, but the rest produce p-values fairly close to the .05
level of significance. Table 4 produces significant results (at the .05 level) in all of the five lines
of business after excluding Group Life insurance. Three of these five are significant at the .01

Do the results tend to support the idea that companies try to improve after experiencing bad
complaint ratios? The preceding analysis suggests that they do. We cannot say with certainty that
they use the complaint ratios themselves to decide if they receive too many complaints; they may
use more informal measurements or they may use different criteria than we used for detecting
improvements. Nevertheless, the results strongly suggest that insurers are concerned about
getting too many complaints and that they do indeed take steps to improve bad complaint


We are also interested in seeing if complaint ratios are correlated between different lines of
business. To analyze this question, we calculated the Spearman rank correlation coefficient (rS)
for Personal Passenger and Homeowners’ insurance for each of the years for which we have
data, shown in the next table:

                       TABLE 5. CORRELATIONS FOR PERSONAL

   Year      2006            2005           2004
    rS       0.496           0.593          0.582
   p-value 2.24E-08        4.44E-12       1.41E-12

So, for example, the correlation relating complaint ratios of Personal Passenger insurance and
Homeowners’ insurance is .593 for 2005. This bears a little explanation. To begin with, the
correlation coefficient itself lacks a reasonably good interpretation. However, its statistical
companion, the coefficient of determination, or rs2, can be interpreted as the amount of
variability in one variable that can be “explained” or accounted for, by the other. The rs2 for 2005
is .352 (.5932). So 35.2% of the variability in the complaint ratios for Personal Passenger
insurance can be “explained” by the variation in complaint ratios for Homeowners’ insurance
(and vice-versa; the relationship holds in both directions).
It might help also to show a graph of the complaint ratios for the two lines of business for 2005:

 Personal Passenger Ranking

                                    0   20   40     60        80   100   120
                                              Homeowner Ranking

From this it can be seen that the correlation between the two isn’t that dramatic. The third row of
Table 5 contains the p-values of the coefficients. The p-values are extremely low, indicating that
the correlations are highly significant, but this doesn’t mean that they’re particularly meaningful,
just that there’s a relationship (maybe a small one) that can be detected in a statistical test of
hypotheses. So if we’re trying to answer the question of whether or not companies tend to have
comparable complaint ratios in separate lines of business, the answer is “Yes, but it’s not a
particularly close relationship.”


Complaint ratios vary greatly by line of business. Complaints in property and liability insurance
are consistently much higher than in life insurance. This might be explained by the fact that
most complaints arise from the handling of claims, with underwriting, policyholder service, and
marketing and sales all together accounting for a minority of all complaints.

All insurance markets are dominated by large insurers, and the analysis of complaint ratios for
small companies is very difficult because of statistical problems. With a small premium volume,
small fluctuations in the absolute number of claims will cause the complaint ratio to fluctuate
wildly. Accordingly, the complaint ratios of very small companies are essentially meaningless.

The analysis supports the conclusion that larger companies have lower complaint ratios than
smaller companies. This was true for every line of business in every year of our analysis.

Using two different definitions of “bad” complaint ratios, we found statistically significant
results at the .05 level in all lines of business (not including Group Life insurance, which had too
few complaints to analyze). In three of the five lines of business the results were significant at
the .01 level. This is strong evidence that insurance companies are concerned about their
complaint ratios and it seems probable that they take steps to improve their service if complaints
We also looked at the correlations between lines of business and found that there was a
significant correlation in rankings of complaint ratios for companies engaged in different lines of
business. However, while significant, the correlations were not particularly meaningful.


[1] National Association of Insurance Commissioners (NAIC) Consumer Information Source
 [2] Query, J. T., Hoyt, R, E, & He, M. Service quality in private passenger automobile
     insurance. Journal of Insurance Issues,2007, 30(2), 155-165.
[3] Venezian, E. Complaint ratios: What (or where) is the beef? Journal of Insurance Regulation,
     2002, 20(4) 19-46.

To top