Methodology by monkey6

VIEWS: 134 PAGES: 22

More Info
									                    Heuristics as a marketing research tool

Emilee Da Costa
Research Executive
Synovate SA

This paper examines the usefulness of heuristic decision rules in current day
marketing research. A Take-The-Best (TTB) decision rule is utilised with empirical
survey data to determine its ability to predict Most Often brand. The heuristic
decision rule is compared to a more traditional additive approach. The results
indicate that there is merit in further developing the use of heuristic decision rules
in marketing research. However, there are a number of difficulties associated with
the use of heuristic decision rules, and these difficulties would need to be addressed
before researchers are able to implement heuristics.

                                          PG 1
1.1 Introduction
There is a strong focus within the marketing research industry on driver analysis.
Researchers hope to identify what drives a person to use a specific brand. In order to do
this, a dependent variable is typically selected (such as purchase intention) and attributes
are statistically ranked in order to establish their effects on this dependent variable. This
methodology generally uses relatively complex statistical techniques to model human
decision making behaviour.

We as marketing researchers have realised however that this process does not provide
results that correlate with real world behaviour. Heuristic decision rules have been
identified as a potential tool for improving results in this field.

Heuristics are simple decision rules that are designed to help explain how people make
everyday decisions. They are not based on an exact science or statistical analysis. Rather,
they have been developed, mainly by researchers in the field of cognitive psychology, to
describe the decision making process.

There have been a great number of academic papers written on the plausibility and
validity of heuristics. Many of these papers use simulation studies to illustrate the validity
of heuristics in decision making. That they are a credible description of human decision
making is not disputed. However, the vast majority of these simulation studies focus on
the ability of heuristics to select an optimal brand or a brand with a high level of utility.
Very few studies have attempted to utilise heuristics empirically to asses their relevance
in a real-world setting.

This paper employs a Take-The-Best heuristic decision rule with empirical data to assess
its potential for use in current day research practises. The paper also provides an
overview of heuristics decision rules and discusses characteristics of various different

                                              PG 2
1.2 An overview / introduction to heuristics.
Heuristic comes from the Greek word heuriskein, meaning “to find”, also leading to the
word, eureka, meaning “I found it (out), (Hoffrage and Torsten, 2004, pg. 439). The term
heuristic has had a number of different meanings in the English language, but in the
1950s and 1960s, Herbert Simon and Allen Newell introduced the term to refer to
methods for finding solutions to problems. Simon and Newell were leading researchers in
the field of cognitive psychology and through their work, the term heuristic came to mean
a useful shortcut, or a rule of thumb for searching through a problem set (Hoffrage and
Torsten, 2004).

However, despite this early work on heuristics, in the 1960‟s statistical procedures such
as ANOVA, Bayesian methods and regression techniques all became popular research
tools. Researchers began to use these tools as models of cognition and soon after this,
people began to view cognitive processes as mere approximations of statistical
procedures (Goldstien & Gigerenzer, 2002, pg.75). Many researchers, attempting to
model human decision making, would start with a method that is considered optimal
(such as a regression technique or ANOVA), eliminate some aspects or calculations and
then propose that the human mind carries out this simplified or “naïve version” when
making decisions (Goldstien & Gigerenzer, 2002, pg.75).

Regression techniques, ANOVA and Bayesian methods were all considered “rational” or
“optimal” strategies and were thus widely used to analyse human decision making.
Today, we are beginning to think differently about the decision making process. After
disregarding heuristics for many years, researchers are now beginning to consider their
use in current day practise. The human mind does not work like a statistical calculator,
weighting up and comparing all information relevant to a decision. It has become
apparent that consumers are often faced with incomplete information and limited time
and that they actually make decisions in quite simplistic manners.

An alternative view to optimality is that of bounded rationality as proposed by Herbert
Simon back in the 1950s. Simon noted that it is usually too difficult for a person to

                                           PG 3
calculate an optimal strategy. The human mind is ill-equipped to identify all the brands
and attributes in a choice set, assign weights and scores, compute the weighted utility
from each alternative brand, compare the utilities and select an outcome. People are
forced to use simple ways to make decisions because they are unable to mentally
compute all the required calculations otherwise. Thus, Simon proposed that people
satisfice (search for “good enough” solutions) rather than optimize. With this in mind,
Simon noted that people are only partly rational when making decisions, and partly
emotional / irrational for the remaining part. This concept led to the term “bounded
rationality” being formed.

The theory of fast and frugal heuristics was largely founded on Simon‟s concept of
“bounded rationality” by someone named Gerd Gigerenzer. Gigerenzer is a leading
expert in the field of heuristics and has carried out a number of very noteworthy studies.

He noted the following in his book Simple Heuristics That Make Us Smart,
       “In an uncertain world, there is no optimal solution known for most interesting
       and urgent problems. When human behaviour fails to meet these Olympian
       expectations, many psychologists conclude that the mind is doomed to
       irrationality. These are the two dominant views today, and neither extreme of
       hyper-rationality or irrationality captures the essence of human reasoning. My aim
       is not so much to criticize the status quo, but rather to provide a viable
Gigerenzer proposed that an emphasis on speed and frugality provide an acceptable
illustration of human decision making (Newell et al, 2002). Fast and frugal heuristics are
based upon these concepts of simplicity and frugality. Thus, they hope to select an
acceptable alternative in a very short amount of time using limited computational skills.

Gigerenzer points out that fast and frugal heuristics are justified by their psychological
plausibility and adaptedness to natural environments. In other words, there are many
times in the real world when we are forced to make a decision in a short amount of time

                                            PG 4
and with scarce information, and it is in these situations, that using a fast and frugal
approach to decision making may be an excellent idea [Gigerenzer et al, 2001].

In the book “Bounded Rationality: The Adaptive Toolbox” by Gigerenzer and Reinhard
Selten, the hypothesis of an adaptive toolbox is presented. Gigerenzer & Selten argue
that the human mind is equipped with a number of simple decision rules, or heuristics,
which are all relevant for certain situations and problem settings. Thus, one type of
heuristic may be used when deciding which DVD player to purchase and another may be
used for deciding what to eat for lunch. According to this theory, no heuristic rule is a
general-purpose tool for all occasions, rather the human mind is equipped to be able to
adapt it‟s decision making process to the environment at hand.

                                             PG 5
1.3 Discussion of various Heuristics and common characteristics

It is important to note that there are three core characteristics of all heuristics:
        -   a search rule which describes how information is searched for
        -   a stopping rule which states when one should stop searching for information
        -   and a decision rule which explains how a decision should be made based on
            the information acquired. (Hoffrage & Reimer, 2004).

All heuristics are made up of these basic building blocks.

When investigating heuristic decision rules, it is customary to illustrate the choice set in
the following manner:

                   Attribute 1         Attribute 2         Attribute 3         Attribute 4
Brand 1            0                   0                   1                   1
Brand 2            1                   1                   0                   1
Brand 3            1                   1                   1                   1

With the attributes displayed as the columns and the alternatives, or brands, displayed as
the rows. The score for each brand on each attribute is then shown in the relevant section
of the table. This matrix like depiction of the data allows one to take the concept of a
specific heuristic decision rule and then calculate exactly which brand would be selected
if this rule was implemented.

There are other factors to consider when thinking about heuristics. Firstly, there are
decision rules that are known as compensatory rules and those that are known as non-
compensatory rules. Bettman and Payne (1988) noted that a compensatory strategy is one
that processes all of the available information and takes into account the good and bad
aspects of each alternative. A non-compensatory strategy on the other hand is one that
only considers some of the information at hand. This is done in order to reduce

                                              PG 6
information processing demands. Heuristics are non-compensatory decision rules that do
not use all possible information in the decision making process.

Another important aspect of decision strategies is whether they are alternative based or
attribute based strategies. In an alternative based strategy, the ratings for that alternative
on every attribute are read and processed before the strategy moves on to the next
alternative, each alternative in the choice set is processed in this way, i.e. one at a time. In
an attribute based strategy, the ratings for all alternatives on one attribute are read and
processed before the strategy moves on to considering the next attribute. Heuristic
decision rules can be attribute or alternative based, although there are many more
attribute based heuristics than there are alternative based heuristics.

Weighted Additive Rule
A weighted additive rule is a compensatory alternative based strategy which is considered
to be a baseline strategy in terms of accuracy and effort, i.e. it is able to provide the most
optimal decision (if no time constraint is applied) but uses a great amount of effort to
compute. WADD could be thought to be similar to a linear regression type technique
whereby all scores are weighted, summed and compared across the dataset.

Recognition Heuristic
The recognition heuristic is perhaps the most simple heuristic decision rule. It simply
states that if only one alternative or brand is recognised then that should be the one
chosen. No information is gathered and no comparisons are done. Many advertisers rely
on people using this type of heuristic when shopping. Consumers are faced with a
number of different brands, and they simply select the brand whose name they recognise.

Take the Best Heuristic
Newell et al. (2003), provide the following description of Take the Best:
Take the Best (TTB) is a fast and frugal decision rule whose guiding principle is “to take
the best and ignore the rest”. Unlike, many traditional “rational” decision strategies which

                                             PG 7
consider all possible information (e.g. linear regression), TTB simply uses the “best‟
piece of information available in a given situation.

TTB operates according to two principles. The first is the recognition principle which
states that if only one alternative is recognized then that should be the one chosen. The
second principle comes into play when more than one alternative is recognised. In this
situation, people are assumed to have access to a reference class of cues, or attributes,
which are subjectively ranked (weighted) according to their validity (or ability to identify
an optimal alternative). People are then thought to search through the cues (or attributes),
starting with the one with the highest weighting, until they find one that is able to
discriminate one alternative from the others.


Consider the following choice set with three brands (X, Y, Z) and four attributes,
displayed in descending order of importance.

                      Attribute 1        Attribute 2        Attribute 3        Attribute 4
Brand X                    1                    1                0                   1
Brand Y                    1                    0                0                   0
Brand Z                    0                    1                1                   1

In this situation, a person using TTB would begin by considering all three brands on
attribute 1. Brand Z would be dropped from the choice set as it does not meet the scores
of the other two brands on that attribute. The person would then move on to consider
Brand X and Brand Y on attribute 2. Only Brand X fulfils this criteria and Brand Y
would then be dropped from the choice set. At this stage the person has only one brand
left and this will be the chosen brand. None of the other attribute ratings are considered

                                            PG 8
To put this in a real world setting, imagine you walk into a store to purchase a beverage.
You know that you want a fizzy soft drink, and you decide that taste is the most
important feature to you. Second to taste is packaging, as you would prefer a can to a

You decide that you want a cola flavoured soft drink. There are a number of soft drinks to
choose from, but only two colas – Coca-Cola and a store brand label. You now have only
two products to choose from. There is a Coca-Cola can, but the store brand only comes in
a bottle. You pick up the Coca-Cola can and proceed to the checkout.

Elimination by Aspects
Elimination by Aspects (EBA) is another well known heuristic. It is a non-compensatory,
attribute based decision rule. It begins by determining the most important attribute and
the cut-off value for that attribute. The cut-off value for that attribute may simply be a yes
/ no value, i.e. a specific brand either does or does not have the stated attribute.
Alternatively, the cut-off value may be an actual threshold value, whereby the person has
a minimal level which is required for the brand to stay in the decision set. The cut-off
value indicates that the decision maker will accept all alternatives (brands) that have
scores above the cut-off with regards to that specific attribute. Thus, starting with the first
attribute, all alternatives with scores below the cut-off value are eliminated from the
choice set. This strategy continues from the first attribute onto the second attribute and so
forth, until only one brand remains [ Payne et al. 1988].

Satisficing heuristic
Another commonly used heuristic is the Satisficing rule (SAT). This rule considers one
alternative at a time, in the order that they occur in the choice set. Every attribute of an
alternative is compared to a pre-determined cut-off value and if the alternative‟s score for
a certain attribute is below the cut-off value then that attribute is eliminated from further
consideration. The first alternative that passes the cut-off values for all attributes is the
one that is chosen. This means that a choice can be made without the decision maker
considering information on all of the alternatives. If no alternative passes the cut-off

                                              PG 9
value on all of the attributes then a random choice is made [Payne et al. 1988]. Thus, the
SAT rule is also non-compensatory but it is alternative based unlike many of the other
heuristic rules that are attribute-based.

                                            PG 10
2. Methodology

This paper focused on an empirical investigation of the TTB rule. TTB, as previously
mentioned, will select the brand that performs “the best”, i.e. has the highest score
compared to other brands on a specific attribute. A modified version of a WADD rule
was used for the purposes of comparison with the TTB rule. A simple additive rule, with
no weights, was utilised to determine the most optimal brand (i.e. the brand with the
highest ratings) in each choice set.

A computer algorithm which incorporated the concept of TTB and WADD was written
and used to analyse the data. This algorithm was written in R, a statistical software
package. Four datasets were utilised, these datasets were based in the following
countries and categories: Toothpaste category in the United Kingdom, Toothpaste
category in the United States, Laundry category in the United Kingdom and the
Laundry category in Spain. The overall sample size for all four datasets was 1145

Within each dataset was a set of attribute ratings and a number of brand choice
indicators, such as „last brand bought‟ and „most often brand‟. For the purposes of this
paper, Most Often brand was selected as a variable with which to measure TTB‟s
predictive abilities, i.e. the computer algorithm will use a TTB decision rule to predict
which brand a respondent will select; this selection will then be compared to the data
collected for Most Often brand. Note that this analysis is done on a respondent level as
opposed to an overall level.

The question used for Most Often brand was:

Q.   Which ONE of the following brands do you buy most often?

The question used for the laundry attribute ratings was (with a very similar version used
in the Toothpaste category):

                                           PG 11
Q.   Displayed is a list of statements that may or may not be used to describe each
     laundry detergent brand. Please read each one and select the brands displayed, that
     you think best fits the statement. You can choose one brand, more than one brand
     or choose NONE or even DON‟T KNOW.

           1. High quality
           2. Trusted
           3. Value for money
           4. The brand for me
           5. Priced just right
           6. Is becoming more popular
           7. People say good things about it
           8. Has packaging that stands out
           9. Gets clothes really clean
           10. Has a nice fragrance
           11. Gentle on my clothes

This type of question resulted in binary attribute ratings.

In the Laundry datasets there were 11 brands in the Spain dataset and 10 brands in the
UK dataset. The Toothpaste datasets consisted of 17 brands for the UK dataset and 9
brands for the US dataset.

When analysing data with heuristics there are a number of factors that one must
address. These factors result in difficulties that can all be approached in different
manners. The next section discusses these problem areas.

2.1 Attributes Order of Entry
One of the key difficulties when working with heuristics is the order of entry of the
attributes or importance weightings. In real life, people naturally attach a level of
importance to certain attributes, for example a person may consider “taste” to have a far
greater importance than “packaging” when selecting a beverage. But, the very nature of

                                            PG 12
a heuristic decision rule is to only focus on some of the information at hand and if the
packaging attribute is considered first, a person may end up with a beverage that has
fantastic packaging but poor taste.

To illustrate, the problem setting could be shown like so:
                       Packaging          Trusted Brand        Taste
Brand X                1                  1                    0
Brand Y                0                  1                    1

In this situation TTB will start by looking at Packaging and select Brand X. At this
point there is only one brand left and no further attributes will be considered, Brand X
will be the chosen brand.

But, if the problem setting were shown like this:
                       Taste              Trusted Brand        Packaging
Brand X                0                  1                    1
Brand Y                1                  1                    0

Now it is clear to see that TTB will select Brand Y as Brand Y has the highest score on
the Taste attribute, and after Brand X is dropped there are no further brands in the
choice set.

Thus, for marketing researchers using TTB to analyse data, it is critical to decide which
attribute one should start with. In an ideal world one would be able to calculate the
importance of each attribute for each respondent and use this information to modify the
order of the attributes for each respondent. A practical manner in which one might be
able to achieve this is through a stated importance question being included in the
questionnaire, i.e. each respondent is asked the attribute ratings question and is then also
asked how important each attribute is to them.

                                              PG 13
If this type of question was not included in the questionnaire (and it was not included in
the datasets utilised for this paper) then alternative methods need to be considered.

For the purposes of this study, the importance of each attribute in relation to Most Often
brand was calculated. This was done using an unweighted additive rule and by
excluding each attribute one at a time to see the effects on overall predictability of Most
Often brand. The attributes were then ordered according to their relative importance.
Note that this approach resulted in an overall ordering of attributes as opposed to a
respondent level ordering of attributes (which would be more ideal).

This is not an ideal method for weighting attributes; it is simply one approach that was
viable for the purposes of this study. If marketing researchers are to use heuristic
decision rules with any level of value, then this is a key area which will need to be

2.2 Ties
There are many cases in marketing research when respondents provide exactly the same
attribute ratings for two or more brands. This results in a tied situation where neither a
traditional WADD type rule or a heuristic decision rule, such as TTB, will be able to
select a brand. The choice set would be whittled down to the few brands with identical
attribute ratings. At that point no type of decision rule would be able to select just one
brand, as none of the remaining brands would outperform any of the others based on the
information at hand.

For the purposes of this study, market share information was utilised to resolve ties. The
thinking behind this is that when a person provides tied attribute ratings, they are
indicating that they are indifferent towards selecting any of those brands, i.e. they would
be equally satisfied with any of them. Thus, when they are in a store environment and
are looking for that particular item, they will select whichever brand is presented to
them first, or is the only one available, or is perhaps on promotion. This is usually the
brand with the larger market share. Thus, in the situation where two or more brands

                                            PG 14
were tied, the computer algorithm was set to select the remaining brand with the larger
estimated market share. The estimated market share per brand was available in the
dataset from previous analytical work that had been conducted.

2.3 Market effects
When considering the predictive power of heuristic decision rules, it is important to
consider what they can and cannot do. Heuristic decision rules are designed to explain
how people evaluate different choice sets and select a brand. They do not take into
account the myriad of market place effects that may influence which brand a person
actually purchases.

This study used Most Often Brand as a dependent variable. Yet we as researchers know
that the brand purchased most often can be highly influenced by factors such as
availability, family pressure, promotional activities and such like. Thus, one will only
ever be able to partially predict a variable like Most Often Brand with heuristic decision
rules like Take The Best.

2.4 Choice of Heuristic
Many researchers have been drawn to TTB perhaps because of its simplicity and ease of
use. But it is foolish to forget that that there are actually quite a large number of
different heuristic decision rules with varying characteristics and features. To focus only
on TTB is to ignore many other potential options.

A tricky problem arises when we attempt to analyse data with heuristics. A computer
algorithm that analyses everybody using TTB or EBA or SAT clearly ignores the
guiding principle of heuristics that proposes that people make decisions in different
ways. It is highly unlikely that there is one particular decision rule that is used by the
majority of people. Instead, people are likely to have a toolbox of decision rules (as
proposed by Gigerenzer) which they pick and choose from depending on the problem at
hand. The difficulty with trying to implement this theory is that it is very hard to know
which respondent uses which type of heuristic decision rule. In fact, this is almost

                                             PG 15
impossible to determine. Current academic research hopes to determine a number of
characteristics or personality traits that might be associated with the use of certain
heuristic decision rules. If this research is successful, then one would be able to segment
a database perhaps according to personality type and then use a different heuristic
decision rule for different segments of respondents. But, this is not yet possible, and for
the purposes of this study, a TTB rule was used for all of the respondents.

                                            PG 16
3. Results
This section reviews the results that were obtained from the analysis of the entire
dataset (with the Laundry dataset and Toothpaste dataset combined). For a more
detailed review of the results obtained within the individual datasets, please refer to
Appendix 1 and 2.

3.1 Overall Results
The overall dataset consisted of a combination of the laundry and toothpaste datasets,
with a total of 1145 observations.
                Most Often            TTB              WADD
N                   1145              1145              1145
Mean                5.84               5.79             4.89
Mode                6.0                6.0               6.0
Std. dev            3.45               3.31             3.19

The number of correct predictions (shown below) suggests that TTB performs better
than WADD when trying to predict Most Often brand.
                      TTB            WADD
No. correct           775              757
% correct             67.7             66.1
Correlation          0.624            0.595

The TTB rule has a correlation of 0.624 which indicates a relatively strong positive
linear relationship between the Most Often brand data and the TTB predictions.

Comparing the TTB model to the WADD rule, we can see that the WADD rule does not
perform as well as a TTB decision rule. The WADD rule has a lower number of correct
predictions and a lower correlation with Most Often brand. TTB does not vastly
outperform the WADD rule, but the results are comparatively more favourable for TTB.

                                              PG 17
Researchers need to find new methods for analysing human decision making if we are
ever to improve predictions. Statistical tools such as ANOVA and regression have not
provided the answers that we need. Research has shown that the human mind is ill
equipped to behave like a statistical calculator and for this reason, heuristic decision
rules may prove to be an excellent analytical tool.

This paper used a relatively simple decision setting in order to test a TTB heuristic
decision rule. The results showed that even when using simplistic methodologies,
heuristics performed well with complex empirical data. The heuristic rule was also able
to outperform a more traditional additive rule. These results clearly indicate that there is
merit in further developing heuristic decision rules for use in the field of marketing

In order to improve the results obtained from analysing empirical data with heuristic
decision rules, it is suggested that the following factors are examined:
      The order of entry of attributes, or importance weightings of the attributes;
      The effect of tied attribute ratings;
      The limitations of heuristics with regards to market effects;
      The choice of which heuristic decision rule to utilize.

                                               PG 18
Appendix 1: Review of results obtained from analysing the Laundry dataset.

TTB correctly predicted 431 respondents‟ Most Often Brand while WADD correctly
predicted Most Often Brand a total of 429 times.

              N         No. Correct       % Correct      Correlation
TTB           688       431               62.65          0.655
WADD          688       429               62.35          0.634

Within the laundry dataset there were 97 instances where the respondents gave exactly
the same attribute ratings to two or more brands (i.e. 97 cases with tied attribute

The above table displays a correlation of 0.655 for the TTB rule. This indicates a strong
positive linear relationship between the TTB predictions and the Most Often brand
scores. We can see that TTB performs moderately better than the WADD rule. But, the
differences between the two rules are minimal for this dataset.

                                            PG 19
Appendix 2: Review of results obtained from analysing the Toothpaste dataset.

Looking at the number of correct predictions, we see that TTB correctly predicted Most
Often brand 344 times and WADD correctly predicted Most Often brand 328 times.
            N         No. Correct      % Correct    Correlation
TTB         457       344              75.27        0.583
WADD        457       328              71.77        0.547

Within the toothpaste dataset there were 63 instances where the respondent‟s attribute
ratings were tied.

Here we have a correlation of 0.583 indicating a relatively strong positive association
between the TTB predictions and the Most Often brand. These results indicate that the
TTB rule performs better than a WADD rule.

                                           PG 20

Bettman et al. (1993). Correlation, Conflict and Choice, Journal of Experimental
Psychology: Learning, Memory and Cognition, Vol. 19, No. 4, pp. 931 – 951.

Bower, B. (1999), Simple Minds, Smart Choices, Science News Online, Vol. 155, No.
22, 1999. Available online:

Broder, A. (2003), Decision Making With the “Adaptive Toolbox”: Influence of
Environmental Structure, Intelligence, and Working Memory Load., Journal of
Experimental Psychology: Learning, Memory and Cognition, 2003, Vol.29, No.4, pp.
611 – 625.

Goldstein, D., Gigerenzer, G. (2002), Models of Ecological Rationality: The Recognition
Heuristic, Psychological Review, 2002, Vol. 109, No.1, pp.75 – 90.

Gigerenzer, G., Todd, P. (1999), Simple Heuristics that Make us Smart, Oxford
University Press, Oxford, 1999.

Gigerenzer, G., Selten, R. (2001 ) Bounded Rationality: The Adaptive Toolbox , MIT

Hoffrage, U., Torsten, R. (2004), Models of Bounded Rationality: The Approach of Fast
and Frugal Heuristics, Management Revue, Vol. 15, issue 4, 2004.

Hogarth, R., Karelaia, N. (2005), Simple Models for Multiattribute Choice with Many
Alternatives: When it Does and Does Not Pay to Face Trade-offs with Binary Attributes,
Management Science, Vol. 51, No.12, pp. 1860 – 1872, 2005.

                                         PG 21
Newell, B., Shanks, D. (2003), Take the Best or Look at the Rest? Factors influencing
“One Reason” Decision Making, Journal of Experimental Psychology: Learning,
Memory and Cognition, 2003, Vol. 29, No.1, pp. 53 – 65.

Newell et al. (2003), Empirical test of a fast-and-frugal heuristic: Not everyone “takes-
the-best”, Organizational Behaviour and Human Decision Processes, Vol. 91, 2003, pp.
82 – 96.

Payne et al. (1988). Adaptive Strategy Selection in Decision Making, Journal of
Experimental Psychology: Learning, Memory and Cognition, Vol. 14, No.3, pp.534 –

Wikipedia (2008), Bounded Rationality, available online from:

                                           PG 22

To top