Heterogeneity Stabilizes Reciprocal Altruism Interactions

Document Sample
Heterogeneity Stabilizes Reciprocal Altruism Interactions Powered By Docstoc
					J. theor. Biol. (2001) 209, 87}95
doi:10.1006/jtbi.2000.2248, available online at on

            Heterogeneity Stabilizes Reciprocal Altruism Interactions
                    MICHAEL A. FISHMAN*, ARNON LOTEM                    AND   LEWI STONE

     Department of Zoology, Faculty of ¸ife Sciences, ¹el Aviv ;niversity, ¹el Aviv 69978, Israel

             (Received on 25 October 1999, Accepted in revised form on 9 December 2000)

         In considering the phenomena of reciprocal altruism few would dispute that there are
         di!erences in individual quality*in particular, that for some individuals, at least on occasion,
         the cost of doing favors will exceed the potential of future bene"ts. That is, at any given time,
         a typical population is heterogeneous with respect to the a+ordability of reciprocal altruism.
         However, methodological limitations of the traditional analytical framework*Single ¹ype
         (symmetric) Evolutionary Game ¹heory2have restricted previous analytical e!orts to address-
         ing populations idealized in terms of their averages. Here we use the methods of Multitype
         Evolutionary Game ¹heory to analyse the role of individual di!erences in direct reciprocity
         interactions. Multitype analysis shows that non-idealized populations possess an ESS pro"le
         wherein individuals who cannot a!ord reciprocity (low-quality) defect, while individuals who
         derive net bene"ts from reciprocity (high-quality) cooperate. Furthermore, this cooperation is
         implemented via unmodi"ed tit-for-tat (TfT) strategy. Hence, our results may help resolve
         a long-standing problem concerning the evolutionary stability of TfT in direct reciprocal
         altruism. Finally, this di!erence between idealized and real populations is not restricted to
         direct reciprocal cooperation. Previously (Lotem et al., 1999) we have demonstrated evolu-
         tionarily stable indirect reciprocal cooperation among high-quality individuals in hetero-
         geneous populations.

                                                                                     2001 Academic Press

                  0. Introduction                          scious inheritable trait maintained by Darwinian
                                                           "tness advantages, was "rst advanced by
Altruistic behavior is commonly attributed to
                                                           Williams (1966), and given rigor by Trivers
inclusive "tness, or reciprocity (Krebs & Davies,
1993). The idea of reciprocal altruism, albeit one
                                                              By using game theoretical analysis Trivers has
motivated by conscious calculation, was "rst pro-
                                                           shown that an individual may help an unrelated
posed by Darwin: &&as the reasoning powers and
                                                           conspeci"c whenever: (i) the cost to the donor is
foresight2become improved, each man would
                                                           less than the bene"t to the recipient, and (ii) the
soon learn from experience that if he aided his
                                                           favor is likely to be returned at a latter date.
fellow-men, he would commonly receive aid in
                                                           Hence, in order for reciprocal altruism (coopera-
return''. (Darwin, 1871). The idea of reciprocal
                                                           tion) to persist, cooperators must protect
altruism in its modern form, i.e. as an uncon-
                                                           themselves from exploitation by individuals who
                                                           accept favors, but do not reciprocate*defectors.
                                                           Trivers, aided by W. D. Hamilton (Trivers, 1971,
  * Author to whom correspondence should be addressed.     p. 39), resolved this issue by postulating that
E-mail: ma"                               cooperation is conditional, i.e. a cooperator will

0022}5193/01/050087#09 $35.00/0                                                                2001 Academic Press
88                                                 M. A. FISHMAN E¹ A¸.

keep helping an unrelated individual, unless the                   (a) The costs of reciprocity are less than its
latter refuses to reciprocate.- These theoretical                      bene"ts for all individuals involved.
results soon gained empirical support from ob-                     (b) The costs of reciprocity exceed its bene"ts
servations detailing reciprocal exchange of favors                     for all individuals involved.
in group-living species (Fisher, 1980; Seyfarth &                  (c) The costs of reciprocity exceed its bene"ts
Cheney, 1984; Wilkinson, 1984).                                        for some individuals, and are less than
   The concept of helpfulness conditional upon                         these bene"ts for the rest. Hence, at any
opponents' reciprocity was further developed by                        given time, the subject population consists
Axelrod and co-workers (Axelrod & Hamilton,                            of two quality classes: low-quality indi-
1981; Axelrod, 1984; Trivers, 1985) resulting in                       viduals*who cannot &&a!ord'' reciprocity
the formulation of tit-for-tat (TfT) strategy: a TfT                   vs. high-quality individuals*who derive
player will punish defection by refusing help in                       net bene"ts by exchanging favors.
turn, but otherwise will cooperate. In particular,
these authors demonstrated that a population of                     By adopting a representation in terms of popu-
TfT players cannot be invaded by defectors, i.e.                 lation averages, cases (a) and (b) can be analysed
TfT is an Evolutionarily Stable Strategy (ESS) in                in terms of symmetric (single-type) evolutionary
the context of the TfT vs. defector contest. How-                game theory (cf. Nowak et al., 1995; Brembs,
ever, Selten & Hammerstein (1984) have shown                     1996), which addresses situations where all con-
that TfT is not an ESS under biologically plaus-                 testants have the same choice of game strategies
ible conditions. Brie-y: mutants that lost the abil-             and receive the same payo!s for any particular
ity to make TfT-type evaluation of opponents,                    interaction (Maynard Smith, 1982). However,
and thus help others unconditionally, can in-                    such a simpli"cation would reduce case (c) to
crease through genetic drift in a population of                  either (a) or (b), and therefore is inappropriate.
TfT players, and a population containing sub-                    Thus, to analyse case (c), we must use the
stantial fraction of these unconditional altruists               methods of Multitype Evolutionary Game ¹heory
(UA) can be invaded by defectors.                                that allow analysis of contests between con-
   The demonstration that TfT is not an ESS led                  testants di!ering in the strategy choices available
to the concerted e!ort to formulate cooperation                  to them, or having the same strategy choices but
strategies superior to TfT in the sense of not                   di!erent payo!s for some of the possible interac-
being subject to invasion by unconditional altru-                tions (Cressman, 1992; Weibull, 1996).
ists, reviewed by (Nowak et al., 1995; Brembs,                      Our multitype analysis shows that individuals
1996). The alternate approach of this paper is                   who cannot a!ord to reciprocate (e.g. young,
based on the following considerations. Even if the               sick, handicapped, or those that simply do not
bene"ts of receiving help are the same for all                   have su$cient resources at a given time) will
members of a population, there are di!erences                    defect by default. Thus, these phenotypic defectors
(genetic or phenotypic) among individuals with                   (Lotem et al., 1999) are a special case of
respect to the ability to donate help. Hence, for                phenodeviation*a name for the disruptive e!ects
any type of exchange of favors, there are three                  of the environment on genotype expression pro-
possible situations.                                             posed by Thornhill & M+ller (1997) in their semi-
                                                                 nal work on developmental stability. Because
   -Trivers used a metaphor, known as the Prisoner1s Dilem-      defection by default is a phenodeviation, and thus
ma (PD), from game theory. In this game, the two players
have the choice of cooperating or defecting. The payo! to        cannot be eliminated by natural selection, it is
each when they cooperate is greater than the payo! for           a persistent feature of real populations.
mutual defection, but less than the payo! to a defector             The persistence of defection confers an advant-
playing against a cooperator. Finally, the payo! to a cooper-
ator playing against a defector is the least of all. A single-
                                                                 age on TfT players vis-a-vis unconditional altru-
stage PD game can be shown to have a unique stable               ists. This advantage is absolute, i.e. TfT players
solution*mutual defection (cf. Fudenberg & Tirole, 1996,         always have higher "tness than unconditional
Section 1.1.3). Trivers have shown that in an open-ended         altruists, leading to the elimination of the latter
series of PD games between two opponents*a Repeated
Prisoner1s Dilemma (RPD), a conditional cooperator does          from the population. As was shown previously
better than a defector.                                          (Axelrod & Hamilton, 1981), in the absence of
                                HETEROGENEITY AND RECIPROCAL ALTRUISM                                                89

unconditional altruists, TfT is an ESS. Thus, we
arrive at an ESS pro"le in which individuals who
derive net bene"ts from reciprocity play TfT,
whereas individuals that cannot a!ord reci-
procity defect.
   Since an individual's ability to reciprocate
(quality) changes with time, our results can be
interpreted in two, not necessarily mutually ex-
clusive, ways. (i) A population might be perma-
nently divided into high-quality cooperators and
low-quality defectors. (ii) Individuals can switch
                                                            FIG. 1. Here 0(r(1 is the probability that an indi-
behavior as their capacity for reciprocity (quality)     vidual requesting help has been requested to help recently
varies. In this latter interpretation, our ESS result    enough for its response to be remembered: defection is
represents cooperators who occasionally defect.          punished with a refusal to help (!), and cooperation is
Thus, our derived strategy pro"le is reminiscent         rewarded with cooperation (#). Alternatively, with prob-
                                                         ability 1!r, there is no de"nite memory of previous interac-
of the evolutionarily stable &&mistake-making            tion, and therefore the TfT is not motivated to refuse help.
TfT'' strategy proposed by Boyd (1989).                  Thus, the value of r depends on the probability of repeated
   Finally, the stabilizing e!ect of phenotypic          interactions and the "delity of memory/individual recogni-
                                                         tion. Considered in a di!erent way, r is the probability that
defection, is not restricted to direct-reciprocity       the favor will be repaid, if the recipient is another TfT.
interactions. Previously (Lotem et al., 1999),
we have shown that the presence of phenotypic
defectors, introduced as a modelling assumption,         indiscriminately; defectors (DE) that solicit, but
stabilizes cooperation in the analogous situations       never donate help; and conditional altruists, or
of indirect reciprocity interactions (Nowak &            TfT players*who retaliate for each defection by
Sigmund, 1998a, b).                                      refusing help in the next interaction, but other-
   This paper is organized as follows. In Section 1,     wise act as unconditional altruists. Thus, unlike
we introduce a notation, one that we "nd conve-          the invariant responses of the UA and DE
nient for multitype analysis, and recapitulate the       players, the response to a request for help by
work of our predecessors in this notation. That is,      a TfT player depends on its memory of previous
we formulate a symmetric game theoretical                interactions. That is, a TfT player always helps
model addressing cases (a) and (b), and show that        unconditional altruists and other TfT players,
in these situations defector strategy is the unique      but helps defectors only when it lacks informa-
ESS. In Section 2, we extend the single-type             tion to classify them. Denoting the probability
model of Section 1 to a multitype model of the           that an individual requesting help has been re-
two quality classes of situation (c). To facilitate      quested to help recently enough for its response
presentation we con"ne some of the technical             to be remembered by 0(r(1, we obtain the
details of the analysis to the appendix.                 TfT response scheme summarized in Fig. 1.
                                                            Let us denote the average (per capita) accumu-
                                                         lated bene"ts of receiving help over a lifetime by
  1. Symmetric Model (Idealized Populations)             B, and the average lifelong costs of donating help
   We start the analysis by constructing a sym-          by C; we use capital letters to distinguish these,
metric game theoretical model for idealized              per lifespan, payo!s from the per encounter
populations. That is, we address the issue of            payo!s more usual in the literature (cf. Nowak
direct reciprocal altruism in terms of population        & Sigmund, 1998b).?
averages. This yields situations (a) or (b), i.e. the
(average) costs of reciprocity are either less (a), or
greater than (b) its bene"ts. As discussed in the           ? If the per encounter bene"ts and costs are given by b and
introduction, we consider three evolutionary             c, respectively; the probability of t encounters per lifespan is
                                                         given by w (0(w(1); and, on the average, individuals
game strategies (heritable behavior phenotypes):         take turns soliciting and being solicited for help: then B"b/
unconditional altruists (UA) that help others            (2(1!w)) and C"c/(2(1!w)).
90                                               M. A. FISHMAN E¹ A¸.

  In these terms the payo! matrix for donating                excluding the strictly dominated strategies*has
help is given by                                              the same ESS solutions as the original system (cf.
                                                              Weibull, 1996, Chapter 3.2.1). Therefore, defec-
         UA      TfT         DE                               tion is again the unique ESS.
         !C !C              !C          UA
                                                 .     (1a)
         !C !C !(1!r)C                  TfT                      2. A Multitype Model for Heterogeneous
            0       0          0        DE
                                                                 In this section, we analyse a situation where
Note that the entry i}j represents the costs of help          the costs of reciprocity exceed its bene"ts for
given by (an average player of ) strategy-i to (an            some individuals, and are less than these bene"ts
average player of ) strategy-j. Thus, to calculate            for the rest. We start by dividing the population
the bene"ts of receiving help, we transpose matrix            into two classes: low-quality individuals for
(1a) and substitute #B for !C. This yields                    whom costs of reciprocity exceed its bene"ts vs.
                                                              high-quality individuals for whom reciprocity
            UA      TfT      DE                               yields net bene"ts. The membership in a class is
                                                              not necessarily hereditary*a reader might "nd it
            B           B     0       UA
                                            .          (1b)   convenient to think of these quality classes as
            B           B     0       TfT                     juveniles and mature individuals, respectively.
            B     (1!r)B      0       DE                      We shall denote the frequency of low-quality
                                                              individuals by 0(q(1 (cases q"0, 1 have
The payo! matrix, P, for both giving and receiv-              been addressed in the previous section). We re-
ing help, the game matrix, is obtained by adding              tain C as the average for the accumulated lifelong
matrices (1a, b) to obtain                                    costs of altruism in the high-quality class, and
                                                              denote the corresponding value for the low-qual-
       UA        TfT          DE                              ity class by D. We retain the use of B for the
                                                              accumulated lifelong bene"ts. As discussed
      B!C        B!C          !C                UA
P"                                                    . (2)   above, C(B(D. Using r as in Section 1, and
      B!C        B!C        !(1!r)C             TfT           using the subscripts H and ¸ to denote the qual-
        B       (1!r)B            0             DE            ity classes, we have the following payo! matrices:
                                                              P , P , P , P . Here the "rst subscript de-
                                                                && &* *& **
  As discussed above, in this section we consider             "nes the focal (recipient of the payo!, the row
two possibilities.                                            strategy) and the second subscript de"nes the
   (a) ¹he costs of reciprocity are less than its
bene,ts, i.e. C(B. In the appendix we show that,              P
if C(B, then system (2) has a unique ESS solu-
tion, DE, i.e. defectors displace individuals using                     UA  TfT               DE
                                                                          &     &               &
alternative strategies, resulting in a population                       B!C B!C               !C         UA
consisting of defectors only.                                 "(1!q)                                        &,
                                                                        B!C B!C !(1!r)C                  TfT
   (b) ¹he costs of reciprocity are greater than its                       B     (1!r)B         0        DE
bene,ts, i.e. C'B. If C'B, then every element                                                               &
of the third row of P is greater than the corre-                                                             (3)
sponding elements of its "rst and second rows.
                                                                        UA  TfT              DE
That is, at any composition of the population,                            *     *              *
defectors have higher "tness than cooperators                           B!C B!C              !C         UA
                                                              P "q                                         &
(UA or TfT), leading to the elimination of the                 &*  B!C B!C !(1!r)C                      TfT ,
latter from the population. Formally, UA and                                                               &
                                                                          B     (1!r)B        0         DE
TfT are strictly dominated by DE, and can be                                                               &
excluded, i.e. a reduced system*obtained by                                                                  (4)
                              HETEROGENEITY AND RECIPROCAL ALTRUISM                                     91

P                                                              UA        TfT          DE
 *&                                                              &          &           &
           UA  TfT              DE                             B!C       B!C          !C        UA
             &     &              &                    (1!q)                                       &
           B!D B!D                          UA                 B!C       B!C !(1!r)C            TfT ,
                                               *                                                   &
           B!D B!D !(1!r)D                  TfT ,                B       (1!r)B         0       DE
             B     (1!r)B            0      DE                   DE
                                               *                    *
                                                 (5)              1       UA
                                                         !qC                 &                          (7)
                                 *                              1!r       TfT ,
         B!D      B!D          !D         UA                         0    DE
P "q                                         *                               &
 **  B!D          B!D !(1!r)D             TfT .
                                             *            UA          TfT       DE           DE
            B     (1!r)B         0        DE                 &             &       *             *
                                                       ((1!q)B (1!r) (1!q)B 0) DE , (0)
                                                 (6)                                     *
Note that the payo!s depend on the fre-                   Since all low-quality individuals are (pheno-
quencies of the two quality types in the popula-       typic) defectors, the ESS solutions of system (7),
tion. For example, every element of the P
                                             &&        and hence system (3}6), have the form
and P is multiplied by (1!q) because this
is the probability to encounter a high-quality                             (x*, DE ),                   (8)
opponent.                                                                         *
   The general mathematical framework for              where x* is an ESS solution of the system
analysing evolutionary stability in games with         obtained by combining the payo!s for interacting
two types of players was developed by Cressman         with DE to the payo!s for high-quality vs. high-
and co-workers (Cressman & Dash, 1991; Cress-          quality interactions. That is, we add the row
man, 1992). The speci"c case of system (3}6),          elements of the reduced P to each element of
however, can be analysed by taking advantage           the appropriate row of P to obtain

                                  UA                      TfT                 DE
                                     &                       &                  &
                               (1!q)B!C                (1!q)B!C               !C            UA
                                                                                               &        (9)
                           (1!q)B!(1!qr)C (1!q)B!(1!qr)C !(1!r)C                            TfT .
                                 (1!q)B                (1!q) (1!r)B               0         DE

of the fact that, similar to the case for symmetric    Since q, r'0, every element of the second
games discussed above, strictly dominated              row is greater than the corresponding element
strategies in multitype games can be excluded          of the "rst row. This is due to the fact that
without a!ecting the ESS solutions of the              the burden imposed by the presence of
game (cf. Weibull, 1996, Section 5.6.1). To            phenotypic defectors on unconditional altruists is
wit, since B(D, every element of the third row         greater than the corresponding burden, eqn (7),
of P and P is greater than the correspond-             on the TfT players. Formally, TfT strictly
     *&        **                                                                            &
ing elements of the "rst and second rows. Hence,       dominates UA *and therefore, as discussed
UA and TfT are strictly dominated by DE                above, we can exclude UA . This yields a
    *            *                                *                                 &
and can be excluded. That is, we see that              reduced system
low-quality individuals &&must'' defect. Exclu-
sion of UA and TfT yields a reduced                        p p
               *              *                        P " RR RB
system:                                                 &  p p
                                                            BR BB
92                                                 M. A. FISHMAN E¹ A¸.

        TfT       DE                                                Condition rB'C is both necessary and su$-
           &         &
" (1!q)B!(1!qr)C !(1!r)C                            TfT          cient when we consider competition between TfT
                                                       &.        players and defectors (Axelrod & Hamilton,
        (1!q) (1!r)B                    0           DE
                                                      &          1981). However, as detailed in Section 1 and in
                                                       (10)      the appendix: eqns (A5, A6), because in the ab-
                                                                 sence of defectors there is no di!erence in "tness
    We see that p 'p , i.e. defector playing
                    BB  RB                                       between TfT and UA players*TfT playing
against defector does better than a TfT playing
                                                                 populations can be invaded by unconditional
against defector. Hence, a population of defectors
                                                                 altruists and a subsequent mixed population can
cannot be invaded by TfT players. Consequently,
                                                                 be invaded by defectors. This result, however, is
defection is an evolutionarily stable strategy of
                                                                 only obtained when we neglect the heterogeneity
system (10).
                                                                 of real populations. In heterogeneous popula-
    If p 'p as well, then defector playing against
        BR    RR                                                 tions, there are individuals who cannot a!ord
TfT does better than a TfT playing against TfT
                                                                 reciprocity, eqns (3}6), and therefore defect by
(exploitation pays better than cooperation).
                                                                 default*phenotypic defectors. In the presence of
Hence, a population of TfT players can be in-
                                                                 these phenotypic defectors, unconditional altru-
vaded and taken over by defectors. Thus, if
                                                                 ists have lower "tness than TfT players, eqn (9),
p 'p , defection is the only ESS of system (10).
  BR     RR                                                      and can be excluded. Thus, in heterogeneous
However, if p 'p , then cooperation pays bet-
                 RR  BR                                          populations, the situation reduces to the com-
ter than exploitation. In particular, a population
                                                                 petition between (high-quality) TfT players and
of TfT players cannot be invaded by defectors.
                                                                 (high-quality) defectors, modi"ed by the presence
That is, as discussed in connection with eqn (8),
                                                                 of phenotypic defectors.
system (3}6) has two ESS solutions (DE , DE )
                                           &     *                  Although phenotypic defectors prevent de-
or (TfT , DE ).
            &    *                                               stabilization of cooperation by unconditional al-
                                                                 truists, their presence is not an unmixed blessing.
           p !p "(rB!C)!qr(B!C)                                  Because TfT players help defectors (in particular,
            RR BR
                                                                 phenotypic defectors) when in doubt (Fig. 1), the
                    "rB(1! ) ( !q),                      (11)    presence of phenotypic defectors imposes a bur-
where                                                            den on TfT players, and decreases their ability to
                                                                 compete with defectors. Thus cooperation per-
         C                  rB!C  1 r!
        "   and           "      "     .                         sists only if the frequency of phenotypic defectors
          B                r(B!C) r 1!                           (q) is less than , i.e. can be thought of as the
                                                                 tolerance capacity (in analogy with the carrying
  Since C(B, (1. Thus, for cooperation to                        capacity term of the logistic equation), for the
be more productive than exploitation, we must                    burden of phenotypic defectors. We summarize
have q( . Since q'0, we must have '0,                            these results in Fig. 2.
which in turn requires r' , i.e. rB'C. As dis-
cussed in Section 1, rB is the expectation of re-
payment when dealing with another TfT player.
Hence, we obtain the unsurprising conclusion
that cooperation can persist only if the expected
repayment exceeds the investment.A

   A This result is analogous to the Hamilton1s rule of kin
altruism (Hamilton, 1964). According to Hamilton's rule
help may be donated to a relative if the degree of relatedness
times the bene"t to the recipient (the inclusive "tness bene-
"ts to the donor) exceeds the donor's costs. That is, in both
cases apparently altruistic acts are undertaken only when          FIG. 2. Here the inequalities on the paths from the origin
they yield net bene"ts to the &&altruist'' (Nowak & Sigmund,     ( ) to endpoints represent the conditions that must be satis-
1998b).                                                          "ed if the strategy pro"le(s) at the endpoint to be ESS.
                                HETEROGENEITY AND RECIPROCAL ALTRUISM                                              93

  Thus, we see that cooperation is possible if           formulated following the demonstration that TfT
a population contains individuals who cannot             playing populations can be invaded by uncondi-
a!ord reciprocity, but the frequency of such, low-       tional altruists and subsequently by defectors
quality individuals is not too high.                     (Selten & Hammerstein, 1984). Our work demon-
                                                         strates that TfT is evolutionarily stable in
                                                         competition with unconditional strategies in het-
                   3. Discussion                         erogeneous populations. However, it is by no
   The analysis of reciprocity undertaken in this        means certain that an analogous analysis of
paper shows that reciprocal altruism can be              the competition between TfT and some of
stable when individual variation is taken into           the more sophisticated conditional cooperation
account. Genetic and/or phenotypic di!erences            strategies will demonstrate domination by
in ability among individuals, in particular in indi-     the former. Thus, examination of the e!ects
vidual costs for donating help, create three dis-        of heterogeneity on the functioning of condi-
tinct situations. When the costs of reciprocity are      tional cooperation strategies, and of their
either less than its bene"ts or exceed its bene"ts       relative merits in heterogeneous context, though
for all individuals involved, then the situation can     beyond the scope of the present paper, is the next
be analysed in terms of the single-type (symmet-         logical step in game theoretical investigation of
ric) evolutionary game theory, and reciprocity is        reciprocity.
not an ESS (Selten & Hammerstein, 1984). How-               Above all else, the current study illustrates that
ever, the most likely situation is when the costs        individual variation is more than just a noise, and
of reciprocity exceed the bene"ts for some               thus the study of the evolution of behavior in
individuals, but are less than the bene"ts for the       terms of population averages may yield mislead-
rest. That is, the subject population consists of        ing results. In terms of mathematical methods,
two quality classes: low-quality individuals*who         our results highlight the usefulness of multitype
cannot &&a!ord'' reciprocity vs. high-quality            evolutionary game theory in analysing real (het-
individuals*who derive net bene"ts from reci-            erogeneous) populations. On the empirical level,
procity, and must be analysed using the methods          they illustrate the importance of studying vari-
of the multitype evolutionary game theory                ations in quality in relation to the behavioral
(Cressman, 1992). Multitype analysis shows that,         phenotypes.
given the appropriate conditions, such a two-
class population exhibits an ESS pro"le wherein            We would like to use this opportunity to thank the
individuals who cannot a!ord reciprocity defect,         unknown referees for encouraging evaluation and
while individuals who derive net bene"ts from            constructive criticism.
reciprocity cooperate by playing unmodi"ed
   The stabilizing e!ect of individual variation                              REFERENCES
derives from the fact that individuals who cannot        AXELROD, R. (1984). ¹he Evolution of Cooperation (reprinted
a!ord reciprocity defect by default. The persist-         1989). Harmondsworth: Penguin.
                                                         AXELROD, R. & HAMILTON, W. D. (1981). The evolution of
ent presence of these phenotypic defectors abro-          cooperation. Science 242, 1390}1396.
gates the ability of the unconditional altruist          BOYD, R. (1989). Mistakes allow evolutionary stability in the
mutants to invade TfT populations*see Lotem               Repeated Prisoner's Dilemma game. J. theor. Biol. 136,
et al. (1999) for similar e!ects in indirect reciproc-   BREMBS, B. (1996). Chaos, cheating and cooperation: poten-
ity. Thus, contrary to the results in idealized           tial solutions to the Prisoner's Dilemma. OIKOS 76,
populations, persistent cooperation based on un-          14}24.
                                                         CRESSMAN, R. (1992). ¹he Stability Concept of Evolutionary
modi"ed TfT strategy is possible in real (hetero-         Game ¹heory. Berlin: Springer-Verlag.
geneous) populations.                                    CRESSMAN, R. & DASH, A. T. (1991). Strong stability and
   As discussed in the introduction, a large num-         evolutionarily stable strategies with two types of players.
ber of conditional cooperation strategies able            J. Math. Biol. 30, 89}99.
                                                         DARWIN, C. (1871). ¹he Decent of Man, and Selection in
to discriminate against unconditional altruists           Relation to Sex (reprinted 1981). Princeton, NJ: Princeton
(cf. Nowak et al., 1995; Brembs, 1996), were              University Press.
94                                               M. A. FISHMAN E¹ A¸.

FISHER, E. A. (1980). The relationship between mating         the payo! for playing strategy x3X against
 systems and simultaneous Hermaphroditism in the              a strategy y3X is given by
 coral reef "sh Hypoplectrus nigricans. Anim. Behav. 28,
FUDENBERG, D. & TIROLE, J. (1996). Game ¹heory, 5th                 u(x, y),x 3 P 3 y"(1!y !rx y ) B
 Printing. Cambridge, MA: MIT Press.
HAMILTON, W. D. (1964). The genetic evolution of social
                                                                               !(1!x !rx y ) C.            (A2)
 behavior. I & II. J. theor. Biol. 7, 1}52.                                             
KREBS, J. R. & DAVIES, N. B. (1993). An Introduction
 to Behavioral Ecology: A11, 3rd Edn. Oxford: Blackwell       ¹he ESS criterion (Cressman, 1992, Section 2) is
 Scienti"c Publications.                                      given by
LOTEM, A., FISHMAN, M. A. & STONE, L. (1999). Evolu-
 tion of cooperation between individuals. Nature 400,
 226}227.                                                     Strategy x*3X is an ESS if for any x3X, xOx*
MAYNARD SMITH, J. (1982). Evolution and the theory of         implies
 Games. Cambridge: Cambridge University Press.
NOWAK, M., MAY, R. M. & SIGMUND, K. (1995). The
 arithmetics of mutual help. Sci. Am. 272, 76}81.                           (x*, x),u(x*!x, x*)'0 or
NOWAK, M. & SIGMUND, K. (1998a). Evolution of indirect
 reciprocity by image scoring. Nature 393, 573}577.
                                                                                       (x*, x)"0
NOWAK, M. & SIGMUND, K. (1998b). The dynamics of                                   
 indirect reciprocity. J. theor. Biol. 194, 561}574.
SELTEN, R. & HAMMERSTEIN, P. (1984). Gaps in Harley's
 argument on evolutionary stable learning rules and in the                  (x*, x),u(x*!x, x!x*)'0.       (A3)
 logic of TfT. Behav. Brain. Sci. 7, 115}116.                           
SEYFARTH, R. M. & CHENEY, D. I. (1984). Grooming, allian-
 ces and reciprocal altruism in vervet monkeys. Nature 308,       Evolutionary games do not necessarily have an
 541}543.                                                     ESS point solution. There is also the possibility
THORNHILL, R. & M"LLER, A. P. (1997). Develop-
 mental stability, disease and medicine. Biol. Rev. 72,       of a set of solutions exhibiting neutral stability
 497}548.                                                     among themselves, while being ESS like in
TRIVERS, R. (1971). The evolution of reciprocal altruism.     comparison with the strategies not in that
 Quart. Rev. Biol. 46, 35}56.
TRIVERS, R. (1985). Social Evolution. Menlo Park, CA:
                                                              set*evolutionarily stable sets (ES sets). Formally
 Benjamin/Cummings.                                           (Cressman, 1992, Chapter 6), a proper subset
WEIBULL, J. W. (1996). Evolutionary Game ¹heory, 2nd Edn.        LX is an ES set if every x 3 satis"es condi-
 Cambridge, MA: MIT Press.                                                                   H
                                                              tion (A3) versus every x, , while being neutrally
WILKINSON, G. S. (1984). Reciprocal food sharing in the
 vampire bat. Nature 308, 181}184.                            stable relative to other strategies in .
WILLIAMS, G. C. (1966). Adaptation and Natural Selection:         Obviously, we cannot check every strategy in
 A Critique of Some Current Evolutionary ¹hought. Prince-     the continuous set X for being an ESS (let alone
 ton, NJ: Princeton University Press.
                                                              an element of an ES set). Hence, we enumerate
                                                              the ESS solutions as a two-step process. First, we
                       Appendix                               "nd all the potential ESS solutions by using the
                                                              fact that every ESS solution is a Nash Equilibrium
               Methods and Notation                           point (though not vice versa) and therefore if
  Let +e , e , e , be the standard basis of R,               x*3X is an ESS, then u(e , x*)"u(x*, x*) or
and let us denote the strategy set for the game               x*"0, ∀j.
given by eqn (2) (payo! matrix P) by                              Consequently, every solution of the system of

     X"+x e #x e #x e " x , x , x *0                                         [u(e !x, x)]x "0 ∀j           (A4)
                                                                        H        H

             and x #x #x "1,.                        (A1)     on X is a potential ESS. Once we have derived all
                                                           the potential ESS solutions, we apply the ESS
                                                              criterion (A3).
That is, e , e , and e represent the pure strategies             In the speci"c case of system (2), there is one
UA, TfT, and DE, and their convex combina-                    potential ES set solution and two potential ESS
tions represent mixed strategies. In these terms,             solutions.
                              HETEROGENEITY AND RECIPROCAL ALTRUISM                           95

  The potential ES solution                         However, since x Oe but
          "+ e #(1! )e " 3[0, 1],            (A5)
                                                                 (x , e )"0 and
represents the state where defectors are absent,
                                                              (x , e )"!r(B!C) (1! ).       (A8)
and therefore there is no di!erence between un-                
conditional altruists and TfT players. Since
                                                    x is not an ESS.
              e 3     and e ,                          Finally, we have a potential ESS solution
                                                    wherein defectors displace both unconditional
                                                    altruists and TfT players
                  (e , e )"!C.              (A6)
   is not an evolutionary stable set.               x "e and         (x , x)"C[x #(1!r)x ].
   Next, we have a potential ESS solution where-                                    (A9, A10)
in TfT players and defectors coexist
                                                    Hence, since 0(r(1, x (DE) is an ESS for all
  x " e #(1! )e :          "        .       (A7)    parameter values.