VIEWS: 23 PAGES: 20 POSTED ON: 11/15/2009
Evolution of risk and ambiguity aversion Jan Burian University of Economics, Prague Outline • Decision in situations with missing information (risk and ambiguity) • Neural systems responding to missing information • Multi-agent model of evolution of risk and ambiguity aversion Risk and ambiguity • Risk – Information needed to determine the result of our decision with certainty is missing – But we have information about the probability distribution of possible results • Ambiguity – Even the information about the probability distribution is missing Expected Utility Hypothesis • Assumptions: – Rational agent is able to assign subjective utility to any reward – In ambiguity situations the agent is able to assign subjective probability to the possible rewards – The expected utility is independent on: • The probability of rewards in other alternatives • The ambiguity in the alternatives • Then the expected utility is given by the sum: EU = ∑i pi U(rewardi) and the agent will choose the alternative with maximum EU Allais problem (Allais, 1953) Experiment 1 Game A Reward $1 million Experiment 2 Game B Game C Probability 89% 1% 10% Game D Probability 89% 11% $5 million 10% Probability 100% Reward $1 million $0 $5 million Reward $0 $1 million Reward $0 Probability 90% Average decisions are A in Experiment 1 and D in Experiment 2 Experiment 1 1.0U($1M) > 0.89U($1M) + 0.01U(0$) + 0.1U($5M) 0.11U($1M) > 0.01U(0$) + 0.1U($5M) Experiment 2 0.89U(0$)+0.11U($1M) < 0.9U(0$) + 0.1U($5M) 0.11U($1M) < 0.01U(0$) + 0.1U($5M) Interpretation of this contradiction: Falsification of the independence of expected utiĺity on the probability of rewards in other alternatives Ellsberg paradox (Ellsberg, 1961) 90 balls in a bag, 30 red, the rest consists from blue and yellow. Bet on some combination of colors, when the ball with appropriate color will be chosen the reward will be 100$, otherwise 0$. Experiment 1: Bet on red or blue ball. Average bet is on red. 1/3U(100$) + 2/3U(0$) > PbU(100$) + (1- Pb) U(0$) 1/3>Pb Experiment 2: Bet on red and yellow or blue and yellow ball. Average bet is on blue and yelow. 1/3U(100$)+PyU(100$)+PbU(0$) < 1/3U(0$)+PyU(100$)+PbU(100$) 1/3 < Pb Interpretation of this contradiction: Falsification of the independence of expected utiĺity on ambiguity. Interpretation of Allais problem and Ellsberg paradox • There exists a decision-making mechanism responding to missing information • This mechanism is behaviorally manifested as an aversion to risk and ambiguity • Neural basis of such mechanism? – Neuroeconomy (Glimcher & Rustichiny, 2004) • Interpretation of the experimental data of neuroscience in the conceptual framework of psychology and economy Neural Systems Responding to Degrees of Uncertainty (Hsu et al., 2005) • fMRI scanning of striatum, orbitofrontal cortex (OFC) and amygdala during decision tasks similar to Ellsberg paradox – All the scanned systems displayed higher activity • Striatum has been more active in risk situations and less active in ambiguity situations • OFC and amygdala have been less active in situations of risk and more active in ambiguity situations • Decisions of people with focal lesions in OFC confronted with decisions of a control group with focal lesions in temporal lobe – People with lesions in OFC displayed lower aversion to both risk and ambiguity Evolution of risk and ambiguity aversion • If there exist neural system responding to missing information (behaviorally manifested as the aversion to risk and ambiguity) then this aversion, at least partially, must be determined by evolution and not only by culture • We can model the dynamics of such evolution • The model should be deliberately simple to show the key-parameters affecting the dynamics Elements of the basic model • • Population with a constant number of reproducing agents Aversion to risk and ambiguity – – – – The only genetic information of agents Genotype is also phenotype Represented as a probability of risky decisions The average aversion will converge during the evolution to a semi-stable range of values • • Reward = better probability of reproduction Types of behavior 1. Certain success but only basic probability of reproduction 2. Risky behavior with possible actual advantage in reproduction but also with possibility of instant death • Repeated decisions – – The decisions are repeated in cycles equivalent to reproduction cycles We consider the maximum number of reproduction cycles as the length of life of the agent Sexual (hermaphrodite) or non-sexual The offspring inheres the average aversion of the parents modified by a random mutation rate • Reproduction – – Parameters of the basic model • Key parameters: – Length of life: L<1; 30> – Winner advantage: a – Probability of death: Pd • Parameters important for convergence of the average aversion: – Number of reproduction cycles: 3000 – Mutation rate: 0.1 • Parameters important for statistical reliability: – Size of the population: 250 agents – Number of populations: 20 (we use the average aversion from these populations) Results of experiments Top row: Evolution of average risk and ambiguity aversion for different lengths of life (a=4, Pd=0.1, L{1, 7, 13, 19, 25, 30}) Bottom row: Histograms of the average aversion after 3000 iterations Results of experiments The dependency of the value of average aversion on length of life for different combinations of winner advantage and probability of death Top row: Pd=0.1; a{0.5, 2, 4, 32} Bottom row: a=1; Pd{0.2, 0.1, 0.05, 0.025}) Results of experiments The dependency of the value of average aversion on length of life for a=1 and Pd =0.1 for non-sexual reproduction Analytical solution • • • • Aversion: p Basic reproduction reward: r = 1/N Winner reproduction reward: rw = (1+a)r Probability to survive (in one cycle): Q = (1-p)+(1-p)(1- Pd) = 1- Pd - Pd p • Reproduction reward (in one cycle): R = pr + (1-p)(1- Pd) rw • Total reward: QG + Q2G+…+ QmaxG = Q [(1-Qmax)/(1-Q)] G Summary: • Average risk and ambiguity aversion: – grows with the length of life • Interpretation: the negative utility of death grows for agents with longer lives (these agents loose their ability to reproduce in future periods) • The shape of the dependency of average aversion on length of life is similar to the S-curve – grows slower for higher winner advantage and lower probability of death – grows slower for non-sexual reproduction Questions, objections and answers • What exactly does this model explain? – The model explains the influence of environmental parameters on the dynamics of aversion evolution. It explains the interaction between environment (parameters) and agents (aversion) in the simplest case. • You get from your model what you have put in. – Yes in some sense. But the aim of this model was not to find out something surprising but to explain in more general way something what we already know about humans (and possibly other kinds of animals). This model establishes a basic model where we can identify the key parameters of the evolution of aversion and which can serve as a starting point for future more complex models. Questions, objections and answers • Our decisions are complex, it is not one system, and they cannot be reduced to one number. – Of course, this is true about the decision systems of biological organisms, but the evolution of these systems in organisms could share some general properties, and this model and other very simple basic models could help us to identify these general properties. • What is the relation between this model and neural systems in our brain? – There is no direct relation. This model shows in which circumstances an evolutionary process will develop a behavior preferring the decisions with certain outcomes instead of decisions with uncertain (although possibly rewarding) outcomes. But this model doesn’t show how exactly this kind of behavior will be morphologically based in the structure of the organism. Questions, objections and answers • The really interesting question is “Why did the aversion emerge?” and not “How to model the dynamics of this presupposed evolution?”. – I show that in some classes of evolutionary processes the evolution must lead to the preference of the decisions with certain outcome. This we call aversion to risk and ambiguity. So in fact … if we just assume that the decisions agent must make have different levels of uncertainty of the outcomes and the agent can choose between these types of decisions (which seems to be a plausible assumption), then we also showed why the aversion emerges. • OFC and amygdala are involved in our emotional responses, is this involvement of emotions considered in the research on risk and ambiguity aversion? – Yes. In some sense we can say, that the Expected Utility Hypothesis didn’t recognized the role of emotions in our decision making. It did expect purely rational decision making. Or more exactly it didn’t count with the kind of emotions that is responding to risk and ambiguity.