Documents
Resources
Learning Center
Upload
Plans & pricing Sign in
Sign Out

Rescorla-Wagner

VIEWS: 10 PAGES: 9

									Rescorla-Wagner Assignment                                                       1


                             THE RESCORLA-WAGNER MODEL

      The Rescorla-Wagner model is a "simple" mathematical model that attempts
to simulate changes in the strength of association between CSs and USs.
Mathematical modeling is a way of developing precise theoretical explanations.
A good model of some natural process (e.g., conditioning a dog to salivate when
a bell rings) will not only simulate the facts already documented, but will also
suggest circumstances and outcomes that have not yet been observed. Our
confidence in the model will grow if, when the suggested circumstances are
experimentally created, the results predicted by the mathematical model actually
occur. On the other hand, if the predictions of the model are not confirmed,
then we know that the learning process inside the animal is different from that
of the particular mathematical model, and that we must either propose a
different mathematical model or seek an explanation in some other medium.
      Mathematical modeling has been used extensively in theoretical psychology.
One of the most impressive models in the field of associative learning is one
proposed by Robert Rescorla and Allan Wagner. The Rescorla-Wagner model is
impressive for three main reasons: a) it accounts for most of the basic facts of
conditioning, b) it suggests novel circumstances and outcomes (most of which are
downright counterintuitive), and c) it is mathematically elegant (simple). For
these reasons, the model has been a major export to other areas of psychology
(social cognition, human judgment about causes and effects) in situations in
which an associative account of the data is desirable.
      The model simulates changes in the strength of a learned association
between a CS and US. This is what learning curves presumably depict. Early in
conditioning the association is weak and, correspondingly, CR strength is low.
As CS-US pairings accumulate, more learning takes place and the association
grows stronger. CRs occur more often and are of greater magnitude. Ultimately,
when learning is complete, the CS-US association reaches its final high level
and a vigorous CR is observed on every trial.
      The Rescorla-Wagner model describes a learning mechanism by which the
strength of associative connections is adjusted on a trial-by-trial basis. The
model learns "episodically" as a result of contiguity between events. Thus, it
is a contiguity-learning device in the tradition of Aristotle and Pavlov.
However, unique characteristics of the model result in its giving rise to
contingency-learning effects of the kind originally thought to be irreconcilable
with a contiguity mechanism. The model consists of the following formula:

ΔV = αβ(λ - VALL), where

V= the current associative strength of a given CS. This corresponds to the
current ability of a particular CS to evoke a CR (positive value = excitation,
negative value = inhibition).

ΔV = the change in V (increment or decrement) on a given trial. For example,
if a previously reinforced CS is extinguished by presenting it without the US,
the CS's associative strength (its strength of connection with the US) will
diminish. The result is negative value for ΔV.

λ = the maximum associative strength the US is capable of supporting. Generally
speaking, as US intensity increases, so does λ. Thus, a US of zero intensity
(i.e., no US) has λ = 0.
Rescorla-Wagner Assignment                                                           2


VALL = the sum of the associative strengths (V values) of all CSs present on a
given trial. For example, if CSA, CSB, and CSC are presented simultaneously in
compound (i.e., ABC), then VALL = VA + VB + VC. On the other hand, VALL for
compound AB is simply VA + VB. If CSB is present, then VALL = VB. The quantity
VALL represents what the subject expects on a given trial, based on the CSs
present. The associative strength of absent CSs does not change, nor does it
contribute to VALL.

(λ - VALL) = The difference between the US obtained on a trial and that expected
on the basis of the CSs present. In most cases, this difference is the
"balance" which remains to be learned.

α = a rate-learning parameter for a given CS. This parameter is an index of
the "salience" of the CS. Although for any particular CS α is fixed, its value
may differ across CSs from 0 to 1.0. A loud banging noise that is very
"salient" would have a relatively high α (e.g., .25). A barely perceptible
noise will have a relatively low α (e.g., .0025).

β = a rate of learning parameter for the US.        A potent US will have a value near
1.0. This may vary from 0 to 1.0.

 APPLICATIONS OF THE MODEL

      To facilitate reading (not to mention typing!), instead of the symbols
CSA, etc., we will identify events with letter names. In the following
simulations, there are four CSs with the labels A (α = .25, light), B (α = .25,
tone), C (α = .1, vibration), and X (α = .25, noise). We assume that β = 1.0,
and that λ = 100 when the US is presented and λ = 0 when the US is omitted.

      Let's create a table for each application so we can "track" changes in the
various components of the model on a trial-by-trial basis. In the first
example, a light (A) and vibration (C) occur on alternate trials and are always
reinforced with the US (+). The last two columns of the table allow us to watch
the growth of the A-US and C-US associations. Each CS's strength only changes
on trials in which it is actually presented and reinforced with the US. We
assume that learning is permanent and there is no forgetting between trials.

      Trial Event λ     VALL    ΔVA     ΔVC    VA      VC

      1     A+    100   0       25      ---    25      0
      2     C+    100   0       ---     10     25      10
      3     A+    100   25      18.75   ---    43.75   10
      4     C+    100   10      ---     9      43.75   19
      5     A+    100   43.75   14.06   ---    57.81   19
      6     C+    100   19      ---     8.1    57.81   27.1
      7     A+    100   57.81   10.55   ---    68.36   27.1
      8     C+    100   27.1    ---     7.29   68.36   34.39
      9     A+    100   68.36   7.91    ---    76.27   34.39
      10    C+    100   34.39   ---     6.56   76.27   40.95

      As you can see the learning process is faster for A than C. This makes
sense because A has a greater "salience" than C. Notice, the greatest changes
occur on early trials.
Rescorla-Wagner Assignment                                                        3


      The greatest strength of the Rescorla-Wagner model has been the successes
it has seen in compound conditioning experiments. Consider how the model
explains Kamin's blocking effect. In blocking, one CS is first trained as a
strong excitor (it predicts the US). A second, redundant CS, then joins the
original CS and the two CSs together continue to signal the US. Lets denote
this procedure as A+ then AX+ to reflect these two stages of training. Kamin
found little conditioning to the added CS, X. Can the Rescorla-Wagner model
simulate blocking?

Group A+ then AX+

      Trial Event λ       VALL    ΔVA     ΔVX     VA      VX

      1     A+      100   0       25      ---     25      0
      2     A+      100   25      18.75   ---     43.75   0
      3     A+      100   43.75   14.06   ---     57.81   0
      4     A+      100   57.81   10.55   ---     68.36   0
      5     A+      100   68.36   7.91    ---     76.27   0
      6     AX+     100   76.27   5.93    5.93    82.20   5.93
      7     AX+     100   88.13   2.97    2.97    85.17   8.90
      8     AX+     100   94.07   1.48    1.48    86.65   10.38
      9     AX+     100   97.03   .74     .74     87.39   11.12
      10    AX+     100   98.51   .37     .37     87.76   11.49

      Note, X has acquired very little associative strength by the tenth trial,
VX = 11.49. A few pages from now we discover that if 5 AX+ trials had occurred
without the prior conditioning of A, then X's strength would have been
considerably higher, i.e., VX = 48.44. Thus, in this simulation, A can be said
to have blocked learning to X. Thus, according to the model, CSs only acquire
associative strength if they provide new information about the US. Conditioning
does not proceed every time the CS and US are paired. Here, X is paired with
the US but is not strongly associated with it because the US is already well
predicted by A. If we had trained A until its associative strength was 100
before introducing the AX compound trials, X would not have acquired any
associative strength at all. The blocking was incomplete in this example
because A had not reached its final level (i.e., VA was not 100).

      Pavlov found that a particular type of compound training procedure allowed
a stimulus to become a conditioned inhibitor (it signals an expected US would
not occur). Using A and B to represent two CSs, lets see what the model
predicts. In this simulation A is the "excitor" and B is the "inhibitor".

Group A+, AB- (trials alternated)

      Trial Event λ       VALL    ΔVA     ΔVB     VA      VB

      1     A+      100   0       25      ---     25      0
      2     AB-     0     25      -6.25   -6.25   18.75   -6.25
      3     A+      100   18.75   20.31   ---     39.06   -6.25
      4     AB-     0     32.81   -8.20   -8.20   30.86   -14.45
      5     A+      100   30.86   17.29   ---     48.15   -14.45
      6     AB-     0     33.70   -8.43   -8.43   39.72   -22.88
      7     A+      100   39.72   15.07   ---     54.79   -22.88
      8     AB-     0     31.91   -7.98   -7.98   46.82   -30.86
      9     A+      100   46.82   13.30   ---     60.12   -30.86
      10    AB-     0     29.26   -7.32   -7.32   52.8    -38.18
Rescorla-Wagner Assignment                                                     4



      Eventually, after many more trials than just simulated, we would find that
VA = 100 and VB = -100. Thus, the model simulates Pavlov's basic conditioned
inhibition finding. In this situation, B is a conditioned inhibitor. It
carries negative associative strength and cancels A's excitation when the two
are presented in combination.

      Now, consider a three group experiment in rabbit eye-blink conditioning
where each group receives the compound AX (e.g., light and noise presented
simultaneously) and paired with a US (e.g., air puff to the eye). These AX
compound trials alternate with another kind of trial. For Group X WEAK,
alternate trials consist of stimulus X presented alone (i.e., without being
followed by the US). For Group X STRONG, alternate trials consist of X paired
with the US. For Group CONTROL, these alternate trials are empty; that is, no
CS or US occurs. Thus, all groups have the same experience with respect to the
AX compound and, particularly, with respect to stimulus A and its contiguity
with the US. Therefore, common sense or "old contiguity theory" might well lead
us to expect all groups to show the same level of conditioned responding to A
after, say, five pairings of AB with the US. When an experiment of this design
is actually conducted, however, the groups differ dramatically with respect to
their conditioned responsiveness to A. The Rescorla-Wagner model provides us
with an account.

Group X WEAK (AX+, X-)

      Trial Event λ        VALL   ΔVA     ΔVX     VA      VX

     1      AX+   100    0        25      25      25      25
     2      X-    0      25       ---     -6.25   25      18.75
     3      AX+   100    43.75    14.06   14.06   39.06   32.81
     4      X-    0      32.81    ---     -8.20   39.06   24.61
     5      AX+   100    63.67    9.08    9.08    48.14   33.69
     6      X-    0      33.69    ---     -8.42   48.14   25.27
     7      AX+   100    73.41    6.65    6.65    54.79   31.92
     8      X-    0      31.92    ---     -7.98   54.79   23.94
     9      AX+   100    78.73    5.32    5.32    60.11   29.26
     10     X-    0      29.26    ---     -7.31   60.11   21.95

Group X STRONG (AX+, X+)

      Trial Event λ        VALL   ΔVA     ΔVX     VA      VX

     1      AX+   100    0        25      25      25      25
     2      X+    100    25       ---     18.75   25      43.75
     3      AX+   100    68.75    7.81    7.81    32.81   51.56
     4      X+    100    51.56    ---     12.11   32.81   63.67
     5      AX+   100    96.48    .88     .88     33.69   64.55
     6      X+    100    64.55    ---     8.86    33.69   73.41
     7      AX+   100    107.1    -1.78   -1.78   31.91   71.63
     8      X+    100    71.63    ---     7.09    31.91   78.72
     9      AX+   100    110.6    -2.66   -2.66   29.25   76.06
     10     X+    100    76.06    ---     5.99    29.25   82.05
Rescorla-Wagner Assignment                                                     5


Group CONTROL (AX+)

      Trial Event λ       VALL    ΔVA    ΔVX    VA      VX

      1     AX+     100   0       25     25     25      25
      2     blank   0     0       ---    ---    25      25
      3     AX+     100   50      12.5   12.5   37.5    37.5
      4     blank   0     0       ---    ---    37.5    37.5
      5     AX+     100   75      6.25   6.25   43.75   43.75
      6     blank   0     0       ---    ---    43.75   43.75
      7     AX+     100   87.5    3.13   3.13   46.88   46.88
      8     blank   0     0       ---    ---    46.88   46.88
      9     AX+     100   93.76   1.56   1.56   48.44   48.44
      10    blank   0     0       ---    ---    48.44   48.44

      The expected results of a test with the A alone after 10 trials follow:
Group CONTROL = 48.44, Group X WEAK = 60.11, and Group X STRONG = 29.25.
Although the numbers we have used are strictly hypothetical, the expected
results for A are quite interesting. Response strength in the presence of A
differs markedly depending on the treatment of X: If X is strongly associated
with the US (Group X STRONG), less associative strength is acquired by A than if
X is weakly associated with the US (Group X WEAK). The control group in which X
was not presented outside of the AX compound lies in the middle. This
demonstrates a principle called the competition hypothesis: CSs compete for the
level of associative strength supported by the US, a result confirmed in an
experiment conducted by Wagner and Saavedra (Wagner, 1969).

      The preceding simulation makes it easy to see how the Rescorla-Wagner
model, which assumes a contiguity mechanism, can account for contingency
learning, if one includes the incidental "background" or "contextual" stimuli
that are inevitably present during learning. For example, a tone (A) that is
paired with a fear-evoking shock produces makes a rat afraid of the tone. But
this pairing occurs in a particular context--a distinct experimental chamber
(X). Thus, the surrounding contextual cues of the chamber are also paired with
shock. If the shock does not occur in the absence of the tone (context), then a
perfect positive contingency would exist (ΔP = +1.0). This situation resembles
the events experienced by Group X WEAK. The context is not reinforced during
the intertrial interval (X-) and the tone and context are reinforced together
(AX+). In this case, the model predicts a short-lived increase in responding to
the context (X) followed by a decline to zero. The tone becomes strongly
excitatory. Alternatively, if extra shocks are presented in the intertrial
interval, this would reduce the magnitude of the positive contingency (i.e., ΔP
is less than +1.0) and conditioning to the tone should decline. This situation
resembles the simulation of Group X STRONG. (i.e., the context is reinforced
during the intertrial interval and this undermines conditioning to the tone.
The important thing to appreciate is that we have an explanation of contingency
learning via a contiguity process. Isn't that neat!
Rescorla-Wagner Assignment                                                     6


Examples:
Assume:

      Beginning Associative Strengths: VA = 100, VB = -100, VC = 0, VD = 50.

      Parameter Values: αA = αB =αC = αD = 0.25, β = 1.0, and λ = 100 (big
      US), 50 (medium US), or 0 (no US)

Example 1: A is followed by a medium US for 1 trial:

                                           (+100)
      Trial   Event    λ    VALL   ΔVA       VA

      1       A+       50   100    -12.5    87.5

      because ΔVA = αΑ β(λ - VALL) = 0.25 X 1.0 (50 – 100) = -12.5

      and thus, new VA = oldVA + ΔVA = 100 + (-12.5) = 87.5

Example 2: AD is followed by no US for 1 trial:

                                                    (+100)   (+50)
      Trial   Event    λ    VALL   ΔVA      ΔVD     VA       VD

      1       AD-      0    150    -37.5    -37.5   62.5     12.5

      because VALL = VA + VD = 100 + 50 = 150

              ΔVA = αΑ β(λ - VALL) = 0.25 X 1.0 (0 – 150) = -37.5

              ΔVD = αD β(λ - VALL) = 0.25 X 1.0 (0 – 150) = -37.5

      and thus, new VA = oldVA + ΔVA = 100 + (-37.5) = 62.5

                   new VD = oldVD + ΔVD = 50 + (-37.5) = 12.5
Rescorla-Wagner Assignment                                                       7


Questions:
For All Questions Assume:

      Beginning Associative Strengths: VA = 100, VB = -100, VC = 0, VD = 100.

      Parameter Values: αA = αB =αC = αD = 0.25, β = 1.0, and λ = 100 (big
      US), 50 (medium US), or 0 (no US)

1) Create a table that shows what is expected to happen if C is paired with a
big US for 10 trials (i.e., C+ trials, λ = 100). (1 point)

2) Create a table that shows what should happen if C is paired with the medium
US for 10 trials (i.e., C+ trials, λ = 50). (.50 point) Compare the results with
this table with the table you created for Question #1. (.25 points) Does the
Rescorla-Wagner model correctly predict the influence of US magnitude on
conditioning? (.25 points)

3) Create a table that shows what will happen to the associative value of A (VA
= 100) if it is followed by “no US” for 10 trials. (1 point) Have experiments
shown that excitors lose positive strength when they are followed by no US? (.25
points). Create a table that shows what will happen to the associative value of
B (VB = -100) if it is followed by “no US” for 10 trials. (1 point) Have
experiments shown that conditioned inhibitors lose negative strength when they
are followed by no US? (.25 points)

4) Assume the BC combination is reinforced with the big US (i.e., BC+). Model
the changes in associative strength of these two CSs during 10 consecutive BC+
trials. (1 point) Compare the trial-by-trial strengths of VC in this question
with the results you obtained in Question 1. To do this, draw a line graph with
trials on the X-axis and strength, Vc, on the Y-axis. There should be two lines
representing the strength of C, one line for Question #1 and one line for
Question #4 (.25 points). What is the name of the phenomenon shown in the graph?
(.25 points)

5) According to the Rescorla-Wagner model, what changes should occur in the
associative strengths of A and D if they are presented in compound and
reinforced for 10 trials with a big US (i.e., AD+)? (1 point) What is the name
of this phenomenon where a loss of associative value occurs despite pairings
with the US? (Hint: see Page 88) (.25 points)

6) Complete the following table (1 point) Do you think the discrimination will
eventually be learned (Vall = 0 on AD trials, and Vall = 100 on each of the A
and D trials)? Why or why not? (.5 points)

                                                 (100)   (100)
      Trial   Event   λ     VALL   ΔVA    ΔVD    VA      VD
      1       AD-     0     ____   ____   ____   ____    ____
      2       A+      100   ____   ____   ____   ____    ____
      3       D+      100   ____   ____   ____   ____    ____
      4       AD-     0     ____   ____   ____   ____    ____
      5       A+      100   ____   ____   ____   ____    ____
      6       D+      100   ____   ____   ____   ____    ____
      7       AD-     0     ____   ____   ____   ____    ____
      8       A+      100   ____   ____   ____   ____    ____
      9       D+      100   ____   ____   ____   ____    ____
Rescorla-Wagner Assignment                                                        8



  7) Model 10 trials in which the AB compound is reinforced with a big US. (1
point) Does VA become abnormally strong with strength greater than 100. What is
the name of this phenomenon? (.25 points)
Rescorla-Wagner Assignment   9

								
To top