Operant Conditioning II
Review of schedules
• Reinforcement schedule and extinction
• Partial reinforcement extinction effect –
– Persistent responding during extinction after
partial reinforcement training.
Responses per Min
• Behavioral persistence
• PREE in everyday life
– Slot machines
• Why does PREE occur?
– Discrimination hypothesis
– Role of emotion: frustration theory
• Primary frustration
• Anticipatory frustration
Other paradoxical reward effects
• Magnitude of Reinforcement Extinction Effect
– Conditioning with large reward extinguishes faster
than conditioning with small reward
• The Frustration Effect: greater responding
after nonreward than after reward.
Other “emotional behavior”
• Extinction-induced frustration
• Effects of frustration
– Aggression: Azrin pigeon studies
– Enhance behavior
• Adjunctive behaviors – example: schedule-induced
• the frustration effect:
• Extinction spike – increased responding in first
– Terminate behavior
• E.g., negative contrast
• Successive Negative Contrast (SNC)
– Model of anxiety
Other things that affect operant conditioning
• Free reinforcers -degraded contingency effect
– Reduces the level of instrumental behavior
Theories of Reinforcement
• Two questions:
– 1. What makes something effective as a
– 2. How does a reinforcer produce an increase in
the probability of the reinforced response?
• Reinforcers as stimuli
• The Law of Effect
– Only answers question 2.
• Drive Reduction Theory
– Clark Hull
– Relies on homeostasis
– Drive state exists
• E.g., hunger, thirst
– Reinforcer works by reducing drive state
– Importance of motivation
– Does this always work?
• Incentive motivation
– Reinforcers are effective because they increase
– They are arousing
• Brain Stimulation
– Something is reinforcing because it activates a
certain part of the brain – the nucleus accumbens
and other areas.
– Direct stimulation studies
– No deprivation is needed, satiation does not
– Rewards typically generate activity in the Nacc no
matter the type of reward.
• Even pictures of attractive people
Reinforcers as Responses
• Other theories conceptualize a reinforcer as a
– Food is not a reinforcer, the act of eating is.
• David Premack
• What makes eating a reinforcing response?
– Not a drive state. The existence of two responses that
differ in their likelihood of occurrence when the
organism is given free access to both activities.
• Behaviors can be ranked to preference
• Premack principle – The opportunity to perform
the higher probability response will serve as a
reinforcer for the lower probability response.
• Problem: preferred behavior changes over
– Can even be transient
– “what we obtain too cheaply we esteem too
• Premack only answers question 1
• Behavioral regulation theory
– Also responses as reinforcers
– Idea of response deprivation – some responses are free to
occur, some are restricted
– Conditioning situation puts a limitation on certain
– behavioral bliss point – form of homeostasis. How an
organism distributes its activities normally
• Depends on the person
– Instrumental response allows us to obtain behavioral bliss
– Answers both questions 1 and 2
Does Punishment Work?
• Skinner said no.
– Nearly every experiment he did that used
punishment found that it did not reliably decrease
• Subsequent research: It can work. Must be
done in a certain way.
• Must be immediate
• Must be consistent
• Must be severe initially (Immunization to
Side-effects of Punishment
• Increased aggression
– Studies with animals
– Person learns that aggression is a way to cope
• abusive parents come from abusive homes
• Generalization of fear
– Can generalize to desired behaviors
– Ex. Overbearing parents, fear of school, learned
• Punished behavior is not forgotten, it's
– behavior returns when punishment is no longer
• Punishment does not necessarily guide toward
– Combination of punishment and reward can be
more effective than punishment alone