Docstoc

Basic concepts Reinforcement and punishment

Document Sample
Basic concepts Reinforcement and punishment Powered By Docstoc
					Operant conditioning at the NC Zoo
Category: Brain & Behavior • News
Posted on: September 22, 2008 8:36 AM, by Dave Munger

You might think the zoo is an odd place for psychology bloggers to meet up. But on
Saturday not only did Greta and I get a chance to connect with some of our readers and
fellow bloggers, we also received some fascinating insight into the psychology of
zookeeping. Our group toured the North Carolina Zoo, led by Jayne Owen Parker, Ph.D.,
the Director of Conservation Education of the Zoo Society.

As we strolled from exhibit to exhibit and listened to Jayne's comments, we were struck
by how frequently psychology enters into the daily routine of managing a zoo. Through
operant conditioning, the animals are trained to assist the zookeepers in practically
every zoo function, from feeding, to grooming, to medication and contraception.

Operant conditioning is simply the use of rewards and punishment to modify behavior,
and examples of this process abound at the zoo.

When Jim and Nora were younger, we visited the zoo quite regularly, and one of our
favorite animals was the elephant (or "Dumbo" as our kids called them). But the NC Zoo
provides the elephants with a generous enclosure, and it seemed that every time we
visited, they were at the far end of their space, to the consternation of children who
wanted to get a close look. Don't elephants like toddlers?

On Saturday, we were excited to see an elephant right up near the viewing area:
Jayne told us that this wasn't a coincidence. In the last decade, zookeepers realized that
their feeding schedule was affecting the elephants' behavior. Every day at closing time,
they let the elephants into the barn for feeding. For several hours, as feeding time
approached, the hungry pachyderms gradually sidled towards the gate, so they'd get their
food as soon as the gate opened. So the zoo placed a new gate near the front of the
enclosure, and now at the end of the day the elephants go through that gate and then
back through a chute to the barn to get fed. So now the elephants sidle towards the
excited zoo visitors in anticipation of dinner, and everyone is happy! (By the way, there's
a great article on elephants in this month's National Geographic.)

At the elk and bison exhibit, Jayne explained that these animals would be content to
remain in their large "prairie" enclosure day and night, year-round, even in rain, snow,
and sleet.
But the keepers occasionally do need to get the animals into the barn, for instance if a
tornado or hurricane is approaching. So they trained all the animals to respond to a sound
played over a loudspeaker. Now whenever the sound is played they go to the barn to get
a reward. This is done daily, even though it's not strictly necessary, so that the keepers
know the animals will respond in an emergency situation. In fact, every animal in the zoo
is trained to come to a unique sound, so that they can all be rounded up quickly, or a
single species can be isolated as the situation demands.

Conditioning, Jayne told us, was a much safer and effective way to care for the animals
than older methods like shooting the animals with tranquilizer darts. Even dangerous
animals like grizzly bears can be trained to rear up onto a special apparatus, placing
their claws and snouts in designated spots, and exposing their razor-sharp teeth for
brushing. "The animals love to be trained," she said. The sea lions used to get so excited
in anticipation of a training session that their behavior became unnatural for several
hours each day, so trainers had to vary the training schedule in order to avoid this
problem.

For more on the visit and the meetup, check out Bora's post, which includes links to all
the other bloggers' posts about the event. (And thanks, Bora, for taking the pictures in
this post!)
Basic concepts: Reinforcement and punishment
Category: Analysis
Posted on: August 6, 2007 9:52 AM, by Dave Munger

It would be difficult to come up with a more frequently confused concept in psychology
than reinforcement and punishment. In fact, "reinforcement" and "punishment" aren't
difficult to understand on their own: Reinforcement simply means any means of
increasing or encouraging the designated behavior; punishment is any means of inhibiting
or decreasing the designated behavior.

It was only when B.F. Skinner devised the "positive" and "negative" descriptors that he
became the bane of college students for generations to come. Rather than "positive" and
"negative," things would have been much simpler if he had used the terms "by adding"
and "by removing."

The easiest concept to remember is "positive reinforcement." If you want to train a rat to
push a lever, you can reward it by offering a food pellet every time the lever is pressed.
The rat will learn very quickly to push the lever. This is positive reinforcement, or
"reinforcement by adding."

But people quickly become confused by the term "negative reinforcement." It's much
easier to understand if you think of it as "reinforcement by removing." If the behavior you
want to reinforce is pushing a lever, you can also train the rat by removing something
when the lever is pushed. For example you could play an annoying sound, and pushing
the lever could stop the sound. The rat will quickly learn to press the lever whenever the
sound is played. This is negative reinforcement, or "reinforcement by removing."

Things get trickier when you talk about punishment, or decreasing a behavior. What is
"positive punishment"? It's punishment by adding, remember? Suppose you want to train a
rat to stay in one corner of his cage. You could play the annoying sound whenever it is in
the wrong part of the cage. This is exactly the way "invisible fences" for dogs work: a
sensor on the dog's collar detects the electrical current from the buried wire and plays a
high-frequency sound whenever the dog crosses it. Eventually the dog learns to stay in its
yard and out of the neighbor's flower bed. This is positive punishment, or "punishment by
adding."

And what about negative punishment? That's probably the trickiest concept of all. What
would "punishment by removing" be? Well, think again about training the rat to stay in
one corner. You could attach electrodes to its brain and give it a pleasurable electrical
stimulation. Then when it left its corner, you could remove the stimulation -- that's
punishment for going to the wrong spot in the cage. With a nice enough stimulation, such
as simulating sex, it wouldn't take long to get the rat to stay in its corner all the time.
You could also think of this as positive reinforcement: you're rewarding the rat for the
desired behavior of staying in its corner. However, the two concepts are different. You
couldn't in any way say that the first example of providing food when a rat presses a
lever is "negative punishment."

Each of these can be effective methods of training, but positive reinforcement seems to
get all the good press. Why is that? When used correctly, positive reinforcement can
indeed be extremely effective. Rats learn very quickly to press the correct lever if it
always results in a food reward; you can even train them to press a lever dozens of times
to get a single food pellet. It can also be a great way to teach your kids good behavior: if
you give a young child a sticker every time she brushes her teeth, she'll do a lot of tooth-
brushing.

Punishment, meanwhile -- especially positive punishment -- is discouraged in most child-
rearing advice books. Why? If it's effective for training rats and dogs, won't kids learn in
the same way? The problem is that children live in a much more complex social
environment than dogs and rats. You want your child to stay in his seat at the restaurant,
but you also want to talk to your friends. From your perspective, yelling at him when he
gets out of his chair is punishment, but from his perspective, that's the only time you're
talking to him. While he may not like being yelled at, he prefers it to being ignored, so
from the child's perspective, you're not punishing him for getting out of his seat, you're
reinforcing that behavior. It would probably be better to reward him for staying in his
seat by including him in the conversation, or letting him play with a favorite toy if he
stays seated. (Though this second option is problematic because the ultimate goal is for
children to stay in their seats and not play with toys in restaurants. Maybe you should
just get a baby sitter.)

But there are times when punishment is really the only effective way to train your kids.
You don't want to use a reward schedule as the only way to prevent dangerous behavior
like running into traffic: one mistake can be tragic, and you can't practically reward your
child every time she doesn't run out into the street. A time-out when she doesn't listen to
your warnings or heads in the wrong direction is probably more effective.

Though it sounds "cruel," we have often found very negative punishment to be very
effective. If Jim and Nora were fighting over a toy, the quickest way to get them to stop
fighting was simply to remove the toy ("punishment by removing"). It didn't take long for
them to learn that trying to get a toy by stealing it from your sibling was not effective!
ShareThis

Comments
#1
Alfie Kohn argues against all these behaviorist methods -- particularly rewards -- in his
book, Punished be Rewards. The argument is quite compelling:
Kohn demonstrates that people actually do inferior work when they are enticed with
money, grades, or other incentives. Programs that use rewards to change people's
behavior are similarly ineffective over the long run. Promising goodies to children for
good behavior can never produce anything more than temporary obedience. In fact, the
more we use artificial inducements to motivate people, the more they lose interest in
what we're bribing them to do. Rewards turn play into work, and work into drudgery.
One example that he gives for this phenomenon is that some parents now offer money (a
reward) in exchange for good grades -- but grades are supposed to be the reward. So it
seems that you end up in an escalating arms race scenario as the reinforcement loses its
appeal and you have to constantly sweeten the deal. This effect is probably more
prominent when reinforcement is at the social-isolation/love-withdrawal level than at
lower order needs like the food/water/sleep/pain level, but we'd normally consider the
latter to be abusive.
Posted by: Mike-2 | August 6, 2007 1:13 PM
#2
Mike-2, I had to post my first ever comment in order to correct some fallacies in your
post.
1. As stated by Dave Munger, it is important to understand the term "reinforcement" and
not confuse it with "reward." If the reinforcement is actually "reinforcing," then it must
increase the behavior. If it "loses its effectiveness," it does not mean that artificial
reinforcement does not work; it means that this particular reward has lost its reinforcing
value.
2. In fact, behavior theory predicts this: When reinforcement is presented on a fixed
schedule (you get this every single time you do this), it loses its effectiveness and can be
easily extinguished. For example, if you get A for every B, then A would eventually lose
its reinforcing properties and the frequency of B would decrease.
3. Artificial reinforcement, due to its effectiveness, is often implemented to establish a
new or difficult behavior. Once the behavior has been established, the goal is to reduce
the frequency/artificiality of the reinforcer.
4. As a behaviorist, I would not recommend that a parent reinforce good grades with
money. Instead, I would recommend them to reinforce good study/homework habits
using naturally occurring consequences, such TV/computer/phone privileges.
5. There exist a mountain of published data on the effectiveness of reinforcement.
I would argue that the examples you provide do not prove that artificial reinforcement
does not work. Instead, they're perfect examples of how not understanding the
intricacies of reinforcement/punishment may lead to difficulties when implementing
these strategies in the real world.
Posted by: Terri | August 6, 2007 4:34 PM
#3
the examples you provide do not prove that artificial reinforcement does not work.
Terri, that's a relief, because I never attempted to say that reinforcement doesn't work.
What I am saying is that, assuming that we want our children to grow up to be moral,
hard-working and successful adults who think for themselves, helping them learn to
develop intrinsic motivations is preferable to a purely behaviorist approach of carrots and
sticks, which don't teach anything other than to seek out carrots and avoid sticks.
Rewards and punishments may teach a child to mouth the words 'I'm sorry' for hurting
someone, but they cannot teach genuine remorse or compassion. Perhaps as a
behaviorist, you see no distinction between the two, or simply don't value the inner
qualities. That's fine, but for parents who do, behaviorist approaches are inadequate.
Posted by: Mike-2 | August 6, 2007 7:32 PM
#4
Mike-2, I don't see anything in the post above, or in Terri's comment, that suggests that
reinforcement is the only type of learning that occurs in children. In fact, Dave's
restaurant example specifically states that a behaviorist approach isn't best in all
situations. I'm not sure who's telling you it's the only tool you have as a parent, but
they're not on this blog.
However, there are some things you might want to teach a child before that child is
ready for abstract reasoning. Speech development is one area where reinforcement from
parents is huge. Parents who attend to and repeat verbalizations reinforce them. Later
reinforcement selectively rewards verbalizations that approximate words, frequently by
giving the child whatever (s)he has learned to ask for. It has its place.
Posted by: Stephanie Z | August 6, 2007 9:15 PM
#5
As it happens I am just beginning to read Kohn's Punished By Rewards, so I too had it in
mind reading this article. I'd be very interested to read a review of the book by someone
with the appropriate psychological training (who has actually read the book, not just
someone's short quotes from it, natch :-)
Posted by: Mathematician | August 7, 2007 8:07 AM
#6
Stephanie, the overwhelming majority of parenting books focus exclusively on behavior
modification. The only serious disagreement is whether certain types of punishment or
rewards are more effective -- the traditionalist favoring the former, and the modern,
"permissive" parent favoring the latter.
This blog post follows those modern parenting assumptions, e.g. "the ultimate goal is for
children to stay in their seats and not play with toys in restaurants." To me, the ultimate
goal would be for children to be respectful of others when in public places, and
behaviorist methods are not helpful for that goal.


Posted by: Mike-2 | August 7, 2007 5:31 PM
#7
How can you say behaviorist methods aren't helpful for having children behave or be
respectful in public places? A quick "I like the way you said thank you to the waiter" is
repeating/reinforcing good manners. Also, pointing out how patient and grown up acting
a child is when sitting quietly and waiting his turn to speak is another easy bit of positive
reinforcing a parent can do while in a restaurant. What it takes is a parent who is
attentive to good behavior rather than bad behavior. Behavior modification isn't just
about toys, candy and money - those are low level rewards that should be replaced for
more subtle, social rewards as soon as possible. Punishment should be used sparingly.
Taking a toy away to resolve an argument is fine. Shock collars of any sort should only be
used as a last resort. Of course, the less creative and emotionally in control a parent, the
more likely they are to resort to punishment.
Posted by: guinea pig | August 8, 2007 2:01 PM
#8
How can you say behaviorist methods aren't helpful for having children behave or be
respectful in public places?
I didn't say that. Behaviorist methods can be effective for modifying behavior, they just
don't instill the values that most parents would like their children to have. Most people
recognize that there's a distinction between merely behaving respectfully and real
empathy, and my point is that, by their very nature, behaviorist incentives can't teach
empathy. Rewarding empathy is actually counter-productive, because it focuses the child
solely on their own needs instead of understanding them in the context of others' needs.
Posted by: Mike-2 | August 8, 2007 10:06 PM
#9
Guinea pig, you might be interested to read the book, but basically, the argument is that
even the kind of apparently innocuous behaviourist methods you mention can, in the long
run, *decrease* the likelihood of the behaviour you intend to encourage. Two questions
arise: does it really, and why? To answer the first we need experiments, which of course
are hard to design well; but many are reported in Kohn's book, with references to the
original research. The answers to the second are mostly speculation, but for example, we
may suggest that rewarding the child with e.g "I like the way you..." causes the child to
pay attention to the praise not the action that elicited it. The action gets reclassified as
something which is done to get praise, rather than something which is done for some
other reason. That is, the intrinsic motivation for doing the action is actually decreased.
I think it's complex, and I don't fully understand the issues: don't mistake me for someone
who thinks Kohn reports the One True Answer. I do think it's interesting, though, and a
lot chimes with my experience. For example, independently, in my university we are in
the process of decreasing the amount of summative assessment (coursework which
contributes to the students' final mark) because it is looking as though it's having the
opposite effect to that intended: instead of ensuring that students engage with the
courses, it's actually stifling their enthusiasm. Of course that's a complex issue too, don't
get me wrong!
Posted by: Mathematician | August 9, 2007 5:34 AM
#10
I really enjoy the discussion and posts on this blog. Everyone here is much more well
versed in pyshcology, but I do hear what Mike-2 is getting at. I am hearing that using the
behaviorist's approach can change the surface, but it doesn't help change the deep inner
truths that we want to teach our children. If that is true, then how would one go about
teaching things like empathy, respect, passion, and love? Or are they learned by changing
behaviors?
Thanks for the great reads everyone.
Posted by: The Happy Rock | August 9, 2007 4:46 PM
#11
Interesting thoughts, Happy Rock. I remember being taught empathy and respect though I
can't say what method was being used. One of my earliest memories is of my dad asking
me how I could forget to fill my dog's water bowl because he was totally dependant on
me and totally devoted to me. Guilt! I still have an image of that dog looking up at me
with big sad eyes and to this day I never forget to fill my dog's water bowl at least once a
day.
And, Mathematician, I thought we'd be further along with using behaviorism by now. I
volunteered briefly doing ABA with autistic toddlers. Some of the things that could be
accomplished with these kids by using behaviorial techniques were truly amazing. But I've
also observed people working with disabled adults using techniques I considered
inappropriate for the situation or sometimes they were just too heavy handed. I guess
toddlers are just cuter so people are more motivated to be creative with them. The
biggest problem with behaviorism is that it has as much potential for harm as good and
can be used as a weapon rather than a teaching method.
Posted by: guinea pig | August 9, 2007 9:53 PM
#12
I hope this thread isn't completely 'dried up' yet, because it is a discussion I've been
looking for. I recently read 'Unconditional Parenting' by Kohn, and previously read
'Punished by Rewards.' I have many questions, both as a parent of four and as a
Pediatrician (who regularly dispenses parenting advise for normal as well as
developmentally delayed children).
It seems that Skinner's behaviourism is pervasive in our culture. Most parents I see (in the
rural southeast) use it exclusively. In fact, I notice a strong correlation between old-
school 'behavior modification' and strong arm controlling parenting and religiosity. One of
the main complaints from these parents about the kids is their 'disrespectfulness' (a hard
thing to quantify anyway). Also these parents see Kohn's ideas as simply letting the kids
'run around like wild Indians.' I try to stress the importance of modeling good qualities so
that the children learn their value by example, then expecting what has been modeled
from them, but not manipulating behaviors via reinforcers. Most people don't get it.
The negative effects of controlling parenting techniques are widespread. It starts with
night-time sleeping, then picky eater behavior, then potty training. It moves right in to
conflicts with older children described above (i.e. behavior in the restaurant, etc).
I know it sounds like I'm completely a Kohn supporter, but I do see how reinforcers do
things like teach language, but that is subconcious. I'm talking about artificial reinforcers
i.e. consequences are engineered by the parents, rather than natural. I think kids
respond to being controlled variably. Some may work against the controlling by doing the
opposite of the desired action. Some may grow up and become controllers themselves.
Any comments on the above incoherant ramblings would be appreciated!
Posted by: 7zcata | August 16, 2007 3:57 PM
#13
Sorry, the only other comment I wanted to add is that it seems that B.F. Skinner behavior
mod. techniques work wonderfully well on autistic children. Is this because they develop
abstract thinking later? Is it because our controlling techniques on typically developing
children are more subtle but no less utilized?
Posted by: 7zcata | August 16, 2007 4:02 PM
#14
I probably shouldn't wade too deep into this conversation, but one point I'd like to make
is that Skinner's techniques are extremely difficult to implement correctly. Even with
rats, experimenters must be highly trained to ensure they aren't rewarding the wrong
behavior.
As I mention in the original post, the problem isn't so much that Skinner's techniques
don't work, it's that all sorts of additional variables are present in a real human
environment. Using positive punishment to try to make your child sit in his chair is very
difficult to accomplish in real life.
Posted by: Dave Munger | August 16, 2007 4:06 PM
#15
Dave: so basically, behaviourism may work as a parenting technique, but only if the
parent has been trained to PhD level in it? That would explain a lot :-)
(I saw a cartoon once with the caption "Children are beautiful behaviourists - and so
much better at it than their parents.")
7zcata: could a lot of it be simply that people don't like being controlled by other people
- i.e., regardless of how much they like the thing they're doing or the
punishment/reward, if they realise that they are being controlled by someone else, they
resent it? This feels right to me, and it's consistent with what you say about genuinely
natural consequences being better than engineered ones. Then this would imply I think
that lacking a fully working theory of mind might stop that resentment, and thereby
make behaviourist techniques work better in autistic children (and rats :-) than in
neurotypicals. I've never been sure what people mean by "abstract thinking" in this
context btw, nor why it should be relevant. Certainly by the time children reach abstract
thinking in the Piagetian sense, the parenting-child relationship has an established style -
it seems to make little sense to use behaviourism until then, and then switch.
Posted by: Mathematician | August 17, 2007 5:12 PM
#16
Just because you can, should you? IOW, have you considered if it is morally right to treat
children as lab rats? Would you go out to dinner with an adult loved one, and expect that
person just to sit there, silent, doing nothing, while you had a conversation with another
person?
Posted by: basta | September 7, 2007 11:51 AM
#17
Basta:
Of course it's not a good idea to treat children like rats. For one thing, the cages are
much too small (I'm KIDDING).
Seriously, though, as I mention several times in the post and comments, trying to get kids
to sit still through a meal is a complex and difficult task, and positive punishment is
probably not the best way to accomplish it.
That doesn't mean that all of Skinner's techniques are bad in all situations. Does anyone
have any objection to the strategy of removing a toy when children are fighting over it?
That's negative punishment, and if applied properly, it works.
We also think it's a good idea to teach kids to sit still at the table, since that's part of
having good manners. Including them in the conversation at dinner is an important part
of that process, but some of Skinner's techniques can also help.

				
DOCUMENT INFO