Tuesday, March 15, 2011

Learning Theory 101: Extinction

In my post on operant conditioning, I mentioned that one of the possible (though rare) consequences for a behavior is that nothing happens. If nothing happens in response to a behavior during the learning phase, the behavior is typically unaffected one way or another.

However, if nothing happens in response to a behavior after the dog has learned it, extinction can occur. Extinction happens when a previously learned response is reduced or disappears, and understanding the process of extinction- and the accompanying concepts of extinction bursts and spontaneous recovery- can help you affect your dog’s behavior.

Two types of extinction
When a behavior has been learned through classical conditioning, extinction happens when the conditioned stimulus is no longer associated with the unconditioned stimulus. Let’s go back to Pavlov’s dogs, who learned that the sound of a bell (the conditioned stimulus) predicted the coming of food (the unconditioned stimulus), and began salivating when they heard the bell ring as a result. The salivation became an automatic response for them, but if the bell rang and then the food didn’t come enough times, they would stop salivating when they heard it. That would be extinction.

When a behavior has been learned through operant conditioning, extinction happens when the consequences no longer follow the behavior. Remember that behaviors only increase due to reinforcement, so what this means is that if a behavior is no longer reinforced, the concept of extinction tells us that the behavior will go away. Although extinction works for both classically learned behaviors and operantly learned ones, I’ll discuss them in the context of operant behaviors today for simplicity’s sake.

How long does it take?
Extinction can happen quickly, or it can take a very long time. There are three main factors affecting how quickly a behavior extinguishes.

The first is the reinforcement history. If a dog has been reinforced for doing something many times, it will be difficult to convince him that behavior is no longer going to pay off. The second is whether or not there has been a history of a delay between the behavior and the reinforcer. If the dog is used to waiting for his treat after doing something, when a reinforcer doesn’t come at all during extinction, he may assume he just needs to be more patient. The final factor has to do with the reinforcement schedule used. Using an intermittent schedule- that is, only giving a treat sometimes- makes the behavior harder to extinguish because the dog thinks that he might get the treat next time instead.

How do you do it?
If you’re trying to get rid of a behavior through extinction, the first step is to figure out what’s reinforcing it. Then, you need to remove that reinforcer. That’s it!

Sound simple? It really isn’t. We humans are notoriously inconsistent, which means that our dogs are often unintentionally and accidentally reinforced for things. Since intermittent reinforcement strengthens a behavior, making it resistant to extinction, this works against us. For extinction to work, you must be absolutely consistent in not reinforcing the undesirable behavior.

Complicating matters is the fact that some behaviors are self-reinforcing, meaning that the dog gets some kind of internal satisfaction from performing the behavior. Unfortunately, you can’t control that. And of course, the dog gets to define what’s reinforcing, so you may think you’ve figured out what’s driving the behavior, only to find that your efforts are having absolutely no effect on the behavior.

Is that why it’s getting worse?
Surprisingly enough, no. If you implement an extinction procedure and find that the behavior is getting worse, it actually means you’re on the right path. This is due to the absolutely maddening phenomenon known as the extinction burst, which is the sudden and temporary increase in the behavior’s frequency or intensity. It’s easy to understand why this happens when you think about it- imagine you have just put some money in a pop machine, pushed the button, and… nothing. No soda! So you push the button again, then harder, figuring that you must not have done it quite right. That is an extinction burst.

Extinction bursts typically happen almost immediately after you’ve quit reinforcing the behavior, and are usually short-lived. They do sometimes come back intermittently, but each time they do, they’re shorter in duration, have a lower intensity, and happen less frequently. Just ride them out, and eventually you’ll find that the behavior is gone entirely.

A word of caution
Even if you manage to successfully get rid of the behavior, it’s always subject to something called spontaneous recovery, in which the behavior comes back despite the lack of reinforcement. Generally, if you avoid reinforcing it, it will extinguish again rapidly. However, spontaneous recovery does remind us that it is best to prevent undesirable behaviors from starting in the first place.

The bottom line
Keep these things in mind as you train your dog. If you want him to continue to do a behavior reliably, you can create a resistance to extinction by having a long reinforcement history, by delaying the time between the behavior and the reinforcer, and by using an intermittent reinforcement schedule.

On the other hand, if your dog has a long history of jumping up or only sometimes gets food when he counter-surfs, these behaviors will be harder to get rid of, too. With undesirable behaviors, it is always best to manage the situation so that your dog can’t practice the behavior and thus be reinforced for it.

I hope this series on learning theory has helped you develop a decent understanding of how to condition responses in your dog, and how to uncondition them if need be. If not, I hope it's at least been enjoyable to read! I know I’ve really enjoyed writing them- it’s helped really cement some of the concepts in my brain. Although there is a ton more to learning theory, I think I've covered the major points. Please let me know if there are any other aspects that you’d like me to cover- I'd be glad to do it in the future.


andrea said...

Extinction burts drive me BANANAS still .. after many years and KNOWING it's coming they still push every button I have somehow ;)

greta series - thanks ... pats to Maisy

Sam said...

I've done lab work on the partial reinforcement extinction effect and the results are absolutely astounding. Pigeons who are reinforced on an FR1 respond SO much less during extinction trials than pigeons on higher FR or VR schedules. VI schedules also produce a behavior that's resistant to extinction, but VR takes the cake.

Laura, Lance, and Vito said...

I don't use a whole of extinction in my training. The only place I can think of where I use it is with barking temper tantrums. Those attention seeking dogs who bark/whine/have a huge fit since they're not getting what they want. In that case I prefer to stand on the leash so that it's short enough where the dog can pretty much only lie down and be comfortable and just ignore everything. But in some sense you could also call that negative punishment since I"m taking away the dog's freedom.

For most situations I prefer actually training incompatible behaviors and teach the dog what I want them to do instead. In some situations I change the meaning of the old annoying behavior. For example, teach the dog that barking means I leave the room, jumping means I turn my back. Basically extinction plus negative punishment. Or the story that I've used in class (no idea if it's true, got it from another trainer) of a woman whose dog would bark and bark until it got a treat and the person always gave in eventually. She chose to whenever the dog barked to run down the laundry room and open up the dryer to show the dog what was inside. Barking=crazy mom going to laundry room and neg punishment for lazy dog. But who knows if that's true :)

Joanna said...

Re: running to the laundry room and opening the dyer -- I've also heard this technique mentioned a couple of times, I remember the example used being "jump up and open and close the kitchen cabinet". So the dog learns that yes, his barking has a consequence, but it's consistently one that is completely useless to him. It's definitely an interesting tactic, but I've never used it or had a reason to recommend it.

Crystal said...

Sam, have I mentioned how jealous I am of your lab work?

Anonymous said...

Hi Crystal, I stumbled on your blog by chance and look forward to reading about Maisy! I have a 3 yr old lab mix, she is reactive as well mostly on leash. In the last two years she has come a long way in learning impulse control and now actively competes in flyball. We hope to eventually do rally and agility. Your blog looks like it will be wonderful for gathering more info on how to work with a reactive dog.
Patty (lifewithsophie.tumblr.com)

Crystal said...

Hi Patty!

Sophie is adorable! It sounds like you've done a great job with her- I can't imagine doing flyball with Maisy. Way too much stimulation for her! Too bad- it looks like such a fun sport. :)