Shedd Animal Training Seminar: Basic Operant Conditioning

In the last hundred years or so, science has learned a lot about animal training. In fact, we have learned so much that Ken stated definitively that training is a technology. That is, the laws of learning are always true, no matter what species we are working with. In that sense, training can be compared to the laws of gravity: no matter what you drop, it will fall downward. Of course training, like gravity, can be influenced by outside factors. If you drop a pen during a tornado, it may fly sideways or appear to hover in the air, but that’s not because gravity has ceased to work. Likewise, the laws behind training are still at work, even when the results are unexpected.

The laws of learning have been broadly grouped into two main categories: operant conditioning and classical conditioning. Ken focused on operant conditioning not only because it is more easily observed and understood by beginners, but also because operant conditioning depends on the animal to think and make choices. Classical conditioning works on a much more instinctive level, and does not result in animals who are actively participating in the training process.

Now, if you’ve ever been to a basic training seminar- or even read a book on the topic- you’ve probably been exposed to the four quadrants. Despite this almost universal approach to explaining the basics of training, Ken didn’t even mention them. This was by design; not only are the quadrants a bit difficult to wrap your brain around at first, but it is also somewhat unnecessary.

What you really need to know about the laws of learning can be summed up in Thorndike’s Law of Effect: behaviors which result in a satisfactory outcome will be repeated, while those that result in discomfort will not. Or, to put it simply, behavior is a function of past consequences.

Consequences come in opposing pairs:
Reinforcing or Punishing
Positive or Negative
Unconditioned (inherent) or Conditioned (learned)
Proximate (immediate) or Distal (in the future)


Beluga whale receiving reinforcement. Photo by Kate Mornement.



Ken believes that the best consequences are the first of every pair; it is far better to have positive reinforcement which is immediate and inherently satisfying. And of all of those consequences, the most important is the use of reinforcement, no matter what form it takes. If you reinforce the behaviors you like, Thorndike’s law tells us that you will see more of those behaviors, which is the ultimate goal of training.

There are three main things to keep in mind when using reinforcement. First, you need to be sure that what you are not mixing up the idea of a reward with reinforcement. Rewards are things we provide that we believe will be an incentive to perform a behavior, but that may or may not actually be something the animal finds desirable. For example, most people consider chocolate to be a great reward, but it gives me headaches, so I would not change my behavior to get some. Second, while inherently reinforcing consequences like food are best, we can certainly teach animals to enjoy and even work for things like petting or praise. Finally, and perhaps most importantly, the timing of reinforcement is the key to successful training; reinforcers should be given as soon after the behavior as possible. Reinforcing in a timely manner with a low-value item will yield better results than poorly timed reinforcers, even if they are very highly desired.

In future posts, we will discuss some of the questions that arise when considering these pairs of consequences: How do you ensure that your timing is good? How do you teach an animal to enjoy something so much that it can be reinforcing? How do you elicit the behaviors you want… and what do you do when the animal doesn’t do what you want? And if you have specific questions about the basics of operant conditioning, please ask in the comments!

Kathy Sdao Seminar: Good Foundations

Pretty much all dog training is based on some form of conditioning, either operant or classical. Since classical techniques tend to be less flashy (read: tedious and boring), most trainers choose to focus on operant ones. Kathy even admitted that while classical conditioning is a bit like watching paint dry, it's so incredibly powerful that it deserves to be the star once in awhile. And of course, Kathy did an amazing job giving classical conditioning the billing it deserves.

Both types of conditioning are built up over repetitions. The difference is that operant conditioning is about consequences, while classical conditioning is not. Operant conditioning seeks to influence the dog's behavior by pairing his actions with rewards like food. The dog only gets the food if he behaves in a certain way. By contrast, classical conditioning doesn't care about what the dog is doing at all- its only goal is to create a direct association between two stimuli. There is no behavioral criteria, and the dog gets the food no matter what he does or doesn't do.

Instead, classical conditioning focuses on the dog's feelings and his reflexes by repeatedly pairing a neutral stimulus with something that elicits a strong reaction from the dog. For example, if everytime a dog hears a bell he gets some food, he'll soon learn to expect some food whenever he hears a ringing sound. Classical conditioning can also be used to change a previously learned association, typically from bad to good. This is called counter-conditioning.

This form of conditioning is incredibly useful when working with fearful, reactive, anxious, or aggressive dogs, but is often poorly understood or glossed over. It does take a bit of time to do it right, and people often try to cut corners, or skip over it entirely. It's a shame, really, because if you can change a dog's feelings about something, the behavior will often change as a result. After all, if children are no longer scary, there's no need to lunge and growl at them, right?

The problem with classical conditioning is that it really doesn't hold up well for the long-term. Yes, you can create associations pretty easily, and you can even change associations with a bit more work, but classical conditioning is incredibly fickle. Unlike operant conditioning, where behavior can be maintained with intermittment rewards, classical conditioning can extinguish quickly if the association is not maintained.

This means that classical conditioning is not a holistic plan. It's simply too exhausting to continually maintain the pairing of scary thing=good thing. The person will get tired, or will screw up. It's also impractical to constantly have treats (or other good things) on you. Ultimately, Kathy said, you'll need to switch to an operant technique, whether that's CAT or BAT or Control Unleashed or whatever.

So why bother with classical conditioning at all? Why not just cut straight to the chase and use an operant strategy from the start? The biggest reason to do this is because most dogs with fear/reactivity/anxiety/aggression issues are okay... until they're not. And usually, once they're “not,” they've gone over threshold, and are now in that fight-or-flight mode where they can no longer think. Then it's too late. You can't teach the dog anything. You can't reinforce an acceptable behavior because the dog isn't physiologically capable of learning anymore. Operant conditioning becomes impossible.

So, you start at the beginning. You pair the sight of a trigger with good things, over and over again, no matter what the dog is doing, so that the dog no longer sees the trigger as a scary thing, but rather, as a thing which results in chicken. You'll know this association has happened when the dog gets demanding- he'll see the trigger, and instead of reacting, he'll be obnoxiously nudging your hand or staring at your chicken pocket. And since he wants that chicken so badly, it's pretty easy to ask him to do something to earn it.

In other words, what classical conditioning does is give you the foundation on which to build those operant behaviors. It helps your dog relax enough to think. It gives you enough time to intervene. It gives you the space you need to begin training the behaviors you want instead of constantly focusing on the ones you don't.

Of course, it's not as easy as I make it sound. If it was, Kathy wouldn't have been able to spend a full day discussing classical conditioning. Indeed, Kathy shared some really interesting- and important- tips to make the most of classical conditioning. If you're going to go through all that work, you might as well do it right! I'll share that info with you in the next post.

In the meantime, I'd love to hear from other people with reactive dogs. Did you do classical conditioning as a foundation? What was your experience like?

Clicker Expo 2011 (Chicago): Eva Bertilsson and Emelie Johnson Vegh- Let's Make Some Noise!

Maisy hates things that move under her feet, and she's not too fond of strange noises, either, so when I heard about this session on building confidence in dogs by using noise and movement, I knew I had to go. At the same time, I was also a bit skeptical. My general experience has been that when dog people talk about building confidence in dogs, it's in mildly worried dogs, not the truly terrified ones. I'm often left with the feeling that, yes, that might work... but not for my dog.

I did not leave Eva and Emelie's presentation feeling that way. To the contrary, I felt like they really, truly get what an anxious dog is all about. I believed that they have met dogs like Maisy and worked through their issues. They did a great job of breaking down the concepts, explaining ways to make it easier, and offering alternative solutions. I was super impressed with them, their presentation, and their ideas.

Eva Bertilsson and Emelie Johnson Vegh are agility people, and their book, Agility Right From the Start, has received a lot of positive buzz. So naturally, they are concerned about dogs learning to love noise and movement because of the agility obstacles, especially the teeter. However, they were quick to say that their session was about more than just agility, and that noise and movement is not simply a distraction to overcome. Our lives are full of noise and movement, and if a dog doesn't learn to enjoy it, his quality of life suffers.

Although they did discuss applications to agility, and used agility-related videos, as a non-agility person, I did not feel this detracted from the presentation in the least. They did a great job of presenting the concepts of noise and movement as something people like me could use to help their non-agility dogs. Again, I was so impressed.

Eva and Emelie shared that there are several ways to approach noise and movement with dogs who are cautious or fearful of it. The first is habituation, in which the dog simply ceases to respond to the stimuli. When a trainer helps a dog to habituate to noise or movement, the goal is for the dog to not mind or not notice when it happens.

The second method is to teach the dog to like the noise or movement through classical conditioning. This isn't a bad approach, and in fact, Eva and Emelie said you need to do this step before you can move on to the next one. Basically you teach the dog that when they hear a noise or feel something move, great stuff will happen. It's not contingent upon the dog's behavior; the dog is simply experiencing awesome treats and super fun games and toys as a result of noise or movement.

Eva and Emelie shared that this is a great thing to do for all agility dogs, even the ones with a solid temperament. If they love noise and movement, they'll be eager for it to happen, and you'll get a far more enthusiastic performance from your dog. For the worried dogs, though, it's vital. When you are pairing the noise and movement with good things, you'll need to gauge whether your dog was simply cautious or whether he was truly scared. Depending on where he's at, you'll need to adjust your training. Dogs on the cautious end of the spectrum can receive relatively low-value treats and work for many quick repetitions. Dogs on the more fearful end of the spectrum, though, might only be able to do one or two trials a day, and will need the awesome thing- whether treats or play (and they prefer play because it's a higher energy activity that results in a better, happier response)- to happen for a very long time.

With conditioning like this, it is vital that you get the order correct. The noise or movement happens first, then the awesome thing. This is especially true if your dog dislikes noise and movement and you do it the other way around, you'll associate the treat or the toy with bad feelings. They called this “backwards conditioning,” and said it will cause you far more headaches in the long run.

Another suggestion for truly worried dogs, is to be sure that you never use noise or movement as an aversive. Don't say “ah-ah,” don't make “sst” noises, don't yell “no!” Doing so will work against you. Your goal is for noise and movement- any noise and movement- to predict great things, not punishing things, no matter how small that punishment might be.

The third method- and the one Eva and Emelie prefer- is to teach the dog to create and then demand noise and movement through operant conditioning. While classical conditioning is controlled by the handler, operant conditioning gives control to the dog. Since worried dogs often feel like the world is out-of-control, giving them the power to exert influence on their world can be transformative.

They start by simply teaching the dog how to create the noise through the handler. They do this by treating the noise or movement as a secondary reinforcer, just like a clicker. They will shape behaviors using a noise or some movement in the place of the click. You can use any noise- jingle some keys, or hit two spoons together, or shut a cupboard door- and then follow it up with a treat. Using movement is a bit more difficult, but they recommended having the dog on a very small wobble board or even in your lap, and just moving slightly in place of the click. This results in a dog who wants the noise or movement to happen. It also lets him control how often the noise happens. They did say it should be offered, voluntary behavior, because if you use a cue to get the behavior, it can both wreck the cue (because the cue will predict something scary), and it takes away the dog's choice.They demonstrated this concept with pretty simply behaviors- turning, raising, or lowering a head, for example.

Next, they teach the dog how to create the noise and movement on his own. They shape the dog to do things that like knocking over a book or a soda can, putting metal spoons in metal buckets or on baking trays, etc. Start small- knocking a book over onto carpet is a fairly muffled noise- and build up to larger noises, like knocking over huge stacks of pots and pans. You can choose to click either the dog's movement or the product of his movement (ie, the noise/movement). Eva and Emelie said it doesn't really matter which, as long as you are consistent.

Finally, they work towards having the dog demand the noise and movement. This is really just another level of “create,” but it's a more intense, active version. During this phase of training, you will make creating the noise or movement more difficult by adding resistance. For example, if you're teaching him to shut a cupboard noise (which will both move and bang when it closes), you might hold it lightly so that the dog must push it with more intensity.

No matter which level you're working at, you want to keep your sessions very short, with very slow escalation in the amount of noise or movement. They recommend doing 1 to 3 reps at a time. Any more than that and the dog's brain starts to produce adrenaline because of the fear reaction, which makes it harder for him to learn from the process. They also recommend varying the intensity. The first rep might be very easy, the second quite hard, and the last somewhere in between.

Always watch your dog's expression as you start a session. Eva and Emelie said that the way a dog approaches a session tells you more about how he's feeling than the way he reacts during the session. That said, if you see any signs of worry during a session, do a very easy rep, and then take a break- a long one. They recommended taking breaks of a few days to a few weeks long so the dog has time to forget that it was scary. Remember the goal is to make noise and movement fun for your dog, not simply something to tolerate. Always make it a fun, easy game.

These are some pretty cool ideas, and I know my words do not do their concepts justice. How can you boil down 90 minutes of brilliance into a thousand words? Like I said- I walked away really believing that this would work with Maisy, and that doesn't happen too often! Their video was amazing, and really helped clarify how to use these concepts.  Still, I hope that this at least gives those of you with cautious or fearful dogs some ideas of how to proceed. I'd love to hear from anyone who has tried these ideas!

Learning Theory 101: Operant Conditioning

As I mentioned earlier in this series, according to the behaviorism branch of learning theory, learning takes place through two types of conditioning. I’ve already told you about classical conditioning, so today it’s time to talk about operant conditioning!

History
Although the term “operant conditioning” was coined by B.F. Skinner in the 1930s, it actually has its roots much further back. In 1905 (only two years after Pavlov presented his first paper on classical conditioning), Edward Thorndike published his Law of Effect. This law basically says that behaviors that have good consequences will happen again in the future, while those that have bad consequences will be less likely to happen again. Despite the fact that the Law of Effect really sums up operant conditioning quite well, Skinner expanded so much upon Thorndike’s work that it is sometimes called Skinnerian conditioning (it is also sometimes called instrumental conditioning), and Skinner is widely referred to as the father of operant conditioning.

Definition
Like classical conditioning, operant conditioning is the process of forming associations between two things. However, while classical conditioning forms a direct and reflexive association between two stimuli, operant conditioning works by forming an association between a voluntary behavior and the subsequent consequences.

In other words, the dog learns that his behavior causes stuff to happen, so he either repeats it or avoids it in the future. Although classical conditioning creates an automatic response, operant conditioning implies that the dog thinks about his behavior, and then deliberately and voluntarily acts in his own best interest.

It is called “operant” conditioning because the dog is operating on the environment (the technical terms for “doing stuff”).

Consequences for (Almost) Every Action
Since operant conditioning is defined by the idea that consequences drive behavior, we need to know what the possible consequences are. Logically, there are three options: something good will happen, something bad will happen, or nothing will happen at all. It’s rare that nothing happens at all. For example, if you are standing and choose to sit, it may look like nothing has happened as a result, but in reality the stress on your joints has changed. Whether that’s a good thing or a bad thing is another matter entirely, but the point remains: most behaviors are followed by some sort of consequence. As a result, we will discuss the other two consequences today.

This means that the core tools of operant conditioning are the good consequences and the bad consequences that happen. When something good happens following a particular behavior, the dog is more likely to repeat that behavior. This is called reinforcement. When something bad happens after the behavior, the dog is likely to avoid doing that same behavior in the future. This is called a punishment. All of dog training- heck, all learning by any species- happens because a particular behavior was either reinforced or punished.

The Good Lord Giveth, and the Good Lord Taketh Away…
Both reinforcement and punishment can happen in two ways: something can be added to the situation, or something can be taken away. These two eventualities are respectively referred to as “positive” and “negative.” Since we tend to think of these words as value judgments, this is probably the hardest part of learning theory to understand. It is therefore important to understand that in this case, “positive” doesn’t mean good, and “negative” doesn’t mean bad. Instead, think of these two terms in the mathematical sense: something has been added or something has been removed from the situation.

As a result, there are four possible consequences following a dog’s behavior:

Positive Reinforcement (R+): This is where something is added to the situation (positive), and the behavior increases as a result (reinforcement). The behavior increases because whatever added was desirable, and the dog wants it to happen again. You can think of R+ as a reward.  
Example: You ask your dog to sit. He does, so you give him a treat. Next time you ask, he sits again. Because you added a treat and the sitting behavior increased, it is R+.

Positive Punishment (P+): This is where something is added to the situation (positive), and the behavior decreases as a result (punishment). The behavior decreases because the thing that was added was unpleasant, and the dog doesn’t want it to happen again.  
Example: You are walking your dog, and he pulls on the leash, so you give a collar correction. Next time you walk with him, he doesn’t pull. Because you added a collar correction and the pulling behavior decreased, it is P+.

Negative Reinforcement (R-): This is where something is taken away from the situation (negative), and the behavior increases as a result (reinforcement). The behavior increases because whatever was taken away was annoying or unpleasant, and the dog is glad it’s gone. You can think of R- as escape or relief from something unpleasant.  
Example: You ask your dog to sit. When he doesn’t, you pull up on the leash, putting pressure on the collar around his neck. You maintain this pressure until the dog sits, then release it. The next time you ask him to sit, he does. Because you took away the pressure and the sitting behavior increased, it is R-.

Negative Punishment (P-): This is where something is taken away from the situation (negative), and the behavior decreases as a result (punishment). The behavior decreases because whatever was taken away was awesome, and the dog wants to keep it next time. You can think of P- as a fine or penalty.
Example: Your dog jumps up on you because he wants your attention. You ignore him and walk away. Next time your dog wants attention, he doesn’t jump on you. Because you took away your attention and the jumping up behavior decreased, it is P-.

Putting it all Together
It is important to remember that the dog defines if the consequence is reinforcing or punishing. While you may think you’re giving the dog something awesome, he may disagree. For example, some dogs love carrots. Maisy doesn’t. If I offered her a carrot as a positive reinforcer, I might be surprised when she walked away instead of doing what I asked. On the other hand, Maisy loves to be squirted in the face with water- something that is often recommended as a punishment. If I squirted her in the face to get her to stop barking, it’s very likely that the barking behavior would actually increase. The ultimate test of whether something was a reinforcer or a punisher is if the behavior increases or decreases in the future.

I’ve also found that the quadrants tend to work in tandem. For example, the standard advice for pulling on leash is to “be a tree” when the dog pulls. By stopping the forward movement, we are hoping to decrease the pulling by taking away the dog’s ability to get where he wants to go (negative punishment). When he stops pulling and instead moves back to loosen the leash, we start moving forward again, thus increasing the behavior of keeping the leash slack by adding in movement (positive reinforcement).

Because more than one quadrant is being used for the same skill, it can be difficult sometimes to figure out which one is at work. Therefore, while understanding operant conditioning will make you a better trainer, I don’t see the need to over-think things. Don’t worry about which quadrant you’re in, and whether or not it’s “acceptable” to use it. Instead, focus on the results you’re getting.

If you aren’t getting what you want, consider why that is. Are the consequences that you’re providing actually reinforcing or punishing? Is there something else at play? Unfortunately, you cannot control every consequence. Some behaviors are self-reinforcing, and you’ll need to think critically and act creatively. By understanding operant conditioning, you’ll be able to more easily parse out what’s going on and then get the results you want.

Finally, timing counts. No matter which quadrant you're using, your timing must be good. The consequences need to happen as soon after the behavior as possible so that the dog can make the association between the two. The process works best when the consequence comes immediately after the behavior. This is part of why clicker training has become so popular: it allows the trainer to bridge the time between the behavior and the consequence, making it clear to the dog why he's being rewarded.

Still, no matter how you train, no matter what methods you're using, operant conditioning is at work. If you like what your dog is doing, reward him for it! If you don't, punish him. Just remember that punishment doesn't need to be painful or scary. I much prefer negative punishment- removing things the dog likes- and have found it very effective.

I hope this post has helped you understand operant conditioning better. It's a complicated subject, but I find it fascinating!

Sources
In addition to the links in this post, you may find the following websites interesting:
This site has a nice summary of operant conditioning, plus a bonus video of a pigeon in an operant conditioning chamber (also called a Skinner Box).
A Brief Survey of Operant Behavior, by B.F. Skinner.
An in-depth historical look at operant conditioning.
Learn more about Thorndike here.
Two great posts by Patricia McConnell: Are you all positive? and The Positives of Negatives and the Negatives of Positives.

Ian Dunbar Seminar: Providing Feedback to Our Dogs

One of Ian’s biggest criticisms of dog training today has to do with how we are providing feedback to our dogs. He believes that the vast majority of training is done with what he calls “Non-instructive Quantum Feedback,” while dogs would be much better off if they received “Instructive Analog Feedback.” It’s an interesting distinction, and today I’d like to spend some time explaining the difference, and why Ian’s so adamant that we switch over to the latter.

First, let’s break down what he means by these terms, starting with “Non-instructive Quantum Feedback.” “Non-instructive” means that the training simply provides consequences, either desirable (click and treat) or undesirable (collar correction), but does not explain why the action was correct or incorrect, nor to does it explain what the dog ought to do in the future. “Quantum” means that that the feedback can be counted in measured in some way. This makes it a simple response, and devoid of emotion.

“Instructive Analog Feedback,” on the other hand, is basically the opposite. “Instructive” means that we tell the dog what we want, either prior to the behavior (by using a lure), or after the dog does it wrong (by explaining what we wanted instead). “Analog” means that the feedback expresses value, which is difficult to measure. Ian does this by using his voice.

You will notice that this does not reference the four quadrants in any way. As I’ve previously written, Ian finds most of learning theory to be unnecessary to dog training, and he says the quadrants fall into that category. Instead of looking at feedback in one of those four ways, he sees it as binary: things either get better, or they get worse.

Instead of worrying about terminology (which is pointless anyway, because your intention and the dog’s perception may not match up), Ian says there are three types of feedback that people use. We reward the behavior somehow, we punish the behavior somehow, or we do nothing. Similarly, there are three ways of combining this feedback.

First, there’s the “old way,” of punishing all incorrect behaviors, but doing nothing when the dog gets it right. In this example, the dog often has no clue why he’s getting punished. It also relies on the use of painful consequences, such as a collar correction, which Ian says (and I agree) are unnecessary for dog training.

Then, there’s the “new way,” of rewarding the desired responses and ignoring the rest. Shaping falls into this category, and Ian isn’t a fan of it. He believes that dogs find shaping frustrating due to the lack of feedback when the trainer is silent. While I certainly believe that it would be frustrating to go “several minutes” without a click, if you find this happening during your shaping sessions, you’re doing it wrong. Shaping should split the criteria up into tiny increments; I once heard Kathy Sdao say that, during a shaping session, you should be clicking roughly every three to five seconds.

Ian also said that he doesn’t like shaping because if you click the wrong thing once, the dog will persist with that action, and you’ll be stuck there for long periods of time. Certainly you get what you click, but a single mis-click is pretty easy to overcome, especially if your criteria is split out well and your rate of reinforcement is high.

Finally, there’s the Ian’s preferred method: using both rewards and punishment. He believes using both helps the dog figure out the task faster. Keep in mind that he believes it’s possible to punish the dog without pain, so he is not combining collar corrections with treats. Instead, he’s using feedback that is instructive and analog, and he uses his voice to accomplish this.

Basically, he uses his language and his emotions to provide feedback to the dog. Not only can he tell the dog if he’s right or wrong, but also how well he did. This allows Ian to let the dog know whether the behavior was average or if it was truly exceptional. It also allows us to inform the dog how serious his misbehavior was, ranging from, “that wasn’t quite it” to “holy crap, that was dangerous!” Ian says this allows the dog to receive far more information about his actions than a simple click and treat, or a buzz and shock from a collar.

Personally, when I’m teaching Maisy new tasks, I use primarily shaping. I do not tell her when she’s getting it “wrong”- but then, I don’t think a dog can be wrong when shaping anyway, since the whole point is that the dog is supposed to offer behaviors and you choose the ones you want to work with. I have tried using no reward markers- when you tell the dog they’re on the wrong track- but I found that they cause Maisy to give up. However, I do add verbal feedback when Maisy does something amazing. Click- jackpot- and lots of praise. Although Ian seems to believe clicker trainers don’t do this- and maybe purists don’t, I guess I don't know for sure- I don’t see why I can’t use my voice in conjunction with clicker training.

As for what I do when Maisy performs a known behavior incorrectly, well, that’s a discussion for another day. Ian talked a lot about how he approaches this: he uses what he formerly called an “instructive reprimand.” However, once he found out that people interpreted “reprimand” to mean something harsh, like yelling, he renamed the method “repetitive reinstruction as negative reinforcement.” This is a very interesting method, and it deserves its own post. I’ll do that very soon.

In the meantime, I’d love to hear about how you give your dog feedback, especially when teaching new behaviors. I know there are many ways to train, and I can’t wait to hear how different people approach this!

Ian Dunbar Seminar: Is Learning Theory Useful for Dog Trainers?

Ian Dunbar is not a heretic- he wants you to know that. Operant conditioning is real. It’s been proven. The experiments behind the theory were repeated thousands of times, and they were validated. It’s great science.

It’s just not useful for dog trainers. Or at least, not most of it. Ian believes that only about 10% of learning theory applies to us because learning theory was laboratory-generated. That is, the experiments were implemented, monitored and controlled by computers, carried out using Skinner boxes, and the animals used were “simple” and had “few interests.” In contrast, we are humans, and we train real dogs in the real world.

If we choose to use learning theory in training, then we must learn to train like a computer. That’s not necessarily bad- computers have some pretty good traits. They are tireless. They are completely consistent in both monitoring the trainee’s behavior and in providing feedback. This allows them to have very clear criteria. By contrast, we humans often have unrealistic or unclear criteria, and are inconsistent in our observations and feedback. So, why wouldn’t we want to train like a computer?

Well, computers as trainers have some drawbacks. They can not qualitatively assess an animal’s performance- they can’t see cute or flashy behaviors and train that into the final product. While they can give feedback, they cannot give instructive feedback. All a computer can do is say yes or no- provide a click and treat or a buzz and shock. They cannot explain why the animal was wrong or what he should do instead, and they cannot explain how important or urgent compliance is. Humans can.

Then there is the matter of reinforcement schedules. Ian identified seven reinforcement schedules: continuous, fixed interval, fixed ratio, variable interval, variable ratio, random, and differential. Ian explained that six of these seven schedules will maintain a behavior, but only one will improve behavior. The one that will improve behavior- differential reinforcement- is the one that computers cannot use. (Personally, I disagree. Computers may not be as good at it as we are, but there is no reason a computer couldn’t reward faster responses. In fact, they might be better at that than I am- I do not have a stopwatch in my head.)

Because we humans cannot be as consistent as computers, and because computers cannot provide instructive feedback the way we can, Ian sees no need for us to try to emulate computers. He finds this to be especially true because most dog owners don’t need nor want the precision that comes about from training like a computer. As a result, he really doesn’t have any use for the vast majority of learning theory.

So what does Ian like? Thorndike’s Law of Effect, which more or less says you should reward the good stuff and punish the bad stuff. Ian says this is simple, elegant, and pure. It doesn’t get into complicated and confusing types of rewards or punishment which cause endless arguments on the internet. Thorndike tells it like it is.

Again, Ian’s orientation as a trainer of pet dogs is obvious. The average dog owner doesn’t care about precision, and doesn’t have the consistency or patience needed to sort through the various quadrants and schedules, so I understand why Ian thinks we should avoid discussing learning theory with clients. We need to quit worrying about the science and terminology and just train. We should help them, not confuse them. It’s hard to argue with that.

Still, as a dog geek, I struggle with this idea. Personally, I enjoy understanding the science behind what I’m doing. Ian said it himself: learning theory is valid. I like thinking about what I’m going to do. I love planning my sessions. I also think it’s fun to take data and evaluate what I’ve done with the goal of doing better next time.

As a competitor, I want precision. I enjoy the challenge of being consistent enough to get amazing results. I strive to be as clear as possible in my criteria. In many ways, I do try to train like a computer, and I don’t think that limits me. I enjoy pairing clicks with not only treats but also heart-felt praise when my dog does something exceptional. I see no reason to have to choose between computer or human. That’s sort of the beauty of being human, after all: I can think outside of the box and combine the best of both approaches.

I know I’m not the normal dog owner. I spent Halloween weekend at Ian’s seminar, and I’m spending hours writing up my notes for this blog, after all. I would ask all of you if you’re normal dog owners, but I suspect I know the answer to that. You are reading this, after all.

Instead, go ahead and analyze what Ian said. Tell me how it makes sense, and then how it confuses you. Tell me how you use, or don’t use, learning theory with your dog. I know you’ll have lots to say!

Small Changes Yield Big Results

She might have her chin down, but she's sure not relaxed!

I’ve known for a long time that Maisy is very sensitive to my moods. At her appointment with the veterinary behaviorist on Monday, Dr. Duxbury saw that, too. She said in her report that during the appointment, “Maisy was exquisitely responsive to [me] but most of her 'relaxed behaviors' (e.g. resting her head on floor, laying on one hip) appeared to be highly reinforced operant responses vs. truly relaxed.”

She went on to acknowledge that this is a tricky area in which to work; while you need to start somewhere, it’s easy to fall into the trap of operantly relaxed, where it’s clear that Maisy is “on” and working, and not actually relaxing. With that in mind, Dr. Duxbury made some suggestions on how I can tweak my current handling in order to promote real relaxation as the medication begins to work.

Relax
Unsurprisingly, Dr. Duxbury said that because Maisy takes so many cues from me, I need to be make sure I’m relaxed. She advised me to make a conscious effort to breathe and relax my neck and shoulders. This is much easier said than done!

Slow down
Dr. Duxbury told me that I need to slow down both my physical and verbal interactions with Maisy. She highly recommended that I don’t use a marker with Maisy because it “keeps the response very operant,” and “interrupts and… ends the behavior.” I quit using a clicker for behavior modification long ago, but I think I will reduce my use of a verbal marker, too. I will still praise Maisy occasionally, but Dr. Duxbury cautioned me thatI need to slow down and speak softly so that I’m not so exciting that it encourages Maisy to get up.

Likewise, I need to be mindful of how I deliver food rewards. She said I need to be much slower in my hand movements, because I have a tendency to be “so quick with [my] food rewards that [I] encourage quickness in her response.” A lot of my quickness has developed because I want to get the treat to Maisy while she’s still performing the behavior. Dr. Duxbury acknowledged that I’m right to be concerned about this timing, but recommended using negative punishment- that is, the removal of the food treat if Maisy tries to leave her relaxed position- to help ensure I’m rewarding the position I want.

I tried this out in my reactive dog class on Tuesday, and it worked fabulously. I put Maisy in her crate, and when she moved into a more relaxed position, such as lying her chin on the floor, I very softly said “gooooooood giiiiiiirl.” I then offered her a treat, using slow, fluid motions. Predictably, she would lift her head or even jump up. When she did, I (slowly) moved my hand away. When she put her chin down, I again moved (slowly, fluidly) to give her the treat. Initially, I allowed very small movements to get the treat, but by the end of class, Maisy was able to lie much quieter as she took the treat. Incidentally, she also rolled over flat on her side, and was possibly the most relaxed I’ve ever seen her at class- even her legs (which usually stick straight out without touching the ground) were becoming soft and loose.

Rest
Dr. Duxbury said that while it has previously been necessary for me to manage Maisy by providing her with a lot of stimulation (such as through training or playing ball), I need to be careful that this doesn’t prevent her from resting. I will need to reduce the amount of management I do, especially the number of treats I give, and I will need to start allowing her to entertain herself.

To that end, while I should continue the relaxation protocol, I need to do it both in her crate and outside of her crate/on a mat. She said these should be considered two completely separate exercises. Further, she recommended routinely encouraging Maisy to rest quietly on her mat for 20-30 minutes a day instead of trying to keep her engaged in an activity. As the medication begins to take effect, I will hopefully see Maisy shift over into actual resting instead of operant relaxation. When I see this, she advised me not to interrupt her because “rest is also its own reward, and [I] can just allow it to happen.”

Balance
Finally, I need to remain attentive to Maisy’s needs. If there is a lot going on in the environment, I may need to switch back to management temporarily. As she wrote, “it is appropriate for [me] to move her away [from the situation] and fire treats at her as rapidly as necessary to reward her for mentally staying with [me].”


I really appreciated that Dr. Duxbury took quite a bit of time during our appointment to help me practice these steps. I learn best by doing, so even though all of this is stuff I’ve been told to do before, having her coach me on how and when to give treats was very helpful.

I am very encouraged by the success we had at class this week, and so I think I’m going to start over with the relaxation protocol. Maisy’s response has been very operant, and that is not at all what I want with it. Instead of following the program so rigidly by treating at the completion of each exercise, I will try to treat only for relaxed responses. I’ll continue with the slow, fluid treat delivery, too. I’m very excited to see how this goes!

Pat Miller Seminar: Two Ways of Treating Aggression (and Reactivity)

My last entry on the seminar was quite awhile ago, but I really did want to say more about it! Since this blog is about reactive dogs, I wanted to share what Pat said about them.

Pat says that almost all aggression and/or reactivity is based on some sort of stress, whether it’s because someone is threatening the dog’s territory or resource, or because the dog is scared or unsure. Since it’s based in emotions, most people choose to treat it classically, but in recent years, some people are adding operant components to the treatment. Let’s analyze the two, shall we?

Classical Conditioning
Since aggression is based on stress, it makes sense to treat that underlying emotion. If, for example, the dog displays aggressive behaviors when he sees a big scary dog, we need to teach him that bog scary dogs actually aren’t scary. In fact, they’re awesome! We do this by pairing something awesome (usually high value treats) with the sight of the big scary dog.

In order to do this effectively, we must keep the dog calm enough that he can see the trigger and eat the tasty treat. If he displays aggressive behaviors, we need to reduce the stimulus (usually by moving further away) and increasing the value of the treat. Pat recommends working with a single stimulus, at the same intensity, for 20 minutes or longer before switching to a new trigger or a new level of intensity. This helps the dog learn that seeing a big scary dog is actually a predictor of good things.

Sometimes, detractors will say that this is foolish, as we are simply bribing our dogs. Done correctly, this is not the case. We aren’t trying to prevent a reaction or distract the dog. We aren’t holding the treat out as a lure. Instead, we are allowing the dog to form a new association.

Operant Conditioning
Over the past ten years or so, various trainers have been coming up with ways to treat aggression in an operant, yet still “positive” manner. Pat talked about one of the first of these methods: CAT, short for Constructional Aggression Treatment, and showed a video of her doing CAT with a dog named Juni. I’d seen the video before, at the last seminar I saw with Pat, and I have to admit: I didn’t like the procedure then.

To do CAT, you stand with your dog (or you tether him), and bring a trigger to his threshold. You don’t go over the threshold, you simply bring him to the point of noticing, but not reacting. When the dog offers a socially appropriate behavior, or when the dog reduces the intensity of his aggressive behavior, the trigger retreats. Basically, you’re shaping calmer behavior, with the ultimate goal of “switch over,” which means the dog begins to offer affiliative behavior.

The reason I objected to CAT before was because it upset me when the dog did go over-threshold (and anyone working with a reactive dog knows that despite your best efforts, it happens). I didn’t like that the handler ignored her dog while he was upset, either. I also thought it looked like it could be flooding. On seeing the video a second time, though, I concluded that it probably wasn’t. To truly be considered flooding, the dog must be exposed to the stimulus at full intensity, and that exposure must remain until learned helplessness occurs. In other words, the dog would have to shut down emotionally. However, this is not what happens in CAT; it’s a highly regulated level of intensity, and the dog’s behavior controls whether the stimulus remains or not.

CAT is not without its critics, and there are even offshoots of the procedure attempting to make it more positive. Still, I’m no longer opposed to it like it was. Although I don’t think it should be the starting place for treating aggression, it is a useful tool for some dogs.

That, and… during the seminar, I found myself wondering: is this what I did with Maisy? I know that my plan wasn’t exactly CAT, as I did counter classical conditioning along with it, but when I did misjudge a threshold and she went over, I simply ignored it, and waited for a moderate duration (ten to twenty seconds) of calm, appropriate behavior before I rewarded her with attention or treats.

It worked, and it worked quite quickly. It did cause some stress and confusion in the short run, so I suppose that it was part of that 1% of the time where I’m not strictly “positive,” but Maisy quickly reached “switch over,” and has been pretty amazing ever since then. She seems far less stressed and much more interested in social interactions.

Anyway, I really enjoyed hearing Pat speak about aggression again. It’s amazing how much a year’s worth of reading and experience and, well, growth, allows you to take new things away from familiar ground.

99% Positive

Ever since the Suzanne Clothier seminar, where she said that being “all positive” is impossible, I’ve been obsessed with how we ought to use consequences in training.

I suppose that Suzanne could criticize my current obsession; I recognize that I may be overthinking this issue, after all. But let’s face it: this is who I am. I think. A lot. I am fascinated by science and research, and I love dog training, so really, it ought to be no surprise to anyone who knows me that I think about this stuff often. So, given all that, I’m going to continue to think about this stuff.

I’ve come to the conclusion that it is, indeed, impossible to use only R+ methods in training. But that doesn’t mean I can’t try. While there may be times that it’s necessary to use some of the other principles of operant conditioning, my personal goal is to remain firmly in the R+ area 99% of the time.

The other night, I went out with some friends. As it always does, our conversation turned to dog training. We were comparing different methods, specifically the various pros and cons of luring, shaping and capturing behaviors. I shared that for the first year, I lured every behavior with Maisy exclusively, but that she seems to learn faster through shaping.

The next morning, on one of the many dog-related email lists I belong to, the incredibly delightful Crystal Salig responded to a thread about teaching the recall by tugging on the leash. She said:

I used to mildly coerce the first part of recall also- just very light pressure on the neck with the leash and released it when the dog started to move- it wasn't harmful, it just wasn't as good as a completely uncoerced recall. I'm still trying to find a less aversive way to stop when a dog is pulling on leash so that it doesn't hurt or surprise the dog as much. When I switched from lure-reward to clicker training, I started to become even less aversive than I was before- seeing how much faster the dog learned if I didn't even lure him let alone use a physical prompt.

This all got me to thinking about my recent venture with Maisy, specifically her sniffing while on walks. Basically, if she stopped to sniff something without having been cued “go sniff,” I’d been telling her “let’s go” and kept walking. If she didn’t keeping walking, the resultant tug on the leash was a natural consequence of her behavior. And while this approach has been working- Maisy knows that “let’s go” means it’s time to stop sniffing and keep walking- I’ve felt bad about doing it.

Crystal Salig’s post helped me understand why the leash tug didn’t sit right with me. Although my action wasn’t inhumane- it’s not physically painful, nor did it appear to be causing stress on Maisy’s part- it did involve coercion, as minor as it is. I commented to my friends that Maisy learns better when she thinks something is her idea, which is why shaping seems to go faster than luring does. Tugging the leash makes Maisy stop sniffing, but it’s not her idea, so while it’s working, it’s slow and frustrating.

This doesn’t mean I think that what I’m doing is wrong. I wouldn’t have tried it if I did. But I do think there has to be a more positive way to teach this. I’m pretty sure that it includes using Premack.

So, here’s the first draft of my plan: I’m going to be vigilant on our walks. I will heavily reinforce the behavior I want- walking in a loose heel. I will also reinforce eye contact, as paying attention to me is incompatible with sniffing. I will pay close attention to her, and when I see the early signs of sniffing (and I think I can pick them out), I’ll call her back to me, ask for some kind of behavior (heeling, a sit, eye contact- it doesn’t really matter what), and then reinforce the behavior by cuing “go sniff.”

I’m quite sure this will not be the final draft of my plan. These things seem to evolve over time, after all. But from now on, all training has to pass the “feel test”- if it doesn’t feel right, then I won’t do it. And in the process, I hope to become 99% positive.

Pat Miller Seminar: Functional Analysis

Last weekend, I had the good fortune to attend a single-day seminar with Pat Miller in Wisconsin. (No, I don’t spend all my time going to seminars, but yes, I’ve been lucky in the canine education department lately.) I had seen Pat previously, and the seminar was basically a scaled-down version of what I had seen a year ago. Even so, there were some very thought provoking moments, and of course, I want to share them with you!

Pat spent some time talking about operant conditioning, which I always enjoy thinking about. Of course, since I do spend a fair amount of time reading about operant conditioning, it was mostly review. (I’ll assume this is the case for most of my readers, but if anyone’s interested, I’d be glad to write an Operant Conditioning 101 post.)

What I found really helpful was Pat’s discussion of “Functional Analysis.” This is where we look at a behavior systematically using the mnemonic of “ABC” to remind us to look at the antecedent of the behavior, the behavior itself, and the consequence following the behavior, which either reinforces or punishes the behavior. The consequence is defined not by what we think should happen, but rather what actually happens.

In other words, if we do something as punishment, but the behavior doesn’t reduce in frequency or intensity? It wasn’t a punishment. If the behavior continues, it was simply aversive, and the dog has learned nothing. If, on the other hand, the behavior increases, our action was actually a reinforcer.

Although we can often correctly predict how the consequence will be perceived by our dog, we will sometimes be wrong. For example, Maisy thinks that getting squirted in the face with water is superawesomefun. If I were to use a squirt bottle as a punisher and didn’t pay attention to her response, I would be in for a pretty big surprise when the behavior I thought I was punishing actually increased. Since the dog’s subsequent behavior determines which of the principles we utilized, Suzanne Clothier is probably right: some of us trainers think too much.



Pat’s discussion on functional analysis yielded something else that was pretty cool: we actually have two opportunities to change behavior. You can manipulate the consequence so that behavior either increases or decreases, but you also have an opportunity to change behavior before it even happens. If we change or prevent the antecedent, we might be able to change the behavior. Doing it this way does seem a little iffier to me, but it’s definitely possible!

How? Well, management is one way. Although management has a high risk of failure at some point, it doesn’t always fail. And management is a good, useful, and sometimes necessary thing, not to mention often easier. I’ve written before about things I do to help manage Maisy’s reactivity.

For behaviors rooted in emotions, like reactivity, we can also prevent the behavior by changing the dog’s feelings about an antecedent. For example, if a dog always barks and lunges at a bicycle, we can change the dog’s feelings about the trigger through classical conditioning. (Again, I assume my readers are familiar with classical conditioning, but if you want a post on it? Just let me know!)

This discussion of functional analysis took topics that I already understood, and helped me think about them in a more sophisticated way. Has it changed the way I train? No, not really, but it has helped me understand why I do what I do a bit better, and I’m quite grateful for that. So, thank you, Pat, for the great discussion!

Suzanne Clothier Seminar: Wrap Up

Wow, who knew that a two day seminar could inspire over a month’s worth of blogging? I want to thank everyone who commented on these blog posts. The discussions we’ve had over the past month have really helped me think about the things Suzanne said in a far more sophisticated manner than I could have alone.

I just want to touch on a few of the highlights from those conversations today. If you haven’t, I encourage you to go back and read the comment threads. There are a lot of smart, dedicated, and talented people in there sharing differing perspectives. Although we dog trainers will probably never agree 100%, it’s nice to consider other ideas, either to refine our own thoughts, or to strengthen our positions.

For me, I have found that the seminar and resulting discussions have strengthened my commitment to positive training, even if the term is a bit of a misnomer. As everyone noted, it is impossible to use solely positive reinforcement. I strive to teach enough foundation skills that I rarely need to stray from that principle of operant conditioning.

Still, there are times when a consequence is needed for a less-than-desirable behavior. The challenge is to find such a consequence which is neither physically painful nor which causes excessive emotional stress. Of course, there is the challenge of defining how much stress is too much, but I’m afraid I have yet to figure that one out. So far, it seems to be a matter of knowing the dog well enough to be able to stop while we’re ahead, but that’s a rather ambiguous answer, and one which is undoubtedly frustrating for the less experienced trainers out there.

I have decided that for my dog, the best way to deal with unwanted behaviors is to use Premack’s Principle. I’ll admit, while I understand the principle intellectually, I don’t quite get why it works so well. At any rate, I’ve had some amazing results with Premack, and so have others.

When it comes to our “silly tricks”- my name for competition behaviors which really only matter because I have a goofy hobby, and not because they’re vital life skills- the consequences for an incorrect response is generally removing the reinforcement or doing a time out. Time outs work well; Maisy loves to train, and she loves to spend time with me. Removing my attention for a short period of time is a far more effective punisher than withholding a food treat.

I do occasionally use mild verbal corrections, but Maisy is so sensitive that I have to be careful with using these. I try to avoid them, as well as no reward markers because they tend to frustrate both of us. Similarly, I use some pressure/release techniques with her, such as body blocks or light physical pressure, but I have to be careful with these, too- she’s incredibly sensitive to physical touch. In fact, I once tried using a body wrap on her, which are widely promoted for reducing anxiety. It did not go over well, and in fact, actually caused more anxiety. Although both verbal corrections and physical prompts can be useful tools which fall on the more positive end of the “consequence spectrum,” they are things which I must use sparingly with my dog.

Which brings me to my favorite part of the seminar: Suzanne’s repeated insistence that we view all dogs as individuals. I love that she says training is humane only when we check in with the dog regularly in order to get his perspective. Can you do this? Is this okay with you? How can I help you? These questions focus on building up the relationship in the name of training, and I’ve always said that training is only about the relationship between me and my dog anyway.

Finally, I think the biggest benefit I got from the seminar was learning to give Maisy the information she needs to be successful. Her statement that dogs look to their people for clues on how they should react really encouraged me to look at what role I play in Maisy’s reactivity. I’ve always known that Maisy is sensitive to my moods and reactions, but being forced to confront that reality at the seminar has really improved my awareness of my own body. Making a conscious effort to remain calm and confident in the face of triggers has gone a long way in soothing Maisy’s fears.

All in all, that weekend was one well spent. I really think I grew a lot as a trainer as a result of the things Suzanne said, and again, I really appreciate each and every one of your comments.

Suzanne Clothier Seminar: Where Clicker Trainers Go Wrong

It came as no surprise that Suzanne considers herself a humane trainer. I knew that by reading her book, by the fact that she calls her method “relationship centered,” and by the observation that the positive training community has rallied around her. So, I was completely unprepared for what she said next:

“All positive training is one of the stupidest things I’ve ever heard.”

Come again? Aren’t you for training our dogs in positive ways? What do you mean, it’s “stupid”? Needless to say, this statement really set me reeling. But you know what? She had some valid criticisms, and I think it’s important that as clicker trainers, we consider what she says, and find ways to improve our training as a result. Again, my understanding of her criticisms are my own, based on stray comments she made. This entry cannot be considered a complete commentary, and I only hope that it’s accurate. That said, in reviewing my notes, I found four ways we often go wrong:

Clicker Trainers are Too Cerebral
As I mentioned previously, Suzanne believes that consequences are useful in training dogs. Although I disagree with some of the consequences she believes are acceptable, I do concede that they can have a place in a thoughtful training program, and I’ve since found some ways to improve my training with Maisy as the result of using them.

The problem is, clicker trainers sometimes shy away from consequences entirely because we’re thinking too much. I know I fell into this category! When I was considering a consequence, I would think about which of the operant conditioning quadrants it fell in. If it fell in the “wrong” quadrant, I wouldn’t do it. Instead, Suzanne encouraged us to look at our dogs, and to decide if the consequence is humane or not. As long as we aren’t hurting or scaring our dogs, we’re probably okay.Instead of thinking about our actions based on some chart, we’re better off thinking about our dog.

Clicker Dogs Have No Impulse Control
Okay, this is a bit of an over-exaggeration of what she said, so don’t go around quoting this verbatim. What Suzanne actually said is that when she sees a dog throwing behaviors, she knows two things: it was clicker trained, and it has low impulse control. This does not mean that all clicker dogs struggle with impulse control, but it can be a by-product of the training method if we don’t get adequate stimulus control over the behaviors we teach our dog, and if we don’t take the time to teach our dogs how to relax.

I love having a dog who wants to work, but sometimes I think I’ve created a monster because I haven’t worked that much on teaching her how to just be. Sometimes, I don’t use the clicker at all because it is creates too much arousal. She loves it so much that she loses control over herself.

Clicker Trainers Micromanage Their Dogs With Cookies
There are two sub-points here. First, clicker trainers tend to rely too much on management, and don’t spend enough time teaching their dogs to be responsible for themselves. (I wrote about this previously in this entry.)

Second, clicker trainers tend to rely on cookies far too much. Suzanne is not against cookies, she just thinks we need branch out. She said that what we really need to do is harness intrinsic motivation. We need to teach our dog to work for other things. Life rewards such as getting to go for a walk or using play as a reinforcer fits here, but Suzanne is big on using praise and the social relationship as a reinforcer, too. Cookies are great, but they’re the icing on the cake of social relationships.

Clicker Trainers Rely on “Recipes” Too Much
Finally, Suzanne was critical of clicker training because it can result in the handler treating the dog like a computer: we train them based on stimulus and response, or input and output. We would find it insulting to be treated this way, so why wouldn’t our dogs? Instead, we need to see the whole animal, and tailor our plans to their needs and preferences. Even counter conditioning will fail, she said, if we don’t keep the dog safe. Training recipes might be a nice starting point, but we need to go beyond them.


As I mentioned, I know I fall in the “think too much” category. I also recognize that my use of the clicker could have contributed to Maisy’s impulse control problems, because I'm just not that good at getting things under stimulus control. So far as micromanaging with cookies… well, guilty again. I do rely a lot on management, and I’m just learning how to use things other than food as a reinforcer. I think I am safe on the last point; but I think my strength lies in being able to evaluate various “recipes” and tweak them so that they work to both Maisy’s strengths and my own.

So, fellow clicker trainers, do you recognize yourself in any of these descriptions? If so, what are you going to do to improve your training?