Ken Ramirez Seminar: Non-Food Reinforcers

 
One of the big objections people have to clicker training is “all that food.” They always want to know when they can stop using it, an attitude that used to baffle me. I mean, I get that people who are active in dog sports need their dog to perform many behaviors for a single treat, but when there are no rules, what's the problem? It's not hard to stick a handful of kibble in your pocket, after all.

Well. Leave it to Ken to not only be entertaining, but to also convince me that non-food reinforcers are both valuable and necessary (mostly because it is much easier to perform husbandry behaviors on a sick animal who is refusing to eat when you have a non-food reinforcer available). He also presented a very thorough method for creating non-food reinforcers, and gave us some tips on how we should and shouldn't use them once they've been established.

Let's start at the beginning: what are non-food reinforcers? Well, obviously, they're not food, but Ken was a bit more scientific than that. When Ken talked about reinforcers, he broke them down into two categories: primary reinforcers and secondary reinforcers. A primary reinforcer is something that is inherently reinforcing; the animal doesn't need to have any experience with it to understand that it is a good thing. Typically, these reinforcers satisfy biological needs, and food is the ultimate primary reinforcer (that's why it is so useful in training). By contrast, secondary reinforcers are something the animal needs to learn is desirable.

Despite the fact that secondary reinforcers are learned, Ken made the point that secondary reinforcers can be very, very powerful. In fact, they can sometimes be more powerful than primaries because of what they represent. For example, money is a secondary reinforcer- the paper itself has no inherent value. However, society teaches us that money is a desirable thing because of what it can buy, and this association is so strong that humans will do some very boring or unpleasant tasks in order to obtain it. In fact, we are more likely to take a job that pays money than one that provides food and shelter.

(As a side note, play can be looked at as both a primary and a secondary reinforcer. Often the act of playing- running or chasing, for example- is innate, making it a primary reinforcer. However, the objects used in play, like balls or tug toys, are secondaries because the dog needs to learn what they are used for. A ball that is not thrown is neither interesting nor reinforcing to most dogs.)

Another way to think of secondary reinforcers is as a “reinforcement substitute,” which emphasizes the fact that secondary reinforcers only become powerful through conditioning. Ken is very, very systematic in the way that he creates secondary reinforcers. His approach is so thorough and slow, in fact, that I suspect some readers will be turned off by it. This is partly because he's found that the more time you spend conditioning them, the more powerful they will be, but also because he believes that if you use secondary reinforcers improperly, it can lead to a lot of frustration. Since frustration is sometimes inherent in training, he tries to minimize it whenever possible, something which is both kind to the animal and practical when working with wild animals who are less tolerant of human mistakes than the domesticated dog.

Ken creates secondary reinforcers in almost exactly the same way he trains a behavior. He starts by choosing a stimulus to act as a reinforcer. This stimulus should be one that is useful- that is, it is easily accessible, and not overly cumbersome to implement. He also thinks it works well to use something that is novel to the animal; choosing something the animal has habituated to and now ignores is going to make things much more difficult. One of his favorite secondary reinforcers is clapping.

Then he does straight-up classical conditioning: he presents the chosen stimulus, and then immediately follows it with a primary reinforcer. So, he claps, and hands over a bit of food. Clap, food, clap, food, until the animal seems to understand that the clapping predicts the food. This shouldn't take long at all unless the animal finds the stimulus aversive (if the animal is sound-sensitive, for example), or if your primary isn't that exciting.

Then he asks the animal for an easy, well-established behavior. This is something the animal already knows well, and has a very strong reinforcement history for. In dogs, the behavior of sit is often a good choice. When the animal does the behavior, the trainer will present the new stimulus, and then give a primary. For example, the dog sits, the trainer claps, and then he gives a treat. Ken will do this daily for several weeks, although the length of time will vary based on the animal, his relationship to the trainer, and his past reinforcement history.

The next step is to cue the same easy, well-established behavior, and then reward with only the new stimulus. Here, it is truly acting like a reinforcement substitute, as the dog will sit and receive only clapping as his reward. Ken will do this a maximum of three times during a training session, and he'll spread it out so that the animal is also getting primary reinforcers for other correct responses in between. Again, he'll stay at this step for several weeks.

This cycle repeats, except now Ken will cue a harder behavior, though it should still be well-established. When the animal responds, he'll give the new stimulus, and follow it by a primary. So, he'll cue, for example, a roll over or a stay, clap, and then give a treat. He stays at this level for several weeks before cuing the harder behavior and using the stimulus as a reinforcement substitute. Again, he continues doing this for several weeks.

Once this process has been completed, you're ready to use your new reinforcement substitute in training... but Ken has a few rules before you do. The most important is the 80/20 rule, which is actually more of a guideline, but basically, he says that you should use primaries approximately 80% of the time, and secondaries approximately 20% of the time. He never uses the same secondary reinforcer twice in row, although he might use two separate secondaries in a row. He always treats the new secondary as a behavior and occasionally “recharges” it so that it retains its strength. Finally, he recommended that novice trainers use secondaries only to maintain existing behaviors, and not to create new ones.

Yes, this is a very regimented way of moving away from food reinforcers, but Ken has a very convincing story to support the importance of being so systematic. It involves a new trainer trying to use tongue scratches to reward a killer whale, with a very poor result. I won't spoil the story for you (and Ken tells it so much better than me anyway), but trust me when I say that I completely understand why Ken is so thorough. His advice is that we never take any reinforcer for granted, and to work to build up as many reinforcers, both primary and secondary, as possible.

If you've found any of this even remotely interesting, you totally need to see Ken speak about it. Imagine this information, only peppered with incredibly funny and informative stories about dolphins, seals, penguins, and yes, even some dogs. He's also got illustrative videos, and I always enjoy seeing familiar concepts used with exotic creatures.

What about you guys? Has anyone used such an in-depth process to create secondaries? Do you see value in doing it with your dogs? What types of non-food reinforcers do you use... or would you like to start using? I'd love to hear your thoughts!