Thursday, July 8, 2010

High Value or Low Value? Choosing the Right Reinforcer

I am a clicker trainer. As a result, I use a lot of reinforcement when I work with Maisy. When I first started out, each click resulted in a treat purchased from the pet store. Since Maisy ate everything I gave her, I really didn’t think too much about the relative values of what I gave her. A click meant a treat, end of story.

As I’ve learned more about training, I’ve branched out into using toys and activities as a reward, and I pay attention to what she likes best (generally people food). I have made lists of all the things she likes, and I’ve ranked them by value. Some of you have played along with this game, too, which delights me. But while I enjoy making lists and categorizing the items, the exercise doesn’t have much practical value if you don’t do something with the information afterwards. Which is why today, I want to talk about how I use this knowledge in training.

Who hasn’t heard someone- generally a new dog owner or inexperienced trainer- say, “But he does it at home!” We all know why this is, of course: the dog wasn’t adequately “proofed” for the distractions present in a new environment. Every time we increase the difficulty, whether by introducing distractions, or by increasing the distance or duration at which the dog is expected to perform, we need to increase the reinforcement.

Now, this can (and should) be done by increasing the frequency of reinforcement, but it can also be done by increasing the value of the reinforcement. I like to match the highest value reinforcers with the most difficult tasks. If Maisy responds to “come” across the backyard, that might earn her a piece of kibble, but coming away from a dog at the fence might earn her a piece of hot dog, or even a rousing ball-chasing game.

This recall, done during Maisy's first visit to the State Park, will earn her several pieces of beef liver.

However, even when I don’t increase the difficulty of an exercise, I do like to reward the best responses with the best things. Using the recall as an example again, if Maisy comes away from the fence slowly or hesitantly, I’m going to reinforce her response- after all, it was difficult, and she still came. However, if she comes quickly and with enthusiasm, I might give her an even better reward: potato chips. (Remember, on Maisy’s lists, there is pretty much nothing better in the world than a potato chip. Not even hot dogs. You will probably need to use something different for your dog.)

If I don’t have something of higher value, I might use a jackpot, which is simply 5 or 10 of the good treat, given one at a time over the course of 15 to 20 seconds, paired with effusive praise. The use of jackpots is questionable- some people say they don’t really work- but I use them anyway. Even if they don’t work, they can’t hurt.

However, it is important to remember that when following this rule, you really do need to save the outstanding reward for the outstanding response. If you use the high value treat for a simply adequate response, two things could happen. Either you reinforce a less-than-optimal response, thus limiting your progress. Why work hard if you’ll get the tastiest stuff for a mediocre effort?

The other possibility is that you may devalue the treat. Kay Laurence talks about this in her book Teaching with Reinforcement (which is fabulous, and you ought to read it if you’re interested in reinforcement based training). Her argument is that dogs often find novelty reinforcing. Part of what makes a high-value reward so high value is its relative scarcety. I have certainly noticed this with Maisy. She might go crazy for something the first time she gets it, but if I then use it on a regular basis, it no longer provokes the over-the-top response it got before. As a result, if you use high value rewards for run of the mill behaviors, you will likely use that high value reinforcer so often that it becomes kind of boring. Once you’ve discovered what your dog really values, it’s wise to save it for the really special moments so that you can maintain the value of that reinforcer.

In the past, I haven’t been so good about doing this. I’ve used treats rather indiscriminately, and haven’t spent much time thinking about the relative values of the rewards I’ve given, nor have I considered if the response earned that level of reinforcement. Lately, though, I’ve tried to pay attention to this. I generally keep two types of food treats on me at all times: a “regular” value reward (usually a kibble and store-bought treat mix), and a high value reward (generally freeze dried liver or salmon jerky). Both exceptional responses and responses which were difficult (due to distraction) earn the high value reinforcer, along with effusive praise and a lot of attention. Maisy loves attention. I really think it’s helped, too. Her recalls have improved a lot over the past few months, and her moving downs are simply lovely now.

Your turn, now. Have you tried using high value reinforcers like this? If so, tell me about it. I’d love to hear some examples, as well as whether or not you’ve seen it make a difference.


Ally, Eclipse, Teddy and Kira said...

I have a slightly reactive male golden retriever that has kidney issues and food allergies/sensitivities so a lot of treats and human foods don't work for us. At home or low-stress situations we use his kibble or the canned version mixed with canned pumpkin in a food tube, for high stress we use pieces of turkey bacon. He loves a ton of other foods but they all cause issues with either his kidneys or allergies. We can use chicken as well but on the few and fewer occasions that we now need it we use turkey bacon.

The first time I used turkey bacon we started off with just strips of turkey bacon as rewards (hypervigilance outside is what we were working on combating with just eye contact in the driveway). Using that with a click and treat for looking back at me or offering an alternate behaviour in 10-15 minutes he was doing all of his positional commands (he was trained as a service dog but due to health and stress in public is now just my pet) and offering great eye contact besides cats being within 10 feet of him and dogs barking. Since then we can just add the turkey bacon to a cup of kibble and mix it all together and it works for work outside although walking past barking dogs in yards still requires plain turkey bacon, inside we stick to kibble.

The use of high-value rewards has been invaluable in our situation! Great post!

Crystal said...

Oooh, it can be SO hard when your dog has dietary restrictions. It's good that you can use turkey bacon (and that you can even mix it with kibble!) as a high value reinforcers.

Have you tried using any non-food reinforcers? Were they helpful? I know that sometimes, non-food reinforcers do NOT help my dog, especially when we're dealing with reactivity- she's already over-aroused, and adding something else arousing (tug, throwing a ball, etc.) would only make the situation worse.

katie said...

Another fascinating post! My Maizey LOVES food. Period. Doesn't matter what and I have had the hardest time discerning that one thing is better than another. I always have various treats on hand, from fresh asparagus(loves) to turkey dogs(loves), peanut butter (loves), squeezee cheese (loves), kibble(loves). . . well you get the idea. So how do you use the tool of high value reinforcers when your dog loves everything? Hmm... I think you have just insprired a future lessons from 4 legs post!:))

I do use jackpots and fully agree "Even if they don’t work, they can’t hurt." And I think one of her highest reinforcers is my enthusiasm level. She loves nothing more than crazymomlady jumping up and down clapping, hooting and hollering! (eyes rolling here)

Crystal said...

Katie, I "tested" different foods with my Maisy- I wrote about it here:

Have you tried doing that, to see what your Maizey chooses?

Still, some dogs just love food, and I think you're right on with the enthusiasm- the same treat, delivered with more praise and overall attention, is definitely worth more to my dog. A shyer dog may not respond the same way, but for many, this will work.

Another thing to consider is how you deliver the food. Again, the same treat, but thrown so that Maisy has to chase it is more interesting than the treat handed to her. I plan on writing more about this in a future post, because reinforcement delivery can be a huge part of the value... and the results.

And finally... there's play and activities. Food's great, but chasing a ball is pretty awesome.

Kristen said...

You know I hate having to decide what to buy/use as treats....

We tend to not use a lot of super high value reinforcers because those are more expensive or more work to get ( extra trip to the store or extra preparation).

I'm much better about telling students to find and use these things than I am about doing so.

With my dogs I don't tend to surprise them with novel things...because there are a lot of novel things they wouldn't touch/eat/go near. I've accidently taken novel things to fish two weeks ago. It even "contaminated" my other treats. (but they like tuna treats and other fish products).

And the dead bird is a high value reinforcer in that my dog LOVES to smell it and wants to chase it (when thrown). But he's not yet to the point where retrieving it is reinforcing ('ewww the feathers!')

Crystal said...

I always test out novel things before using them as a reinforcer, for just that reason- you never know what they'll like. For example, my dog loves lettuce. And basil. And tomatoes. And watermelon. And... I don't think she know she's supposed to be a carnivore.

As for dead things... I admit, I will let my dog smell them (I'll even point them out to her), but it never occurred to me to throw them...

Anonymous said...

Ira is such a foodie that even kibble is fairly high value to him. It's so easy to reward him! I usually use store-bought moist treats for him, but I'll pull out the fresh meat at matches/trials.

Ally, Eclipse, Teddy and Kira said...

I've experimented with using non-food rewards like physical praise, ear rubs, rump tickles that get him going crazy if he's not anxious at all but if we're working on walking calmly outside or anything outside food is the only that gets his attention off making sure everything is okay, not even his favorite balls or stuffed animals help!

Anonymous said...

Great to find you and thanks for the link to the Fearful Dogs' Blog! I've added a link to yours.

When rewards are better than expected we (and dogs) get bigger hits of dopamine, the feel good neurotransmitter, in our brains.

Crystal said...

Ooooh, that's cool! I love learning about brain chemistry stuff... I didn't know that about dopamine, but it makes sense.

Thanks for reading, and for adding a link to my blog. I love yours- it's so good! :)