Clicker Expo 2011 (Chicago): Helix Fairweather- Accelerating Success with Data Driven Training

There are two kinds of presentations at Clicker Expo: lectures and labs, which use attendee's dogs. I love this format, because it means that you get to see how a given idea works on a “real” dog (ie, not the presenter's). Helix Fairweather presented a lab on using data in training, and I really enjoyed the hands-on component. It took a topic that could have been dry and boring and transformed it into something fun and interesting.

This group- and dog- learned all about data-driven training.

 

Is your training data-driven? If you're like most people, probably not. Still, it's worth making the effort. Helix identified a number of reasons this is true: Data allows you to see progress in black and white: you know that your dog is doing better because the numbers prove it. Data helps you plan for your next session and can show you where there might be gaps in your training. Keeping data gives you a record that can help you learn how your dog learns.

So what data should you keep? There are two main things: rate of reinforcement, and success rates.

A sample training log. Click to embiggen.

Knowing your rate of reinforcement is especially helpful during the initial stages of shaping, because it lets you know if your criteria is too hard or too easy. Count out ten treats, then time how long it takes your dog to earn them. Divide by ten. For most tasks, the dog should be earning a click and treat every 6-10 seconds. Anything less means it's probably too easy, anything more means it's probably too hard.

Of course, these numbers may change if the behavior requires a lot of distance or duration, so repeat this step twice more. The average amount of time it takes the dog to earn ten treats should go down in each subsequent trial, because this indicates that the dog is learning. If it stays the same, or worse, goes up, try to figure out why.

During the lab, we all broke into groups. Each team would work with a dog on a given task three times, timing the rate of reinforcement. The conversations about why the numbers changed the way they did were very interesting. Everything from the way a treat got tossed to distractions in the room impacted those numbers. Figuring it out- and then compensating for it in the next set of trials- was almost like being a detective!

After you've attached a cue to the behavior, you can calculate how often the dog responds correctly to the cue. Count out ten treats. Cue the behavior. If the dog responds correctly, click and treat. If he doesn't, put one treat aside. Repeat until all ten treats are gone. Then look at how many treats you set aside and calculate your dog's success rate; each treat represents ten percent, so if you have two treats left, the incorrect response rate was 20%, and the success rate was 80%.

You can also keep track of how often your dog offers the behavior uncued. Ideally, your dog will only do the behavior when you've cued it, and keeping track of how often he does it without the cue will help you determine if you are achieving stimulus control or not.

No matter what you track, Helix said that it's a good idea to look at each data point separately as well as together. For example, it's a great idea to compare your success rate to your rate of reinforcement. A success rate of 100% with a really low rate of reinforcement is not good if you want quick, snappy responses to your cues. Similarly, a high rate of reinforcement with a high rate of uncued behaviors indicates that your dog doesn't quite understand the cue yet.

If you're thinking this is a lot of work, you're right! In the lab, we had teams of 4-5 people per dog to time, count, and evaluate what was going on. The punch line was that it's easy when there are lots of people to share the work, but not so easy when there's just you and the dog!

One of the best ways to track your data is by video recording your sessions. Doing so will allow you to concentrate fully on the dog, and not on counting or analyzing. Then, later on, you can go back and count and time things to calculate the various rates. It also allows you to see how things like treat delivery, body language or environmental distractions affects your dog. I have to admit, I hate watching myself train. I'm always really embarrassed by my sloppy handling skills, missed reinforcement opportunities, and unclear criteria. Still, I know that in the long run, this makes be a better trainer, too.

Helix also suggested using a voice recorder. That way you can talk as you're training (or immediately afterward) to summarize how things went. It's quicker and easier than writing during a session, although you will want to go back and transcribe those notes for future use.

I haven't actually started to keep data in training. I know I should, but I've been lazy about implementing it. I don't even have any excuses- at least not any good ones. But what about you guys? Do you take data, or record your training sessions in some way? What do find most helpful? What are your tricks and tips for getting the most out of it? If you don't take data, do you think it would help if you started? Why or why not?