OK, wait a doggone minute! How is it that in Zip’s last lesson, I was being all poetic about how the behaviors didn’t matter all that much, but all of a sudden we are zeroing in on just one thing? And it sounds so…cold! How did we get there? Does this mean that Marge has given up on bonding and positive reinforcement and creating fun for her puppy (and rainbows and fairies while we’re at it)?
Of course not! What Marge has done is make learning impulse control a win/win situation. With good teaching of impulse control (including what people call “Leave It,” “Zen,” or “It’s Yer Choice,”) dogs learn that when they control themselves around stuff they want, they can get even better stuff! As Sue Ailsby says:
It’s not my job to control the animal. It’s the animal’s job to control herself. It’s my job to put the animal in a situation where she can learn what I want her to know as quickly and easily as possible.
Sue calls it Zen, since the way to get the thing is to leave the thing alone. It’s just something else to learn, and Zip has already had many lessons in “learning is fun!”.
That Puppy Sure Sits a Lot!
For a puppy that didn’t have any formal training sessions on “sit” Zip sure sits a lot. How did that happen? While Marge may not have done any training sessions on “sit,” she was still teaching Zip to sit by reinforcing that behavior when he offered it. Since, as Marge would say, Rewarded Behavior Continues, Zip started sitting more. When barking doesn’t work to get out of a pen, he’ll try sitting and will get rewarded (you can see this in the movie). If dashing towards the door doesn’t work, he’ll try sitting. That’s how highly reinforced behaviors can start to fill in the blanks. I love seeing puppies put two and two together and try it out, like Zip does.
Having default, highly reinforced behaviors are one of the lovely things about positive reinforcement training. At first, when teaching impulse control, any behavior but lunging toward the desired object or goal is usually reinforced. But soon, the trainer can select out of these other behaviors that she has already been reinforcing what she’d like to have. You can see that Marge is building in eye contact and a general orientation to her in all these situations, as well as sitting.
By the way, one of the reasons Marge hasn’t done any formal “sit” training is because she wants to teach Zip a “tuck” sit and just hasn’t gotten around to it. Zip turns 10 weeks old today. She has plenty of time.
What Do They Practice?
So, what did Marge show us in Lesson 3? Zip is working on impulse control in the following ways:
Waiting for permission to grab the tug toy. Getting the permission by looking at Marge.
Staying away from food in Marge’s hand (at puppy level). Getting the food by looking at Marge.
Being quiet in his pen when Marge approaches.
Sitting quietly to get his leash put on (see, she is teaching sit, but she still has yet to say the word!)
Waiting to go out the door. Getting permission by looking at Marge.
Reorienting to Marge after they go out the door together.
Not only is he learning to control his impulses, he is learning to look to Marge when he wants something. A huge part of impulse control is focus on the handler. And Marge has been building that since Day 1.
Using Positive Reinforcement to Teach the Dog Not to Do Something
So many of us came to dog training because our dogs had behavior problems. We wanted them to Stop. Doing. That. And that is also one of the main questions that people ask about positive reinforcement based training: how to you teach a dog not to do something? Today’s whole movie, plus the two before it, do exactly that, but you have to know what to watch for. When you increase some behaviors, some others decrease without a whole lot of work. Some of the things that Zip is learning not to do are:
Lunge for the toy
Run off with the toy (since Marge has made herself the entertainment center–and also because the toy has a handle on it!)
Help himself to food without permission
Jump around when Marge puts his leash on
Run out the door without permission
Go nuts once he gets outside
And countless other behaviors that humans do not prefer!
All without a harsh word, a stern look, being forced into a position or held in place, or any kind of physical punishment.
It would probably be good to decrease this behavior–photo credit Wikimedia Commons
Lots and lots of people think that if you withhold the treat you are punishing the dog. Some will ask the above question in a gleeful, challenging way, feeling certain that they have caught the positive reinforcement based trainers in an inconsistency. But let’s see what is really happening.
Here is a scenario. In the past, you have given your puppy attention and played with him when he jumped on you. But he’s getting big and you really don’t want him jumping on you anymore. You decide to teach him to sit to greet you. He already has a good reinforcement history for sitting, so the likelihood that he will do it in any given situation is fairly high.
So here you are with your excited pup and you are clicking and giving a treat whenever he sits.
Pup sits. Click/treat.
Pup sits. Click/treat.
Pup jumps on you. Nothing.
Pup sits. Click/treat.
Pup sits. Click/treat.
OK, what happened when the pup chose to jump instead of sitting? You didn’t click. The treats stayed in your hand, your pocket, or the bowl. (You meanie!) You stood still and didn’t react. You are paying for sits, not jumping up.
But lo and behold, the jumping up starts to decrease! Decreasing behavior means punishment, right? You must have punished your puppy for jumping!
No. Let’s look at the definitions of positive and negative punishment.
Punishment
Positive punishment: Something is added after a behavior, which results in the behavior happening less often. Example:
Antecedent: You approach your dog.
Behavior: Dog jumps on you.
Consequence: You step on the dog’s back foot, hard. (I’m not recommending this, of course. Just want a clear example of positive punishment.)
Prediction: Jumping up on you will decrease.
Negative punishment: Something is removed after a behavior, which results in the behavior happening less often. Example:
Antecedent: You approach your dog.
Behavior: Dog jumps on you.
Consequence: You turn around and leave.
Prediction: Jumping up on you will decrease.
In the positive punishment example you added painful pressure to your dog’s foot. (Please don’t ever do this.) If the dog finds having his feet stepped on sufficiently painful, jumping will decrease. In the negative punishment example you removed your presence and attention from the dog. If he likes your presence and attention well enough, and if you are consistent, (and if there is no competing reinforcer–that’s a big if!) this also will cause jumping on you to decrease.
So that’s what positive and negative punishment look like. Now back to our original example. Let’s map it out as well.
Antecedent: You approach your dog.
Behavior: Dog jumps on you.
Consequence: You just stand there.
You don’t respond with physical actions or increase or decrease your attention. Admittedly, this is hard to do, and remember, the lack of response has to be from the dog’s point of view. Even looking down at them is a response. Future blog on this point!
The cookies staying put
Nothing was added: therefore no positive punishment. Nothing was removed: therefore no negative punishment.
(By the way, some people who are very new to learning theory think that the above example is negative reinforcement. Sit, give treat = positive reinforcement. Then jump, withhold treat = negative reinforcement. No, no, no! It has an attractive symmetry, but that is not what the term means at all. Here’s a review.)
So What Is Happening?
OK, back to the first scenario, where you are working on sits with your puppy. Let’s say that after that one time when the puppy jumped and you didn’t treat, the puppy didn’t jump up again. Jumping on you decreased during training. Let’s also say that that decrease continues over time. Why isn’t that punishment again?
Because punishment is not the only process that involves a decrease in behavior. There is another: extinction.
Extinction is the nonreinforcement of a previously reinforced response, the result of which is a decrease in the strength of that response.
In other words, extinction is what happens when the behavior you used to do to achieve some thing doesn’t work anymore. So you stop doing it.
So here comes the big question, especially for those folks who think they’ve somehow caught us out on the withholding the treat business.
How Humane is Extinction?
As with so many things, the answer is, “It depends.” But in this case there is a pretty clear demarcation. In the Humane Hierarchy, extinction by itself is at the same level of negative reinforcement (which involves an aversive) and negative punishment (which involves a penalty for behavior). Not great as first choices. We know that from life. If a machine we use all the time stops working, or a method we use of interacting with another person we care about suddenly gets no response with no explanation, we are left high and dry. It is not fun.
However, extinction also happens in tandem with a process called Differential Reinforcement of Alternative Behaviors (DRA). This is how trainers who aim to train primarily with positive reinforcement use it. (There are other differential reinforcement methods, but this is a good general one to discuss right now.) It consists of reinforcement of an alternative behavior while reinforcement for the target behavior is withheld. Done with some care and skill, it can involve very little frustration for the animal, and it is one step closer to the “most humane” end of the Humane Hierarchy. And this is what is happening in the example above. As long as the trainer is being quite clear that sits are being paid for, the fact that jumping up on her no longer gets attention is not so hard on the pup. He has another thing he can do to get something good. He gets attention and food.
The trainer has communicated to the pup a new behavior to “fill the hole” where jumping used to be.
I’m borrowing this great example of how DRA works from my friend Kim Pike. Let’s say the soda machine at a workplace is not working. People will push the button repeatedly. Some will perhaps pound on the machine or kick it. This is typical when extinction is in play by itself. The people have no alternative, and get frustrated. (I’ll be covering extinction bursts and and extinction aggression in a later post.) Gradually people will stop going to the machine and give up pushing the buttons. Individuals will probably forget, and now and then go try the machine again, then perhaps give it another kick or shove. But after a while no one goes to the machine anymore.
But when the soda machine is fixed, there will likely be a crowd of people ready to buy their sodas. It’s easier than going to the corner store, and involves less planning than bringing drinks from home. The behaviors attendant to getting a soda are all still fluent and easy for people to perform. And they once again get reinforced.
However! What if, when the machine broke, someone immediately set up a system where folks could buy a soda they liked as well or better for less money? Perhaps there was a cooler, or an honor system with soda in the fridge. If that alternative were in place immediately, would the thirsty people typically have experienced the same level of frustration at the broken machine? Nope! (Except perhaps for the engineers and mechanics, grin.)
And the most important question: What will the folks who just want a soda do when the machine gets fixed? As long as the cheaper, better alternative is still available, they will keep heading for it. The machine will have become irrelevant. Maybe once in a while someone will forget, and go to the machine. But they’d then remember that they can get a better drink, cheaper, out of the fridge.
This is what we are doing when we allow an extinction process in tandem with positive reinforcement of an alternative behavior. We clearly offer the animal an attractive alternative and remind them of it to keep it front and center. It’s important that the reinforcer for the new behavior be the same or better than that of the old behavior. This makes for a process with much less frustration.
Extinction in a Specific Circumstance
In my post, How Do I Tell My Dog She’s Wrong? I address “failing to click” during a training session. I feature a short video example from the great trainer Sue Ailsby teaching her young Portuguese Water Dog, Sync, to stand and stay. In the video you can see Sync’s immediate bounce back after the couple of times she tries something other than a stand and doesn’t earn a click.
In that case, sits and downs are not going to decrease into oblivion in every situation, as we might want the jumping up to do in our other example. But they will go into extinction during training sessions of “Stand” and later when Sync learns a cue for it. Since dogs can discriminate this easily, it also tells us that when we want a behavior to go away completely, we need to practice reinforcing our alternative behavior in many locations and situations.
Conclusion
So in answer to the critics, no, withholding the cookie in itself is not punishment. And if used in tandem with reinforcing another behavior, it is quite humane. If we put even a moderate amount of thought and planning into the situation, we can set the dog up to succeed. There will be minimal frustration when he does miss the mark on occasion and fails to earn the treat.
Stay tuned for Part 2 on extinction. I’ll be talking in more detail about what happens when extinction is used by itself, and comparing that with differential reinforcement in some human and dog case studies.
