Originally published in December 2012; expanded and revised for 2019. The video in this post was featured at Tate Behavioral’s ABA Conference in October 2019 by Dr. Megan Miller.
A poisoned cue is a cue that is associated with both reinforcing and aversive consequences. Poisoned cues were probably the norm for a period in some types of training, and still are common. If you tell your dog to “sit” and he gets a cookie if he sits but gets a push on the butt or jerk on the leash if he doesn’t, then “sit” is a poisoned cue. The term was coined by Karen Pryor.
According to the well known TV personality (I won’t call him a trainer) Cesar Millan, “calm submissive” is a very desirable state for your dog to be in.
Is Zani calm? Check out her left front leg before you decide.**
OK, I don’t even want to begin to address the word submissive. So far I’m just thinking about “calm.”
Here is a dictionary definition:
Free from excitement or passion; tranquil.
Zani shut down
My friend Diana and I were discussing seeing the difference between a shut-down dog and a calm dog. I have some video footage, soon to be published, of Zani in a shut-down state. (It was long ago and she had been rude to little Cricket, who snarked at her. For whatever reason, that time it upset Zani greatly. She shut down and tuned the world out for about half an hour afterwards.) Diana said it would be great to contrast the shut down state with photos or video of Zani when she is calm, so as to help demonstrate the difference.
So I started thinking how I would film Zani being “calm.”
First of all, I realized calm is not a behavior. It is an emotional state, but it can sometimes be observed by physiological signs. I would say they include:
slow to moderate heart rate
slow to moderate breathing
relaxed muscles or muscles being used smoothly
lack of signs of arousal or excitement
I tried to list positive signs first, but it is easiest to see calm as a lack, yes? Like the definition: free from excitement. In a dog we might notice:
lack of barking
lack of panting
lack of excitement
lack of trembling
lack of running around
lack of jumping on people or chewing the furniture
…ad infinitum.
Calm as Contrast
I’ve also realized that in English, “calm” is frequently used as a contrast word. What picture does the following sentence bring to mind?
Henry calmly got out his wallet and removed his driver’s license.
What did you visualize? I bet 9 out of 10 people visualized Henry being stopped by a police officer. The word “calm” in such a sentence would be emphasizing that Henry is cool under pressure, and/or innocent of any law breaking.
Did anybody visualize Henry sprawled on a couch, watching TV and drinking beer, reaching idly into his wallet to take a look at his license or show it to his girlfriend? <<crickets chirping>>
Yet even if Henry were a really coolheaded guy, he would probably be much more calm in that situation than when being confronted by a police officer.
The more I think about it, the more examples I can think of where “calm” is used to in a situation where there is something exciting or stressful going on. “Julie was calm in the face of danger.” “David is calm under pressure.” We even say that dogs give “calming signals.” They are generally stress indicators. Calm is usually noted as a (desirable) reaction to something stressful. Whereas the word relaxed, though related, describes a physical/mental state only and doesn’t necessarily imply as much about the surrounding environment. So it’s kind of hard to photograph “calm.” It’s comparative.
Zani and Summer look at a cat
Here’s a “calm by contrast” photo. Summer and Zani are looking at a cat. Summer is starting an agonistic pucker of her mouth and is standing up. Zani, by contrast, is sitting. She is watching attentively but not braced as readily for action. She is more calm than Summer. But is she “calm”? Maybe about as “calm” as Henry was when taking his driver’s license out for the cop.
Calmness in Dogs
At first I couldn’t decide whether to say Zani is calm most of the time or never. In a dog as well adjusted as she is, one tends to take a certain amount of calmness for granted.
But actually, living with Zani around the house, I would rarely call her demeanor “calm.” She’s either asleep, or she is active. When she’s interacting with the other dogs or me she is alert, in the game, responsive, high energy, even wired. And it was pretty telling that I couldn’t find many pictures or videos in which she looks “calm.”
We Glupling Trainers tend to work on calmness with dogs for whom overstimulated emotional states are a problem. In other words, it’s for their benefit at least as much as it is for ours. My dog Summer is reactive. My dog Clara is feral and also easily overaroused. These dogs need help being calm. So we practice things like Dr. Karen Overall’s Relaxation Protocol and straight-up relaxation, as in this video I made with Clara.
I’ve always been sure Zani, like most dogs, would benefit from those exercises, but frankly, it hasn’t been high on my list. That is, until I recorded a bunch of footage of my dogs doing crate and mat exercises this morning. From watching the recordings, I saw that I have probably underestimated the stress in Zani’s life. Up till now, I haven’t worked on calm with her because she is a trouble-free dog for me. I’m feeling a bit like a self-centered jerk after watching that footage. Some dogs are amazing for putting up with us at all. She is very sensitive. But that’s a topic for my next installment.
Here are the “calmest” pics I could find of Zani, but in the ones where she is cocking her head, she is working me for a treat. She tends to snap to attention when I get the camera out.
Zani fairly relaxed
Cesar is Confused
Isn’t that a nice way to put it?
Cesar often calls dogs “relaxed” or “calm submissive” when they are motionless but frightened out of their wits, as indicated by trembling, stiffness, rolling eyes, or the release of urine. You can see him do that in this video analysis of “Showdown with Holly,” if you can stomach it. At 5:12, he says, “So you see, just relaxation…” I think he says that basically because Holly is lying down. It’s clear when the camera turns her way that Holly is far from relaxed. But Cesar is not famous for his ability to read dog body language.
For Cesar Millan, “calm submissive” means, “I can do stuff to this dog and it won’t react.” It is equivalent to what we would call “shut down.”
In contrast, the most important state for my dogs to be in, at any time, is “happy.” After that, I value alert, responsive, cooperative. Excited some of the time, calm when appropriate.
As I write this, Zani is sprawled at my feet in the position I call “flounder,” as in the very first photo at the top. She’s lying flat on her side with her head down. Is she calm? No. She is completely alert, offering that funny behavior, trying to get me to give her a treat. And that’s perfectly OK with me. For now. But I need to observe and analyze just how much of the time she is “working.” Maybe I, too, have been guilty of assuming that a dog that doesn’t bother me is “calm” a lot more often than she really is.
I really thank Diana for her part in helping me to see this.
How about you? Can you define “calm”? How would you take a picture of it? Have you observed or filmed your dogs being calm?
** A note about the first picture. This is not an appeasement display. But neither is Zani relaxed. She is offering that behavior because she thought it up and I have reinforced it. You can see that she is holding her right front leg stiffly out from her body. Her eyes are staring straight ahead and not soft. She is working for a treat.
Every so often, in the midst of a discussion about operant learning, someone will write,
The quadrants* don’t matter. Talking about the quadrants just confuses people and makes them pay more attention to theory than what is going on in front of them. To be truly humane we need to pay attention to the individual dogs and how they react to each teaching method.
I really wonder about that.
Of course the dog is the arbiter of what is pleasant and what is aversive and to what degree. But how good are most people at reading dogs, really? How many pictures and videos can you find on the Internet in 30 seconds that look roughly like the one below?
Public Domain Image of Unhappy Dog Being Held Too Close By a Human
[If you are new to the blog or new to the concepts or nomenclature of operant learning, you may just want to skip down to the movie. It is an example of what I will be discussing here: using theory to inform our practice. It is particularly geared towards folks to whom the theory is new. Or if you really want to go for it, here is a whole post, including a video, that gives examples of all the processes of operant learning.]
Setting up “the quadrants” and observation of the dog as exclusive from one another is a false dichotomy. That is a rhetorical fallacy that implies that there are only two choices when there are actually more, or implies that two choices are mutually exclusive. It doesn’t have to be either or, folks, and I will put forth that that attitude can be harmful.
Learning theory and dog body language observation inform each other. Why encourage people to depend on just one and not the other? Why leave a gap in people’s understanding about the processes of learning, certain ones of which have been shown repeatedly in research and real life to have undesirable side effects?
I know that many pet owners who have hired trainers look at them like they have two heads if they start speaking about learning theory. I get that. Clients often just want a method to solve the problem. But when someone is eager to learn and serious about working with their dog, I think it’s a disservice not to share the nuts and bolts of how animals and humans learn. And it’s a disservice to discourage Internet discussions about the processes of learning. Yes, I know they can be tiresome. But just like with any other aspect of humane training, there are always people new to the subject who can benefit.
The more I see people objecting to “the quadrants”, the more I notice that most of them are attempting to veil their use of less humane techniques.
Generalization of a behavior is one of the steps to fluency. One of our ongoing goals with dogs is to help them generalize. So I hope that trainers and teachers would want to help their human students generalize as well. With the humans it’s not only about generalizing behaviors, but about learning concepts and generalizing them as well. Studying the processes of learning and recognizing and naming them helps with this. If negative punishment in one situation stressed my dog out, wouldn’t I want to keep a special eye out for other negative punishment scenarios? Why would I not want that conceptual assistance?
I also know from painful personal experience, and observation of, like, all of YouTube, that reading the body language of a dog and getting past one’s own assumptions is a difficult and time-consuming task. It’s easy for an experienced trainer to say, “Just look at the dog.” But can all students really do that and perceive what the dog is saying? I don’t think my observation skills are below average, but I gotta say, it took me lots less time to get the basics of operant learning processes than it did to learn to read dogs well. I’m still working very hard on that. Being informed by the theory about what kinds of situations to look out for can really make a difference for anyone who is learning to observe and learning the language.
There was a video making semi-viral rounds on the clicker training community recently, often accompanied by comments like, “The power of positive reinforcement!” The video has an adorable, tiny young Yorkie with a bow in her hair doing all sorts of tricks. I saw it posted on a list of thousands of people, and not one person spoke up to discuss the stress signals the dog seemed to be throwing. (Not to mention that some of the tricks might have been physically too demanding for a pup.) Perhaps people were just being polite. I didn’t say anything myself because I had dealt with enough controversy that week, sigh. But the dog did not appear delighted with the training interaction at all.
I’m not linking to the video here, but will send a link privately to any curious folk who make a request through email (sidebar) or a comment.
It sure confirms my doubts about the advisability of just having everybody depend on “watching the dog.”
Examples
The following two stories are true. They both happened to me. One tells how my observation of dog body language led me to analyze and classify the reason for my dog’s stress. After the classification I could be alert to other similarly stressful situations. The other example tells how being informed of what quadrant/process I was using made me question a decision I had made, gain more empathy for my dog, and change my behavior. My (beginner) knowledge of learning processes helped in both cases.
Example #1: From Body Language To Learning and Generalization
My puppy Clara has always loved doing stuff with me and has great attention and a great work ethic. However, I have noticed that shaping can be quite stressful for her. I even wrote a post about shaping and stress. I started thinking about why it might be so. I realized that with my imperfect skills, the changing of criteria was hard on her. Riding the little extinction trails where one version of something ceases to be marked and reinforced and another behavior is desired was quite hard for her.
In the photos, Clara is doing a fast counterclockwise circle, which is a default stress behavior for her. Ironically, the behavior we were working on was, “Relax.” (We’re doing a lot better with that now.)
Clara circling during a shaping session (1)
Clara circling during a shaping session (2)
I have since learned more about shaping and know that if it’s done with careful manipulation of the environment like Skinner suggested and the great trainers can do, there can be much less of this type of stress. I like to think that my skills have improved. Clara has also grown up a little, and doesn’t think the world will end when she doesn’t get clicked.
But my realization that extinction in shaping was hard on Clara made me both more empathetic to her situation and also proactive in avoiding extinction in other scenarios. Her stressed body language made me analyze the cause of the stress, and being able to put a term to it allowed me to learn more about it and look for similar problems in other situations.
(Extinction is not one of the four operant learning processes that people call the quadrants. Extinction is when a behavior that has been previously reinforced ceases to get reinforcement. It is a process that can happen with both operantly learned and classically conditioned behavior. What’s important for my point is that it is a learning method that is often under my control and that I can choose whether or not to employ, and one that can definitely be stressful.)
Example #2: From Quadrants to Empathy
Early in my life with my dog Zani I picked her up and carried her into a public place. She is very friendly and immediately started to struggle to get me to put her down. Since I was just learning about reinforcement, and had learned that what you reinforce is what you get, I decided to hold onto her until she stopped. I didn’t want to positively reinforce her struggling by giving her immediately what she wanted. She finally stopped, I waited a few seconds, then put her down.
I mentioned this episode to my teacher, who said, ah yes, you used negative reinforcement instead! Up until that moment it had not trickled into my head that I had been using a mild aversive. Zani did not want to be held. She was struggling to get away. I not only hung onto her but I had tightened my grip until she figured out that struggling wouldn’t work. (There could be an element of positive punishment in here as well. But the duration of the tight grip, and the requirement for Zani to come up with a different behavior to escape it, even if the behavior was relaxing her body–these indicate that the major process was that of negative reinforcement.)
I grew up spending a lot of time in the country and was around a fair number of small animals and farm animals. Holding or holding down a struggling animal with force was just something you took for granted. You had to do it sometimes “for their own good” and it was something I was absolutely comfortable with. I was 50 years old before I realized that there are things you can do to help prevent you and your animal from getting into this situation in the first place, and ways you can give them more of a say about things. And it was in part because my teacher reminded me of the different processes of operant learning. This led to empathy for Zani on my part, and for me not only to work on that specific situation but to be more aware of the negative reinforcement moments in the future.
Education about Learning Theory
Here is an example of the kind of thing that I believe can be helpful. This is a video I made that demonstrates what negative reinforcement can look like, and shows the same behavior trained with negative reinforcement vs positive reinforcement. It is a modest attempt at linking the theory, practice, and dog body language.
I’d be interested to know what the rest of you think. Can we train humanely without knowing learning theory? For me, the theory definitely helps.
Four quadrants of operant conditioning
*NOTE “The quadrants” is not optimal nomenclature in learning theory. I use the term throughout this piece because that is how the argument is almost always stated, and people might not know what I was talking about otherwise. Better nomenclature is “the processes of operant learning.” “The quadrants” is just a description of the shape of the diagram they fit in, as Dr. Susan Friedman points out.
I’m an auditory person. I grew up in a whole family of musicians. I love language and sound and music of all sorts. For the first half of my life I lived almost exclusively in the company of musicians, and in the second half I have few musician friends. This second half has made me conscious of the ways we musicians are different!
At work, I’m the one who gets asked to check the voice mail when we can’t understand the name. When there is a strange noise, eyes turn to me for identification of it. I can almost always tell if someone can’t hear me well, and I can immediately tell if someone is not listening to me (these are very different!). I can also ignore visual stimuli very well if I am listening to someone or something. (With regard to visual skills, I’ve been told someone could come in and rearrange the furniture in my house and I wouldn’t notice….)
So naturally I am interested in my dogs’ perceptions of sound and verbal cues. Frequent readers will know that I am honest about my limitations and frailties as a trainer, so I think you all will believe me when I say that ironically I seem to have three dogs in training who all have rather low aptitude for verbal cues. I.e., it’s probably not just my limitations in this case. So we all have to work extra hard on words.
Verbal cue discrimination training, where you teach a dog to respond only to the correct word, can be stressful for any dog. If not done with care, the dog can have a very high error rate, which is discouraging to many dogs. So I gave a lot of thought about how I could reduce that error rate.
In A Secret for Training Two Dogs I described my strategies for teaching one of my dogs to stay on a mat while I trained another. I briefly discussed my methods for releases. I have chosen to use each dog’s name, spoken in a certain tone, as an individual release. Dr. Patricia McConnell demonstrates this method in “Examples of Wait with Multiple Dogs.” This post covers how I went about teaching the discrimination of each dog releasing only on her own cue.
Does She Know the Cue At All?
Most of us at some point discover that our dogs don’t know their verbal cues nearly as well as we think they do. Here’s an experiment for those who have puppies or inexperienced dogs and haven’t worked on this before. Try this if your dog is familiar with Sit and Down, but not Stand.
Cue your dog to sit. Then look at her just like you are giving her a real cue (be as convincing as you can) and say “(Your dog’s name), Purple.” Or some other word that doesn’t sound a bit like “Down” or any other cue she knows. Most dogs will promptly lie down.
It usually turns out that your dog didn’t really know the word, “Down.” She didn’t need to, since whenever you have said something to her when she is sitting, you meant for her to down. So you can say anything and she will do it.
The transition from responding whenever “human-says-a-word” to learning to listen to the verbals can be difficult and stressful. That’s why I decided to apply the principles of reduced error learning.
Reduced Error Learning
I don’t use the term “errorless learning” because it is both impossible in real life situations and sets a depressingly high standard for most people (and also, I have to add, the most well known studies involved lab animals that were food deprived. I just don’t want to be associated with that). I wrote about this in Errorless Learning II. I have adopted Susan Friedman’s terminology of “reduced error learning” because I think it’s more realistic.
The original concept as promoted by Skinner is great. I do absolutely follow the practices of this kind of learning, which I would describe as “setting your dog up to succeed and to reduce stress in learning, including with creative manipulation of the training environment and props.”
An example of this is the process of making the right choice easier at first during an olfactory discrimination, such as the cups game. If you are teaching your dog to foot target the inverted cup that covers a smelly treat, first you start with only that one cup. Encourage her to use her nose to smell the cup and treat. Let her repeatedly practice touching the cup with the treat under it. Lift the cup and give her the treat each time.
To start the discrimination, you introduce a second cup without a treat, but you introduce it way over to the side where the dog can’t reach it. You gradually move it closer and closer while the dog is still touching the correct cup. In this way you have made the correct choice easy and the incorrect choice hard, and the dog is gaining a reinforcement history for touching the cup with the treat. Only after this process would you start mixing the cups up.
The opposite of this process would be to put out multiple cups with only one with a treat under it, and mix them up each time the dog gives a try. Even if your dog knew a foot target, there would be no clue as to which cup to touch. And even if it seems like it would be obvious for them to touch only the smelly cup, well, I’m here to tell you that my hound couldn’t do it, even when I made it much easier than a bunch of mixed up cups. With several cups, the failure rate is apt to be so high that many dogs will quit after a few attempts. This is the difficulty with trial and error learning.
Applying Reduced Error Learning to Cue Discrimination
Verbal cue discrimination means you teach your dog to respond only to the correct verbal cue and not other words. The way this is generally done is to repeat the cue for one behavior several times, and reinforce correct responses. (If you are not getting correct responses, you aren’t ready to work on cue discrimination.) After about four of these, say a completely different word instead. If the dog doesn’t do the behavior (yay!), or hesitates, quickly mark and reinforce.
Note that this is harder for the dog than firing off a bunch of different cues the dog knows. Because in this exercise the dog must be discriminating enough and confident enough to do nothing if the word is not a real cue. Plus, in so many situations we reinforce clicker dogs for guessing. The first time you practice this it can be like pulling the rug out from under the dog’s feet.
So how can we reduce stress and errors? In addition to choosing words to begin with that were very different from the correct cue word, I also chose to use at first a different tone of voice and/or volume for the cue. I took pains to make the non-cues as far away in the auditory sense as they could be from real cues.
Clara correctly stays on her mat when I chirp out a nonsense word
The Process
Since the whole point of individual releases is that one dog comes and the other/s stay put, I practiced with each dog by herself, going through the following steps to insure that she learned to respond to her own release cue and not the other dogs’.
Special note: I heavily reinforce my dogs for being on their mats, and I don’t require them to move when I give the general release cue, “OK.” Because of this I incorporated a hand target and/or other body language at first to encourage them to move, then faded it. Others would probably not need to do this.
With the dog on her mat, I called her with her release word followed by invitation to hand target or other body language that invited her to move.
Then I called her with her release word without a hand target.
Then I said a word that was very different from the dog’s release word and in a different tone (I blurted it out, high and squeaky). I reinforced her for not moving. If she got up, I quietly escorted her back to the mat, walking side by side with her to avoid using body pressure. (This hardly happened at all, which was one of my goals.) If the dog did get up, I made the non-cue word even more nonsensical. Quieter. Or perhaps I turned away. Anything I could think of to make it less cue-like. Once she started getting it: lather, rinse, repeat.
I started interspersing the dog’s release word. I reinforced when she came, and for the other words, I reinforced when she stayed. If she stayed for her own release word, I beckoned her a little. If she came for another word, I quietly escorted her back to the mat as described above.
I gradually worked into using a normal tone of voice for the non release words. I continued to reinforce for correct behavior/s, staying or releasing appropriately.
The final step was to work in the other dogs’ release words to make sure the subject dog wouldn’t release on them. At this point I was saying all the words exactly the same way without helping the dog. The goal was that she released for her own and was steady for the others.
This method worked very well for Clara and Zani. Clara in particular got it very fast, and I loved how she lay there very relaxed on the mat while I said the other dogs’ release words.
Summer had the hardest time. She alone started offering other behaviors for the non-cue words. That meant that the first few times I used a non-cue word, I had to withhold reinforcement or else reinforce a random behavior performed on the mat. In most cases she tried her “rewind” trick, a backwards inchworm move. I figured out to reinforce very fast, before she was able to move, and we got through it.
But then after I got her to stay still on the mat through the non-cue words, she lost confidence about coming on her own release word. She was not getting the difference.
I did some extra sessions with Summer. After I reviewed the video I realized what the problem was. My squeaky cues were actually prodding her to action. She is a bit sound sensitive and I think they stressed her out a tiny bit. In any case she responded by trying something, anything. So I did the obvious, and instead of squeaky blurty non-cues, I said very quiet ones. That did the trick. I was able to raise the volume almost immediately, and she is catching up to the other dogs.
I’m getting close to my goal of having all my dogs present and unfettered while I train one, with the others reinforced for their self control on their mats!
Note to all my international readers and viewers: I will be happy to make more translations of this movie, if you want to help. Thanks to Stéphanie and Nathalie, if anyone wants to volunteer to translate, I can send a text document that has all the English from the movie, with spaces left for translation. It takes me only a couple of hours to change the text in the movie, and I can usually do it within a week or two of receiving the translation, depending on what else is in the queue. Hoping to get some takers!
And of course if you want to translate any other movie or post I would be flattered and will work with you on that.
Poor Clara had her yearly vet visit this past week. She is my feral dog, and although I have two socialization sessions with her every week and she is making great progress, I have not worked with her at the vet. Going to the vet is graduate school, and she’s just to 7th grade or so.
So what do you do when you have to put your dog in a situation for which they are not ready to be comfortable? What I did was take lots of food for distractions and get through it as fast as possible.
For Clara, getting shots or other procedures that cause a bit of pain is not the problem. Being in a building with other people and dogs in close proximity, and being handled and restrained by strangers is. This is by far the most frightened she has ever been at the vet’s, probably because Continue reading “Dog Facial Expressions: Stress”
Zani waiting at the back door. Her reinforcement for this polite behavior is the opportunity to go outside.
Sacrilege!
Is it possible that in some cases, using the Premack principle in choosing reinforcement for our dogs is not the best choice? Can attempting Premack cause problems?[1]*A technicality, but it’s important. Notice I haven’t said “Premack didn’t work.” That’s like saying that reinforcement didn’t work. Reinforcement is defined … Continue reading
In my experience, yes. It can go wrong with some behaviors, with some dogs, and especially with some inexperienced trainers (yours truly takes a bow).
Premack’s principle states that more probable behaviors will reinforce less probable behaviors. In other words, you can use an activity the dog really enjoys to reinforce something that is ho-hum. You can reinforce loose leash walking with a tug session. You can reinforce sitting politely while the leash is attached with going for a walk. Premack is all about life rewards.
Premack is often suggested in situations when a dog really, really wants to do something, so much so that they are having a hard time with self control. My dog Zani loves little kids. If I wanted to apply Premack to this situation, I could use the opportunity to visit with them (if they were interested and it was OK with Mom) to reinforce her walking calmly up to them without pulling. Visiting with children could be a more potent reinforcer than a really good food treat for Zani. So the Premack principle can turn a distraction into a reinforcer. For my dog Summer, being brought close to children would be punishing. She’s nervous about them.
By the way, Premack applies to punishment, too. Many of David Premack’s experiments involved punishing a behavior by inducing the animal to subsequently perform an undesired behavior. We can think of examples of this easily in our life with dogs. If the only time we take a dog into a certain bathroom is to take a bath, and he hates baths (and we haven’t done anything to mitigate that), the behavior of wandering into that bathroom with us will decrease.
There is one obvious answer to the question posed by my title. Premack is not a good choice when the behavior is never acceptable.For instance, my young dog Clara loves to pounce on and body slam my other dogs. She would love it if I allowed that, but of course I don’t. I teach incompatible behaviors and I interrupt it. And I try to give her opportunities for very physical play with me, with some firm ground rules.
But there is another situation in which Premack is not the best choice, and it can be hard to recognize, especially for pet owners and anyone who is trying to teach their dog without an in-person expert teacher.
In my experience, Premack may not be a good choice when the desired behavior triggers stress, arousal, or a strong emotional response from the dog, or if the behavior results from these conditions.
Summer waiting at the back door. What is wrong with this picture?
I think this can be an insidious problem, since behaviors and situations the dog gets really excited about are precisely what prompt people to recommend Premack. If you spend any time at all on dog training Internet discussion groups, you know that whenever someone describes something the dog is passionate about (squirrels) someone else is going to suggest using Premack. This advice comes as regular as clockwork. Give the dog contingent access to the squirrels.
I’ve gotten so I flinch every time I see those recommendations come rolling in. It may work out just fine. But the newbie trainer who is describing the problem may not have a correct assessment of the situation, and/or the skill to use the Premack reinforcer.
I can relate three personal experiences where Premack didn’t work out for me. And I mean, spectacularly didn’t work out. My own inexperience was part of the problem, but that’s my point.
1. Reinforcing loose leash walking with a chance to run towards a squirrel, with my dog Summer. This was a disaster. I was brand new to training, but it seemed like such a good idea, made to order. What I didn’t know then was that Summer has a very high prey drive, is hyper-vigilant, and very environmentally sensitive. I also didn’t know that I really needed to have taught her more about LLW itself (using food). But instead, I jumped right into Premack. When we would see a squirrel I would require a few steps of LLW, followed by a quiet sit. Then I would release her and we would run together to the squirrel and she would lose her mind. When I got tired of circling the squirrel tree with her, I had to figure out a way to get her away. Her capability of going for a normal walk was completely gone by that point.
If you are going to allow the dog some kind of engagement with the environment as a reinforcer, I think there is a prerequisite to being able to make it work. You need a way to get them back, and it seems to me that you need to train this first. You need your dog to be able to recover from a potent emotional response fluently. These are challenging things to do, and usually not in place if you are having a big issue with distractions in the first place.
By the way, I used sniffing as a reinforcer for loose leash walking with moderate success with my dog Zani. I allowed stopping to calmly sniff as a reinforcer for walking nicely on leash. But in her case, I had a little more experience than I had had when I tried it with Summer and the squirrels. I taught Zani a cue to go sniff, “Beagle!” And a cue to come back to my side, “With me!” I practiced the pair of behaviors in boring environments before taking it on the road, and I taught Zani the correct position for LLW to begin with already with food.
2. Reinforcing Clara for not jumping up to lick my face by letting her lick my face on cue, with four paws on the floor. Ouch. Another newbie error on my part. It seemed like such a no-brainer. I mean, if she is dying to jump up and get my face, that seems like a great candidate for Premack, right? Well in our case, wrong. I recently wrote a whole post about the face mugging problem and all the things I tried. I was well on my way to trying Premack when I thought to ask my teacher about it. She took a look at Clara, and said that her jumping up at my face did not look like a happy behavior. It was stress-related. So even if I had succeeded in teaching her how to lick my face without the danger of breaking my jaw, I might have ended up with a situation like Summer at the door (see below).
3. Reinforcing sitting politely at the back door with going outside with Summer. This is a lovely method for two of my dogs, Zani and Clara. See Zani’s photo above. It is one of the most commonly recommended uses of the Premack principle in dog training. But again, it didn’t work for Summer. You would think that something she wanted so badly—to charge out into the yard checking for cats, squirrels, and other varmints—would cause a very prompt, snappy sit at the door. Not so. As you can see in the video, sometimes she can’t sit at all. And if she does sit, she will not accept a treat. She is what is often called “over threshold.” She is anticipating what might be in the yard, and is having a big emotional response to that. She is also showing the fallout of years of conflict with me at the door. I didn’t cope with her behavior well, especially at first. I nagged her because I was completely oblivious to what was going on. I made the situation worse.
By the way, Zani is also at the door and can be seen at 1:24 in the video in an exemplary calm sit, even though she is excited to go out, too. She is not drowned in excitement and stress hormones.
I fully acknowledge that a better trainer could have managed this situation better. She could have taught Summer first to be calm in the face of the potential excitement. Then worked up to using the Premack reinforcer when she could keep her wits about her. I should have aborted the project when my behavior was obviously stressing her out. But that’s my point. Premack is often recommended to beginners and to us non-professionals. And it can really backfire without some experienced eyes on what is happening. When I first started doing this years ago I had no idea why Summer’s sit was not more reliable. This method seemed to work for everybody else. To be perfectly frank, I read her body language as “sulky.” I thought she was being a bratty adolescent; moving slowly and giving me a dirty look because I didn’t let her out fast enough.
You might think that I would have run into a problem with her stress at the door just as badly if I had used food as the reinforcer for a calm sit. But using food diffuses Summer’s overexcitement, and doesn’t feed into it. (Many trainers have noted that food tends to have a calming effect when training behaviors, as opposed to using tug or other high arousal activities.) She has practiced her frozen shutdown followed by running out in a frenzy for years now. But reinforcing a sit near the door with a high-value food treat instead, and doing training sessions in this area of the house, are changing the potential reinforcement map in my favor. The excitement of the outdoors pales a little, which is good. She starts thinking of other ways she can earn the treat. Hmmm, how about reorienting to me after she goes through the door? Great!
Premack Successes
Let this post be a cautionary tale. But lest it appear that I am saying not to use Premack at all, let me mention some Premack reinforcers that have worked really well for me.
The two ball game: reinforcing Clara for releasing the ball by throwing another ball (this works with one ball, too, but was easier for me to teach with two)
Tug and flirt pole releases: reinforcing them with the resumption of the game (I should mention that I don’t think I would have succeeded with this one without the help of my teacher, though)
Putting on the leash: gets reinforced by getting to go somewhere
Agility sequences: reinforced for Summer with play in the water hose
Loading into the car crate: getting to go somewhere
Getting and staying in a down when I walk in the room with something in my hands: gets reinforced by getting to sniff what is in my hands (guess who: Clara)
Walking nicely on leash: reinforced by opportunities for Zani to sniff
Most behaviors: reinforced by eating food treats. Gotcha! Eating is a behavior. So really, everything is Premack.
I’m always discovering hidden genius in the Training Levels. Sue Ailsby talks about using Premack or life rewards plenty. She seems personally to be a master at transitioning to life rewards. But she uses food first. Using doors as an example: Level 1 Sit, Step 4: Dog sits by an open door. A whole Step dedicated to using food treats to teach the dog self-control around a door. Level 3 Zen: this whole Level behavior is entirely about self-control around doors, and you don’t send the dog charging out as a reinforcer once! Using food can diffuse the emotional potency of doors to the outside. It makes the door area just another training environment.
So now, almost 6 years into our relationship, Summer and I are spending a whole lot of time doing “silly dog tricks around doors.” To undo the problem I helped to create–with this particular dog–by trying to use the Premack principle first.
What about you all? Am I the only one who has made some poor Premack choices or implementations? And can anyone help me come up with a more general–or more specific–guideline for when Premack might not be the best idea? I don’t think I have ever seen this discussed online.
*A technicality, but it’s important. Notice I haven’t said “Premack didn’t work.” That’s like saying that reinforcement didn’t work. Reinforcement is defined by its effects on future behavior. If the behavior didn’t increase, then there was no reinforcement. Likewise, you can’t say, “Premack didn’t work.” If what you tried didn’t reinforce the behavior, there was no Premack.
Ever since she arrived at my home at the age of 10 weeks, Clara has been a challenge.
One of her more problematic behaviors was her mugging of my face whenever it got within range. It happened all the time. How many times a day do you lean over your puppy, or lean over in her presence to pick up something off the floor? Most often something that she either dropped or shouldn’t have. Answer: a lot. Except not me, anymore, because she shaped me not to. If a strong, speedy puppy came barreling at your head every time you bent over, you might modify your behavior, too. So I do this embarrassing dance whenever I need to pick something up: distracting her, sneaking past, or trying to move REALLY FAST (which of course makes her all the more excited when she does catch me). Continue reading “Get Out of My Face! Teaching an Incompatible Behavior”
Zani rolling over in a shaping session that we both enjoyed
This is an expansion of a post about a possible cause of stress in shaping that I sent to the Training Levels Yahoo group.
Shaping involves extinction. That is, ceasing to reward something that has been repeatedly rewarded. In the real world, for humans and observably for other animals, that is stressful. The classic examples are when an elevator stops coming when the button is pushed, or when a candy machine just sits there after you put in the correct change and push the button. What usually follows? In the elevator case, repeated pushing of the button. Harder, faster. With the candy machine, all that, and possibly pounding, shaking, yelling. If you think about an animal’s behavior being tied to survival, something suddenly not working anymore is a danger signal. Oh oh, this place or this method that I was relying on no longer provides food. I’m going to have to start all over again and find somewhere or something else.
We are taught that when we suddenly stop rewarding something that a dog has been rewarded for, to be ready for an extinction burst. That is, the behavior rises in frequency and intensity before it fades away. Extinction is not fun for the dog in this circumstance! It is frustrating.
OK, back to shaping. When we shape, we are introducing tiny little extinctions over and over again. That’s how we get successive approximations to the final behavior. “Fido, THAT behavior is not getting paid for anymore, it is up to you to figure out something that is.”
When I see the really great trainers shape, there is another characteristic besides their ability to detect the tiniest behaviors and differences in behaviors to reinforce. Another skill is that they are constantly watching the animal’s demeanor, as much as its actual movement, and are responding to that. They can keep that extinction process as gentle as possible and keep the animal trusting that the world hasn’t come to an end when they stop clicking for something. And of course these two skills go together. Seeing and responding to the tiniest movements does tend to keep the rate of reinforcement high.
Also they think empathetically. There is a clinically proven human tendency (the “curse of knowledge”) to assume that when we have something visualized or auralized in our heads, that the others around us automatically will see it, hear it, understand it quickly. Great teachers learn that this isn’t the case. And great shapers keep in mind all the time that the animal may not have a CLUE to what they themselves have so clearly in their heads.
Finally, with our pet, service, and performance dogs (i.e. dogs who live with us) it comes down to the trust account. It needs to be very high for some animals to enjoy shaping as much as we ourselves might. They have to trust us that the lack of a click, and a little extinction, is not the end of the world. I will admit to making mistakes about this. Shaping is so cool; it’s like being handed a shiny new toolbox with all sorts of fun things inside. I’m a pretty empathetic person but I will tell you that I have gotten overexcited about this tool and plowed on through signs of big frustration from my animals. I have recordings that I will probably never show anyone else of shaping sessions I did very early on with both Zani and Clara. They went on for several minutes. We got to our goal (MY goal). But neither dog was having fun after the first minute or so. They were showing stress and frustration. Zani was whining. Clara was spinning, which is her superstitious and stress related behavior. I was pressing on towards the goal insensitively.
Another thing to keep in mind is that the shaping process is usually reinforcing–to the human! Shaping is incredibly cool! We dangle it in front of trainers who are considering “crossing over.” Look what you’ll get to do with your dog! Many of us need to be careful about going overboard.
Just like any other activity, some dogs are going to intrinsically enjoy shaping more than others. But we are trainers, right? If using shaping is important to us, we need to find ways to make sure it is fun for the dog. A little stress may be a good thing in life, but if an animal is chronically averse to training activity we like, it’s time to do something about it. We probably need to gentle down the extinction process. And mind our trust accounts.
A few thoughts on how to do this:
Watch the dog and get to know her signals.
Pay attention to how long you are waiting if you are withholding the click. That’s when the extinction stress can build up.
Start with very short sessions: just a few clicks.
Be willing to stop before achieving a pre-ordained goal (This is a hard one! We tend to be so goal oriented.)
Have an environmental cue that lets the animal know when you are shaping and when you aren’t.
My friend Lynn says, Teach it! Think of it from the learner’s point of view.
Lynn also says do little sessions of “shaping nonsense.” Make sure both you and the dog approach it as a game.
Don’t do like I did with Zani and start shaping with a brand new rescue dog just because you can. I wish I had built up our trust a little better before doing that.
Here are my three submissions to ShapeFest 2012 a few months ago. I’m pleased with my dogs’ demeanor in all of these. Clara is still the most serious, but showed only a few little stress signs. Her main stress behavior is a counterclockwise spin. She does a couple of spins starting at 2:20 but it’s hard to tell how much is stress and how much is just a behavior she is trying. Since her pace is not frenetic, my guess it that they were mostly offered behaviors.
Shaping Zani to roll over
Shaping Summer to mount a platform, using playing in the hose as the reinforement
Shaping Clara to do a distant paw touch
I bet some of you out there have some other suggestions about making sure shaping is fun. Care to share?
