eileenanddogs

Category: Reinforcement

That Damn “Roll Out the Carpet” Trick

That Damn “Roll Out the Carpet” Trick

Tan dog with black muzzle sits next to a tightly rolled up maroon carpet. She is sitting on the tiny part of it that is not unrolled.
Clara dutifully sitting on her sliver of mat

I picked the “Roll out the carpet” trick from the novice trick list from Do More with Your Dog because it looked fun and more trick-like than a lot of the other behaviors. We had been doing things like sits and downs and walking on leash and targeting. This was more like a real trick. It would be new, but still looked like a fairly straightforward one because Clara knows how to push things with her nose.

The definition of the trick is:

Dog will use his nose to unroll a rolled-up carpet. Carpet can be a yoga mat, rug or towel and should be roughly 5 feet/~2 meters in length.

DMWYD Novice Trick List

I have rolled food up in towels for Clara before as enrichment, so that seemed like an obvious way to practice. So I took a 5-foot rubber-backed rug and rolled it up with treats inside, and she promptly unrolled it to get the food. I had Clara do this for a couple of days. Easy Peasy.

But that was for practice. Luring is allowed, but I’m not sure about luring-and-eating-as-you-go-along. Even if it’s allowed—the rules for Novice tricks are pretty loose—to me, it’s not in the spirit of the trick. So the next time we practiced, I rolled up the carpet with no treats. Guess what happened? See the photo above?

Clara gave the rolled-up carpet a good sniffing all over, then sat on the little strip that wasn’t rolled up and looked at me. There was obviously no food in there, so why should she bother? Maybe starting with a loaded-up carpet wasn’t the best idea after all!

I had thought the original discriminative stimulus (cue) to get her to unroll the carpet was the rolled-up carpet. But it was the rolled-up carpet with treats in it. I had annihilated a giant lure (perhaps 20 treats) in one blow. Why should she bother with an empty carpet?

Back to Square 1. I realized I was going to have to actually teach the trick instead of coasting in on previous behaviors.

First Teaching Attempt: Get Clara’s Nose in the Right Spot without Treats

I started rolling up an empty carpet and shaped a nose touch in the correct area to push the carpet. This wasn’t hard. She would sniff when she approached the rolled carpet, anyway. So I turned that sniff into a little nudge. And I was thoughtful about my treat delivery, aiming for the little crack under the roll of the mat so I would direct her nose right back to the correct area when she went for the treat.

Tan dog with a black muzzle and ears is putting her nose under part of a rolled up carpet and receiving a treat
Crappy photo of my glorious treat placement. She always scrunched herself up to stand on the mat, because guess why?

However, I had two problems. One was that she has an enormous reinforcement history (there’s that problem again!) for lying down on mats or anything matlike. Possibly the most reinforced behavior in her life. So even though I kept my rate of reinforcement high for the nose touches, whenever there was even a momentary lull, her first choice was to lie down on the mat.

Tan god with a black muzzle and ears is lying down on a maroon carpet. The very end of the carpet is turned over, showing the white backing.
This is what we do on mats: lie down.

The second problem was yet another behavior that was stronger than the nose push: a foot target. She would sometimes hit the unrolled part of the carpet with her foot or stand on it.

Standing on it was incompatible with unrolling it for sure! And once she would start these other two highly reinforced behaviors, it was not likely she would find her way back to the nose touch. So I didn’t just leave her to figure it out. That would have been too frustrating. I would interrupt, ask for a nose touch to my hand or simply toss a treat, then start us over again.

I did succeed in shaping the gentle sniff under the rolled part of the rug into a nudge, then a push. Sometimes she would give a big push and the whole thing would unroll! I reinforced well for that, but again, I didn’t feel like it was in the spirit of the trick. It happened frequently when I used a yoga mat instead of the rubber-backed carpet runner, so we stuck with the latter.

I was getting the nudge, but I had a problem. I needed to thin my reinforcement schedule and get enough pushes from Clara to unroll the carpet completely before I reinforced. But I had these two other behaviors lurking, ready to pop out the minute Clara didn’t get reinforced for a nose touch. I knew if I tried to thin my schedule now, the first time I didn’t reinforce a nose push (because I wanted a second one), she would try one of the other behaviors instead.

Extinction and Thinning the Ratio Schedule

I’ve made it no secret that I generally pay my dogs for every behavior. You can see my article on it here and another by Dr. Eduardo Fernandez here. You can also look up Nevin’s work outlining the arguments for rich reinforcement schedules creating behaviors that are resistant to extinction (Nevin, Mandell, & Atak, 1983).

I do have a few exceptions to using a 1:1 ratio schedule with my dogs. For loose leash walking, I have extended the number of steps between reinforcers. I probably reinforce on a VR15 (steps) or so. I have also trained stationary duration behaviors where the reinforcers get fairly spaced out. For instance, there can be time periods between reinforcers measured in minutes when I am reinforcing Clara for staying on a mat while I work in the kitchen. I have at least one behavior chain (retrieve) where I generally only reinforce the terminal behavior. Finally, just living with my dogs, sometimes I randomly don’t reinforce for everyday behaviors. But I probably reinforce daily behaviors far more than most people. For instance, I still reinforce 10-year-old Clara with food virtually every time she pees or poops in the yard.

What I haven’t asked for from Clara, since back when I was working on the Training Levels, is multiple iterations of the same behavior for one reinforcer. What Sue Ailsby calls “twofers.” I found this out the hard way early in our trick training endeavor. Clara could not do puppy pushups unless I reinforced every behavior, or at least every other one. Doing six iterations, as is required for the trick (sit, down, sit, down, sit, down), was not possible for us. On the third cue or so, if I failed to reinforce for a sit or a down, she “assumed” she was wrong and started hopping around and throwing behaviors, usually a stand or a hop. I got an extinction burst. How humbling. I hadn’t worked hard enough on cue recognition.

We had an even worse situation with unrolling the carpet, because my goal was to cue her to unroll the carpet, which meant nudging it up to five or six times before it was all the way unrolled, then reinforce. Multiple nudges for one terminal reinforcer! I knew the nudging was still weak enough that I needed to reinforce every single one for a while. Because as soon as I would space out the reinforcement, in would pop into the foot targets and lying down. And I don’t want to put her through extinction without a really clear idea of what she can do for reinforcement.

Then I realized what I should do.

Backchain It!

I don’t think I’ve ever written about backchaining here. I don’t teach many chains. Backchaining means you start with the last behavior of a behavior chain first and work backward. There are several benefits. One is that you load a lot of reinforcement onto the final behavior (stay tuned to see the result in the video below). Another is that because of this, the dog is working toward the more familiar part of the chain that has gotten more reinforcement.

I can think of three behaviors I backchained. First, I backchained a retrieve with Summer and Clara. I also backchained Clara to drop a ball into a bowl using this video as a model. That’s a good video that shows how backchaining can work, if you are curious. It can be almost magical. I also backchained stopped contacts in agility.

Here is how I I used backchaining to get out of the rug trap.

I folded over only the very end of the rug. Clara had enough practice with pushing at the rug that she happily unrolled the little end I had folded over. I didn’t load it with treats, but she had enough experience by then that she would do it without seeing the money on the front end. We did many reinforced repetitions of opening one fold. Many. Then I folded it over twice. Oops, too soon! Got a down and a foot target. Went back to the beginning with just one fold, worked up again to two, and voilà! She pushed it hard enough (or pushed twice) to unroll both folds! Lots of reps of that, too. So we continued, working backward, with me rolling the rug more and more. I sometimes gave interim treats. She was giving multiple pushes rather than one constant one, which was fine with me. I didn’t want her to go from feast to famine, but I wanted her to gradually learn I would pay well if she performed the nose push multiple times to get to the end criterion: unfolded rug. That was backchaining.

While working on the trick, I also remembered she knows how to get food out of a rolling food toy, so I got out the Tricky Treat Ball and fed her some of her breakfast in there. It seemed like a good idea to build some more repetitions of nose pushes however I could get them.

As we got close to success with the backchaining, I added a cue, “Push,” and started using a conditioned reinforcer, “Good girl,” instead of the intermittent treats to let her know she was on the right track. As for that verbal cue: I was cheating a little. It didn’t matter what I said. Clara didn’t instantly learn the specific meaning of “Push.” If I were to say “Push” to her when she was lounging on the couch, for example, she wouldn’t start hunting for a rolled-up rug to nudge. It’s contextual. “Lady says something in a certain tone while I’m standing on the rug, so I will do the thing I just did.” But hey, it worked.

A tan dog with a black face and tail pushes an orange ball filled with food with her nose
I always think of the Tricky Treat Ball as “Summer’s food toy,” but Clara gained some fluency at it after she aged out of trying to eat the toy itself.

Progress Video

The video shows the steps we took and our victory a couple of days ago. For such a simple-seeming trick, this feels like quite an accomplishment. But I know exactly why it was a challenge for us, and I’m pretty pleased I could thread my way through all those heavily reinforced but “wrong” behaviors to tease out the right one.

I love the last iteration of the trick on the video. She pushes the rug several times and ends up with a small flap of it still folded over at the end. She looks at me, she looks at the rug. I am holding my breath, waiting for the reinforcement history to burst into the picture. But the folded over flap, because of the backchaining, became a pretty good discriminative stimulus for “push with your nose to unfold that.” She pushed both sides of it to open the rug flat all the way!

She never did one constant nose push, but it doesn’t appear to be a requirement, so I don’t think we’ll bother. We have already learned a lot by working on this trick!

Speaking of learning a lot, shout out to Marge Rogers for not saying, “I told you so!” She’s been trying to get me to train more tricks for years!

Copyright 2021 Eileen Anderson

References

Nevin, J. A., Mandell, C., & Atak, J. R. (1983). The analysis of behavioral momentum. Journal of the Experimental analysis of behavior, 39(1), 49-59.

Clara’s Stand Disaster and Why She Still Hops

Clara’s Stand Disaster and Why She Still Hops

tan dog with black muzzle stands on all four feet on a mat
This calm stand happened during a time when we weren’t working on it, of course

I considered titling this post “Eileen’s Stand Disaster,” but I thought that might be too confusing. Clara was the one standing, but the disaster part was definitely on me.

Thousands of people worldwide have used Susan’ Garrett’s fun method for teaching the stand and gotten fabulous results. I wasn’t one of them, but I blame myself, not the method.

The method is to have the dog in heel position in a sit, and to use a hand target above the dog’s head to get them to leap into the air, hit the target with their nose, then land on four feet. It’s a fun, flashy behavior. But the activity got Clara over-aroused, and I didn’t know how to handle that.

When I casually mentioned on social media that Clara and I had bombed using Susan’ Garrett’s method, a group of angry fans came for me. But wait! I am a Susan Garrett fan, too! I took part in the very first Recaller’s Class, and that was probably the time I saw all three of my dogs at their happiest. I respect her work and have used lots of her methods, both in and out of agility settings.

My failure with that stand method was just that: my failure. Between Clara’s temperament and my weaknesses as a trainer, we had a wreck that kept on wrecking. Not every method works for every trainer’s skill level with every dog.

What Did the Disaster Look Like?

I messed around with the method for a couple of years, alternating between trying to make it work and trying to reteach stand another way. Here’s a video from that time. It’s pretty embarrassing, but things had been even worse earlier. You can see (and hear from my yelps) that Clara’s arousal level was a tremendous problem. I know better now how I helped ramp her up. I literally fed into it by feeding too rapidly and never adding duration. Shark creation.

And no, I don’t know why the half-squat position with her back legs crept into her stand!

I do kind of wish I had that “leapfrog” behavior at 0:05 on cue, but I know better than to work on it! Another thing to notice is at 0:50 when, after trying unsuccessfully multiple times to get her to stand with a forward hand target, I give up and say “OK.” That, unfortunately, became the verbal cue for the behavior. What to me was a release cue, to her meant “stand.” You’ll see it in subsequent videos.

Clara is not always ramped up. She can melt into a mat and relax, and has successfully relaxed through every session of Dr. Karen Overall’s Relaxation Protocol in several locations. But jumpiness became part of the stand sessions.

Susan says in her video: “Even a six-year-old can do it.” Sigh. But I couldn’t, not with this dog.

Teaching Stand Successfully with Previous Dogs

I taught standing on cue to two other dogs with no problems. I taught Summer while training for competitive obedience and rally. I believe I used targeting but forward, not high in the air as in Susan’s method. Later, I taught Zani an adorable kickback stand using Sue Ailsby’s clever luring method: you lure them backward so they don’t step forward. Stand was the first and maybe only behavior I taught Zani by luring with food. I had to persuade her to follow the lure, since we had done so much leave-it practice.

This video of Zani is from 2018, after her spinal cord concussion. She couldn’t do a tight sit because of the effects of the spinal cord injury on her right rear leg, but she could still do the kickback.

Thanks for indulging me. Yeah, I posted this video partly to calm my ego, but also, damn, Zani was so cute!

What Were the Problems with Teaching Clara to Stand?

Clara gets aroused when she moves. A common problem, but I had not experienced it with any dogs before Clara. For instance, when we practice loose leash walking—ouch! She gets excited and super chompy when she gets treats while in motion. So when I taught her to leap from a sit or down and touch my hand in the air, then jammed food in her face when she hit the ground, I got her all worked up.

I was trying to capture the stand before she moved, so I shoved the treat into her mouth reallyreallyfast. And, predictably, got shark behavior in return.

Also, when Clara got confused about what we were doing, she would land in a sit or a down and stay there. I responded by giving my release cue to get her to move. She would stand, and I would reinforce. So the release cue (“OK”) became the cue for stand. By the time I added a real cue (“Brace”), I had already accidentally but firmly taught her to listen for “OK.” And it was unhandy that Clara popped into a stand whenever I tried to release her from anything.

I kept starting over with the process, so I got no duration.

I had an over-aroused, grabby dog who would bounce around nervously whenever I tried to train stand. Whatever I did with my reinforcement mechanics left her manically continuing to try other behaviors.

How I Solved the Problems

Clara already knows how to stand, of course, as does any dog with typical mobility. She stands dozens of times every day, meaning she performs both the motion of standing up from a sit or down, and the duration behavior of standing around. Standing in everyday life doesn’t arouse her; I created the arousal in our training.

A few months ago, I started over one more time. I took a page from my previous training and looked up my video on capturing stands. I thought I had made the video with Zani, but there was Clara, in 2015, with a nice little start on a fairly calm kickback stand! LOL, that’s what can happen when you write lots of blog posts and make lots of videos over the years! I think I had done that before I got her all overwrought with the jumping and targeting method! In the video, you’ll see her roach her back a couple of times, but her rear legs are in a much better position than the ultra-squatty stuff reinforced during the Bad Times.

So this year, I copied the steps from my own video.

First, I captured quiet stands for a while (not the behavior of standing up, just standing). I changed the picture from the method that didn’t work for us and stopped using the heel position setup. After sessions of capturing, I set her up facing me and waited. She offered movement into a stand quickly (but not nervously!). I was careful not to feed too fast, and I added duration as soon as I could. (One of my tragic flaws as a trainer is neglecting duration.) And I used a new cue—a flicking hand signal. No more “Brace” or “OK.”

This video from April 2021 through today shows our progress. She was still crouching somewhat with her hind legs in April, but that is mostly gone now.

Reinforcement History: Ghosts of Behaviors Past

But my “sort-of “victory isn’t the point of this post. I did some decent problem-solving. I was very patient. I got back a pretty nice stand behavior that we can continue to work on. You’ll see it in our next trick training video as well. But the nervous behaviors I reinforced during the stand practice still pop up frequently in training sessions of all sorts.

Behaviors rarely diminish all the way to zero. Clara unfortunately has a big reinforcement history for the debacle-stand. Here are a few examples of the attendant behaviors reappearing. And at the end, she does a perfect stand from a down (in response to my release cue) while I turn my back to go turn off the camera!

The concept of reinforcement history comes straight from Thorndike’s Law of Effect.

Of several responses made to the same situation, those which are accompanied or closely followed by satisfaction to the animal will, other things being equal, be more firmly connected with the situation, so that, when it recurs, they will be more likely to recur; those which are accompanied or closely followed by discomfort to the animal will, other things being equal, have their connections with that situation weakened, so that, when it recurs, they will be less likely to occur. The greater the satisfaction or discomfort, the greater the strengthening or weakening of the bond.

Thorndike, 1911, p. 244

The matching law quantifies the Law of Effect. The percentage of the time a behavior is reinforced will be reflected in how often the animal performs the behavior. So it’s a numbers game. If I stop reinforcing Clara for popping into a stand after I give the release cue (the last behavior in the above video), the stand will gradually decrease in that context. But the leapfrogging around in training sessions will be hard to eliminate because it got a ton of reinforcement in the past and gets chained into other behaviors I reinforce. You can see me reinforcing these chains throughout the video. The chain problem is not impossible to fix, but I need to sit down and think about whether I (with my own limitations) can carry out a plan without frustrating Clara, whether it’s worth it to try.

In the meantime, I’m pleased I am getting a semi-normal stand.

Copyright 2021 Eileen Anderson

Reference

Thorndike, Edward L (1911) Animal intelligence: Experimental Studies. Macmillan.

Tan dog with black muzzle and tail sits in "front" position and gazes up at a woman standing
Clara thinks “front” is a great game. Stay tuned for more tricks!

P.S. Although I didn’t categorize it that way, this post was born of the trick training we are working on. We aren’t through with tricks by a long shot!

I Just Show Him the Water Bottle and He Behaves—I Don’t Have to Squirt Him!

I Just Show Him the Water Bottle and He Behaves—I Don’t Have to Squirt Him!

Some people make claims like the one in the title out of true ignorance. They can’t identify how the behavior change is working. I’ve been there. It’s easy to believe that if one can get a dog to do something without discomfort or physical force in the moment, the training method is benign. We forget what transpired before.

There are others who make claims who, I suspect, do understand the method they are using. For them, it’s a game of “let’s pretend I’m not using force.” Some trainers use those statements to entice customers that their methods are humane or based on positive reinforcement. Some may have an interest in throwing fog into arguments on social media.

These methods are the topic of this post. Here is why waving a stick (at a dog who has been hit with one), or showing the spray bottle (to a cat who has been sprayed by one), and countless other things that don’t touch the animal are working through aversive control.

The Little Whip

When I was a kid, we had horses. I rode from a young age until we moved to town when I was about 15. For gear, we usually used hackamores and perhaps a bareback pad. More often bareback. Very rarely did we actually saddle up the horses or use bridles. Before the equine folks step up to the podium, I now know that the hackamores, with their pressure on the sensitive nose, were likely not comfortable either. But it appeared that the hackamores were less intrusive to our particular horses than the bitted bridles we also trained them to accept.

But don’t be misled. The methods we used were not kindly, except compared to those of some of our neighbors. We used pressure/release, yanking on the lead rope, kicking with our heels, smacking the horses with the reins or a whip, and using the reins to turn or stop the horse. I may have had spurs; I know my sister did.

We didn’t use positive reinforcement when riding. There were no appetitives involved except whatever pleasure the horses got from getting out in the world to walk and gallop around, and the feed we gave them before and after, as we were preparing for and cooling down after rides.

A quirt, or small whip. Except for the metal, it looks like a great tug toy!

I used a quirt, a short whip. It looked something like the image to the right. I don’t remember where I got it or whose idea it was. But I remember using it when I rode.

When I wanted my horse to go faster, I would swing the quirt around behind me to strike her on her butt. I’d do that a few times until she had sped up to my liking. We all knew how to do that with the ends of the reins, too.

I noticed after I had used the quirt for a while that I didn’t actually have to hit her anymore. With her excellent peripheral vision, she would see me swing the quirt forward, winding up to land a blow on her butt. She started speeding up when she saw the quirt moving and before I actually hit her with it. I adapted my behavior, whether out of kindness or efficiency, I don’t know. But I rarely hit my horse after I learned that all I had to do was to threaten her with the little whip.

Even at that young age, I realized what was happening, although I didn’t have the words for it. I do now. In response to my use of the quirt, my horse was changing her behavior from escape (speed up to make Eileen stop hitting her) to avoidance (speed up sooner to prevent Eileen from hitting her).

Escape and avoidance are the two faces of negative reinforcement. My horse’s behavior was under aversive control.

What Did I Think about It?

I could have gone around saying, “Using the quirt isn’t cruel; I don’t touch her with it.” I don’t think I said that because I understood that the quirt worked because I had hit her with it, and could hit her with it. The movement of the quirt had become a threat. That’s still aversive control.

If I had never hit her with the quirt, if she hadn’t gained that history, she would have had no reason to speed up in response to the swing of it unless the movement itself scared her. But she would probably have habituated to the movement if there had been no following slap. There would be no threat.

Note: If this post appears on the websites Runbalto, Scruffythedog, Snugdugs, or Petite-Pawz, or frankly, anywhere else, know that they are reposting without permission and in most cases without credit. This is my intellectual property, not theirs. I haven’t had time to file DMCA takedown notices yet.

Spray Bottles

When I was in my late teens and living on my own, I got a cat. Nobody I knew then talked about training cats. We lived with the “cat” things they did or interrupted them in unpleasant ways, usually yelling or using a spray bottle with water. Some people even used lemon juice or vinegar.

I used a spray bottle with water. I found out, over time, that the spray bottle worked the same way as the quirt. I remember using the spray bottle when my cat would get on the dining room table. I’d spray him as long as I needed to until he’d jump off. This was the escape flavor of negative reinforcement. He made the aversive stimulus stop with his action of jumping down.

But the same thing happened with the spray bottle that had happened with the quirt years before. It took fewer squirts to get him to move, and finally, all I had to do was wave the squirt bottle in his direction or even walk over to get it. I didn’t have to spray him at all. This was avoidance. Still negative reinforcement.

Was there also positive punishment involved? Maybe. I don’t remember for sure whether the behavior of getting on the table decreased, but I don’t think so. So there may not have been P+. But there was definitely negative reinforcement, two flavors of it.

It would have been easy to eliminate, decrease, or prevent my cat from getting on the table to begin with. I could have used management and positive reinforcement. I could have provided him with several elevated beds and perches. And I could have taught him to target my hand or a target stick so I could move him off the table using positive reinforcement. I did not know of those options then.

Is Avoidance Better than Escape?

Most dogs will work to escape or avoid body pressure

You will hear people proclaiming that they don’t have to use force anymore.

  • “I don’t have to vibrate the collar anymore; he behaves when I just make it beep.”
  • “I just show him the spray bottle.”
  • “I just start to roll up a newspaper and he shapes right up.”
  • “I just walk toward him and he pops back into a sit.”
  • “I don’t have to throw the chain anymore; she stops when I wind up to throw.”

Is this force-free training? Of course not. There would be no avoidance if the animal hadn’t experienced the unpleasant thing first. And not usually just once. They likely experienced it repeatedly until 1) they learned how to make it stop, and 2) learned the predictors that it was about to happen and responded earlier.

In learning to avoid the unpleasant stimulus, the animal may prevent pain or even injury. So of course those are benefits. But is that an advantage to brag on? What about the pain or injury it took to get there? “I don’t have to whip the horse anymore. That was so unpleasant that she learned how to avoid it.” Yay?

How to Tell When Avoidance Is Involved

Avoidance is complex. A lot of behavior scientists have put their minds to the question of why an organism will work for the goal of nothing happening. I’m not even going to get into that here, but if you are interested, most behavior analysis books have a section on it.

Besides being complex, avoidance can be hard to spot. Again, it’s because we don’t see a blatant aversive in use. Think of the videos by aversive trainers of a bunch of dogs on platforms lying very still for long minutes. We don’t see them getting hit, yelled at, or shocked. But they are usually frozen and shut down. They have learned that the way to avoid being hurt is to stay on their platform. Body language is one tell. They are often crouched, not relaxed. Their eyes are either fixed on the human, or they have checked out and are going, “La la la” in their heads. They are not casually looking around the room or wagging their tails.

But the other thing to look for is this. Do you see any appetitives in the picture? Is anyone going around giving the dogs a nice morsel of food every few minutes or even more often? Rewarding them with a game of tug? Granted, some trainers use both aversives and positive reinforcement. So even if you do see food, there still may be aversives involved. But if you see frozen dogs not moving a muscle and no food or toys in evidence, you are probably seeing avoidance.

Another easy place to see it is in traditional horse videos. Horses are so attractive and look so beautiful being put through their paces that we dog people can often be fooled. There will be some nice verbiage about the natural method or the “think” method or what neuroscience proves. But look for the appetitive. Look for the yummy treat or the butt scratches. Something the horse enjoys, not the relief of something uncomfortable stopping. If you don’t see the fun stuff, the good stuff, you are probably seeing aversive control. The horse is performing because of discomfort or the threat of it: avoidance.

Things That Can Work through Avoidance

  • Squirt bottles
  • Shock or vibration collars, both manually triggered or as part of boundary systems
  • Prong collars
  • Choke collars
  • Bark collars
  • Body pressure
  • Eye contact
  • Citronella spray
  • Whips
  • Plastic bags on a stick
  • Verbal threats
  • Chains or “bean bags” that are thrown near the dog
  • Penny cans
  • Picking up a stick or anything you might hit your dog with

Aggression

The use of aversive tools and methods can prompt an aggressive response. Some of the milder aversives are probably less likely to do that with the average animal. But it’s the animal that gets to define “mild” or not. I watched a YouTube video of a domestic cat aggressing at a woman who is threatening to spray him with a spray bottle. I’m not embedding or linking it because I don’t want to give it that support, but it’s among the first hits if you search for cats vs. spray bottles on YouTube. Here’s a description (not an exact transcription):

A small orange tabby cat is sitting on a wooden table next to a potted plant. A woman’s arm and hand come into the frame. She is holding a squirt bottle. The cat squints his eyes when the spray bottle first appears. She shoves the spray bottle nozzle into his face as she says things like, “Back up from the plant.” “I said, back up from the plant.” The cat responds to her movement and statements by repeatedly slapping the woman’s hand holding the bottle with his paw. He meows and whips his tail around. He actually advances on the hand with the spray bottle rather than retreating. She finally squirts him point-blank in the face, and he shrinks back a little and moves laterally but doesn’t get off the table. He goes to the other side of the plant. There are at least three aspects to her threat: the spray bottle itself, her advancing on him, and her verbal threats.

But this cat is not showing avoidance. He retreats only when sprayed directly in the face, and then only a few steps. But instead of avoidance, his go-to method is to lash out. My cat was more easygoing and merely worked to avoid the spray.

I’m not ashamed to say I was rooting for the cat in the video, but with a mental caveat. He’s lucky he’s small. If this were a large dog or a horse, similar behaviors would be extremely dangerous for the human, and the animal would be in danger of being euthanized for aggression. Even the small cat could be in danger of losing his home of life if he escalates further, except that his owner is making money on YouTube.

I include this story for two reasons: progressing to avoidance is not inevitable, and we can’t predict what kind of aversive use will elicit an aggressive response.

Avoidance Doesn’t Earn You a Pass

Teaching behaviors through escape and avoidance is generally unpleasant for the learner. Even in situations where we can’t see anything bad happening. if the animal is working to avoid something, something bad did happen. It could happen again, and the animal knows it.

Copyright 2021 Eileen Anderson

Positive and Negative Reinforcement by Jack Michael: A Misconstrued Article

Positive and Negative Reinforcement by Jack Michael: A Misconstrued Article

Thank you to the readers who helped me with this paper. Any mistakes are my own.

Some terminology in behavior science is notoriously hard to get one’s head around. One of these terms is negative reinforcement. Not only is this learning process itself a challenge to understand, but the terminology itself is counterintuitive. Behavior scientists specialize in training, teaching, and learning, so naturally, if a term from their own field trips people up, they are going to analyze the problem. The terminology for negative reinforcement has already been changed once, in the 1950s to early 1960s. There has been more discussion since then. This post is about the article that started the more recent discussion, and how it is often misunderstood in the animal training community.

In 1975, psychologist Dr. Jack Michael published an article named, “Positive and negative reinforcement, a distinction that is no longer necessary; or a better way to talk about bad things” in the journal Behaviorism.

This journal article is widely mischaracterized, in my opinion. It is commonly quoted by people who use aversives in training and seek to minimize that when discussing or defending their methods. And certainly, the title sounds very promising for just that purpose. But only if you ignore the last phrase about “bad things.”

Some people claim the article says that the distinction between the learning processes of negative reinforcement and positive reinforcement doesn’t exist or is immaterial. They say that the difference between positive and negative reinforcement is blurred and can’t always be determined. Some say that Dr. Michael dismisses all the possible reasons for maintaining a distinction between the two. This is false (see page 43 in the paper).

Michael’s paper centers on better ways to make descriptions of and determinations about the contingent processes of operant learning. The claim that Michael states that there is little difference between positive and negative reinforcement is false. This claim misrepresents both the focus and the conclusions of the article. Note again the last part of the title: “A better way to talk about bad things.” 

In the article, Michael asks whether we need to make the distinction between what we call positive and negative reinforcement. His final answer is yes, that we need the distinction. He concludes, “We need to make the distinction in order to have a name for the bad things in our world.” (page 43)

Dr. Michael is concerned about terminology on two fronts:

  1. He wants to get rid of positive/negative and present/remove in the descriptions for different types of reinforcement.
  2. He wants to find a better nomenclature to indicate when an aversive is involved.

He proposes a solution, which I will describe below.

There are four major parts to the paper: a history of the usage of the terms for reinforcement and punishment, a critique of the current terminology, a section that explores whether we need the distinction or not (his answer: yes), and a proposed solution. I’ll summarize each briefly. The following four sections are headed with the subtitles used in the paper.

1. A Brief History of the Distinction Between Positive and Negative Reinforcement

This section comprises 75% of the paper and is devoted to a retrospective of the usage of the terms for reinforcement and punishment, starting with Skinner in 1938. As some people know, what Skinner initially called “negative reinforcement” is what we now call punishment.

A textbook published in 1950 by Keller and Schoenfeld (1950) used different terminology, and in 1953 Skinner reversed his usage of the terms in his own textbook, defining them as we know them today. There was a period of transition—Michael mentions that it had to have been especially tough for the students who attended courses at the same time that employed different textbooks—and by the 1960s Skinner’s revised usage, what we use today, was in common use.

I am not going into detail here, but Dr. Michael did. Eight and a half of the eleven pages of the article are dedicated to the changes in definitions and usages of the terms and the resultant confusion. This is a major focus of the article and a major part of his criticism of the use of “positive” and “negative” with regard to reinforcement.

2. What is Wrong with the Present Usage?

In this section, Michael says, “Since 1953 there must have been thousands of man-hours spent in the attempt to prevent the learner of behavioral terminology from equating [negative reinforcement] with punishment…”

Even though I am not credentialed in that field, I know what he means. I have spent many hours myself figuring out the processes of operant learning, with bonus time on negative reinforcement, and many hours as well trying to pass on my basic understanding to others. Reinforcement, punishment, and the plusses and minuses can be confusing, especially since almost all the words used have other meanings or common metaphorical uses.

Michael goes on to describe another problem that includes semantics, in this often-quoted section:

Another difficulty with current usages is that the critical distinction between positive and negative reinforcement depends upon being able to distinguish stimulus changes which are presentations from those which are removals or withdrawals, and these latter terms are not very satisfactory descriptions of changes. The circumstances under which we have a tendency to say “present” certainly seem to differ from those where we say “remove” in vernacular usage, but some of these differences are irrelevant to a science of behavior, and there are a number of circumstances where the distinction is not easily made.

Michael, 1975, p. 40

Note that he is not saying that it’s difficult to detect the differences between aversive and appetitive stimuli. The issue he objects to is the use of the terminology of presenting and removing stimuli.

…In other words, from the point of view of the behaving organism presentations and removals are both simply types of environmental changes. If they differ, the difference must not be based upon the variables controlling the person who causes the change.

Michael, 1975, p. 40

This section merits careful reading. His major objections to the concepts of “presenting” and “removing” are that they focus unnecessarily on the actions of the environment or a third party and that they have societal and linguistic baggage (e.g., he mentions that removal can sound negative). He says what is really important to the subject organism is simply that something changed, and it is the point of view of the subject that we should be concerned about describing. We don’t need to talk about adding or removing stimuli, we need to describe bad changes and good changes from the standpoint of the subject. (Michael uses the terminology of “bad” and “good” throughout the article, which is also a deviation from standard practice.)

In other words, it appears that “present” and “remove” are abbreviations that can sometimes stand in place of a more complete description of both the pre-change and  post-change condition. The abbreviation is usually possible in the case of unconditioned reinforcements, although even here it must always be possible to infer the characteristics of both pre- and post-change conditions if we are to imply behavioral significance.

Michael, 1975, p. 41

An interesting point: he states “present” and “remove” are incomplete descriptions. He is not arguing to ignore the nature of the circumstances the organism finds itself in. He is arguing against shorthand. He is arguing that we need to describe the state of the environment and the nature of the change more accurately in order to determine the learning process in play.

There is much more in this section about presentations and removals not being specific enough for scientific usage, and it is in this section one really gets a sense of Dr. Michael’s concerns.

He also addresses an argument that has been going on for a long time in behavior science. It goes like this:

You can’t tell the difference between positive and negative reinforcement if you train using food because you don’t know if you are adding food or removing hunger.

Various people

Experts in the field discuss this question earnestly and with goodwill. But you will also see it glibly thrown into arguments by trainers who seek to mask their use of aversives (it’s is a favorite among force trainers). I humbly offer my own study of this question, but here’s a surprise. Dr. Michael addresses this very situation in his paper.

When we say that we present a food pellet to the rat the listener can always assume that the pre-change condition is one in which no food is available. We could say that we remove the “no-food” condition, but then the behaviorally important aspect of the change would remain to be described. When we say that we terminate a 50 volt electric shock, the subsequent “no-shock” condition can generally go without further description, but if it were described alone little information would be provided.

Michael, 1975, p. 41

He is saying that only one description is typically accurate in a particular situation because the other one fails to describe crucial parts of the situation. Again, he is arguing that we need to analyze the reinforcement situation with information about the environment before and after the change, not by focusing on one stimulus and whether someone “presented” or “removed” it. What is happening from the animal’s point of view? Is it a “good” change or a “bad” change, and does it involve a bad thing (aversive)?

3. Why Do We Bother?

In this section of the paper, Michael examines possible differences between negative and positive reinforcement and discusses whether each particular aspect could or should be the reason we need to make a distinction.

As we find ourselves applying behavioral analysis to more and more complex human situations we find it increasingly difficult to distinguish between presenting and removing, or we find an increasing number of situations that seem to involve both. A fairly common response to this situation is to avoid making the distinction, and simply refer to the relevant environmental change as “reinforcement,” without attempting to determine whether a positive reinforcer is being presented or a negative removed. One might well ask, then, why we bother making the distinction even in those cases where it can easily be made.

Michael, 1975, p. 41

If your goal is to generally minimize fallout of the use of negative reinforcement, you can cherrypick the above paragraph without continuing and make it look like Michael is saying the distinction between positive and negative reinforcement is unnecessary. On the contrary, this is the section where he specifically rejects that interpretation. He considers four reasons for making the distinction. He discounts the first three as follows.

  1. Are the (behavioral) strengthening effects of R+ and R- different? He answers that they are not any more different than the differences between different forms of R+.
  2. Do R+ and R-  involve different physiological structures or processes? He doesn’t think trying to make this distinction is a good idea in view of the changing field, but he leaves room for future research. This article was published in 1975, before most of the current discoveries that showed exactly that: that different physiological processes are likely involved (Overall, 2013, p. 69). 
  3. Should we keep the current terminology so as to warn people only to use “positive,” not “negative”? He appears to be asking whether we should actually appeal to the double meaning of positive. Again he answers no, that we shouldn’t base a scientific definition on a social distinction. (Note that in using the term “social distinction” he is referring to the words “positive” and “negative,” not to the actual learning processes.)

So he rejects three reasons for making the distinction between positive and negative reinforcement. Then, in a section that is rarely quoted, he goes on to answer his original question in the affirmative, saying that we do need a way to distinguish the difference. He says:

The layman frequently finds it necessary to identify an environmental event or condition as one which he doesn’t like, which he attempts to escape, or avoid. He may refer to such an event as “bad” (without the moral implications of this term), “undesirable,” “unfavorable,” etc., and he also has “punishment” to use as a contrast with “reward.” A science of behavior also needs a way of identifying such events.

Michael, 1975, p. 42

And finally:

We need to make the distinction in order to have a name for the bad things in our world…

Michael, 1975, p. 43

He is arguing that we need the distinction between what is currently called negative and positive reinforcement so as to be able to specify when a “bad thing” is involved. So it is incongruous that this paper is cited in support of arguments to blur and erase the use of aversives.

4. The Solution

Michael spends so much time focusing on confusing terminology in the paper that it is strange he doesn’t devote more space to making his solution clear.

But here is what he wrote.

So, the solution to our terminological problem is to refer to the good things as reinforcers and reinforcement and call the bad things punishers and punishment. One set of terms refers to changes which have a strengthening effect on the preceding behavior; the other to changes which have a weakening effect. The distinction between two types of reinforcement, based in turn upon the distinction between presentation and removal simply can be dropped.

Michael, 1975, p. 44, bold added by Eileen

The last sentence of that quotation can also be taken out of context in a misleading way. A hasty reader, or one with an agenda, can claim Michael is saying that there is no difference between the learning processes we call R+ and R-. But he has already said that we need to specify when there is a bad thing involved. He is arguing not to base the distinction on the terminology of presentation and removal. 

Finally he writes:

The arguments set forth above convinced me about 6 years ago to stop making the distinction between negative and positive reinforcement and to refer to the bad things as punishers and punishment.

Michael, 1975, p. 44

That is the way he achieves his goals of getting rid of the terminology of presentations and removals and finding a better way to describe the “bad things.”

It’s a shame that Dr. Michael doesn’t give some examples of applying his terminology. But I would suggest a couple of examples, following his lead. Both of these are what we would now call negative reinforcement.

  1. In a shock experiment with the goal of increasing behavior, the learning process could be called using reinforcement with the punisher of shock.
  2. In an escape protocol where an animal’s behavior is reinforced by giving them more distance from a scary thing, the learning process involved could be called using reinforcement with the punisher of a feared stimulus.

It seems clunky at first, but once you realize a bad thing (“punisher”) can be involved in reinforcement in only one way, escape/avoidance, it falls into place.

Dr. Michael makes it clear that we need to stipulate when there is a bad thing included as part of the learning process. He also states that what we call negative reinforcement includes a bad thing, and presents a cogent argument that the differences between what we currently call R+ and R- are important and are distinct from each other in real-life situations.

Epilogue

In 2013 (yes, I’ve been working on this post for seven years), I tried to contact Dr. Michael to ask for some examples of how he applied his terminology: how and when he made the distinction that a punisher was involved. I reached his wife, who said he was not able to discuss such things any longer due to dementia. He passed away this year: November 13, 2020.

Dr. Michael’s paper prompted several others in the same vein, questioning the terminology of “positive” and “negative” with regard to reinforcement. In my reading, the arguments had some of the same flavor but were not exactly the same. I’ve included those articles in the references below. My arguments above apply to Michael’s 1975 article alone.

These papers usually get a footnote in behavior science textbooks, but the standard nomenclature hasn’t changed to reflect the ideas put forth, which Michael himself later noted (Michael, 2005). I recently heard a behavior analyst being interviewed in a podcast voice a similar concern with “presentations and removals.” She mentioned that in her work it is most important to observe whether behavior is under aversive or appetitive control, and those are the classifications she uses.

And permit me one moment of editorializing: I don’t know any trainers who don’t use negative reinforcement. Even the kindly act of letting an animal leave or take a break from a difficult procedure means that R- is a planned part of a training plan. Most of us would agree that allowing escape is less intrusive than flooding, but we also try mightily to train with enough skill that the animal doesn’t want to leave in the first place. So my aim here is not to preach purity, although I try to avoid the use of R- in every possible way. My argument is with people who are disingenuous about their use, and who cherrypick quotes from this paper to attempt to obfuscate the contingent processes of operant learning.

Rest in peace, Dr. Michael, and I hope my efforts here have done this famous paper justice.

References

Baron, A., & Galizio, M. (2005). Positive and negative reinforcement: Should the distinction be preserved?. The Behavior Analyst28(2), 85-98. 

Baron, A., & Galizio, M. (2006). The distinction between positive and negative reinforcement: Use with care. The Behavior Analyst29(1), 141-151. 

Chase, P. N. (2006). Teaching the distinction between positive and negative reinforcement. The Behavior Analyst29(1), 113. 

Iwata, B. A. (2006). On the distinction between positive and negative reinforcement. The behavior analyst29(1), 121. 

Keller, F. S., & Schoenfeld, W. N. (1950). Principles of psychology: A systematic text in the science of behavior.

Lattal, K. A., & Lattal, A. D. (2006). And Yet…: Further comments on distinguishing positive and negative reinforcement. The Behavior Analyst29(1), 129.

Michael, J. (2006). Comment on Baron and Galizio (2005). The Behavior Analyst29(1), 117. 

Michael, J. (1975). Positive and negative reinforcement, a distinction that is no longer necessary; or a better way to talk about bad things. Behaviorism3(1), 33-44.

Nakajima, S. (2006). Speculation and Explicit Identification as Judgmental Standards for Positive or Negative Reinforcement: A Comment on. The Behavior Analyst29(2), 269. 

Overall, K. (2013). Manual of Clinical Behavioral Medicine for Dogs and Cats. Elsevier Health Sciences.

Sidman The Distinction Between Positive and Negative Reinforcement: Some Additional Considerations

Skinner, B. F. (1938). The behavior of organisms: an experimental analysis.

Photo Credit

Skinner Box diagram credit Wikimedia Commons.

Copyright 2019 Eileen Anderson

A Great Substitute for Canned Spray Cheese for Dog Treats

A Great Substitute for Canned Spray Cheese for Dog Treats

If the idea of giving junk food to your dog appalls you, don’t read this. But I will say that my concoction of goopy stuff is healthier than the original.

I won’t make you read the history of the world before presenting the recipe. To save some of you from scrolling down, here’s my best substitute for canned spray cheese. But feel free to read the story of my experimentation. It will probably help with your own. Also, Cheez Whiz is a U.S. product; I have ideas for my friends in other countries at the end of the post.

Continue reading “A Great Substitute for Canned Spray Cheese for Dog Treats”
The Stages of Crossover

The Stages of Crossover

When I crossed over to training with positive reinforcement, I had no idea how much my behavior and even my belief system would need to change. I had to question my faith in some long-held cultural assumptions and learn to rely on scientific observation and analysis.

Crossing over was a lengthy process for me, and even now, after more than 10 years, I occasionally fall back onto old assumptions and behaviors. I wonder sometimes if I am the only one so vulnerable to cultural programming. But a quick look around social media says no, I’m probably not.

Continue reading “The Stages of Crossover”
Oops! I Trained the Better Than Perfect Recall!

Oops! I Trained the Better Than Perfect Recall!

What if your dog’s recall is so good that she comes before you call her?

The little movie featured in this post shows the myriad ways a smart dog can mess up your plans.

small black dog running to come when called, to a recall

The Original Goal: Film That Recall!

This post brings together a lot of ideas I like to explore. Among them are cues, offered behaviors, and stimulus control. And like many of my posts, this one features an unintended consequence. I’ll explain.

Continue reading “Oops! I Trained the Better Than Perfect Recall!”
The “Invention” of Cues in Training

The “Invention” of Cues in Training

Hat made out of folded newspaper

Once upon a time, there was a girl who decided to teach her dog some tricks. She figured out that if she gave her dog something he liked after he did something she liked, he was liable to do the thing again. So she taught him some simple tricks using food and play as reinforcement.   

As she went along, her dog started finding playing training games lots of fun in and of themselves. But she still used food and play. He liked earning his “pay” and she liked giving it to him. She didn’t see any reason to stop.

This girl was unusual in that she didn’t try to tell her dog what to do in words. She realized what is not obvious to so many of us: he didn’t speak English. Things worked out just fine because he could generally discern from context and her gestures what she wanted to work on.

She used a little platform to teach him to pivot in a circle. He would put his front feet on the platform and walk around with his back feet and rotate. He got good at this and soon could spin in both directions. As soon as he saw the platform he would run over to it and start to pivot, although she could ask him to stop with a hand signal.

Continue reading “The “Invention” of Cues in Training”
Doesn’t Intermittent Reinforcement Create a Stronger Behavior?

Doesn’t Intermittent Reinforcement Create a Stronger Behavior?

This article was first published by Clean Run – The Magazine for Dog Agility Enthusiasts, in August 2017. I changed the title after publication in this version. Please see the note about that at the end of the article.


Three dogs looking through a fence. Continuously reinforcement. A recall trained via variable reinforcement probably won't get their attention.
If I’ve trained recall on a variable ratio reinforcement schedule, how likely are my dogs to come away from the fascinating distraction behind the fence?

Do I have to carry around treats or toys forever?

This is a common question from trainers who are new to positive reinforcement techniques. And most of us have heard the following typical answer.

Continue reading “Doesn’t Intermittent Reinforcement Create a Stronger Behavior?”
Herrnstein’s Matching Law and Dog Training

Herrnstein’s Matching Law and Dog Training

Chocolate cookies on a cookie sheet. The baker may do other activities while the cookies are baking as long as she shows up at the right time. Her behavior follows the matching law.
When we bake cookies, some reinforcement is on a variable interval schedule.

What Is the Matching Law?

Have you heard trainers talking about the matching law? This post covers a bit of its history and the nuts and bolts of what it is about. I am providing this rather technical article because I want something to link to in some other written pieces about how the matching law has affected my own training of my dogs.

Continue reading “Herrnstein’s Matching Law and Dog Training”
Copyright 2021 Eileen Anderson All Rights Reserved By accessing this site you agree to the Terms of Service.
Terms of Service: You may view and link to this content. You may share it by posting the URL. Scraping and/or copying and pasting content from this site on other sites or publications without written permission is forbidden.