eileenanddogs

Category: Extinction

All That’s Unpleasant Does Not Punish

All That’s Unpleasant Does Not Punish

I’ve written a lot about the behavior science definitions of reinforcement and punishment. That’s because they can trip us up so easily. Something can be attractive, but not always reinforce behavior. Something can be unpleasant, but not serve to decrease behavior even when it looks like it should. This story is about a natural consequence that seemed like it would decrease behavior but didn’t.

Continue reading “All That’s Unpleasant Does Not Punish”
Resistance to Extinction Can Be Incredibly Annoying

Resistance to Extinction Can Be Incredibly Annoying

black and brown hound dog mix sitting and staring in an example of behavioral resistance to extinction
Zani is too cute to be annoying, isn’t she?

I’ve written before about the concept of “resistance to extinction” and how it’s not all that it’s cracked up to be when we want strong, fluent behaviors from our dogs.

Continue reading “Resistance to Extinction Can Be Incredibly Annoying”
A Quadrant by Any Other Name is Still a Cornerstone of Operant Learning

A Quadrant by Any Other Name is Still a Cornerstone of Operant Learning

This 2003 edition book is $4.89 on Amazon. Contents: priceless.

There is a science that deals directly with how organisms learn and how to use that information to change the environment in order to change behavior. It’s called applied behavior analysis (ABA). It is the applied version of behavior analysis, which was referred to as the experimental analysis of behavior earlier in the 20th century.  It is descended from the work of the behaviorists such as Skinner and is a sub-discipline of psychology.

It is a rich field of study. Universities offer graduate degrees. At the same time, it is approachable. Many of the entry-level ABA college textbooks currently in use are readable to someone with a strong high school education and certainly to someone with a college education. They are generally self-contained, in that they don’t require a lot of previous exposure to terminology to be able to work through.  The books contain fascinating information about what makes us tick, why we do what we do, and how we might go about changing behavior if we needed to. They also teach skills in ethics and kindness.

Because they are written by experts in learning, the texts are generally well organized, interesting, and approachable. A sidebar in Paul Chance’s Learning and Behavior starts off, “What would you do if, while camping miles from the nearest hospital, you were bitten by a poisonous snake?” It goes on to discuss superstitious behavior. Other sidebars are titled “Punks and Skinheads,”  “Variable Ratio Harassment,” and “Learning from Lepers.” I’ll leave you to go find out the subject matter. This topic is a goldmine for the curious. It is relevant to everyday life and can teach knowledge and skills that are very practical. If you buy older editions of textbooks, as I usually do, the prices are quite reasonable. (For instance, here’s a link to Paul Chance’s Learning and Behavior, with the oldest editions first. You can scroll forward to newer editions as your pocketbook allows. The most recent edition is 2013.)

Like any field of study, ABA has its own terminology. When we first encounter it, two things typically happen. First, we think we know it already. Who doesn’t know what punishment is, right? Motivating operation—doesn’t sound too hard to figure out! Then we go a little deeper, and even though the words are familiar, the concepts may not be. Some are extremely unfamiliar. That can cause dismay. One of the problems in the dog training world is that a lot of people get stuck at that point.

Continue reading “A Quadrant by Any Other Name is Still a Cornerstone of Operant Learning”
How to Make Extinction Not Stink

How to Make Extinction Not Stink

[In operant learning], extinction means withholding the consequences that reinforce a behavior.  –Paul Chance, Learning and Behavior, Fifth Edition, 2003

Extinction not stink

This post is Part 2 (a year later!) of But Isn’t it Punishment to Withhold the Treat?

In that post I discussed the common error of arguing that withholding a treat from a dog in a training session (or other time) comprises punishment. On the contrary, when nothing is contingently added or taken away but behavior decreases, the process at work is extinction, not punishment.

But that is not to say that extinction is automatically better. In Dr. Susan Friedman’s Humane Hierarchy for behavioral intervention (see graphic below), extinction by itself is at the same level as negative punishment and negative reinforcement. They are roughly at the same level of (un)desirability, and the level of unpleasantness of any particular technique would depend on the circumstance and individual animal. Dr. Friedman makes a point to say that these three are not ranked in any particular order of overall undesirability.

Extinction is often overlooked when considering or analyzing methods. People often mix it up with negative punishment. It’s a bit of an oddball learning process since it applies to both operant learning and respondent conditioning. In operant learning it is sometimes jokingly called the “fifth quadrant.” The important thing to me is that its unpleasant effects can vary wildly, from practically nil to complete misery.

Dr. Susan Friedman's Humane Hierarchy: From bottom to top: Health, Nutrition, and Physical Setting; Antecedent Arrangements; Positive Reinforcement; Differential Reinforcement of Alternative Behaviors; Extinction, Negative Reinforcement and Negative Punishment; Positive Punishment.
Dr. Susan Friedman’s Humane Hierarchy

Extinction can be very, very frustrating. Here you are with these behaviors that you have been performing for such a long time that they are habitual, and all of a sudden they don’t work anymore! But there are ways to use extinction in combination with other processes to make it much less hard on the learner. And in fact, if we look at the Humane Hierarchy a little more closely we will see that extinction is actually lurking in another of the levels, partnering with something much nicer.

Let’s explore this by way of a thought experiment.

keysExtinction Scenario #1. You get your car out of the shop after a tune-up. You buy a pint of your very favorite ice cream, or other perishable treat. You realize you need one more thing from the store, so you lock the car and go back in. When you come back out, you try to unlock your car with a remote. It doesn’t work! You press the remote again and again. You press it harder. You aim it differently. No go. Then you try unlocking the door with the key. That doesn’t work either! You jiggle and jiggle the key, and try the different doors. Nothing works. You bang on the car doors. You can’t get into the car using the methods that you have always used. You are starting to cuss now. Your ice cream is melting. You finally yell at the car and it opens!

You drive back to the repair shop and ask the guy what the heck he did to your car. He said your car doors now work by voice control. Apparently he thought that sending you off to find that out on your own would be the best way to teach you.

Two questions. 1) Was that learning process fun? 2) What are your feelings towards the mechanic?

That is a description of the process of extinction. A behavior that has previously been reinforced is no longer reinforced. In this case it was actually two behaviors: opening the car with the remote and opening it with the key. Both used to be reinforced by your gaining entry to the car. Both stopped working with no warning. Stinky!

Three characteristics of extinction are the extinction burst, an increased variability of behavior, and aggression. We got all three.  When your normal methods for opening the car door didn’t work, there was a big burst of behavior from you as you tried stuff. You unconsciously started adding variety in how you performed the behaviors. And you started doing everything a little harder and banging on stuff. None of that was fun for you.

Now let’s try a different version of the scenario.

Scenario #2 When you first go to the mechanic, he tells you about a new option to have your car respond to voice commands, including that if you opt for the upgrade, in some cases the old methods will not work. You decide that it sounds good.*  Your mechanic takes 10 minutes to go over the voice commands that you will use with your car, including that you practice unlocking it with your voice.

When you stop off to go to the store and return to your car, if you are like 99% of the human race, that huge reinforcement history for using your remote or keys during your whole driving career kicks in and you initially try to use one of these to open your car. But the practice of the new behavior is fresh in your mind, so as soon as the remote doesn’t work, you remember to give the voice command. Your car unlocks!

But old habits die hard. You will probably be hitting that remote or trying your keys for quite some time, each time you approach your car. The old behaviors will diminish slowly as their reinforcement histories fade into the past and the practice of the new successful behavior overshadows them. However, there will be comparatively little frustration. You are never in the dark about what behavior will actually work. You’ll probably perform the old behavior once, go “oops!” and immediately use your voice without wasting much time.

Not so bad!

What About Dog Training?

Here are the dog training corollaries to Scenario #1 and #2 above.

Let’s say you want to address the following behavior problem: When you get out your dog’s leash, your dog gets excited and runs around getting all aroused, barking and jumping on things.

Scenario #1 You have never trained your dog to do anything, but you’ve had enough of the overexcitement. So you decide you aren’t going out that door until your dog sits calmly for you to put the leash on. So you take your dog into the front room and pick up the leash. Dog runs around. You just stand there. Dog jumps on you and on the furniture. Runs around and barks. This goes on for about 5, maybe 10 minutes. Finally your dog wears out and sits down and looks at you. You take one step towards him, holding the leash out to attach it. He gets all worked up again and you have to wait out another few minutes of excited activity. This happens over and over.

From your dog’s point of view, the rules have changed. All that previous barking and running around have been reinforced by getting to go outside. Many people frankly don’t have the stamina to outwait a dog in this situation, and will finally break down and take the dog out anyway, which worsens the problem (by finally reinforcing the behavior they’ve made it more persistent). If you do succeed and the dog calms down in 20 minutes on that first day, it may take a bit less the next day. But since this is completely new to your dog and you are asking so much of him when he is already wildly excited, it will take a while, and be a frustrating process for him

Cricket sit at attentionScenario #2 You have trained your dog to sit in all sorts of situations and for all sorts of reinforcers. He sits for his supper. He sits to go outside. He sits to greet people. He sits at the agility start line. He can hold a sit stay while you run around and play tug with another dog. So when you decide to teach him to sit calmly to put the leash on, you first practice some sits for treats in a random room of your house. Then you do the same in the room where you keep the leash. Then you pick up the dog’s leash and look at him expectantly. If he starts running around you wait. When he makes contact again you give him the expectant look. He will likely sit pretty soon. Treat!! He may jump up again when you approach him, but he is already learning.

This fits a pattern he is familiar with: sit and something good happens. You can use treats to reinforce those sits in this new situation so he doesn’t have to wait so long for the ultimate reinforcement, going out. You practice in small steps until you can put the dog’s leash on while he sits calmly. Depending on the dog and what you have trained, you may be able to take him straight out the door calmly that first day, or you may practice a few more days just putting the leash on and off before you go out the door.

Defining the Difference

Take a look at Dr. Friedman’s diagram again. See the area just below “Extinction, Negative Reinforcement, and Negative Punishment”? It is called “Differential reinforcement of alternative behaviors.” Guess what? That one corresponds exactly with both of the Scenarios #2 above. Dr. Friedman’s definition is, “Differential reinforcement is any procedure that combines extinction and reinforcement to change the frequency of a target behavior.”

Instead of being gobsmacked by the normal behavior not working anymore, the learners, dog and human, are given a big fat clue about what is going to work to get what they want. That clue is the positive reinforcement of an alternative behavior.

Extinction is part of all differential reinforcement training methods. Those methods are on a more humane rung of the hierarchy because the animal is given immediate opportunities for positive reinforcement. This can be done either by reinforcing successive approximations (shaping), or by separate practice of the desired behavior before it is evoked in the situation where the undesired behavior is likely.

So when someone says to you, “Neener neener neener, you use punishment when you withhold a treat,” say, “No, that’s extinction.” Then if they say, “Neener neener neener, you use extinction and that’s mean,” say “I use it in combination with differential positive reinforcement.” And make sure you do!

Be the mechanic who shows his client ahead of time what is going to work, instead of the one who sends him off with no clue.

* The car thing is a deliberately ridiculous scenario. Obviously, to cause a car’s keys and remote not to work would be horribly dangerous, and hardly anyone would consent to that even if it allowed one access to a new feature like voice commands.

Related Posts and Pages

But Isn’t It Punishment if You Withhold the Treat? (Extinction Part 1)

R+ Misconceptions

I never got to the issue of “ignoring” in these extinction posts. So I guess there is going to be a Part 3.

© Eileen Anderson 2014                                                                                                             eileenanddogs.com

But Isn’t it Punishment to Withhold the Treat?

But Isn’t it Punishment to Withhold the Treat?

It would probably be good to decrease this behavior
It would probably be good to decrease this behavior–photo credit Wikimedia Commons

Lots and lots of people think that if you withhold the treat you are punishing the dog. Some will ask the above question in a gleeful, challenging way, feeling certain that they have caught the positive reinforcement based trainers in an inconsistency. But let’s see what is really happening.

Here is a scenario. In the past, you have given your puppy attention and played with him when he jumped on you. But he’s getting big and you really don’t want him jumping on you anymore. You decide to teach him to sit to greet you. He already has a good reinforcement history for sitting, so the likelihood that he will do it in any given situation is fairly high.

So here you are with your excited pup and you are clicking and giving a treat whenever he sits.

  • Pup sits. Click/treat.
  • Pup sits. Click/treat.
  • Pup jumps on you. Nothing.
  • Pup sits. Click/treat.
  • Pup sits. Click/treat.

OK, what happened when the pup chose to jump instead of sitting? You didn’t click. The treats stayed in your hand, your pocket, or the bowl. (You meanie!) You stood still and didn’t react. You are paying for sits, not jumping up.

But lo and behold, the jumping up starts to decrease! Decreasing behavior means punishment, right? You must have punished your puppy for jumping!

No. Let’s look at the definitions of positive and negative punishment.

Punishment

  • Positive punishment: Something is added after a behavior, which results in the behavior happening less often. Example:
    • Antecedent: You approach your dog.
    • Behavior: Dog jumps on you.
    • Consequence: You step on the dog’s back foot, hard. (I’m not recommending this, of course. Just want a clear example of positive punishment.)
    • Prediction: Jumping up on you will decrease.
  • Negative punishment: Something is removed after a behavior, which results in the behavior happening less often. Example:
    • Antecedent: You approach your dog.
    • Behavior: Dog jumps on you.
    • Consequence: You turn around and leave.
    • Prediction: Jumping up on you will decrease.

In the positive punishment example you added painful pressure to your dog’s foot. (Please don’t ever do this.) If the dog finds having his feet stepped on sufficiently painful, jumping will decrease. In the negative punishment example you removed your presence and attention from the dog.  If he likes your presence and attention well enough, and if you are consistent, (and if there is no competing reinforcer–that’s a big if!) this also will cause jumping on you to decrease.

So that’s what positive and negative punishment look like. Now back to our original example. Let’s map it out as well.

  • Antecedent: You approach your dog.
  • Behavior: Dog jumps on you.
  • Consequence: You just stand there.

You don’t respond with physical actions or increase or decrease your attention. Admittedly, this is hard to do, and remember, the lack of response has to be from the dog’s point of view. Even looking down at them is a response. Future blog on this point!

The cookies stayed put
The cookies staying put

Nothing was added: therefore no positive punishment. Nothing was removed: therefore no negative punishment.

(By the way, some people who are very new to learning theory think that the above example is negative reinforcement. Sit, give treat = positive reinforcement. Then jump, withhold treat = negative reinforcement. No, no, no! It has an attractive symmetry, but that is not what the term means at all. Here’s a review.)

So What Is Happening?

OK, back to the first scenario, where you are working on sits with your puppy. Let’s say that after that one time when the puppy jumped and you didn’t treat, the puppy didn’t jump up again. Jumping on you decreased during training. Let’s also say that that decrease continues over time. Why isn’t that punishment again?

Because punishment is not the only process that involves a decrease in behavior. There is another: extinction.

Extinction is the nonreinforcement of a previously reinforced response, the result of which is a decrease in the strength of that response.

In other words, extinction is what happens when the behavior you used to do to achieve some thing doesn’t work anymore. So you stop doing it.

So here comes the big question, especially for those folks who think they’ve somehow caught us out on the withholding the treat business.

How Humane is Extinction?

As with so many things, the answer is, “It depends.” But in this case there is a pretty clear demarcation. In the Humane Hierarchy, extinction by itself is at the same level of negative reinforcement (which involves an aversive) and negative punishment (which involves a penalty for behavior). Not great as first choices. We know that from life. If a machine we use all the time stops working, or a method we use of interacting with another person we care about suddenly gets no response with no explanation, we are left high and dry. It is not fun.

However, extinction also happens in tandem with a process called Differential Reinforcement of Alternative Behaviors (DRA). This is how trainers who aim to train primarily with positive reinforcement use it. (There are other differential reinforcement methods, but this is a good general one to discuss right now.) It consists of reinforcement of an alternative behavior while reinforcement for the target behavior is withheld. Done with some care and skill, it can involve very little frustration for the animal, and it is one step closer to the “most humane” end of the Humane Hierarchy. And this is what is happening in the example above. As long as the trainer is being quite clear that sits are being paid for, the fact that jumping up on her no longer gets attention is not so hard on the pup. He has another thing he can do to get something good. He gets attention and food.

The trainer has communicated to the pup a new behavior to “fill the hole” where jumping used to be.

Japanese Drink Vending Machine
 

I’m borrowing this great example of how DRA works from my friend Kim Pike. Let’s say the soda machine at a workplace is not working. People will push the button repeatedly. Some will perhaps pound on the machine or kick it. This is typical when extinction is in play by itself. The people have no alternative, and get frustrated. (I’ll be covering extinction bursts and and extinction aggression in a later post.) Gradually people will stop going to the machine and give up pushing the buttons. Individuals will probably forget, and now and then go try the machine again, then perhaps give it another kick or shove. But after a while no one goes to the machine anymore.

But when the soda machine is fixed, there will likely be a crowd of people ready to buy their sodas. It’s easier than going to the corner store, and involves less planning than bringing drinks from home. The behaviors attendant to getting a soda are all still fluent and easy for people to perform. And they once again get reinforced.

However! What if, when the machine broke, someone immediately set up a system where folks could buy a soda they liked as well or better for less money? Perhaps there was a cooler, or an honor system with soda in the fridge. If that alternative were in place immediately, would the thirsty people typically have experienced the same level of frustration at the broken machine? Nope! (Except perhaps for the engineers and mechanics, grin.)

And the most important question: What will the folks who just want a soda do when the machine gets fixed? As long as the cheaper, better alternative is still available, they will keep heading for it. The machine will have become irrelevant. Maybe once in a while someone will forget, and go to the machine. But they’d then remember that they can get a better drink, cheaper, out of the fridge.

This is what we are doing when we allow an extinction process in tandem with positive reinforcement of an alternative behavior. We clearly offer the animal an attractive alternative and remind them of it to keep it front and center. It’s important that the reinforcer for the new behavior be the same or better than that of the old behavior. This makes for a process with much less frustration.

Extinction in a Specific Circumstance

In my post, How Do I Tell My Dog She’s Wrong? I address “failing to click” during a training session. I feature a short video example from the great trainer Sue Ailsby teaching her young Portuguese Water Dog, Sync, to stand and stay. In the video you can see Sync’s immediate bounce back after the couple of times she tries something other than a stand and doesn’t earn a click.

In that case, sits and downs are not going to decrease into oblivion in every situation, as we might want the jumping up to do in our other example.  But they will go into extinction during training sessions of “Stand” and later when Sync learns a cue for it. Since  dogs can discriminate this easily, it also tells us that when we want a behavior to go away completely, we need to practice reinforcing our alternative behavior in many locations and situations.

Conclusion

So in answer to the critics, no, withholding the cookie in itself is not punishment. And if used in tandem with reinforcing another behavior, it is quite humane. If we put even a moderate amount of thought and planning into the situation, we can set the dog up to succeed. There will be minimal frustration when he does miss the mark on occasion and fails to earn the treat.

Stay tuned for Part 2 on extinction. I’ll be talking in more detail about what happens when extinction is used by itself, and comparing that with differential reinforcement in some human and dog case studies.

Related Page and Post

Eileenanddogs on YouTube

Theme: Overlay by Kaira Extra Text
Cape Town, South Africa