Leaving the Scene: Clarifying the Science of Negative Reinforcement

Hares escape a lot (if they are lucky)

Hares escape a lot (if they are lucky) — Photo credit, Wikimedia Commons

Negative reinforcement is really, really easy to get mixed up about. Recently I read something that quite bothered me until I did a little research and figured it out. I’d like to share what I learned with you. What I’m talking about is this:

When I take my dog away from the thing he is concerned about, I am adding distance. Therefore this is positive reinforcement.

This is contrary to what you would read in most learning theory books, but it has its own seductive logic. We are primed to associate the word “add” with positive reinforcement (or positive punishment).  I love puzzles and problems, so I decided to do my best to tease out what, exactly, the problem with this statement is.

I  actually found two problems:

  • One is the confusion between positive and negative reinforcement.
  • The other is the focus on the word “distance” rather than the aversive thing itself.

Positive and Negative Reinforcement in Training

First, a review of definitions.

In positive reinforcement, the consequence of a behavior is the appearance of, or an increase in the intensity of, a stimulus. This stimulus, called a positive reinforcer, is ordinarily something the individual seeks out.–Paul Chance, Learning and Behavior, 7th edition

The stimulus can be lots of things. An object (food or toy), an event (door opening). The dog learns that performing a certain behavior (e.g. sitting), makes this thing available. If it is a desirable thing in that context, the dog sits more often.

In negative reinforcement, a behavior is strengthened by the removal, or a decrease in the intensity of, a stimulus. This stimulus, called a negative reinforcer, is ordinarily something the individual tries to escape or avoid.–Paul Chance, Learning and Behavior, 7th edition

Again, it can be an object (snake) or event (fire alarm ringing). The dog learns that performing a certain behavior makes the thing stop or retreat, or lets him get away from it. If the thing is aversive for the dog in that context, the behavior that makes it go away will increase.

It gets a little confusing because  there are some situations where it is difficult or impossible to determine whether negative or positive reinforcement is in the foreground. Sometimes it’s hard to tell you if you are taking away something or adding something, or perhaps both.

I’ve written about this recently :

There are indeed some situations where it is very difficult or impossible to determine the major process. The classic example is that of thermostat adjustment. If you are a little bit cold and go turn your thermostat up 2 degrees and the furnace comes on, are you adding heat or reducing cold? Most people would say the latter (negative reinforcement), but that heat can sure feel pleasurable and luxurious in its own right when it comes on. The situations that don’t fit well into one process or another generally have in common that they deal with a continuum of states and not discrete things that are added or removed.

However, although there are these gray areas, there are many other examples of negative reinforcement that are more clear cut, where it is a real strain to characterize the removal of something icky as the addition of something nice.** I will discuss some examples below.

Examples

Here is one of the “strained” examples and how one well known behaviorist approaches it. It has been argued that turning off an electric shock in response to an animal’s behavior is actually “adding a shock free environment,” (and that adding makes it positive reinforcement).  That argument has been neatly dismantled by Dr. Murray Sidman.

He reminds us that a positive reinforcer must be something the animal is willing to work (perform behavior) to get. And if you take the bad thing (in this case, shock) out of the picture, a “shock free” environment is meaningless and can’t even be defined, much less worked for.

Let’s apply that method as a litmus test to some other examples. Let’s take out the “icky” thing and see what we have left.

These examples involve either negative or positive reinforcement or both.

• First, food. If you remove the drive of hunger (relieving the state of hunger can be negative reinforcement), is food a positive reinforcer? Yes. Anyone who owns a dog or eats dessert knows that. And I’ve written a whole post about it. You don’t have to be hungry to enjoy and be willing to work for food. Eating food can be both negative and positive reinforcement. (But as I pointed out in my previous post, experiments indicate that the positive reinforcement process is more powerful.)

• Now, how about using an umbrella to protect oneself from the rain? There is an unpleasant condition (getting rained on), you perform the behavior of obtaining an umbrella and opening it over your head, and you escape the rain. This is in most textbooks as a classic example of negative reinforcement. What happens if we say that using the umbrella is really positive reinforcement because you are adding the state of “freedom from rain” or even “dryness”? Let’s follow Dr. Sidman’s lead and take away the rain or other unpleasant weather. Would the behavior of opening an umbrella over your head get reinforced by the “addition” of a dry condition?  No. Carrying around and opening an umbrella is a tiresome, expensive behavior. We wouldn’t do it to add something so ill-defined and meaningless.

A black scorpion -- photo credit, Wikimedia Commons

A black scorpion — photo credit, Wikimedia Commons

• So now to tackle the scenario in the subject. Escaping scary things. Let’s say you are phobic of scorpions. If you accidentally get close to one, it is a great relief to leave the area and go somewhere that you believe is scorpion-free. You escape the scorpion. Now, remove scorpions from the picture. Completely. Not just that scorpion, or even scorpions in general, but the threat or mildest hint of scorpions. They don’t exist. What does a scorpion-free environment look like? Well, anything, right?  As long as there are no scorpions. And is it a positive reinforcer? Well, for starters, we can’t even describe it.  Anytime you start thinking of the environment you ran to as reinforcing, it’s because you are comparing it to one with scorpions or some other scary thing.

The Huge Variety of “Scorpion-Free” Environments

A scorpion free environment

A scorpion free environment — photo credit, Wikimedia Commons

Your scorpion free environment could range from freezing to firestorms to 200 mph winds to vacuum to a 70 degree Sunday afternoon,  and all could be scorpion free. That makes the “scorpion-free” environment impossible to nail down to define.

Not only that, but Sidman points out that the environment could be changing.  As long as it doesn’t have a scorpion in it, it qualifies as scorpion free. But reinforcers generally need to sit still. A piece of meat doesn’t usually morph into a paperclip, nor does a book turn into a clock. If they did, they would lose their reinforcing qualities for hungry people or bookworms respectively.

Reinforcers are definable and describable. I don’t believe a “scorpion-free environment” is.  Compare that to the simple description of a scorpion (ick). That’s very concrete. And to the clear action of becoming aware of the scorpion and getting away from it. The scorpion is extremely well defined. And many people (including me) can’t get away from them fast enough.

Can Distance Itself be a Reinforcer or Punisher?

Here is the second problem. Even if you are in the camp that believes that any negative reinforcement situation can be equally argued to be positive, there is still a big problem. Most of us have learned that using the word “add” is an indicator that positive reinforcement (or punishment) is at play. Something is added to the environment after a behavior that in the future leads to the increase (or decrease) of the behavior.

So at first reading, the idea of adding distance or space from something scary sounds at least similar to positive reinforcement. You are adding something (maybe). But let’s go back to Sidman’s exercise. If you take the aversive, scary thing out, what you are “adding” is nothing at all. If you are standing at the 50 yard line in a football field and move to the 20 yard line, what have you added? The distance or space are meaningless except as escapes from the aversive.

So here’s the important part. It is not distance that is being added or removed.  It is the aversive or reinforcer. Distance is merely a description of the escape (or approach) process. The aversive is the scary monster. Only it can be removed or added. Controlling one’s distance from it is just a way of describing the mechanism of the appearance/disappearance of the aversive.

Also, it is a trick of wording. If you wanted, instead of “adding” distance you could say you were “removing” proximity. Focusing on distance is a red herring. And it neatly removes the real aversive from the picture.

I wrote the following in jest, but the more I think about it, the more I realize that what I have written for the other quadrants is exactly parallel to the claims about adding distance being positive reinforcement. It’s just that the other quadrants are easier to understand, so it’s easier to be aware of flawed logic.

Distance

These are the results if we treat distance as the focus. They are obviously untrue. That’s because the aversives and reinforcers are not “distance.” They are the scary monster, the cookie, and the stick. If you talk about “adding distance” what happens to the actual thing we are trying to get away from? It falls out of the equation.

So did this help at all? Does it make sense? Hope so!

Coming up:

Eileenanddogs on YouTube

**Note:

I should mention that there has been a movement for some time to do away with  the distinctions between positive and negative reinforcement and positive and negative punishment. This is mostly because of the few indefinable cases like the thermostat one above, and because the processes of reinforcement and punishment are difficult enough to discuss and explain without the plusses and minuses. But it is not usually argued that there is no difference between adding and subtracting at all. I’m working on a summary of that research, so stay tuned. But in the meantime–you will still see the plusses and minuses in any learning theory book.

 

Share Button

About eileenanddogs

Passionate amateur dog trainer, writer, and learning theory geek.

Eileen Anderson on Google+

This entry was posted in Behavior analysis, Dog training hints, Fear, Negative Reinforcement, Operant conditioning, Positive Reinforcement, Premack Principle, Reinforcement, Terminology and tagged . Bookmark the permalink.

21 Responses to Leaving the Scene: Clarifying the Science of Negative Reinforcement

  1. Sharon Wachsler says:

    This is a terrific post! I was just trying to explain the four quadrants the other day to someone (interested and willing, not argumentative, just for clarification) WRT horses. But the one that always trips me up is positive punishment. R+ and P+ are easy to define, and R- is relatively easy for me to get (and this post clears it up even further). Have you done a post on P-? If not, would you like to? 😉

    • Hi Sharon, great to see you! I’m not sure whether you mean positive or negative punishment, but I think you mean negative? Let me know. I’ve got one that involves positive punishment coming out soon.

  2. Terrific explanation, as always!!! You have no idea how important your contribution is to helping people really get a solid understanding of the science of training and behavior modification. Thanks so much for continuing to hammer away at reducing ignorance. I find these reminders very helpful.

  3. Nicola says:

    Oh my goodness Eileen. When I read your post, the first time, it reminded me of a lecturer who used to say, “If you’re not confused by now, you haven’t been paying attention.” LOL.
    I got it second time. 🙂
    Thank you. This post is excellent.

  4. Naomi Heck says:

    Wonderful post! I loved your explanation and examples. It makes it very clear.

  5. Vicky says:

    For sure I’m wrong, since I must be the only one who thinks this, but I think that the learning quadrant is one of the worst inventions in the history. Something so confusing that almost 100 years after its formulation we still need post like this to explain it, it’s, for me, just wrong formulated. I just try to explain it to my clients in a more simple way: if the outcome is pleasant, the behaviour should increase, if the outcome is unpleasant, the behaviour should be reduced. (I skip Skinner and go back to Thorndike, ha, ha). In this way you can concentrate in understanding how you can make the consequences more or less appealing to your dog to change the behavior. If my dog pulls, I stop: the outcome for the dog is unpleasant in that situation. I don’t want them to worry about adding or removing something. My dog don’t worry about it either (I think).
    Well, it was just a thought, great post anyway.
    (I hope you can understand the general meaning, my English is not very good, sorry.)

    • Hi Vicky! Thanks for commenting. Your English is great and I know what you mean. In English it is a real effort to get past the emotional and value judgmental definitions of positive and negative, so I often do wish Skinner had picked something else!

    • Tegan says:

      I agree with you, Vicky. Though this is a great, well argued post that makes a claim… Does it really matter? As long as we’re getting a change in behaviour (and the change we want), it’s not really important what mechanism it’s by (as long as it’s humane).

  6. Pingback: Only if the Behavior Decreases! | eileenanddogs

  7. Pingback: Only if the Behavior Increases! | eileenanddogs

  8. Pingback: My Dogs Are Not in Charge | eileenanddogs

  9. Pingback: 8 Common Dog Training Errors: Cautionary Tales | eileenanddogs

  10. Ingrid Bock says:

    Aw, this is wonderful. And it’s funny, too. I’m still laughing. Sharing it to any friend in a scorpion-free environment.

  11. Pingback: Let’s Talk about Using Aversives in Training. I’ll Go First! | eileenanddogs

  12. Pingback: Wie Tiere lernen - Verstärkung | Chakanyuka

  13. Pingback: 7 Effects of Punishment - eileenanddogseileenanddogs

  14. Pingback: But Isn't it Punishment to Withhold the Treat? - eileenanddogseileenanddogs

  15. Pingback: Natural vs. Contrived Negative Reinforcement - eileenanddogseileenanddogs

Comments are closed.