I cannot add anything to what has already been said so eloquently of the tragic, untimely passing of veterinary behaviorist Dr. Sophia Yin. In her honor I am posting two of her helpful handling videos and linking to Steve Dale’s tribute to her.
These papers are often quoted by people who seem motivated to rehabilitate negative reinforcement, although the papers are generally more about nomenclature, and not whether negative reinforcement is humane.
Before we go on, here is a working definition of negative reinforcement and some examples:
Something is removed after a behavior, which results in the behavior happening more often.
Some examples are:
The buzzer of your alarm clock goes on until you get up and turn it off.
You get rained on until you open an umbrella.
A trainer pinches a dog’s ear until she opens her mouth to accept the retrieve object.
Negative reinforcement can be involved in something as trivial as scratching an itch to something as serious as running for one’s life from a predator. There is a vast range of severity. It’s not all about pain.
When we consider dog training, we need to make a distinction regarding handler mediated negative reinforcement and automatic reinforcement. Stepping in and putting a behavioral requirement on the removal of an aversive differs from the myriad ways that dogs take action in their own lives to remove an aversive, be it mild or extreme.
Finally, there are some borderline cases where it is hard to determine whether the process involved is positive or negative reinforcement.
That is what I’m writing about today.
Borderline Cases
The classic borderline case is the thermostat. When it’s too cold and you go adjust the thermostat by two degrees, are your actions reinforced by the subsequent pleasant feeling of warmth, or the relief from the uncomfortable cold? People use the borderline cases to support arguments made in favor of doing away with the distinction between R+ and R-.
Those who like to argue that negative reinforcement is “not so bad” also like to bring up this example, even though it is not typical of reinforcement scenarios.
I ran across one of these ambiguous situations recently in my own life and am going to share and analyze it here. Let’s see whether the fact that it could go either way makes the negative reinforcement any more benign.
Tile photo credit—Wikimedia Commons
Personal Example: My Shower
I am an indifferent housekeeper. I am prone to clutter, and tend to barely keep up with the dog hair on the floor and the dirt the dogs track in.
I have a bit of a problem with mold in my house, and my shower had recently gotten pretty bad, such that even with a thorough cleaning I couldn’t get it to look nice. I have tried several times in the past to change my behavior about that, but failed.
So when it got moldy again about four months ago, I made a thoughtful plan and tried again. First, I threw out and replaced my shower curtain liner and in-tub mat. I scrubbed the shower and tile and sprayed it with bleach. I did this repeatedly over the course of a few days until it was beautifully clean.
Then I thought about antecedents and reinforcers regarding the shower cleaning behavior and made a plan to maintain the shower and keep it clean.
I purchased two kinds of shower spray: one with bleach and one without. Both claim to keep the shower clean just by spraying on. (Bear with me. I’m not much interested in the details of housecleaning either, but they are relevant here.) My goal was to arrange antecedents to make the desired behavior as easy to maintain as possible.
I then adopted a loose schedule of using the cleaner with bleach a couple of days a week and the less noxious (but also probably less effective) one a few times a week. I wasn’t sure exactly how much would be necessary to keep the shower clean, but was ready—gasp—to do something every day if I had to.
So far I have kept up—it’s been a few months now—and the shower/tub is sparkling clean.
Question: What is Maintaining the Behavior?
Shower photo credit: Wikimedia Commons
Is it negative reinforcement or positive reinforcement?
Let’s map out our contingencies. We are talking about a reinforcement scenario (not punishment) because we are increasing/maintaining a behavior: spraying stuff on the tub and tile.
Positive reinforcement version
Antecedent: Schedule says it’s time to spray down the shower with cleaner
Behavior: I spray cleaning agent on the shower tile
Consequence: Shower looks and smells pleasant and clean
Negative reinforcement version (escape)
Antecedent: There is mold in the shower
Behavior: I spray cleaning agent on the shower tile
Consequence: Mold is gone
Negative reinforcement version (avoidance)
Antecedent: There is the threat of mold in the shower
Behavior: I spray cleaning agent on the shower tile
Consequence: Threat of mold is relieved
So which scenario is it and does it matter?
Can We Tell By Observation?
First, let’s think about whether there is any way that a person observing my behavior could tell. Is there a special way to apply shower spray that shows one’s motivator is to prevent mold? Or is there an indicator that one loves the look of a sparkly clean shower?
Behaviors maintained by negative reinforcement tend to be minimal. The person or animal does the very least he or she can do to get the result. I believe this has shown to be true in the workplace, and can also often be observed in dogs that are trained using aversives only.
As Aubrey Daniels says:
Positive reinforcement maximizes performance, while negative reinforcement gets a level of performance that is just enough to get by, just enough to escape or avoid some unpleasant consequence.—Bringing out the Best in People, Aubrey Daniels
In the case of the shower, could we tell by watching? If we observed my behavior over time, we could note whether I sprayed the whole shower or just the parts that tend to get moldy. We could also note whether I made efforts to determine the minimum amount of work it takes to keep a shower clean (or mold-free) using the methods I chose.
Also, we could try to tell whether I took any enjoyment out of the clean shower. Do I go out of my way to admire it? Do I polish parts of it to make it extra sparkly? Do I shower even when I don’t need to because I find it so pleasant?
But since I’m a human being with many motivations, I think it would be a little difficult for an onlooker to tell what is driving my shower cleaning behavior. I may use minimal efforts because I want to save on cleaning supplies or I like to make a game out of efficiency. When I look at the shower, I may be looking for flaws, not admiring my handiwork.
But I know which it is!
So Which Is It?
What is driving my behavior is the threat of mold. I hate it. I remind myself to notice how nice the shower looks, but that is an incredibly weak reinforcer for me.
Even though I have worked out a system with minimal effort and virtually no elbow grease, I HATE having to spray stuff to maintain the clean shower. There is no pleasure in it for me, before or after. I am continually trying to figure out whether I can skip a day, or two, or maybe leave off the bleach version for a while. The situation is doubly frustrating because I feel like I can’t mess up. Because if the mold comes back even a little, it will be that much harder to eradicate. So I don’t even know where the boundary for “minimal” is, but I am sure trying to find it.
This is almost a purely negative reinforcement scenario for me.
Application to Dog Training
I have previously written about two situations in which it could be hard to tell the difference between positive and negative reinforcement in dog training. One is when training with food if the dog has been deprived. The behaviors that allow a starving dog to eat are negatively reinforced as her hunger is assuaged. Likewise, a game of hiding from your dog could involve either positive or negative reinforcement.
However, I think the most common situation where positive and negative reinforcement can be confused is when dogs are said to work for praise. Yes, you read that right. Compared to food and play, praise is a very weak positive reinforcer for most dogs, and often nonexistent unless it has been deliberately paired with a primary reinforcer and/or the bond with the human is very strong. More often, praise is a safety signal, a sign from the human that, “You have done the right thing and I won’t hassle or pressure you anymore.”
So we may think our dog is working “for the joy of a clean shower” when she really is working to escape the mold. And, unlike humans, dogs tend to be a little more obvious about how happy they are with an interaction or a method, if we can just learn to pay attention.
Take-Home Lessons
Even if it is a negative reinforcement scenario, cleaning the shower is one of those fairly benign sounding applications. Perhaps I sound like a pretty spoiled person to be complaining about it. I know that I am privileged for that to even be on my radar as a problem, for sure. But you know, when searching for photos to use with this post, I got grossed out. And even though I found a couple of moldy tile pictures on Flickr that would be permissible to use, I ultimately decided against it because they were disgusting. I didn’t want icky pictures of mold on my blog.
I have been describing an “automatic” negative reinforcement process. My own actions directly remove the aversive, the threat of mold. How would I feel if someone used the threat of mold to get other behavior from me? Easy answer. I wouldn’t like them very much. Especially since I am so easy to please with food or money, grin. Really, why on earth would someone want to use a threat instead?
These kinds of analyses of everyday activities are helpful to me. I hope they are helpful to others, and I hope I didn’t over-share. I have contemplated trashing this post several times, but then I thought perhaps it would help someone understand negative reinforcement just a little better. When one is first learning about the processes of learning, negative reinforcement methods can sneak in, seeming like magic. Look, I didn’t have to hurt my dog or give it food either! That’s one of the main reasons I write about it so much. It can be quite insidious.
Got any personal negative reinforcement stories?
Related Posts
Why Scratching an Itch Is Not the Same as Performing a Force Fetch
So how many of you with puppies out there wish sometimes that you could flip a switch to turn them off, just for a little bit? Catch your breath, do the dishes, sit down for just a minute?
I have it on pretty good authority that most of the puppies would also appreciate having an off switch, too! Just as human babies can get all wound up without knowing how to come down on their own, puppies get overwound too.
Some of the advice that gets passed around is off the mark. Owners of high-energy dogs are told to exercise them more and more to burn off the energy. Every time the dog leaves the house it’s for a rousing run or play time. While stimulation and exercise are vitally important, taken by themselves, they can actually exacerbate the problem of being wound up. The dog rehearses a pattern of arousal.
That’s why learning to relax and settle is an important life skill. Marge is really good at teaching it, in my opinion. She teaches “relax” as a behavior, just like teaching sit, down and come. And it’s a win/win for puppy and caregiver.
Lots of other trainers have methods for teaching this behavior, too. Sue Ailsby teaches it in her Training Levels program. Nan Arthur has a method in Chill Out Fido, Laura VanArendonk Baugh has a whole book about it, and Emily Larlham has some videos. I have some resources here in the blog as well. You can search the blog under “1,000 Treats” to see Clara’s progress in relaxation.
The goal of all of these methods is far beyond just getting the dog to stay still. It is to teach the dog to chill out and relax.
From Practice to the Real World
Being able to recover and think through increasing levels of arousal can be taught. Most people play with their dogs and puppies without breaks. But breaks allow the puppy to reset, and to learn how to transition between different states of excitement and arousal. They also can keep the pup from going over the top.
In the movie, you will first see Zip relaxing in a non-challenging situation. Then Marge transitions him back and forth between relaxing and getting up to play. Marge works with lots of puppy owners, and has them start with play increments of 5 seconds (one banana, 2 banana, up to 5.). Reset/relax, then start again. Gradually increase duration and difficulty.
At 1:06, watch Zip’s right front leg. He is not just lying down; he is relaxing his muscles. Later you can see him also change his breathing when asked to relax. I’ve watched the movie several times, and keep seeing other aspects of the relaxation. In the last tug session, between the 2:00 and 3:00 minute marks, Zip is growling–a symptom of high arousal for him. You can see how hard he has to work to control himself when Marge asks him to release the tug and relax. “Ohhhh I wanna bite that shoe……but I won’t.” This is yet another version of impulse control.
Take note as well, how Marge reinforces Zip for the relaxed behavior. She is using food rewards, delivered with soft body language right to his mouth. Nothing active, no tossing treats. This is in contrast to the active play with the toy during the “up” states.
The final part of the movie shows a real world application. You can’t see it in the movie, but while Zip is chilling on the floor at the animal hospital, there are two very active toddlers and another dog nearby. This is where you can see yet another benefit of playing tug with a puppy (with a rule structure such as Marge uses). Environmental stressors can also bring about an aroused state. A dog doesn’t have to be jumping around to get over-excited. But playing tug has helped Zip learn how to “come down” from that state, and his lessons carry over beautifully to the new environment.
Just like last time, this is another lesson on how to teach a puppy not to do something using positive reinforcement-based training. Notice all the things Zip is not doing?
Biting
Running around screaming
Stealing the toy and running away
Leaping up to investigate the other dog or the kids at the vet
All because Marge has “filled in the blanks” with desirable behaviors, and is teaching Zip at a very young age how to calm down.
How about you all? Does your puppy have an off switch? Also, any guesses about Lesson Five? Because we have left out something BIG!
Elicited Aggression: If you hurt or scare your dog, she may hurt your other dog or your kid.
Generalization (related to #1 and #2 above): If you scare or hurt your dog, she can become afraid of (or aggressive toward) other things associated with your actions, like locations and objects.
Apathy: If you hurt or scare your dog a lot, she may become apathetic and not do much of anything.
Conditioned Suppression/Learned Helplessness: If you hurt or scare your dog a lot unpredictably, she will live in a state of fear and also may not do much of anything.
Injury: If you hurt your dog you could cause him injury.
Reinforcement of the Punisher: If you hurt or scare your dog regularly, your actions will easily be reinforced and become habitual. On the occasion that your actions don’t work to interrupt or decrease behavior, you will tend to escalate the hurt.
Copying: If you see someone training their dog through pain or intimidation, it can influence you to do it yourself.
These are the things you risk if you use pain, fear, force, coercion, intimidation, or even startling to train your dog. The effects are not limited to training “tools” such as are featured in the picture below.
Not all of them will happen all the time. But they are all possible, and we can’t know ahead of time which dogs (and which owners) will be strongly affected by the use of aversive methods.
That’s the short version. For scientific references, check the resource page described and linked below.
This resource page cites articles, most of them classics from peer-reviewed journals, on the above types of fallout. It is provided for people who need or want to investigate the original sources.
Most types of aversive fallout are so well documented that the reader can check out the original article and follow a cascade of research following it.
Besides classic sources for the above effects, I’ve listed the main studies that document side effects of painful or scary training for dogs, and also a couple of other important references. Like many of my projects, the page is ongoing.
If it is helpful to you, please share it. If I have left out something important, please let me know!
I think it’s so interesting when someone says that.
Every so often I hear or read a confident claim from someone saying that desensitization/counterconditioning (DS/CC) wouldn’t work on them. They seem offended at the very idea that food would help them overcome their fear. The nerve of anyone to even mention it!
Actually, I can relate just a little. Maybe the idea is a tiny bit threatening. So many of us hold our individuality near and dear. Americans especially, I suspect, are taught (conditioned, hah!) to view conditioning as some kind of mechanistic insult to our personhood. It evokes scary thoughts of self-serving mind control as in Brave New World.
But could any of us really be immune from associative learning and respondent conditioning? Do the processes of learning apply to us or not? Are we really such exceptional super-humans that we know in advance that a method that operates in a fairly predictable psychological and physiological manner just….won’t?
What’s wrong with this picture?
Shoving Cookies
To start with, the phrase about “shoving cookies” signals either a basic misunderstanding or a deliberate misrepresentation of the process of counterconditioning/desensitization. The claim usually goes something like this:
You could shove all the cookies at me you want but it wouldn’t make me feel any better if I were trapped in a room full of [spiders, clowns, snakes, etc.]
I could end the post right here and just say that’s a straw man argument, because it is. But I’m interested in what’s behind it, so I’ll keep going.
Desensitization consists of exposing a subject to the thing they fear in graded exposures, starting with a form that is dilute and nonthreatening, and working up to full exposure to the scary thing. Counterconditioning consists of changing an emotional response (usually from fear to neutrality or to a positive response), by pairing the trigger of the undesired response with something that evokes desired emotional response. Combining these two methods creates a non-threatening but very effective way to alter phobic fear responses.
So of course if the process were started with a full-blown exposure to the scary thing, those cookies probably wouldn’t do much. Cookies wouldn’t help a person who is already scared out of their wits. However, that scenario is not DS/CC. The desensitization component ensures that we never start at such an over-the-top exposure.
What’s This About Starting with a Low-Grade Exposure?
Performing desensitization/counterconditioning when addressing fears ensures that we start with a low intensity, manageable form of the scary stimulus. This is done for at least two reasons. One is ethics, and the other is efficacy. I have discussed the ethics issue in my publications on thresholds. Since we can’t explain to animals what we are doing or get their “buy-in,” there are issues of consent that are not present with humans. It is more ethical to start with a non-traumatic exposure to the scary thing.
But the second reason, efficacy, applies to humans as well as other animals, and this is why the quote above about creepy things vs. cookies doesn’t cut it. We start with a tiny, controlled exposure to the scary thing precisely so that the emotional reaction to it doesn’t overpower the positive response to the pleasant stimulus we will use for our counterconditioning.
The anxiety-producing stimulus must be presented at a low enough level that the parasympathetic nervous system response to food (or specifically for humans, the practice of trained relaxation or other internal technique) is stronger than that of the sympathetic nervous system’s fear response. (See Behavior Therapy Techniques: A Guide to the Treatment of Neuroses, by Joseph Wolpe.)
Pavlov’s Work as an Example
Let’s compare classical conditioning, which doesn’t usually necessitate desensitization, and counterconditioning, which usually does. We can use Pavlov’s work with salivating dogs as a starting point.
Pavlov’s dogs learned via respondent conditioning to salivate when a sound, probably a bell, immediately preceded the delivery of food. They most likely did not have any negative association with the bell to begin with; it was probably a neutral stimulus. Because of the bell being paired as a temporal precursor to food, the dogs’ physiological response to food (including salivation and other internal preparation of the GI system) transferred to the sound of the bell. This straightforward classical pairing changed the dogs’ simple auditory response to the bell to an appetitive one, apparently without a hitch.
But what if there had been a dog who was deathly afraid of the sound of a bell beforehand? When the bell rang, the dog’s sympathetic nervous system would have kicked in, with a cascade of biochemicals and physiological responses. Just like for the person in the room full of spiders, snakes, or clowns, that situation would have provided a strong competing response that would inhibit the ability to relax or respond to even the yummiest food.
Depending on the order of events, the dog could even get reverse conditioned, and the fear triggered by the bell could get associated with the food. Likewise, the person in the room full of spiders being showered with cookies may not want a cookie again for a long time.
So for the bell-fearing dog and the spider-fearing person, we would use desensitization in addition to the counterconditioning. We would start with a very dilute, weak exposure to the scary thing. Then the seesaw would drop on the side of response to the conditioning stimulus being the more powerful process at work.
This is also why we use something like steak and not kibble when doing counterconditioning on dogs. We take every opportunity to get the strongest positive response possible.
So again, our people with their off-the-cuff denials are not describing DS/CC at all.
Let’s explore how DS/CC is typically done with a human. I’ll volunteer for the thought experiment.
My Phobia: Crawdads
A crawdad ready to get me. Also called crayfish (the formal name) and crawfish. (Public domain image from Wikimedia Commons.
I spent a lot of time playing in a murky creek as a kid. Crawdads creeped me out. I was afraid of being pinched by one, underwater where I couldn’t see it. I felt like they were ugly and sneaky. And I was also really, really grossed out by the dead ones I would see sometimes. They would be in the still, shallow water, and have algae and murk on them. I would think they were alive, just lurking, until I moved the water around them and they would slowly disintegrate. Ewww!
However, crawdads are just not all that dangerous. They generally are not going hurt you very badly, especially if you leave them alone. Certainly, they are not as potentially harmful as bumblebees (which I don’t fear abnormally) or the non-poisonous snakes I had as pets. I believe I was pinched by a crawdad one time. It was completely underwhelming. I felt this little pinch on my big toe, and then it was gone.
The comparative harmlessness of crawdads makes my little phobia a perfect candidate for DS/CC. There is no ethical problem with making me comfortable around crawdads, as there would be, for example, with escaped convicts or bears.
Let’s Make a Plan
So how would we really go about addressing my irrational fear of crawdads with desensitization/counterconditioning?
First of all, we have many more options with humans than with animals (on this page is just a sampling of the many things that have worked with humans). We don’t have to use “cookies,” although we certainly could. With humans, we can often evoke a competing emotional state by using imagination/cognition or a physical activity. There are lots of things one can use.
Now, the idea of cookies (especially chocolate ones) is enticing, but what if I got full too fast? Satiation can indeed be a problem. And all that imagining or virtual reality described in the resource list sounds like more work than I feel like doing. So, my first choice for a tool to use for my own counterconditioning is that potent secondary reinforcer for humans: money! How about if some rich person funded my DS/CC by arranging for me to be paid $50 every time I perceived a crawdad in any form, proceeding up a desensitization hierarchy? (I’m actually already thinking about how I would go around looking for crawdads.)
To do it properly I would need to establish a hierarchy of exposures, so here is what I came up with. In real life, I would do this in consultation with a knowledgeable psychotherapist, who would also monitor the sessions to gauge the exposure levels and my response.
Steps of Graded Exposure to Crawdads
Each iteration of each step will pay $50, with the option of adjustments by the supervising psychologist.
Read a story in which crawdads are peripherally mentioned.
Write the word “crawdad” myself on a piece of paper. Note that this might be a good first step for the people who write posts about how DS/CC couldn’t work for their fear, since they have already shown that they can write about it. In contrast, there are people with phobias so severe (emetophobes come to mind) that they can’t even stand to see the word that is associated with their phobia.
Watch more videos, gradually increasing numbers of crawdads, movement, size, and murk.
View, in person, a single crawdad in a tank, such as in the setup in Step 9.
View more crawdads in tanks, raising criteria with movement, numbers, and intensity as with the photos and videos.
Put my hand in a tank with a baby crawdad for it to investigate.
Do the same in a tank with several babies.
[I’m leaving out steps of deliberating touching juvenile or adult crawdads because going that far is not necessary for my needs and could hurt the crawdads.]
Listen to the song, “Crawdad Hole.” I originally had this as an early step, since it’s a very pleasant tune, but then I actually listened to the lyrics, which include a whole bag of crawdads breaking and the crawdads were “back to back.” Oops! Imagining multiple moving crawdads was too intense for an early exposure, but is probably okay about now.
Go explore a creek (something I love to do, by the way) where there are known to be crawdads, but stay on the bank.
Go back to the creek and actively look for crawdads (probably won’t find any). Just for fun, the fee has just gone up and I will get $200 if I see a real crawdad in the wild.
Sit by the creek with my feet dangling in the water. Still paying $2o0 for crawdad sightings.
Wade in the creek (with footwear if appropriate). Still paying $2o0 for crawdad sightings.
I would get paid for each iteration of any one of these tasks. Under the care of the psychotherapist, I could repeat tasks, or add interim steps as needed. (In case the potency of the $50 tempted me to linger unnecessarily on a particular step, the psychotherapist would observe me for possible “fake” responses, and would be free to make alterations in the protocol accordingly.)
For the sake of thoroughness, I added more steps to the list than would probably be necessary for me. You might want to note how many steps there were before the real-life crawdad made an appearance. That is the beauty of desensitization, and why it’s really a shame that people misrepresent it.
If anybody wants to fund my project, please let me know!
Would it Work?
There’s the question. If I went through standard DS/CC of my crawdad phobia, would it work? Probably! Claiming to be exempt from this well-documented process, whether happening naturally in life or in a protocol, is what’s pretty hard to defend.
Our bodies are wired to make associations. If we didn’t build these associations, we wouldn’t have many of the pleasures in life that we do. We do have to be a bit more creative and work a bit longer if we are “rehabbing” a stimulus with negative associations rather than a neutral one. Yeah, it seems kind of silly for the sight of a crawdad to predict $50, but if it did, I bet my feelings about them would change. And with a bit of finesse, the added pleasure from mucking about in creeks without fear–access to pleasurable activities–would kick in as I weaned off the money.
To make one comment about dogs: I have watched that exact process with my feral dog Clara, and it is thrilling. For a long time, strangers predicted spray cheese. Now they predict comfortably hanging out with her dog and human friends, being able to walk up and give interesting people a good sniff, being able to go in completely new environments and explore without worry, and of course being able to sniff good pee-mail. She actually chooses these activities rather than hanging around for the food. She is free to choose environmental reinforcers. She was formerly prevented from that by her fears.
So I bet I could go from cartoons to plush toys to photos of itty bitty cute (really?) ones all the way up to the gnarly crusty crawdads hiding in the mud. This is not some kind of faith on my part. It’s in keeping with what I have learned that science says about the process, creating a protocol that is in keeping with what we know, and personal observations that are in concert with that knowledge.
Can One Resist?
You do kind of have to wonder whether the people who are adamant that it wouldn’t work would actually cooperate. Could one actually resist counterconditioning?
Stay tuned for a future blog on trying to resist respondent conditioning. There is an interesting story in the literature. But in the meantime, let me leave with some words by Dr. William Mikulas, a behavioral psychologist and also the author of this cool book:
Counterconditioning of fear/anxiety takes place outside of cognitive control, so does not require cognitive acceptance or beliefs. But, of course, cognitive co-operation makes it much easier!!
Dr. William Mikulas, excerpt from personal email, August 2014
*There are also issues of order and timing that would affect which response would “win.” We’re leaving them aside for now.
.jpg){kind=link}