“Errorless” Learning

Addendum, 2/9/13. Please be aware that there are some historical inaccuracies in this post, mostly related to the origin of the method and term Errorless Learning. The mistakes affect some of my conclusions as well. Please read Errorless Learning II if you read this post, or instead of reading this post. –Eileen

You will never hear me say, or see me write, “It’s only semantics.” I grew up in a family full of passionate readers, English and education majors, and teachers. Not to mention musicians, who are often quite obsessed with passionate about language as well.

Semantics is very important to me because it deals with whether or not we understand each other. Here is part of the definition from Wikipedia: “It [semantics] is often used in ordinary language for denoting a problem of understanding that comes down to word selection or connotation.” In other words (ha ha), if I use a word or phrase and it means one thing to me and to you it means something very different, we instantly have a communication problem, and we may not even know it.

That is the context in which I offer this post. Don’t worry, I’m going to make it to dog training, and there are actually some dog training hints in here.

Zani beginning the shell game

Zani beginning the shell game

There is a newish catch phrase going around the science-based dog training community: “errorless learning.” I am seeing more and more usage of the term, and reading pieces that equate it with the ultimate humane training. I think a lot of folks have picked up the phrase and are using it to mean setting your dog up for success in a general way. I’m aware of some others who associate it with training with positive reinforcement only. Some use it to indicate that they do not use No Reward Markers. (If an error happens in the forest and no one says anything, did it really happen? Sorry.)

But actually the phrase is not new at all. It refers to a specific teaching methodology that has been well investigated by research.  I am going to describe the original research on so-called errorless learning, some subsequent research, and explain why I think the term is currently being misused and perhaps wrongly proposed as a goal in our companion animal training.

Initial Research about Error Free Learning

Skinner box

Herbert Terrace published, “Discrimination learning with and without ‘errors’,” in 1963. The experiments were performed on pigeons in Skinner boxes. The discrimination behavior taught to the pigeons was to peck on an illuminated key for a food reward when the key was lit internally with a red bulb and not peck when the key was illuminated with a green bulb.

At the beginning of the experiment, the key (in the darkened enclosure) was lit bright red. Apparently it is easy to get birds to peck on a colored, illuminated key, and importantly, they generally will not peck a dark key. The pigeons got reinforced for pecking on the brightly lit red key. The key went dark between trials and no reinforcement was available.

The birds were divided into four groups.  After the birds had a period during which the key glowed red and during which they got food rewards for pecking it, the color and brightness of the key were changed according to four different protocols. For the pigeons who learned the discrimination the fastest, called the “early progressive” group,  the procedure was as follows: early in the experiment, after a “dark” period, the key was illuminated at an extremely low level with the green bulb, and no reinforcement was available when the birds pecked at it. (But mostly they didn’t.) This was alternated with periods where the key was illuminated bright red, and the pigeons were reinforced for pecking it. The duration and intensity of the green light were very gradually  increased from a dark key of 5 seconds duration to a bright green key of 3 minutes duration. In short: the key morphed from completely dark to bright green so gradually that to the pigeons it remained “unattractive” to peck.

For the three other groups of pigeons, there were variations in how early the green key was introduced, and whether it was introduced gradually or at full intensity at the very beginning.

The pigeons in the “early progressive” group had an amazingly low error rate. They pecked at the green key well under 1% of the time.

This technique is the ancestor of what we often do nowadays in teaching a discrimination. If I want my dog to touch her paw to a cup with food under it and ignore another cup, I will first have only the desired cup present. I’ll reinforce some iterations of touching that cup. Then when I first introduce the second cup (which is empty) I may put it in an inconvenient place for the dog to touch it and only gradually bring it physically closer. In other words I will make it easy for the dog to be right, sneak in the “wrong” cup so that at first it is just part of the background, and raise the challenge very gradually.

But before we adopt Terrace’s term of errorless learning to apply to such techniques, let’s look at some differences between what he did and what we do–and are even willing to do–with our pets. Here are some of the primary differences between his training situation and ours with our pet or performance dogs:

  • The pigeons were food deprived. They were kept at 80% of their normal body weight for a period starting two weeks before the start of the experiment to the end of the experiment. This is very common in such experiments.
  • The pigeons were isolated in a Skinner box during the experiments.
  • White noise was played to block external sound.
  • The light intensity and duration on the keys were controlled with great precision by an electrical unit.
  • The birds had not been taught anything before.
  • Although it is not stated in the paper, it is fair to assume that the birds had no particular relationship with humans other than being handled; they were not pets.
  • The birds were being taught only one behavior (this is a crucial point).
  • There was no proofing. The birds were not challenged to perform the behavior under any other conditions. It’s fair to assume the behavior wasn’t generalized.

Again, the error rate of the highly controlled birds (only in the early progressive group) was less than 1%.

From the above points, it appears that impressively low error rate was possible at least in part because of the technology available to the experimentors and the extreme control over experimental conditions that was possible for them. We don’t generally have rheostats to gradually change the intensity of lights, or little trucks to drag in the “wrong” object in increments of exactly half an inch.

You Can’t Do This At Home, But…

The conditions under which the pigeons were trained cannot be emulated by the average trainer, for technological reasons, reasons related to environmental control, and also humane reasons.

But I wanted to show how that method is relevant to some teaching strategies we use with our dogs. I made a video of the descendant of Terrace’s method as applied to scent discrimination: the Shell Game. In the video I am teaching Clara to tap only a jar lid that has a treat under it. My goal was to demonstrate the method of sneaking the “incorrect” lid in from the side while the dog is happily bopping the food filled one.

As usual, I got more than I bargained for. For starters, I utterly failed at sneakiness. You’ll see. We survived that. But in addition, I got some interesting footage of Clara making an “error” pretty early on (tapping the wrong lid), which was probably because of my clumsiness and bringing it in too fast. However, although different dogs react differently, making this error appeared to help teach her more about the game. See what you think.

Mine is not a tutorial video. If you want to teach your dog the Shell game, here’s a really nice tutorial by Donna Hill. She uses a different method of helping her dog succeed, and does a beautiful job as usual. By the way, in my video Clara was standing in for Zani and it was her first time ever playing the game. All the paw flailing she did (including belting and grabbing me a few times) was because we’ve been spending quite a bit of time shaping a trick with a lot of paw movement. It took her a while to get that out of her system and figure out the new game. That’s another difference between training in the lab and at home. Whatever else has been reinforced recently or richly will likely creep into the new thing you are working on.

What you see in my video, however clumsily done, and what many people seem to mean by “errorless learning, ” is helping the animal to be right. Terrace’s work went far beyond making it easy for the pigeons to be right, however. Because of his use of technology and the controls available in a laboratory, he made it very, very difficult for the birds to be wrong. Is this a good thing?

Would You Even Want To? The Big Drawback of that Huge Success Rate

OK, so what if we could achieve that kind of low error rate while still being kind to our animals, and let’s further assume that we were able to teach it to fluency in a real life environment. Are there any other problems?

Yes. Back to the pigeons: what if later we needed them to peck the key when it was green instead? Biiiiig problem.

Discrimination Reversal Following Learning without “Errors” by Marsh and Johnson in 1968 demonstrated that pigeons taught to peck a red key and ignore a green one, using Terrace’s method, could not, even after five days, be induced to learn a new behavior of pecking the green key.

For most things we want to teach our dogs, that would be a huge problem.

Granted that there are some behaviors and tasks in the dog world that are standalone, in the sense that you would not be likely to teach a conflicting behavior. Diabetic alert dogs come to mind. As I understand it, they learn to react to one and only one odor for their working lives. (Correct me if I’m wrong, folks.) Cadaver search dogs. Perhaps some other types of search dogs, but not all.

But in the service dog, pet and performance dog worlds, it seems to me that these kinds of needs are rare. Most people teach their dogs both sit and down. Agility and herding dogs aren’t taught left turns only. They learn left and right. Service dogs typically learn to both push and pull, use left and right. They have to be ready to pick some stuff up and not even touch other stuff.

Anyone who has trained a dog, for example, to raise her right paw, got that fluent, then taught her to raise her left paw, is familiar with the period of frustration the dog goes through when the familiar behavior no longer pays off. I have a post related to that about the mini extinction bursts that our dogs undergo in shaping exercises. The research tells us that if we had trained the right paw raise errorlessly (a difficult challenge), the dog’s frustration when trying to learn the left paw raise would greatly increase.

The pigeons learned only to do one thing, and the exercise did not teach them problem solving skills or how to play other training games with humans. And it blew their little minds when they were asked to do something else.

This is the biggest reason I do not have errorless learning as a goal for my dogs, nor do I use the term for the teaching strategies I use and admire. Taking a long view, training them to do one thing using something close to Terrace’s method could set them up for tons of stress and frustration later.

Learning What’s Wrong to Learn What’s Right

It’s a little bit out of style to emphasize the importance of your dog knowing what the wrong behavior is. It smacks of corrections and punishment based training. But as clicker trainers say, the lack of a click is information. In my video, because I lumped a bit and moved the second lid into the picture so fast, Clara made an error fairly early on. Her error consisted of tapping the empty lid. She tapped it a couple of times, got no treat, sniffed and licked it, then proceeded to the correct lid and tapped it. She then ignored the “wrong” lid from then on in that session. I think she learned something really important. There are lids with nothing under them! She is going to have to use her nose to figure it out. It seems to me that learning that at this point was not at all harmful for this dog.

Let’s Add to the Terminology Confusion: Applications to Human Learning

Errorless learning is used very successfully in operant conditioning programs for autistic children. But the process is quite different, since we primates mimic so easily and often we can understand and follow verbal instructions. From this website comes a good definition:

[Errorless learning]: The use of instruction designed to prevent errors or incorrect responses. Typically prompts (artificial cues that provide assistance to the learner about the correct response) are presented so that an individual engages in a behavior that is being targeted. Once the individual is engaging in the behavior appropriately, then these prompts are faded or removed slowly and systematically so that the correct behavior is made with few or no errors.

Here is a lovely little video that shows that technique.

https://www.youtube.com/watch?v=8sYNcSP5VZ0

But think about whether we could apply that method to dog training. The child is learning to perform the initial task through either the verbal instruction, mimicking the hand movement of the teacher, or both. Neither of those are available to us with dogs. If the behavior is new, they don’t already know the verbal cue. And although there seems to be some small evidence of dogs learning by mimicry (of other dogs), you can’t take your average dog, put your hand on an object, then expect them to put their paw on it just like you did.

Aggression

Back to the birds. Terrace later claimed as a by-product of some later experiments that pigeons trained using a trial and error method rather than his “errorless” approach showed aggressive behavior when pecking the wrong key produced no reinforcement. The article is “Behavioral contrast and the peak shift: effect of extended discrimination training” and is available in full online. In that work and in a later study he claimed that these behaviors were not present with his “errorless” cohort.

I have heard this used as an argument for “errorless” learning for dogs. Our dogs might get enraged and aggressive if they make too many mistakes, so we need to absolutely minimize by any means possible the number of mistakes they make. But again, there are big differences in the training environment between the lab and training our dogs at home. Our training is relationship based. And a big part of the job of the human trainer is to monitor the emotional state of the dog as evidenced by its behavior and adjust the task accordingly.

Also, later research did not replicate Terrace’s results; i.e. the aggressive responses were also found in subjects who learned via the “errorless” methods. See Rilling: Extinction Induced Aggression. To me, for the dog to undergo some momentary frustration in small doses seems better than to get a big dose later.

Conclusion

In short: I think the methods used in the original “errorless” learning would be inappropriate, and in some cases inhumane to use on our pets, and the method by which the pigeons learned a discrimination behavior appeared to impede further learning.

I understand why people use the term. They want to clarify that they are doing their best to make the training experience fun and successful for their dog, and to emphasize that their approach is humane. Maybe there is a better way to say that!

Dr. Susan Friedman uses the term, “Reduced error learning.” To me it is more accurate, and doesn’t carry the baggage of Terrace’s term.

As trainers who use learning theory, we know the value and sometimes the difficulty of getting terminology right. And we understand that discussion is a lot more straightforward when everybody has general agreement on terminology. So from what I have learned here, I am encouraging folks to not morph a term that has a scientific, historical meaning into something a bit different, and especially not to attach a glamour to it because it sounds so nice.

I’m all for making it easy for our animals to be right in order to initially learn the behavior, then very gradually raise the difficulty. Of course! That is a basic tenet of effective, humane training. But it seems to me that striving to get an extremely low error rate can have a very high price.

Anybody have any examples of standalone behaviors that would profit from strict errorless methods? I’d love to know. Also, please note that I did not do a complete literature survey on errorless learning. It’s a large topic. Maybe I missed something important. If I did, please tell me!

Thanks for reading!

Coming up soon:

Share Button

About eileenanddogs

Passionate amateur dog trainer, writer, and learning theory geek.

Eileen Anderson on Google+

This entry was posted in Cues, Discrimination, Dog training hints, Operant conditioning, Terminology, Training philosophy and tagged , , , , . Bookmark the permalink.

21 Responses to “Errorless” Learning

  1. Margery Cavins says:

    Hmmmm…. You seem to be saying that the design of the experiment defined the concept or the term. I don’t think that’s the case, but I’ve got nothing to offer “historically” to show that. My bosses used the “errorless” learning concept in designing testing materials for a Precision Teaching Project in 70-74. The term wasn’t used anywhere in ‘our’ written material that I know of, but that’s what I was told was behind our decision to remove whatever had prompted an error from our testing materials. Our tests didn’t involve ‘discrimination’ at all, but simply tested the rate of a specific behavior. So, maybe I’ve been misusing the term for 42 years? Fluency was the key… by removing letters, sounds, words, numbers, etc. that caused an error, the fluency “rate” was achieved rapidly, then the letter would be added in, the rate of correct behavior would drop, but rise rapidly, to fluency, where another letter would be added in. It seemed very efficient with high learner confidence. Not ‘errorless’ learning? I’m not sure.

    • Margery, that’s fascinating. At first I thought you were describing the method used with autistic kids, but it seems that you are talking about yet another twist on it. Removing whatever prompted the error. It’s related, but not quite the same as the prompting thing I’ve seen. Wow. Here’s an article that is a historical link between what you are describing (I think) and Terrace’s work. So they are all related. When I throw off some mental fog I may write a little addendum in my post. I still feel like the term is fraught with too much stuff for me personally to use with dogs but I’m really glad you shared your knowledge about this.

  2. I love this post! I have a knee-jerk reaction to the term “errorless training” because I read a puppy-raising guide by a renowned vet who kept talking about “errorless housetraining” and “errorless chew-toy training,” so every time Barnum had an accident or didn’t understand what the hell to do with a chew toy (or wasn’t hungry), I thought I was A Horrible Failure and my puppy would be RUINED!

    Basically, my point is that the term is incredibly loaded because it sets up an impossibly high standard (errorless = perfection, no?). So, ironically, this idea/technique that is supposed to set up both dog and person to succeed ended up putting a HUGE amount of pressure on me, Barnum, and my household.

    I know that’s not what your post is about; it’s about how the term came to be from lab experiments and how people are applying that term today. But, as you said, words/semantics DO matter.

    This was a fascinating post, and I learned a lot. Also, I discovered with Barnum, because I tried to start out with no aversives and only success (DIDN’T work! LOL), that then when he was used to only succeeding, and I would accidentally make it TOO hard (not realizing I was lumping), that it was super stressful for him. However, when I became a better splitter and he learned that the absence of a click was not the end of the world, he — like Clara — learned to make use of the errors.

    I agree with your conclusions. My favorite part in the video is when — after Clara has tried the second lid and gone back to tapping the correct one — there is a moment (1:36) where she comes back for the next rep, glances at the 2nd lid, and then turns and focuses on the correct one. It was a glorious moment of seeing her think, “No, that’s the no-pay lid. I’ll ignore that.”

    • Thank you Sharon! I agree that there is so much more baggage with this than I talked about, and you described it well. First is errorless learning literally possible (no), and second, if it were, is it desirable (well…)? You answered that one nicely. Some day each dog has to learn that in some situations, some behaviors don’t get a click. And each dog is so different. Is it better to learn about mistakes sooner or later? That’s one of the big, subtle challenge of training and I think it does our dogs a disservice to assume that a protocol of artificially and extremely minimizing errors at all times is the way to go. My post was already too long so I cut out a whole section asking how we could possibly go for “errorless” in cue recognition or stim control, when the whole point is to teach the dog the boundaries of the cue. Anyway, I’m so glad you liked it. I’ve been holding this one back for a while and I’m so glad it proved helpful.

  3. P.S. It’s too bad Clara doesn’t have more personality. Also, I wish she would wag her tail more. ;-b

  4. Marjorie says:

    Interesting post as always, I learn so much. My dogs are requesting you post somthing on “errorless training.”

  5. Mary says:

    The out takes are a riot, Clara knew the game before you even started, lol. Her enthusiasm is great. I seem to remember a contest video on the Karen Pryor web site of a black poodle learning discrimination (it was 2-3yrs ago). It was an amazing demonstration. I’ll try to find it when I get a chance.

  6. Yes, I’ve seen that video, too. If you run across it it would be great if you posted it here. About the outtakes: I had recently had two or three long shaping sessions with Clara where I was shaping her to cross her paws. I had gotten her to lifting her left foot, including that I reinforced her foot targeting my knee by accident. Bad mistake. I did get the foot lift but decided to discontinue that project for a while….

    • Mary says:

      I understand, I shape with Rex too and it’s amazing how they can remember what was working in a previous session, I make so many mistakes clicking that lead Rex into unintended behaviors that I’ve learned to go with the flow. About half the time I get the behavior I had in mind and the other half Rex gets to decide. Rex enjoys shaping sessions so it’s lots of fun either way.

  7. Pingback: Läsvärt #2 | klickerklok.se

  8. Pingback: Errorless Learning II | eileenanddogs

  9. Pingback: Is It Really Just a Tap? Shock Collar Training Explained | eileenanddogs

  10. Pingback: Lumping It: A Public Service Announcement | eileenanddogs

  11. Pingback: Superstition Ain’t the Way | eileenanddogs

  12. Pingback: Dogs Who Like to be Petted or Touched | eileenanddogs

  13. Pingback: Let the Treat Fit the Feat | eileenanddogs

  14. Pingback: Contest! The Faces of Summer | eileenanddogs

  15. “If an error happens in the forest and no one says anything, did it really happen?”…..made me laugh out loud……. 🙂 Thanks for this piece (and its addendum), Eileen! I just read Susan Friedman’s new review article on errorless learning in the new Journal of Animal Behavior Technology and was considering writing an essay about it for The Science Dog. Then…..I found your two pieces and realized, “No need, Eilleen’s got this one covered”! Great essays – Thanks as always for your clear thinking and writing, Eileen! Linda

    • Eileen Anderson says:

      Thanks, Linda! I was hoping you saw Part 2… Part 1 has some problems by itself. But Susan is one of the ones who has convinced me that sharing the learning process can be valuable!

Comments are closed.