eileenanddogs

Tag: errorless learning

Errorless Learning II

Errorless Learning II

Seems like I’ve been eating a lot of humble pie lately. Pull up a chair and have a slice with me, won’t you?

This is an addendum to and correction of my post on Errorless Learning.

A knowledgeable Internet friend  gently let me know a few things about the origin of Errorless Learning that I had incorrect in my earlier post. She was generous with her time and patient with my learning curve. These misconceptions affected some of my conclusions as well. I’m letting my earlier article stand, since there is still much in there that I think is worthwhile and accurate. But I’m linking them and I hope anyone who read it will read this as well.

Mine were common misconceptions. Up until today, the Wikipedia article on errorless learning, along with quite a few other posts and articles, attributed the term to Herbert Terrace and cited his experiments with pigeons as the beginning of its usage, as did I. But in 1963, behaviorists, psychologists, and educators had been discussing errorless learning for 30 years.

Errorless learning was an instructional design introduced by B.F. Skinner in the 1930s. There was controversy at the time, and following, about whether errors were necessary in learning a behavior. Skinner wrote, “…errors are not necessary for learning to occur. Errors are not a function of learning or vice-versa nor are they blamed on the learning. Errors are a function of poor analysis of behavior, a poorly designed shaping program, moving too fast from step to step in the program, and the lack of the prerequisite behavior necessary for success in the program.”

It helps to know that Skinner was responding to the then-famous 1898 Thorndike paper called “Trial and Error Learning” which posited that learning was a slow and laborious process. Skinner’s response was that it didn’t  have to be–with proper planning, the teacher could grease the skids for the learner and learning could be achieved through “trial” only. Hence “errorless learning.”

Skinner’s system has been summarized really well elsewhere so I’m not going to go into it at length in this post. I may later, since it seems overshadowed by Terrace’s work and is fascinating. My friend summed it up as an anti-lumping program: through good planning make the correct behavior so easy as to be almost inevitable, slice the desired behavior/s thinly and maintain a very high rate of reinforcement. This method is highly humane, in contrast to what I was objecting to in my other post. But it’s more than “setting the animal up for success” as we think of it in a general way. As I understand it, it entails much more planning and prompting than we usually see, even in clicker training these days.

Terrace’s work, on which I based my objections in the other post, was an example of how low the error rate could be pushed down with extreme planning and control of the environment and training. I still maintain that the level of control he maintained with pigeons in the lab would be not only unfeasible but some aspects actually undesirable in our home environments. And I maintain, if not skepticism, an appreciation of the challenges inherent in teaching a later conflicting behavior if one behavior were taught with controls similar to Terrace’s, and especially with the number of repetitions. However, these concerns cancel out. If we can’t attain the control and terrifically low rate of error Terrace got in the first place, we aren’t going to have the problems that Marsh and Johnson‘s pigeons had in learning a conflicting behavior.

Geikie, a greater sulfur-crested cockatoo, in a color discrimination session with a high rate of reinforcement
Geikie, a greater sulfur-crested cockatoo, in a color discrimination session with a high rate of reinforcement

Another astute reader questioned the logic of saying that the design of the experiment defined the concept or the term (thanks Margery!). Turned out she was right on target, since the concept and the name predated the experiments. Not only that, but the educational work she described doing herself in her comment on  the original post is directly related to the early learning methods for humans that Skinner worked on. Which were also very humane, pleasant, and natural for the learner.

The Method vs the Term

Which brings me back to one of my original gripes. The term itself. But first I can now say what is good about the term. It refers to the fact that a subject does not need to make errors to learn a behavior.

What I don’t like about the term remains: it is inaccurate and unattainable for lots of behaviors for most of us in real world training. Jesus Rosales Ruiz writes in a piece that discusses errorless learning in a positive way,

“At each step of the program, the learner has a reasonable chance of success….Good shaping is characterized by high rates of reinforcement and low use of extinction.”  (emphasis mine)

Dr. Rosales Ruiz trains in an academic setting, but also in the real world. “Reasonable chance” of success is not 100%. And “low use of extinction” is not 0%. These are much friendlier terms for the average trainer. But as my friend reminds me, if we treat the term “errorless” as an ideal and put our mind and heart into setting the animal up to succeed as much as possible, we will get really good training.

The Rosales Ruiz article is available as a download in this post in Mary Hunter’s great blog. It is a must-read.

I have a final objection to the term, not the method. It is probably the most important of all that I have said.

I don’t like it because it can make people feel bad, and thereby discourage them. Approaching zero errors requires not only great control over the learning environment, but great skill, which many of us don’t have. For literal people, the term itself, coupled with the amazingly low error rate in Terrace’s article that is always cited, can make the whole concept daunting and unapproachable.

In my other post I suggested a couple of candidate behaviors for errorless learning; behaviors for which there is no potential conflicting opposite behavior, such as scent work like a diabetic alert. There is another obvious candidate in the dog world: house training. We really would prefer no mistakes, and hardly anybody anymore would say that is is important that the dog have an accident in the house and be punished for it for their understanding to become complete. It’s a great example. However, I know several people who thought they were utter failures because they failed to live up to the “no accidents” and errorless approach that is written about in at least one puppy book, including dire predictions of terrible things that will happen if there are errors.

Is this use of a pee pad in error?
Is Summer’s use of a pee pad in error?

Which brings me to my final point. If the role/goal of the teacher is to set the student up for success and make the desired behavior the easiest and most likely, shouldn’t the term itself focus on success and not errors? Also, Skinner himself said that it was about the teaching, not the learning.  I would never have written any of these rants if the term had been something Enhanced Chances of Success Teaching instead of Errorless Learning. But I doubt if the whole educational world is going to change its nomenclature just for me.

Grins. Thanks for reading!

Coming up soon:

Eileenanddogs on Youtube

P.S. A Project

The Wikipedia article has been amended to begin with Skinner’s role in errorless learning and now has several references for that. However, the bulk of the article is still about Terrace’s work, which is actually just one of the many offshoots.  Skinner’s method and the discussions from the 1930s on need to be the focus. It would be great if somebody who knows about that history, or wants to read up on it, would further edit the article.

“Errorless” Learning

“Errorless” Learning

Addendum, 2/9/13. Please be aware that there are some historical inaccuracies in this post, mostly related to the origin of the method and term Errorless Learning. The mistakes affect some of my conclusions as well. Please read Errorless Learning II if you read this post, or instead of reading this post. –Eileen

You will never hear me say, or see me write, “It’s only semantics.” I grew up in a family full of passionate readers, English and education majors, and teachers. Not to mention musicians, who are often quite obsessed with passionate about language as well.

Semantics is very important to me because it deals with whether or not we understand each other. Here is part of the definition from Wikipedia: “It [semantics] is often used in ordinary language for denoting a problem of understanding that comes down to word selection or connotation.” In other words (ha ha), if I use a word or phrase and it means one thing to me and to you it means something very different, we instantly have a communication problem, and we may not even know it.

That is the context in which I offer this post. Don’t worry, I’m going to make it to dog training, and there are actually some dog training hints in here.

Zani beginning the shell game
Zani beginning the shell game

There is a newish catch phrase going around the science-based dog training community: “errorless learning.” I am seeing more and more usage of the term, and reading pieces that equate it with the ultimate humane training. I think a lot of folks have picked up the phrase and are using it to mean setting your dog up for success in a general way. I’m aware of some others who associate it with training with positive reinforcement only. Some use it to indicate that they do not use No Reward Markers. (If an error happens in the forest and no one says anything, did it really happen? Sorry.)

But actually the phrase is not new at all. It refers to a specific teaching methodology that has been well investigated by research.  I am going to describe the original research on so-called errorless learning, some subsequent research, and explain why I think the term is currently being misused and perhaps wrongly proposed as a goal in our companion animal training.

Initial Research about Error Free Learning

Skinner box

Herbert Terrace published, “Discrimination learning with and without ‘errors’,” in 1963. The experiments were performed on pigeons in Skinner boxes. The discrimination behavior taught to the pigeons was to peck on an illuminated key for a food reward when the key was lit internally with a red bulb and not peck when the key was illuminated with a green bulb.

At the beginning of the experiment, the key (in the darkened enclosure) was lit bright red. Apparently it is easy to get birds to peck on a colored, illuminated key, and importantly, they generally will not peck a dark key. The pigeons got reinforced for pecking on the brightly lit red key. The key went dark between trials and no reinforcement was available.

The birds were divided into four groups.  After the birds had a period during which the key glowed red and during which they got food rewards for pecking it, the color and brightness of the key were changed according to four different protocols. For the pigeons who learned the discrimination the fastest, called the “early progressive” group,  the procedure was as follows: early in the experiment, after a “dark” period, the key was illuminated at an extremely low level with the green bulb, and no reinforcement was available when the birds pecked at it. (But mostly they didn’t.) This was alternated with periods where the key was illuminated bright red, and the pigeons were reinforced for pecking it. The duration and intensity of the green light were very gradually  increased from a dark key of 5 seconds duration to a bright green key of 3 minutes duration. In short: the key morphed from completely dark to bright green so gradually that to the pigeons it remained “unattractive” to peck.

For the three other groups of pigeons, there were variations in how early the green key was introduced, and whether it was introduced gradually or at full intensity at the very beginning.

The pigeons in the “early progressive” group had an amazingly low error rate. They pecked at the green key well under 1% of the time.

This technique is the ancestor of what we often do nowadays in teaching a discrimination. If I want my dog to touch her paw to a cup with food under it and ignore another cup, I will first have only the desired cup present. I’ll reinforce some iterations of touching that cup. Then when I first introduce the second cup (which is empty) I may put it in an inconvenient place for the dog to touch it and only gradually bring it physically closer. In other words I will make it easy for the dog to be right, sneak in the “wrong” cup so that at first it is just part of the background, and raise the challenge very gradually.

But before we adopt Terrace’s term of errorless learning to apply to such techniques, let’s look at some differences between what he did and what we do–and are even willing to do–with our pets. Here are some of the primary differences between his training situation and ours with our pet or performance dogs:

  • The pigeons were food deprived. They were kept at 80% of their normal body weight for a period starting two weeks before the start of the experiment to the end of the experiment. This is very common in such experiments.
  • The pigeons were isolated in a Skinner box during the experiments.
  • White noise was played to block external sound.
  • The light intensity and duration on the keys were controlled with great precision by an electrical unit.
  • The birds had not been taught anything before.
  • Although it is not stated in the paper, it is fair to assume that the birds had no particular relationship with humans other than being handled; they were not pets.
  • The birds were being taught only one behavior (this is a crucial point).
  • There was no proofing. The birds were not challenged to perform the behavior under any other conditions. It’s fair to assume the behavior wasn’t generalized.

Again, the error rate of the highly controlled birds (only in the early progressive group) was less than 1%.

From the above points, it appears that impressively low error rate was possible at least in part because of the technology available to the experimentors and the extreme control over experimental conditions that was possible for them. We don’t generally have rheostats to gradually change the intensity of lights, or little trucks to drag in the “wrong” object in increments of exactly half an inch.

You Can’t Do This At Home, But…

The conditions under which the pigeons were trained cannot be emulated by the average trainer, for technological reasons, reasons related to environmental control, and also humane reasons.

But I wanted to show how that method is relevant to some teaching strategies we use with our dogs. I made a video of the descendant of Terrace’s method as applied to scent discrimination: the Shell Game. In the video I am teaching Clara to tap only a jar lid that has a treat under it. My goal was to demonstrate the method of sneaking the “incorrect” lid in from the side while the dog is happily bopping the food filled one.

As usual, I got more than I bargained for. For starters, I utterly failed at sneakiness. You’ll see. We survived that. But in addition, I got some interesting footage of Clara making an “error” pretty early on (tapping the wrong lid), which was probably because of my clumsiness and bringing it in too fast. However, although different dogs react differently, making this error appeared to help teach her more about the game. See what you think.

Mine is not a tutorial video. If you want to teach your dog the Shell game, here’s a really nice tutorial by Donna Hill. She uses a different method of helping her dog succeed, and does a beautiful job as usual. By the way, in my video Clara was standing in for Zani and it was her first time ever playing the game. All the paw flailing she did (including belting and grabbing me a few times) was because we’ve been spending quite a bit of time shaping a trick with a lot of paw movement. It took her a while to get that out of her system and figure out the new game. That’s another difference between training in the lab and at home. Whatever else has been reinforced recently or richly will likely creep into the new thing you are working on.

What you see in my video, however clumsily done, and what many people seem to mean by “errorless learning, ” is helping the animal to be right. Terrace’s work went far beyond making it easy for the pigeons to be right, however. Because of his use of technology and the controls available in a laboratory, he made it very, very difficult for the birds to be wrong. Is this a good thing?

Would You Even Want To? The Big Drawback of that Huge Success Rate

OK, so what if we could achieve that kind of low error rate while still being kind to our animals, and let’s further assume that we were able to teach it to fluency in a real life environment. Are there any other problems?

Yes. Back to the pigeons: what if later we needed them to peck the key when it was green instead? Biiiiig problem.

Discrimination Reversal Following Learning without “Errors” by Marsh and Johnson in 1968 demonstrated that pigeons taught to peck a red key and ignore a green one, using Terrace’s method, could not, even after five days, be induced to learn a new behavior of pecking the green key.

For most things we want to teach our dogs, that would be a huge problem.

Granted that there are some behaviors and tasks in the dog world that are standalone, in the sense that you would not be likely to teach a conflicting behavior. Diabetic alert dogs come to mind. As I understand it, they learn to react to one and only one odor for their working lives. (Correct me if I’m wrong, folks.) Cadaver search dogs. Perhaps some other types of search dogs, but not all.

But in the service dog, pet and performance dog worlds, it seems to me that these kinds of needs are rare. Most people teach their dogs both sit and down. Agility and herding dogs aren’t taught left turns only. They learn left and right. Service dogs typically learn to both push and pull, use left and right. They have to be ready to pick some stuff up and not even touch other stuff.

Anyone who has trained a dog, for example, to raise her right paw, got that fluent, then taught her to raise her left paw, is familiar with the period of frustration the dog goes through when the familiar behavior no longer pays off. I have a post related to that about the mini extinction bursts that our dogs undergo in shaping exercises. The research tells us that if we had trained the right paw raise errorlessly (a difficult challenge), the dog’s frustration when trying to learn the left paw raise would greatly increase.

The pigeons learned only to do one thing, and the exercise did not teach them problem solving skills or how to play other training games with humans. And it blew their little minds when they were asked to do something else.

This is the biggest reason I do not have errorless learning as a goal for my dogs, nor do I use the term for the teaching strategies I use and admire. Taking a long view, training them to do one thing using something close to Terrace’s method could set them up for tons of stress and frustration later.

Learning What’s Wrong to Learn What’s Right

It’s a little bit out of style to emphasize the importance of your dog knowing what the wrong behavior is. It smacks of corrections and punishment based training. But as clicker trainers say, the lack of a click is information. In my video, because I lumped a bit and moved the second lid into the picture so fast, Clara made an error fairly early on. Her error consisted of tapping the empty lid. She tapped it a couple of times, got no treat, sniffed and licked it, then proceeded to the correct lid and tapped it. She then ignored the “wrong” lid from then on in that session. I think she learned something really important. There are lids with nothing under them! She is going to have to use her nose to figure it out. It seems to me that learning that at this point was not at all harmful for this dog.

Let’s Add to the Terminology Confusion: Applications to Human Learning

Errorless learning is used very successfully in operant conditioning programs for autistic children. But the process is quite different, since we primates mimic so easily and often we can understand and follow verbal instructions. From this website comes a good definition:

[Errorless learning]: The use of instruction designed to prevent errors or incorrect responses. Typically prompts (artificial cues that provide assistance to the learner about the correct response) are presented so that an individual engages in a behavior that is being targeted. Once the individual is engaging in the behavior appropriately, then these prompts are faded or removed slowly and systematically so that the correct behavior is made with few or no errors.

Here is a lovely little video that shows that technique.

But think about whether we could apply that method to dog training. The child is learning to perform the initial task through either the verbal instruction, mimicking the hand movement of the teacher, or both. Neither of those are available to us with dogs. If the behavior is new, they don’t already know the verbal cue. And although there seems to be some small evidence of dogs learning by mimicry (of other dogs), you can’t take your average dog, put your hand on an object, then expect them to put their paw on it just like you did.

Aggression

Back to the birds. Terrace later claimed as a by-product of some later experiments that pigeons trained using a trial and error method rather than his “errorless” approach showed aggressive behavior when pecking the wrong key produced no reinforcement. The article is “Behavioral contrast and the peak shift: effect of extended discrimination training” and is available in full online. In that work and in a later study he claimed that these behaviors were not present with his “errorless” cohort.

I have heard this used as an argument for “errorless” learning for dogs. Our dogs might get enraged and aggressive if they make too many mistakes, so we need to absolutely minimize by any means possible the number of mistakes they make. But again, there are big differences in the training environment between the lab and training our dogs at home. Our training is relationship based. And a big part of the job of the human trainer is to monitor the emotional state of the dog as evidenced by its behavior and adjust the task accordingly.

Also, later research did not replicate Terrace’s results; i.e. the aggressive responses were also found in subjects who learned via the “errorless” methods. See Rilling: Extinction Induced Aggression. To me, for the dog to undergo some momentary frustration in small doses seems better than to get a big dose later.

Conclusion

In short: I think the methods used in the original “errorless” learning would be inappropriate, and in some cases inhumane to use on our pets, and the method by which the pigeons learned a discrimination behavior appeared to impede further learning.

I understand why people use the term. They want to clarify that they are doing their best to make the training experience fun and successful for their dog, and to emphasize that their approach is humane. Maybe there is a better way to say that!

Dr. Susan Friedman uses the term, “Reduced error learning.” To me it is more accurate, and doesn’t carry the baggage of Terrace’s term.

As trainers who use learning theory, we know the value and sometimes the difficulty of getting terminology right. And we understand that discussion is a lot more straightforward when everybody has general agreement on terminology. So from what I have learned here, I am encouraging folks to not morph a term that has a scientific, historical meaning into something a bit different, and especially not to attach a glamour to it because it sounds so nice.

I’m all for making it easy for our animals to be right in order to initially learn the behavior, then very gradually raise the difficulty. Of course! That is a basic tenet of effective, humane training. But it seems to me that striving to get an extremely low error rate can have a very high price.

Anybody have any examples of standalone behaviors that would profit from strict errorless methods? I’d love to know. Also, please note that I did not do a complete literature survey on errorless learning. It’s a large topic. Maybe I missed something important. If I did, please tell me!

Thanks for reading!

Coming up soon:

Theme: Overlay by Kaira Extra Text
Cape Town, South Africa