I Failed to Falsify—Twice! (Falsifiability Part 2)

Graffiti on a brick building that says, "False"

Photo credit carnagenyc: see bottom of page

I want to share just how tricky this falsification stuff can be. In the last few weeks I’ve received two comments from readers that pushed me to rethink some things I’ve written. They were both presented very constructively, offering some ideas in the spirit of good dialogue and the search for truth. They included fascinating questions that were a bit technical and they got my attention. This post may be overly nerdy for some readers. But it has given me great pleasure to learn from my generous commenters and write about I’ve figured out so far.

Both of the comments were about falsifiability, and related to these two posts:

  1. Do You Dogs REALLY Want to Come In? In this post, I pointed out that my dog Zani was more likely to stay in the yard when I cued her to come in using a questioning tone. I stated that the reason was that the reinforcer involved was lower value, not because she intrinsically understood that I was offering a choice.
  2.  Falsifiability or Falsehood in Dog Training? (Part 1). In this post I explained the concept of falsifiability. I claimed that if we are truly following the science, then we should be able to explain what would falsify the tenets of learning theory that we study and use to train. In other words, we should understand the science behind what we do so well that we could state what it would take to prove it wrong.

 Both of the comments are in the comment sections for the falsifiability blog.

That Damn Diagram

The original (faulty) diagram

The first comment was about a diagram I had created. In the falsifiability blog, I showed a diagram of different responses one might get if one asked dog trainers how their methods could be falsified or disproved. I had a good time creating it, especially when including all the evasive or incoherent answers one might get. But the joke was on me. As commenter “A” pointed out, the one answer I gave as “correct” was incorrect. And it was supposed to be the crux of the diagram. Here’s how it went. 

The question was, 

As a science-based trainer, how would you falsify your methods?

The answer I offered as “correct” was, 

For learning theory in general, there would need to be multiple replicated studies that were deemed by experts to be robust enough to be considered strong evidence and included in learning theory and cognition textbooks.

The same would be true on a smaller scale for individual techniques.

Do you see the problem? I didn’t falsify anything. I didn’t even put forth anything to falsify. I just described where one might look and described the nature of evidence that could add evidence for or against…some stuff…involving learning theory.

I have updated the answer on the graphic on the original post. Now it reads:

Some findings that could falsify important aspects of learning theory would be multiple, well controlled, replicated studies that found things such as the following:

Behaviors followed contingently by an appetitive stimulus do not increase or maintain (given no other consequences or factors attributable to antecedents).

Behaviors that are followed contingently by the removal of an appetitive stimulus do not tend to decrease (same disclaimer).

Painful aversive stimuli that are used to punish or negatively reinforce behaviors have no fallout, and have the result of influencing an animal to be more confident, enthusiastic about training, and less fearful.

This is a better response because it explains how to falsify specific tenets of learning theory upon which we rely for our training.  

Falsifying the Hidden Assumption

The next reader’s comment is more complex, but incredibly instructive. In the falsifiability blog, I offered some rough evidence of what could falsify my previous claim that my dog responded more frequently to the cue that was tied to a higher magnitude reinforcer. I didn’t go into detail, but I offered some names in the scholarship world where a person could find research on the topic, then I wrote:    

The hypothesis could be falsified if this body of research was overturned with the results of new, replicated studies that showed no correlation between the animal’s response and the quality of the reinforcer, or an inverse correlation.

I think I did OK with this. But my commenter detected an assumption that was buried deeper in my thought processes. I had also written the following major point in the post:

When we want a dog to respond reliably to a cue, we use high value reinforcers on a dense schedule.  We also limit access to reinforcers for the behaviors we don’t want. So what would we do if we want the animal to have a more of a choice? The opposite! We would use a weak-ish reinforcer ourselves so as not to stack the deck in our direction.

Three disks on the ground with arrows on them. Can we falsify this whole "choice" thing?

Choices again!—Photo credit Derek Bruff: see bottom of page

My commenter wrote the following, which includes a great question:

Skinner would not, of course, agree that you have given your dog any more of an option by using a lower-value reinforcer on a CR [Ed: continuous reinforcement] schedule, but I suspect you have good reasons to use and hold on to the term. I guess the succinct way of asking this is—Why do you think your dog is “choosing” to come in or stay out, rather than just exhibiting a weaker but no less determined behavior? What sort of evidence would you need to falsify this part of the hypothesis? And, if you do think she is choosing in the low value recall case, why would you say your dog is NOT choosing, or has less choice in, responding to her RRR [Ed: highly reinforced recall] cue, rather than just saying she chooses the high value reinforcer more often, but just as freely?

Bingo. She got me. I can’t amass evidence for my claim that offering lower value reinforcers gave my dog more choice, nor can I falsify it. The act of making a choice is an unobservable process. We see only the behaviors that result from the choice (if there is one).

When we discuss “choice”:

  • We need to say what we mean by the term;
  • We need to know a lot about the cognitive abilities of dogs (do they think of it as “choice”?); and
  • We even need to consider freedom of the will. As in, do any of us—of any species—have “choice”? The neuroscientists are saying “not likely” to that question these days. (Which I think is funny because it’s the radical behaviorists who always get accused of thinking of organisms as robots.) 

To put it another way, if I have a cue to which a dog responds 95% of the time and a cue to which a dog responds 50% of the time, she is “making a choice” in both situations—or in neither. I don’t know which it is. But my commenter pointed very succinctly that I can’t claim that I am giving my dog more of a choice by offering one type of reinforcer instead of another. I can say only that the odds of her performing the behavior are probably changing. To repeat what my commenter said, “she chooses the high value reinforcer more often, but just as freely.”

Ironically, I wrote that post partly as a gripe against making unfalsifiable claims about dogs’ understanding of language. But I jumped right into the unfalsifiability trap myself. 

Am I Going To Quit Offering the Lower Value Treat for Going in the House?

Hell, no! I can’t say what’s going on in my dog’s head. I can’t claim that I am “giving her more choices.” But I can operationalize what I am doing when I offer that reinforcer. I am putting more opportunities for reinforcement in her environment. I am offering enrichment. And I think we can agree that that’s a good thing without bringing the mysteries of choice into the discussion.

Related Posts

Photo Credits
Photo of “FALSE” graffiti by carnagenyc via Flickr and under this license.
Photo of arrows by Derek Bruff via Flickr and under this license.

Text and diagram graphic copyright 2017 Eileen Anderson

Share Button
This entry was posted in Choice, Critical Thinking, Research, Terminology and tagged , , . Bookmark the permalink.

6 Responses to I Failed to Falsify—Twice! (Falsifiability Part 2)

  1. Aside from all the interesting ways we get “caught” in falsifiability traps, which has been a fascinating read, your last paragraph was so salient. We humans are a stingy bunch, and if I had one wish for all my students it would be to freely and lovingly provide more enrichment and reinforcement for dogs. In human social groups, eating together is an opportunity for bonding, for just plain feeling good in the company of others. So, I find ways to share that with my dogs as well, both while I’m training and while I’m not – just for fun! I agree with your dogs. #bestdoggraphicever #usefood #dontbestingy

    • Eileen Anderson says:

      Thanks, Anne! I agree with all that, and I’m really starting to think it’s a much better way to talk about the stuff people are trying to get at with “choice.” Thanks for the comment. (Glad you like the image!)

  2. Very interesting once again. When it comes to true choice, and this is more philosophical than scientific, and based on the assumption free will exists, it would be better to have similar consequences for both choices. One should not outweight the other. If you give a cookie for coming inside, you should also give a cookie for staying outside. It’s the choice-making you reinforce, not the option chosen, when both options are equally reinforced.
    I’m not saying this is good dog training necessarily, but it’s a way to present the dog with true choice. So you could do an experiment, and N=1 trial. You can determine her current chance for getting inside when you recall her. Then you can have a period where you transition to a reinforcement schedule for choice-making, then after an adjustment time you can determine the chance of her getting inside when you recall again. Repeat this process over months and see if there’s a difference.
    You want to prove there is a difference between the two, because if there is no difference that would indicate she had true choice already (which was your claim).
    I hope I said that the right way around (it’s late). Of course the assumption that true choice is possible must be made, but science does allow for “under certain conditions” conclusions and you also need to ignore the fact that either getting inside or staying in the garden may have a reinforcement history. This type of experiment would work better in a standardized trial, where you can make sure both have equal reinforcement history.

    Abou the learning theory: It’s a hard one, because the quadrants of operant conditioning you are referring to are defined by the outcome. So no, I don’t think the quadrant is falsifiable. That doesn’t mean other parts of learning theory aren’t falsifiable. One thing that should be falsified way more often are things like “cookies are reinforcing for my dog”. Are they? In what circumstances?

    • Eileen Anderson says:

      Agnes, wonderful points. Your suggestion of equalizing the consequences is a great one. I don’t know if you are aware of the experiments of Catania, but he and others did lab experiments about choice where they tried to separate possible reinforcing effects of choice itself from the effects of the results of the choice (an appetitive stimulus). I talk about the Catania studies a bit in this post about choice. You’ve suggested a way we possibly could even “do this at home,” although it’s not something I’d want to train, as you mentioned.

      Good point about quadrants, since they are not hypotheses. More like definitions or descriptions. I think I’ve seen them called empirical descriptions. I’m going to ponder another rewrite, but probably will leave that stand for a while. Observing different outcomes would serve to falsify the Law of Effect, perhaps? You have given me great food for thought. Thanks!

  3. Lauries says:

    “We are in favor of more opportunities for reinforcement!” made my day! Beautiful article, and this picture is so lovely!

Leave a Reply