This test that you keep using……

The availability heuristic is a common cognitive error that influences our ability to make accurate decisions. It is operating full-force whenever we base a decision upon evidence that is easily available (i.e. dramatic, obvious, easily measured) but that may not actually reflect reality. In practice, this means that we pay more attention to evidence that is salient (obvious and dramatic) and tend to ignore evidence that may be more compelling but not quite so sensational.

Take for example, shark attacks. The feeling that shark attacks are far more common and that we are at greatly inflated risk than we actually are occurs because of the extensive and sensationalist media coverage that a single shark encounter attracts. As a result, when you consider going to the beach this summer, an image of a shark pops into your mind because such an image is highly available to you. However, while shark attacks can and do happen, an examination of the actual risk is much lower that our perceptions lead us to believe.

Shark      Coconuts                     PERCEPTION                                                             REALITY

I will return to the significance of the availability error shortly. Let’s now turn to an important dog topic – the expression of food-related aggression in dogs. (There will be a tie-in, I promise. 🙂 )

Background information: Food-related aggression (FA) is a specific subtype of resource guarding in dogs. It’s expression  can vary in intensity from a dog who simply shows tenseness near his food bowl, to freezing, growling, or biting a person who interferes with the dog while he or she is eating. Most of the standardized behavior evaluations that are used by shelters and rescue groups include an assessment for FA. For reasons of safety, many use a fake plastic or rubber hand that is attached to a long stick for this test. Although procedures vary somewhat, the test for FA involves interfering with the dog while he is eating from a bowl, first by placing the fake hand into the bowl and pulling it away and then by attempting to push the dog’s face away from his food by pressing the instrument alongside the dog’s face. The validity of this test, meaning its ability to correctly identify dogs who do (and do not) truly have FA, is an important issue because dogs who exhibit FA during a behavior evaluation are almost always identified as an adoption risk, which can lead to reduced opportunities for finding a home, and at some shelters, to automatic euthanasia.

Assess A HandTESTING FA WITH FAKE HAND

2004 Study: Despite its ubiquitous inclusion in behavior tests, few studies have actually examined the reliability of the fake hand test for FA. A few years ago, a group of researchers at Cornell conducted a study with dogs who had  a history of various forms of aggression, including FA (1). They found a positive and statistically significant correlation between showing an aggressive response toward the fake hand and previously exhibited aggression in the dog. However, the relationship was weak and a substantial number of dogs who were NOT aggressive also tested positive (i.e. reacted to the hand) when tested. The authors recommended the use of caution when using a fake hand in behavior tests because of the high number of both false positive and false negative responses that they found. A limitation of this study was that because the researchers used dogs with a known history of different types of aggression who were already in their permanent homes, they could not make conclusions about the predictive value of the test. To do this, we needed a study that examined how well the fake hand test, when administered to dogs in a shelter environment, correlates with dogs’ future behavior when living in homes. Such a study was published in September, 2013 in the journal Applied Animal Behaviour Science (2).

2013 Study:  Dr. Amy Marder and her colleagues at the Center for Shelter Dogs in Boston, MA  tested a group of 97 dogs using a standardized canine behavior evaluation that included a test for FA. Dogs showing extreme aggression or multiple forms of aggression were excluded from the study for ethical and safety reasons. Following testing, all of the dogs were adopted into homes. Adopters of dogs who showed food aggression (FA+) were provided with additional instructions for handling the dog during feeding times, but the dogs themselves received no additional training or behavior modification prior to adoption. Adoptive owners were surveyed to assess the dog’s behavior in the home at 3 days, 3 weeks and 3 months following adoption.

Results: Of the group of 97 tested dogs, 20 dogs (21 %) reacted aggressively to the hand and were classified as FA+; 77 dogs did not react and were identified as FA-. Of the 20 dogs who were classified as FA+, approximately half (11/20, 55 %) were reported by their owners to show food-related aggression while in the home and nine of the FA+ dogs (45 %) showed no signs of food aggression when in the home. Of the 77 dogs who were classified as FA-, the majority (60/77, 78 %) were also FA- when in their adoptive homes. However, 17 dogs from this group, 22 %, did show signs of FA when in the home, even though they had tested negative for FA while in the shelter. A final result was that the majority of owners of dogs who were showing FA in the home reported that they did not consider their dog’s behavior to be problematic and that they would definitely adopt the same dog again.

Take away for dog folks:

  1. The authors found that the negative predictive value of the test was high since 78 percent of dogs who tested negative in a shelter environment showed no food aggressive behaviors when in their adoptive home. (This is good).
  2. The positive predictive value of the test was low since only 55 percent of dogs who tested positive in the shelter environment showed food aggression when in the home (This is bad).
  3. Owners may perceive food-related aggression as much less problematic than do shelter staff and may have little trouble managing dogs who are reactive around their food bowls.

If that information does not give you enough to chew upon, let me contribute an additional question to this controversial (and apparently quite polarized) topic. What do these data say about the test itself?

There is really no question that the data presented in this study, along with the Cornell study, suggest something additional. Realizing that this is a sacred cow to those who are highly committed to their fake hands, I offer up the suggestion that perhaps the fake-hand test is not measuring what its users think it is measuring. (In other words, it is not a valid test of FA).

Keep UsingThis test that you keep using……..

Here is why (stay with me here; this gets long but it is worth the ride…..): The researchers reported positive and negative predictive values for the fake hand test (numbers noted above), but they also had data available to calculate two additional measures of a diagnostic test’s validity. These are referred to as sensitivity (a test’s ability to correctly identify all positive responses) and specificity (its ability to correctly identify all negative responses). I went ahead and punched these numbers using the data that the paper provided and found this:

  • Fake Hand Test Sensitivity = 39 % This means that 39 percent of the time, the fake-hand correctly identified FA in the dogs who actually had it. The flip side of this statistic is probably more important. It also means that almost 2/3 of the time (61 %), the fake-hand either incorrectly identified a dog who was FA- as being FA+ or missed the identification and labeled a dog who was FA+ as being FA-. Although sensitivity values are considered to be a relative measure, I do not think anyone would try to argue that 39 percent success rate signifies a valid test. (Especially in light of the fact that a positive result for this particular test can mean the end of life for the dog).
  • Fake Hand Test Specificity = 87 %. This means that the majority of the time, if the fake hand says a dog is non-reactive around his food bowl, it is correct. Only 13 percent of dogs who tested FA- actually had FA. While this is a desirable value for the test, high specificity alone is not enough.
  • Supporting data? This was not the first study that has examined the use of the fake hand in behavior evaluations, but it is the first study that has measured the predictive value of the test. It is important to note that to date, there are no published studies that provide data showing that using a fake hand to diagnose food reactivity in dogs is a highly reliable test. None.

Which begs the question – Why do temperament tests that are used with shelter dogs continue to include the fake hand as a test for food aggression?

IllogicalSeems a bit illogical, doesn’t it?

There are a few possibilities:

  1. It is simple and measurable: Unlike much of what we do in behavior and training, the Fake Hand test is pretty easy to administer and to score. Therefore, it is a shoe-in for being included in a battery of tests that can be quickly administered to a lot of dogs and by personnel who have varying levels of expertise.
  2. The use of the fake hand is well-established: Many, but not all, of the behavior assessment tests that are used in shelters today include a test for FA that uses a fake hand (3,4). Many of these tests are highly standardized and include specific training programs for shelter staff who administer them. However, while proponents of the fake hand insist that a set of clear and very specific steps are used in the test’s administration (i.e. how far to stand away from the bowl, how many times the dog’s face is pushed, how to manipulate the bowl), such protestations are a moot point since none of the specific guidelines for administering the tests have been validated either.
  3. The results are dramatic and salient – i.e. AVAILABLE: A dog who reacts aggressively when a fake hand is shoved in his face while he is eating provides us with an example of the availability heuristic in action. Aggressive responses in dogs elicit dramatic and involuntary reactions in those who witness the response –  a rush of adrenaline, a bit of fear, perhaps even a little bit of the “stopping to watch a car wreck” feeling, if you will.  Just as we react strongly (and illogically) to reports of shark attacks, so too might an evaluator react emotionally to an aggressing dog. The fallout is that the aggression that is provoked by a fake hand during a behavior test may acquire more significance than it actually has in real life.  (This is supported by Dr. Marder’s results when interviewing owners of FA+ dogs, who did not see FA as such a big deal). And, because the provoked aggressive response in the dog is dramatic and obvious, the evaluator now feels compelled to do something about the reaction that was provoked – special adopts, no adopt, euthanize.

        shark3         Pos FA Test                   AVAILABILITY HEURISTIC – DRAMATIC IMAGES STICK WITH US

Here’s a bombshell….Perhaps poking a dog in the face with a fake hand while he is eating in a shelter environment is not a valid way to test for food aggression:  The sensitivity statistic of 39 % suggests that at least some (if not the majority) of dogs who react when tested with  a fake hand are not showing FA. At the very least, this paper and this particular statistic suggests that the presumed test for FA using a fake hand is not testing for the thing that proponents think it is testing for. Additionally, the availability error may lead those who regularly administer this test to assign excessive significance to FA because of the salience of provoked responses in the test and highly inflated perceptions of risk to owners. Given that the fake hand test leads to decisions that severely reduce a dog’s chances of being adopted into a home or may even result in the death of the dog, this is a possibility that must be raised and considered.

Cited References:

  1. Kroll TL, Houpt KA, Erb HN. The use of novel stimuli as indicators of aggressive behavior in dogs. Journal of the American Animal Hospital Association 2004; 40:13-19.
  2. Marder AR, Shabelansky A, Patronek GJ, Dowling-Guyer S, D’Arpino SS. Food-related aggression in shelter dogs: A comparison of behavior identified by a behavior evaluation in the shelter and owner reports after adoption. Applied Animal Behaviour Science 2013; 148:150-156.
  3. Barnard S, Siracusa C, Reisner I, et al. Validity of model devices used to assess canine temperament in behavioral tests. Applied Animal Behaviour Science 2012; 138:79-87.
  4. Taylor KD, Mills DS. The development and assessment of temperament tests for adult companion dogs. Journal of Veterinary Behavior 2006; 1:94-108.