Consider the (Caregiver) Placebo Effect

Most people are familiar with the concept of a “placebo effect”, the perception of improved health while unknowingly receiving a sham (placebo) treatment that in reality should have no benefit at all. Growing up, my mother referred to this as “giving someone a sugar pill”. The assumption is that because we believe that we are receiving an actual treatment, our mind tells us that we should feel a bit better. Then amazingly, we do feel better. We notice a reduction in symptoms and ultimately conclude that the “medicine” must be working. The irony is that placebos actually can be powerful medicine (or something), at least for some people, for some diseases, some of the time.

placebo-comic

Placebos and Us: The effects of placebos in human medicine are well-documented and are described with human diseases of almost every type. The highest level of placebo effect is seen with diseases that have subjective symptoms that are patient-reported and difficult to measure directly, that tend to fluctuate in severity, and that occur over long periods of time (i.e. are chronic). Examples include depression, anxiety-related disorders, gastric ulcer, asthma, and chronic pain. In medical research, an average placebo response rate of 35 percent is reported, with rates as high as 90 percent for some health conditions. By any standard, that is a whole lot of sugar pill response going on.

placebo

Placebo Control Groups: Although the reasons that we respond to placebos are not completely understood, medical researchers universally accept the importance of considering them when studying new treatments. Studies of new drugs or medical interventions include placebos as control groups to allow unbiased comparisons with the treatment or intervention that is being evaluated. Any effect that the placebo group shows is subtracted from the effect measured in subjects who are receiving the actual medication. The difference between the two is considered to be the degree of response attributable to the treatment. If a placebo control group was not included, it would be impossible to differentiate between a perceived response (placebo) and a real response to the treatment. Today, double-blind, placebo-controlled clinical trials are considered to be the Gold Standard of study designs by medical researchers. The “double-blind” part refers to the fact that in addition to having both a placebo group and a treatment group, neither the researchers nor the subjects know which subjects are getting the treatment and which are getting the placebo. (For more information about double-blind research trials with dogs, see “Thyroid on Trial“).

What about Dogs? Can a placebo effect occur with dogs? Possibly, but things work a bit differently where our dogs are concerned. Most obviously, while highly communicative in many ways, dogs cannot specifically tell us what part of their body is in pain, how intense that pain is, if it is abating, or by how much. Rather, we use our knowledge of a dog’s behavior and body language to determine how he is feeling. As their caregivers, we are the recorders and the reporters of our dogs’ health, symptoms, and response to treatments. Similar to human studies, this is most relevant when the symptoms are things that are not easily measured using medical tests and that are more subjective in nature.

A second important difference is that dogs are basically always blinded to treatments. Although they may understand that something different is being done to them (or that there is a strange pill buried in that piece of cheese), most people will agree that dogs do not have an understanding that they are being medicated for a particular health problem or are on the receiving end of a new behavior modification approach. As a result, unlike human patients, dogs lack the specific expectations and beliefs about health interventions that may be necessary for a placebo effect to occur directly. However, because it is the owner who reports many symptoms and changes in health to their veterinarian and also who conveys subjective information regarding the dog’s response to a given treatment, a different type of placebo effect may be in action with dogs. This is called a “caregiver placebo effect”. As with human maladies, the conditions for which this type of placebo effect has been described in dogs are those that involve subjective measures of health (pain, activity level, appetite) and that have a tendency to fluctuate in severity.

Let’s look at two examples – the caregiver placebo effect in dogs with osteoarthritis and in dogs with epilepsy.

Does Your Dog Hurt Less? Osteoarthritis is a painful and progressive health problem that can seriously impact a dog’s quality of life. A variety of medical and nutritional treatments are available today for afflicted dogs. These range from NSAIDS (ex. deracoxib, meloxicam), nutrient supplements (ex. glucosamine, chondroitin sulfate) to alternative medicine approaches (acupuncture, cold laser therapy). Researchers who have studied these treatments use subjective measures of lameness in which dogs’ owners and veterinarians numerically rate their dog’s degree of pain, mobility, and interest in daily activities in response to treatment. Some, but not all, studies also include objective measurements of arthritis that quantify the amount of weight-bearing in the affected legs and weight distribution in the body.

Arthritis Studies: In virtually all placebo-controlled studies of this type, a substantial proportion of owners and veterinarians have reported improvement in the placebo-treated dogs. However, when measured using weight-bearing techniques, the dogs in the placebo group showed no change in or a worsening of disease. Michael Conzemius and Richard Evans at the University of Minnesota’s College of Veterinary Medicine decided to quantify the actual magnitude of the placebo effect in this type of experimental trial (1). They analyzed the data from 58 dogs who were in the placebo control group of a large clinical trial that was testing the effectiveness of a new NSAID. All of the enrolled dogs had been diagnosed with osteoarthritis and had clinical signs of pain and changes in gait and mobility. This was a multi-centered design, which means that each dog’s own veterinarian conducted the bi-weekly evaluations of gait and lameness. Both owners and veterinarians completed questionnaires that measured whether the dog showed improvement, no change, or worsening of arthritis signs over a 6-week period. Neither the owners nor the veterinarians knew if their dog was receiving the placebo or the new drug.

Results: Half of the owners (50 percent) stated that their dog’s lameness decreased during the study, 40 percent reported no change, and 10 percent said that their dog’s pain had worsened. When these reports were compared with actual change as measured by force platform gait analysis, the caregiver placebo effect, (i.e. thinking that improvement occurred when there was either no change or an actual worsening of signs), occurred in 40 percent of owners. The veterinarians performed no better. A placebo effect occurred 40 to 45 percent of the time when veterinarians were evaluating dogs for changes in gait or pain. This means that not only were the owners strongly invested in seeing a positive outcome, so too were their veterinarians. This effect occurred despite the fact that all of the human participants were aware that their dog had a 50 percent chance of being in the placebo group or the drug group, and that there was no way to be certain which group their dog was in.

Seizure Study: This study used an approach called a “meta-analysis” which means that the researchers pooled and then reexamined data collected from several previous clinical trials (2). Veterinarians from North Carolina State University College of Veterinary Medicine and the University of Minnesota reviewed three placebo-controlled clinical trials that examined the use of novel, adjunct treatments for canine epilepsy. During the treatment period owners were asked to record all seizure activity, including the length of the seizure, its intensity, and the dog’s behavior before and immediately following the seizure. The pooled results showed that the majority of owners of dogs in the placebo group (79 %) reported a reduction in seizure frequency in their dog over the 6-week study period. Almost a third of the owners (29 %) said that there was a decrease of more than 50 percent, the level that was classified in the study protocols as indicative of a positive response to treatment.

What’s Going On? Well, several things, it appears. The most obvious explanation of the caregiver placebo effect in dogs is owner expectations of a positive response when they assume an actual treatment is being administered to the dog. Whenever we introduce a new medication or diet or training method and anticipate seeing an improvement in our dog’s health, nutritional well-being or behavior, we naturally tilt toward seeing positive results and away from seeing no change (or worse – a negative effect). This is a form of confirmation bias – seeing what we expect to see and that confirms our preexisting beliefs. In fact, an early study of the caregiver placebo effect in dogs found that when owners were asked to guess which group their dog was in, the owners who said that they were certain that their dog was in the treatment group (but was actually in the placebo group) demonstrated the strongest placebo effect (3).

Such expectations may be an especially strong motivator when we are dealing with maladies that have affected our dog for a long time, infringes upon the dog’s ability to enjoy life, and for which we feel that we are running out of options. Osteoarthritis and seizure disorders were the health conditions studied in these papers, but I can think of several other problems with our dogs for which we may easily succumb to the power of the placebo effect. These include chronic allergies, adverse reactions to food ingredients, anxiety-related behavior problems and even cancer.

Cognitive Dissonance: Another factor that may contribute to the caregiver placebo effect is finding oneself in a state of contradiction. When we invest time and money (and hope) into a new treatment for our dogs, it follows that we will naturally have high expectations that the treatment will work. Indeed if it does not, we may experience cognitive dissonance, the uncomfortable feeling caused by holding two contradicting beliefs in one’s mind at the same time. For example, “I was told that giving my dog dehydrated gooseberry rinds would cure his chronic itching; these rinds are expensive and hard to find. He does not seem any better…… Uh oh. This is not a good feeling….”

Psychologists tell us that our brain reduces this discomfort for us (without our conscious awareness, by the way) by simply changing our perceptions. “Oh look! I am sure that the dehydrated gooseberry rinds ARE finally working. It just too some time – several months in fact. Still the effect MUST be the gooseberry rinds. YAY!”  In this case, convincing oneself that the dog does seem a bit less itchy, her coat is a bit healthier and overall, she does really seem to be feeling better, immediately solves this problem for the brain and for our comfort level.

cognitive-dissonance

The Hawthorne Effect: Finally, a related phenomenon that is common enough to have earned its own name is the Hawthorne Effect, also called observation bias. This is the tendency to change one’s behavior (or in our case how one reports their dog’s behavior) simply as a result of being observed. The Hawthorne Effect suggests that people whose dogs are enrolled in an experimental trial may behave differently with the dog because they know they are enrolled in a trial that is measuring many aspects of the dog’s life. In the case of the arthritis studies, owners may have altered how regularly they exercised their dogs, avoided behaviors that worsened the dog’s arthritic pain, or began to pay more attention to the dog’s diet and weight.

The point is that when people are enrolled in a research trial or are starting a new medical treatment, diet, or training program and are being monitored, they will be inclined to change other aspects of how they live with and care for the dog as well. These changes could be as important (or more important) than the actual treatment (or placebo). This is not necessarily a bad thing, mind you, and is another reason why we always need control groups, but the occurrence of the Hawthorne Effect emphasizes the importance of recognizing that the thing that we think is working for our dog may not actually be the thing that is doing the trick.

Take Away for Dog Folks: When trying something new with our dogs, might we, at least some of the time, in some situations, be inclined to see improvement when it does not truly exist? When interpreting our dog’s response to a novel therapy or supplement or training technique are we susceptible to falling for the sugar pill? It seems probable, given the science. It is reasonable to at least consider the possibility that a placebo effect may be influencing our perceptions of our dog’s response to a new food, a new supplement, a new training technique or a novel treatment. This is especially true if the approach that we are trying has not been thoroughly vetted by research through double-blind, placebo-controlled trials. While the development of new medications and foods and training methods is exciting and important, we must avoid the tendency to see improvement from something that is novel simply because we expect and desire it to be so.

CITED STUDIES:

  1. Conzemium MG, Evans RB. Caregiver placebo effect of dogs with lameness from osteoarthritis. Journal of the American Veterinary Medical Association 2012; 241:1314-1319.
  2. Munana KR, Zhang D, Patterson EE. Placebo effect in canine epilepsy trials. Journal of Veterinary Internal Medicine 2010; 24:166-170.
  3. Jaeger GT, Larsen S, Moe L. Stratification, blinding and placebo effect in a randomized, double blind placebo-controlled clinical trial of gold bead implantation in dogs with hip dysplasia. Acta Veterinaria Scandinavica 2005; 46:57-68.

Excerpted from: “Beware the Straw Man: The Science Dog Explores Dog Training Fact and Fiction” (2015).

Beware Straw Man Cover

Excitable You

There is a common cognitive bias, the Fundamental Attribution Error,  that is central to the way in which we view others and make judgements about their behavior. It is supported by a large body of research and is one of the most common errors that our brains make on a regular basis. The Fundamental Attribution Error refers to our tendency to explain the behavior of other people in terms of their internal disposition, such as personality traits, innate abilities, and motives, rather than to the external (situational) factors that may actually be exerting a much stronger influence on them. This lapse in judgement occurs (especially in Western cultures) because we tend to assign high value to what we assume to be an individual’s character and personality traits, while at the same time we underestimate the influence that situational factors and context can have.

Fundamental-Attribution-Error

THE FUNDAMENTAL ATTRIBUTION ERROR

We all are susceptible to committing this error and it is usually only through conscious control that we can keep it in check. A common example occurs when we are driving and someone cuts us off in traffic. We immediately label the offending driver as “a jerk” (or worse) rather than consider that he might be driving to the hospital (or with his dog to the veterinarian) on an emergency and would not normally behave so rudely towards other drivers. This is not to say that unpleasant people do not exist, but rather, that humans have a natural tendency to jump to depositional (personality) explanations for another’s behavior and are less inclined to consider situational explanations.

FAE Homer

The Fundamental Attribution Error came to mind recently when I was reading a paper that examined dog owners’ reports about their dogs’ behavior, specifically about excitable behavior. The study was conducted at the Animal Rescue League of Boston and the Center for Shelter Dogs and was published in the journal, Animals (1).

The Study: The authors note in the introduction that the term “excitable behavior” in dogs is both poorly defined and under-studied. They then provide a diverse list of undesirable behaviors that have been reported to  fall under the umbrella of excitable dog behavior. These include jumping up, mounting, destructiveness, mouthing, grabbing clothing, digging, some forms of barking, rough play, pulling on lead, and (my particular favorite) “dogs who respond poorly to commands and are difficult to control”.  Study objective: The purpose of the study was to use an on-line survey to collect information regarding owners’ experiences with their dog’s excitable behavior and to report the behaviors that are prevalent in excitable dogs. Methods: The study group was self-selecting. Participants checked a box in the survey that asked if their dog was “highly excitable or highly energetic”. Only those owners who answered “yes” were included in the study; owners who answered “no” were excluded. The remainder of the questionnaire included questions about the dog’s demographics and problematic behaviors, and the degree of frustration that the owner had with those behaviors. Results: The study group included 175 owners, the majority of whom said that they were very frustrated with their dog’s behavior and found it difficult to manage. Most of the dogs were spayed/neutered and were young adults (average age; ~ 3 years). Almost half of the dogs (44 %) were identified as either purebred Labrador Retrievers or Lab mixes. The two most frequently reported problematic behaviors were jumping up and mouthing (without discomfort to the person). Other commonly reported undesirable behaviors included general disobedience, unwanted barking, pulling on the leash, destructive behavior and “not listening to commands”. The scenarios in which excitable behaviors were most likely to occur included when the owner arrived home after an absence and when the owner was playing with the dog. Some owners also reported excessive excitement when the dog was meeting new people. Conclusions: The authors concluded that “The majority of owners in this self-selected sample were very frustrated with their excitable dog”, that “Many of the dogs in the sample had other behavior problems”, and that their results could be used to “…..provide better education to owners of excitable dogs(Emphasis mine).

Hmmm……..Yes, in case you were wondering, I do have an opinion about this.

soapbox

I THINK I’M GONNA NEED A BIGGER BOX

There are several problems with this study, in terms of both its methodology and the conclusions that were made. Let’s start with that pesky thing called the Scientific Method, which requires the use of both a representative sample and sufficient controls to prevent bias and capricious conclusions.

Sampling bias: In the authors’ words “The focus of this study is on owners’ experience with their excitable dogs.” Therefore, it must have seemed logical to them (i.e. it felt like a good idea at the time) to simply ask owners to tell them if their dog was one of those (poorly defined) excitable dogs. By this logic, an excitable dog is a dog who is excitable (according to their owner). Circular reasoning does not a representative sample make. And here’s a big surprise; the owners who identified their dog as “highly or extremely excitable” were also very frustrated with their dog’s behavior. Wow. Who knew?

Control Group

Absence of controls: At the start of the survey, owners were asked if they would describe their dog as “highly excitable or highly energetic”. Only those who answered in the affirmative were included in the study. Owners who answered “no” were not allowed to complete the survey (i.e. a possible control group of dogs was purposefully excluded). The authors went on to report that excitable dogs are likely to show problematic behaviors of jumping up and mouthing, along with a myriad of other associated problem behaviors. However, without a control group to compared the frequencies of these behaviors to, what do we actually learn from these data?

Absolutely nothing

ABSOLUTELY NOTHIN’

Here’s why: Let’s say that a control group was used (i.e. correct scientific methods were followed). So, hypothetically, let say that the control group included a similar number of age-, sex- and breed-matched dogs who were representative of the general population of dogs. Their owners completed the same survey and answered the same questions. The reported frequencies of problematic behaviors in the experimental group (dogs identified as excitable) were then compared with the frequencies of the same behaviors in the control group. Here are some possible outcomes of this hypothetical study:

  • Jumping up: In the actual study, 60 percent of owners of excitable dogs said that their dog jumped up to greet when they returned home after an absence. If (hypothetically) a similar proportion of owners in the control group, let’s say 62 % for reason of argument, stated that their dog jumped on them when they returned home, then the proportion of jumping up in excitable dogs did not differ from the proportion of that problem in the general population of dogs. And, if jumping up was not over-represented in the excitable dog group, then jumping up is NOT a problem associated with excitable dogs. (Rather, it is just something that dogs do).
  • Pulling on leash, destructive behaviors, not listening to commands: You see where this is going. The plethora of unwanted dog behaviors that the study participants vented about in their surveys cannot viewed as indicative of an excitable dog because the frequencies of these behaviors were never compared to their frequencies in other dogs. This problem is exacerbated by the fact that the owners placed their dogs into the self-described category of excitable dog in the first place. Lots of dogs pull on lead, bark and do not listen. All that we learned here is that owners like to complain about these behaviors and welcome the opportunity to label their dog as “excitable”.

no control group

Wait, there’s more.

The Fundamental Attribution Error: The authors state: “In general, disobedient, destructive, chasing and barking behavior problems were the most commonly reported behaviors by owners of excitable dogs“.  Excluding the occasional dog who cheats on his income taxes or robs the town bank, I think that this list of unwanted behaviors pretty much covers everything that owners complain about in young, untrained dogs. (What are the “non-excitable dogs’ doing to annoy their owners, one might ask)? While this sounds facetious, I actually am serious. If the purpose of this study was to allow a group of self-identifying owners of excitable dogs to air their (numerous) complaints about their dogs and to give their perceptions a voice, then by definition, the authors are assuming that excitable dogs differ in some fundamental way from other dogs. I would argue that they have no evidence of such a thing and moreover that classifying certain dogs as excitable is ill-founded and not in the best interest of any dogs, regardless of the researchers’ noble intentions.

Encouraging dog owners (and dog professionals) to commit a fundamental attribution error by labeling dogs as inherently “excitable” provides tacit permission to blame the dog’s personality or intrinsic nature for undesirable behaviors, rather than looking carefully at situational factors that may be influencing the dog. The outcome of such perceptual differences could be devastating:

FUNDAMENTAL ATTRIBUTION EXPLANATIONS (THE EXCITABLE DOG):

  • My dog must have been born this way. (Solution: none)
  • He was abused/abandoned/neglected by his previous owner and it made him hyperactive. (Solution: none)
  • He’s a Lab, Lab-mix, Pittie (*Insert any breed stereotype here) (Solution: none)
  • She’s a hyper-active dog. (Solution: none)
  • He’s an excitable dog. (Solution: none)
  • She’s a bad dog. (Solution: Get rid of the dog).

This mindset leads an owner to the conclusion that their dog’s behavior is immutable and that their own degree of responsibility is minimal or nonexistent. Alternatively, where do situational explanations lead us?

SITUATIONAL EXPLANATIONS (UNWANTED EXCITED BEHAVIORS):

  • He is rarely exposed to new people, places, and dogs. (Solution: I need to socialize him and take him with me more often).
  • She does not receive regular exercise. (Solution: I need to incorporate several types of daily exercise into our routines).
  • He has not had consistent training (Solution: I will enroll him in a training class).
  • She is crated and left alone for many hours of the day. (Solution: I will hire a dog-walker or use a reputable doggy day care).
  • I may have unrealistic expectations for my young dog’s behavior. (Solution: I will ratchet down my expectations so that they are more in line with what is reasonable to expect of a young, happy and exuberant dog. I will love my dog).

Let’s avoid making the fundamental attribution error with our dogs. Because we have complete control over what happens to them, the outcome can be much worse than simply calling someone a jerk.

Nuff said. Off box.

Cited Study: Shabelansky A, Dowling-Guyer S. Characteristics of excitable dog behavior based on owners’ report from a self-selected study. Animals 2016; 6, 22; doi10.3390/ani6030022.

Excerpted from: “Only Have Eyes for Your: Exploring Canine Research with the Science Dog” (2016).

coversnip

 

 

 

What’s Your Dinner Ritual?

The Case dogs have an evening dinner ritual. This ritual has not changed much in the last few years and deviates very little in its nightly performance. It begins, like clockwork, at 8:15 pm and is currently directed by Cadie, our senior Golden girl. Mike (my husband) typically feeds the dogs their evening meal, so is her usual target. As the self-appointed “dinner getter” Cadie takes her responsibilities very seriously. She is in charge of checking the time (apparently every 15 seconds after 7:00 p.m.), of carefully tracking potential human movement towards the utility room where the dog food resides, and of counting dogs to determine when everyone is in the house and ready to eat.

Chip Cadie Vinny Cooper Dec 2012CADIE AND HER BOYS

When all key factors are in place, Cadie declares “Game On!” and the ritual begins in earnest. First comes the unrelenting stare; laser-beam eyes capable of burning holes through flesh. Cadie’s style is impressive; she sits rock solid still, barely breathing, eyes fixed on Mike’s face. If the stares do not elicit the desired response (dinner), she gradually inches closer until she is perched on the couch, Snoopy vulture-style, her cute little snout hovers inches above Mike’s face. If there is still no food-related movement, she adds the woofing; persistent little barks timed at two-second intervals for maximum annoyance. The paw on the arm is added last and occasionally Cadie feigns a dramatic hunger-induced swoon. (Okay, I made that last part up, but it really seems like something she would try). Finally, if all else has failed and it looks like dinner may not be forthcoming, Cadie enlists her second-in-command, Vinny the Brittany, to help.

With two dogs hovering with pleading eyes, Mike finally gets to his feet and walks towards the utility room. An explosion of happiness erupts! It is time for a DOG PARTY!!! Four dogs, all running, spinning, barking, more spinning, joyous, joyous Dinner Time, Dinner Time – A time for celebration! As Mike measures food into bowls, he sings the Case Family Dinner Time song (Who wants dinner? Who wants dinner? Everybody does! Everybody does! ). All four dogs crowd around for the measuring into bowls; Cadie keeping a keen eye on portions. Once the food is doled out, sitting is required prior to eating and all of the dogs adhere to the single hard and fast dinner rule – eat only from your own bowl. When everyone has finished their meal, the dogs are then allowed to play musical bowls, each thoroughly inspecting and licking every bowl. Finally, dinner complete, everyone goes outside for a potty break, knowing that tomorrow will be another day, complete with a new joyous opportunity for food and the celebratory dinnertime ritual.

Malamutes at dinnerSO, WHAT IS YOUR DINNER TIME RITUAL?

Do you have a dinnertime ritual with your dog? Does your dog have very specific and endearing “dinner-getting” behaviors? Do you have a particular way of responding to these? And, tell the truth now…….do you have a dinnertime song?

Food  is love (emotional brain): When we think about our daily lives with our dogs, we consider many shared enjoyments. And with our dogs, just as with our human family and friends, dinner time is not only about nutrition and food – it is just as much about joy and affection, and ritual. Indeed, there is perhaps no other aspect of our lives with dogs in which we show love more consistently than with the decisions that we make about what, how, and when we feed them. And it is exactly these feelings that cause choosing the best possible food or method of feeding for our dogs to weigh so heavily upon our minds.

And, it is science (rational brain): It is a fact that nutrition is a science that is governed by the same scientific principles and methods as all of the hard sciences such as biology, chemistry and physics. However, for most of us, applying the principles of sound nutrition to our dogs’ daily lives does not feel like science. Rather feeding our dogs feels like love, and caretaking, and nurturing.  And indeed, providing good nutritional care should feel good. Without question, the deep love and commitment that we have for our dogs is essential for caring for them well.

Using both: Still, we need evidence, scientifically acquired evidence, to make informed decisions about nutritious foods and healthful feeding practices for dogs. Critical thinking skills enable us to sort out reliable evidence from information that is based upon conjecture, anecdote, and belief. The good news is that emotions and rational thought are not mutually exclusive, and in fact can play quite nicely together in your decision-making brain. Loving our dogs and wanting the best for them (our emotional mind) plus a set of well-honed critical thinking skills (our rational mind) can work together quite efficiently to help us to make wise food choices. While emotions are essential for decision-making and can influence us in many positive ways, we must also be aware of (and avoid) the cognitive traps that emotions can lead to and that clever pet food marketing campaigns often rely upon. Once you have these skills in place, the resulting smart food choices (coupled with a really cool dinnertime song) can help you to enjoy your dinnertime rituals with your dogs for many years to come.

This essay is excerpted from Chapter 1 of my new book “Dog Food Logic“. If you enjoyed this piece would like to read more, the book is available from the publisher (Dogwise) and on Amazon:

 Amazon Cover                                            Amazon Cover              AMAZON                                                                    DOGWISE

What the Dog’s Nose Knows

In “Thyroid on Trial” we used an evidence pyramid to examine the various types of scientific studies that have been used to examine the hypothesis that there is a connection between hypothyroidism and aggressive behavior in dogs. We followed this example up through most of the levels of the pyramid, finishing at the penultimate level, the randomized, controlled trial.

At the tippy top of the evidence pyramid is the systematic review. As I mentioned in that piece, systematic reviews are somewhat of a rarity  in canine science because they entail an enormous amount of work and time and because such reviews are only possible once there is a robust body of published information about a topic. So, I was recently quite delighted to find a newly published systematic review of a topic that many dog folks have an interest in – Canine Scenting Ability.

Big NoseWHAT DOES THE NOSE KNOW?

It is generally accepted that dogs possess a phenomenal sense of smell. These talents are regularly demonstrated in the many ways that dogs work and play at various scenting tasks. Dogs are trained to find illicit drugs, explosives, and land mines for the police and military, to identify the presence of mold, insects and toxins in homes and public buildings, and in recent years, to diagnose the presence of cancerous tumors or changes in blood glucose levels in human patients. On the recreation side of things, anyone who enjoys tracking or canine nose work with their dog knows that these canine sports are not only interesting and enjoyable for owners, but are great fun for the dogs as well.

tracking                   Canine Nose Work                            TRACKING                                                            NOSE WORK

Evidence in support of the dog’s keen scenting ability comes from many sources, each of which is associated with varying degrees of reliability. Information ranges from anecdotes (“Fluffy detected Aunt Marge’s diabetes and  saved her life!”), to media reports of heroic finds and rescues, to controlled scientific studies. The diversity of the dog’s olfactory talents is also expressed by the wide range of items that dogs can be trained to locate and identify.

The Systematic Review: Knowing this, a group of researchers at Freie University’s College of Veterinary Medicine in Berlin, Germany conducted a systematic review of all of the published research regarding the domestic dog’s ability to successfully detect selected odors. They published their results in the peer-reviewed journal, Applied Animal Behaviour Science in late 2013. Here is what they found:

  • Papers: A group of 31 papers was initially identified. Of these, 2 were excluded for duplication and 14 were excluded because they did not meet the criteria for a systematic review (i.e. they did not include original research). This left 14 published studies for the review.
  • Odor substrates: Biological substances were used in all of the studies. These included termites/ants (2), cancers (7), tree snakes (1), microbial growth (1), parasitic infection (1), bird/bat carcasses (1), and estrus in cows (1).
  • Training methods: All of the studies used reward-based training methods. Five of the papers specifically mentioned that clicker training was used. One paper reported that using positive reinforcement to train dogs resulted in greater sensitivity to the test compound than that observed in an earlier study that had used aversive stimuli to train dogs to detect the same compound.
  • Duration of training : Training period durations ranged from just a week to more than a year. While the authors assumed that more difficult tasks or training dogs with no previous training may have resulted in the more extended training periods, this was difficult to ascertain given the small number of studies and the limited information that was provided.
  •  Trainer’s experience/skills: Most of the studies did not report the trainers’ qualifications or experience, making it impossible to evaluate the impact of training skills upon success. Because a trainer’s qualifications is expected to impact training duration and success, the authors note that this is an area in which additional study is needed.

Success Rates: Sensitivity and specificity were calculated in 6 of the 14 studies (see “This test you keep using” for an explanation of these two measures) and response rate (number of correct finds) was reported in 8 of the 14 studies:

  • Sensitivity: Dogs’ sensitivity (ability to correctly identify positive samples) ranged between 88 % and 100 %.
  • Specificity: Dogs’ specificity (ability to correctly identify negative samples) ranged between 91 % and 99 %.
  • Number of Correct Indications:  Dogs correctly found or indicated a positive sample 35 % to 98 % of the  time.
  • These results suggest that dogs are capable of successfully detecting and indicating a variety of biological substances, ranging from different forms of cancer, to insect or reptile infestations, to carcasses and biological states.

Study Limitations: All of the published studies had one or more limitations. While these do not invalidate the positive results, they do suggest the need for future research that is well controlled, blinded, and which includes multiple samples.

  • Blinding: Only 6 of the 14 studies used methodologies that blinded handlers and researchers to sample type and position. Lack of blinding may cause handlers or researchers to inadvertently influence or interpret a dog’s behavior  (i.e. Clever Hans Effect).
  • Sample replicates: Only 3 of the 14 studies reported using more than a single positive sample (i.e. replicate) during testing. A lack of replication prevents the measurement of sampling error, which is an important criterion when attempting to quantify successful odor detection. In addition, while 8 studies reported using different samples during training and testing, the remaining six did not include information about sample reuse so it is not known if different samples were used during training and testing in those studies.
  • Consistency: Only 4 of the 14 studies met the most important evidence-based science criteria of being double-blinded, measuring test sensitivity and specificity, and using unique samples during testing. However, these 4 studies varied in methodologies, types of detection,  and training durations of the dogs, making it difficult to compare results.

Take Away for Dog Folks: This systematic review shows that dogs can be successfully trained to detect a wide range of biological substances and that training methods that focus on positive reinforcement are the approach of choice for this type of training. In addition to providing a thorough review of a topic, systematic reviews also function to identify gaps in the research that may require additional study. In this case, further exploration of the specific methods and durations used to train scent detection work in dogs along with careful blinding to prevent handler signaling or sample contamination appear to be needed. Personally, I found it to be very exciting that 31 papers were found on the topic of scent detection work in dogs, 14 of which met all of the (rather strict) criteria for a veterinary medicine evidence-based review, and that of these 14, the majority reported successful or highly successful scenting responses in the dogs.

What the dog’s nose knows seems to be quite a bit indeed! 

smell birthday cake

Reference: Johnen D, Heuwieser W, Fischer-Tenhagen C. Canine scent detection – Fact or fiction? Applied Animal Behaviour Science 2013; 148:201-208

Thyroid on Trial

Every day, we are bombarded with new information about dogs that arises from a variety of sources – via the internet, through our smart (or not so smart) phones, from our colleagues, friends and family, and of course from our neighbor Joe next door (who happens to know a lot about dogs). In this day and age of information overload, it should come as no surprise that deciding which information is trustworthy and which to view with a healthy dose of skepticism is increasingly difficult. Lucky for dog folks everywhere, science comes to the rescue once again.

science to the rescue

Even within science however, all evidence is not created equal. Though not quite as dramatic as walking on water, the construction of a handy “evidence pyramid” helps us to sort the various categories of scientific study and to rate the types of information that they provide.

Evidence-Pyramid

Interpreting the Pyramid: First, keep in mind that this graphic presents only scientific evidence and does not include the host of other types of information that we come across each day, such as anecdotes, testimonials, stories/experiences, and (non-expert) opinion. (For a review of all of these types of evidence, see my new book, Dog Food Logic).  As you move from the bottom to the top of the pyramid, the amount of information (published literature) decreases, but its relevance and reliability generally increases. Starting at the bottom:

  • Editorial and expert opinions: Experts in a field will often produce textbooks and review papers that can provide a good foundation about the chosen topic. This information presents a helpful summary, but because it is not reporting results of a scientific study, cannot provide evidence that supports or refutes a new scientific hypothesis.
  • Case reports: These are individual reports, usually published by a practicing veterinarian, of one or more dogs with a condition who are found to respond to a particular treatment. People are often surprised that case reports are not regarded as stronger scientific evidence. However, while case reports may generate new hypotheses, they cannot be used as strong support for an existing hypothesis because of their anecdotal nature and the lack of control groups (remember our Steve Series).
  • Case-controlled and cohort studies: Case controlled studies occur when the researcher finds cases (dogs, in our example) that have the condition under question and then matches and compares those cases with other dogs who are similar, but lack the condition.  Similarly, cohort studies compare large groups of dogs with or without a condition, over time. While providing a type of control, results of these studies are limited because showing a statistical relationship (usually correlation) between two groups does not mean than one factor necessarily caused the other.
  • Randomized controlled trials (RCT): This type of study is the most important and reliable source of scientific research. It includes methodologies that use the scientific method, reduce the potential for bias (via randomization, blinding, and the use of placebos) and allows for comparison between intervention groups and carefully selected control groups.
  • Systematic Reviews:  At the pinnacle of scientific evidence, this is is a specific type of review in which experts in a field assess all of the relevant studies and the data (called a meta-analysis) that address a particular topic.  Systematic reviews require enormous commitments of effort, time, and money and are only possible once a hypothesis has been studied in  depth. Therefore, these studies are few in number, especially for many topics of that are important to dogs. (For a good source of a few systematic reviews in canine health see Best Bets for Vets).

AN EXAMPLE – THYROID ON TRIAL

Thyroid Gland

An example: I recently came across a great example of a hypothesis about canine health and behavior that progressed from a few initial case reports, through case-controlled studies and culminated recently with the gold standard – the completion of a randomized, controlled trial. The issue had to do with a common endocrine disorder in dogs, hypothyroidism, and its potential relationship with aggressive behavior.

Background information: Thyroid hormone is produced by the thyroid gland. The active form of this hormone regulates cellular metabolism and so has effects in virtually all body systems. The condition of hypothyroidism refers to a reduction in thyroid hormone production and resulting clinical signs.  Hypothyroidism generally develops in middle-aged or older dogs and certain breeds show a genetic predisposition. Documented clinical signs of hypothyroidism include lethargy, decreased interest in exercise, weight gain, changes to coat quality and hair loss, and skin problems such as seborrhea, hyperpigmentation and secondary bacterial infections.

anxiety              defensive threat                       Is there a relationship between hypothyroidism and aggression in dogs?

Hypothesis: In recent years, it has been speculated that certain types of aggressive behaviors in dogs may be related to suboptimal or low thyroid hormone levels. Starting with case reports, this hypothesis has gradually worked its way up the evidence pyramid:

  • Case reports:  Two case reports were published in 2002 and 2003. Together, they involved a total of 5 dogs with owner-directed aggression, who were subsequently also diagnosed with hypothyroidism. The dogs responded to thyroid hormone replacement therapy with a reduction in aggressive episodes, leading to the hypothesis that some cases of aggression in dogs may be associated with hypothyroidism (1,2).
  • Case-controlled studies: A case-controlled report was conducted a few years later (3). Records of over 1500 dogs were reviewed. Of these dogs, 61 per cent were classified as either hypothyroid or with suboptimal thyroid function. A statistically significant correlation was found between thyroid dysfunction and dog-to-human aggression in this group of dogs (p < 0.001). However, two subsequent case-controlled studies failed to find a connection between thyroid hormone levels and behavior problems (3,4). Conflicting results – jury still out.
  • The RCT: Most recently, in 2013, the connection between suboptimal thyroid hormone levels and aggression in dogs was examined using the Gold Standard of designs – a double-blind, randomized, placebo-controlled study (5).  Dr. Nick Dodman and his colleagues at the Tufts Cummings School of Veterinary Medicine enrolled a group of 40 dogs, all of whom were exhibiting owner-directed aggression and were also diagnosed with suboptimal or low thyroid hormone levels. Following screening and a 2-week pre-treatment (baseline) period, the dogs were randomly assigned to either the treatment group (thyroxine replacement therapy) or to the control group (placebo). Neither the researchers nor the owners knew which group each dog was assigned to. Dogs were medicated twice daily for a period of 6 weeks. During both the baseline period and throughout the study period, owners recorded the number and type of aggressive episodes that their dog exhibited.

results

RESULTS: 

  • Enrollment and attrition: The highly specific inclusion criteria for the dogs in this study (aggressive dogs that had borderline or low thyroid function) coupled with difficulties associated with working with dogs with aggression problems led to a relatively small initial sample size (n = 40). In addition, attrition was high, due to owner non-compliance or other problems.
  • Frequency of aggressive behavior within groups: The frequency of aggressive episodes significantly decreased in both groups from baseline levels over the six-week experimental period. The change in the control group demonstrates a significant placebo effect in this study.
  • Treated group vs. placebo group: Owner-measured aggression did not differ between the treated group and the placebo group during the first five weeks of the study period. During the final week of the study (week six), dogs who were treated with thyroxine and had normalized serum thyroxine levels showed slightly lower frequencies of aggression when compared with dogs who were receiving the placebo, but this difference was not statistically significant  (P = 0.08).

Take away for dog folks:  This type of design is truly the “gold standard” of experimental studies for several reasons:

  • Methodology: An RCT design reduces bias and allows relevant comparisons because a matched control group is used, treatments are randomly assigned, and neither the experimenters nor the subjects (in this case the owners of the dogs) know which treatment each dog is receiving. In addition, the inclusion of a placebo treatment (as opposed to simply not treating the control group at all) allowed the researchers to measure and account for a placebo effect (which clearly was important in this study).
  • Clinical trials using dogs in homes: This study was a clinical trial, meaning that it was conducted with owned dogs living at home with their owners. This differs from studies conducted with dogs living in kennels (typically at a university setting or at a pet food company’s kennel). If you remember back to The Steve Series, a cornerstone of the scientific method is selecting a study sample that is representative of the population that you will make conclusions about. Therefore, while we can control many of the “variables of life” with kenneled dog studies (making those studies much easier to conduct and to detect differences) , such a sample is by definition, less representative of the population of dogs than is a sample that includes dogs living at home.

Challenges: In-home clinical trials with dogs are wrought with enormous challenges, all of which make it difficult to demonstrate real effects when they exist and which can require larger sample sizes to detect any true differences. These include:

  • Variations in daily life: Every owner lives with his/her dog in ways that are idiosyncratic to that person’s demographics, lifestyle, and values. These differences all impact an owner’s perceptions of his dog’s behavior (in this example, displays of aggression) as well as a tendency to show a placebo effect (see below).
  •  Dog differences: Generally speaking, the differences among pet dogs enrolled in a clinical trial are going to be greater (spread more widely around the mean) than those among a group of kenneled dogs. The most obvious difference is the variability in living situations and daily routines among households. These are not present when studying a group of dogs who are housed under the same conditions and experience the same daily routines. For a researcher, this means that being able to identify a treatment effect (in this case, a measurable reduction in aggressive episodes in dogs treated with thyroid hormone), is much more difficult when studying dogs in homes compared with studying dogs in kennels.
  • Owner perceptions and compliance: When the owner is the data collector  in a study (which is sometimes the only feasible approach with in-home studies), there will be error (variability) introduced by the different perceptions among owners as well as by varying levels of compliance. Extreme non-compliance usually leads to removal from the study, but this too is a problem since removing subjects from an already limited sample will further reduce the power of the experiment (i.e. the ability to detect a true difference when it exists).

placebo

  • Placebo effects: Just as in studies with human subjects, the placebo effect is a real effect that must be accounted for in dog studies. When owners are aware that their dog is enrolled in an experimental trial, even though they are blinded to the treatment that their dog is receiving, the mere participation in the study will affect their perspective of their dog’s behavior and their judgement of possible effects or side effects of the treatment (that their dog may or may not be receiving). Including a placebo control group in a study that includes subjective measures of behavior (such as measuring the number and intensity of an aggressive response) is even more important since subjective scales are generally less reliable than objective measures.

Bottom line? The RCT that examined the effects of thyroid hormone replacement therapy on borderline or frankly hypothyroid dogs with owner-directed aggression showed a slight numerical reduction in aggression that was not statistically significant. As a result, the researchers concluded that thyroid replacement therapy could not be wholeheartedly recommended as a treatment for aggression in hypothyroid dogs and that additional studies of this type may be helpful to further examine this potential connection.

Personally, I think that this is also an excellent example of the progression of science from a set of initial case reports, followed by case-controlled studies, culminating in a randomized, controlled, clinical trial. An examination of the final  study illustrates the enormous commitment of labor, time, and money that is required when conducting clinical trials as well as the importance of including placebos and double-blinding in scientific studies. Kudos to the investigators – not only for conducting what was clearly a very challenging clinical trial, but also for reporting informative negative results in a peer-reviewed journal.

Control Group

References:

  1. Fatjó J, Stub C, Manteca X. Four cases of aggression and hypothyroidism in dogs. Veterinary Record 2002;151: 547-548.
  2. Dodds WJ, Aronson LP. Behavioral Changes Associated with Thyroid Dysfunction in Dogs. Proceedings 1999 American Holistic Veterinary Medical Association Annual Conference, pp. 80-82.
  3. Carter GC, Scott-Moncrieff JC, Luescher AU, Moore G. Serum total thyroxine and thyroid stimulating hormone concentrations in dogs with behavior problems. Journal of Veterinary Behavior 2009; 4:230-236.
  4. Radosta LA, Shofer FS, Reisner IF. Comparison of thyroid analytes in dogs aggressive to familiar people and in non-aggressive dogs. Veterinary Journal 2011;192:472-475.
  5. Dodman NH, Aronson L, Cottam N, Dodds JW. The effect of thyroid replacement in dogs with suboptimal thyroid function on owner-directed aggression: A randomized, double-blind, placebo-controlled clinical trial. Journal of Veterinary Behavior 2013;8:225-230.

 

Your face is gonna freeze like that (Part 4)…….(a.k.a. Why we need multiple Steves)

So, here we are, with three study groups described in Part 3 of this series. Each group consists of 25 young adult dogs, representing a range of breeds and breed-types. This collection of dogs is considered a sample of the population that we are testing. In this example, we identify the population as all young adult dogs living in homes.

Many dogs

A FEW DOGS FROM OUR SAMPLE

It’s all about variance: Let’s start by all agreeing that dogs vary. They come in different sizes, breeds, personalities, with different past experiences and of course with differing home lives and owners. Individuals also vary in their rate of learning, interest in different games and tricks, and motivation and talent for different dog sports – agreed? It is exactly these differences that require the use of both a representative sample of dogs and statistical analysis of results.

Let’s illustrate this concept with….dogs, of course: For any given measure (for example, height at the withers, coat length, degree of territorial behavior, or as in our example, tolerance of handling) the ways that individual dogs vary is measured with a statistic called (aptly) variance. Variance is then used to calculate a numerical measure called a standard deviation. Standard deviations (SD) and their close cousin, standard errors, provide an estimate of how much variability there is within and between groups of dogs around the mean (average) of a given measurement. The SD is affected by a number of things, one of which is the type of sample that we choose to use in an experiment. Let’s say we chose two sets of samples for our study. The first (below left) comes from ALL dog breeds (and mixes); let’s call this the Chihuahuas to Great Danes Sample. The second (below right) comes only from dogs who are members of the herding or sporting groups and who range in size from an average Border Collie to an average Golden Retriever. Below each sample photo is a Bell-shaped curve that represents the expected variance among individual dogs within each of the samples.

toy-chihuahua-and-great-dane             border_collie_and_golden_retri~AP-UK5TBN-TH

  CHIHUAHUA TO GREAT DANE                         BORDER COLLIE TO GOLDEN RETRIEVER

            (LARGE SD)                                                                          (SMALL SD)

High Standard Dev                       Low Standard Dev

CHIHUAHUAS TO GREAT DANES: When we study a sample of dogs that is representative of the entire dog population, the sample is expected to have relatively high variance (lots of spread around the mean), which represents all of the naturally occurring differences among dogs of that population. In our example, the study of priming, this sample of dogs is expected to vary widely in terms of how well or poorly they accept having their feet, ears and mouths handled because it includes dogs with a wide range of temperaments, learning ability, and innate handling tolerance. The plus side of this type of sample is that we can make conclusions from it that applies to all dogs (not just the Steves of the world)! So, is there a down side? Well, the greater the variance, the more difficult it is to detect true differences caused by our treatment (in this case, priming) when it exists. Think of it as having a bunch of noise in the background that interferes with our ability to “hear” (measure) the effects of a treatment or intervention.  This variability, while accounted for using statistics, can make it relatively difficult for us to detect a priming effect.

BCs TO GOLDENs: We could of course select a less “noisy” sample. If our sample was restricted to include just BCs to Goldens, the variability in responses would be smaller as there are fewer naturally occurring differences among individuals in this group. We would have an easier time detecting (statistically speaking) an effect of priming because the “noise level” of naturally occurring variance is lower.  The downside here – I am sure you are ahead of me on this one – BCs and Goldens, beloved though they are, are not representative of all dogs. We would be limited in the conclusions that we could (should) make from a study that used this sample. Unfortunately, because this is real life, we do see studies in which samples are unrepresentative of the population that they are intending to study…..more about this in a later blog…

RESULTS: For now – let’s get back to our hypothetical study – What might we learn about the effects of priming from our Study of Multiple Steves?

The three study groups and the 3-week protocol were described in Part 3. Here is a table showing the mean (average) weekly scores for the two control groups and the treatment group for just the touching feet portion of the experiment. The + after each mean is the standard deviation and as you now understand, this represents the “spread” of individual dogs’ scores  around the mean within each group. Below the table is a line chart showing the change in “foot handling score” for the three groups during the study period.

GROUP

MEAN SCORES   (WEEKLY) + S.D.

PRE-TEST

WEEK 1

WEEK 2

WEEK 3

Negative Control (No training)

1.8 ± 0.34

1.7 ± 0.34

2.0 ± 0.34

2.2 ± 0.34

Positive Control (touch-treat only)

1.6 ± 0.31

3.0 ± 0.31

3.6 ± 0.31

4.0 ± 0.31

Test Group (Priming and touch-treat)

1.2 ± 0.36

3.4 ± 0.36

4.9 ± 0.36

4.9 ± 0.36

Steves Graph 2

RESULTS: What do the numbers in the table (mean values + SD ) and the line chart tell us? Well nothing, actually, without doing a further statistical test. We can use a statistics procedure called repeated measures ANOVA with this type of study design; a test that allows us to account for measuring “feet touch tolerance” in the same dogs on multiple occasions AND also compares the three groups to each other (pretty cool!).

The results of this statistical test tell us this:

1. Dogs with no training (negative control group; blue line) did not show significant improvement in feet handling scores over the study period. Notice the relatively flat line going from the pre-test at week 0 to three weeks.

2. Dogs who were trained with either touch-treat alone (positive control; greenline) or with priming and touch-treat (test group; red line) showed significant improvements in their feet handling tolerance over the three-week study period.

3. Dogs who were trained with touch-treat alone (green line) had significantly higher feet handling scores than dogs who had no training (blue line). Notice the distance between the two lines and the fact that the blue line has a positive slope, indicating improvement in scores.

4. Dogs who were trained with priming plus touch-treat (test group; red line) had significantly higher (better) feet handling scores than dogs  who had no training (blue line) and than dogs who were trained with touch-treat training alone (green line). Notice the gaps between the lines and the steepest slope in the red line.

WHAT CAN WE CONCLUDE?  We conclude that this study found that including priming prior to touch-treat training when teaching feet handling tolerance to young adult dogs significantly improved the success of the training procedure. (And if our scores were similar for ears and mouths, we could include that priming is effective in those handling exercises as well).  (NOTE: Remember that this study was hypothetical and has not actually been conducted….yet….might be a nice project for a Masters student……. 🙂 )

And that, in a nutshell, is why we need Multiple Steves!

THANKS FOR TOONIN’ IN FELLOW DOG FRIENDS – MORE SOON! TIME FOR A SWIM WITH MY PAL, VINNY!

Vinny goes for a Paddle

Your face is gonna freeze like that (Part 3)…….(aka: How many Steves?)

The mental manipulations that we described in Part 1 are actually a form of psychological priming. Priming occurs when the way in which a person responds to an event (stimulus) is influenced by a previous stimulus. Perhaps most intriguing is the fact that much of this influence takes place outside of our conscious awareness. In part 2,  we presented our working hypothesis that priming may work with dogs; specifically priming dogs to feel happy and relaxed will enhance learning during touch-then-treat training.

So, can a dog’s emotional responses and behaviors be primed? Like most people who live with and love dogs, my intuitive response is “Of course they can“! But, intuitions aside, if we are being good canine scientists, how might we collect actual data that will either support or refute this hypothesis?

Suppose that our friend Terry decides to test the theory on her dog Steve.

single Border Collie

STEVE

The Steve Study:  Terry sets up a study protocol of daily training sessions with Steve. In each session, Steve is primed  using a favorite food treat. Terry decides to use 10 repetitions of reinforcing eye contact, something Steve readily and regularly offers, as her priming stimulus. She follows this portion of the session with a series of touch-then-treat exercises in which Steve’s front paws, mouth, and ears are each handled gently (see below). Terry repeats the entire routine once per day for a period of three weeks.

Touching Ear Touching feet Touching teeth

Data Collected: For each training session, Terry rates Steve’s response to the three types of handling (ears, paws, mouth) using a five-point rating scale that ranges from “does not accept at all” (score of 1) to “accepts completely” (score of 5). This type of scale, called a “Likert scale” is commonly used in studies that rate subjective experiences. Terry scores Steve’s responses before the experiment begins (pre-test) and daily (after each session) until the end of the study. Here are Steve’s mean (average) weekly scores, before and after testing, for each handling exercise:

HANDLING

PRE-TEST

WEEK 1

WEEK 2

WEEK 3

EARS

4.1

4.8

4.8

5.0

PAWS

1.4

3.5

4.2

5.0

MOUTH

2.1

3.2

4.4

4.4

Results and conclusions: Steve’s scores show that he was quite tolerant of having his ears handled at the start of the study (mean score 4.1), but that he did not tolerate handling of his paws (mean score 1.4) or of his mouth (mean score 2.1) at the start of Terry’s study. All three areas showed improvement over the 3-week training period. Steve completely accepted Terry’s handling of his ears and paws and showed a great deal of improvement in allowing Terry to work with his mouth by the end of the three-week study. Terry concluded that “priming is an effective training tool that leads to  improvement in a dog’s acceptance of handling exercises

Is Terry justified in this conclusion? Does Steve’s response tell you if priming enhanced classical conditioning (touch-then-treat training) in dogs? In a word, NOPE.

There are several problems with using a Steve Study (or a Muffin study, or a Rover study or a Cooper study). Some of these may be immediately obvious to you; others perhaps less so. Here are four reasons that Steve’s response (though very nice for Steve and Terry) should not be used to make conclusions about our hypothesis:

1. Confounding factors: Several other factors may have influenced Steve’s behavior. The passage of time alone (especially in an adolescent dog) may have caused him to more readily accept handling. Second, Steve may have started to tolerate handling regardless of any type of training intervention, simply in response to the daily scheduled interactions with Terry. Third, his response may have been due to touch-then-treat alone, regardless of priming. These uncontrolled factors are why all studies need to include a control group.

2. Experimenter bias: Terry was not blinded to the treatment and had expectations that it would be effective. This leads to a common cognitive error called confirmation bias; more about this in a later blog). In this case Terry’s bias is also caused by a placebo effect and would lead her to err on the side of seeing improvement in a subjective measure where it may not have existed.

3. Steve ain’t stable: No, this is not a derogatory statement about Steve. What I mean is that Steve, like all biological creatures, varies in his day-to-day behaviors. That variation, which occurs both within an animal and between animals, must be accounted for when trying to determine if an animal is actually responding to a treatment or if the results that we are seeing are purely by chance and are caused by normal day-to-day fluctuations. In this case, fluctuations in behavior or learning, but this applies to all biological processes.

4. One Steve = Anecdote (not a study): Last but not least, the fact that Steve varies day-to-day in his behavior is further complicated by the fact that ALL dogs vary from one another (we all know this, of course). In this particular case, we would expect that all dogs vary in the degree to which they respond to priming (if priming works). The trick with research is….here is the punch line……separating the normal variation that occurs within and among dogs from the variation that may be caused by our treatment (in this case, priming) is why we need to study groups of dogs and why we always need one or more of those groups to be a control group This is also the reason that we need statistics – to keep us from making incorrect conclusions that occur because Steve just happens to be an unusually smart dog (which of course, he is), because of normal differences among dogs, or because Steve just happens to be having a good day.

So, instead of just testing our hypothesis on just Steve, we actually need to test it on a sample of the population (i.e. multiple Steves):

many BCs

MULTIPLE STEVES (sample size: n = 10)

OOPS! WAIT you say! These are all Border Collies! Don’t we need a sample that is representative of all dogs in the population? Yep – we sure do……

Group-of-dogs-resized-600

STUDY SAMPLE (n = 9)

This looks a bit better. Let’s test our hypothesis on a study group, a sample that of dogs that is representative of the population of dogs that we would like to make conclusions about. Let’s start with the group of happy fellows above, but keep adding until we have about 75 young adult dogs available for our study. What are the treatment and control groups that are needed for this study? To thoroughly control this study, we need three groups:

Treatment (Experimental) Group: Also called the “test group”, this is the group of dogs that receive the experimental treatment; in this case this is a series of Priming + Touch-then-Treat training (25 dogs or n = 25).

Positive Control: Positive control groups are used when we need a control that is expected to have a positive result, allowing the researcher to show that the protocol was  capable of producing results. In our study, this would be a group of dogs who are not primed, but are trained to accept handling using Touch-then-Treat only. (n = 25)

Negative Control: Negative control groups are used to make  sure that no confounding variables affect results and to factor in any likely sources of  bias. A negative control can also be a way of setting a baseline. No training. (n = 25)

Study Protocol: Dogs are randomly assigned to one of the three groups. (We might also “block” dogs across treatments, which matches dogs by sex, age, breed or other sources of variance that we want to control; more about this in another blog). We use the same study protocol, with one exception. The person who scores each dog after their daily training (or no training session) with Terry, is not Terry. Rather, we will use a scorer who is “blinded” to the study treatments and is present only to score each dog’s response to handling, with no knowledge of the treatment.

In the final part of this series, we will look at data from the Multiple Steves study and what conclusions we might make from our hypothetical study. For now though, you may be asking……WHY IS THIS APPROACH IMPORTANT? Well, let me tell you…….gotta first climb…..

Up on my Soapbox…..

soapbox   

        Testing new ideas using the scientific method protects us from making Steve Study mistakes. Sure, you may see improvements in your dog’s coat, vitality, agility performance, or health when you switch him from a cooked to a raw diet, when you decide to go grain-free, when you train him to balance his front feet on a ball, or when you use the ointment for his ear infection that Joe next door (who knows a lot about dogs) concocted and gave to you. However, without adequate study that includes groups of subjects, control groups and (gasp!) statistical analysis, you cannot know if something that has not yet been studied is actually doing what you think it is doing. While not infallible, the scientific method  is constructed to prevent or minimize bias, to test sample groups that represent a population, and to prevent us from coming to conclusions based only upon our biases, our intuitions and our dear beloved, and very smart dog, Steve.

to be continued[1]