The problem with observer bias in studying animal intelligence

Why it’s important to look at how studies are done

I’d like to step back from the fun stuff, the stories of how animals show their intelligence, and let you know about some of the problems with studying animals, starting with the potential for bias by the person(s) conducting the study.

What is observer bias?

The website Psychology Tips defines the problem this way: Observer bias refers to the systematic errors that occur when the observations or interpretations made by an observer are influenced by their own preconceived notions, prejudices, or expectations.

Probably the best way to show what observer bias looks like is to go back to the famous Clever Hans story. Clever Hans was a horse whose owner claimed he could do basic math. For example, when his owner asked what 4+4 was, Hans would ‘count’ by tapping his hoof, until he got to 8.

Sounds impressive, but Hans couldn’t count when his owner wasn’t present. Hans was actually picking up subtle clues from his owner. He was always rewarded for a correct answer, so figured out how to get his reward. I would call that clever, but not in the way he was originally thought to be.

No, we don’t know what these meerkats are thinking Photo by Dušan veverkolog on Unsplash

Observer bias is hard to control. Scientists often go into a field of study with an idea of what they are likely to find. Starting out this way, it’s hard for them not to find what they expect to find without significant controls on their study. Luckily, there are ways of doing this.

Ways to control for observer bias

Perhaps the most common method for ensuring observer bias doesn’t enter into a study is to create a ‘blind’ study where the observer doesn’t know what the expected outcome is. In human research there is one further step- the double-blind study where neither the researcher nor the subject knows what the expected outcome is.

Needless to say, this is not a major concern for many animal studies where the animal being observed or participating in a test can’t know what the researchers are looking for. However, as Clever Hans showed, captive and companion animals can often read subtle clues from humans to intuit what that person is hoping to see, or to guess whether they’re likely to get the reward they’re looking for.

One way to mitigate this potential bias is by using two observers. In the case of Clever Hans, it was an independent scientist who decided he needed to see how well Hans could count without his owner present. Two people, even if they share the bias of knowing the predicted result, are less likely to record exactly the same observations, allowing for discussion and comparison of their findings.

In the wild, and probably valuable as well in controlled studies, in addition to the use of two observers during a study, whenever possible using audio and video footage to be reviewed by someone not directly involved in the study design and goals can help control observer bias.

Earlier animal behavior studies often had problems with observer bias

A 2012 paper in the Canadian journal Ethology: international journal of behavioural biology found that only a handful of papers on animal behavior provided information regarding how the authors had included at least one ‘blind’ component in their study.

The good news is that animal behavioral researchers are getting better at including control for observer bias in their studies. In another article, published in 2024, the authors of the above paper once again reviewed the 2020 publications of the five major animal behavior journals reviewed in the earlier paper. Control for bias in papers accepted had increased from 10% in 2012 to 50% in 2020. This shows an increased interest in utilizing best practices, which in turn make the papers produced viewed as more reliable proof of their premise.

While this is great progress, there’s still more that can be done to improve control for observer bias, especially with the advances in audio and visual technology.

And we as individuals can be thinking about how to recognize how little we know about what animals are thinking. Our observer bias often appears in the form of anthropomorphism, giving human interpretations to what animals are doing. But, in truth, we rarely do know what they are thinking.