While we consider ourselves to be fundamentally rational beings, our rationality is ruled by intuition – that is, whether or not we believe a thing to be true. As adults, our rational thought process has a significant impact on what we believe, but in the end if we can’t get over the feeling that something isn’t right, most of us will go with our gut feeling – intuition. Of course, sometimes our intuition just doesn’t line up with the facts.
During World War II, the Royal Air Force was taking a beating. German anti-aircraft guns were taking down too many planes; it was time to bulk up the armor on the underside of the planes. But where should it be reinforced? Intuitively, it made sense to check the returning planes for bullet holes and add armor plating to the areas that were shot up the worst. But a mathematician named Abraham Weld said no, you’re doing it wrong. Instead, he suggested, add armor to the returning planes where there were, on average, the least bullet holes.
What was wrong with the first plan? It ignored an important fact: they were looking at the wrong planes. Weld pointed out that the planes they were drawing their conclusions from had already made it back; on these planes, the areas with the most bullet holes were the least dangerous areas to get hit. These planes couldn’t have many bullet holes in the most vulnerable areas – if they had, they wouldn’t have made it back!
Drawing conclusions from raw statistics is likely to result in confirmation bias – the tendency to favor information that supports our hypotheses. The best way to counter this problem is to actively look for information that proves your hypothesis is wrong. This forces you to look at the data from another angle, making it more likely for you to find the holes in your theory – no pun intended.
For more on the topic of airplanes and confirmation bias, read about drill instructors’ bias. That’s not a technical term; I just made it up. Having a hard time seeing another side to your hypothesis? Ask for another opinion. Thanks for reading!