The Planes That Started It All
During the war, engineers studied returning bombers and marked where they had the most bullet holes. The obvious solution was to add armor where there were the most holes.
A mathematician reversed the logic: the bullet holes were visible where the plane could survive them. The armor had to go where there were no marks - because planes hit there did not return.
The mistake was not in the calculations but in the sample. It included only surviving planes, so the damage map showed safe hits, not dangerous ones.
Where You See the Same Pattern
- Advice from successful people: we hear from those who made it and not from those who did the same thing and fell short.
- Reviews and ratings: happy and furious customers write them, while the silent majority does not appear in the statistics.
- Old things: "They used to make things to last" - only what did not break has survived to reach us.
- Investments and the market: companies that shut down disappear from reports, making past returns look higher than they really were.
That is why success stories are dangerous. You cannot infer a cause from them: if thousands of people who did not make it had the same habits, those habits explain nothing, and you have to look for an explanation in what you cannot see.
The check is simple and almost always uncomfortable: ask where the other half of the data is. Who tried and did not make it? What happened to those who did not return? Usually, it turns out that no one collected this data.
The reverse application is useful, too. When you decide what to learn from someone else's experience, look not at those who succeeded but at those who were in a similar position and fell short: the causes are clearer there.
Finish reading in the Miqo app
1 more minute, then a quiz at the end. Find it under “History → Great Mistakes”
Point your phone camera here to open the App Store or Google Play.
Sources
- The example of armoring bombers is associated with Abraham Wald's work in the Statistical Research Group during World War II: he showed that the distribution of damage on returning planes was skewed and proposed reinforcing areas without bullet holes. Systematic selection bias is described in statistics and regularly appears in reviews, success stories, and return data