#6: Incorporating New Information into Your Estimates
Gradually getting closer to the truth by continually updating estimates
Once, a friend and I were trying to figure out who was the secret boyfriend of another friend. The person wouldn’t tell us (yet), so we had to guess.
We settled on one person, and we thought it might be him with 55% confidence. Then, in discussing the likelihood, we eventually bumped our confidence to 94%. For example, after realizing our friend was a workaholic and wouldn’t have time for dating new people outside of work, we concluded it had to be someone at the same company. That led us to increase our confidence from 55% to 80%. Then, a few other facts, like seeing them talking once in the hallway, nudged it up higher and higher.
Were we correct? YES. Eventually, over dinner, we told our friend that we had a guess about who it was.
"Okay? Who do you think?" our friend said.
"We think it is John [a pseudonym]," we said, giving our 94% prediction.
"You're right," our friend said.
What we did well was continuously update our estimates intuitively based on small pieces of information and insights.
The more mathematically sound approach is to use the Bayesian belief-updating equation. “In simple terms, the theorem says that your new beliefs should depend on two things: your prior belief (and all the knowledge that informed it) multiplied by the "diagnostic value" of the new information.”1
But starting with a simple approach can build confidence and habits.
Also, the simpler approach is not unreasonable. Individuals who consistently make highly accurate predictions about future events (e.g., superforecasters) rarely explicitly use Bayesian updating.
The superforecasters are a numerate bunch: many know about Bayes' theorem and could deploy it if they felt it was worth the trouble. But they rarely crunch the numbers so explicitly. What matters far more to the superforecasters than Bayes' theorem is Bayes' core insight of gradually getting closer to the truth by constantly updating in proportion to the weight of the evidence.2
References
Tetlock, P. E., & Gardner, D. (2015). Superforecasting: The art and science of prediction. Broadway Books.
Tetlock & Gardner, 2015, p. 170
Tetlock & Gardner, 2015, p. 171