“Trust your gut.” It’s a cliche we’ve all heard (and probably even expressed at one time or another), an exhortation to not overthink decisions in realms as diverse as business, sports, dating and politics.
But is it actually good advice?
In a new paper published in the Journal of Behavioral and Experimental Economics, a trio of British economists applied some brain power to the question of gut feelings and found that people who second-guess themselves make considerably worse decisions than those who stick with instinct. The researchers focused on prediction accuracy in sports betting but said their findings would apply in any realm where people have to make educated guesses about the future.
The economists gathered data on 150 users of a popular sports betting and prediction website. This was a representative sample of a subset of bettors vying to predict outcomes for all 380 soccer matches in the English Premiere League during the 2017/2018 season.
The final data set consisted of 57,000 individual predictions, each consisting of a user’s best guess of the final score of a given match. Users were allowed to issue score predictions and to revise those score predictions up to the minute the match started.
Revisions were infrequent: 6 percent of the predictions changed – with the typical user revising 15 out of 380 predictions – and most of those occurred within minutes of the initial forecast. The authors speculate that in such cases, users typed in a prediction and looked up some relevant information on the match before quickly changing their mind. But there also were a number of instances involving large segments of time – days, even weeks – between prediction and revision; such betters waited nearly two days, on average, to switch gears.
The researchers specifically wanted to know whether the revised predictions were more accurate than unchanged ones.
In theory, there are a lot of reasons to believe this might be the case. A person would presumably revise a prediction after obtaining new information, such as an analyst’s match forecast or a team roster change.
In practice, however, the opposite was true: Revised forecasts accurately predicted the final match score 7.7 percent of the time. But the unaltered forecasts were correct 9.3 percent of the time. In other words, revised forecasts were about 17 percent less accurate than those that had never changed. “Game players would have been better off sticking with their first judgments rather than ever revising,” the authors write.
So where did the second-guessers go wrong? For starters, the researchers controlled for match-to-match and player-to-player variation – it isn’t likely the case, in other words, that matches receiving more revisions were more difficult to predict, or that bad guessers were more likely to revise their forecasts.
The researchers found that revisions were more likely to go awry when forecasters dialed up the scores – by going, say, from predicting a 2-1 final score to 3-2. Indeed, across the data set the bettors systematically underestimated the likelihood of a 0-0 draw: an outcome anticipated 1.5 percent of the time that actually occurs in 8.4 percent of matches.
That suggests a general forecaster bias toward scoring as opposed to non-scoring – or more broadly, toward things happening rather than not. Few people, after all, watch a soccer match and hope that nobody scores.
One final, tantalizing piece of evidence: Forecasts revised after a longer period of time were considerably less accurate than those changed within a few minutes. This suggests that overthinking it, or “over analysis” in economics parlance, was a key driver of
the revised forecasts’ relative inaccuracy.
In the end, the authors write, their findings “could have relevance to other contexts where judgmental forecasting explicitly takes place and which have real economic importance, such as in company management and planning, financial markets and macroeconomic policy.”
For those of us not in the prediction business, the findings can be boiled down to a simple message: Trust your gut.