In his 2004 book “The Wisdom of Crowds”, James Surowiecki makes an interesting case that if you want to make a correct decision then large numbers of ordinary people can provide better advice than a small number of experts.
Surowiecki claims that if you ask a large enough sample of diverse, independent people to make a prediction or estimate a probability, and then average those estimates, the errors each individual makes in coming up with an answer will cancel themselves out. Each person’s guess has two components: information and error (or signal and noise). The process of averaging tends to cancel out the errors, leaving the information.
On the other hand, small groups of experts tend not to have a diversity of opinion and nor have they necessarily come to their opinions independently. Experts are often vulnerable to “groupthink”. This was evident in the period before the global financial crisis – which 99% of economists failed to predict. Thus, it is hard for business and government planners to manage risks effectively.
Foreseechange has been researching the Wisdom of the Masses concept since 2005. We prefer the term “masses” to Surowiecki’s “crowds” because the latter implies a group of people in close proximity and, therefore, not as diverse and independent as the masses.
The Wisdom of the Masses is quite good at forecasting political and economic outcomes.
We have found that the concept of the Wisdom of the Masses is rather more complex than suggested by Surowiecki. In addition to the signal and noise components, most individuals are prone to bias of various sorts. These include optimism and pessimism, and perceptions that there will be great change or that there will be little change. These biases do not always cancel out when taking an average, so to forecast with the wisdom of the masses, we may have to exclude biased opinions. The Wisest of the Masses are a small segment of the total population whose predictions are judged, on the basis of analysis of their responses across the board, to contain little bias.
The opinions of the masses are important in two other ways, not mentioned by Surowiecki.
First, the wisdom of the masses provides insight into what consumers are expecting to happen. The opinions will be a factor influencing their behaviour. For example, if a rise in unemployment is expected then some consumers may be more reluctant to take on additional financial commitments.
Second, by studying variations over time and location in estimated likelihoods of events we can gain insight into what factors drive changes in expectations.
Wisdom of the Masses is available in standard reports and can be applied in custom studies.