In The Signal and the Noise, the New York Times’ political forecaster and statistics guru Nate Silver explores the art of prediction, detailing discussions with expert forecasters from economists to big time gamblers. Maria Kuecken finds that Silver’s writing style is straightforward and accessible, peppered with anecdotes, charts, and references which keep new readers engaged.
The Signal and the Noise: The Art and Science of Prediction. Nate Silver. Allen Lane. September 2012.
In this era of Big Data, we would like to think that we’re getting better at making predictions. But as the information piles up, the list of hypotheses to test inevitably grows longer. The truth is out there, but it is becoming increasingly difficult to find. Such is the central proposition of Nate Silver’s The Signal and the Noise in which he elevates prediction as both a problem and solution. Silver takes a balanced look at the issues plaguing prediction across various fields as well as how these problems can be improved.
Nate Silver’s name is by now familiar in social science circles from his FiveThirtyEight forecasting model, which debuted during the 2008 US general election. But before his election days, he attained recognition by developing a probabilistic system to predict baseball statistics while working a consulting day-job. Thus, Silver holds a unique vantage point from which to critique the science of prediction, one that is motivated not so much by a particular topic but by a global interest in the field itself.
With so much information constantly produced and advanced computer programs to boot, it is relatively easy to churn out data-driven forecasts. But a data-driven claim does not a good prediction make. Much of the information out there is simply noise “which distracts us from the truth.” If we sift through enough of this noise, we are likely to come up with relationships that seem meaningful when they don’t truly exist or predictions that are way off the mark from reality. Instead of being cautious about overfitting and uncertainty, Silver notes that “our bias is to think we are better at prediction that we really are.” Not only this, but forecasters’ priors and perverse incentives influence the results they present as well as the types of hypotheses they are willing to test in the first place.
It turns out, quite unsurprisingly, that all fields are not created equally when it comes to making predictions. In some cases, the increasing volume of data has been a great boon for accuracy. Weather forecasts, for example, are introduced as a positive example. Though dynamic weather systems are innately chaotic, with a minute change producing potentially massive effects down the line, a combination of computer generated scenarios and human ingenuity to catch errors helps to underpin their forecasting success. The incredibly complex system of events that generates an earthquake, on the other hand, still prevents scientists from predicting when they will strike. Moving away from hard sciences, economic and political forecasting are influenced heavily by narratives and prior assumptions. These examples are simplifications of issues treated in-depth in Silver’s book, but they help to illustrate why the process of generating predictions, and good ones in particular, is difficult to navigate.
So how can we make better forecasts or, at least, be better forecast connoisseurs? Those anticipating an easy solution will be disappointed – if anything, readers will come to realize that we know much less about the world than we think we do. Rather, Silver offers an insightful prescription on how to approach prediction, applying it to a variety of fields from chess to climate change in order to show its versatility.
Silver advocates a Bayesian approach. Broadly, such a framework means that we should base our predictions on conditional probability – in other words, “the probability that a theory or hypothesis is true if some event has happened.” He uses several cases to drive the point home, one being a now-classic example: We suppose that a woman in her forties has a very low chance of developing breast cancer. When she takes a mammogram, she receives a positive result. It might seem that she is now highly likely to develop breast cancer, given that a positive mammogram is incorrect only 10% of the time. However, the likelihood that this forty-something woman will develop breast cancer is still miniscule – why? Because very few women in this age group have cancer at all, even with a positive mammogram. Guarding against such false positives is one way a Bayesian approach can help us to keep our predictions in check. If we subscribe to this thought process, our imperfect estimates are just a start. We can and should update them as we obtain new and better quality information. Thinking probabilistically can put us on the road to generating more reliable predictions.
The downside of such statistics-centered books is that they have a tendency to grow tiresome. However, Silver’s writing style is straightforward and accessible, peppered with anecdotes, charts, and references to get his points across without losing the reader. He manages to provide a degree of information that is detailed while remaining engaging. Better yet, he tackles a variety of topics but intentionally chooses those for which information is widely available so as to be accountable to his diverse readership. Overall, Silver’s modesty (and honesty) about the limitations of forecasting is refreshing in an era when all too many claims are fact-checked and found wanting.
Even a pro like Silver acknowledges that we can never fully rid ourselves of our priors, even with the best of intentions and the most modern statistical tools. We should therefore do our utmost “to be less subjective, less irrational, and less wrong.” For researchers and forecasters, it’s a reminder that introspection is always in order, that we must be open about uncertainty and bias. For students and those interested in prediction, it’s a reality check about the boundaries of our knowledge, particularly when faced with dauntingly ‘big’ questions about the global economy, natural disasters, and climate change.
Good predictions may be hard to find, but so are good books about statistics. The Signal and the Noise is definitely one of them.
Maria Kuecken is a Ph.D. candidate in Economics at the Paris School of Economics—Paris I Pantheon-Sorbonne University. Maria teaches graduates and undergraduates in Applied Econometrics and Development Economics. Specializing in development economics, her own research focuses on the determinants of educational quality in developing countries. She has blogged on a variety of issues for the European Journalism Centre, worked on educational projects in Rwanda, and interned in the Health Division of the Organisation for Economic Co-operation and Development. Read more reviews by Maria.