Review of Sam Savage's "The flaw of averages"

If you ever thought that probability calculus and statistics are boring, incomprehensible or just irrelevant for real life, then The Flaw of Averages might make you think again.

The book’s author, Sam Savage,  a Consulting Professor at Stanford University, and a Fellow of the Judge Business School at the University of Cambridge, has set himself as task to explain in plain English how people make avoidable mistakes in assessing risk in the face of uncertainty – and, with respect to this specific point, the book is an unmistakable success. (By the way, Savage also happens to be the son of one of the great statisticians of the last century, which is probably why the name is not completely unfamiliar to you if you have taken a course on the subject).

The book is organized in three main sections.

In the first section, Savage lays the foundations of the book.  Savage points out that, when people plan for the future, they often replace uncertain outcomes with single numbers, the averages. He uses several real life examples (some of them hilarious) to illustrate why this approach is, more often than not, wrong, and leads, on average, to negative outcomes.

Savage then goes on to explain that uncertain outcomes are not just described by their averages, but by their whole distribution. He then quickly moves on to discuss the behavior of combinations of uncertain numbers, and the importance of understanding interrelated uncertainties. Subsequently, he discusses decision trees and the value of information (guess what: you should not be willing to pay any price to make better informed decisions!).

In case (a) you have taken a university level course in statistics (b) actually understood the practical implications of what you have been thought (c) remembered how to apply the tools you have learned, then you will probably already have guessed that this section in the book will teach you nothing substantially new – basically, what Savage tells you is that the expected value of a function of several variables is not always the value of the same function applied to the expected value of these variables. However, this is no reason to think you will learn nothing from the book.

Indeed, the originality of this section is different.

First, even if you think you master the principles of probability calculus, the examples provided by Savage will show you that you often fail to apply these principles when confronted with real life applications . To be honest, I made several hugely embarrassing mistakes myself – the price for revealing one (to you in person, not on this site) is a Westmalle Trippel http://en.wikipedia.org/wiki/Westmalle_Brewery .

Second, Savage is a master teacher and storyteller. He really does succeed in explaining all the essential elements of probability theory (including far-from-obvious topics such as Jensen’s inequality) in a very accessible and humorous style. Every teacher of probability and statistics should read this book to understand how his subject can be brought to life – as a result, his student might even become actively interested in the subject.

 In the second section, Savage provides several applications of the theory: financial portfolio theory, the capital asset pricing model, derivative pricing, corporate investment theory, supply chain management, the war against terrorism, health care, gender issues, World War II and climate change. Again, Savage illustrates brilliantly that problems of staggering complexity can be brought to life, and, more importantly, can be explained in such a way that the student actually ends up understanding what the subject is about (want to understand how the boffins at Long Term Capital Management almost blew up the world economy in 1998? –read the book).

Finally, in the third section, Savage presents his own solution to the problem: Probability Management. This review is not the right place to discuss this approach in detail. Essentially, Savage argues that, when faced with uncertainty, decision makers should be running thousands of simulations, showing how random events affect all relevant variables simultaneously, and illustrating the interactions between these variables. The book contains links to websites that offer commercial software that can be used to run simulations on standard spreadsheet packages (the website also contains demo versions).

This part is the weak link of the book. While the two first sections are light-hearted but very informative, this section mostly reads as a big advertisement for the services Savage has on offer. Instead of providing detailed and informative examples on how Probability Management could improve decision making, Savages mostly discusses personal anecdotes on its intellectual conception. It’s still a lot of fun, no doubt about that, but it’s rather disappointing after the brilliant start of the book. But, then, I suppose that the objective is really to make us download the demo versions and find out their usefulness for ourselves.

Let me summarize. There’s an old joke stating that a statistician is someone who drowns in a river that has an average depth of 30 cm. This joke show a complete lack of understanding of what statistics is about: Savage shows that people who drown in these circumstances are precisely people who fail to understand that one should look at the complete distribution of an uncertain variable, and this is precisely what statisticians do. To quote from the book: the best models are the models you no longer need because they have changed the way you think. If this book can change the way a few hundreds of thousands people worldwide think, it will certainly contribute to make the world a better place.