[book review] The Signal and the Noise: Why So Many Predictions Fail — but Some Don’t

Nate Silver’s The Signal and the Noise is a great entry to know how much information is out there and how much we don’t know about it. All the data in the world will not make us predict the future yet knowing how to analyze it is a something that I think all of us has to know.

I think we don’t value statistics enough and this book tells me how much I need to know more about it.

The Signal and the Noise

As much as I want to be a great fan of statistics, I was never really good in math but luckily the statistics subject wasn’t that hard during university. If you read The Black Swan by Nassim Nicholas Taleb then you would have an idea how much we under utilize statistics in our daily lives. This book will give you insight on how statistics is being used in different industries and how wrong we are in analyzing the data that we are given.

Nate Silver was already known in the baseball world before the US presidential elections last 2008, but the elections made him a household name after successful predicting the winner in 49 out of 50 states. If you follow just a bit of US politics like I do (via The Daily Show and Colbert Report 😛 ), then you would have definitely heard of him.

A former accountant, Silver created his predictive baseball algorithm on the side which eventually led him to better opportunities. The book discusses several industries where prediction and forecasting is their main role. Jobs like political pundits, weather forecasters, chess computer programmers, and even poker players rely a lot on statistics. This varied mix of industries got me really into the book and showed me how important statistics is.

The book was released in 2012 and a good start to any “statistics” book is to start with the 2008 economic crisis and how nobody saw it coming. Of course, he does give the typical hindsight is 20/20 routine but Silver does have a bunch on insights on why a lot of people saw it coming and how fiscal irresponsibility and ballooning property prices were good indications of the things to come.

A ironically funny chapter in the book is about political pundits and how they constantly get things wrong. Silver is in the pundit game and he knows what he is talking about especially about television pundits. It’s just hilarious for me especially since I get my “news” from comedy news show.

The book does have a good analogy on two types of thinkers: hedgehogs and foxes.

• Hedgehogs are personalities that believe in Big Ideas. People who won’t change their minds on predictions even though new evidence presents otherwise.
• Foxes on the other hand are people who believe in little ideas and use the data to formulate their predictions.

The book has better examples and explanations of this.

Moving away from politics, the book goes into weather forecasts and how this industry has improved over the years. If you are like me and you look at weather predictions all the time then I’m sorry to tell you that we’ve all been had. I used to live in tropical countries and man the weather forecasts are at best “a guide”. Unless it’s for a major storm, 20% of rain means nothing to me and I would always carry an umbrella. Having only two seasons, tropical countries have a hard time predicting the weather (or so I thought). When I moved to Japan, I was quite surprised on how “good” their predictions are and I would always follow their advice on what to wear and whether to bring an umbrella or not. Alas, a few months later, I still carry my umbrella all the time. Expecting a good weekend picnic is not an accurate science too. Apparently, weather forecasts are much much better now than they were 30 years ago which I totally believe. The advancement of weather forecasting technology would definitely have given us more data to work with. However, one key point I got from the book is the biases that are introduced into the forecasts as well as 5-day forecasts. The book explains this more, let’s just say that I only look at forecasts with confidence two days in advanced now.

Living in Tokyo, the topic about earthquake prediction definitely got me interested. If you’ve lived here for a few weeks, you’ll know that earthquakes is as common as the wind. Silver discusses how earthquake predictions are always wrong and why we always want to look for patterns in earthquake predictors. I really believe that the technology to predict earthquakes are still far off and until then, all we can do is be prepared. I felt a couple of moderate earthquakes here and to be honest, I won’t know how I’ll react once another big one comes. I am ready with my emergency pack and all, but looking at the statistics a big one will definitely hit sooner or later. The book has a good analysis of earthquake occurrences that he will later discuss but on the topic of terrorism.

Another chapter that got me piqued was the chapter about climate change. Although I still don’t understand why there are still people who don’t believe that climate change is a real thing is beyond me, he does have some information on why. Comparing date from the past, sample sized, and different predictive algorithms has a big part on this. Although just looking at what is happening with our weather patterns should make you think that climate change is real. Larger storms, longer winters, hotter summers, and random freak fests should give us an indication that something is wrong with our nature and we are contributing to this destruction should be freaking obvious. The recent storm in the Philippines, Haiyan, is another single point example of this.

I really enjoyed this book and if you are like me who likes to know how much I don’t know, then you would appreciate this book at lot. Looking at different industries and professions in a different angle will just make you appreciate them more and all the thought that goes into their jobs that we take for granted.

[book review] So Good They Can’t Ignore You: Why Skills Trump Passion in the Quest for Work You Love

Constant self-improvement is everybody’s goal. If you are still struggling which self-help book to read, make sure you read this one. Grounded on real life and none of that secret crap, So Good They Can’t Ignore You is based on something real, tangible, and accessible to us normal folk who actually have jobs, don’t have a million dollars, and not part of the “exception”.

For those unfamiliar with Cal Newport when you were in school, then I suggest checking out his website at http://calnewport.com/. I finished university by the time he was writing his study hacks blog, but I saw a number of my interns reading his blog and I can see how effective his techniques are. Even if you’re outside school you can definitely get some tips on his study habits and tackling a similarly academic setup.  Familiarising yourself with his content will give you better context on how he came up with his processes in the book.

Cal Newport has a great summary of the book at his blog. I’ll try to do what he did to himself, using his own book to analyse himself, where he went wrong and where I can improve. I’ll go the rules one by one so that you can see it in action and why I really like this book.

Rule #1: Don’t follow your passion

The book details how follow your passion is bad advice. Follow your passion is something you hear all the time especially when expose yourself to different communities where people can become experts or successful with what they do. We always hear about people taking the plunge and going straight to things they really love like moving to another country (ha!), quitting their day jobs and starting a business or something that is truly out of the ordinary. I would separate this from risk taking as I’m a big believer that calculated risks is part of life, but rather big jumps that make us seem to believe that there is a perfect job out there waiting for us to quit our current ones and take that “dream job”. He calls this the passion hypothesis. The book has several examples on why figuring out our passion before hand is almost impossible or not financially viable. For some reason, a huge percentage of us think that our hobbies or things we enjoy can be made as a career and we sometimes mistake this as a passion in life. Being the practical human beings that we are, we have to take a look at things in a much more different light. Rather than dreaming of somewhere else we can be, we have to assess where we are right now and see how realistic this “passion” really is.

Seeing as where I am right now, I think I got trapped into the passion hypothesis as well. Thinking that a change of scenery would change me perhaps? I’m not really sure… My goal of moving is part of me wanting to live in another country, understanding a different environment, and getting more international experience. However, if I went by the book, there’s still one thing that I didn’t capitalise enough of which is discussed on rule #2.

Rule #2: Be So Good They Can’t Ignore You

Here he talks about the alternative to passion hypothesis, building career capital. This is my favorite chapter as it’s the most controversial and gives you the alternative to all the follow your passion advice.

Career capital is creating enough money in your personal bank so that your options will open up in the long run, creating a name for yourself with the craft/job/career that you have chosen. The few examples in the book are of a TV writer, a musician, and a venture capitalist. Most of us being knowledge workers, it might not be as clear why he choose those examples, but it’s definitely more relatable than other books. Here he defines the word deliberate practice as a way to continually improve ourselves.

I’ve heard of deliberate practice before and it’s definitely a painful process. However, deliberate practice is such a good way to measure and see if you are actually challenging yourself to keep on improving. For some reason, when I look back a few years ago, I seem to engage in more deliberate practice. Maybe because we grow more comfortable with what we do, that we try not to put ourselves in uncomfortable situations. I’m trying to be more conscious now whenever I feel uncomfortable while I’m studying, writing, or any creative endeavour and I easily realise that I’m going through that phase of discomfort.

Studying is a great measurement if we are being uncomfortable enough to know that we are actually learning. Whenever, we study and seem to find the work easy to digest, then probably that is beyond your level. However, whenever we try to dodge or rack our brain, that’s the sign that we’re putting something foreign and is most likely actually learning. It’ll take a long time and it’s a crappy feeling but that’s how you learn. Newport shows several examples of this and it’s great seeing such a feeling being validated.

Deliberate practice creates career capital for oneself. With knowledge workers, where immediate feedback and deliberate practice is not as common, it creates an open field for us to strive for excellence as we can constantly be conscious of what we do with the goal of deliberate practice. This method is much more easy to spot with professions that has immediate feedback like musicians and athletes. However, being a normal knowledge worker doesn’t give us that opportunity to solicit feedback as obvious as other crafts. None the less, knowing where and what to improve is already a step ahead of the majority.

Have I built enough career capital for myself? I would think I did but the reality is different. Building career capital has to be very deliberate and conscious. I did create value, expanded my network, and specialised but career capital has to be more than this. The thing about career capital is it has to be niche and valuable enough. Did what I use to do enough for this? Of course there are more factors to this which will be explored in rules 3 & 4 but this question has been going in my mind more that ever.

Rule #3: Turn Down a Promotion

Of course this will make more sense for people who currently have jobs so my interpretation will be different. In our desire to take more control and autonomy, we will usually associate it with a better role in the company. As we continually improve and create more career capital, more people will notice, especially your boss, and a promotion will follow soon. The lesson in this chapter is we should know the right time to advance. Just because we are given a chance to advance doesn’t mean we should take it. A promotion doesn’t necessarily mean more autonomy and definitely doesn’t mean more opportunities to build more career capital. He cited several examples here from different fields and if I look back, I really applaud people who would rather choose more control, autonomy, and challenges than a better title on paper.

I know several people who went through this path and it inspires me more to do the same. Focus on a niche and valuable skill and tough it out.

Rule #4: Think Small, Act Big

Having a mission is one of the main points of this book. But with the theme of the book, it’s something that you have to discover for yourself and not built into us.

This is probably the most difficult part of the book as this will take time and a lot of exploration. I’m a big believer that we’re here in the world to find our own purpose. Not find in the sense of finding a rare antique. It will definitely not be there by travelling and doing something extreme like one of those once in a lifetime experience. I think this book sums it up by creating more value in what we do and from there seeing and discovering what our mission can be in life. I like the term “adjacent possible” wherein the big ideas found in any field is new combinations of existing ideas. I think this sums up our great purpose in life especially if you just don’t want to “exist” in this world. To be honest, this will be a lifelong struggle for me as I search and create more value for myself. My move to Japan is part of my goal to learn more about the world, in a very slow way. I’m the type of traveller who really wants to live in a country and not just be there.

If there’s a part of this book that will be the hardest to ingest, it would be rule 4.

So Good They Can’t Ignore You will be part of my re-read list as it does a good job of reminding us how we should continually improve ourselves and not be trapped into this passion hypothesis. Reading this book definitely reminds me of Linchpin by Seth Godin in a different sense. It surely reminds me of the Survivorship Bias article at youarenotsosmart.com which makes his thinking more palatable and acceptable in this day in age of mediocracy.

We live in such a great era that opportunities are made and not presented to us. The more value we create for ourselves the more opportunities open up. This book has definitely got me thinking of how to approach our lives in a way where we are not directed by our imaginary desires and our magical passions that a dream job exists out there. Survivorship bias is a reminder on why the law of averages exists and why we only hear about heroes.

The courage culture bug definitely hit me and maybe I did move too soon to Japan without creating as much career capital as I can. In the theme of the book, I won’t say how much I love taking risks and challenges but this is one of the reasons.

If you are at the stage of your life questioning your job daily and why you are doing what you are doing, (which a quick look at my FB feeds says majority of you) then I would definitely recommend this book. Much more approachable and realistic than your typical self help crap, So Good They Can’t Ignore You will definitely be up there in my recommended list.

[book review] The Omnivore’s Dilemma: A Natural History of Four Meals

Food, food, food. If you eat food, then you better read this book. Easy to chew, a joy to digest.

Michael Pollan’s first book, the Omnivore’s Dilemma talks about the four paradigms of food production, from industrial, pastoral (big-organic and self-sufficient) and personal.

The book starts from the large producers of corn then goes down all they way to a hunter gatherer experience.

I was engaged throughout the whole book as the topic about food is so personal. Something we hold and touch everyday, something we put in our mouths and resides in our bodies for hours. Nothing is more personal than food and yet we take it for granted.

Starting with industrial agriculture, Michael walks us through a brief history of corn farming and how it has shaped the modern farms of America. How big corporations were able to control the corn production and how much money farmers lose for every yield of corn. How much corn has infiltrated our daily lives, from breakfast, lunch, and dinner. Even from driving and the tools we use at the office. He shows how much corn has been a part of the modern world beyond it’s intended use of being food.

From industrialized agriculture, he moved towards a more vile scene of industrialized farming. A place where cows, pigs, chicken, and other animals are grown to serve the purpose of feeding a “higher creature”. He visits several farms hoping that he’ll be able to track the progress of the calf he bought as he wanted to follow it’s journey from birth to plate. It proved to be a difficult task just like following a husk of corn from the field to the plate.

Industrialized farming and agriculture has it’s fair share of enemies from liberal environmentalists and the general public who has seen those scare videos from PETA showing them how animals in industrial farms are like prisoners in a concentration camp. One common complaint about the book is its lack of macro economic analysis of the whole food industry. Sure, it’s a great way to mask the book from being a capitalist view on food but there is a reason on why corn can grow so close to each other and why cows and chickens live in dire conditions to feed our insatiable appetite for food.

The author achieved his goal in leaving a bad taste in my mouth upon reading the section on industrialized farming and agriculture. How we can enjoy out of season vegetables and meats from all over the world without batting an eyelash. How we can have meals that looks like and tastes like chicken and yet is made from corn. How we can afford to throw out totally edible corn to the sea just to balance out the over supply. His goal is for us to rethink where our food comes from and the sacrifices made for us to enjoy a single meal.

The first section reminded me of the few weeks I was in the US, and how large the portions are and more importantly how much meat I consumed, even the salad tasted like meat. Compared to most of the meals I eat here in Asia, eating in States felt like a feast. Large portions, lots of condiments and lots of meat. Reading how they are produced made all the bad memories of all the weight I gained on these trips. No surprise on why USA is suffering from an obesity epidemic, I can’t imagine having to eat enormous portions everyday.

The pastoral section started by introducing us to organic farming, a relatively recent fad in the states which of course gets so much interest from the hipster/hippie folk, and how badly we fall for well written stories. He visits several organic farms and sees first hand how manipulated the meaning of organic has been. How farmers abuse the system of lax rules in the effort to pass off their products as organic. He visits several farms in California and see little difference between industrial CAFOs and the so called organic farms.

I greatly enjoy books that open our minds on marketing gimmicks, especially about common misconceptions that somehow become common knowledge, one of which is the organic farming movement. It’s clear that even though they are marked organic, the process, once seen, can hardly pass off to be organic. Crowded chickens, unnatural farming, and assembly line like processing just can’t be called organic in it’s pure essence. Luckily, I’m not a big sucker for organic products and I’m lucky enough to be living in Singapore where (hopefully) the animals and plants don’t go through so much trouble that they have to grow out of unnatural circumstances. This does bring to light how we are easily fooled not just by the companies creating the products but by our ideal pictures of how farms are supposed to be run. Lush grassy knolls, chickens and cows running free across the fields, a happy family running the farm with their farm truck used to haul feeds to the barn, this is what we associate organic farming with, a beautiful farm operating just like in the books we used to read when we were young. Pollan shatters this notion and exposes us to the hidden truth.

There is some hope to all this madness as he visits Joel Salatin’s Polyface farm. Here he discovers how Polyface tries to maximize the energy of the sun by admitting that they are grass farmers. In essence, every farmer is a grass farmer as this is the first organism that harnesses the sun’s energy, that then gets eaten by cows which then gets eaten by us. Salatin doesn’t believe that cows and chickens should be trapped in a cage but rather, he grazes them in his huge farm. The cows eat grass then move around the farm with the chickens a day behind the cows which then feeds on the cowpies and other by-products of the cow, as was designed by nature. It’s a sight to behold as Polyface has gotten down what farming is as to how the animals were designed to live. Here’s a short clip from USATODAY to give you a better idea. Pollan iscern which type of farming is better. But again, this is where Omnivore’s Dilemna fails. The macro economic benefits of feeding a nation is something that Polyface can’t tackle. Their costs are higher and their yields are lower compared to industrial farms. Salatin and Pollan makes a convincing argument that the face value of food produced by CAFOs are cheaper, yet we pay for it through other means like damage to our health and the environment, something that is very hard to quantify, unlike a price of a dozen eggs.

The final part of the book tells how humans have lived before industrialization and farming, through hunting and gathering. Here, he learns how to hunt wild pigs and gather wild mushrooms through the forests of California. It’s quite a fascinating part of the book as he gets to meet people who hunt for fun, while other who hunt and gather for profit. His goal at the end of his expedition is to create a meal from ingredients that he has harvested himself. From the fowl, to the mushrooms and other ingredients.

Here, he shot his first gun with mix feelings. He hesitated gathering mushrooms as he is unsure if it was the “safe kind”. He went through a lot of introspection especially when he was hunting as it was his first time taking a life. The ethical hunting also became an issue with his guides giving good explanations on why they continue to hunt.

This section had an air of ‘hipsterness’ for me. It didn’t seem as right as sustainable farming as most of the hspends a week helping out at Salatin’s farm to understand how he can sustain such farming methods in our industrialized world. In the end, after trying their products, the chickens tastes more “chickeney” and the eggs more “eggy”.

Polyface is a great counter example of how our race for profits has made us alter even the natural tendencies of animals so that we can maximize yield. Salatin acknowledges the limits of nature and his farm runs along with how the animals interact with nature and other animals and plants. He has accepted the fact that the environment is not mono-specie, something that large farms do, but rather animals interact with other animals. Cows don’t just eat one species of grass. Chickens don’t eat just one type of feed. Cows graze on their own and eat grass upon their discretion. Chickens eat after the cows have trimmed the grass. This is nature at its best and that interaction is something that industrialized farms can’t mimic or don’t even bother about. Their cramped cows and their manufactured feeds will produce beef, but is it worth all the toxic byproducts, the diseases contracted by the animals due to their environment? I haven’t seen a large farm to be able to pass judgement but just using logic one can dunters are doing it for game or to get a better tasting patte. I do believe that there are pros to hunting and most of the hunters are not invading any territory or over hunting but the thought that this game is used to feed those with better palates is just alien to me. Coming from a third world country where food is scarce, every morsel of food is sacred and is valued. It doesn’t mean that we don’t appreciate good food, it just means that there is not enough for everybody and one has to share, else feel the guilt of eating expensive meals while a family outside can barely feed themselves (socialist in me talking).

He concluded the book by sharing a meal he created from scratch with his friends. With the pork and wild mushrooms he gathered, pasta he kneaded himself and other more “natural” ingredients. He analyses the whole food industry, through the corn fields and cattle farms, the grazing cattle in Salatin’s farm and wild pigs of California. He doesn’t struggle to realize how much we have altered the environment in an effort to produce more and more food beyond our needs. How we have destroyed nature in order to grow animals beyond their natural environment. How we have created products that the main ingredients are beyond recognition. His analysis again gets you think of what you eat and where it comes from.

An easy read and a great eye opener for something very personal, The Omnivore’s Dilemma is something you should chew slowly else you might spit it out.