Today I am reviewing Big Data: A Revolution That Will Transform How We Live, Work and Think. A photo of a horse in full stride is one thing, but when you combine hundreds of photos of the horse running along a path, you get a movie ("motion picture") that suddenly provides the viewer with a new level of insight about things that are impossible to get from the single photo (e.g., gait or speed).
This is an example of when a change in the scale of data leads to a change in state; the data take on completely new properties. That a change in scale leads to a change is state is the premise behind Viktor Mayer-Schönberger and Kenneth Cukier's Big Data, which seeks to analyse the impact of the world's rapidly escalating rate of data accumulation. The authors provide a background of how data accumulation has occurred historically and how changes in the way we collect and store data (and the decreasing cost of doing so) are contributing to an ever more rapid rate of accumulation (4x the rate of growth in the world economy), which is providing insights that weren't possible even in the very recent past.
There are countless interesting examples about companies and governments are using this information, but the core point is that big data is accompanied by three major shifts:
- As N approaches the full set (N = all), new insights are available that are not available when using more traditional research methods where N is some subset of the population.
- Where N = all, we have to embrace real-world messiness rather than tiny, perfectly controlled studies.
- Big data leads to superb insights into correlations, but leaves causality as unclear as ever. This does not limit its efficacy. (Focus on the what, ignore the why).
Of relevance today is that there's an interesting discussion of NSA data collection on page 156, presciently filed in the "Risks" chapter. Overall, this book was a worthwhile read that succinctly provides insight into why some companies are benefiting from big data, which can help inform your investment theses. For example, I often thought of GameStop's (GME) massive data trove from which it gains insight into buyer behavior and shares with game developers and device manufacturers to help solidify GME's position a valuable partner -- rather than the opponent that many short sellers believed it was.
Disclosure: This book was provided by the publisher for review.