One of the core tenets of the internet, internet media and internet advertising is that everything is measurable. Advertising and targeting users has become a data-driven science, and businesses are more efficient because of it. Wix (A Benchmark company) for example, tweaks the time, text and offers in its email marketing based on detailed conversion analytics gathered every minute, optimizing marketing at an incredible frequency.
What I find surprising therefore, is the disparity in data that we accept from internet measurement companies.
Fundamentally, there are two ways to measure internet traffic and engagement: sampling and real data (which requires pixel placement). Sampling was the state-of-the-art measurement for TV and radio, because that was truly all that was technically feasible. Nielsen (NYSE:NLSN) built an enormous business out of sampling audiences for TV, retail shopping and anything else that needed to be measured. The question is: Why should we settle for sampling in the internet era when everything can be measured?
This is how Compete describes its data collection methodology:
How We Get Our Data:
Compete's data comes from a statistically representative cross-section of 2 million consumers across the United States who have given permission to have their internet clickstream behaviors and opt-in survey responses analyzed anonymously as a new source of marketing research.
By contrast, Quantcast uses a pixel to accumulate real full data. From Quantcast's methodology:
In a major advancement over traditional panel-based measurement methodologies, Quantcast couples machine learning with massive quantities of directly measured data to deliver detailed audience data in real-time for all forms of digital media including websites, video, widgets and advertising campaigns.
Using Seeking Alpha (A Benchmark company), you can see a clear example of discrepancies in unique users between real measurement (Quantcast for Seeking Alpha) and sampling (Compete.com for Seeking Alpha)
Compete.Com estimates that Seeking Alpha has 1.8MM uniques per month:
Quantcast puts Seeking Alpha's U.S. user number at 3.9MM uniques per month:
Quantcast, using a pixel on Seeking Alpha's site, measures more than double the U.S. users for Seeking Alpha than Compete measures using its sampling data, and also suggests a significantly larger growth curve.
But let's dig more. Most of the Alexa data is relative (that is relative to other sites), so it is hard to find apples-to-apples comparisons. But Alexa's estimates on Seeking Alpha's U.S. vs. International traffic vary pretty widely from Quantcast.
Alexa estimates that 67% of Seeking Alpha's users are in the U.S.:
Quantcast knows that almost 83% of Seeking Alpha's users are in the U.S.:
I would love to compare Page Views (PV) and Page Views Per Visit (PPV) between Quantcast and Compete, but I could not bring myself to pay the premium membership for the sampling data at Compete. Contrasting Alexa data and Quantcast data, you can see an almost 100% discrepancy in Page Views Per Visit when looking at the Seeking Alpha data:
Alexa estimates that Seeking Alpha's Daily PPV per user is under 3:
Quantcast knows that Seeking Alpha's Daily PPV per user is around 8:
As I started perusing these sites, I found a few nuggets interesting. First of all, you are not required to publish your Quantcast data publicly, but it does provide a measure of transparency that is reassuring to readers, partners and advertisers. For example, I tried looking up Techcrunch on Quantcast but the Data was hidden by the owner. The Compete.com data for TechCrunch showed a slide in users, but my assumption is that were we to have the real data, we would see a similar reversal to what Seeking Alpha showed. Huffington Post is available on both Compete and Quantcast, with Compete showing almost a 10% drop before the recent rise, and Quantcast showing only up and to the right (I assume the big spike in traffic is Osama Bin Laden driven).
I think the conclusions are pretty clear as to the advantages of direct measurement, yet many entrepreneurs who come pitch us quote the Alexa data in particular very frequently.
As investors and users begin not tolerating measurement methods that are sample-based or estimated, and as more of media moves to IP based systems such as IP TV and internet radio, those measurement companies using sample and panel based methods could begin losing market share. This has real implications for Nielsen and its decades-long market leadership in sample based measurement. The skill set and methodologies for real time and real data approaches are different both in terms of technology and cost structure than sample-based methods, and offers an opportunity for disruption in the value chain.`