Google's Trends for Websites - Challenge to Comscore and Alexa?

I need to start this post with an important disclosure. I was on the Board of ComScore for nine years from the summer of 1999 until earlier this month. And I am still a shareholder in ComScore through the Flatiron partnerships. In no way is this post intended as a plug for ComScore. And I cannot and will not disclose any confidential information about ComScore in this post.

Now that I have that on the table, I'd like to share my impressions after a first look at Google's new Trends For Websites service. I've been involved in the web measurement market for almost ten years and we use third party measurement services every day in our job as investors in Internet companies. I've blogged about our approach (USV's) to web measurement in the past.

In my opinion, there is no service that gets third party web traffic measurement exactly right. We like to triangulate at Union Square Ventures. We use ComScore actively and believe their data is the most accurate of all the services. But it is not perfect and it is particularly weak at measuring web sites with small (sub 500k uniques) audiences. We also use Compete, Alexa, and Quantcast on almost a daily basis. And we will most certainly start using Google's Trends For Websites.

There are a lot of misconceptions about third party measurement services out there. They are on display in the comments in this Techcrunch post. People say things like "ComScore's data is not worldwide" or "Compete doesn't have good statistical algorithms." Neither statement is true. ComScore, Compete, Hitwise, and NetRatings have been around for a long time and have invested heavily in the statistical talent that is required to get a third party measurement service right. And certainly a bunch of the leading services, particularly ComScore and NetRatings have built large international panels and report international data.

Google's service reminds me more of Alexa than anything else. And I don't think it's particularly accurate. Here's an example. We run monthly queries of all the third party services against our entire portfolio and share that data internally in our firm. I am very familiar with the numbers on our portfolio companies, both what their own analytics and server logs are telling them and what the third party services tell them.

Here are three charts of Twitter, Etsy, and Indeed. I picked these three services because they have large enough audiences which makes them fairly easy to measure by third parties and have been around a while and are well known.

This is what ComScore says the worldwide unique visitor trends have been over the past year:

This is what Compete says:

And this is what Google Trends says:

Since we are investors in these three companies and know what their internal numbers are saying, I can safely say that ComScore and Compete are lower but directionally correct. Google is like Alexa in that they don't report absolute numbers but even if they did, they are not directionally correct on this particular set of companies.

Once again, this story demonstrates that many in the tech blog world don't really understand the third party measurement market. Google announces that they are offering a website traffic measurement service and everyone instantly assumes it is going to be great. I don't think it is, at least not yet.

Mike Arrington points out in this post that,

all the data is being gathered by Google for the product is from Google users (their toolbar, for example), the data for Google’s sites would be skewed to 100% of all Internet users. It points out an inherent flaw in the product, and I’m not sure Google can easily solve it.

This is an important fact. Everyone who provides third party measurement starts out with a data set that is skewed in some way. The trick is to understand how your data set is skewed and apply statistics to take that bias out. Firms like ComScore, Compete, Hitwise, and NetRatings who sell their data have invested heavily for many years in reconciling their data to server logs and internal analytics. And that makes their data better.

If Google wants to be a player in this market, they'll have to invest time and energy doing the same. It will be interesting to see if they will do that. If not, they'll be more like Alexa. A helpful tool - but not one that you can completely rely on.

Comments

[David Jackson:]

Fred, I think the most interesting part of Google Trends for websites is this comment from its FAQ ( www.google.com/intl/en... ):

16. How are you using Google Analytics data?
The Google Analytics data in Trends for Websites comes from the anonymous opt-in data sharing setting in Google Analytics. This setting enables website owners to share their data in an anonymous form. To find out more about this setting, please read the FAQs related to Google Analytics Data Sharing. Individual site level information from Google Analytics isn't currently used in Trends for Websites. Instead, the Google Analytics data is used in an anonymous and aggregate fashion and combined with other data sources to calibrate macro-level insights into website traffic patterns, site visitation across geographies, and related websites and searches.

This means that Google has the potential to use actual, hard data, not panels and statistical extrapolations like the other stats providers.

2008 Jun 22 04:40 PM

[David Jackson:]

Just to clarify the above: Even though Google isn't using site specific data from Google Analytics now, it could easily do that in future as people become more comfortable with sharing data.

2008 Jun 22 04:43 PM

[Matthias:]

And to add a small feature that you might have not noticed:

When you click on "log in" on Google trends (with a valid google account), they also show absolute numbers.

2008 Jun 23 07:49 AM

[B.M.:]

Yet another guy who thinks just because he's on the board of a company he's an expert. I am on the board of no company, and I can tell you for sure that the world's leading search engine has a massive massive advantage when it comes to determining which websites are popular or not. The only reason Google wouldn't be more accurate than all the other competitors, is bc they knowingly choose to NOT use some of their information sources (such as toolbar for instance) in order to promote privacy. Since there are many many humans that would trade their privacy for a nickel, this won't likely be a deterrent.

2008 Jun 23 10:40 AM

[seamtp123:]

Lets not forget googalytics, from which they have hard data on nearly every site ...worth measuring.

2008 Jun 23 11:49 AM