Is Microsoft Screening Its Search Results?

I think I might have stumbled onto one reason why Google (GOOG) hit a wall with AdSense - the same reason why the proposed acquisition of Yahoo (YHOO) by Microsoft (MSFT) should be opposed.

I publish a half a dozen websites which all have something in common - they are non-commercial and family friendly, and all carry Google search and AdSense. These websites used to be indexed and show up in search results at MSN, Google, and Yahoo.

About three months ago, I noticed that these websites completely disappeared on MSN search but are still on Yahoo and Google. Nothing has changed on these sites that would make MSN remove it from its index.

In trying to understand what happened, I used MSN to search for information using common keywords and examined the results. One thing immediately caught my attention - none of the results returned by MSN involved websites carrying Google search and AdSense. There were a few websites listed that carry the ads (but without the Google search box) that were listed farther below in the search results.

Is MSN blocking, or delisting from its index, websites that carry Google AdSense and the Google search box?

If MSN and Yahoo combine, and the combined internet company would have the majority of the internet's traffic, what would prevent MSFT from stifling the competition by blocking from the new MSN-Yahoo search engine those websites carrying Google search and AdSense?

As a web publisher, I'm concerned that the successful acquisition of Yahoo by Microsoft will force me to drop Google search and AdSense if my sites were to show up in a new MSN-Yahoo search engine.

I'm bringing this matter up because I'm not sure if this situation is unique to me, or if there is a pattern out there. Perhaps this is ongoing on a larger scale? This could be nothing, or this could be the tip of an iceberg.

Disclosure: none

Comments

[MuddlingInvestor]

You are right on point! My blog is excluded from MSN results as well, when showing first on both Google and Yahoo for "Muddling Investor" search.

[Chuck]

I went out to google and searched for sherry gil seekingalpha and got about 10 replys back, one for this article.
I went out to msn.com and did a search for the same, and only got Butkis... with suggestions about Sherry Wine, etc.
...
Very Interesting !!!
Very !!

[Sherry Gil]

As a follow up to my article above, I just did a log analysis on one of my websites for this month of February and here is a summary of the results:

The total number of visitors is 35,063 (1,252 daily average)coming from 18,314 unique IP address. The visitors coming from search engine referrals are as follows:

Google - 16,135
Yahoo - 7,360
MSN - 0 (zero)
Ask Jeeves - 148
AOL - 11

Lycos, Netscape, Altavista, Search.Com, Mamma, Excite, AllTheWeb has each less than 10 referrals.

Why zero referral from MSN? If any web publisher would like to compare log results please email me at [e-mail].

[JGHoward]

A search for my name and town brought up my website on Live Search with no problems, and I have Adsense on every page. In fact, I'm impressed with how many other relevant results were returned.

I seriously doubt Microsoft would stoop that low. It's these kind of tactics that people revolt against with fervor. Take a look at how badly Comcast is doing because they are alledgedly filtering Internet content. It's all over the media, and Microsoft has enough problems without that kind of attention.

[Sherry Gil]

What I wrote in my original article is that

"One thing immediately caught my attention - none of the results returned by MSN involved websites carrying Google search and AdSense. There were a few websites listed that carry the ads (but without the Google search box) that were listed farther below in the search results."

OK, let us examine JGHoward's website. His website carries Adsense but not the Google search box so based on my comment his website could still show in the MSN search engine but farther down the line.

I searched for the keywords MEDIA DESIGN AND PRODUCTION which is the exact title page of his home page so this should rank very high on the search results.

In Yahoo his website was ranked 7th, on Google it was ranked 34th, but on MSN it was ranked 79th.

I keep the printout of my test search if anyone interested to see.

[Scott Berry]

Hmm. If you switched to FIOS TV, and somebody pointed out to you that there were channels and programs that weren't being delivered--perhaps even a major network--how long do you think it would be before you changed provider to satellite or cable?

If in fact MSFT is filtering, they're shooting themselves in the foot for sure. By default they are not returning the best search results, and after all people us a search engine to get results, not ads.

[Sherry Gil]

In today's world where there are thousands of websites containing information on a certain topic, it is almost a certainty that there would be several that would have the same degree of relevancy to a word being searched.

Say, if I searched for the word WXYZ and there were ten results with the same 90% relevancy, and for some reason two of those slipped out of the list, would I notice? No. Would I still get what I was searching for? Yes, because there were still eight others with the same degree of relevancy. And most likely, I would not need all eight of them anyway - half would probably serve my purpose. So, who would notice if any subtle screening were going on?

[Jon Clark]

I also did a search for my website containing adsense. In a very competitive market I am #13 and #17 respectively for my main keyword out of 18 million and 174 million plus pages in Google and Yahoo respectively. Not a thing in MSN....???

[Sherry Gil]

What is intriguing about all these is that I noticed the disappearance of my sites in MSN only in recent months. And not long after, Google hit a wall on its Adsense and Microsoft make a bid to acquire Yahoo. Coincidence? Sounds like a perfect storm to me.

[David MRowell]

Interesting theory. I've always noticed an enormous disparity in traffic to my site from Google vs other search engines.

My site has both Google Adsense ads and the Google Search feature on just about every page.

I just did some checking, and here is a table of results for referrals from search engines for the last week in Feb in both 08 and 07. Apologies for the screwy formatting.

Year 2008 2007
Google 66,000 66,000
Yahoo 2,860 2,812
MSN 547 1,466

Google numbers are estimates because my web analysis program lists out every different google domain (for different countries) separately, and I get a bit lazy when it comes to adding in, eg, 24 more referrals from Lithuania. :)

You can draw your own conclusions from these numbers as you wish. Suffice it to say that Google and Yahoo referrals remained close to identical, while MSN referrals dropped to a third their earlier level.