Does Microsoft consider the server environment a site is hosted on when applying relevancy rankings for their
search results? If this were indeed true, wouldn't manipulation of this sort render their SERPs as biased and
Is MSN Search manipulating their search
results to give more weight (relevance) to sites being hosted on IIS environments? Would you even consider using a
search engine that was thought to be manipulating their results? Discuss at
If the study conducted by Ivor Hewitt (which was first reported
/.) continues to yield the results his
less thorough tests did, he may be on to something that could cast some doubt towards one of the more promising search
engines to hit the market in some time.
The story goes like this: in an act of curiosity, Ivor began testing keywords from the Google Zeitgeist in both Google's
search and MSN Search in an effort to document ranking differences. However, the conclusions he developed
indicated a lot more than simple SERP variations:
the whole is seems that the MSN search engine is indeed placing IIS hosted sites higher in the results more frequently than
other webservers. Frequently the MSN search is placing more IIS servers in the important top 10 results than Google even where
result sets from a query have actually returned fewer IIS servers overall on MSN.
Which means, if you follow Ivor's logic, MSN Search may be giving preferential treatment to sites hosted on the IIS
environment. Understandably, Ivor's conclusion is quite troubling and leads to a number of questions: Is MSN
manipulating their search results to give higher rankings to IIS sites? If so, does this not render MSN Search as a null
and void search engine?
Perhaps some other points need to be discussed before Ivor's observations get accepted as gospel.
First and foremost, many critics of Ivor's conclusions immediately noticed his range of keywords was quite limited. In
response to this, Ivor administered the test again, this time using
1000 random keywords instead of the Zeitgeist. However, reactions were the same: not enough keywords were tested to
formulate a strong hypothesis. Ivor is currently re-conducting his test, and this time, he's using 10000 keywords.
However, results from this latest test have not been posted.
Another point made by a number of people has to do with quality level of sites hosted on IIS environments, as opposed to
those hosted on Linux or Apache environs. The general counterpoint indicated perhaps lesser quality sites, blogs, and
vanity sites were being hosted on Apache/Linux, while the more optimized; search engine-friendly sites are apart of the IIS family (to which
iEntry's IT Manager,
Jay Fougere replied: "that's
a straw man argument").
Do you think someone should tell Google and Amazon their respective vanity sites would be better off as a member of the IIS family?
Now, before everyone runs out to purchase their tinfoil hats, consider the additional points made by Ivor:
So what's going on, why the bias?
I have no idea; I doubt it's all a big conspiracy... but some possible explanations spring to mind:
Perhaps the MSN search has simply been coded by developers used to talking to IIS machines and so it just does that job better?
Perhaps the MSN spider is taking advantage of some specific IIS features to provide enhanced indexing?
However, Jay has some thoughts about Ivor's theory as well:
"I don't buy it -- HTTP is HTTP is HTTP -- there is nothing in it that even allows for indexing. I mean a site is made
up of pages with links -- you write a program to crawl those pages (follow every link). There is nothing there to help or
protocol, HTTP, is the same for Unix/ IIS /Apache /insert favorite webserver here. However, each server environment
will return a header with what it is running, if you ask it. Although, many admins tell their servers to announce themselves
as servers they are not -- to confuse hackers."
Although the results from Ivor's extensive keyword test are not available, the early implications from previous tests
have caused some arched eyebrows. However, it's almost silly to suggest such a thing simply because of the public
relations mess MSN Search would have to handle if Ivor's data isn't coincidental. While it's true that MS has had their share
of PR nightmares, one of this magnitude would be quite damaging to their credibility.
No one would use an engine suspected of manipulating their results, and the folks at MSN Search know this as well.