One way to keep your website pages out of Google’s index is to use a robots.txt file – this tells search engines where they can and can’t go (which folders are “disallowed”).
Bing has quite an extensive one, telling other search engines not to waste their time ploughing through its search results.
Unfortunately, when Bing launched www.bing.com/social (Bing’s combined search of Facebook and Twitter updates) back in June, it forgot (I presume) to update its robots.txt file (which had previously, and still does, disallow results from the more limited forerunner – bing.com/twitter).
As a result, a search for raoul gazza gascoigne in Google returns Bing social search results pages at the top.
Bing search results pages are, er, top of Google’s search results
And while Google’s site search operator isn’t that accurate, it appears to suggest that it has indexed 168,000 pages of Bing search results in the last month or so.
A lot of Bing social pages are in Google’s index
None of which explains why Google considers a search results page from one of its competitors full of recycled tweets and updates to be the most relevant results …
You might also like
- BUSTED: bing defender is Microsoft employee – and have they deleted my post on bing’s usability from bing?
- Google shows social connections in Flickr sitelinks
- Want an SEO job? Check out the Daily Mail’s robots.txt file …
- Google’s improved search pages: no results
- Google Circles: a new social network (just not yet)
Leave a comment!
