If you’re keeping up with web-related news, this should be high on your personal radar: Google recently accused Bing of copying their search results. This has been covered on broadcast news, in major newspapers, and of course in the tech blogsphere.
Here are some links on the issue:
Fox News, SearchEngine Land, Matt Cutts (video is very interesting and worth the 40 minutes), David Pogue (NYT, may require registration), Nate Silver (NYT), more here, and here, and of course what started it all.
I posted a variant of this for my students to answer/discuss, and this space is for my take on the issue…
There are two topics here – the original conceit, which regards search spam destroying the basic value of searching; and secondly, the stunning PR coup perpetrated by Google on Microsoft.
I also learned Blekko is a fancy directory service; a competent version of dmoz.
Matt Cutts is a better communicator than Harry Shum, and Vivek Wadhwa and Rich Skrenta appear to be friends.
Onward to the issues… The PR coup is Google changing the subject away from search relevance and into a plagiarism issue for Microsoft. Read Nate Silver’s blog at the NYTimes for some statistical musings on the possibilities, but I think Microsoft got caught…
All toolbars collect data and do an ET-phone-home routine. BUT – so do most anti-virus products, and I expect many of them are selling data back to Microsoft – and this may amplify honeypot results. I don’t think these guys want to get into that part of things and reveal too much about what data traffic really moves around between engines and browsers and toolbars and anti-virus (wireshark is very enlightening). But the Microsoft emphasis on “no one reads EULAs” is a blatant tell, at least to me.
The real issue regarding relevance of search is obviously a topic which Google aims to avoid discussing. Hence we have a lot of dissembling and yapping about algorithmic approaches and so on. Google doesn’t know how to deal with the problem; neither does Microsoft, and neither wants to go the directory route to solve it. But Google does understand it must be solved, and soon.
I am wondering if the spam issue might be the real reason behind the recent shakeup at the top of Google – Schmidt may recognize the real danger while the founders are still blind to criticism.
Back in ’02 I attended a conference for the professional journal industry (mostly medical but some engineering thrown in) and recall a speaker who predicted the eventual demise of Google – indeed he felt all “free” search engines would eventually fail – due to the internal conflicts of the basic business model.
I think we’re beginning to see this play out.
The next-gen search will have a subscription model – pay a fee and block the spammers or search for “free” and accept the less useful results.
I agree there are likely technological methods available to block the spammers, but the incentives have to be there to implement them. I’ve often thought selective economic boycott would do wonders to the email spam problem… along with public execution of some of the purchasers of spamvertised goods.