I was forced to carry out this relatively simple search engine relevancy test after one incident: when my wife kept browsing through multiple websites generated by search engines and still could not find what she was looking for, wasting a good part of her Saturday evening. Frankly, I was in two minds while thinking whether I should pick up a low competition keyword or a high competition – but I finally decided to take a VERY competitive word to check where all these Page Ranks, Links, Artificial Intelligence algorithms would bring us to? I selected 4 Search Engines for my test: Google, Yahoo, MSN, AskJeeves. Having done so, I turn off all my search engine optimization knowledge and just turn into a regular internet user who wants to buy… one of those new thing called “satellite radio” (Overture indicates 1279 searches per day (October, 2006), Wordtracker shows 799 daily inquiries). OK, done with the theory, let us turn to practice now. I am going to type in “buy satellite radio in USA”, that is going to be logical enough for someone who wants to buy a radio (not read some news or reviews about it), and buy it in USA not abroad (since delivery to any state is available as long as you have a credit card and have a permanent post address). So, let’s see what will the results be.

OK, first result in Google – C-Span website, telling me what is satellite radio and where the headquarters of Sirius Satellite Radio Inc. is located. OK, thanks to Google, now I know what satellite radio is but still don’t know where to buy one…Second result: America’s number 1 way to buy cars on-line. Fortunately in a small article describing radios for cars I was able to find a tiny link to the page selling what I am looking for – at last! But E-BAY CANADA in the top ten Google – oh, good gracious, that is a blow below the belt, Google completely ignored the last part of my search request “in USA”. What an inexcusable mistake for the search monster! Luckily, Google was able to recollect Amazon and E-bay that saved it from a complete collapse. My relevancy mark: 5 out of 10, with all my respect to the Internet giant.

Yahoo – very relevant result, gives RadioShack, Best Buy and Sears as the first three results – pretty good choice of satellite radios – moreover if somebody asked me to put this trio in the order of importance (based on provided choice and consultation) I would do just the same! I would add here WalMart and Target for a complete list of the biggest stores selling electronic items. Very good results with Yahoo, giving consistent relevant output! Yahoo takes 6 point out of 10.

MSN. Absolutely fantastic results!!! Not only MSN gives me an electronics store website – it takes me to the page selling exactly what I am looking for – satellite radios! Pretty high prices there on Infinitipartsusa (looks like Nissan is doing very sophisticated radios), but let’s say I am a very busy person and pay no attention to its cost. Second result goes to Satelliteradiousa – I get the page with the description of satellite radios and the history of XM Radio – but site’s navigation is user-friendly, I can clearly see that I am just one click away from the page where radios for sale are displayed. Unfortunately third result is just the same site, but this time I see a page telling about the history of Sirius radio. MSN need to be a little more diverse in its results. MSN takes 7 relevant results out of 10.

AskJeeves – best results! Not only do I see pictures of satellite radios, not only do I see their prices – I can even enter my zip code to see local stores deals. Absolutely relevant result goes to AskJeeves with a simple web site called Shopping.com and its page devoted to entirely satellite radios. But second result (I am not taking into account supportive results) upset me – the page was not found, what a failure for AskJeeves! There is nothing left for me but to give 10 out of 10 for AskJeeves. All of the results were relevant; I would be able to buy a satellite radio on any page of the generated results.

Even though all the results were so different there was one website that managed to be present in the results of all four search engines, that is www.usatoday.com. I wonder if this site comes out every time somebody is looking for something “in USA”.

Conclusion. The purpose of this test was not to carry out an elaborate testing of the existing search engines, but to find out how user-friendly they are, taking a random key phrase for this experiment. Unfortunately, we only witnessed that the greater the share of the search engine is, the more commercial it becomes and in the long run more irrelevant. Developers of the search engines are certainly pursuing very difficult tasks, and I honestly respect their work; but it becomes clear that the future of the search engine industry lies in relevant and request-friendly results as the most important criteria of search engine effectiveness and accuracy.