Objectively Comparing Competing Search Engines?

aendeuryu asks: "My default search engine of choice is, like most of you I assume, Google. That said, some complaints about Google over the years do seem to have some merit -- basically, that sometimes the indices aren't always updated, that it's too easy to manipulate via googlebombing or legislation, and that maybe too many of its featured services never get out of beta stage. Maybe the fact that Google has gone so long without significant competition is enough to make one at least begin to ask questions about it possibly becoming stagnant. Personally, I'm so used to doing things the Google way (and achieving acceptable results quickly) that I'm not really interested in switching -- case in point, all the above links referenced were quickly found via Google. However, what am I missing out on by not giving (for example) Yahoo search a shot? Or, more to the point, how would one go about trying to effectively and objectively compare competing search engines? In what areas have people found Google to have become obsolete for their purposes? Have less ignorant people than myself figured out ways to test a competing search engine's efficacy for themselves?"

Dont bother

[nb caffeine]

If you know how to use google to achieve your results, whats the issue? If a better search comes along, im sure it will be posted on slashdot (twice), so you dont need to worry about missing out.

Re:Dont bother

[TyfStar]

I will agree that 99% of the time, google is perfect for what I need. Stick a few words together, add an extra one, and VOILA, you have what you're looking for.

The times that I have had problems is when I am not exactly sure what I am looking for in a few quick words. I can put it together in a question, such as "What is my house in Utah worth?" or "Why are flamingos pink?".

in those cases, I usually do ask.com That will get me going on a few pages, at which point I will know more clearly what I'm looki

Re:Dont bother

[ShieldW0lf]

Thing about google that really pisses me off is that for a lot of the things i search on, it pops lots of sites that require pay registration and doesn't make it clear. I'm not going to pay for these sites when there are so many free resources available to find my answers, but I have to waste my time visiting them over and over. Experts-exchange [experts-exchange.com] in particular really pisses me off. I don't know if there's something better out there, but I'm actively looking for one, because using google just wastes too much of my time.

I know everyone loves google, and I use it too, but I find that where it used to be an efficient way to find information, it's becoming less and less so as time goes on because of this sort of crap. As far as I'm concerned, if I need to pay to access the information, google should not be indexing that information and putting up links to the sign up page for me to waste my time with when the answer is already freely available elsewhere and that freely available source is in their index. If I wanted to use pay sites to provide my answers, I wouldn't be using google in the first place, would I?

Re:Dont bother

[Jaycatt]

Oh, don't get me started about Experts-exchange. Most of the time someone there is asking the exact question I'm trying to find the answer to. However, once you've paid (and I did, once, because the person I was working for was under time pressure) you inevitably discover the "answer" is something you're already tried. Just try getting your money back. Of course, after registering, you can always ask your own question. Didn't help me, though, since I couldn't wait a week (or however long it takes) for

Re:Dont bother

[CvD]

Um... yeah, you can pay, but you can also scroll down about 2 pages of advertisements and a listing of catagories and find the answers lodged beneath. Unless there are yet more answers to these questions?

For example: a question about Java [experts-exchange.com]. The question first, then the SIGN UP! bla bla, then a bunch of catagories, but if you scroll down further, you'll find answers to the questions, including the 'accepted answer' and such.

Hope this is useful to someone.

Cheers

Re:Dont bother

[Peer]

Experts-exchange.com does require registration to see the answers, but the google-cache always contains the full page with the answers.

Re:Dont bother

[iamhassi]

duh, google it [google.com]... answer [64.233.179.104] found on the first page:
"Flamingos are not born pink. They are white at birth. However, a substance -- called carotenoids -- in the foods they eat produce the bright pink color.

Flamingos would lose their shading if they could not eat carotenoid-filled foods like plankton, shrimp, or -- as handlers at the Philadelphia Zoo have found -- carrots. "

Re:Dont bother

[aconbere]

While I'm sure you had good intentions when posting this I decided to do my own little research (just to see what the other search engines returned) and I got rather different information that you did.

These are the top 4 results for "Why are Flamingos Pink?" (entered without quotation marks) in the top 3 search engine providers Yahoo!, MSN, and Google.

Yahoo!

http://www.finelinefeatures.com/pink/
http://www.shopping.com/xGS-Pink_Flamingos~FD-113~ NS-1~linkin_id-3058014
http://www.thewildones.org/Animals/flamingo.html Contains Answer
http://www.overstock.com/cgi-bin/d2.cgi?cid=48422& PAGE=PRODUCT&PROD_ID=962864&fp=F

MSN

http://199.216.204.14/project04/legends2004/why_fl amingos_are_pink.htm
http://home.nycap.rr.com/useless/pink_flamingo/
http://www.cat1234.com/id56.htm
http://www.straightdope.com/columns/010518.html Contains Answer

Google

http://www.straightdope.com/columns/010518.html Contains Answer
http://www.thewildones.org/Animals/flamingo.html Contains Answer
http://www.amazon.com/exec/obidos/tg/detail/-/0060 000244/104-8146884-9815145?v=glance Sells Answer
http://webexhibits.org/causesofcolor/7.html Contains Answer

As we can see... google out preforms all three, offering 3 sites that actualy contain the answer in the top 4 results, two of which are in the top two. CLEARLY providing better results on at least this topic than either Yahoo or MSN.

Anders

Re:Dont bother

[anagama]

• overstock.com

Sites like this come up all to frequently, even in google. What be really sweet, would be a way to EXCLUDE certain sites. Maybe it's already possible ... I'm sick to death of the cruft that shows up in a search.

Re:Dont bother

[tepples]

What be really sweet, would be a way to EXCLUDE certain sites.

For each site that you want to exclude, add a term along the lines of -site:overstock.com to your query.

Re:Dont bother

[joeykiller]

[Google] CLEARLY providing better results on at least this topic than either Yahoo or MSN.

Yes, at least on that topic. But here's my favorite: Try searching for WHO IS THE WHO on Google and Yahoo. Google is nowhere near an answer on page one, but Yahoo's first result is The Who's home page.

I know that this probably is caused by algorithms and how the different search engines treat stop words, but still: It seems as if no one search engine is best at everything yet, although Google currently (probably)

Re:Dont bother

[Hank the Lion]

When you are looking for a phrase, instead of loose words, enclose them in quotes on Google.
Googling for "The Who" gave me mostly relevant results.

Re:Dont bother

[joeykiller]

If you had read the entire thread, you'd see that the topic was searches phrased as questions - "Why are flamingos pink", etc. Someone pointed out that ask.com was best at answering these kinds of questions, while someone else proved that Google was best.

I merely tried to point out that in some cases -- such as this -- searches phrased-as questions can return no relevant answers at all on Google.

Another thing: I may have composed my search in an idiotic fashion. But don't you think most people are idiot

Re:Dont bother

[iBod]

>>I find the most effective approach for me is to sue Google by default

What a litigious world we live in where you have to sue your search engine (by default) when you don't get the results you want!

Re:Dont bother - why? Parallel to OS Wars

[drhamad]

I know how to use Windows to achieve the necessary results better than the Mac or Linux. Does that means I should never try to use the Mac or Linux? Does that mean that I won't achieve better results if I learn to properly use the Mac or Linux?

Re:Dont bother

[aminorex]

> it will be posted on slashdot

Or at least mentioned in the comments: vivisimo.com

Re:Dont bother

[oni]

I can remember when searching for a page on the internet was like searching for a fish in the ocean

heh, I remember when we had to prepare our gopher searches on punch cards and wait days for machine time to run them, only to find that the research paper we thought we'd found was actually ascii porn with little popup jcl terminal windows selling "CHEEP A5PRIN" (because nobody had invented viagra!). And once you're name got out there, your bitnet account would be so full of spam that you wouldn't even want to use your wyse terminal! But you know what? We were thankful for the opportunity to be on the Internet.

you kids today...

Re:Dont bother

[OldManAndTheC++]

I remember when we had to prepare our gopher searches on punch cards and wait days for machine time to run them

Meh. You think that was bad? Why, I remember when we had to hardwire our Internet searches on plugboards and read the results off of a teletype. Let me tell you, it was pretty tough rendering a web page on a machine without any memory. And every now and then some joker would wire the AC from the wall into a board just for laughs. No, we didn't mind the odd electrocution - it was all part of the fun of the Golden Age of computing.

Back in those days, spam was SPAM, and it came in a can. And we liked it!

Damn kids are soft these days...

[Thud457]

I remember having to walk uphill in the snow both ways to the mailbox to mail my google queries in! [fury.com]

dogpile.com

[Bad-JuJu-Man]

personally I prefer dogpile. I like the organization of results much better.

Alternates

[hambonewilkins]

Here's my own experience.

Yahoo search is okay, not as nice as google, but a good second.

Alltheweb.com has found things google hasn't, but in general I rarely use it.

I rarely use MSN because it was awful all the times I tried it. Same for Altavista.

In general, if I'm searching for something I'll use google first and then Yahoo and Alltheweb to catch anything that google may have missed.

Re:Alternates

[lowrydr310]

If Google doesn't find it, then it's not worth looking at...

I know that's not true, but generally if what I'm looking for isn't in the first two or three results pages of Google, then I give up.

This has only happened to me a few times (not finding what I want with Google), however it does bring up an interesting point. I trust Google results so much, is it possible that all the search results can be misleading or wrong information?

Re:Alternates

[krgallagher]

" Here's my own experience."

Here is my alternative. It is called Copernic Agent [copernic.com]. It is a desktop application that searches multiple search engines returns the results sorted by relevance. It will then let you further refine your search by searcheing aginst the actual pages in the result list. There is a free version that is crippleware. I bought the personal version and it was my favorite tool for searching job sites when I was unemployed.

Bizarre MSN search results

[Anonymous Coward]

I was looking through my website's logs and noticed a ton of MSN bot hits. Then I noticed one coming from their search page. The search term was "UTC+flash" and my site was listed third in the search results.

My site has nothing to do with UTC or Flash. Turns out, it indexed my lame little archive page that displays article dates in UTC format. One of the article titles was something like "Flash Storm," so it indexed the "UTC" portion of the previous article's date and the word "Flash" that began the ne

astalavista.box.sk

[cyfer2000]

Some times I used astalavista.box.sk [astalavista.box.sk]

Re:Alternates

[X]

This is an interesting demonstration of just how much subjectivity effects perceptions about search engines.

Alltheweb.com produces the same results as Yahoo search (basically ever since Yahoo merged with Overture). Yet you describe them as being distinct and with different qualities. You even will search on one after searching with the other.

Re:Alternates

[XorNand]

While your list is a good start, Phil Bradley has an excellent, and extremely comprehensive, analysis [philb.com] of how to use different search engines to find different things. Definitely bookmark-worthy.

I use Google for almost everything

[Snarfangel]

...but I have to admit the AltaVista search engine for pictures is pretty nice. I use that when I want to search for pictures of a particular size for wallpaper.

Re:I use Google for almost everything

[Tribbin]

Try deviantart.com for wallpapers. I can spend hours just clicking along these art-galleries.

Re:I use Google for almost everything

[cbreaker]

The only problem with Deviantart for me has always been that it's painfully slow, from whatever connection I use (home on Cable, work on a pair of OC3's, friends house, etc.) Maybe some times of the day it's better, and I always hit a slow time. Who knows.

But the site is very cool and there's some great stuff on there making the wait worth it.

I quite like Google.

[Morlark]

Now sure enough Google has its faults, but I do still use it as my primary search engine. I do dislike Google never-ending cookies, so I've blocked them, and my Google bookmark contains all my preferences. I've not really noticed any problems with Googles indices not being updates (except in the silly image search, and I don't really use that for any serious purposes). Having said that, I also do find Yahoo to be a very acceptable alternative. I should probably try it out more so as to see how they compare

Re:I quite like Google.

[nkh]

I hate Google: I don't want Google's never-ending cookie so every time I try to search something, Google redirects me to a completely random local version (fr/dk/se/de/...) when I just ask for google.com! The second problem is that this random page shows me results in the local language with a higher priority than the english (standard) pages.

I haven't found a good alternative to Google though...

Re:I quite like Google.

[neil.pearce]

if you visit Google's No Country Redirect [google.com] page, it'll set a cookie to stop you being redirected to your national page when visiting google.com
Not sure why you end up at different fr/dk/... domains though

Subjective

[fembots]

I think you have said it already, Google is good for returning acceptable results quickly, but acceptability is something very subjective.

Even by comparing keyword search side by side, one can still consider a worse result better, but who's to judge except the user?

I kept using Yahoo until it's not giving me results that I think are good enough, then I switched to Google, and I'll keep using Google until it's not returning good enough result.

Appalling

[Anonymous Coward]

Hello.

I have been browsing your internet site for several hours and am generally impressed with your coverage of IT related issues. However, when I saw an article on Google I just had to voice my opinion. I would just like to say how increadibly appalled I am with the Google internet search engine. My main concern with Google is how easy it makes for malicious people to find information on the now illegal Bittorent computer software.

Some background information on Bittorent and what makes it so dangerous:
1. The Bittorent computer software allows distribution copyrighted material.

2. In doing so it inadvertently causes excessive use of bandwidth. Now you might say that this is fairly harmless, but is it really? The effects of electromagnetic radiation pollution caused by this cannot be underestimated. Just think of the millions of wired and wireless connections lighting up and emmiting those deadly electromagnetic rays and all the innocent men, women and children being exposed to them.

Every bittorent user has blood on his (or hers) hands. From this point on, I am boycotting Google and advise any person with a shred of decency to do so too.

Re:Appalling

[adjwilli]

Every bittorent user has blood on his (or hers) hands.

Is this person serious?

Re:Appalling

[PoPRawkZ]

Err... that isn't blood. Time to get out the medical dictionary and learn to distinguish different bodily proteins.

Re:Appalling

[madstork2000]

I hope this gets modded +1 Funny, I have not laughed so hard.

It took me a while to get that this was satire. But just incase it wasn't a funny satirical post but instead a trolling astroturfer; I'll explain it better of the overzealous sladotters out there who are going to rip on this guy without comprehension.

If we follow his his warped logic we should boycott everything for example:

Here is background on the trucking industry and why it is so very dangerous:
1. Trucks should be banned because they allow

Just stick with what works.

[StikyPad]

Personally, I'm so used to doing things the Google way (and achieving acceptable results quickly) that I'm not really interested in switching -- case in point, all the above links referenced were quickly found via Google. However, what am I missing out on by not giving (for example) Yahoo search a shot?

I ask my wife the same thing. Honey, I'm used to doing things your way.. and I always get acceptable results from you.. but what am I missing out on by not giving (for example) Veronica a shot?

At least Google will never make you sleep on the couch, or give them half of all your assets. Hopefully.

I tried others...but I never changed my home page

[nsasch]

I open my browser, and see the Google page up and running. I started with Yahoo, I tried meta search engines, altavista, a9, and many others, but I never change my home page to be the other ones. I know Google, I know how to use the results and to view pages all in HTML and to get the cache and to search sites that link to me, or search a specific site. It's easy in the other sites, but I already figured Google out. Google works for me, when I find the wrong thing, I just add "-wrongword" to the end and I find what I need. I see all the blogs and misindexed pages, but I've never really suffered from Google Bombing or any of the other problems that are mentioned.

Why not find out ....

[TheViffer]

Alternative search engines [google.com]

Sarcastic answer

[hambonewilkins]

Go to google and type in "better search engines"

Re:Sarcastic answer

[ZP-Blight]

I followed your logic, searched for this term and google brought up this article! Now i'm trapped in an endless loop, help!

Re:Sarcastic answer

[Secrity]

Ewwwww, I did that and clicked on "Web Searching Tips" [searchenginewatch.com/facts/], now I feel so dirty and I just know that I caught anything.

Try this..

[Anonymous Coward]

This should give you an answer [justfuckinggoogleit.com]

Re:Try this..

[jonbryce]

which tells us that Lycos and Altavista are better search engines than Google?

What I use

[jonbryce]

If I am looking for adverts, or to buy something, I will use Yahoo. It's classified ads sections are much better than anything Google can offer.

If I am looking to buy something offline, I use yell.com.

If I am looking for software, I use something like freshmeat or one of the rpm search facilities.

Otherwise, I use Google.

Try Yahoo

[AndreySeven]

When I am not getting satisfactory results using Google(about 30% of the time), I try Yahoo, and I usually find what I am looking for. If this keeps up, I might start my searches using Yahoo.

Precision and Recall

[xyzzy]

Unfortunately, comparing search engines is a nearly impossible task, since they probably aren't indexing the same data.

When you measure a search technology, the values you typically look for are precision and recall. precision says "of the X results you gave me, how many of them are relevant". recall says "in the world, there were Y possible pages you could have found, but you gave me X of them".

you can't measure recall for a public search engine, but you can measure precision. Take a set of sample queries, and some users. Have them perform the queries, and go through the first ~100 pages and give them a "thumbs up" (relevant) or "thumbs down" (not relevant).

Your overall score will measure precision: if at N=100, all 100 were relevant, that's 1.0. if only 50 were judged relevant, precision is 0.5.

You can estimate recall by judging say 1,000 documents (phew). Then sample precision at N=10, 100, 500, etc, assuming that is an "exhaustive" list of documents in the world.

A simple way to test recall

[benhocking]

If your name is somewhat rare, search on your name in the search engine.

Re:A simple way to test recall

[xyzzy]

Unless you know for sure all the places that your somewhat rare name is mentioned, and that the search engine you're working with has visited the site, this won't work.

Needs to be more complex though

[Sycraft-fu]

Because relivance is more complex. There's a number of additonal considerations:

1) HOW relivant is a page, and is that page more highly ranked? It doesn't do me any good to have 99 slightly relivant results and 1 highly relivant result, if that one is at the end. So you have to measure how relivant the page is, and how high it appears in teh search and weight that.

2) The ability to find the correct page. Sometimes it's not that you are looking for general inforamtion on a topic, there's a specific page yo

Re:Precision and Recall

[xyzzy]

Just take the first 1,000 that come out of the search engine when you give a query!

Metacrawler.com

[cthrall]

Try a metasearch [metacrawler.com] and let the server figure it out.

Re:Metacrawler.com

[cmclean]

Try a metasearch and let the server figure it out.

So I hit up metacrawler.com for "sendmail tips". Just for the heck of it.

Result #4: Tips on EBay, Find Tip items at low prices.
Result #5: ServSafe Alcohol (R) Training Program, Comprehensive interactive training for those who serve alcohol.

Erm, what the hell? Leaving aside the fact that these are sponsored links thinly disguised as real results, they seem to lack relevance somewhat.

Re:Metacrawler.com

[TykeClone]

Result #5: ServSafe Alcohol (R) Training Program, Comprehensive interactive training for those who serve alcohol.

That may just be telling you that sendmail can drive you to drink

Re:Metacrawler.com

[gte910h]

ServSafe used to be called TIPS

Re:Metacrawler.com

[ThisIsFred]

Result #5: ServSafe Alcohol (R) Training Program, Comprehensive interactive training for those who serve alcohol.

Oh, by the way, if anyone is interested: I'm running a ConsumeSafe Alcohol (R) Training Program, whereby I offer comprehensive "interactive" training for those who wish to consume alcoholic beverages. Just bring a 12-pack with you and I'll show you how it's done.

Re:Metacrawler.com

[2short]

Huh, I never would have guessed MetaCrawler was still around. They used to be my search engine of choice. They generally returned a link to a page that had the information I wanted somewhere in the top 5-10 hits. But sometime in 99 I realized I wasn't looking through the top 5-10. I was just picking the right one, almost without fail. Without even realizing it, I was scanning down the list, looking for the magic words "found on Google". So I ditched the middle man.
Since then, when I haven't found w

Teoma

[xzap]

Teoma has this great feature called Related search which is very useful. Basically if you look for a particular topic, the search engine identifies all related topics and offers you a one click access to all of them. Makes the search equally usable for both a rookie and a domain expert using the same search term.

Presentation

[dwcasey]

One thing I like about askjeeves and a9.com is the way the present the search results. I think the next step is to improve on the presentation of the results (data) to make it more usable/accessable. Hit up askjeeves and run a search. The preview feature is pretty nice. And check out a9.com searches with their Site Info mouse-over.

I hate to say it, but...

[Raindance]

I hate to say it, but I think your quest to directly compare search engines "objectively" is pretty problematic.

Frankly, I think you're on the right track when you ask, "What am I missing out on by not giving Yahoo search a shot?"

Likewise, I think you're on the wrong track when you go on, "Or, more to the point, how would one go about trying to effectively and objectively compare competing search engines?"

Comparing the results of searches is necessarily subjective. Only that first question has a real answer.
RD

Re:I hate to say it, but...

[xyzzy]

So you are saying that there is no yardstick by which you can compare search engines? I don't believe that to be true. It's just hard.

If there isn't, how are you going to answer your question #1 -- gut feeling? By missing out - do you mean parts of the web? usability features?

As I said in an earlier post -- it's nearly impossible. But that doesn't mean you can't come up with a reusable metric to make an objective judgement.

Search Engine Watch

[br0ck]

These types of issues are discussed ad infinitum at SEW [searchenginewatch.com].. particularly in the forums.

Any algorithm can be gamed

[Ars-Fartsica]

This is the dilemma for any centralized algorithm, as soon as you are number one you are exploited, thus relatively increasing the utility of as-of-yet unexploited competitors.

Here's a nice comparison

[Zordok]

I got this from a friend who works at yahoo...

http://www.langreiter.com/exec/yahoo-vs-google.htm l [langreiter.com]
Sorry if it gets slashdotted.

Re:Here's a nice comparison

[jericho4.0]

What an interesting tool.

hahahhaha

[MattW]

I think the coolest thing here is to enter "search engine" so you can compare what they think of each other easily ;)

Listen to the Buzz

[femto]

You don't have to bother evaluating better web based technologies. When they are worth using others will tell you about them. It's the nature of the web.

For example, a professor of the university department in which I worked came back from Digital Research Labs, enthusing about a great new search algorithm the designers of Digital's Computer Aided Design software had come up with. A short time later Altavista was 'it'.

The same happened a few years later. The buzz from collegues and those on the web was about a new search engine called Google.

The short answer is, "Don't go looking for the 'next search engine'. It will find you."

Wikipedia

[chiapetofborg]

I love wikipedia. I basically use it as my default search. Unless I think that the question I have is non encyclopedic. acronymfinder for acronyms, babelfish for translations, imdb for movies, and well, for everything else I use google. It has integrated everything else I need. Yes it is subjectable to googlebombing and similar ilk (I should know, I work for a SEO company), but its *way* easier to "hack" Yahoo, MSN, Altavista and others. Googleboming is much harder (and therefore more reliable) than the others.

Punctuation

[Daetrin]

If in the process of comparison you find a search engine that can actually handle punctuation please let me know. Altavista used to be able to do it, but sometime in the last few years in the process of "competing" with google they dropped the feature.

I _used_ to go to altavista everytime i had a search that involved specific punctuation, usually some kind of coding question. Now i just get frustrated with google while trying to find some related term i can add in that will give me the results i want.

Google Is Good Enough For Most

[TheFlyingGoat]

While most of the complaints about Google are warranted, Google works well enough even for those of us in the computer industry. If 99% of the time I find what I'm looking for in the first page of links, and the results are returned quickly enough, why would I go elsewhere?

Yahoo and Altavista worked ok for me before Google came along, but the clean interface and good results drew me in. So, the only thing that would convince me to switch to a different search engine would be if Google started cluttering

Why Google works

[onyxruby]

OK, here it, why Google has become king.

1. Simple interface, quickly loads.
2. No graphical Ads
3. Paid results are clearly ads and seperated from real results.

That's it, that's why Google is king. Until Yahoo, MSN search, Ask Jeeves and the like get those three points, they will continue to be second fiddle.

Re:Why Google works

[jerw134]

1. Check out MSN Search [msn.com]. Maybe not quite as simple as Google's, but it's far from complex. It also loads very quickly.
2. MSN Search has no graphical ads.
3. MSN Search separates the paid results just as clearly as Google does.

So, when was the last time you looked at MSN Search? Last year?

Re:Why Google works

[jerw134]

Hell, my previous post applies to Yahoo Search [yahoo.com] as well. If you're going to be a Google zealot, at least try and make some points that are actually valid.

One way to test

[mcguyver]

I usually test search engines by typing in popular keywords that spammers generally go after, ex:

phentermine
home loans
poker
mesothelioma
viagra
miserable failure

Then look at the sites that rank at the top. It's very easy to tell which search engines are more succeptible to manipulation. A quick look at the backlinks for sites favorably ranking in those competitive keywords tells you how that SE is doing.

Here's my opinion on the race between Google, Yahoo & MSN. Google has more sites that are authorities in the top results and Google penalizes over optimization however extreme examples of over optimization continue to show up in Google. Yahoo is a moderate success and does a fair job of filtering out spammy sites as well as authorities like wikipedia - wikipedia will always rise to the top in G but not in Y - and this is good for Y because you get more variety. MSN does an average job of filtering out blog spam but new sites are too favorably ranked and this is because MSN is new and has no recorded history of URLs. My personal preference is to use G simply because it loads the fastest in my browser... Maybe it's also worth pointing out that my company has several URLs ranked favorably in the terms listed above - looking at the change in rankings over time certainly helps give insight into which SE is better. MSN & Y are by far easier to manipulate than G but G gives the most traffic.

search.yahoo.com

[popo]

Many people don't realize that Yahoo! has a scaled down (Google like) search interface which is actually pretty sweet: http://search.yahoo.com

Lately my Google results have been so Google bombed that I've been going back and forth between the two. I can't say for sure yet, but I may be in the middle of a bit of a personal transition.

Depending on what you're searching for, Google is often so front-loaded with dead-end advertiser links that its results aren't really worth much. Although it has to be said, it depends what type of a search user you are, and what types of things you're looking for.

Google is still the king of advanced search.

Vertical Search

[tobes]

Google is great for finding web pages, but why limit your searches to such a broad domain? Vertical search seems to be where all of the innovation is occurring. My guess is in the future we'll have very specific search engines tightly integrated with their relevant platforms.

For example you could do your music searching on your iPod or stereo, your yellow pages searches on your mobile phone, your video searches on your pvr. Of course it makes sense to expose a web front end to these engines as well, but it

3 cheers for objectivity.

[GeorgeMcBay]

Nothing screams objective like this article displaying the Google logo.

MSN's sandbox test searchpage

[standsolid]

I personally think Microsoft's sandbox search engine front-end is pretty nifty.

Too bad the search results aren't nearly as up to par as google's results (in my opinion)

http://start.com/1 [start.com]

Simple Method

[Salamander]

I've stuck with Google for a while, but I used to do surveys pretty often. My approach was to start preparing a couple of days in advance, by keeping notes about things I was searching for. Then I'd take three or four of them, usually the ones that I'd had the most trouble refining, and try them out on a bunch of search engines. For each, I'd keep track of how many searches I had to do and how many junk pages I had to get through before I could get to something useful on that subject. It usually became clear pretty quickly which search engines were allowing me to make efficient use of my time and which were wasting my time.

Another thing you might want to do is check out some of the newer "clustering" or "concept map" search engines such as Vivisimo or Kartoo, to see whether they suit your searching style better. They're really quite different from the search engines we've gotten used to, so the metrics I just described don't quite work for them. That doesn't mean they're better or worse - just different.

Yahoo seems lazy

[MattW]

Since I tend to have to do some SEO for sites, I tend to keep an eye on how search results are returned... one thing I've noticed is that Yahoo seems far more easily manipulated by URLs - ie, it seems to weight something like, "www.goats.com/goats" high for the term "goats" even when the site has little or nothing to do with goats.

Also, Yahoo and MSN both seem extremely poor about figuring out the "right" url to link to. It's almost as if they index the first thing on any domain they come across, instead of trying to figure out where on the site most people link to, so you'll often find yourself deep-linked into a site where you'd prefer to be looking at a higher-level page to start. Google deeplinks too, but it seems to be only when it's really more relevant to the content.

I don't use a9 much, but it seems like google with a different skin. I swear sometimes they're snarfing google's results and storing them. Not that this is all bad, since Google's results tend to be some of the best, but it's still eerie.

Who Cares?

[blackmonday]

"..too many of its featured services never get out of beta stage"

Who cares? Froogle and Google News (and for that matter, Gmail and Google Maps) are functional, and can be used today. Why would you let the word "beta" get in your way? Is there an unfulfilled promise?

basic benchmarks

[OmniVector]

i wrote some benchmarks of the three major search engines (downloadable here: asn1.tar.gz [otierney.net]). Basically google had the fastest results, the most results, and tied with yahoo for the best quality results (actual usefulness of what was returned).

Teoma used to be good...

[Newer Guy]

Teoma (teoma.com) used to be good, but lately I find it's not giveing me a smany choices as google, nor are they as good.

Surprisingly, I still use Ask Jeeves (www.ask.com) for things - and find it finds things that Google has completely missed!

I guess you have to use a combination of several to really find everything you want - though Google by far is the best one.

I base it on bot/spider visits

[KlomDark]

Looking through the logs for my website, I see Googlebot visiting nearly every day, followed (recently) by MSNBot. (Actually, in raw count, I'm seeing that MSNbot has just recently surpassed the number of requests as Googlebot. Would need to do some in-depth analysis to see if those are requests for the same thing over & over, but in raw requests...) I pretty much never see anything from Yahoo cataloging my site.

What's weird I'm noticing is that I don't see anything from something like a Yahoo bot at http://klomdark.servebeer.com:443/analog/report.ht ml#browsum [servebeer.com], but Yahoo is giving more traffic (http://klomdark.servebeer.com:443/analog/report.h tml#refsite [servebeer.com]) than MSN.

Google still leads however. I wonder where Yahoo is getting it's data, unless it's from a crawl previous to fall 2003, as I'm not tracking logs from that far back. Strange.

Search Engine Watch

[Guidii]

Or, more to the point, how would one go about trying to effectively and objectively compare competing search engines?

For some time now, Search Engine Watch [searchenginewatch.com] has provided a good editorial and comparison on various search engines. They focus on marketing topics, but also tend to talk a lot about the underlying technology, etc.

A recent roundup of engines is at http://searchenginewatch.com/links/article.php/215 6221 [searchenginewatch.com].

In the old days...

[TheAwfulTruth]

No single search engine had won out so yo uhad a bank of search engines that you always scrolled through. What one engine didn't have another would.

Well a hell of a lot of those "old" search engines are still around! And they have become better over time. Google at one time was so much nicer than the others that people sort of got "lazy" and stopped browsing qround the engines. But everyone else didn't just curl up and die.

So just start engine hopping again. Try Google first if you must, but then try Yahoo, search.msn, alltheweb or search.com or other meta search engines that search all the real search engines for you.

Multiple sources of info have always been and always will be better than one giant conclomerate of info such as Google is becoming.

My search engine interface project

[Dacta]

(Slightly premature announcement coming up.. but hey - it's Open Source so that's okay, right?)

I've just started a (Java) project to interface to a number of search engines. It might be a good place to start if you feel like doing some coding. See https://argos.dev.java.net/ [java.net] - there is no release yet but the code is in CVS.

It currently supports Blogdigger, Feedster, Del.icio.us, Google, MSN and Yahoo (and Google Desktop search). I'd like to include Ask.com, too, but they don't provide a programatic interface and I refuse to screen-scrape.

In my opinion none of the other search engines are close to Google in quality of results. I've found (to my surprise) that Ask.com gives me the second best results (they bought the old Teoma search engine, which was always okay. It had an index almost the size of Google's, which neither MSN or Yahoo can match yet.)

Re:Questions

[TheViffer]

It will cost ya.

Google Answers [google.com]

Re:Questions

[Yurian]

An excellent question-answering engine is BrainBoost. [brainboost.com]. It's currently slow, but damn impressive sometimes.

The other day I needed to know, for obscure reasons, the number of heroin addicts in Dublin. This is the kind of info that you know is probably on the web, but is going to be hard to find with Google.

I used BrainBoost - "How many heroin addicts are there in Dublin?" [brainboost.com], and, bam, first line of the result - "There are 13,000 heroin addicts in Dublin."

That's damn impressive. Out of curiosity I tried to see if I could find the same info with Google - it was fairly tough. Took three or four searches, eventually resorting to

"there are * heroin" Dublin

which is a fairly specialized search that average users probably wouldn't be able to construct. The BrainBoost search, on the other hand, was completely natural, my granma could have done it.

So, thumbs up for BrainBoost for question answering.

Still, it's not the kind of thing you'll want every day. For day-to-day search, Google is the tool, but BB is worth a look.

Re:Questions

[amokrun]

That kind of engines are indeed nice. Still, they have their own oddities. For laughs, I tried to ask the system whether moon is made of cheese.

It so turns out that moon is indeed made of cheese!

"is moon made of cheese?"
"The Moon is Made of Cheese"

I guess it still takes some time before that kind search engines become more popular than the traditional ones.

Re:Questions

[funny-jack]

Interestingly, you also get the same result if you actually do a Google search for your original question: "How many heroin addicts are there in Dublin?" [google.com] In the summary of the first result: "... There are an estimated 13000 heroin users in the Dublin area. ..." I'll give you the benefit of the doubt here and assume that, as you have supposed, these results have changed since you did your searches.

I have actually found searching for a plain english question to work in a number of other instances, as well.

Re:Questions

[somethinghollow]

"You mean like Google Answers?"

I don't think so. Google Answers a) costs money, and b) queries humans instead of a database (and that takes time).

AskJeeves is free and quickly queries a database.

What Google Answers provides should be more accurate, since humans can determine what you mean better; however, pay-per-"search" would out price simple questions like "Who wrote House of Leaves?" AND make them pointless to ask since, by the time you recieved an answer, it probably isn't pertanant. Google Answer

Re:Try them out yourself

[Anonymous Coward]

Yeah! In fact, people shouldn't go to school, either. They should go out into the real world and find out everything for themselves, like how to read and write. Never give anyone advice, and if someone ever asks for advice, you should punch them in the face. Information should never be shared between people. Ever.

Re:Try them out yourself

[Chess_the_cat]

Your ideas are intriguing and I wish to subscribe to your newsletter.

dumb joke, get over it

[Thud457]

Apparently Ask Jeeves reccommends MSN search. [ask.com]

Re:other search engines

[Baricom]

Try this: yahoo! vs. google [langreiter.com]

Re:Clusty.com

[lucabrasi999]

FYI -- Clusty is owned by the folks that created vivisimo [vivisimo.com].

Re:Major reason Yahoo is better...

[nmg196]

-1, Wrong
This is absolute rubbish. Google DOES crawl dynamic pages quite happily. It's crawled all of my sites with no problem.

Neither (no) search engine crawls dynamic sites where there are no links to the dynamic content (eg where you HAVE to search using keywords to find the content) but Google and Yahoo are happy to index any dynamic page which is directly linked to even if it has lots of parameters in the URL. Google has indexed 15000 dynamic pages on a directory site of mine quite happily.