TCS - Search Engines 101

Search Engines 101

by Lloyd Colston
Tulsa Computer Society
From the August 1999 issue of the I/O Port Newsletter

http://lloyd.colston.com/library.htm

Not all "search engines" are search engines. For our discussion today, we will be talking of search engines some of which are directories and only that. Let's start with the first directory on the Net.

Scott Yanoff's Special Internet Connections was formerly known at the Internet Services List. It has not been updated since 1995 but still has some value for the beginner.

Following this effort came Yahoo. Most people think of Yahoo as a search engine when in reality it is a searchable directory. It adds features left off of Yanoff's work such as frequent updates, customizable content, and a search capablity (not to mention the advertising around it). Another directory is The Open Directory Project which looks a lot like Yahoo used to do before advertising hit. Like Yahoo this service utilizes real people to search the net and edit the pages. The value to this is the content is kept up-to-date with very few dead links, i.e. links that no longer work.

However, as the net grows, the directories became harder to maintain. Thus began the age of the true search engine. This device, mostly unattended, searches the net for content and indexes it where it feels one should put it with minimal human involvement.

Examples of Search Engines are:
Magellan
Infoseek
Altavista

As the net has grown, some consolidation has occurred. While there are a great many search engines besides these, some have been bought out by larger firms seeking to capitalize on the advertising potential of the net. The Open Directory Project already mentioned was started as NewHoo just a few months ago but is now part of the Netscape network. Magellan is now part of the Excite.com network. Infoseek has now become part of the Go.com effort. Even Altavista has had its part in the consolidation effort.

Focusing on AltaVista, this search engine is the most comprehensive yet most overwhelming for a new user. It is so difficult that they have started modeling a "Ask your question in plain language" format stated by AskJeeves. No longer does the user have to know Boolean searches, plus or minus signs, etc. Just ask a question and you get an answer. Correct answers still cost more.

Altavista was so overwhelming to the beginner that a number of online tutorials exist. Some of the better ones are Welcome to the AltaVista Search Tutorial, Using AltaVista - A Tutorial, and AltaVista Search, tutorial.

AltaVista, being one of the most comprehensive engines, is also one of the most overwhelming because it gives one so much information. When searching AltaVista for "Where can I find an altavista tutorial?" in plain English, AltaVista found 5,977,500 Web pages. Thankfully it ranks the most relavant toward the top.

At the next level of search engine is the multiple-search-engine-search-engine. These sites search ten or more search engines and return only a handful of responses to the user.

The most popular are:
MetaSearch
MetaCrawler
DogPile

Of the three, MetaSearch is the less featured because it requires you to click each search engine after it has formatted the search for you. DogPile is slower because it returns parts of the searches to you in separate windows. MetaCrawler is the most comprehensive, returns the searches to you in a reasonably short period of time, but has come to the advertising frenzy of the Net (much like is found at the bottom of this page).

All three share the same cause and that is to bring you the most accurate information in the shortest period of time. MetaCrawler and Dogpile seem to do a better job of that effort.

There are search engines that cater only to UseNet and Mailing lists. There are search engines that focus only on Asia, Europe, etc. However, if you learn these few and focus on only one or two of them, the reader will be far along in their own Net Adventure.

If you need further information, please feel free to email the author.