Your Federal Quarterly Tax Payments are due April 15th Get Help Now >>

Effective Web searching by dfhrf555fcg


									                                    Effective Web searching
Search Engines
Search engines allow you to do a targeted search for specific information. You type in keywords that
are run against a database of Web pages - in much the same way that you may have searched for
periodical articles using an electronic database. But there are at least 5000 search engines, both
general and specialised. So…

Which search engine should I use?
Be aware that:

      Any search engine covers only a small fraction of all the pages on the Web. Even the major
       search engines index at best only about a third of documents on the Web
      Different search engines do not necessarily index the same Web pages
      Different search engines have different frequency of updating
      Some search engines search all the text of a document whereas some search only the first few
       sentences, so different engines give different results and different relevance rankings
      Search engines are developing all the time. Expect them to change

Have a look at
For reviews, comparisons and information about search engines:

      Search Engine Watch - - look at the section "Reviews,
       ratings and tests" which will take you to details of the size of the different engines, how
       frequently they update, how they index, popularity etc.
      Search Engine Showdown - also gives access to
       reviews and analysis.

For details of search engines that specialise in particular subject areas or countries:

      All -
      The Search Engine Guide - and click on "Internet search
      Search Engine Colossus - - excellent international
       directory of search engines

Of the most popular search engines:

      Google - is highly regarded: it covers over 3000 million pages in an
       innovative way
      AlltheWeb - also has in the region of 3000 million pages indexed
      Alta Vista - (over 1500 million pages indexed), is old-established
       (1995) and has good advanced search features.
      Hotbot - is medium-sized (approx 1000 million pages indexed)
       and has good advanced searching features.
      Teoma - is a relative newcomer with a slightly smaller database (just
       under 1000 million) but it is designed to find Websites that are authoritative rather than merely
       relevant, and it suggests ways of refining your search.

You can find more links to search engines at Search Engine Watch - - click on Search Engine Listings. Try also the other guides to
search engines listed above.

Some searchers prefer to use the smaller engines so that they are not overwhelmed with results, but
as long as you use the appropriate keywords and the engine ranks the results, even a return of
several thousand hits is not really a problem as you should only need to look at the first few. (Search
engines tend to rank results by relevance by checking whether your keyword(s) appears in the title or
in the first few lines or paragraphs or by counting how often it appears in total in the web page.) But
clearly it is better to structure your search so that the search engine finds exactly what you want.

Metasearch engines
Because all engines index the Web differently you will get the best results by using more than one
search engine. Metasearch engines (also called multi-search engines) are useful because they do
this for you – they send your query to several different search engines at once and collate the
results. Here is a small selection.

      Ixquick - – among the best. Good relevance ranking.
      Vivisimo - – organises your results into categories. Easy to use.
      Metacrawler - – long established and popular
      Ask Jeeves - – accepts your question in plain English, but can give
       irrelevant results
      Dogpile - - returns results from 10 leading search engines.

Be aware that some metasearch engines time out, so if any search engine site is too slow to respond
its results may be excluded. Also, with some metasearch engines (but not Ixquick) you do not have
the same level of control over your search statement because the one way of phrasing a search does
not cater for all the sites that a metasearch engine visits. Metasearch engines quite often change the
selection of search engines they use and with some (e.g. Ixquick) you can “customise” your search by
including or excluding particular search engines.

How to search
Web search tips
Search engines work differently and it is not easy to give general rules - but here are some points in
common to many search engines. To get the best out of them you should use their help screens. You
may also like to experiment with the advanced search options that some engines have which allow
more complex searches and the use of Boolean operators (linking search words with AND, OR,
NOT). Some search engines provide a helpful search form which makes it easier to structure your
search. You can often filter your search by date or speciality search options such as people's names,
media type (video/audio), language, etc.

Search strategy
Everything you may know about search strategy as applied to the library catalogue or a database
indexing periodical articles also applies to search engines e.g.

      Be specific - do not search on very general topics
      Use more than one keyword if possible to narrow a search (but no more than 6-8 as above this
       number search engines tend not to work effectively)
      Think of synonyms for your topics and join them with OR e.g. bedsores or pressure sores
      Think of different (especially American) spellings for your keywords e.g. color/colour
      Some search engines (e.g. AltaVista and Hotbot) accept truncation. * is almost universally
       used e.g. medic* will search for medicine, medicines, medical, medics, medicinal, medication,
       medicament, medicate etc. simultaneously

Some special search strategies to use with search engines

      To search on an exact phrase or a name, put it into double quotes e.g. "shakespeare’s
       globe". Many search engines have this feature and those that do not will ignore the
       quotes. Without the quotes some search engines will tend to look for shakespeare and globe
       as separate words anywhere in the document and not necessarily together as a phrase. Try
       this out in Google and compare results. With some search engines there is a search form
       allowing you to specify "this exact phrase".
      Beware that a few search engines automatically link words with OR rather than AND so if a
       document contains only one of the words you have entered it will still be picked up.
      Many search engines allow you to put a + in front of a word (or phrase) that must be in any
       documents found, e.g. +polio +vaccine would look for documents including both
       words. (Sometimes there is a search form that allows you to do the same thing by using an
       option for "all of these words".) In this instance you could of course use quotes to search more
       precisely for documents containing the phrase “polio vaccine”, or truncate the second word
       (vaccin*), if you wanted “vaccination” as well as “vaccine”.
      As a general rule place your main subject first in your search statement as some search
       engines use the first word to determine relevance ranking
      Some search engines are sensitive to upper and lower case when searching for names
      With some search engines you can search in the title of documents, rather than anywhere in a
       document. This can make for more precise searching

Gateways – general and subject-specific
Gateways (also known as portals, hubs or directories) are collections of Web resources arranged by
broad topic. You are first required to choose a subject area and then browse the resources or do a
keyword search using an inbuilt search engine. The resources are usually indexed, annotated and
evaluated by experts.

Gateways usually have smaller coverage of the Web than search engines, and therefore tend to give
results which are fewer in number but more relevant. They are better for browsing for general
information about a subject rather than looking for a specific piece of information, which is the job of
search engines. And most of them have the advantage over search engines that the Websites you find
have been checked for quality.

Gateways can be:

a) General, covering all subject areas
The most popular general gateway of all is Yahoo - - with over 2 million sites

In fact, an increasing number of search engines give the opportunity to browse a directory of
resources, in “Yahoo” style. These include Google - and Alta Vista -
Beware that Yahoo and other directories based on search engines include large amounts of
recreational material, and the resources are neither selected on the basis of quality nor evaluated.

By contrast BUBL LINK - is recommended as an annotated, selective, evaluated
subject gateway to resources. Although it is aimed at the higher education community it still contains
some very useful sites suitable for the FE sector. You can browse a directory of subjects, or look for
your subject in an alphabetical list.

b)       Subject-specific

          RDN: Resource Discovery Network - an excellent collection of subject
           gateways developed in the UK and designed to bring you Websites relevant to teaching,
           learning and research
          Pinakes - Links to many major subject
           gateways are at

Look particularly for those gateways that select and evaluate the resources rather than just linking to
anything and everything on a topic.

The Deep (or Invisible) Web
In addition to the straightforward Web pages, there are other categories of freely available information
on the Web which search engines cannot index. These include searchable databases (e.g. laws and
cases, phone books, dictionaries, library catalogues), information that is dynamically changing (e.g.
share prices, rail and airline timetables), and files in non-textual formats such as PDF and Excel (often
used for statistical information) and multimedia. This whole area is known as the Deep Web (or
Invisible Web) because it is at a level of depth that is invisible to most normal search engines. This
does not mean that it is difficult to access, but that it needs to be accessed in a different way. It is an
increasingly important area of information.

How can you search for information on the Deep Web? Here are some starting points. Unfortunately
all are somewhat USA-biased in their content, but some of them have links to a good number of non-
US resources.

          The Deep Web - - this tutorial from the
           University at Albany in New York gives more information about the Deep Web

          Direct Search - - lengthy list of searchable databases
           on many academic topics. Not all that easy to use. You can select the category you need, but
           be prepared to browse around
          CompletePlanet - offers a directory of over 100,000 resources
           including approx. 13,000 "deep" Websites. Also keyword searchable. Detailed search tutorial
      - directory of high-quality Deep Web
           databases. Browsable subject categories and sub-categories.

Alternatively many searchable databases and other Deep Web information resources are included in
directories and subject gateways. For example SOSIG, the Social Science Information Gateway, has an extensive section with statistical resources.

With some search engines e.g. Google, you can search for images or for data in formats such as PDF
or Excel. With search engines it is sometimes possible to find a searchable database on a topic by
adding the keyword "database" to your search.
For further information about the Deep Web have a look at

      Those dark hiding places: the Invisible Web revealed, by Robert J. Lackie,
      CompletePlanet's Deep Web FAQ,

Evaluating Web resources
There are few, if any, quality controls on the Web, so beware! Data can be inaccurate, unreliable, out-
of-date, false, biased, here today and gone tomorrow. When you have found a resource do not take it
at face value but try and check its authority. Some suggestions follow:

      Authorship and authority
       Does the information have a stated author or creator (whether personal or corporate) or is it
       anonymous? If it is a person, what is his/her educational or occupational background? Is
       he/she an unknown individual or someone connected with a respected professional
       association, educational institution or commercial publisher? Does the information seem to be
       Be aware that individuals often create their own home pages out of vanity and as a means of
       self-publishing. The information may be of great value but will not have been through the peer
       and editorial review or quality control intrinsic to scholarship so should be regarded as possibly
      Currency
       When was the information last updated? Look for a date at the bottom of the page. Are any
       links to other websites still working?
      Viewpoint
       How objective is the information? Is it slanted in a particular direction? You should be aware of
       personal and political bias and the use of the Internet for propaganda, campaign literature and
       activist publishing. It may be difficult to check this, but examine any statement of scope or
       introductory explanation.
       Is the information promotional i.e. is it commercial advertising or trying to sell you
       something? More and more businesses are using the Internet as a marketplace.
      Comprehensiveness and level
       How comprehensive is the information? What kind of audience is it aimed at? Does that level
       match your needs?
      Accessibility and ease of use
       Is the site well-designed and easy to find your way around, or are there design features which
       get in the way? Are links to other pages obvious? Can you print easily from it?

The following are very good tutorials on evaluation of Internet resources, including checklists of
evaluation criteria and a quiz, see:

      Internet Detective - - requires
       registration (free)
      Virtual Training Suite – which has a really good FE section
      Tonic - The Online Netskills Interactive Course - - requires registration (free)

The Future
Existing search tools, however advanced, cannot keep up with the amount of data and the rate of
change on the Internet. A different approach is coming from artificial intelligence in the form of bots
(short for robot) and intelligent agents which can be customized to search the Web on your behalf
gathering information relevant to your interests. Some of them will alert you by email to new changed

If you want to find bots and agents or learn more have a look at BotSpot -

Copernic - is a free intelligent metasearch engine which you download to
your desktop. It sends your search to many more engines than a normal metasearch engine, and
allows you to manipulate the results. There are more advanced versions of Copernic available for

And remember:

      The Web is not only a source for text documents, but also images. Many search engines
       include an option to search for images. For more information on the use of images, go to the
       TASI (Technical Advisory Service for Images) Their "resources" and
       "image sites" links give more detail about image searching.
      When you find something useful or interesting save it as a Bookmark or Favorite
      Everything on the Web is covered by copyright!

Some really useful websites

   • will give you a list of UK government papers sites
   • which contains a selection of Government titles covering a
       broad range of topics including the economy, work and welfare, health, transport and the
   • online street map
   • gives you information about the rail network, including train
   • Transport for London home page
   • British Telecom homepage including telephone directory
   • advice and guidance service supporting individuals and
       organisations in making effective use of ILT
   • TechDis provides an advice and information resource via
       extensive web-based databases and an email helpdesk on provision for disabled staff and
       students in the further, higher and specialist education sectors
   • electronic yellow pages
   • online directory enquiries
   • good range of public service information
   • music lyrics
   • Internet Movie Database, the “biggest, best, most award-winning movie
       site on the planet”

To top