| IMD |
| Page: Definition Spider |
Spider:
Also called web
Spider, web crawler, web robot or bot. A program or automated
script which combs the Internet, visits websites
, gathers all relevant data about it – especially
an index
of all keywords appearing on the site - and sends it to
a search
engine index data
base.
Before a new site is not entered in a search engine’s index it cannot appear in the search result pages. Therefore webmasters want their site visited by Spiders as soon as possible. They can either submit their site manually (for Google you can do this here) or – better as a lot faster if done correctly – put their link out on those other sites which are often visited by spiders, e.g. blog sites, social bookmarking sites or other popular high traffic websites . Spiders follow these links and arrive at the new site sometimes within 24 hours.
[Spider comes from Old English spibra = spider]
