Search Engines - The What, How, and Why of Web Crawlers

Surveys have shown that when potential customers want to find a website for information, or to purchase a product or service, they do this in one of three ways. They may have heard about the site from a friend or read about it in an article. They may find the site by clicking a link found on another site. The most popular option, by far, is that they find a website by using a search engine.

In fact, 90% of Internet users find what they are looking for by using a search engine. Since it is the search engine that ultimately brings a prospect to your website, it is imperative that you have an understanding of how these search engines actually function and how they decide what information to show the person that requested the search.

When you submit your website to a search engine, it will send a robot, often called a spider or crawler, to index your site. This automated program will visit your site, read it's content, read the Meta tags, and follow any outgoing links that you may have. The spider then reports all this info back to a data bank where it is stored and indexed. It will also do the same for any sites that you have linked to. The spider will revisit your website periodically to check for additions and changes. The frequency of these return visits will vary from one engine to another.

All search engines employ some sort of ranking system to determine what the website is about and how important each website or web page is. The algorithm varies slightly from one search engine to another, but basically they analyze the frequency and location of your keywords on the web page. They also look at how your web pages link to other web pages. Using this information, the search engine can determine the subject of your web page and decide how 'important' your website is.

When someone requests a search on a search engine, it actually searches its database or index to find the information for you. It is not really searching the web in real time. It will return results to the searcher according to the ranking it has assigned to the pages. Each search engine will rank a page differently because they don't all use the same algorithm, but their common goal is to present the most relevant search results first.

A webmaster needs to keep in mind that potential customers will probably only look at the first few listings before making a choice and clicking away from the search results. Yes, it does matter where you site shows up in the search results! Keyword research and quality content using these keywords effectively should be every website owner's first priority. The success of your website depends on it.

About The Author-- Kevin Sinclair is the publisher and editor of Net MLM Profits, a blog dedicated to providing information and tips for network marketers to succeed online. Net-MLM-Profits.com

Article Source: Articles island - Free article submission and free reprint articles