Before you go through SEO you have to know that search engines are not humans. The differences between how humans and search engines view web pages aren’t, must be cleared to everybody. The difference between humans and search engines are that search engines are text-driven. Although technology advances very fast, search engines are faraway from intelligent creatures that can sense the beauty of a cool design or enjoy the sounds and movement in movies. As an alternative, search engines crawl the Web, looking at particular site items (mainly text) to get an idea what a site is about. This short explanation is not the most accurate because as we will see next, search engines execute several activities in order to deliver search results – crawling, indexing, processing, calculating relevancy, and retrieving.
Work Procedure of Search Engines
First, every search engines have a piece of software called crawler or spider (or Google bot, as is the case with Google) which crawl the Web to see what is there. Crawlers follow links from one page to another and index everything they find on their way. Though the number of pages on the Web is over 20 billion, it is quite impossible for a crawler to visit a site daily. Rather visiting sites daily search engines just to see if a new page has appeared or if an existing page has been modified. Sometimes crawlers will not crawl your site for a month or two, so during this period of time your SEO efforts will not be rewarded, and also there is nothing you can do about it, so just calm down.
But you can check what a crawler sees from your site. As already talked about, crawlers are not humans i.e. they do not have sense, therefore they do not see images, Flash movies, JavaScript, frames, password-protected pages and directories. If you have tons of these on your site, you’d better run the Spider Simulator below to see if these goodies are viewable by the crawlers. If they are not viewable, they will not be crawled, not indexed, not processed, etc. – in a word they will be absent for search engines.
After a page is crawled by the search engines, the next step is to index its contents. The indexed page is stored in a gigantic database, from where it can later be retrieved. Basically, the process of indexing is recognizing the words and expressions that best describe the page and assigning the page to particular keywords. For a human it won’t be possible to process such volumes of information but usually search engines deal just fine with this task. Sometimes crawlers might not get the meaning of a page right but if you help them by optimizing it, it will be easier for them to classify your pages correctly and for you – to get higher page rank.
When someone search with some keywords a search request comes, the search engine processes it – i.e. it compares the search keywords in the search request with the indexed pages in the database. Since it is most likely that more than one page (practically it is millions of pages) contains the search keywords, the search engine starts calculating the relevancy of each of the pages in its index to the search keywords.
There are various algorithms to calculate relevancy. Each of these algorithms has different relative weights for common features like keyword density, links, or Meta tags. That is why different search engines give dissimilar search results pages for the same search keywords. What is additional, it is a known fact that all major search engines, like Yahoo!, Google, Bing, etc. periodically change their algorithms and if you want to stay at the top, you also need to adjust your pages to the latest changes. This is one reason (the other is your competitors) to dedicate permanent efforts to SEO, if you’d like to be at the top.
Retrieving the results is the last step in search engines’ activity. Basically, it is simply displaying the search results in the browser – i.e. the endless pages of search results that are sorted from the most relevant to the least relevant sites.
Major Search Engines and Their Work Procedure
Although the fundamental code of operation of all search engines is almost identical, the minor dissimilarities between them lead to major changes in results relevancy. Different factors are significant for different search engines result relevancy. There were times, when SEO specialists joked that the algorithms of Yahoo! are intentionally made just the reverse of those of Google. While this might have a little bit of truth, it is a matter a fact that the major search engines like different things and if you plan to overcome more than one of them, you need to optimize carefully.
There are many examples of the differences between search engines. As an example, for Yahoo! and Bing, on-page keyword factors are of primary importance, while for Google links are very, very important. Also, for Google sites are like wine – the older, the better, while Yahoo! normally has no expressed preference towards sites and domains with tradition (i.e. older ones). Thus you might need more time till your site gets mature to be admitted to the top in Google, than in Yahoo!
Related posts:
I never thought I would find such an everyday topic so enthralling!