You have to use SEO to lure the spiders and convince them you have the goods. As always, user optimization (UO) is just as important – actually delivering those goods!
You want your site to be spidered, but spiders don’t convert – visitors do. Without the spiders, the visitors never find your site; but without conversions you site is useless.
The sheer volume of information on the web engineered the development of search engines. As the internet continues to grow, it is necessary for the spiders to be crawling more web pages at an increasingly greater frequency. Search engines can produce significant percentages of your traffic, especially if you don’t use advertising or backlinking for traffic purposes.
Unfortunately competition has indirectly lowered the overall satisfaction level of website visitors. Traffic often goes to # 1 results, so this is a huge incentive for websites to reach this position. Of course, good SEO is needed to be ranked #1 in the SERPs, so certain rules have to be followed.
When webmasters & SEO’s analyze Google and other search engines to determine what properties they need to push in order to rank number 1 on the results pages, the real need for UO becomes overlooked,.
Some misguided webmasters will sacrifice human visitor experience to the spider gods. When a robot visits your site, it will see the optimization and you may be ranked highly – but when a visitor enters, they may look around in disgust at no formatting, layout or graphical interfacing. An unattractive, uninformative, unusable website will not garner conversions. No conversions means no sales and no revenues, and eventually a dead site. This results in a website that does not produce profit, even if it ranks in the top slot.
You have to find an equitable balance between optimizing websites for spiders and keeping it convenient for users. Even if you dropped a few spots below #1, you could still out-perform a more highly ranked competitor in terms of revenue generation and conversion rates. with proper UO.