Google is a very fast search engine. In our experience, Google crawls new sites very rapidly. We’ve actually seen some of our websites land on the first page of Google results before Bing and Yahoo even have a chance to index them. This gives you an idea of how swiftly Google moves.
That said, there is still some confusion as to how often Google indexes sites. This is because crawl frequency varies on a site by site basis, and the factors that influence the crawl frequency are many of the same factors that influence your site's rank in the search results. In other words, no one outside of Google really knows what determines how often a site is crawled. But observation has illuminated several important techniques.
1. Fresh content and links. If you add new content to your site on a regular basis, Google will crawl your site more often in order to retain an accurate index of your site. If other sites on the web consistently add new links to your website, Google will take this as another strong signal that your site is important and being updated frequently. The last thing Google wants is to miss part of your website. If you can convince Google that it needs to come back to your website as often as twice per day just to keep up with you, Google will do so. Add fresh content as often as possible.
2. A comprehensive sitemap. An easily navigable sitemap that includes links to every page on your website and is consistently updated when new pages are added is invaluable.
3. Rapid server response time. Google's crawlers dip in and out of a lot of sites very swiftly, and if a Google crawler gets hung up trying to access your site because your server response time is slow, you may miss out on being crawled, which will likely delay your next crawl.
The bottom line? Fresh content and new links are critical. A sitemap will help. And reducing your load time will prevent hiccups.
