More sleep for the AdSense Crawler
Let's imagine that you're the AdSense Crawler. You're bouncing along all over the Internet, visiting publishers' sites, and letting the AdSense system back at Google know what the pages are all about. Then one day, you run into a site that asks you for a login and password. "Huh? I don't have a username and password to this site. How am I going to crawl the pages behind that login?" This is something our crawler sees every day here at AdSense. The result is that your users end up with poorly targeted ads and the AdSense Crawler ends up with sleepless nights, wondering what could have been -- if only it had crawled those pages. We've recently launched a new feature called Site Authentication to take care of this problem. Using Site Authentication, you can give our crawler access to your login-protected pages by passing it information to log into your site. For example, let's say your news site has a premium content area, with articles that only payi...