Crawling the infinite scroll

February 13, 2014


Link copied to clipboard
Author PhotoBy John Mueller, Webmaster Trends Analyst

You think infinite scroll is cool? Search-friendly infinite scroll is even cooler! We just announced on the Webmaster Central Blog how to make infinite scroll pages more search-friendly, helping search engines access all the individual items in the category/gallery so the content is available in search results.

With some implementations of infinite scroll, crawlers aren’t great at scrolling down or clicking “load more”, so they may not crawl items displayed after the initial page load. To help the crawler see all the content, we recommend converting the infinite scroll page to paginated series by using the HTML5 History API. (Of course, the pagination is seamless to the user.)

infinite scroll page diagram
Infinite scroll page is made “search-friendly” when converted to a paginated series.
Each component page has a similar <title> with rel=next/prev values declared in the <head>.


Here’s a demo of infinite scroll with pagination. The demo isn’t production-ready, but the key search engine-friendly points to note are:
  • Coverage: All individual items are accessible. With traditional infinite scroll, individual items displayed after the initial page load aren’t discoverable to crawlers.
  • No overlap: Each item is listed only once in the paginated series (that is, no duplication of items).

For the full details, check out our new post on the Webmaster Central blog.


John Mueller is a Webmaster Trends Analyst in Zurich, working with webmasters and Google engineers to make the web better. In his time off, he builds robots with his kids to take over the world (or at least mow the lawn).

Posted by Scott Knaster, Editor