Infinite Scroll has setup but no new elements added to the list?

When you're scraping a website that applies scroll-down/show more button to load more content, you may encounter such a scenario:

Even you've set up the scroll-way with Octoparse, you still find no new items added to the loop item list. 

Here is the reason:

Octoparse will automatically generate a loop mode based on the structure of the website. If the auto-generate loop mode cannot detect new elements when you scroll down and load more items, the reason behind would be the auto-generate fixed list. 

You may follow this tutorial to check and solve the problem. Take the picture below as an example:

You may notice that the current "Loop mode" is under the "Fixed list" mode.

Unlike the "Variable list", "Fixed list" will only include items according to the fixed list of XPath you enter to the text box. Therefore, when dealing with dynamic items, or when the number of items would vary in different pages, you need to switch the loop mode to the "Variable list" because it can automatically detect and match all the items corresponding to a certain XPath.   

 To solve this problem, you also need to modify the XPath. Check this tutorial Locate elements with XPath to know more about XPath.

If you don't know much about XPath, you may still generate the XPath by observing the similarities among the XPath in the "Fixed list".

Taking the picture below as an example, all the XPath in the "Fixed list" share the same tag "H4".

You can then enter the new XPath "//H4" into the text box of the "Variable list" to see if it is viable or not. If you scroll down to load new content and find new items added to the loop item list, then the new XPath you enter is viable.

This method applies to most simple cases. If it is a difficult one, you may still need to learn how to write XPath to solve the problem or contact us at any time for assistance.

Artículo en español:  ¿Se ha configurado Infinite Scroll pero no se agregaron elementos nuevos a la lista?

También puede leer artículos de web scraping en el sitio web oficial

Author: Vanny