Page scroll-down

Some websites (like eCommerce) requires scrolling down to the bottom of the page for loading additional content. In this case, you can set up Page scroll-down to have the page scrolled automatically in Octoparse. This is especially useful for extracting a web page that uses infinitive scrolling or lazy loading.

How to set up scroll-down

1) Double click the Go to Web Page action(or a Click Item) or click the   icon to access the setting menu. Then find the Page scroll-down options by clicking open the "After loading page" section.

2) To set up page scroll-down, check the box for "Scroll down the page after it is loaded"

3) Choose how you'd like to scroll the page

There are two ways of scroll-down to select from: scroll to the bottom of the page and scroll for one screen. 

Scroll to the bottom of the page: Octoparse would scroll down to the bottom of the page directly without stopping in the middle of the page. You should consider using this option when the page only loads elements when you reach the page bottom, like google play reviews.

Scroll for one screen: Octoparse would scroll for one screen only each time. Consider using this option if the page loads content as your scroll down the page continuously. 

It is recommended that you first test the options out by scrolling the page manually, then pick the one that works best for your target web page. 

4) Set up Repeats and Wait time

Next, enter a number for Repeats which is the number of times you'd like to repeat the scroll-down (ie. the number of scrolls). Again, you may want to scroll the target web page manually first, just to find out how many scrolls you'd need to load all the required information. 

Then select a proper wait time between every two scrolls. You may want to pick a longer wait time for pages that take longer to load.

If you have questions, you are welcome to submit a request here. Our support team will get back to you later.

Artículo en español:  Desplazamiento de página hacia abajo

También puedes leer artículos de web scraping en sitio web oficial

Author: Yina

Editor: Isabel