While scraping data from the web, Octoparse takes the reloading as the signal to execute the action, such as "Click item" and "Click to paginate". For the web page using AJAX, it updates new contents without reloading. As there is no reloading, Octoparse doesn't receive the signal to act and would be stuck in the last step. As a result, we may get zero, or much fewer extracted data than we expect.
To fix this issue, you will need to set up “AJAX Load” timeout under “Advanced Options” of the “Click Item” action.
AJAX timeout setting in Octoparse is based on the time it takes for web pages to load.
You can also set up a higher value as long as there is enough time for the next page to load.
In fact, many websites update new data with AJAX configuration such as Amazon, eBay, Google etc. Sometimes, if Octoparse takes so long to scrape data after clicking an item, you should also configure AJAX timeout to make sure there’s enough time for detail pages to load.
If you do have questions about AJAX, read more details here
Artículo en español: ¿Por qué se detiene Octoparse después de hacer clic en “Next” ?
También puede leer artículos de web scraping en el sitio web oficial