Dealing with pagination (clicking on a "Next" button)
FollowPagination, also known as paging, is the process of dividing a document into discrete pages, either electronic pages or printed pages. Setting up pagination is a crucial process to get plenty of data from multiple pages. Common ways of pagination include paging with a "Next" button, a "Load More" button, or with an infinite scroll.
In this tutorial, we will introduce the way to deal with pagination with a "Next" button in Octoparse.
There are two ways to set up pagination with a "Next" button:
1) Use the auto-detect algorithm to set it up
2) Set up the pagination manually
You may need this example link to follow through:
https://www.amazon.com/s?k=sunglass&ref=nb_sb_noss_2
1) Use the auto-detect algorithm to set it up
- Click the "Auto-detect web page data" on the Tips panel.
- Click the "Check" button to inspect which button is identified as a "Next" button.
The button selected will be highlighted on the page.
If Octoparse fails to recognize the correct next page button, click "Edit" to select the next button manually.
- Uncheck the "Add a page scroll" and click "Create workflow" on the Tips panel and you will see a workflow with pagination created.
- Test the workflow by clicking "Pagination" and "Click to paginate" to see whether Octoparse is able to move to the next page.
If you find Octoparse is able to go to the next page, that means your pagination is successfully set up.
2) Set up the pagination manually
If the auto-detection does not find the next page button, you can create the pagination manually.
- Click the next button, then choose "Loop click next page" on the Tips panel
|
Tips! If there is no "Loop click next page" option, you can select any option with "Loop click" to generate a pagination loop, like "Loop click single element", "Loop click single button", and "Loop click single link". |
- Test the workflow by clicking "Pagination" and "Click to paginate" to see whether Octoparse is able to move to the next page.
If you find Octoparse is able to go to the next page, that means your pagination is successfully set up.
If you need any assistance with your data project, feel free to submit a request here to contact our support team anytime!
Author: Joy
Editor: Yina
