Dealing with pagination (with a "Next" button)

Pagination, also known as paging, is the process of dividing a document into discrete pages, either electronic pages or printed pages. Setting up pagination is a crucial process to get plenty of data from multiple pages. Common ways of pagination include paging with a "Next" button, a "Load More" button, or with infinite scroll.

In this tutorial, we will introduce the way to deal with pagination with a "Next" button in Octoparse.

There are two ways to set up pagination with a "Next" button:

1) Use the auto-detect algorithm to set it up

2) Set up the pagination manually

1) Use the auto-detect algorithm to set it up

You may need this example link to follow through: https://www.amazon.com/s?k=sunglass&ref=nb_sb_noss_2

1. Click the "Auto-detect web page data" on the Tips panel.

2. When the auto-detect process is completed, check whether the "Paginate to scrape more pages" option is ticked. (Usually, this option will be shown on the Tips panel automatically.)

3. Click the "Check" button to inspect which button is identified as a "Next" button. 

When Octoparse fails to recognize the correct next page button, click "Edit" to select the next button manually. 

4. Click "Create workflow" on the Tips panel and you will see a workflow with pagination created.

5. Test the workflow by clicking the "Pagination" and clicking "Click to paginate" to see whether Octoparse is able to move to the next page. If you find Octoparse is able to go to the next page, that means your pagination is successfully set up.

2) Set up the pagination manually