What if the search button is not working?

You might encounter some websites that don’t have a search button or the button is not clickable. For example, if you open https://www.bukalapak.com/ in your browser, you will find that you can only conduct searches with the “Enter” key, not the search button.

If the target website’s search button is not clickable in the browser, it won’t work with the Octoparse. Therefore, Text/ keyword input is not suitable in this case, instead, we suggest you generating the link of search results in your browser. Then directly copy and paste the search results URL on Octoparse can avoid complicated steps.

(Check out the complete tutorial of How can I extract data with a list of URLs?）

It will require you to get a list of URLs outside of Octoparse.

• Step 1. Input different keywords into the target website in your browser, generating URLs and observe the pattern of URL.

• Step 2. You can use our batch generating URL function or an Excel sheet to generate the URLs automatically if the URLs are based on a predefined pattern. 

Here is a more detailed instruction of how to use Octoparse predefined pattern.

Batch generate URLs based on a predefined pattern

With URL Batch Generate feature, you can easily generate numerous URLs following specific patterns by modifying various parameters of one given URL.

This feature would be especially useful for scraping from many pages from a particular website. Use the URL generator to quickly generate all the page URLs and scrape all the pages simultaneously. No need to go through the pages one by one.

· Select "Advanced Mode" and click "+Task" to create a new task

· Select "Batch generate"

· Input the URL as a base for batch generate

· Highlight the selected URL parameter, and click "Add parameter"

· Select from the four Parameter Type options to define the pattern you need

· Click "Save URL" to save the list

Editor: Yanni