Sometimes, you may encounter such a situation that you need to click some items first to display the information hidden behind. How to scrape this kind of data?
Let's take this website as an example: https://www.nastygal.com/black-croc-structured-mini-backpack/AGG77323.html
In this website, if you want to view data under the fields of "Details and Care" and "Shipping and Returns", you need to click the tags respectively.
In this case, if we want to extract data under "Details and Care", how are we going to achieve it?
Here are two ways to help you achieve it:
- Click the tag to display the data first, and then extract data you want
- If the data can be found in the source code of the web page, you can extract data directly
1. Click the tag to display the data first, and then extract data you want
- Click the "Details and Care" tag
- Select "Click element" on the "Action Tips" panel
- Uncheck "Auto Retry" for the "Click" item and click "Save"
- Octoparse has detected AJAX automatically. You can input the AJAX timeout based on your network condition.
- Then, select the data and click "Extract text of the selected element" on the "Action Tips" panel
- Click "OK" to save
|
Tips! If you want to learn more about AJAX, here are the related tutorials you might need: If the new tab option is selected automatically, please uncheck that.
|
2. If the data can be found in the source code of the web page, you can extract data directly
- Open the web page with your browser, and then right-click to select the option of "Inspect Element" to view the source code
From the source code, we can find that even though we do not click the "Details and Care" tag, we can still find the corresponding data under the tag.
Now, we can go back to Octoparse and continue scraping.
- Click the switch button
in the top right corner of the built-in browser to switch to the browser mode - Click the "Details and Care" to display the content
- Click the switch button again to switch back to the edit mode
- Then, you can select the data and click "Extract text of the selected element" on the "Action Tips" panel
- Click "OK" to save
Author: Vanny
Editor: Yina
Was this article helpful? Contact us
any time if you need our help!