Why do we need to use "Wait before action"?
In some cases, the previous step may require some time to complete, such as Enter Text, or the current page requires a longer time to load. Wait before action would thus be needed or it may lead to some errors like missing data.
Wait before action can also be used to avoid being blocked by the website due to fast scraping. Slowing down the scraping would make the process more human-like.
How to set up "Wait before action"
- Click on one action in the workflow. You can find the "Wait before action" in the "Option" of the settings at the bottom of the workflow.
- Tick "Wait before action" and select a proper time.
If you want to wait for the page to load some specific elements before scraping, "Wait until a designated element appears" is really useful. You need to enter the XPath of the element and Octoparse will either wait for the setup wait time or wait for the element to appear before going on to the next action.