"Wait before action" is a function that could be set to every action in the workflow. It will let the task to wait for some time before the action is executed.
Why do we need to use "Wait before action"?
In some cases, the previous step may require some time to complete, like manual input, or the current page requires a longer time to load. You will need to set up "Wait before action" for steps like these; otherwise, it may lead to some errors like missing data.
It is also used to avoid being blocked by the website due to fast scraping. Slowing down the scraping would make the process more human-like.
How to set up "Wait before action"
- Click on one action in the workflow. You can find the "Wait before action" in the "Option" of the settings at the bottom of the workflow.
- Tick "Wait before action" and select a proper time.
If you want to wait for the page to load some specific elements before scraping, "Wait until a designated element appears" is really useful. You need to enter the XPath of the element and Octoparse will either wait for the setup wait time or wait for the element to appear before going on to the next action.
1. "Wait until a designated element appears" must be set up along with the "Wait before action" to avoid endless waiting.
2. If you are not sure what's the XPath of the element, you can check What is XPath and how to use it in Octoparse. Or use to select an element from the web page and Octoparse would generate an XPath automatically.
If you have questions, you are welcome to submit a request here. Our support team will get back to you later.