Why do we need to use "Wait before action"?

In some cases, the previous step may require some time to complete, such as Enter Text, or the current page requires a longer time to load. Wait before action would thus be needed or it may lead to some errors like missing data.

Wait before action can also be used to avoid being blocked by the website due to fast scraping. Slowing down the scraping would make the process more human-like.


How to set up "Wait before action"

  • Click on one action in the workflow. You can find the "Wait before action" in the "Option" of the settings at the bottom of the workflow.

  • Tick "Wait before action" and select a proper time.

6622.gif

If you want to wait for the page to load some specific elements before scraping, "Wait until a designated element appears" is really useful. You need to enter the XPath of the element and Octoparse will either wait for the setup wait time or wait for the element to appear before going on to the next action.

23788.png

TIPS:

  1. Wait until a designated element appears must be set up along with the Wait before action to avoid endless waiting.

  2. If you are not sure what's the XPath of the element, you can check What is XPath and how to use it in Octoparse. Or select an element from the web page and Octoparse would generate an XPath automatically.

bmnnk.gif
Did this answer your question?