What are concurrent runs? (V7.3)

May 11, 2022 06:09
Updated

Tasks that are running concurrently means they are being executed simultaneously either in the cloud or on your local machine. Yet, there are still minor differences to be noted.

1. Local concurrent runs basically mean executing more than one task on the local machine. The Free Plan is limited to two concurrent local runs while all the other plans allow for unlimited concurrent runs.

2. Cloud concurrent runs can be easily checked by filtering for "Running" for Cloud Status.

Question: When a plan comes with 6 cloud servers (such as the Standard Plan), does it mean there should always be 6 concurrent running tasks in the cloud?

Answer: Not exactly. When an account is assigned with 6 cloud servers, it is possible to have 6 tasks running concurrently in the cloud. However, in efforts to achieve maximum extraction speed, Octoparse will always try to split up the task into smaller sub-tasks. Once split, each sub-task will be run on a separate cloud server for faster data extraction. In this case, one task can take up more than one cloud server (Learn more about task splitting ). Of course, if task split is not needed, you can always disable "task split" such that you'd always get 6 tasks running concurrently in the Cloud.

Tips!

1. If task spit is not intended, select "Disable task split" in "Setting" (for Cloud Extraction).

2. Consider limiting the number of tasks running concurrently for faster data extraction. Having fewer tasks running in parallel will free up more cloud servers to the limited tasks hence speed up the extraction (for Cloud Extraction). Go to Account Setting and select the maximum number of tasks to be run in parallel.