What is cloud live log & history ?

Brief Introduction

Octoparse 8.1 has added the features of "Cloud Live Log" and "Cloud History". When tasks are in the Cloud runs, users can click to view current running details. If the task has been run in the Cloud for multiple times, users can view its Cloud history records.

Plan limits

Only available to premium plan (Standard plan and above) users (including trial users).

1. View Cloud Runs Details (Cloud Live Log )

On the task list, you can click the "More" button to view the task's current running details (for tasks running in the Cloud) or the most recent Cloud run details (for tasks completed in the Cloud).

Under Cloud live log page, there are 4 sections: Task Overview, Sub-task Overview, Task Logs, and Sub-task Details. 

• Task Overview

The section of Task Overview includes: Task status (completed, stopped, running), Run progress (a progress bar), Time started, Time completed, Run time, Data extracted, etc.

Notes:

For associated child tasks, there won't have a progress bar. Check this guide to understand What is parent task and child task in Octoparse?

• Sub-task Overview

This section will show the summary of split sub-tasks, which includes: Task split (the number of sub-tasks the task has been split into), Sub-task status (Waiting, Running, Completed, Stopped).

• Live Logs

For tasks running in the Cloud, here is a workflow showing how the tasks are split in the Cloud.

The "Task Logs" section is to record details on how a task runs in the Cloud, which can help premium users understand the Cloud run details. You can know when the task starts, when it is split, and when the sub-tasks start, etc.

• Sub-task Details

This section shows details about each split sub-task runs in the Cloud, such as its start time, completion time, run time, status, total data extracted (including accumulated data from each re-run), current data extracted (data after the last re-run).

Notes:

1. Sub-task No. is consistent with the sub-task number in the Task Logs.

2. You can filter out sub-tasks based on status. You can select more than one.

3. You can stop and re-run a certain sub-task. When a sub-task is "Completed" or "Stopped", you can click "Re-run" to re-start the extraction. When you re-run a sub-task, if the main task is "Completed" or "Stopped", its status will change to  "Running".

4. If you want to re-run a running sub-task, you need to click "Stop" first to change it to "Stopped" status, and then re-run the sub-task.

2. Cloud History

To view "Cloud history", you can click "..." for on the Dashboards, choose "Cloud runs", and then select "Cloud history". 

• Cloud History Interface

Cloud history will show the details of each Cloud run, including Batch No., Status, Run Started, Run Completed, etc. 

Notes:

The maximum number of the latest Cloud runs that can be recorded varies in different plans (Standard plan: 10, Professional plan: 20, Enterprise plan: 100). If you need more record more Cloud runs, you can upgrade to a higher plan.

• View Data

On the page of "Cloud History", you can click "View Data" to view and export the extracted data of a certain Cloud run.

Notes:

If you want to view all Cloud data or unexported data, you can still go to Dashboard and click "..." for more actions. Click "View data" and then choose "Cloud data" to view all Cloud data.