You are browsing a tutorial guide for Octoparse's latest version. If you are running an older version of Octoparse, we strongly recommend you upgrade because it is faster, easier and more robust! Download and upgrade here if you haven't already done so!
After the completion of a scraping task, it may be necessary to utilize the same task to retrieve data from additional, similar URLs. This article outlines the process for updating the URL list within a task.
1. Single URL update
If the task only has one URL (without Loop URLs), you can use the following steps to make changes to it:
Click on Go to Webpage in the workflow (1), then paste your new URL into the URL box (2) and click Apply to save (3)
2. Multiple URLs update
Octoparse enables users to iterate through a list of URLs with a shared webpage layout by implementing a loop item. It is important to ensure that all URLs originate from the same domain, as this serves as a prerequisite for the successful execution of the Loop URLs function. Failure to adhere to this requirement will result in Octoparse's inability to identify the URLs and carry out subsequent extraction procedures.
Select Loop URLs and click the edit button
Copy and paste the new URL list into the box and click Save
Alternatively, you can click the Edit URLs button at the top of the built-in browser.
Copy and paste the new URL list into the box and click Save
Then, the URL list will be updated successfully. You can also select the other 3 ways to input the new URLs: Batch URL input
3. Update URLs using API
If you are a user of the Professional plan or Enterprise solution, you will have the ability to utilize the Octoparse OpenAPI to automatically update your URLs without the need to access the Octoparse software.