Besides data export to formats such as XLS, CSV, HTML, etc, Octoparse also supports data export to your database (MySQL, SQL Server).
As for data export to the database, there are 2 methods we support:
- Export extracted data to your database manually (For local and Cloud data)
- Auto-export extracted data to your database with schedule (Database Auto-export Tool) (For Cloud data only)
This article will mainly introduce the 1st method - manual export to your database.
- Our 8 Beta's database export feature supports MySql and SqlServer. If you need to connect with other databases or platforms (such as your own websites), please refer to Octoparse API.
- If you need to export data to Oracle, please use the 7 versions.
- Octoparse currently does not support data export to cloud databases such as AWS database, Azure, Cloud SQL database, etc.
Export extracted data to your database manually
Here are the prerequisites before we start:
- A task with data extracted (both local and Cloud data are okay)
- A table with data fields configured in your database to receive data
This tutorial will use the MySql database and a task's local extracted data as an example.
Now, view the local data of the target task.
- Click the "Export Data" button
- Choose the type of your database
- Press "OK"
- Fill up the required fields about the target database settings
After filling up the information, click "Test Connection" to see if you can connect to your database.
If the setting is alright, you will receive a "Connected" notice as above. Then, you can click "Next" to move forward.
Now, we need to configure data mappings.
- Select an existing table from the Data Table drop-down options
- Match the source data fields (data fields extracted) to the target data fields from the designated table
- (Optional)Select any source data fields to use as an identifier (primary key)
- After data mappings are configured, click "Export" and wait for the data export to finish.
Once the configuration is completed, it can be saved for later use. You can name the setting first and then press the "Save" button to save it.
Next time, if you want to use your previous database export settings, you can just choose a saved setting to save time.
- Press the "Completed" button to exit after you receive the "Export Completed" message.
Then, you can go to your database and check if the data is delivered correctly.
1. Once a data field is selected as a primary key, subsequent data records with the same primary key will overwrite the existing ones in the database.
2. If you have trouble delivering extracted data to your database, you can also choose to export data to CSV first. Then uploading the CSV file to your database.
Auto-export extracted data to your database with schedule
If you want to set up automatic data export to a database regularly, then you can check this guide for details: Database Auto-export Tool.