If you've run into any of the issues below, you may want to try setting up IP proxies manually in Octoparse:
- Unable to log into Octotoparse because your company or institution uses intranet and restricts external requests
- Target website is very sensitive to web scraping and implements anti-scraping strategies such as IP blocking to stop scraping activities
In this tutorial, we are going to show how to set up proxies with the latest version to resolve the issues mentioned above.
There are 2 ways to do so -
- Use proxy to login Octoparse (for login issues only)
- Rotate external proxy servers to counteract IP blocking
1. Use proxy to login Octoparse
- Go to setting on the login panel
- Enter proxy information as required
- You can Test to see if the connection is good then Confirm the setup
2. Rotate external proxy servers to counteract IP blocking
Octoparse Cloud IPs are data center IPs, which may be blocked. If a website blocks your IP or our Cloud, you can set up an IP proxy in the task settings. You can use your own IP proxies or use Octoparse built-in proxies.
2.1 Set up Octoparse Built-in proxies
Octoparse proxies are residential IPs, so they work better in avoiding being blocked.
- Open the task you would like to set up proxies for
- Go to task setting
- Tick Access website via proxies
- Click on Use Octoparse proxies
- Select Default or Country/Region you'd like for the IP address (Default means to use IPs from random countries)
- Choose Rotate time for the rotation interval
- Click the Save button on the upper left corner of the interface to save your settings
- Octoparse proxies are only available on version 8.5.4. You can download the new version here.
- It costs $3/G proxies. Make sure you have enough Proxy Credits or Account Credits to use this feature. To enable IP proxies, you must have at least $3 in Account Credits. Account Credits displayed on the page can be used for proxies purchases. You can simply top up your credits by clicking the Add Credits button.
- 1G proxies can usually scrape 500 web pages. The proxies might not work for all web pages. You may top up $3 credits to do testing first.
- Proxies credits cannot be refunded
2.2 Use your own IP proxies
Another way to set up IP proxies is to access your own IP into Octoparse. Your own IP proxies can only be applied in local runs.
- Tick Use my own Proxies
- Click Configure
- Input IP Proxies in the box
- Confirm the settings