When I run my scrap task in the Cloud, I am regularly blocked despite slowing down my scraper.


1 comment

  • Scarlett

    -The default IP pool for the premium account is the US one, which can avoid getting blocked for most cases, but if the website is way too scraping sensitive, we can help you switch your account to the German IP pool.
    -Our IP rotation works like this: When an extraction task is set to execute in the Cloud, the task will be split into sub-tasks(What is "task split" on Cloud Extraction? (Speed up Cloud Extraction)). Every 100 URLs will be one sub-task. One sub-task will be run by one IP, minimizing the chances of being traced and blocked. For auto-switch UA, you can set up the interval as you preferred in the settings:

    -With a higher plan, you get to enjoy more cloud servers, which means you can upgrade your plan to increase the rotation frequency of IPs in the cloud. For example, there are 6 cloud servers(standard plan) to 20 cloud servers(professional plan) scraping the data simultaneously. So the same set of data in the cloud can be scraped 6 to 20 times as fast as with local extraction.

    Comment actions Permalink

Please sign in to leave a comment.