You are browsing a tutorial guide for the latest Octoparse version (8.5.4). If you are running an older version of Octoparse, we strongly recommend you upgrade because it is faster, easier, and more robust! Download and upgrade here if you haven't already done so!
CAPTCHA is a very common anti-scraping technique applied by many websites in different forms.
To help improve the efficiency of scraping, Octoprase can currently handle the three kinds of Captcha automatically: hCaptcha, ReCaptcha V2, and ImageCaptcha.
hCaptcha and ReCaptcha V2 can be resolved similarly, while it is more complicated to set up a resolution to deal with ImageCaptcha.
Follow through this tutorial, and you will be able to have a basic understanding of each Captcha and handle them with Octoparse.
1. What are hCaptcha & ReCaptcha V2?
hCaptcha usually combines:
- an "I am human" button with the logo of hCaptcha
- and simple questions (in pictures) that are easy for humans and difficult for machines:
Most ReCaptcha V2 usually has an "I'm not a robot" button; however, sometimes, it may contain simple questions similar to hCaptcha.
2. How to solve hCaptcha and Recaptcha V2
Click the add step button in the workflow
Select Solve CAPTCHA
Click on Solve CAPTCHA box
Select hCaptcha/ReCaptcha V2 as the CAPTCHA type
Click Apply to save the setting
Note: hCaptcha and ReCaptcha won't be resolved automatically until an actual data run. Thus, you need to turn on the Browse Mode and solve it manually to proceed when creating the task.
3. What is Image Captcha?
ImageCaptcha is the original way in which humans were verified. It can use known words or phrases or random combinations of digits and letters. Some ImageCaptcha also includes variations in capitalization.
4. How to solve Image Captcha
To follow through with the tutorial and resolve ImageCaptcha, you may use this URL: https://democaptcha.com/demo-form-eng/image.html
A. Select the Input Box and Image Box for the Captcha
Click on the Input Box for the Captcha
Select Solve Captcha in the Tips panel
Click the Image Box
Click the Login/Submit/Confirm button to continue (sometimes it can be other buttons such as 'Send' in this specific case)
Click Confirm in the Tips Panel
B. Set up a Captcha Solving Failure
Now, we need to train Octoparse to resolve the Captcha by setting up a solving failure.
Click on the error message (in this case - Some errors were detected in your form: Invalid verification code)
Click Confirm Error in the Tips panel
C. Set up a Captcha Solving Success
Click Set Up CAPTCHA Solving Success to go through the final step
Input the text shown in the Image Box
Click Submit CAPTCHA answer and complete setup
The Image Captcha has now been resolved. The Solve CAPTCHA step will be added to the workflow, and you can also modify the settings under the workflow.
hCaptcha and ReCaptcha V2 can be detected automatically, so there is no need to set XPath to locate them. Image CAPTCHA cannot be detected without XPath. You need to pay attention to the XPath in the settings.
The cost is $1/1K CAPTCHA. One attempt to resolve a CAPTCHA is counted as one CAPTCHA credit. So resolving one CAPTCHA successfully might cost several CAPTCHA credits. You can click on Add Credits to top up. CAPTCHA credits cannot be refunded. We have sent some credits for Standard/Professional plan users for testing. You can test it before paying for the credits.