Octoparse

XPath, Pagination, data cleaning, anti-blocking, API, and more

Beyond Basics

What is XPath and how to use it in Octoparse?

Use relative XPath to locate data outside a loop item

Use XPath to locate email addresses from "mailto" links on any website

Fix field issues (missing, blank or misplaced fields)

Customize element XPath

Locate elements based on nearby text ("following-sibling" function)

Learn how to set up an alternative XPath in Octoparse to improve data field accuracy when the main XPath fails.

Set up an alternative XPath

XPath Cheatsheet for Web Scraping with Octoparse

Regular Expression (Regex) Cheatsheet for Data Extraction

Essential XPath Knowledge for Octoparse Users

XPath

Handling AJAX

Dealing with pagination (clicking on a "Next" button)

Dealing with Pagination (with a "Load More" button)

Dealing with pagination (no "Next" button)

Dealing with pagination (infinite scroll)

Tackle Pagination

Scrape and download files from websites

How to add a custom data field

How to use Octoparse to extract page-level data, including webpage URL, page title, meta description, meta keywords, and HTML source code.

Get page-level data (metadata, URL, title & HTML)

Add the original URL (before redirecting) along with the data scraped

Data Deduplication: Causes, Fixes, and Prevention

Merge field data

Import and export a data field configuration

Data Field Management

Select the correct HTML tag for web elements

How to extract star rating information

Extract attributes of a web element (text, URL, HTML, etc.)

Scrape images from a carousel

This tutorial will teach you how to scrape data from inside a tab.

Scrape data from inside a tab

Extract Data

Learn how to clean data inside Octoparse software.

Refine extracted data (replace content, add a prefix, etc.)

Convert the current time field to another time zone

Reformat the date from DD/MM to MM/DD

Learn how to use Octoparse's AI RegEx tool to clean data

Regular Expression Tool

Data Cleaning

How to set up the workflow if your target data is behind a login

Scrape data behind a login

It helps you to log into the target website and save the cookies easily.

Auto login to a website

Save cookies in Octoparse to keep your login status and avoid logging in repeatedly.

Save and use cookies

Login

How to select a specific option from a drop-down list?

Click through a drop-down menu

Scrape the selected dropdown option as a field

Handle Dropdown Menus

How can I address anti-scraping measures and restrictions when using Octoparse?

Set up IP Proxies

Choose different IP pools for websites with IP restrictions.

Switch IP Pools for Cloud Runs

Resolve Captcha

How to solve CAPTCHA in Octoparse?

What is Cloudflare verification and how to deal with it?

How to deal with Cloudflare security checks manually

Add custom User Agent

Some useful tips for setting up tasks, getting data or refining data.

Smart Hacks in Octoparse

Retry actions

Effective methods to prevent your scrapers from being blocked

Anti-blocking

Octoparse API FAQs

Migrating to Octoparse OpenAPI

Connect to Octoparse APIs with Postman

Exporting Data to Google Sheets with Mixed Analytics API Connector

How can I access my API key and use API in Octoparse?

How to delete a task group

Find Your Task ID

Learn how to import a task file from your local device or export your task as a file to your local device.

Import and export tasks

Organize tasks using Filters

Learn how to manage your tasks through task groups

Task Groups

Run task locally with Octoparse 8.5

Quickly share your task files with others.

Share task via link

Learn how to schedule your tasks to run on a regular basis

Schedule tasks to run

Cloud task completion notifications

Custom Alerts (v8.7.4 and above)

Task Management

Go to Octoparse.com

Octoparse DE

Octoparse FR

Octoparse ES

Octoparse JP

Download

Blog

API Docs

Find answers and get help from Intercom Support and Community Experts

This site employs cookies and other technologies that we and our third party vendors use to monitor and record personal information about you and your interactions with the site (including content viewed, cursor movements, screen recordings, and chat contents) for the purposes described in our Cookie Policy. By continuing to visit our site, you agree to our {websiteTermsLink}, {privacyPolicyLink} and {cookiePolicyLink}.

This site uses cookies and similar technologies ("cookies") as strictly necessary for site operation. We and our partners also would like to set additional cookies to enable site performance analytics, functionality, advertising and social media features. See our {cookiePolicyLink} for details. You can change your cookie preferences in our Cookie Settings.

We use cookies to make our site work and also for analytics and advertising purposes. You can enable or disable optional cookies as desired. See our {cookiePolicyLink} for more details.

Advertising cookies are set by our advertising partners to collect information about your use of the site, our communications, and other online services over time and with different browsers and devices. They use this information to show you ads online that they think will interest you and measure the ads' performance. Social media cookies are set by social media platforms to enable you to share content on those platforms, and are capable of tracking information about your activity across other online services for use as described in their privacy policies.

These cookies enable the website to provide enhanced functionality and personalisation. They may be set by us or by third party providers whose services we have added to our pages. If you do not allow these cookies then some or all of these services may not function properly.

These cookies are necessary for the website to function and cannot be switched off in our systems.

These cookies allow us to count visits and traffic sources so we can measure and improve the performance of our site. They help us to know which pages are the most and least popular and see how visitors move around the site.

You have the right to opt out of the sale of your personal information. See our {cookiePolicyLink} for more details about how we use your data.

Your Privacy Choices

We use cookies to enhance your experience. You can customize your cookie preferences below. See our {cookiePolicyLink} for more details.

Cookie Settings

Link, Press control-option-right-arrow to exit

Empty Help Center

Uh oh. That page doesn’t exist.

Disappointed

Neutral

Smiley

Thinking...

Searching through sources...

Analyzing...

Tickets submitted through the messenger or by a support agent in your conversation will appear here.