You are browsing a tutorial guide for the latest Octoparse version. If you are running an older version of Octoparse, we strongly recommend you upgrade because it is faster, easier and more robust! Download and upgrade here if you haven't already done so!

This tutorial will show you how to scrape the product information, such as product name, price, brand, etc., from Target.com.

2022-05-26_18-05-06.jpg

To follow through, you may want to use the URL below:

https://www.target.com/c/milk-substitutes-dairy-grocery/-/N-5xszh?lnk=MilkMilkSubstit

Here are the main steps in this tutorial: [Download task file here]

  1. Create a Go to Web Page - to open the target website

  2. Save the Cookies - to load product information

  3. Auto-detect the webpage - to create a workflow

  4. Modify the settings of Pagination - to fully load the content on the webpage

  5. Run the task - to get your desired data


1. Create a Go to Web Page - to open the target website

  • Enter the page URL on the home screen and click Start to create a new task

target_0007.jpg
  • Click Option

  • Tick Scroll down the page after it is loaded

  • Set Scroll for one screen

  • Set the Wait time as 3s and Repeats 15 times

target_0000.jpg

2. Save the cookies - to load product information

Since the product information will only be presented if there is a certain shop on Target.com, we need to choose one and tell Octoparse to save our choice.

  • Turn on the Browse Mode

  • Click Please select a Store

target_0005.jpg
  • Input your zip code to find stores nearby > choose the store you need

target_0004.jpg
  • Tick Use Cookie > Click Use cookie from current page > Apply

target1_0001.jpg
  • Turn off the Browse Mode


3. Auto-detect the webpage - to create a workflow

  • Click Auto-detect web page data and wait for the detection to complete

target_0002.jpg
  • Check the data fields in Data Preview and delete unwanted fields or rename them if needed

target_0000.jpg
  • Uncheck Add a page scroll

  • Click Create workflow

target_0001.jpg

4. Modify the settings of Pagination - to fully load the content on the webpage

  • Choose Click to paginate in the workflow > Click Option

  • Tick Scroll down the page after it is loaded

  • Choose Scroll for one screen

  • Set the Wait time as 3s and Repeats 15 times

target_0001.jpg

NOTE: We need to change the XPath of the Pagination. Otherwise, Octoparse may keep scraping the last page and never stop.

  • Choose the Pagination box in the workflow

  • Input the Matching XPath as: //button[@data-test='next' and not(@disabled)]

target00000.jpg

5. Run the task - to get your desired data

  • Click Save on the upper right to save your task

  • Click Run next to it and wait for a Run Task window to pop up

  • Select Run on your device to run the task on your local device

  • Wait for the task to complete


Here is the sample output from a local run:

target00001.jpg
Did this answer your question?