You are browsing a tutorial guide for the latest Octoparse version. If you are running an older version of Octoparse, we strongly recommend you upgrade because it is faster, easier and more robust! Download and upgrade here if you haven't already done so!

A dropdown menu is a list of options that only appears when the user clicks on a piece of text or a button. It is often used for websites to offer a range of choices.

At times you may want to actually scrape the selected dropdown option as an individual data field, for example when the drop-down options entail category information. In this article, we'll take a quick look at the different ways you can capture data on a drop-down list.

Tip: If you need guidance on how to switch between options in a drop-down menu and extract page data accordingly, check out this tutorial on how to click through options in a drop-down menu.

In Octoparse, we can either scrape the option values directly like any other data field on the webpage or scrape them from the loop item.


1. Scrape the option value directly like any other data field on the webpage

For this example, we are going to use a drown-drop list from MyCarInfo to show you how this is done.

  • Click the dropdown menu box

  • Select Loop through the options in the dropdown menu to set up a Switch Dropdown loop

loop_through_dropdown.jpg
  • Click the dropdown menu box again

  • Select Extract text of the selected element

extract_text.jpg
  • Tick Extract data in the loop and click Apply

data_in_the_loop.jpg

We can iterate through the options in the loop list and click Extract data to check if we get the selected dropdown option in the data preview section.


2. Scrape the option value using a loop item

If in some cases the option value won't change even when they are selected, we'll need to scrape the option value using a loop item.

  • Click the dropdown menu box to set up a Switch Dropdown loop

loop_through_dropdown.jpg
  • Add an Extract Data action inside the loop

add_a_step.jpg
  • Tick Extract data in the loop and click Apply

data_in_the_loop.jpg
  • Click the dropdown menu and select the SELECT tag and then the first OPTION

1.jpg
  • Click Extract the text of the element

Tip: You can also switch between the different options from the loop to see if the data change accordingly in the preview section.

Did this answer your question?