How to click through options in a drop-down menu?

July 21, 2020 08:35
Updated

Tutorial for the latest version 8.1 beta is available here. Go to have a check now!

A drop-down menu is a list of items that appear when clicking on a button or text selection. This tutorial will show you how to select options in a drop-down menu in Octoparse.

You may need this example link to follow through:

https://services2.hdb.gov.sg/webapp/BR12AWRentalEnquiry/BR12PSearch.jsp

1) Click on the drop-down menu

web scraping with octoparse - drop-down menus

2) From the Action Panel, click "Loop through options in the dropdown"

3) Switch to the Workflow Mode by toggling the Workflow switch on the upper right corner. A Loop Item had been created and added to the workflow automatically to loop through options in the drop-down menu.

web scraping with octoparse - drop-down menus

4) Click on the Loop Item for the dropdown then refer to the looped items in the list on the right side. Check if all the items added to the loop were desired; If not, refine the list by using XPath function: position().

web scraping with octoparse - drop-down menus

5) Now we are done configuring for the drop-down menu. Move on to select other options or click on the confirmation button to complete the search.

web scraping with octoparse - drop-down menus

Tips!

When a drop-down menu is detected and created in Octoparse, all available options will be selected by default. However, if you want to modify the list by adding or removing items, it can be done via using XPath function position() to specify which options are to be kept in the list.
In this example, our XPath for the drop-down loop is "//SELECT[@id='selTown']/OPTION", by adding [position()="x"] to the end of the XPath, we can specify if we want to include only options of certain positions, ie. position( )=1, position( )=2, or position( )>1, etc. So, assuming we want to remove the first item from the list since it is not a real option but a header, we will use XPath: //SELECT[@id='selTown']/OPTION[position()>1] to include every single option with position greater than 1.