Hi, I'm tracking protest arrests in my city and I'm hoping there is a template that might be close enough to what I need.
I want to leave the name fields blank and search arrests by "Booked Today" and then "Booked Yesterday." Those might need to be two separate steps or functions.
Once search filter is selected (let's say we start with "Booked Yesterday"), the site returns a list of name URLs to each person's booking page. I need to scrape all that data (including bail amounts, etc) for every person on the list, then return to the main page and run the "Booked Today" search. It doesn't matter to me if all the name URLs are clicked simultaneously or if Octoparse loops through the list until all items are clicked and logged.
For example, I'd like the process to look sort of like this:
(select "Booked Today" from drop down) -->click "Search" button-->click name URLs-->scrape all data-->return to homepage
(select "Booked Yesterday" from drop down)-->click "Search" button-->click name URLs-->scrape all data
Does anyone know of a template that would be applicable?
Please sign in to leave a comment.