Real Estate details like photo's

Comments

10 comments

  • Kara

    Hi Titania,

    Thank you for reaching out.

    If you want to click on the list items to get into each detail page to scrape, please follow the instructions in this tutorial:

    Lesson 5: Getting data - Click through links in a list and capture data from each item page

    To scrape the image URLs, please refer to this tutorial:

    How to extract multiple pictures from the webpage?

    Best regards,

    0
    Comment actions Permalink
  • Titania

    Hi Kara,

    Thanks, tried this , getting more data and the images.


    Now work on the sequence and what if empty scenarios (properties options).
    Still looking for things but seems to improve on the results.

    John

    0
    Comment actions Permalink
  • Titania

    Hmm, still run into issues.

    I don't see how to combine the images to the extraction (seem there are two ways, add at the end or get multiple lines (think the multiple lines are better).

    I also have sometimes missing data , then the other data shifts in cells, see property size value 3 line.
    I saw the option to set fixed value in there in case that selected field is not found but seems not to work.



    Also have a bit of a problem with interface, when I change a field (eg sequence) it gets messed up , but there is not always an undo option.

    I do like the way it works and need to spend more time of course.

    Hope you have a suggestion?

    Thanks

     

    John

    0
    Comment actions Permalink
  • Kara

    Hi John,

    Thank you for your reply.

    Regarding your first question "how to combine the images to the extraction ", could you please send over the page URL for checking? And let me know what's the output you'd like it to be.

    By "saw the option to set fixed value in there in case that selected field is not found but seems not to work.", I'm assuming that you've already checked and followed this tutorial Data fetched to the incorrect data fields? to locate the target data field? If it didn't work out when you tried it, perhaps you can send me the specific page URL and I'll help modify the XPath of "Propertysize" as an example for you?

    For the interface issue, if you are using the new version, since it's still a beta version, the system might be not that stable sometimes, we'll resolve the issue with the next release. As for now, please try to click "OK“ and "Save" to save the change, or reboot the software to try again. Sorry for the inconvenience brought by.

    Best regard,

    0
    Comment actions Permalink
  • Titania

    Hi Kara,

     

    On the extract of data.
    Two examples of RE sites:

    http://www.joubertrealty.com/listings/sale-index

    https://wigboldrealestate.com/status/te-koop/

    Both (and actually most RE_Agents) have similar setup.
    A page with listings with some information and then you can click the listing and you get more info including the pictures.

    I would like to get as much listing data so I can feed that into my website to combine all listings on the island (the Househub.biz site). For this site there is an API that I can use to send listing data.
    At the moment I just am exploring the possibilities of Octoparse, export to CSV is what I use.
    Obvious fields :
    Address, type of accomodation, no of bath and bedrooms, Property size, price , description, their reference ID.

    Then the pictures of the listing. The number of images differ. So if I would add columns this may be less handy for further processing, I had at some point the csv with multiple lines for the same listing (all data) and last column a picture link.
    Think that would be best.

    As mentioned , when for a listing on the website information is omitted(eg Sq Footage) then I need to get a default values (for SQ FT that would be 0)  as otherwise the rest of the data shifts one column to the left.
    I tried setting this using the fields (right click) that were found, just looked at the link, I think I will need to check that first as this seems different from what I tried.

    Beta Version.
    Ok , will try that.

    I tried to find whether there are training options (direct sessions) but this seems only possible when you take the Pro Plan.
    But that is for what I am doing at the moment to expensive to get into.
    I am willing to pay for direct training so I can get better grip on the options of the program. Maybe there are options available that I have missed?
    I would think an hour session should get me started pretty good and thenif needed get an extra session to fix any strange issues I run into.

    Appreciate the help!


    Kind regards


    John

    0
    Comment actions Permalink
  • Titania

    Hi Kara,

    hope all is fine.
    Did you have a chance to look at this?

    Let me know if anything needs clarification.

    Regards

    John

    0
    Comment actions Permalink
  • Kara

    Hi John,

    Sorry for the late reply, I was out of the office before.

    If you want to scrape data from the list page and detail page at the same time, please check: How to scrape from the list page and detail page at the same time?

    To scrape obvious fields like "Address, type of accommodation, no of bath and bedrooms, Property size, price, description, their reference ID". Please follow this tutorial to revise the XPath: How to associate data with nearby text?

    You can also refer to the case tutorials on how to scrape real estate data:

    https://helpcenter.octoparse.com/hc/en-us/sections/360002177472-Scrape-Hotel-Real-Estate-Data

    And yes as for "The number of images differs", if you don't want to have the number of image URL columns for each data line to be different, it's better to scrape the source code first, then format out the image URLs as that tutorial introduced.

    Regarding your last question, step-by-step training is one of our paid services. The cost of it is $150-200 per hour. Let me know if you are interested in that.

    Best regards,

    0
    Comment actions Permalink
  • Titania

    Hi Kara,

    Thanks, tried some of these tutorials, but ran into the issue that I use the 8.1 Beta and these tutorials are for previous version.

    Reverted back to older version but then some of the steps (actions) seems not to show.
    Frustrating as I know what I want to achieve with the extract but seems to loose time with the GUI of the program.

    I would like arrange for the training and hope this will allow me to fully use the program features and achieve my goals.

    Could you let me know how this can be best arranged and whether the training goals can be agreed upfront?


    Thanks

    John

     

     

    0
    Comment actions Permalink
  • Titania

    Hi,

     

    I would really like to speed the project up, I would like the step by step training.

    How can we make an appointment?

    Kind regards

    John

    0
    Comment actions Permalink
  • Kara

    Hi John,

    Please check the reply via the ticket you submitted. Thank you.

    Best regards,

    0
    Comment actions Permalink

Please sign in to leave a comment.