Difference between preview data and scraped data

Follow
wpmkoenig
  • Edited

Hi community,

I am very enthusiastic in Octoparse.
Unfortunately, I have a problem right now. I'm scraping some products data from a website including the product thumbnail. In preview of the scraping taks, the img source is displayed correctly (for example https://website.com/123.jpg) for all products individual. But if I start the task, this field won't be filled that way, but with some unusable data like data:image/gif;base64,R0lGODlhAQA...

Does anyone have an idea how I can fix this problem?

EDIT: 
The problem is solved. Because the webpage images were loaded with lazy load, the picture were loaded when scrolling in preview, but not when starting the scraping task. The option "scroll down to bottom" does not help in that case, because loading the pictures is skipped. To load the pictures correctly, when they are loaded by lazy load, you have to scroll for one screen and repeat it enough times. 

 

0

Comments

2 comments

Sort by Date Votes
  • BlueBytez

    Solved how? I understand what you are saying about the lazy load, but as soon as I try solve this problem then even the preview fails to show the URL content the same as the Task run does.

    0
    Comment actions Permalink
  • wpmkoenig

    Yes, currently it is not displayed correctly in the preview, nor in the task. Try to scroll down manually during running the scraping task, but before the scraping of the content starts. For this i suggest to use a wait time after loading the site. In this way the pictures, which you see in the screen, can be loaded. So the URL should be scraped. If this is successfull, choose "Scroll down by one Screen" for enough times in the task configuration, so that this scrolling is done automatically. 

    0
    Comment actions Permalink

Please sign in to leave a comment.

Didn't find what you were looking for?

New post