I would like to just crawl the whole website of ~800 different URLs. So I do not want to include specific fields but just crawl everything on the respective sites.
The websites do not have the same structure.
But I would just like to scrape everything on each site
How do I do this?
Please sign in to leave a comment.