I’ve been trying to download roughly 1,500 pictures from a website for several hours now with zero knowledge of scripting and this is so far very painful so I need help. The website I’m trying to download the content from is a flash-based website so I can’t scrape through like with normal HTML.
Not knowing this, I tried using the Chrome web extension called “Web Scraper” but couldn’t select anything on the webpage and I’m assuming that’s because flash uses so called “binary format”.
So I decided to bust out the good o’l developer tools and this is what I found:
When I visit the Request URL under “Headers” on a separate tab, I can see the image and download it but theres no way I’m gonna do that 1500 times…
After a bit of research on scraping I stumbled upon a program called wget which seemed like it would be helpful in this situation so I downloaded it and tried it on one of the request URLs. It worked perfectly and downloaded the image. After some playing around with it I found out that you can chain a list of URLs in wget to download several pages at a time so I thought to myself that maybe I could copy all the Request URLs from the Headers tab in Chrome, paste them into wget, and I’d be good. However, in order to copy the URLs I have to click on each image individually which would take forever.
You also can’t choose a file in the “Name” window, shift-click another file to copy the list of highlighted material because, as you can see, there are ellipsis (“…”) between the names of each file in order to shorten them (and I’ve tried to expand the “Name” window in hopes that it would show me the entire name but it didn’t).
So is there any way to download several of these files at the same time using some kind of script? Any resources I could use to learn scripting that would be helpful?
Any alternate approaches that could be helpful? Really anything that can cut down my time here would be appreciated. Thanks :).