Friday, 12 October 2012

Using wget to get multiple files from http servers

The Unix wget utility support globbing, but only when pointed to Unix FTP servers. So this could work:

wget ftp://something.com/*.jpg

but this will not

wget http://something.com/*.jpg

What do you do when you want to do this for an http server? A common case is where you have numbers for example, like

http://something.com/0.jpg
http://something.com/1.jpg
http://something.com/2.jpg

etc.

In this case the solution is

wget http://something.com/{0,1,2,3,4,5,6,7,8,9}.jpg

In this case the command will be repeated for all numbers and you get all 10 images in one go.