wget is an incredibly useful GNU tool on Linux. Unfortunately, it doesn’t come with OS X (as of Mountain Lion). OS X includes curl, which is a very handy tool but lacks at least one important feature of wget: the ability to use wildcards to get multiple files at the same time. For example, let’s say you want download all the example files from the IGES specification. With wget, you could type:

wget http://www.wiz-worx.com/iges5x/wysiwyg/igs/*.igs

Here is how to mimic that process with curl and a few UNIX command-line tricks.

1. Download the directory listing and save it in a file.

curl -L http://www.wiz-worx.com/iges5x/wysiwyg/igs > index.html

2. Use grep with regular expressions to parse the .html file, extract the .igs file names and save them in a text file.

rep -o 'f[0-9]*x.igs' index.html > iges_file_list.txt

3. Use a bash loop to iterate over the text file and fetch each file with curl.

while read line; do
curl -O http://www.wiz-worx.com/iges5x/wysiwyg/igs/$line;
done < iges_file_list.txt