Hacker News Reader

Comment by meritt | Hacker News Reader

If you want to download all the PDFs to your current working directory:

    wget -rH -nd -Dubm-us.net -Apdf http://gdcvault.com/gdmag

Edit: for the curious, the flags are:

    -r recursively retrieve all the links on the page
    -H span hosts (default would stay with gdcvault.com only)
    -nd don't replicate directory hierarchy, just download the files to cwd
    -D only download from ubm-us.net
    -A pdf extensions only

View on HN

Consider grabbing the torrent[0] instead so as not to peg the gdcvault server.

[0]: http://thepiratebay.sx/torrent/8940823/Game_Developer_Magazi...

I was just working on this: wget -r -A.pdf http://twvideo01.ubm-us.net/o1/vault/GD_Mag_Archives/

but getting a 403 (forbidden). Now I see I should have tried http://gdcvault.com/gdmag instead of copying the link from right clicking and saying "Save As"

You do have a few more flags than I do. I should look those up.

Edit: Thanks for expanding your post to explain all the flags. Very helpful.

DownThemAll works with Firefox cross platform. Right Click on the page and dta OneClick. Fast Filter .pdf, .zip covers all downloads.

I was disappointed I couldn't find DownThemAll for Chrome but a quick check on the store found an equivalent plugin called DownloadAll.

But consider using --limit-rate <some rate> or -w <some number of seconds> to add a delay so as not to bomb the server.

Safe downloading, everyone; remember to first check the ToS and don't change your user-agent for any reason.

Also here is the list of all pdfs: https://gist.github.com/schnabear/6657115/raw/9b8a2527cfeca6...

You can save this list to a file (e.g. GDMAG.txt) and run

  wget -c -i GDMAG.txt

On Windows there's Flashget 1.73 http://www.flashget.com/en/download.html and Tools, Site Explorer dialog. It's much faster because it downloads multiple files simultaneously.

Once the download is done, you might want to do:

mv GDM_February_.pdf GDM_February_2010.pdf

to make sure the nameing is consistent :)

A cross-platform solution would be using Opera 12's sidebar, going to links, selecting all 'view' links and downloading all at once. Or use Opera to download the torrent.

Is Opera really more cross-platform than wget?

In my opinion it's easier to put Opera (12) on Windows than wget; also, it doubles as a torrent client. Older Opera versions (that still have this feature) had versions for a large number of OS-es.

Wget has already been ported to Windows, and because it's free software, you can try port it to any other OS. Opera, on the other hand, hasn't been ported to nearly as many OSen, and you don't get the source to help with the effort (plus it's much larger than a CLI file downloading utility). Your idea of cross-platform is weird.

BTW, there are also zip files with code and stuff:

    wget  -c -w5 -rH -nd -Dubm-us.net -Azip http://gdcvault.com/gdmag

to get those

How big is the complete archive?

3.34 Gb according to the torrent.

Reply on news.ycombinator.com