Hacker News Reader

Comment by ryankrage77 | Hacker News Reader

You can bulk download these with wget like so;

  wget http://www.ghibli.jp/gallery/{ged,chihiro,karigurashi,ponyo,kokurikozaka,marnie,kaguyahime,kazetachinu}{001..050}.jpg

However they're low quality 1080p jpegs, so you're better off splitting out the frames of a blu-ray rip of the movie using ffmpeg or something.

View on HN

Download them over WebTorrent directly in your browser:

https://instant.io/#e371e76f57923f555c0585910420137cd615911a

magnet:?xt=urn:btih:e371e76f57923f555c0585910420137cd615911a

Cheers, this is such a neat website.

once downloaded to the browser is there an easy to dump the images to disk?

Look for the "Download all files as zip" link.

Wow thanks that's impossible to see. It just blended in with the file list and I completely missed it.

Thank you! I had missed that.

Without addons, I suppose "Save Page" is your only option. Or tediously save each image manually.

In firefox the "Save Page" doesn't save the downloaded images. I'll have to investigate the addon options..

So far on Firefox neither "Down them all!" nor "Download All Images" recognized the torrented images.

Print to PDF and convert to web page? I’d be interested to know if the JPGs move through that one untouched.

bulk image downloader has worked really well for me in the past

...Found the guy who crashed the site ;)

You can add delays with -w 10 (10 seconds) or --random-wait says https://stackoverflow.com/questions/35771287/delays-between-...

--random-wait with the default tries (20) is working for me

Don't forget --timestamping in case the command fails and you want to try again, but only download files you don't already have.

It's really the tragedy of the commons and why we can't have nice things. Ghibli went out of their way and gave something nice, only to have it immediately abused by gluttonous data horders.

Yeah, it's definetly the ~100 people using a shell script to download a couple of jpegs crashing the site. Not millions of regular people looking around.

This is a nice use case for IPFS.

People `ipfs pin` the images to satisfy their data lust, and instantly become part of the growing swarm of computers which are now serving the content to new visitors.

Or just put it on a tracker, like we've been doing for 20 years.

https://instant.io/#e371e76f57923f555c0585910420137cd615911a

magnet:?xt=urn:btih:e371e76f57923f555c0585910420137cd615911a

Thanks for sharing – 8 peers and downloaded in 30 seconds!

This is a nice use case for a regular and boring CDN, caching images and making them available without crashing a web server is a solved problem.

You've described a solution that puts the cost/work on the provider. IPFS and BitTorrent shifts that to the consumers.

€50 per month for 50TB of traffic = 50 mio x 1MB file downloads. The costs are so low, that CDNs are practically free. Oh wait, CloudFlare IS free for such primitive use cases.

You ignored the costs of labor.

Not that it matters, my point stands. The cost exists and different solutions put it in different places.

Or pin them on Skynet. Which is just a better implementation of IPFS.

https://siasky.net/JACbilLSW_T_yzeNlYlw_pIqh_hVEeEMvc9Vz_SBE...

Are there metrics that indicate this, or is it better in other ways also?

The big difference with Skynet is that you pay for professionals to pin the data on your behalf. It works better for unpopular files because you aren't relying on the uptime of people at home.

It also works better for popular files because the professionals generally have great bandwidth and good peering.

There are a bunch of other benefits as well, such as access to an api that allows skynet pages/apps to upload and download. For something like this you wouldn't use those endpoints but if you wanted to make say a decentralized blogger, you could use those endpoints to upload posts that users write.

> because you aren't relying on the uptime of people at home

IPFS doesn't automatically spread whatever you put into your client, so you should never rely on others to randomly host your content for you. In fact, if you start a default IPFS node, add your content then turn it off again, no one is gonna have the content you added.

Would have been interesting to hear a fair/impartial comparison between Skynet and IPFS but your misunderstanding of IPFS is now evident so maybe someone else can fill in the gaps?

Out of their way? Releasing a few pictures they already stored digitally hardly seems like a lot of effort.

Or the studio could have provided a single zip/tar for the data archivist. ; )

A torrent would be easiest on their servers.

Check out https://letsenhance.io/ I've been using it for >year to get print quality images. Works great

To piggyback off this - As a graphic designer, I've tried basically all publicly available upscaling tools, and Gigapixel AI is by far the best one.

If anyone upscales this collection, can they send me a message (all modes of contact on https://maya.land/)? I'm happy to send you some $ to be able to download the higher res copies; seems like a waste to do it twice.

I've given this a shot using Pixelmator (https://www.pixelmator.com/blog/2019/12/17/all-about-the-new...)

Expect an email with the upscaled images sent via WeTransfer.

Thanks!! Appreciate it. :) Let me know if you've got a ko-fi or whatnot?

> low quality 1080p jpegs

imagemagick reports their quality as 75, based on a few I picked at random:

  identify -format '%Q' marnie007.jpg
  75

--- which is not terrible but not great either. The thumbnails are PNG. It's too bad they didn't use that for the full-size ones.

I know shell can expand, but this is my first time seeing somethings like this. I didn't know shell can do things like Cartesian product before.

That's a bashism, fwiw

This was in the BSD C Shell (https://en.wikipedia.org/wiki/C_shell) way back in the early 1980s!

> However they're low quality 1080p jpegs

Good enough for me :) They look nice.

Thanks for the wget tip!

I did not know I could do {001...999} with wget! wow!

That's a shell feature, not wget's.

Curl has it built in:

    curl -O 'https://example.org/[2-4].jpg'

(quotes disable any kind of shell expansion)

Really? I thought the shell could only expand globs if it matches files on the local file system?

The feature is called brace expansion:

https://www.gnu.org/software/bash/manual/html_node/Brace-Exp...

You may also be interested in yet another powerful feature that is parameter expansion:

https://www.gnu.org/software/bash/manual/html_node/Shell-Par...

Filename expansion (nitpick: often mistaken for globbing) is performed last:

https://www.gnu.org/software/bash/manual/html_node/Filename-...

> nitpick: often mistaken for globbing

What exactly would you call globbing if not the feature that is controlled using the nullglob, failglob, nocaseglob and dotglob options as well as the GLOBIGNORE variable.

Globbing has an ambivalent definition given that it’s mostly used for filename matches (especially given its origin as the glob command), but it’s really a specific form of pattern matching using wildcards such as * and ?, it need not apply only to filenames or paths on the filesystem: this matching is also used to match strings in case/in/esac constructs and test’s == !=

https://www.gnu.org/software/bash/manual/bash.html#Condition...

I guess one could say globbing is applying this pattern matching to filter a list.

You're right regarding glob functionality, but the curly braces are a separate shell feature. The most useful idiom I use it for is renaming or copying, e.g.:

cp filename{,_backup}

Thoae aren't globs, if you put an asterisk there it would not work.

Of course, this only works well with non-ancient bash, not the ancient version which ships with MacOS, which will do {1.999} but can't do the leading zeros (you can use printf '%.3d' as a separate step).

brew is your friend here, it has the latest version. Works well.

Afaik macos ships with zsh now, which supports that feature.

It isn't bound by command. You can use it everywhere!

You mean, you didn't know that wget can be given more than one URL argument it can fetch, and that Bash can generate arguments with interpolated numbers using {from...to}.

Could you upload an archive for the others since the site went down? With the note about usage preferrably.

Dumb question but I'm getting 404 errors with the wget command (but the urls work fine in the browser). Any advice?

I got the same error on zsh, but not on bash

I was using fish, wasn't working there either. thanks :)

good enough as photo reference!

Reply on news.ycombinator.com