I first learned about this technique from Cody Brocious' post (http://daeken.com/superpacking-js-demos) about his WebGL demo 'Magister' (http://demoseen.com/windowpane/magister.html). That file has a .html extension, but it's also a valid PNG. In his post, he discusses this technique as well as others.

Cody's goal was to compress already minified WebGL code and achieve some pretty impressive results from a ridiculously small file for a demo contest. He also put the JS loader (which is much smaller than the linked decoder) in a custom chunk on the PNG itself. By doing it that way, you can have a PNG that when renamed also contains a web page and/or more Javascript.

It's an interesting technique, for sure, but I've never been able to find a real use for it beyond the compression. With a content-type sniffing vulnerability or something, it could be an interesting way to deliver content to browsers, but otherwise it's really just a curiosity as best I can tell.