Hijacking HTML canvas and PNG images to store arbitrary text data | Hacker News Reader

Hijacking HTML canvas and PNG images to store arbitrary text data

igorkromin.net

70 pointsikromin41 comments

Similar, prior work: the video game "Spore" let you export a PNG of the creatures (and other things) you made with its many, many editors. It'd output a PNG of the critter, with all the data needed to build it from Spore's parts library encoded in the least significant bits of the pixels; you could easily share it in email/chat/whatever, then drag it into Spore to get the critter.

Lots more appealing than a square of grey noise, lots easier to understand when you run across it a year later...

edit: some more details on this, with examples https://nedbatchelder.com/blog/200806/spore_creature_creator...

Pico8 does the same thing http://pico-8.wikia.com/wiki/P8PNGFileFormat

A clever use of steganography, it seems, but one that would probably be ruined by image "optimizers" that are all too common today.

I suspect that is exactly why they encoded it into the pixel data rather than extra tEXt blocks. Extra blocks and metadata like EXIF and geotags get reaped by optimizers but pixel data is pixel data. (unless you resize or alter color depth.)

Or your favourite social media platform indiscriminately converts PNGs to JPGs.

A video game I contributed to, Gang Garrison 2, stores maps as PNG files where the image is the actual level background, and other data is hidden in a text PNG chunk. Hides data in the file format, not the image proper, but a similar idea.

There is also Jmol, a molecular editor, that allows you to save scripts as .pngj files:

http://wiki.jmol.org/index.php/File_formats/Compressed

Someone did something similar to make a point during the 3d printed gun debate. There's a .png floating around of the PM522 (3d revolver) with the print files encoded directly in the image. IIRC it just requires zipping and changing the extension to load correctly.

That's just concatenating the files together and relying on the zip extractor to ignore the PNG part of the file, rather different than actually embedding it in the image data itself. I remember this being a popular way to upload non-image files to certain imageboard websites a few years back, to get around them only allowing image uploads. JPG+RAR was the usual choice.

The data was visible as a block of jumbled pixels as the bottom, not sure how concatenated files render visually (especially jpg format), so for all I know it may have been exactly what you're describing. Pretty sure I found it on an image board anyway.

That's how it looks when you concatenate two file types. I guess in theory if you make the image a pixel, you effectively get the same output but with way less effort.

Unfortunately, MobileSafari is making it harder and harder to actually save the image. If you gently press and hold, it'll give you the option to Save to Camera Roll. But with the advent of force touch, pressing too hard means that you'll end up navigating to a page with the image on it, losing your original page's state in the process.

It is possible to catch the force touch event, but as far as I can tell it overrides the regular touch event so all you can do is prevent the event and print "you pressed too hard, please release and try again, gently".

When was the last time you actually tried this? Because the reality doesn't match your description at all.

First off, you have to put a pretty decent amount of pressure on it to trigger the 3D touch behavior instead of the long press behavior.

Secondly, when you do trigger 3D touch, it goes into "peek" mode, and you have to keep pressing, and in fact press even harder, before it will "pop" and actually navigate to the image. In the "peek" mode simply swiping up will give you the Save Image action.

And if you're the Hulk and every single touch is a force touch, well, even gripping as hard as I can, it still forces the "peek" mode for a second or two before "popping".

Thank you; peek mode + swipe up does prompt to save the image and I did not know that. Based on user feedback, the "press-and-hold" method is the only one people commonly know about, and the major issue is that if you accidentally peek (or even imperceptibly drift into peek mode for a few ms) it won't work, and you won't get any feedback as to why.

It's handy to have a second way to trigger save, but having to include the explanation in order to teach users how to execute the maneuver is not great UX. "Press gently and hold to save. Or, if your phone supports Force Touch and has it enabled, press pretty hard (but not _too_ hard) and swipe up to save. If it doesn't work, try it again."

Force Touch in general has difficulty in exposing actions and discoverability in general.

I did similar several years ago but built my own index and rendered a character per channel... https://github.com/byteface/GTP/blob/master/sentiment/canvas...

meaning 3or4 chars per pixel. then tried to do boyer moore on a dynamic shader. At the time I felt I had the fastest string searching tool on the web, in the world... https://github.com/byteface/GTP/blob/master/play/simplified/...

it could all the 'the' in moby dick in a split second. which at the time would take textmate a few seconds.

showed a mate who then did his own storage only implementation, closer to what you are doing: https://github.com/claus/PNGDrive

years after saw someone did one that stored data in flckr. Think it was posted here.

To be honest, IMO the idea is senseless as pure storage as theres other better options. like zip. But if you are going to do something with it on GPU that's when it has new potential.

Sigh safari is the new IE. Making everyone else suffer from nonsensical hacks.

But anyway, if the problem is that the json file is opened in a new tab instead of downloading, can't you just encode it with a non-text media type in the data-uri so the browser doesn't understand it?

E.g, data:application/octet-stream instead of data:image/png which is what you claim works right now

data uris do not support setting something equivalent to http’s disposition header, which is what prompted the addition of the download html attribute. data uris also do not permit setting a sensible filename, the other feature the download attribute patches over. safari supports neither, and mobile safari is also in general incapable of sensibly dealing with an anonymous blob like that. (what could it even actually do? what app would you open it with? what would it be named?)

Uh you can definitely set the mime type in data uris. It's like right there as the first part of a data uri...

Download attribute let's you choose a filename. But giving up on that for safari sounds like a decent tradeoff to me (with the current approach you end up with a nonsense png extension in the save file anyway).

Testing with a quick demo: https://codepen.io/anon/pen/wYpBry?editors=1010, it seems like mobile safari will show it as unknown.txt but have options to save to files/dropbox etc. Kind of complicated but probably still less confusing than a png file.

The download attribute is not supported on mobile Safari unfortunately - https://caniuse.com/download/embed/description

https://codepen.io/anon/pen/wYpBry?editors=1010 is very cool, thanks!

I didn't say anything about mime type. I said disposition (which is what you would use to force a file to download, instead of display in browser, using the http protocol), and filename. If you set mimetype to octet, mobile safari, since it does not support the download attribute, cannot name it anything sensible or choose an appropriate file extension.

I always felt like data: urls could obtain a lot more utility if disposition and filename attributes were added to it, but... uhm, browser makers don't seem to agree, and the syntax would make it difficult to add features like that whilst staying backwards compatible.

> Sigh safari is the new IE. Making everyone else suffer from nonsensical hacks.

I can confirm that, most of my time spent for browser support is on IE11 and Safari. Everything in Edge, Firefox and Chrome generally works fine.

I did something vaguely similar storing text in QR codes then converting the black and white squares to audio 'noise', which could be saved as a wav then read somewhere else, by reading the audio 'noise', reconstructing the QRCode and reading that ;) - It's not a million miles away from the 'chirp' app idea.

demo at https://qrdio.com

What I would be interested in is if there would be a way to encode data in an image that is robust to various kinds of jpeg/back/shrinking/enlarging transforms. This is really cool and useful but there's all kinds of normal processes that images go through that would destroy data hidden in an image this way. Perhaps something akin to parity or error correction data hidden somewhere else?

Why, yes, there are ways to do this.

One of the interesting ways is adding is adding noise generated by inverse Fourier transforms.

http://repro.grf.unizg.hr/media/Ante/Radovi/033008_1_FINAL.p...

Sounds like image steganography. If anyone wants a python example here ya go https://github.com/jermainezhimin/culpriteCTF/blob/master/LS...

This reminded me of a steganography blog post I wrote back when I was doing computer security.

It should be possible to binarily concatenate compressed (RAR'd or ZIP'd) text data directly to a JPG.

Then you can save it as an image on the client, or uncompress it in the client browser and read the data.

The differences are 1) the saved image won't look like random pixels-- it'll look like whatever base image you choose and 2) you don't have to worry about writing the encoding/decoding data as bitmap stuff.

Reminds me of that 4chan spam attack which was an image that told users to save it and change the extension to an executable, which would then execute and make more posts with the same image, continuing the cycle. I believe that they started scanning images for random appended data after that

This is not really new. One of my friends built a DAWG into a PNG.

https://en.wikipedia.org/wiki/Deterministic_acyclic_finite_s...

The DAWG contained all french words, in a convenient small picture.

Did s/he publish somewhere? I'm interested in learning more, thanks!

A while back I built a site for converting torrents into banner-sized images, with the idea that people could share them on message boards and such, and there were browser extensions that would detect them and let you open them in your bittorrent app.

What site is it

Depending on the amount of data, you could store it in the URL itself as a fragment identifier

Agreed, but you would not want to use a URL that is larger than around 2k characters, in my case that wasn't enough.

Maybe you could store the image in the browser cache. Then users wouldn't need to manually save and upload the image. Most browsers will keep the image cached until the user manually empty the cache (eg forever).

If you're using PNGs, why skip the fourth alpha channel?

because he used ascii codes which uses 3 numbers and he stored a number on each channel. method requires no table to encode/de-encode

Nope, and I explain it in the article, it's because of the weird behaviour you get with canvas and alpha channel. If somehow a value of 0 made it to the alpha channel, all of red, green and blue channels were reset to zero as well. It was a bit annoying to deal with and a total pain to change my code. Originally I was using all 4 channels.

ah i see now. ur using ascii code as the color value. i used to gen them with a php lib don't recall experiencing this error on alpha. browser thing?

Elite did this way back to store the entire galaxy.

Reply on news.ycombinator.com