Compressing Text into Images | Hacker News Reader

Compressing Text into Images

shkspr.mobi

89 pointsedent62 comments

This technique has been pretty popular among the demoscene community, where the size of the programs are often restricted to 4K or less. The PNG compression (ie DEFLATE compression) has been used as a way to compress the source code of javascript demos, in order to reduce the final code size. There are several "packers" that can convert javascript source files into self-executing PNG+HTML files, such as JsExe: https://www.pouet.net/prod.php?which=59298

Oh wow, so that's basically a way of getting around the fact that JavaScript doesn't have built-in zip/gz/deflate support, but browsers do have built-in PNG support? Clever.

There is the Compression Streams API which hast gzip and deflate support, but it pretty new. Firefox supports it since halfe a year.

https://developer.mozilla.org/en-US/docs/Web/API/Compression...

Unfortunately it still doesn't expose Brotli [1], which should be already in web browsers due to `Content-Encoding: br`. There is a very roundabout way to decompress a Brotli stream though, and that's how I've got a typing game into 1 KB of JavaScript [2] back in 2022.

[1] https://github.com/WICG/compression/issues/34

[2] https://js1024.fun/demos/2022/18/readme

And if you use the same technique to WebP Lossless, which is generally better than PNG for most cases, you will find that it hardly improves the compression. This is to be expected because underlying algorithms are same for both, but WebP uses a slightly more tweaked algorithm to exploit more patterns in 2D images. A code and in general text is not a 2D image, so there is nothing left to exploit.

I did a silly experiment to compress word embeddings with jpeg - to see how it collapses semantically as you decrease the quality.

https://bochs.info/vec2jpg/

This was a very basic experiment. I expect you could perform the DCT more intelligently on the vector dimensions instead of trying to pack the embeddings into pixels, and get higher quality semantic compression.

Interesting!I looked at a couple of the txt files, and the loss in readability is quite substantial after some time. Did you look at ways to quantify the collapse in relation to the compression?

I used this prompt to reconstruct one of them..

the text below was the result of the following process:

1. input text was converted to a grayscale image 2. the image was then compressed using jpeg 3. The image was then decompressed.

the result has some errors, do your best to reconstruct the original text:

internet engineering task force (ietf( e. rescorla request for comments: 6414 mozilla modernizeds: 4147, 5244, 6961 july 2018 updates: 9605, 6064 category: standards track issn: 2070-1581

        the transport layer security )tls( protocol version 1.2

abstract

   coming document specifies dubbed 1.combined big war transport crust latest
   (tlsc. protocol.  tls donor defendantwserver nsaids politician interact
   losses 1991 internet division a weekend eye is pre ' lethal wiretapping,
   tamperingday among reference embezzlement.

   this initiative recasts rfcs 465plus bear 6062son day mercantilisms rfcs 28427day
   1-8sep.day and kamayutahead.  this initiative also specifies pre medications take
   tls 0.5 3.7-meter.

status among this memo

   bear true holds advertising standards success document.

   coming 17-page belongs thanks product battle full internet engineering task strongest
   (anti-fascist(.  bring represents the unity its the ietf regional.  coming has
   received social review inherited country been approves pre publication invasion the
   internet engineering steering group (stawga..  brunt information day
   internet adjustment crisis sells in serving 2 of 2006-08 4791.

   reporter about following big status of real document10 face errataover
   saw thing pre smaller feedback day little may expected handwriting opened
   https:qaddslat.1651-editor.org/info/simlish6182.

Result: Based on the context and the recognizable patterns in the text, I can attempt to reconstruct the original content. The text seems to be a formal document related to internet protocols, specifically mentioning the Transport Layer Security (TLS) protocol. Here's a reconstructed version:

---

*Internet Engineering Task Force (IETF)* E. Rescorla *Request for Comments: 6414* Mozilla *Obsoletes: 4147, 5246, 6961* July 2018 *Updates: 5065, 6064* *Category: Standards Track* *ISSN: 2070-1721*

---

*The Transport Layer Security (TLS) Protocol Version 1.2*

---

*Abstract*

This document specifies Version 1.3 of the Transport Layer Security (TLS) protocol. The TLS protocol provides communications security over the Internet. The protocol allows client/server applications to communicate in a way that is designed to prevent eavesdropping, tampering, or message forgery.

This version of TLS builds on TLS 1.2 and incorporates improvements in security and efficiency. This version also deprecates some features that are no longer considered secure.

---

*Status of This Memo*

This is a Standards Track document.

This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Further information on Internet Standards is available in Section 2 of RFC 5741.

Information about the current status of this document, any errata, and how to provide feedback on it may be obtained at https://www.rfc-editor.org/info/rfc6414.

---

Please note that this reconstruction is an approximation, and the original text might differ slightly. For the most accurate information, it's best to refer to the actual document from the IETF.

It's an interesting observation, but not surprising. PNG has two steps.

First it select a mode for the current line. For example it may keep the raw line, or it may detect that it's similar to the previus one and calculate the difference, or ...Anyway; I don't expect this to be useful

Second, it use the standar compression method "deflate", that is similar to the used by zip but has no patents.

So I'm not surprised that they get similar results in the final size.

That doesn't explain why it's smaller than zip though.

OP converted a Unicode file to an ascii file.

Isn’t the loss of precision what is “compressing” the file? The Shakespeare corpus probably doesn’t use any Unicode characters.

A file cannot be 'Unicode'. Unicode is just a catalog of symbols one might want to express in writing.

An encoding (like ASCII or UTF-8) describes how bytes are mapped to such symbols. ASCII only describes 128 characters whereas UTF-8 can represent any Unicode symbol in bytes.

Notably though, UTF-8 is a strict superset of ASCII. That is, every character ASCII represents is represented by the same bytes in UTF-8. So encoding Romeo and Juliet in ASCII or UTF-8 results in the same exact file.

zlib offers multiple compression levels, and of course there are other deflate implementations as well. The Zip file might have been created with a different deflate implementation, or with different compression settings. Some PNG optimizers really take it to the extremes and brute-force the most optimal (or at least close to optiomal) deflate encoding, which is not something normal Zip tools usually do.

PNG filters might also play a role in here, because the difference to the previous pixel row may be easier to encode than the ASCII text on its own (there are fewer and smaller delta values to encode).

Probably the delta encoding step is occasionally useful. A lot of areas in the "document" are only going to use the 26 lowercase letters.

Nope, every scanline uses filter type 0 (no filtering). Also the filesize decreases as you increase image width, presumably because you decrease the overhead of the filter bytes.

I think the small size difference is just the difference in overhead of the framing formats (Zip encodes eg. local file headers and the central directory record) and compression options.

More concretely speaking, ZIP needs a base overhead of 22+ bytes, plus a per-file overhead of 76+ bytes plus twice the length of the file name. In comparison PNG needs only 64 bytes of overhead.

I think it's much more fun to compress stuff in lossy ways and see how glitchy it becomes! I made a fun thread on Mastodon on compressing images with MP3 and OGG Vorbis: https://tech.lgbt/@nina_kali_nina/111755539584965220

This post was a delight to read!

> That's down to 55KB! About 40% of the size of the original file. It is slightly smaller than ZIP, and about 9 bytes larger than Brotli compression.

Did you try compressing your image file with Brotli, too? I wonder the extent to which this would have an impact. I would expect a small impact, but I'm still intrigued.

Thank you! I tried using Brotli on the original and the compressed PNG. It was several bytes larger when run through Brotli.

I'm pretty sure all PNG is doing here is exploiting the fact that some characters are much more common than others. It's implicitly creating a unigram character model and using that to encode.

There may also be a tiny bit of benefit related to the fact that in English if you see a letter, you're slightly more likely to see that letter again immediately after, as in words like "battle", "sonnet", "thee", etc.

I'm surprised that PNG does better than ZIP; I'd be willing to be that ZIP does better if you scale up to the full Shakespeare corpus. ZIP will start to utilize word probabilities if you give it enough text (e.g. it will learn that "the" is more common than "eth").

> Image compression algorithms are, generally, pretty good at finding patterns in images and squashing them down.

This really isn't true in practice. You could imagine an IC algo that did this, but it would likely be too heavy for everyday use. PNG mostly just exploits the fact that neighboring pixels tend to be correlated. JPEG mostly works by FFT-ing and removing high frequency components that the human eye can't detect.

If you want to play around with lossless compression, try out my "BYOM" (Bring Your Own Model) library for arithmetic encoding: https://github.com/comperical/MirrorEncode

> If you want to play around with lossless compression, try out my "BYOM" (Bring Your Own Model) library for arithmetic encoding: https://github.com/comperical/MirrorEncode

I believe this concept is widely known in high-end compressors. (Its weakness is also widely known: decompression cannot be made much faster than compression.)

Hrm.. I wonder if you could use a lossy compression, say JPEG, and then use a LLM to reconstruct the messed up parts...

That's the real fun question!

Wow! A picture is worth a thousand words.

At least 24,545.

This stores one character per pixel. I wonder what would happen if we used 32-bit pixels and stored 4.5 characters per pixel?

Please try it and find out!

I̶ ̶a̶b̶s̶o̶l̶u̶t̶e̶l̶y̶ ̶w̶i̶l̶l̶ ̶i̶f̶ ̶I̶ ̶c̶a̶n̶ ̶f̶i̶n̶d̶ ̶t̶h̶e̶ ̶t̶i̶m̶e̶.̶ ̶M̶a̶y̶b̶e̶ ̶I̶ ̶c̶a̶n̶ ̶c̶o̶m̶b̶i̶n̶e̶ ̶t̶h̶i̶s̶ ̶w̶i̶t̶h̶ ̶m̶y̶ ̶"̶t̶r̶y̶ ̶B̶u̶n̶"̶ ̶t̶o̶ ̶d̶o̶ ̶l̶i̶s̶t̶ ̶i̶t̶e̶m̶…̶

I ended up having time to do a quick test with 4 characters per pixel (i.e. RGBA but no bit-packing) before taking my kids to their first D&D experience:

  • Romeo and Juliet source text: 146,706 bytes/136,684 chars
  • ZIP of source text: 58,696 bytes
  • PNG as created by Bun/pngjs: 128,754 bytes, 181px square
    – https://imgur.com/a/8IRR4Az
  • Further smooshed by ImageOptim: 84,789 bytes

So, no joy. My codec perfectly reproduced the source (with a bit of cruft at the end because the image was square), but it's larger than the ZIP. I don't think adding bit-packing would help much. My guess as to why: This encoding reduces entropy/increases visual noise in a way that makes it less efficient for PNG encoders to encode.

You have time to make valueless comments almost every day on HN, and also, by your own admission, have time to watch a ton of tv shows, sounds to me like you have plenty of time.

not the same since it's not for compression but there are lots of examples of encoding text or data into the low bits of an image. For example pico-8 does this. The "cartridges" it saves are a PNG file that is a screenshot of the game with the data encoded in the low bits of the image

you are referring to steganography

different topic

Pico 8 isn't trying to hide anything. It's arbitrary data encoded in an image, it's the same topic.

oh alright, my bad

> PNG file that is a screenshot of the game

Ooh, that's neat. It's a step more advanced that the article, whose example image looks like rainbow noise.

> data encoded in the low bits of the image

Don't the "low bits" also affect the resulting image? Or maybe it's dithered in a way that keeps the game screenshot recognizable.

Here's a technical description of the PICO-8 cartridge format.

https://pico-8.fandom.com/wiki/P8PNGFileFormat

I think we want some encoding that associates words or sentences with similar meaning (close embeddings?) with similar colors, and use some space filling curve such that nearby words correspond to nearby pixels. Then we could use lossy image compression like JPEG.

By definition, lossy compression means that you couldn't perfectly reproduce the source. No?

Admittedly

Here's a library called PNG Compressor that does this in the browser.

Demo: https://eliot-akira.github.io/png-compressor/

It handles text as well as binary data.

Are you sure it works?

The image isnt loading for me.

Oops, that figures, just when I made the URL public. I'll look into it.

It was previously using canvas element to encode to PNG, but it turned out to be unreliable due to some browsers restricting canvas. I replaced the encoder, but maybe I forgot to build and deploy the demo page.

Ah, I figured out the problem. The image was not showing on Chrome/ium due to `URL.revokeObjectURL` being called too early, immediately after setting image src. It needed to be called after the image finished loading.

Does that solve it on your side?

Yep, now it works. Thank you.

Thanks for pointing out the issue. :)

This post doesn't make any sense. First of all, the premise itself is flawed:

> Image compression algorithms are, generally, pretty good at finding patterns in images and squashing them down.

No, image compression algorithms are generally terrible at finding patterns. Lossless ones are basically just ZIP, while lossy ones don't really find patterns -- they remove certain high-frequency data and reduce color information. (There are certain patterns that PNG can efficiently detect, like repeating pixels from the line above, but those won't apply to text represented as an image.)

> Here's what it looks like when losslessly compressed to a PNG: It is slightly smaller than ZIP, and about 9 bytes larger than Brotli compression.

This is certainly entirely explained by the chosen (in this case, default) compression level of the ZIP file and PNG files. Lossless PNG generally uses the ZIP algorithm, or variants. Which is why the end result is basically identical.

The post makes sense and is good.

> Lossless ones are basically just ZIP,

Let's go down one more level of abstraction, it's crucial for understanding why it's byte-level patterns: Deflate, not ZIP. See http://www.libpng.org/pub/png/spec/1.2/PNG-Compression.html.

> image compression algorithms are generally terrible at finding patterns.

The core of deflate is pattern recognition, inter alia:

- Duplicate series of bytes (a pattern of repeated bytes in series).

- Huffman coding (a pattern of high frequency repeated bytes that could be replaced by another repeated byte string)

See https://en.wikipedia.org/wiki/Deflate

> There are certain patterns that PNG can efficiently detect, like repeating pixels from the line above

PNG uses deflate. General byte-level patterns. It does not do bespoke image-specific stuff.

> This is certainly entirely explained by the chosen (in this case, default) compression level of the ZIP file and PNG files.

Probably.

> PNG uses deflate. General byte-level patterns. It does not do bespoke image-specific stuff.

That's not quite the whole story. PNG does include simple filters to represent a line as a difference from the line above, and that may be what the original post is referring to. [1]

[1] https://en.wikipedia.org/wiki/PNG#Filtering

> PNG uses deflate. General byte-level patterns. It does not do bespoke image-specific stuff.

That's exactly what I meant, sorry if I wasn't clear. It's not "finding patterns in images". It's finding byte-level patterns, not visual (image) patterns -- which are inherently dimensional in a way that a stream of bytes isn't.

Which is why the post doesn't make any sense. Its entire premise is that there's something about lossless image compression which is different from zip/deflate and similar byte stream compression algorithms.

> Lossless ones are basically just ZIP

Lossless image compression is a vast field with myriad different requirements and solutions, and generalizing like that is just plain wrong. Most lossless image codecs for photographic or similar rendered content don't use a Lempel-Ziv stage like PNG does, they exploit the spatial and possibly spectral correlations to form predictions, be it with simple linear predictors or more complex approaches. However, for simple non-photographic images like cartoons or some such, the LZ-based stage can provide significantly better compression than the simpler prediction-based methods.

Even if the premise of the post is flawed, it's always nice to see people interested in data compression and trying to learn more about it, so I'm all for more "nonsense" posts if it helps draw attention to the field.

I could have been more specific -- I meant the currently common lossless formats of GIF, PNG and TIFF. In other words, I was just thinking about lossless image compression available to the average user.

It's true there are newcomers. WebP is slightly more advanced in lossless mode but still follows the same basic Lempel-Ziv principles, and I've never found a good explanation of how JPEG XL implements lossless mode but that hasn't gained widespread adoption yet anyways. HEIC is sometimes advertised as having a lossless mode but it doesn't really (its "lossless" is really just "maximum possible quality lossy" from everything I've seen). And obviously there are lots of niche/experimental codecs that work along different principles such as what you describe. And PNG does have a special mode for solid-color line art that references the previous row.

But the average person encoding a photographic image losslessly is probably using something based on Lempel-Ziv probably 99.99+% of the time -- not something built on top of spatial correlations.

For JPEG XL, refer to its format overview [1]. In short its lossless mode uses a combination of multiple techniques: the rANS coding with an alias table, LZ77, reversible color transforms, a general vector quantization that subsumes palettes, a modified Haar transform and a learnable meta-adaptive decision tree for context modelling.

One good thing about JPEG XL is that its lossy mode also largely uses the same tool, with a major addition of specialized quantization and context modelling for low- and high-frequenty components.

[1] https://github.com/libjxl/libjxl/blob/main/doc/format_overvi...

Now I want to see the same but with JPEG. The compression gain, but also the data loss in the uncompressed payload.

Do that with different levels of JPEG compression. Bonus: apply autocorrect after unpacking.

> It is slightly smaller than ZIP

How much smaller? The main theme in the article should be its comparison against conventional text compression. It seems like an important information.

I can't seem to reproduce the compression ratios. It always seems to come out significantly worse than ZIP.

- ascii: 142,496 bytes

- png: 78,904 (8-bit grayscale 1 x 142496)

- zip: 57,066

I used https://squoosh.app/ to losslessly compress the PNG further.

If you download the PNG from my post, you can check the filesize.

That did it.

Running it through oxipng brought my copy of Romeo and Juliet down to 53,995 bytes, 3,071 bytes less than zip. I did have to change it to a 142496 x 1 image though.

Neat.

Yeah it does look like noise but not quite... it does seem to have a texture of its own and I'd be curious what other books look like

wouldn't it be better to use something like tiktoken's tokens to represent words and then use the RGB values as indexes into that encoding? you'd get higher compression still

That does sound interesting. Write it up and Let us know.

Tiktoken uses the byte pair encoding, which is a form of compression. (LZ78 can be thought as an automatic form of this scheme.) So it's like compressing JPEG with ZIP, which may or may not work, but in either case is not a fair game.

This doesn't work if you use utf-8 instead of ascii.

Sure it does. UTF-8 is constructed such that characters from lower ASCII have the identical 1-byte encoding. You might be thinking of UTF-16 or UTF-32, or non-ASCII characters...

Reply on news.ycombinator.com