Knusperli: A deblocking JPEG decoder | Hacker News Reader

Knusperli: A deblocking JPEG decoder

github.com

121 pointsgsnedders44 comments

Ah, this is a great idea, and I can see how this could be extended beyond simple deblocking.

In essence, it computes the uncertainty from the JPEG quantization step (the source of most of JPEG’s compression) and uses that information to select a decoding that minimizes a particular error metric (here, discontinuities at block boundaries).

But it’s not limited to just that, in principle. It could optimize towards any decoded block within the quantization range given a suitable prior - and I imagine that this could be paired with e.g. a DNN to faithfully reconstruct an image from a JPEG with much better accuracy.

Additionally, I'd love to see a decoder add noise proportional to the uncertainty and surrounding structure as over smoothed images look unnatural as well. See activity masking demo https://people.xiph.org/~jm/daala/pvq_demo/

Pik can synthesize noise. https://github.com/google/pik

yeah, the deblocked image looks a little softer. Makes sense, the optimization objective encourages it. Sometimes the block boundary aligns with an image boundary. Agree that incorporating a DNN could be a cool solution.

The same should be applicable to MP3s. Also, the continuity measure could include derivatives as well.

DNNs are not magic, and not everything needs them, especially a fucking JPEG decoder, come on.

Could a DNN be thought of a continuous decompression dictionary?

Nice. This is a decode-side thing; it can be used on existing images.

But the site should show more images. Some with hard edges, like text and line art, need to be included.

Why would that be interesting? Jpeg is the wrong format for these types of images anyway, is it not?

That's precisely why it would be interesting. Getting cleaner decodes of jpeg-encoded line art and screenshots would be great.

Some images are only available as badly encoded jpegs.

Fonts suffer from ringing, their lines can actually look better if they are on the block boundaries.

Yes but there still isn't a universally supported "good format" so JPEG often gets used for text.

Because it might still help if someone else made the wrong choice in image format.

See also https://github.com/victorvde/jpeg2png , which tries to maximize smoothness everywhere instead of just at block boundaries.

Maybe these two approaches, jpeg2png and Knusperli, could be combined to get potentially superior end result.

Knusperli would run first, to reduce quantization discontinuities a bit.

Test image still has perceptible blocking. Description suggests it should be possible to reconstruct source values/smoothness staying in stored intervals, and yet there are still visible boundaries.

Lower right corner you can clearly trace 4 horizontal blocks, looks like algorithm tries hard to maintain encoded hard boundaries despite there being gradual color shift. Maybe it needs to look further than direct neighbor block for clues.

Unacceptable. Even though swiss-german doesn't have any spelling rules, I hereby declare a public vote to deport google from switzerland, because it's clearly spelled "Chnuschperli" in proper swiss-german.

Edit: Sorry, I forgot HN is allergic to jokes, obviously this is turning into reddit. I apologize.

So what does Knusperli even mean? It seems to mean “crunchy” or something from a Google search but Translate turns up nothing. Is the meaning at all related to the project, or is it just a cute name?

Yes, it means "crunchy thingy" basically. The name doesn't have any relation to the project. It's small pieces of fish fried in a dough mantle.

I was jokingly complaining about the spelling because the previous names "Brötli" and "Zöpfli" of similar projects were 'properly' spelled. They don't have anything to do with the name either. "Brötli" and "Zöpfli" are basically types of bread.

While Knusperli is a very german way of spelling a swiss german word. Though, it is what you would find in a supermarket.

Yes, too obscure of a joke, sorry :(

No need to 'joke' about deportation. That's what disgusted me.

> No need to 'joke' about deportation. That's what disgusted me.

Yeah, I should have considered global cultural influences more before posting. Sarcastically joking about deportation is probably a bit too swiss for an international audience to recognize it as such. Let me assure your though, this was an attempt by a swiss person desperately trying to be funny.

Obviously it backfired.

Knusperli could also refer to a type of bread.

https://wiewowasistgut.com/schnelle-fruehstueck-broetchen-mi... https://www.deal.ch/deals/brot-des-monats-coop-naturaplan-bi... http://fddb.info/db/de/lebensmittel/kleiner_baeckerei-kondit...

(Also Google's Zopfli and Brotli are already missing the ö / ü Umlauts, so maybe improper spelling is intentional.)

I think visually speaking, while the deblocked version is smoother overall, the blockiness added extra perception of noise/texture. The extra noise almost always increase the perception of sharpness and level of details, and you can see from the braids coming out of the hat that it does look sharper with the blocking version, although the wall is smoother. Would be great to include more sample images in the page.

x264 video codec developer Dark Shikari blogged about [1][2] this "fake sharpness" phenomenon. Early H.264 encoders produced excessively smooth, deblocked video, which made them look worse compared to MPEG-4 ASP (like Xvid), despite former being a newer, objectively better format. Wavelet-transform compressors like JPEG2000 had a similar problem vs. blocky JPEGs.

People's tastes are subjective, though, so it's good to have options.

[1] Archive.org: http://web.archive.org/web/20141124213159/http://x264dev.mul... [2] Archive.is: http://archive.is/K7jKA

I would assume that JPEG like most lossy compression algorithms is specifically defined in terms of the correct uncompressed output for a given compressed input. This means a "deblocking JPEG decoder" is effectively actually just a bad decoder that gives the wrong output (albeit for some outputs maybe you happen to prefer this).

The definitions are this way around because it's the only way to have an opportunity to improve encoding quality over time, if you start by defining how encoding must be done then you lock in bad ideas. Lame for example is far, far better at encoding MPEG audio layer III than the demo encoders shipped when the standard was written, and this is only possible because the standard says how to turn MP3 back into PCM data, not how to encode PCM data as MP3.

That is what I expected as well, but it turns out not to be the case. See also https://www.ece.cmu.edu/~ece548/hw/lab3/faq-do~1.htm and https://encode.ru/threads/2922-Knusperli-%E2%80%94-a-better-....

So could one define an "inverse" JPG standard that works with the current version but has the advantages that tialaramex spoke about?

I thought that by this point in time, we would have more images than Lena to showcase image libraries.

After all, no image compression format will ever beat LenPEG as long as the she is the standard

http://www.dangermouse.net/esoteric/lenpeg.html

Agreed, I believe the image of Lenna should be discouraged, as some people might find the image offending and unwelcoming to women (it originally appeared in a 1972 Playboy magazine issue).

https://en.wikipedia.org/wiki/Lenna

Considering Lena doesn't object and by all accounts is quite happy her image is used, there really is no reason not to use her image. If you real want you can use Fabio instead.

I don't think that's a good standard for a general rule.

Why did you use the pornographic picture for this work related task?

Well the people in the picture don't mind. In fact they're quite happy about the exposure.

Oh well, carry on then.

The Lena image is not pornographic. It is a picture of a Lena naked (well apart from the hat) and if it was a painting nobody would object.

Brace yourself, the social justice warriors have arrived!

Dissenting opinion: I think the blocky version looks better when not zoomed in. The sample on the left is less smoothed, and doesn't look as airbrushed. It looks better when zoomed in, but not when are regular zoom.

This is one of those nice "let's do this properly" projects google does. I'd put into the same category as their work on compression. Unsexy and "so last decade" at first glance, but decidedly useful and universally applicable, given the number of jpg files created since the dawn of time.

That being said, I'd really wish people would stop using lossy formats for everything. I can live with it for simple icons and such (although, why not svg?) but photos? come on! Of course, that would have to come with a nice connection for everyone.

Oh and this is definitely going to make for some head scratching for devs who assume image decoding is somehow deterministic. :D

>their work on compression

Like that time they stole Jarek Duda ANS and are still trying to patent it fraudulently claiming to have invented it 4 years after Jareks public domain publication?

https://encode.ru/threads/2648-Published-rANS-patent-by-Stor...

There is a lot of previous art on this sort of thing, like the ffmpeg spp filter: https://ffmpeg.org/ffmpeg-filters.html#spp-1

It's unclear to me if this method will perform better than those methods.

Seems a new library could use something besides Lenna...

C# wrapper? :D

I'm not sure if you are joking, but it does seem that more interoperability is usually a good thing.. (especially for something like a jpeg decoder - maybe it's less useful to have a toaster interface smoothly with some coffee beans)

Although I guess coffee roasters are a thing..

Assuming interoperability to be a good thing, then, coding something like a c# translation of this is a stimulating and useful intellectual exercise relative to like reading a book or solving a prescribed math problem. Assuming coding problems aren't like fossil fuel for jobs or something, and society hasn't yet reached the state where we can just neglect technology and say 'good enough,' given that (at least according to people I see while locomoting around town), there are still diseases, disabilities (physical and psychological), aging, puns, death, overpopulation, homelessness, variety of experience (I don't know what the correct word for this is, but something like 'non-crunchy-multitude-of-choice'), etc. all which may have solutions with sufficiently advanced technology (with the caveat that it may be like a pandora's box). (You could probably superficially post-justify a solution to these kind of problems in society by strapping an RC radio to everyones head and saying 'karma', but I would guess you can't do that without the being enacting this breaking large parts of the social contract, and essentially ridding life of substance, e.g. 'this thing happened to you because you ate a fish when you were a child, and that fish had a family, and so, relatively, your brain is like a fish-size (lets say chicken because birds are actually pretty smart in the animal hierarchy) to this AI we've constructed, and thus it's eating you, and so therefore, its karmically balanced (that's not really a solution is it, I mean it is, but it's like a military tit for tat solution (even if done subtly) rather than a diplomatic / medical type solution, which would provide more leeway for things like basic fight-or-flight response of humans / miscommunication / speed of actual human thought / etc., which doesn't really fit in the tit-for-tat model)' ). Wow I wrote freaking thesis here (and I didn't really put that long of thought into it, so judge as such, like git commit 1 of an more revised thought).

Curious how this affects visual quality indexes like SSIM.

(Also, come on, it’s 2018, we need to retire Lena)

Reply on news.ycombinator.com