Hacker News Reader

Comment by danbruc | Hacker News Reader

This also seems like something a [convolutional] neural net should be able to learn pretty well and generating training data could be fully automated. Throw everything at it - different fonts, different sizes, bold, italic, different colors, different anti-aliasing methods, sub pixel offsets, pixelation sizes and offsets, JPEG noise, rotations, ... - and maybe also give it some language model to better reason about plausible letter combinations. I would love to see, how good this could be.

View on HN

Already been done! https://github.com/HypoX64/DeepMosaics

Image models are pretty bad at text, generally, so I question how well it would work on text.

You are probably thinking of diffusion generation models. Models in general work absolutely fine with text. Just about the first convolution neural network examples you come across are basic character classification examples.

Great point!

It’s not, though. It seems to be a very shallow understanding of the subject (or just insufficient consideration?) stated confidently (as so often happens these days).

As a sibling comment to yours also points out, there are plenty of examples to the contrary - models taught to recognise text from indistinct images. We’ve all been contributing to this for years via Captchas, and now we’ve got to the stage where our iPhones apply the concept locally to text found in our photo libraries.

It was my understanding that anything trying to recover text is still hard to do well, but I could be wrong - I have not worked with image models in awhile. Classification and OCR is truly well solved though.

For blurred (instead of pixelated) text, take a look at https://arxiv.org/abs/2211.10103.

A neural net was my first guess as to how this would work but was pleasantly surprised at the actual technique used, which makes a lot of sense. The animated gif does a great job of explaining it on its own.

Should be very straight forward with cold diffusion.

Reply on news.ycombinator.com