Comment by JTxt | Hacker News Reader

Comment by JTxt | Hacker News Reader

I was skeptical of the bottom line of the eye chart image. characters are 3x3 pixels. The o should translate to a dot. there is no hole.

Their explanation for it:

"The lines of letters have been recovered quite well due to the existence of cross-scale patch recurrence in those image areas. However, the small digits on the left margin of the image could not be recovered, since their patches recurrence occurs only within the same (input) scale. Thus their resulting uniﬁed SR constraints reduce to the “classical” SR constraints (imposed on multiple patches within the input image). The resulting resolution of the digits is better than the bicubic interpolation, but suffers from the inherent limits of classical SR [3, 14]."

So it's guessing based on the larger characters. Neat.

Here's their last line:

DKN?MULJSPAVMRA?C?OVZO

Here's the actual:

DKNTWULJSPXVMRAHCFOYZG

(from what I can read.)

( http://1.bp.blogspot.com/-VgvutrSWaFk/T4VI-2tDH2I/AAAAAAAAAX... )

This could probably help ocr.

Other than this, I think Genuine Fractals and BenVista PhotoZoom give similar results.

The central pixel in the O is a lighter grey than its four nearest neighbours. That alone could be a reasonable cue for the 'hole', without cues from larger characters.

Sorry, could you clarify what you mean about the eye chart image? I'm amazed at the last line of the chart too. What are you suggesting they did to get that clarity?

Part of their algorithm finds areas of the image that are close matches but have different sizes, and uses the larger areas to create high-resolution replacements for the matching smaller areas. Of course this produces especially good-looking output for images like the eye chart (and also the final image on the page) that have many similar elements repeated at different scales.

Essentially it is looking at the large letters (or pieces of them) to guess how the small letters should look.

And as JTxt notes above, sometimes it chooses the wrong large letter. Even on the third-to-last line it couldn't get one of the letters correct. The output looks like "HKO" while the actual chart has "HKG" -- but since there is no larger "G" for the algorithm to use as an example, it ended up with a different but similar shape. This could probably be improved by the other SR techniques they mention that use libraries of sample images.

Ah, that's incredibly clever, but that must be incredibly expensive. For every pixel window you're trying to resolve, you have to look at (and generate smaller versions) of every other possible pixel windows, right?

The full PDF explaining their method is linked at the top of the page. http://www.wisdom.weizmann.ac.il/~vision/single_image_SR/fil...

[...] there is plenty of patch redundancy within a single image L. Let p be a pixel in L, and P be its surrounding patch (e.g., 5 × 5), then there exist multiple similar patches P1,...Pk in L (inevitably, at sub-pixel shifts). These patches can be treated as if taken from k different low-resolution images of the same high resolution “scene”, thus inducing k times more linear constraints (Eq. (1)) on the high-resolution intensities of pixels within the neighborhood of q ∈ H (see Fig. 3b). For increased numerical stability, each equation induced by a patch Pi is globally scaled by the degree of similarity of Pi to its source patch P. [...]

[...] Assuming sufficient neighbors are found, this process results in a determined set of linear equations on the unknown pixel values in H. Globally scale each equation by its reliability (determined by its patch similarity score), and solve the linear set of equations to obtain H.

Then they also add patches at different scale by making a few more versions of the image which have been scaled down by various amounts, and doing comparisons between the target patch and patches in those images too. If good matches are found, they can then use the original higher-resolution original patch that yielded the match when it was shrunk.

Right. Another example is the only W (on the bottom line) mistaken as a M.

The M first shows on the third from the bottom, but not very clear, but tries to reconstruct it; then it uses that M to replace the M the second from the bottom, finally it guesses that W (only known as a 4x3 pixel blur on the bottom line)is a M.

It's also interesting that the bottom M is skewed out a little on it's top right corner to match the fuzzy W shape.

So where are the '?' coming from? I think I have to call shenanigans / database, there aren't any in the image, or even any shapes like it. But then, I'm not sure what they're implying with this paragraph (this is the whole paragraph, my additions not italicized):

>Fig. 6 [the eye chart] compares our uniﬁed SR result against ground truth. In Fig. 7 [baby with hat] we compare our method to results from [11,13] and [9]. Note that our results are comparable, even though we do not use any external database of low-res/highres pairs of patches [11, 13], nor a parametric learned edge model [9]. Other examples with more comparisons to other methods can be found in the paper’s website.

>So where are the '?' coming from? I think I have to call shenanigans / database, there aren't any in the image, or even any shapes like it.

How about it's what they output when they are not sure?

That requires a database, even if it consists of only one symbol. And then why don't we see it in any other examples?

I added the ? when I could not read it. Normal SR can only do so much with a 3x3 pixel blur.

But their bottom line is mostly correct because their process also uses larger versions of the same pattern to replace the smaller ones.

So, there was not a much larger T, so it made a little fuzzball instead. So I wrote "?".

There are '?' characters in the paper's PDF as well, page 7.

I thought about the same thing (applications on OCR), the results on the eye chart are impressive. It's almost as if someone made a vector out of the enlarged image by hand.

It would be interesting to integrate with OCR on a deep level, preserving tile correspondence within the image to achieve coherent recognition.

Most real-world OCR, though, deals with single-scale fonts, and this reconstruction technique may have limited applicability.

Reply on news.ycombinator.com