Depixelizing Pixel Art | Hacker News Reader

Depixelizing Pixel Art

research.microsoft.com

168 pointswmwong33 comments

Earlier comments: http://news.ycombinator.com/item?id=2601347 http://news.ycombinator.com/item?id=2578706

Some of the previous discussion isn't great (squabbling about dupes, not being able to download the pdf). A common comment was "hasn't this been done before". Many existing tools output higher-res bitmaps, this tool outputs vector images. They did run comparisons to a variety of algorithms, including potrace, which also generates vectors.

When we get stories like this that get repeated multiple times on Hacker News (a few others spring to mind recently) it's a minor annoyance if you're an ardent HN reader but a significant indicator of what HN readers really find interesting on the whole since it's getting voted up each time by a different subset of users who hadn't seen it before.

Things involving novel presentations of algorithms and well presented 'undry' academic papers seem to do very well in this regard..

Even though it does a terrible job staying true to the source material, I'd bet playing an old game rendered through a vector magic filter would have an interesting abstract aspect to it.

In the discussion to one of the previous submissions, somebody posted a depixelated video of Super Mario World.

For reference: http://www.youtube.com/watch?v=o2Fd-4NzB0w

Here's a page where you can compare various algorithms, including this new one, over many classic pieces of pixel art:

http://research.microsoft.com/en-us/um/people/kopf/pixelart/...

Beautiful! I especially love the bits near the end, where they apply it to oldskool Windows 3.1 icons :D

I like the algorithm, but the hq4x seems to produce better fine shapes such as eyes and skeleton bones (albeit in bitmap form) - it would be interesting to see some sort of hybrid algorithm taking the best features of both to produce a vector output.

I feel like hq4x is probably a better choice for upscaling videogames, as it feels like it better maintains the original 'intent' of the art, which was, after all, designed with low-res in mind. The Microsoft algorithm, on the other hand, seems like it would be really useful for recovering graphics that have been downsampled algorithmically (as opposed to graphics originally drawn at low resolutions).

Except that the doom example in the paper (figure 11) shows that down-sampled images aren't recovered well. The posted work is designed for pixel art.

This is not exactly an apples to apples comparison. [hq4x](http://en.wikipedia.org/wiki/Hqx) is meant to magnify by a factor of 4. The comparison images use it for 16x magnification. Would be interesting to run this algorithm after magnifying it with hqx.

I wonder how difficult is it to generalize the algorithm to 3d voxels, and apply it on Minecraft.

Pixels map quite nicely to voxels, assuming you're looking for a 3D extrusion of 2D pixel art.

The Minecraft voxels are decorated by textures that are designed to fit the cube form. A devoxalization is going to look like crap.

I think he means disregarding all that and looking at the raw voxels then applying a modified version of this algorithm, similar to marching cubes.

I know. But when displaying the Minecraft landscape afterwards you either have to leave the disregarded textures out, which looks like crap, or apply them stretched, which looks like total crap, or apply a smoothed gradient, which also looks like crap compared to the original landscape. :-)

Given that he suggested Minecraft terrains, I opined on how Minecraft would look with such an algorithm applied.

Here's a paper experimenting with some ways of doing that: http://users.soe.ucsc.edu/~amsmith/papers/voxel-pcg.pdf

This is an interesting paper. Though I only read the abstract, it seems like real-time performance could be achieved by utilizing GPUs for B-spline calculations. Would be awesome for video games if real-time performance can be achieved!

Needing a high-end gaming machine to run Super Mario World or Space Invader at acceptable frame rates would be fun, for a change (-:

Can you run the algorithm on graphic resource first to produce vector graphics and then do the emulator stuff? Seems to me feasible and you can run two copies of the game (one with vector graphics one without in the background to ensure the correctness of emulator).

If one were to port a game, then before displaying the buffer, one can generate vector graphics before sending it to the frame buffer. While you can potentially display both the buffers, I am sure the vectorization phase will be computationally intensive (thus low fps).

Sorry about the anti-climatic question, but I'm genuinely curious: What's the main application of this research, other than prettifying old video games?

At a guess, it might make CSI-style "zoom in, enhance" manipulations of images and videos more plausible.

This algorithm is specifically designed for pixel art. One of the examples in the paper shows that it doesn't do very well on anti-aliased graphics. Therefore it probably wouldn't be of much use on blurry pictures or videos.

The application is that projects like this are fun to do. Great work on their part.

http://en.wikipedia.org/wiki/Geographic_information_system#D...

Yeah, I'm just learning about this stuff, so I cannot offer a detailed explanation whatever :/

Previous discussion: http://news.ycombinator.com/item?id=2601347

Wouldn't this be much harder if the background had more than one color as well? If that is the case, this new technique may not be applicable to emulators

Well, in the paper they do describe some heuristics for separating figure and ground.

Additionally, a lot of old videogame machines have special built-in routines to place sprites over a background. So if you'd apply the vectorizing algorithm separately for the sprite drawing routine--by not just processing the entire screen after it's rendered, but doing the elements separately--you'd at least not run into the problem that lines from foreground sprites meld with background details.

What is the @article block at the bottom of the page?

A BibTeX entry: http://en.wikipedia.org/wiki/BibTeX

great job, I'd like to see it applied on http://draw.heroku.com drawings

Love it.

Reply on news.ycombinator.com