Pixels and voxels, the long answer
medium.com
medium.com
But I think it's worth considering that:
- "Pixel" is an overloaded term. "Pixel" is a little square, sometimes, because common law - it's being used that way.
- This is an article about art, aimed at artists, and if an artist intends to create a blocky image in two or three dimensions, then what they're doing is "correct" regardless of the terminology they use.
- The pictures are really pretty, IMO. I realize some people don't like the block look, but I'm not alone when I say I love it and it's inspiring. The explanations in the article were nicely done.
- The article does cover sampling. Just not filtering.
- When Alvy Ray wrote his paper, the world was made of blurry CRTs. Now we have really crisp digital LCDs everywhere, and more now than then, pixels on the screen are little squares.
- "pixels" even in the days of CRTs have always meant little squares to a lot of people. Video game art benefitted from a little blur, but tons of artists and programmers have always thought of pixels as squares, and for the most part it's a working analogy.
This is why it's okay to call a pixel a little square, and overly rigid to claim it's not one. (And I think Alvy Ray was being funny and making an important point about math and graphics, for a siggraph audience, not pedantic about what words artists are using.)
Yes, so technically we should probably call the modern rendering technique of pixels and voxels block-based rendering. I would say though it's fair game to call the underlying data "pixels" and "voxels" - in most retro-style games these assets are indeed stored internally at that level of abstraction. The issue now becomes how to render that data on the screen, and blocks are still a way to make this data look good.
While the block is not a pixel or voxel in the strictest sense, I do think it's a reasonable linguistic shortcut to use that word in this context, and it's not unprecedented to use the description of an abstract thing interchangeably for its more concrete representation.
In my opinion, there is nothing to freak out about, and I don't see why this article specifically prompts such a needlessly absolutist response.
As others have said, 8-bit art was made in a context where the "pixels" were viewed through an analogue filter, that being an NTSC display. My understanding is that to a first approximation, such a display (i.e. the electron gun + shadow/aperture mask) acts as a spatial lowpass filter with a resolution of 640x480 pixels, with maybe a 1-pixel-wide horizontal "smearing" effect due to the horizontal movement of the gun. Of course various console + display combinations deviate wildly from this ideal, but the lowpass filtering is the basis for understanding how 8-bit art was expected to be viewed.
It's also worth mentioning that for most of us today, graphics tastes have changed quite a lot, so emulating the original blur and smear of an NTSC display is perceived as far less of a generally acceptable aesthetic solution than rendering rectangular blocks is. Of course, opinions differ, but you can totally vote with your wallet on that.
You're talking about how art is designed with an expectation about the manner in which it is later consumed. To the degree this is even a realistic expectation, you would have to make the same concession to today's pixel artists who are creating works loosely based on an 8bit heritage, but which are often clearly and intentionally distanced from that heritage. You might personally object to viewing original games from the 8bit era on modern systems at all on the grounds of authenticity and distortion of artistic intent, but that's not an objection you can raise against modern pixel artists and their working premises.
Overall, I object to this notion that, say, the average 80s CRT pixel representation is somehow the canonical rendering of the ur-pixel and that we should strive to somehow emulate that. I think we should instead do whatever works for the modern context. You might counter that blocks don't meet this goal for your tastes, or that we should stop using recognizably blocky pixels altogether, but again that's a personal preference.
The actual argument above was not about personal preference at all, it was about what can rightfully be referred to as a pixel and what can't (and I do maintain that in this context linguistic pedantry does not yield any meaningful benefits).
You'd have to imagine it's closer to the original artist's end goals though.
The guy who drew Mario probably didn't intend for us to see a blocky representation of an Italian plumber, they just wanted us to see a plumber.
OTOH, Seurat was consciously making a point that you can represent a person with nothing but colored dots:
https://en.wikipedia.org/wiki/Pointillism#/media/File:Seurat...
> people referred to those little atomic picture cells rendered on the display as "_pixels_"
'These aren't the pixels you're looking for'. As the article points out, the pixels on a CRT do not have a 1-to-1 correspondence with the pixels in RAM. If you understand a pixel as a sample, then the mental picture becomes clearer:
RAM pixels -> video card DAC converts to continuous signal -> CRT resamples again into monitor pixels -> your eye/brain reconverts into a continuous image
Expanding on that answer, I think you're making a case for DRM here. Mario wasn't supposed to be not-blurry so we shouldn't be allowed to see him like that. If that's your opinion, I disagree, but fair enough. It's an entirely different thing though to assert that modern artists should not be making new pixel/block art, because it seems to me that their intent has value, too.
So the question becomes whether you object to that art style, or just the commonly used name for that style. Personally, I think you're going to have a hard time with either one, especially given the fact that the common usage of the word "pixel" has always been blurry, and that you could simply avoid modern "pixel" games at no personal cost and still allow other people to make and enjoy them.
I think there is room to say both "this was the authorial/creatorial intent behind Mario's sprite, which should be acknowledged as being distinguished by being the intent" and to say "I aesthetically appreciate this deviation from the creator's intent, more so than the original intent."
The deviation in interpretation from the original intent is, I think, a new (derivative) creation, in a sense. (Though it may be an accident, and might not have an intent to create behind it.)
But we should appreciate modern pixel artwork as a cool retro-themed anachronism, not an actual representation of what displays rendered in the past. The idea that a pixel is only a 2D square gets in the way of that -- ideally, people could understand that it refers to both a point sample as well as a box-filtered rendering of that sample, and those two senses imply different ways to view a set of pixels.
That is exactly – and only – my concern. The remainder of your post you are arguing against a strawman.
I don't care what modern artists do. You say "let go" like I'm taking a stance on nouveau pixel art; I'm not.
"NTSC...is the analog television system that was used in most of the Americas (except Brazil, Argentina, Paraguay, Uruguay, and French Guiana); Burma; South Korea; Taiwan; Japan; the Philippines; and some Pacific island nations and territories"
https://en.wikipedia.org/wiki/NTSC#NTSC-J
However, Japan uses both 60hz and 50hz power, so I wonder what that does to frame rate.
Graphics assets are no more a picture than a recording of an artist's brush strokes are. They can only be interpreted in the context of a medium, be it CRT scanlines vs. LCD squares or oil-on-canvas vs. ink-on-papyrus.
I just refer to them as "boxels."
I wonder though, would a lot of 'pixel art' actually look better with a different filter instead?
I actually prefer the crisp pixels
of emulated 8 and 16 bit games,
I have no desire to emulate
the crappy monitors we used to use
to play said games. - lordofultima
I actually enjoy gigantic, crystal clear pixels.
- rateoforange
All this reminds me of tge performance practice of old music.
Well yeah, but it's a feature nonetheless. People aren't doing anything wrong by preferring sharp pixels. It's an aesthetic all to itself.
Pixel art and voxel art are intending to evoke feelings of nostalgia but with a modern flare. It's important to the work that the art is visibly blocky. It's a completely artificial limitation, applied completely on purpose for the sake of the art.
What does it mean to start filtering today's pixel/voxel art? What would that achieve for the art? If *xel artists wanted to make it look "better" by smoothing out the blockiness, then it opens the door to asking, "why use low resolution blocks at all, if what you want is more realism?".
Personally, I'd guess realism, e.g., better filters or higher resolution or anything else that tries to smooth out the blocks, is antithetical to the goal, which is in part to specifically emphasize the blocks.
But once you're in 3 dimensions, triangles do a much better job of approximating arbitrary shapes (also, hardware deals with triangles), plus the voxel dimensions have no relationship to your display. You're essentially picking a very clunky filter that obscures what you're trying to represent, out of some misguided analogy to 2D pixels.
Just to be clear, cubic voxels can be great as a stylistic choice, but are inferior to triangles as a general way to describe 3D scenes.
Rather, it is only an infinitesimally small point (an "impulse" or Dirac delta) which, when grid-aligned and ideally filtered, can rasterize as a single pixel.
The only reason pixel-sized square survive supersampling intact is because supersampling uses an imperfect low-pass filter (block-based averaging). Here is a good example showing the shortcoming of straight supersampling: https://people.cs.clemson.edu/~tadavis/cs809/aa/aliasing5.pn...
You NEED to apply a true low-pass filter before rasterizing to completely eliminate Moiré patterns. Whether a round of supersampling occurs before this filtering is irrelevant. And pixel-sized squares don't survive low-pass filters intact, see https://en.wikipedia.org/wiki/Gibbs_phenomenon#Signal_proces...
In the case of your second link, the problem is inappropriately applying a low-pass filter. This makes the edge of the square non-sharp, and adds distortion effects multiple pixels away.
This portion of signal theory only applies to a world made entirely out of frequencies. This causes problems when you try to apply it to realistic shapes. It's great for audio, not so great for rendering. The use of point samples does not automatically imply you should be using it.
If you don't like supersampling, that's fine. But you need to pick an antialiasing method that's compatible with the concept of 'edges'.
Edit:
You added "You NEED to apply a true low-pass filter before rasterizing to completely eliminate Moiré patterns."
I don't think that's right. It should look fine if you use brute force to calculate how much of every texel on every polygon shows up in every pixel. And it should look fine with much more efficient approximations of that. At no point should it be necessary to use filters that can cause ring effects multiple pixels away.
That's not precisely true: discrete blocks or edges can be approximated arbitrarily closely in Fourier frequency space -- you just need to admit higher-frequency components.
'Pixel-sized square' is properly an oxymoron if you view pixels purely as samples -- a 'true' square can only be represented by an infinitely dense set of pixel samples.
But this is a feature, not a bug, because neither computer monitors, nor the human eye, can render nor see a 'true' square either, only successively better approximations to them.
You can approximate those shapes, but that's an approximation. It's not impossible to do the math that way, but there are stumbling blocks to dodge. Like weird artifacts when you low-pass.
> 'Pixel-sized square' is properly an oxymoron if you view pixels purely as samples -- a 'true' square can only be represented by an infinitely dense set of pixel samples.
It's a square equal in size to the square-grid pixel spacing. It's not wrong, it's just being non-pedantic.
> But this is a feature, not a bug, because neither computer monitors, nor the human eye, can render nor see a 'true' square either, only successively better approximations to them.
A monitor does not quite deliver perfect squares, but what it displays is a lot closer to a square than to this shape: https://upload.wikimedia.org/wikipedia/commons/f/f8/Gibbs_ph... So calculating it more like that is not a feature.
Monitors don't display colored squares, period:
https://upload.wikimedia.org/wikipedia/commons/4/4d/Pixel_ge...
Our cone cells also don't see colored squares, and in fact each color's distribution is random:
https://askabiologist.asu.edu/sites/default/files/resources/...
In fact, CRTs phosphors didn't even correspond to logical computer pixels. If your CRT resolution was lower than the maximum phosphor grid resolution of the CRT mask, then a single pixel would indeed spread across more than one tri-color phosphor group.
Signal theory points reminds us that 'square pixels' are just an arbitrary shortcut. We could equally well describe them as hexagons, ovals, or rectangles, or make the grid positions random, and monitors would work just as well. That's why when you blow up a and render it as a giant square with exact edges, it's a very misleading and arbitrary choice.
The basic problem with being petty is that, while you're right on one level, you're wrong on the global scale. In this case, that pixel are square or not is irrelevant. Do you look at your monitor straight on? Do you sit side-by-side with a friend playing a game? In that case, squares become trapezoid from your POV anyway. I'm going to bet you don't mind it, because our brain is very good are pretending perspective distortions don't exist.
The same apply in many domains: color correction, for example. I use to write in printing and designers would be hung up on proof to verify colors. No matters that reader would read teh final product under varying lights anyway, making precisely 'correct' colors moot.
In both cases, what is important is consistency. For prints, that all runs be the same. For monitors, well, as long as you look at a single monitor at a time, you won't notice distortion.
(In a somewhat related anecdote, the piano my parents own is and always has been out of tune. I learned on it and for me, all other pianos tones were unsufferable. What is the right pitch is mostly learned, even though musicians would like us to believe there is but a single harmonious scale.)
In modern pixel/voxel art, however, the whole point is that they are in fact little squares and cubes. Otherwise you don't get the aesthetic this style is aiming for.
It should be noted for historical accuracy, that the origins of pixel art were displayed on relatively blurry CRT screens, and your pixels didn't actually look like little squares either, more like glowy blobs. That's a different point than the cited paper is making, however, which is about the sampling theorem where pixels and voxels are basically data points (that can be interpolated in various ways).
On the other hand, when I played 16 colour 320x200 games like Commander Keen and Duke Nukem (2D) in the 90s on a VGA CRT monitor, the pixels definitely looked like little squares, not blobs.
Edit: maybe I should mention the oldschool ANSI art as well (BBS/demoscene), which were made with various coloured ASCII-blocks, and there the "pixels" were in fact quite obviously little blocks (and sometimes other characters but that's more ASCII-art than ANSI).
There is a fair amount of graphics whose good looks are based around the fact that pixels are little squares. Like orthogonal lines.
What you want is a non-linear filter - it's pretty obvious that you can invent good upscaling filters (like Waifu2x / NNEDI3) that aren't based on FIR theory and can't be used to downscale.
> Or, you could say, the smaller I want filter regions to be.
And that's the opposite of you get from linear filters. There's a resampling filter called sinc that's "perfect" - it reproduces every frequency in the input signal - but it looks terrible for images because it tries to reproduce pixels based on every other pixel in the image. So you get ringing all over the picture instead of sharpness.
>What you want is a non-linear filter - it's pretty obvious that you can invent good upscaling filters (like Waifu2x / NNEDI3) that aren't based on FIR theory and can't be used to downscale.
When I'm looking at really tiny details even those can cause problems, and I just want pixel blobs.
>And that's the opposite of you get from linear filters.
Huh? I'm talking about the footprint of the filter, like the diagrams in the pdf. The box filter and Figure 1 filter sample one pixel, so they have the smallest region, and sinc is the largest possible. What's opposite?
But your intuition isn't wrong. Nearest neighbor can be a bit more "crisp" feeling on certain images, especially natural photos. A lot of photos have enough blurriness in the 1-3 pixels range that nearest neighbor sampling is a good choice if you're just resizing an image by a factor of 2.
But to see why nearest neighbor isn't the best in general, imagine down-sampling a checkboard and what happens as individual checkers become smaller than a pixel. You'll get moire patterns if you don't filter. You'd also expect pixels to turn grey as multiple checkers land there, and maybe not stick to only black or white.
Interestingly, some "sharpening" and up-sampling algorithms do the opposite - they subtract the value of surrounding pixels, rather than add. Those are assuming that some averaging already took place, and trying to un-average, if you will, to restore the original point sample.
The long answer is... really long.
Until that night I hadn't even imagined that holograms would ever become a reality, and I had also never heard of voxels. It really made an impression on me. I hope they succeed in making their machine viable for consumers because I could tell how passionate they are about it, and how hard they've been working.
Essentially, the object to display is sliced up like you do for 3D printing, and a screen of some sort is moved to the lowest Z position, and the bottom slice is projected up onto it.
Then, the screen increments it's Z position, and the next slice is displayed, repeating until the entire object has been displayed.
Thanks to persistence of vision, it appears as though the image is 3D and floating.
Also, it's not necessary to take the displayed path, i.e., triangulate a voxel-based representation then use vector-oriented rendering methods to get the displayed image. There have been lots of work rendering a 2d display by ray tracing through voxel space or even directly splatting voxels into display space.
A lot of research has been directed into efficient representations for large collections of voxels. One really cool sparse volume system, OpenVDB, was open sourced by DreamWorks: http://www.openvdb.org/. Nvidia has papers on sparse voxel octrees that work very well on GPUs: https://research.nvidia.com/publication/efficient-sparse-vox...
Odd that people suggest making it "better" with polygons, as if you're stuck with voxels, as if you didn't choose it consciously, as if you didn't already understand the alternatives. :) Ignore it and keep making awesome games.
For example, voxels can't represent non-right-angle surfaces without adding jagginess, while a triangle can.
(Naturally, one can model a system as a grid but render it as arbitrary shapes, or vice-versa.)
I wonder how far someone could get with that artstyle in general, I haven't seen it that often. There was that RTS game that used it, forgot the name..
Screenshot: http://cdn.dbolical.com/videos/games/1/19/18851/planetary-an...
Do you mean 8 bit armies? [0]
That one looks pretty cute, too, though. :)
Looks pretty different from what I remember, haha.
Which one?
They can also contain meta information, like color and transparency.
To represent pixels or voxels on a screen you need to to render and interpolate them. Because, if you have an image of 4 pixels that you would like to show on a screen with 1080 leds, you need to interpolate the meta data between those pixels.
The best thing about voxels is what you can do with fluid simulations.
In Maya you could create a cube made up of voxels, each voxel had a fuel, temperature etc variables.
You could start a fire from a single voxel, and watch is spread ! Changing the values could make the fire behave differently.
I wish front-end javascript was that cool :( fire simulation on a computer is so much more fun.
FDS and SmokeView, are the defacto fire simulator tools used by fire protection researchers/engineers around the world. Not only can you manipulate the fuel sources, you can manipulate the environments and rooms and watch the fluid and thermodynamics dynamics happen. Radiation, convection, conduction, flow gradiants, for you to see and play around with.
http://www.thunderheadeng.com/pyrosim/
Is a nice gui layer for FDS allowing you to import CAD files for your simulations and gives you some visual control rather than programming in Fortan for FDS.
The control scheme was a weird mix of simulator and shooter.
In Minecraft, the drawing primitives are triangles, the map is made of cubic tiles, and the detail is in the texels.
I've always disliked the use of the word voxel when talking about Minecraft, because I think it applies too loosely. They are mostly map tiles.
I learned SpriteKit and Swift for fun and started making games with my kids. So rewarding and a nice break away from real world programming. But not something I would want to do for a living.