Exploring Euclideon's Unlimited Detail Engine
gameinformer.com
gameinformer.com
As others have pointed out, voxel-based games have been around for a long time; a recent example is the whimsical "3D Dot Game Hero" for PS3, in which they use the low-res nature of the voxel world as a fun design element.
Voxel-based approaches have huge advantages ("infinite" detail, background details that are deformable at the pixel level, simpler simulation of particle-based phenomena like flowing water, etc.) but they'll only win once computing power reaches an important crossover point. That point is where rendering an organic world a voxel at a time looks better than rendering zillions of polygons to approximate an organic world. Furthermore, much of the effort that's gone into visually simulating real-world phenomena (read the last 30 years of Siggraph conference proceedings) will mostly have to be reapplied to voxel rendering. Simply put: lighting, caustics, organic elements like human faces and hair, etc. will have to be "figured out all over again" for the new era of voxel engines. It will therefore likely take a while for voxel approaches to produce results that look as good, even once the crossover point of level of detail is reached.
I don't mean to take anything away from the hard and impressive coding work this team has done, but if they had more academic background, they'd know that much of what they've "pioneered" has been studied in tremendous detail for two decades. Hanan Samet's treatise on the subject tells you absolutely everything you need to know, and more: (http://www.amazon.com/Foundations-Multidimensional-Structure...) and even goes into detail about the application of these spatial data structures to other areas like machine learning. Ultimately, Samet's book is all about the "curse of dimensionality" and how (and how much) data structures can help address it.
In the late 90s at Naughty Dog, I used Samet's ideas (octrees in particular) for collision detection in the Crash Bandicoot games. In those games, the world was visually rendered with polygons, but physically modeled -- for collision detection purposes, at least -- with an octree. The nice thing about octrees is that they are very simple to work with and self-calibrate their resolution dynamically, making them very space-efficient. Intuitively, a big region of empty air tends to be represented by a handful of huge cubes, while the individual fronds of a fern get coated with dozens or hundreds of tiny cubes, because there's more surface detail to account for in the latter example.
I think the crossover point I mentioned earlier will come when GPUs become general-purpose enough to allow massively parallel voxel rendering implementations. That's what surprised me most about this article: they crow that it's a CPU-only technology... why? GPUs excel at tasks involving vast amounts of relatively simple parallel computation.
Prior to the crossover point, we'll see a bunch of cool games that use voxel rendering primarily for gameplay reasons. These games will look chunky compared to their polygonal peers, but will offer unique experiences. Minecraft is a good example. (I'm assuming it's voxel-based, but don't really know.)
Doesn't take away the fact that both games look pretty damn nifty though.
> Voxatron is based on a virtual 128x128x64 display. It's a buffer of 3d video memory that is rendered out to the screen at the end of each frame, much as an old-school 2d display is. You can POKE bytes into the virtual memory, and they come out as voxels. I don't compromise on this -- even the menus are drawn into the voxel display. Hopefully one day I can get hold of a real physical 128x128x64 display and play Voxatron on it with almost no modification.
> The renderer is written in software (C + SDL). Each frame, scan through the virtual video memory back to front and look for voxels that have empty neighbours. If they are exposed, I transform the corners of a cube intro screen space and scan-render the polygon. Shadows are done with a traditional shadow-map but sampled with a filter to get the soft shadows.
I guess I was thinking that the raycasting approach I've seen used in some of the sparse voxel octree stuff I read wouldn't make sense when you want the blockiness of voxels to actually show up very clearly, plus that it looked like you could achieve the same look using polygon based cubes. Guess I should've done a bit more research.
http://www.youtube.com/watch?v=Ku-ICQvQJGI&sns=em
http://en.wikipedia.org/wiki/Comanche_series
No story I've seen on this engine seems to mention that series.
Today voxel-based renderers must allow correctly projected 6-DOF orientation of the viewport, and not just render terrain but arbitrary objects, and integrate together with various lightning and shader effects to stand any chance outside the neo-retro or abstract category.
one thing is that only the landscape is rendered as "voxels" and it's mostly a textured height map, so it can't contain multi-level structures such as bridges, etc. it's best suited for mountain ranges and beaches and things like that.
just the term "voxels" is quite generic already, and doesn't really describe one algorithm over another, except that you're not using polygons.
Indeed. This is certainly possible with modern GPUs, and has been for a while. Voxels are pretty well-suited to GPU rendering. See the research I linked to in another reply in this thread.
I didn't get around to reading their entire article, but if they call it a CPU-only technique they are behind the curve instead of ahead.
Edit: You didn't clearly get what is claimed here. 'Infinite detail' is just a marketing speech, the claim is that the algorithm can render up to screen resolution any data set whose size is limited only by memory. I.e. rendering speed isn't bounded by the size of the data set. The method has nothing to do with compression, and indeed the data set will be huge if everything is 'infinitely' detailed.
Looking at the article, this is clearly using method 3. Lets look at this "new" hierarchical composition. Hiearchical composition is when any given space is composed of smaller objects, and those smaller objects may also be composed of smaller objects.
When you do this in 2D games from 1980, we use the technical term "tiling the crap out of everything". You have a bunch of obviously repetitive pieces. If each tile is 16x16, and there are 256x256 tiles on a map, then one could say that there are 16 million pixels in the world, on a computer that had only 1Mb of memory.
So that right there is your clue that this guy is a fraud. 42 trillion voxels? Are they 42 trillion unique voxels, or are they tiled? [1] Right. (a "voxel" is the 3d equivalent of a 2D pixel).
The thing about tiling is it doesn't just apply to pixels or voxels. Tiling is just a spatial subdivision algorithm that splits space up in to 2d or 3d grids. You can store a color in each grid cell, and then its a voxel. But you can just as easily store the head of a linked list of polygons. And you can also store a pointer to another tile array, and that tile array can then store voxels, or polygons, or more tile arrays. When you do this, its a hierarchical data structure.
Do games/graphics programmers know all about this? Yes. Do we use hierarchical composition? Yes! All the time! Do we use axis-aligned hierarchies? Yes, e.g. octrees? Do we use regular grid composition all the way down? Fuck no. Why not? Because splitting up space along predetermined, regular, axis-aligned divisions is fucking awful for modeling interesting 3D worlds [3]. What you get is a brick world: [2]. Such rigid, regular, axis aligned hierarchy fits the real world poorly.
[1] http://media1.gameinformer.com/imagefeed/featured/gameinform... [2] http://media1.gameinformer.com/imagefeed/featured/gameinform... [3] Ok, minecraft being the exception =)
Interestingly this technology or even this idea isn't anything new. A nice way of summarising it is 3d fractals, sure it's infinite and richly detailed. But it's the same thing over and over again.
Minecraft clearly demonstrates the comparitive heaviness of using voxel arrays to represent a world in contrast to a polygon mesh. This point is trivial, and is not unique to Minecraft.
Additionally I'm finding a large number of comments that begin with "No it doesn't", then either stating an identical argument to the parent, or not attempting to refute the parent comment at all. If you're doing this for "points" then shame on you.
A simple example of the difference this affords is that large dynamic changes(e.g. a tornado 'mod') are the result of simply moving voxels around and letting the world renderer do it's thing. Where a polygon based system has to computationally alter the various surfaces to represent the changes made, which can be quite a bit more 'work' for programming the various scenarios.
Ironically, I see pretty effective tornado effects in movies all the time, using polygons. An audience will happily believe that a tornado is composed of solid pieces of geometry (such as a car, or a cow, or individual roofing tiles) because they've seen it on TV. Tornadoes do not deconstitute matter as far as I'm aware. Thus hierarchical polygon models are ideal for tornadoes.
Also I feel like your example is confused, I'm trying to be polite, but I think you should do some research about what this is before writing a counterpoint like you have. (what you've written isn't really meaningful.)
Let's take an example of a whirl wind in a desert or dusty plain, in a polygon mesh you're not seeing individual sand grains being lifted from the polygon surface then flying around and landing somewhere else forming a pile, as you have suggested. In a voxel simulation you would however see just that.
Now ignoring post production, what you're seeing is a well textured, but (crucially) introduced, particle effect and maybe some surface deformation for the sandy hill that is being destroyed. (This isn't how the effect is done in film by the way.)
Translating (i.e moving) cows and other objects is actually more on par with how voxels work. I.e. moving an object through space and placing it somewhere else. So you were half right in your thinking that way.
To finalise my point, the tornado "mod" example that I cited wasn't me discounting polygons as you've interpreted, it was referencing this video specifically(i.e showing how voxels can trivialise the programming of advanced motion effects): http://www.youtube.com/watch?v=fSEwU5IqZ4A
Because they don't know where raytracing research has gotten to with GPGPUs. Indeed if a few years ago you had told me that GPGPUs would be handling the insanely branching search of raytracing I'd have told you you were mad. Now we have iray. Bottom line: yes, its a staggeringly inefficient algorithm compared to a CPU, but I've got 480 cores!!!
I'll place money that this technique is just raytracing with a nice, hierarchical voxel tree. Plenty of work being/been done on combining such spatial subdivisions with SIMD. Usually the search ends on a polygon with a collision test. The idea here, which I havent come across (but I've not read every paper) is to store the subdivision down to sub-poly level so that you can skip the collision test. I hope they still share surface information between voxels, but maybe not. Essentially there are plenty of researchers who could implement their algorithm right now by adding "return true" for their poly hit test and increasing the subdivision threshold.
As Carmack said 114 days ago, "Nice idea. Be a few years till its viable."
But I'd be surprised if its this team that delivers the goods. They seem woefully uninterested in the wealth of research in the area.
Hacking reflexion by duplicating the scene is a good way to start, because no one can say whether it's a good or a bad solution to a non-trivial problem - it gets the job done for now - but can you have multiple coexisting versions of the world? like a vertical mirror, a horizontal body of water and an underwater section?
To Dell: just stop handwaving the questions already, we get it that you ignored the state of the art and built your own thing, just tell us what it can't do yet, show us your current progress and you'll be met with much less skepticism, and even help, even if you keep your trade secrets, at least show us your specs! You won't be struck down by lightning if you talk - no one ever died from reinventing/copying the wheel and making it so good that the wheel can fly.
I'll go out on a limb here, and suggest it can't do real-time animation, texturing, lighting, shadows, collisions, etc. All the maps / screenshots shown seem to be static, procedurally generated and not artist-created.
Dell keeps throwing around this 'Unlimited Detail' marketing phrase which is very offputting. Until it's seen in action in an actual game it can be nothing but snake oil.
This kind of marketing is one I can't quite understand. I mean, what is he after? Buzz? Funding? What's his endgame? Why & how is he paying people to work on it? Is his marketing strategy to troll the profession & the audience, and he really has something big up his sleeve?
Why is he using the Vaporware approach?
Maybe there could be some middle ground like in the old voxel days, where you would have a static (unlimited detail) background world and some traditional, polygon-based actors in the foreground. Looking at any modern game, just about everything on the screen is constantly moving, so color me sceptical even on that idea.
Also, I would like to know how they do lighting. It looks like they might use precomputed highlights and shadows. Needless to say that this would not be of much use for dynamic lighting.
.
"Dell shows me an as yet unreleased demo – not real-time in this case – that suggests Euclideon has already made some positive inroads in ensuring Unlimited Detail is compatible with existing middleware. The short video shows traditional polygon objects happily coexisting and interacting in the same space as a small Unlimited Detail environment"
Remember: you can hack stuff in games until it looks right. It doesn't actually have to work perfectly from a theoretical standpoint; it just has to work practically without too much additional tuning labor.
I'm not saying voxel worlds solve every such problem; just that there are likely to be as many things that are easier w.r.t. animating elements as there are things that will be harder.
[Edit: maybe I'm not explaining this well, but I guess what I'm saying is that I don't see any reason why any voxels in a voxel world have to be static. The data structures don't force that; in contrast, in a voxel world with a clever data spatial structure, every single voxel can be a dynamic, particulate object, subject to computing power. As I argue above, though, this is not practical until computing power improves enough. Perhaps this is your fundamental point, in which case we're in violent agreement.]
Searching such a large problem space for 1 to 2 million results 25x a second is amazingly impressive... This is what i am most curious about at the moment... Also how he is storing the full voxel point data for any given world that needs to be searchd in real time. Replicated data or not (i.e. similar to GIF color data deduplication) you still have location data for every voxel position or offsets or something that still results in a hellacious amount of data that needs to be searched efficiently.
Havent seen details from Dell or others on either of these aspects that I feel are cornerstones to the engine.
I'm not at all an expert in this area, can someone who is explain what the downsides of voxel graphics are? There must be some serious problems with it, because the technology is well known.
Is there something new that makes this particular engine stand out above the previous engines?
EDIT: If you are spekaing about preprocessing data - yes - you can rotate as much as you want, the only problem will be that object instanced 1000 times with different orientation/scale will take 1000 times more memory than object instanced 1000 times with the same orientation/scale. With polygons there is no difference.
But yeah, the amount of data you'd have to work on would be much greater, but the way I interpret all of this is that he's found algos to do that very fast, regardless of whether it's a lot of data.
With voxels, you'd have to recompute all the voxels of the object on a 3d grid. Also there are issues with ragged boundaries, which can be prevented by using antialiasing, but which is probably trickier (=more computationally intensive) than in 2D...
The displacement of the voxels can also be done with a simple matrix transformation, as far as I see it; same like you'd do for the nodes in a 3d model, same as you'd do it with a 'traditional' 3d pipeline (OpenGL/DirectX) - unless I'm behind the curve and animation itself is done on the GPU nowadays, but I don't think it is.
Ragged boundaries wouldn't be an issue as long as there are enough voxels. What I understand, that's the whole point - you just have lots and lots of voxels, at a much finer resolution than you'd ever want to render at, so that you just (by brute force) render once without having to worry about anti-aliasing, gaps between planes etc.
Yes, but the point is that you have to displace much more voxels than you'd have to displace vertices otherwise. Vertices are only on the boundaries of the object (and quite sparsely at that if you use normal mapping shaders etc wisely) but voxels fill the entire object densely...
I was under the impression that they fill the object's hull
as far as i can tell their voxel tech is just for non-moving objects like the landscape, buildings, etc. animated and moving actors are still rendered as polygons and blended into the scene (according to the article, this already works to a certain degree, but they weren't able to give a live demo, just a video).
Supposing this is legit, I'm cautiously optimistic, homie has probably uncovered some very novel techniques that allow the granular level of detail you couldn't do before with voxels, with optimizations that don't require GPU .. this is pretty amazing.
Edit: They claim in the article to have devised an approach that no-one else has come up with, and they give this as the reason why no-one else is doing something similar.
Part of the point cloud needs to move for animation, which involves re-indexing at least part of the world. Doing this once per frame is probably far too expensive (at least currently), which is why animation doesn't feature in their videos.
That doesn't mean this technology should be dismissed; most world data in 3D scene graphs is static, so it does have applications. I don't see it overtaking rasterised graphics entirely, but I think there is value in it.
I'm not an expert either, so take my comment with a healthy bucket of salt.
Then of course there are the problems with lighting, animation and so on.
You're right that lighting and animation are real problems with voxels. Animation in particular is pretty much impossible, which means you have to fall back to polygons for anything that moves or changes, which in modern games is a lot of stuff.
No no, I explicitly said "ahead of time". It's easy to determine what's on screen when you're about to render the frame, but you have to load every single object in memory because every single object is a "candidate" that may end up on the screen, you just don't know ahead of time. You can't cull the voxel tree, even though 99% of the voxel tree won't be used when rending a frame. So the memory overhead is immense.
Voxel trees don't make it easy to discard things that are far away at all, because there aren't any low-resolution objects that you can use for far-away objects. You have to load many objects in memory even when you use them to render just a pixel or two.
The polygon-based worlds we have today use various low-resolution models for when you view at a distance, and as you get closer the higher resolution models "pop in". This tech advertises that you have infinite detail and that there's no need for objects with different level of detail. This means that the complete point cloud of every object has to be loaded in memory and that's a very real problem and it's in complete contradiction to the claims made by the company that artists can create infinitely detailed worlds without having to worry about vertex counts or whatever. Artists have to compose their world out of re-usable high-detail objects or the sparse voxel octree will be way too large. This means the artists will be much more constrained than they are today, not less.
My understanding is that the problem is that there's just too much stuff that you have to keep in memory if you don't use LoD to distinguish between nearby and far away objects.
Consider this screenshot from rage: http://pcmedia.ign.com/pc/image/article/116/1162072/rage-201...
What are you going to cull? There's just too much stuff to load in memory. Too much stuff that actually ends up on your screen. Too much point cloud data. Now games use low resolution models and prerendered billboards for far away objects. With a voxel octree you're just SOL, no matter how clever your spatial optimizations.
Unless hard drives (or SSDs) become a few orders of magnitude faster I don't see how the data can be fetched from disk quickly enough.
But if they still need to use 'traditional' techniques like LoD, rendering at various resolutions, etc, then the complexity and running speed can't be all that much faster than a traditional approach. The way I've been reading these articles is that somehow he's got a very fast method of deciding which voxels to show, without any pre-processing or optimisation - just brute force. I'm curious to see the first demo they'll come up with, especially since they only now seem to be at a point where they can render without shading, animation etc, and they make it seem as if those are just implementation details that can be added on an afternoon. We'll see.
[1] http://www.flipcode.com/archives/Realtime_Voxel_Landscape_En...
I'd love to see this released, but the article was far too positive and the tone read too much like marketing to me. Some careful, not too aggressive words at the introduction and blessing after blessing afterwards, sprinkled with the seemingly unbiased author's impression and description of a very nice and professional guy. Mhhhh...
Edit: Just saw that user 'Causification' already said something similar, albeit with a good amount of more emotion. Still, I'm going to keep this here as a personal impression of someone that has no clue about graphic engines in general, i.e. my layman's reaction after reading the article.
http://pericles.ipaustralia.gov.au/ols/auspat/quickSearch.do...
I'm also very suspicious of all this recent positive coverage of Euclideon. There was a pretty suspect feature on Euclideon on an Australian game review show called Good Game a couple weeks back, talking with this same guy, and not really making much mention of the skepticism involved (or at all if I remember correctly), and as far as I know no outspoken skeptics have been able to get their hands on it. The clip from Good Game can be seen here: http://www.youtube.com/watch?v=f_ndZ8ETbqU
If each "voxel" took only a single bit to store, this works out to require [42 trillion / (1024 * 1024 * 1024 * 8) = 4 889.44352] gigabytes of memory.
How do you store that while you calculate the screen pixels? Modern PCs only have 4 or 8 gigabytes of RAM!
That's the magic of it -- they are able to create worlds with insane amounts of detail, but in an intelligent way that means they don't need an insane amount of computational power to render it.
honestly, the article is not of the highest quality.
The thing is, the textbooks also have all the things that didn't work or don't work well. Even some educated graphics programmers often fail to understand that representing 3D objects is just a data representation and transformation problem. Sure, a hard one, but there's nothing magic about it. Representing the data as a bag of "atoms" instead of polygons just isn't a breakthrough. The problem is still spacial search. The problem is still a simulation of optical physics. Graphics is just an optimization problem now. Attend an IEEE conference and maybe 5% of papers will be new theory, the rest will be about effective optimization techniques.
When someone comes to you and says "I have this awesome idea because I didn't read anybody else's ideas", just walk away.
Only time will tell though.
If one were in the perpetual-motion machine business, taking a calm attitude towards haters would be pretty much the only option, since there would be no way to argue with them.
Nevertheless, I did watch him interviewed and give the real-time interactivity demo to an AU gamer mag a few months ago (I think it's still on Youtube).
I know a bit about the traditional polygon rendering pipeline and I was quite impressed with his response to his critics.
I'm reserving judgment since there are still big unanswered questions. But I would not expect him to be satisfying my curiosity with answers if his strategy is a legitimate attempt to develop the technology with a bigger partner (e.g. a console company or first-party studio).
Plus, they've been hyping up the same tech for years now, with very little progress. People keep throwing the same criticisms at them, and they keep delivering very unimpressive responses.
If they've indeed scored $2 million of government grants already with this demo, as others allege, then it couldn't be more obvious: it's a zombie company designed to look impressive and leech off investor money, nothing more. It's centered around tech that has already been explored thoroughly by academia and industry, and we know where the practical problems lie in turning it into a viable game tech.
I guarantee he is optimizing his entire company for fancy demos. He'll sucker some idiot big company into buying his technology, then we'll never ever see it in a product.
These things are predictable as clockwork.
All I can say is good for Dell.