Using Rust for Game Development [video]
youtube.com
youtube.com
I found a lot of common ground:
* Trying to write object-oriented code in Rust doesn't work well. Writing data-oriented code does.
* "Fighting the borrow checker" is a sign you might be doing it wrong. Try data-oriented approaches instead.
* Use a structure-of-arrays rather than an array-of-structures.
* Use what I call "state splitting" to borrow mutable references to just the state you need; this works well with the structure-of-arrays approach and is a powerful motivator to use it.
* To build graph-like structures, reach for a Vec of components, and indexes into the Vec for relationships, as your first choice. Self-references, arena allocators, and Rc are viable alternatives, but you should have a good reason to use them instead of Vec.
* Some form of dynamic typing is useful to keep your system loosely coupled (though we ended up with very different forms of dynamic typing, see below).
* Data-oriented approaches have taken root in the C++ gaming community, mostly motivated by performance, but adapt well to Rust, and some of the ideas may be useful in domains beyond gaming.
There were some other points that went beyond what I talked about, but could fit in well:
* A "generational index" is a good way to avoid use-after-free style errors that result from the use of a stale index. The slotmap crate can help.
And now for the things that are different.
* The exact nature of dynamic typing is different. An ECS usually uses a registry of the different component types (anymap is a useful crate) and is quite open-ended in adding new types of components. My xi-win-ui, by contrast, has two main component types, Widget and listener, and does dynamic dispatch on a Widget trait.
This naturally raises the question, which is better? From my perspective, both are valid. The pure ECS approach definitely makes sense when the components are interacting with each other in diverse ways (collisions, damage, etc). In a GUI, the components are mostly interacting with the framework, and have diverse behaviors within standardized interfaces (input, layout, paint).
Thus, one point of my talk is that you don't have to reject all vestiges of object oriented programming, it's perfectly reasonable to hybridize, using data-oriented approaches for state splitting and graph structure, and dynamic dispatch for the behavior variation.
My talk also went deeper into the ideas of "data flow rather than control flow" and the use of an event or command data structure rather than method calls. I'm not sure how deeply these ideas apply to games, but wouldn't be surprised if they do.
[1] video: https://www.youtube.com/watch?v=4YTfxresvS8 , slides: https://docs.google.com/presentation/d/1aDTRl5R-icAF38Di-qJ4...
Could you please explain this in more detail?
# To build graph-like structures, reach for a Vec of components, and indexes into the Vec for relationships
And that also allows you to reference deleted nodes?
instead of this:
struct World {
players: Vec<Player>
}
struct Player {
name: String,
health: i64,
}
struct World {
player_names: Vec<String>,
player_health: Vec<i64>,
}
"Player zero" is no longer an index into a players array, but an index into many arrays, all of which hold certain kinds of data about a player.
> And that also allows you to reference deleted nodes?
... which is why the talk then references generational indices as a way of dealing with it.
> This sounds like it is just going to trade one set of problems for another.
Sure, that's exactly what a tradeoff is.
> what substantiates the claim that it is generally superior?
I don't think the claim is that it is always superior.
Oh, and I would see this refactor of a decoupling. The issue is that, if you’re trying to process Players in certain ways, the fact that the name and health are coupled together in a single struct is an issue. This pulls them apart.
The answer is, they usually don't.
They don't. If you ever end up in a situation where you feel they do, the correct solution is typically instead to split the component into multiple, different components, only some of which will be used for any given entity. This is basically the same as defining a schema for an SQL table. A component is just a set of state, there is no requirement for it to map 1-to-1 to a specific functionality.
> What benefits justify this extremely tight coupling?
The most commonly stated one is speed. ECS was adopted first in game design because it is just so much faster. On modern OoO cpus it's typically something like 5-10x faster than traversing an object graph. On the previous generation consoles (PS3, XB360) with their in-order CPUs and crappy load/store subsystems, it could easily be 20x-50x faster. I don't know how relevant this is to your typical GUI, though; the speedup in an ECS comes from linear memory access, which means that the prefetchers make sure every memory access is an L1 access, which is great when you have a game that has thousands of entities, which don't fit into any cache. But just how many GUIs have enough state to overflow the L1 anyway?
However, speed is not the only benefit. This is somewhat subjective, but having implemented similar logic for ECS and OO based games, I feel that the logic is almost always much clearer, more understandable and less buggy in the ECS versions. Basically, in OO doing things that have cross-cutting concerns tends to get split into many small parts done in multiple places, and it's hard to understand the whole system at once. In an ECS, the logic for one system is implemented in one place, it is always just a transformation that reads in some data, does some computation on it, and writes out some data, without complex control flow. It's so much easier to understand and test.
> It makes it impossible to write generic container types.
An example of a Generic container type is AnyMap, which holds one value of each type (and each value will typically be either a straight Vec for small/common components, or some kind of more complex set for components that hold a lot of data.)
(edit: looked up old numbers and found that 100x was pushing it, even on Xenon. 50x ought to be realistic.)
https://ericlippert.com/2015/04/27/wizards-and-warriors-part...
https://ericlippert.com/2015/04/30/wizards-and-warriors-part...
https://ericlippert.com/2015/05/04/wizards-and-warriors-part...
https://ericlippert.com/2015/05/07/wizards-and-warriors-part...
https://ericlippert.com/2015/05/11/wizards-and-warriors-part...
The final one does offer (imo) a pretty clean solution to the problem he's been exploring, which boils down to viewing the rules of the game as the class definitions and using the things in the game world as data to be plugged in to the rules, rather than the other way around.
Also, fascinating that Rust’s borrow checker sort of encourages you over time to adopt an entity component system architecture instead of a naive OO architecture.
Is this true outside of gamedev contexts?
The idea of operating on individual components of an entity is very compelling in those cases, it's not even novel. We often have done this by pulling data out of datastores in parallel array like requests. The difference with this model is that it creates an elegant interface over the data for an entity that isn't a bastardized version of OO, instead just the data and structures you need present in the context in which your working.
As a non-gamedev, I found this really compelling.
https://internals.rust-lang.org/t/a-working-group-for-rust-g...
At the moment the main focus is just to get organised around some common tooling that everyone can get behind. At this point I'd say the most impactful project to contribute to is gfx-rs.
Restrict is one of those very, very sharp tools that's usually only reserved for cases where you've profiled and done the extensive work to guarantee non-aliasing pointers. The fact that it's just implicit as part of the ownership design of Rust is pretty bad-ass.
"The intended use of the restrict qualifier (like the register storage class) is to promote optimization, and deleting all instances of the qualifier from all preprocessing translation units composing a conforming program does not change its meaning (i.e., observable behavior).
The compiler is free to ignore any or all aliasing implications of uses of restrict.
To avoid undefined behavior, the programmer must ensure that the aliasing assertions made by the restrict-qualified pointers are not violated. ..."
That said, I'm hardly an expert here and the situation on the ground may have changed since last I looked (or I may be completely misunderstanding the discussions that I've peeked into). An example of the problem in this GitHub issue: https://github.com/rust-lang/rust/issues/53105
We do the equivalent of telling the compiler that things don't alias at function boundaries (i.e. restrict) and not much more.
Nobody writes C code with restrict all over the place, which means that not only is this the finest grained level we can give this info to LLVM, it doesn't even necessarily support it well! We had to turn it off for &mut for a long time because of a bunch of bugs.
The weird thing is, to be able to do most optimizations LLVM does its own alias analysis anyway! (I don't know the details).
A lot of program analysis research is focused on improving alias analyses which are usually slow, imperfect, and global. Rust's aliasing info is "free" (you don't need to compute anything extra to get it), local, and perfect (ish). We really could get a lot of wins from doing our own aliasing-aware analyses, or by reordering the code in such a way so as to make it easier for LLVM (e.g., hoisting reads and writes to function boundaries so llvm just sees a bunch of locals)
That said, yes, re-writes are probably a bigger thing here. Especially lately: https://llogiq.github.io/2018/09/06/fast.html
and two years ago: https://news.ycombinator.com/item?id=12232385