The data-oriented design process for game development
computer.org
computer.org
Originally introduced for its particle system, the underlying system is a pure data-oriented framework that is supposed to be extremely fast.
https://docs.unrealengine.com/5.0/en-US/overview-of-mass-ent...
This is a very surprising take to say the least. I always understood data orientation in game dev as exactly that. Can experts chime in on this?
From the article: "DOD promotes solving concrete problems as opposed to generic ones". Given that DOD is biased against unnecessary abstractions, this can improve the code's readability (e.g. no more Abstract Factory Manager Generators) and thus maintainability. As Mike Acton pointed out, it's very useful to be able to easily reason about what your software is doing (not just a particular class, in isolation).
The quote you mentioned seems to be in opposition to the OOP-centric view of the world, and hints to alternative ways of organizing code (PODs and transformations).
I would say the purpose of DoD is most certainly to make fast programs, but guess the point of that quote is just to further emphasise that this is all about data.
An object oriented programmer would propose the program to be built up from objects, the functional programmer would propose to build with pure functions. I would image a DOD programmer would propose to build the program with data transforms, which is a more general thing than just "make program go fast".
Of course making good use of hardware is easier in a paradigm that uses hardware level data transforms as its building block, but I don't think it makes any sense to make that the _defining_ feature of the paradigm.
But you can do DOD without that being your goal, because ultimately DOD is saying that you use “classes of objects and maps between them” as your framework for design. That it allows you to substitute efficient versions for each is a benefit, not a requirement.
Also, DOD isn’t fast unto itself, it just lends itself to fast implementations.
I was thrilled several times when I started exploring ECS. As a long time pseudo OO developer (not a fan) I found that first of all ECS really did help me write fast update loops, and when the loop become slow it was easy to determine what was slow simply by commenting out different system update calls. IMO it really delivers on the performance front of you pay attention.
Second, you can't always do this, but I had cases where once an entity had no more components, it simply ceased to exist, and there was no separate collection simply containing entities themselves. A novel approach compared to OO, and I found it refreshing.
Third, it encourages you to think about each component separately, because they will have their own collection and engine. This makes refactoring so easy, because you are looking at typically a handful of properties at most, and a small dedicated engine routine. This delivers on many of the promises of microservices.
The last obvious but IMO incredibly important benefit I found was the loose coupling that resulted. Sure not everything is isolated, I would often need to share components between services, but usage and purpose was clear and obvious, and I found myself refactoring components with ease. Contrast this with OO design that encourages abstract concepts like User which, it turns out, have the gravity of black holes and can swallow up any number of properties that SEEM like they should go there. Good luck teasing this apart later, and even remembering what half of them do.
I found the concept in the article that humans are more likely to make additive than subtractive changes profound, and it seems we must recognize this and fight against it as developers wherever we can. (It's definitely true in the audio world, where, e.g., people are much more likely to make additive than subtractive EQ changes, unless they're professionals.)
"Data-oriented design (DOD) grew when game developers needed to use modern hardware architectures for performant games, and existing software processes did not meet their needs."
So, I think performance and DOD have been closely tied together from the start. There has been some evangelism to push it out of "it's just for optimization" territory (and reasonably so), but I'm not sure I agree with their notion of "core".
Or, since each of your systems are relatively self-contained, your `update` function will more or less allow you to inspect which order they’re in and when they’ll iterate. I’ve worked on games where that logic has been hidden behind a bunch of vtables, and that made it unclear where/when certain logic was happening.
https://www.gamedevs.org/uploads/data-driven-game-object-sys...
MA, MA, MA
DoD lets you tweak data w/o a recompile. Data is composable and transferable. DoD is highly iterative and that's a major reason why its in the running.
That said, its also nice that DoD can fairly easily be array oriented and that's where a lot of the speed comes from.
As far as DOTS goes, Unity has been advertising the performance pretty hard because in the short term its going to be a step down ergonomically.
Are there real production uses for this or do you consider it a toy?
To give my personal reason: I have absolutely never managed to find SoA style code readable at all aha
> folks will understand things better if they have to type it themselves
I guess it comes down to if you work in a top-down or bottom-up way to create an understanding of a codebase. Generally, I start by looking at the class names to get a general idea of the architecture and make some mental diagrams of it, and then I zoom in to the level of detail needed. I know other people work better by reading individual functions and working upward from there but it's definitely not my case.
But truthfully I’m not a fan of doing this kind of declarative DSL within c++, it’s just not a pleasant experience and would rather use yaml/toml or embedded lua for the data driven interface
Not having to have an extra code generation step is definitely nice though.
The SoA -> AoS difficulty has driven me to julia for my personal projects, there's a library called StructArrays.jl[0] that is quite similar to your project here.
- Either you can use standalone PFR (it does not really change anything as it's almost exactly the same code, just outside of namespace boost) but apparently that is an issue ahah
- Or you can lobby the committee to accept P1061 ; there's already a compiler implementation based on clang-12: https://github.com/ricejasonf/llvm-project/tree/ricejasonf/p...
With it, the code becomes even simpler and would not need PFR at all ; a function such as
std::size_t create()
{
[&]<std::size_t... N>(std::index_sequence<N...>)
{
(std::get<N>(vec).push_back({}), ...);
}
(indices{});
return size() - 1;
}
std::size_t create()
{
auto& [... v] = vec;
((v.push_back({}), ...);
return size() - 1;
}Has anyone else explored this?
In short, I'm using a run of the mill stack (Caddy/Gunicorn/Flask/Postgres) - but with the twist that all my core logic is defined in plaintext SQL files, which get bound into namespaced Python methods by aiosql. Routing, error handling, templating, etc. are all done in Python - but data manipulation and processing are outsourced to the DB level. All database object definitions are laid out in a massive, idempotent "init_db" method that gets called at launch, so I can essentially point the app at a fresh instance of Postgres and rebuild from scratch. The design is primarily driven by my personal distaste for ORMs, but I've found it extremely beneficial in terms of rigid typing, integrity checks, and performance.
And if you are doing data processing, easier to use a data science library like Pandas for Python which implicitly have DOD built-in.