Data-Oriented Design (2018) | Hacker News Reader

Data-Oriented Design (2018)

dataorienteddesign.com

55 pointsdvfjsdhgfv21 comments

Yehonathan Sharvit wrote a post describing the difference between Data-Oriented Programming, Data-Oriented Design, and Data-Driven Design here. it's a 2 minute read or 15 second skim, very brief.

https://blog.klipse.tech/visualization/2021/02/16/data-relat...

it includes a nice summary of all of the above which may help when reading the OP.

"Data-oriented programming" is a made-up thing by a few B- tier thought leaders. Biggest scam since "hexagonal architecture".

Instead of addressing the major performance gaps of functional programming, they instead just adopted “data oriented programming” in order to intentionally confuse people and try to drown out the clear and obvious performance pitfalls with data oriented programming.

Data-oriented programming is a paradigm that treats data of the system as a first-class citizen. Data is represented by generic immutable data structures (like maps and vectors) that are manipulated by general purpose functions (like map, filter, select, group, sort …).

CompSci 101, how to represent information as data. Like every industry, lots of bullshitiers.

What's the issue with hexagonal architecture?

There's no there there. Maybe even less than DOP.

That doesn't explain anything to be honest; what is "there"? why is there no "there" there? Can you explain it like I'm 5?

This clarification (just definitions) is quite helpful, because probably many readers (myself included) would have assumed that Data Oriented Design would be closely related to Data Oriented Programming (the latter effectively being about the design of software systems).

This is an important set of concepts, but I must say the tone here really makes it hard to tune in. Within the first few paragraphs I've already heard how OOP is bad, not in-line with a high performing architecture, that design patterns were a mistake, etc - ok got it!

I think this coming from a game developer makes sense, especially with the obsessive focus on performance. I really appreciate that point of view, especially when I have to optimize something that wasn't considering performance at all. One really does need to architect software with an eye to how it performs from the very beginning, and during development if you want to have something that truly performs well (sometimes you get lucky / you're working on simple software). This reminds me of the recent podcast on Software Unscripted: Things Web Devs Can Learn from Game Devs with Casey Muratori. Similar obsessive focus, similarly off-putting for me (with a dash more arrogance).

That said, there are a lot of situations where a single pointed obsession on performance will not result in a successful outcome. Business software is full of these instances. I'd LOVE a world where we elevated the concern for performance higher than it is now, but there's a range of other considerations that include:

- readable, simple code and architecture

- expressivity and elegance (related to ^)

- the ability to find and hire someone to can work on what you've made

- flexibility, being able to respond to change

- working around political obstacles

- working with less capable / motivated team members

So: combining the various tradeoffs into an approach that includes performance as a core concern but also doesn't sacrifice too many of the other concerns is key. Doing so with a data oriented mindset is probably a good way to accomplish this, but it's just one angle from which to view the overall solution.

Edit: added podcast mention

But data-oriented design usually does result in much simpler designs, readable code, smaller architectures, and easier composition.

What it doesn't do is let some bootcamper pick up some A-B-C-D-E pipeline (whether that's literally some report generation pipeline, or an HTTP ingress to a CRUD DB, or whatever) and plan their whole career around (some subset of) just B. Or let some serial entrepreneur cycle through a cadre of these guys as he burns them out in turn, because to pick the right choices for B you need to understand (and be able to influence the design esp. re. data representations in) A and E.

Nothing actually lets you do that and still produce good software, and we should stop pretending otherwise.

That's definitely useful info. I'm finding Yehonathan Sharvit's work on this both fascinating and persuasive, without the unnecessary strawmen thrown in.

Apologies if I came off as being dismissive of the concepts, I was strictly talking about the tone of the article itself.

Yehonathan Sharvit does not work on this. He works on his "data-oriented programming" bullshit.

”Within the first few paragraphs I've already heard how OOP is bad”

Whenever there is a discussion about ECS/DOD, “Object oriented” acquires a special meaning in that specific context. It is merely about a popular set of coding patterns used to structure games mostly based on heavy inheritance usage (and often also with lots of singletons).

ECS/DOD isn’t exactly a replacement for all OOP in a game code. It is mostly replacing what would be entities, managers, nodes, basically things that exist in multiples and have specific access patterns. Also: lots of DOD/ECS code and libraries are implemented using OOP + classes.

About performance: with the popularization of ECS the focus shifted away from performance alone. It is actually also a very rich paradigm/pattern that is heavily about dynamic composition and separation of concerns, and it achieves those things in a way that is IMO more elegant than the popular OOP approaches historically and currently used in games.

So I would argue that there isn’t really a tradeoff here when it comes to code quality and engineering.

But like I said above it can be implemented with OOP and classes: there is prior art for using this kind of dynamic composition in 90s software and books, for example.

DOD also has a non-architectural component (i.e. not just "use ECS" which is how it's often pitched), which can be summed up as: Represent your data in a way that's efficient to compute with, not a way that models "the real world" (increasingly irrelevant to most business processes anyway!) in your language du jour.

>I'd LOVE a world where we elevated the concern for performance higher than it is now, but there's a range of other considerations that include:

Surely you should now be able to demonstrate that decent performance is none of the things in your list.

Can you explain what you mean, perhaps with more than one sentence and a little less passive aggression?

We always work in a world where we have to make a variety of tradeoffs. If performance takes a higher importance in a given context, tradeoffs can be made on readability, flexibility, hireable skills, etc. Much of the art of our profession is in maximizing the radar graph for the facets that matter most for a particular situation.

You made a list of things that you supposedly gain by sacrificing performance, and I am telling you to prove your claims.

I don’t want to see stupid medium article childish anecdotes. I want to see a real measured study that backs up your host of claims, as you have assertively made them as if a fact not worthy of testing.

Maybe don’t speak so matter-of-factly about things that are not actually measured facts and you won’t have people asking you to source your claims?

>passive aggression

There was no aggression, passive or otherwise. You’re reading in to it, presumably because telling you to provide sources for your claims upsets you for some reason. Try not getting upset when people challenge your incorrect preconceptions.

"You made a list of things that you supposedly gain by sacrificing performance" - I did no such thing.

I said: "I really appreciate that point of view, especially when I have to optimize something that wasn't considering performance at all. One really does need to architect software with an eye to how it performs from the very beginning"

Clearly I'm saying you HAVE TO include performance concerns from the very beginning.

and "I'd LOVE a world where we elevated the concern for performance higher than it is now, but there's a range of other considerations that include ..."

So performance IS IMPORTANT, as well as a host of other concerns. Where did I advocate for "sacrificing performance"?

You said "Surely you should now be able to demonstrate that decent performance is none of the things in your list.". First off, that's not even good grammar, and it's not clear what you're asking for. "Decent performance" was baked in to the paragraphs of text proceeding the list.

"I want to see a real measured study that backs up your host of claims" - what claims? That we have a variety of tradeoffs to make besides just performance? That isn't a "claim", it's just the truth. It's called business. And I certainly don't have to prove anything to someone as off-putting as you.

Feigning ignorance when called out on your nonsense.

I see the mindset of the excuse parade has not changed in a couple years.

“I didn’t make any claims, I only stated that you have to trade away performance to get maintainability and readability. Where are the claims?! Definitely not the claim that performance and readability and maintainability trade off for sure. What claims?!”

Reposts count as dupes when the topic has had significant attention in the last year or so (see https://news.ycombinator.com/newsfaq.html), which this one did:

Data-Oriented Design (2018) - https://news.ycombinator.com/item?id=36571110 - July 2023 (133 comments)

and in any case it looks like there's a related ongoing thread:

Data-Oriented Design (2018) - https://news.ycombinator.com/item?id=38547737 - Dec 2023 (11 comments)

Other past threads, if anyone's curious:

The data-oriented design process for game development - https://news.ycombinator.com/item?id=31510432 - May 2022 (33 comments)

Data Oriented Design – first chapter (2018) - https://news.ycombinator.com/item?id=29292868 - Nov 2021 (19 comments)

Data-Oriented Design (2018) - https://news.ycombinator.com/item?id=20380397 - July 2019 (37 comments)

Data-Oriented Design - https://news.ycombinator.com/item?id=12168828 - July 2016 (5 comments)

Data-Oriented Design (2013) - https://news.ycombinator.com/item?id=11064762 - Feb 2016 (8 comments)

Data-Oriented Design - https://news.ycombinator.com/item?id=6672854 - Nov 2013 (2 comments)

This book helped me understand the similarity between the entity-component-system architecture popular in game engines and the relational database model. Lots of ECS libraries are building fast in-memory relational databases with code-based instead of text-based query DSLs.

Reply on news.ycombinator.com