Principles of Data Oriented Programming
blog.klipse.tech
blog.klipse.tech
In the context of C++ some of us have been calling these programming/design principles "Value Oriented Design". Some talks on the topic:
- Most Valuable Values (Juan Pedro Bolívar) https://www.youtube.com/watch?v=_oBx_NbLghY
- Squaring the circle, value oriented design in an object oriented system (Juanpe) https://www.youtube.com/watch?v=e2-FRFEx8CA
- Objects vs Values: Value Oriented Programming in an Object Oriented World (Tony van Eerd) https://www.youtube.com/watch?v=2JGH_SWURrI
For example I'm porting a parser engine, implemented in another language, to C++ and it wasn't clear what the hierarchy of objects should be for the purpose of RAII (because there are circular dependencies) (the original implementation language was garbage collected). The original implementation loaded a grammar file and directly created objects with pointers (originally, references) to other objects.
I introduced an intermediate layer where the file is first read into data structs which hold the integer values from the file. So instead of objects pointing to objects, the links are implicit because of things having the same index. The representation of the loaded grammar file is now copyable, moveable, immutable, etc. A side effect is it's trivial to tell whether the file loading code is correct or not when the result is just the same data with structure applied to it, rather than having the added dimension of determining whether a graph of objects are correctly relating to each other.
Then I construct the actual parser engine objects from the data representation structs. True that didn't in itself solve the RAII-hierarchy problem, but what it did do is make it easier to isolate that problem to just the domain of how the objects are used and not commingling that with the problem of how the file is loaded.
The epiphany I spoke of is that after this refactor, it became clear: the file is arbitrary. For testing, or for use of the parser with a grammar which does not change, I could dispense with the file load step and just encode the grammar directly in value-structs.
Why I think this is significant is that "the way I was trained" to think of making code like this unit-testable is to mock the file reading interface. That's a lot of work for something that's only necessary because of an over-emphasis on objects and behaviors instead of thinking about data and values.
"Model the data part of the entities of your application using generic data structures (mostly maps and arrays)."
and example
function createAuthorData(firstName, lastName, books) { return {firstName: firstName, lastName: lastName, books: books}; }
For a simple, obvious object like "person" it might work, but for a complicated domain object with many other composed objects this starts to be a pain.
I am not feeling ready to memorize all the field names, I prefer to have an object with documentation for each field.
Also, I would be careful with using it so much as his example:
``` function createAuthorData(firstName, lastName, books) { return {firstName: firstName, lastName: lastName, books: books}; }
function fullName(data) { return data.firstName + " " + data.lastName; }
function createArtistData(firstName, lastName, genre) { return {firstName: firstName, lastName: lastName, genre: genre}; } ```
you can also do `fullName({firstName: 'Elmond', not:'really'})` and have an hard to catch bug (where an actual typesystem would catch it).
You can either use a Protocol for this (PersonName) or a base class (though those are going out of fashion nowadays)
type example = { firstName : string; lastName : string};;
let name x = x.firstName;;
name {firstName = "Joe"; lastName = "User"};;
let name { firstName : string} = firstName;;So adding a value and make it (inherently) incompatible in the value system breaks the benefits of a couple of the six points outlined in the OP (given the version field suggestion).
Just saying. Your mileage may vary. But again, introducing version attributes is most of the time (and that is a warning) _increasing_ complexity.
One of the articles referred to by the op is [out-of-the-tar-pit] which is fundamentally about complexity and WTF it is paradigms, on syntax level and language support. A version field is a counter on higher level on top of anything of it (and therefore in the off-topic domain already to a larger extend) and also ruining any of the value comparison ability (adding the version field exploits the value inequality in DO as per OP making it part of the versioning system) introducing meta-date and IMHO ruining DO.
If you need to encapsulate state to take a short-cut, introduce state. Don't ruin value(s).
Just my 2 cents.
(/edit: better than version attributes are just any attributes as they work towards both directions of change. not that straight forward to deal with at first, but offering more flexibility. it would be that some older value handling functions are incomplete [but compatible] and new ones just fitting. similar, namespacing for attributes are orthogonal as well [as in closure, depends language at task supports them] while version fields are imposing one general forward direction only, IMHO only for last resort if any other kind of consistency is already considered lost beyond recognition [most often this is _not_ true in computer systems, it's just that everyone involved is too f-c-k'ing lazy to take care and then blames others for anything but getting the job done my/your-self. when in doubt ask operations, they may tell you if they have time])
[out-of-the-tar-pit]: https://raw.githubusercontent.com/papers-we-love/papers-we-l... Moseley/Marks 2006
For distributed systems this has been solved in the last two decades (if not even longer), but still systems are pressing against formality, how come?
Continue to write the code to deal with the (always changing) data, maybe that's it when diving data apart from code (:OP/DO#1 ; Perlis 9. It is better to have 100 functions operate on one data structure than 10 functions on 10 data structures.).
Using data doesn't preclude documenting fields, particularly if those fields are assigned a unique name.
For example, in Clojure you'd namespace the keys, so instead of "lastName" one might write :author/last-name instead. This keyword can then be documented or even assigned a type/spec.
(That said, you can't currently assign a docstring to a keyword in Clojure without the use of a third party library, or using a comment or external document. Hopefully a future version of Clojure will make this part of the core language.)
Many of the early PHP applications were written in a style using generic data structures (array that is in PHP) having functions dealing with all these.
There is no free lunch.
If you already have domains you can model in, the strategy to reduce complexity is perhaps more from the DDD book which looks like a higher level concept to me and therefore may be more fitting for higher complexity levels.
(but again, one for sure can shoot in her own foot with DDD as well)
What I like about the DO thing is it maps well on simple REST APIs sending and receiving JSON text. Quite popular these days to say the least if not mentioning Serverless.
A similar trend can also be seen in structural logging.
These systems are often distributed and complex.
As it bothers you in your case, I would tend to say, it's better to stick to a domain if there is one. Across boundaries of domains, values to emit and receive data can work out very well though I can imagine. As so often, it depends.
It may be that complicated domain objects with many other composed objects are not compatible with this style of programming.
Whether or not that's a good thing could be the subject of an interesting discussion. For one thing, negative experiences with the "large complex classes" approach to domain modeling is a major factor in the backlash against object-oriented programming. Lately I've been trying to familiarize myself more with the early literature of OOP, and I'm discovering that even OOP's early pioneers had already had bad experiences with it, and would warn against the temptation to do things that way. OTOH, there's undeniably a certain attractiveness to it, otherwise it wouldn't be so common.
The reason you would use this setup is so you have all your data in contiguous arrays you can SIMD through quickly. If you're doing a lot of operations where n is low then its not that useful (and even detrimental) to orient your data array-wise.
The main benefit of dedicated, encapsulated types is that they can preserve their invariants. Without that each function operating on "data" has to check that those invariants are kept. It's fine in trivial cases e.g. where this validation boils down to checking if a value is present or not - which might even be supported by the language. But I don't see this working in more complex cases.
In my experience many (most?) bugs are caused by the implementation assuming something incorrect about the shape of the data and I feel like this principle only makes it more likely to happen.
These two slides in the Clojure, Made Simple talk are my favorite counterpoint to even shallow objects being easier to work with than data. See starting at 50:00:
If you don’t want to click through, the list of HttpServletRequest methods is bigger than the map itself! Each one is its own little DSL, and they are not consistent with each other. To do anything with the data, you first have to figure out which method gets the bit you want out of the thing. And that’s a simple case—the complexity only goes up from there.
The map OTOH is just a map; you have hundreds of simple, composable functions that you can use to slice and dice the data however you need.
Want the keys? (keys request)
Want the keys of the headers? (keys (:headers request))
The user-agent value? (get-in request [:headers “user-agent”])
None of these are specific to a servlet request; they will work on any piece of data. And it’s trivial and transparent to build more specific functions out of them that fit your problem domain.
However, JavaScript or Clojure are not ideal for demonstrating this methodology in the sense that industrial applications will more likely be built in C++, Java or Python. For example C++ and Java support Apache UIMA, which is an industry standard for data-oriented systems. UIMA (originally developed at IBM before open sourced, and use in their Watson system) manages data as immutable objects (e.g. text, videos) that are enriched with annotations (e.g. syntax graphs, topical tags, subtitles...).
Functional designs are often well-suited to data flow related processing, whereas in OOP, you end up with a pipeline object and various DataStream objects that it inputs and outputs.
In my experience, data-intensive systems often need to: - cater for distributed processing due to large-scale (which calls for Apache Spark, and then PySpark or Scala); - compute-intensive work like machine learning, which may require GPUs or other bespoke hardware (Tensorflow supports GPUs, Google's cloud has TPUs); and - special purpose data structures (e.g. Bloom filters, huge persistent graphs, R* trees...) specific to the nature of the processing (this latter point, I guess, contradicts to the author's claims).
The big one: "Data is immutable". The problem here is that data isn't actually immutable (generally) and mutability isn't actually the problem. The problem is unmanaged references or other dependencies on the mutable data. The "source of truth" becomes muddled which creates the problem of how to keep the various instances in sync (or otherwise handle cases where they are/become out-of-sync). Immutability is a very useful tool since you have have any kind of dependency -- direct, indirect, implied, etc) -- and there's no worry. But you still need a mechanism to manage mutating data. Maybe it comes out somewhere, but the principles of DO don't cover it, which is a rather serious omission.
Also, principle 2 isn't really a principle. I think what it's getting at is that you don't really know the precise type of your data, over time, in a distributed system, so it's good to include the flexibility to handle that. That makes sense to me. but generic data structures aren't necessarily always the right way to handle that.
Data Oriented Programmming is both old and new. In that it does not have the same amount of programming patterns that OOP has. As it is a more bare metal means of programming without a ton of abstraction to ease most programmers into it.
Where I find the idea interesting is concurrent and parallel processes are more natural in the data oriented. And that is through immutablility and ownership as first principles.
I agree. For me it feels like immutable data is a tool, not a universal principle. For example, in low-level C programming or direct control of a register in an embedded context, immutable data isn't a principle, it's just another approach.
What I mean by "universal" is something like SOLID. [1] (Most people only use SOLID for OOP, but Uncle Bob makes it clear in Clean Architecture that he thinks SOLID is universal, and not limited to OOP.)
That said: it does feel like immutable data should be the _default_ approach in many situations. But that's really hard to do in most languages.
[1] https://www.amazon.com/Clean-Architecture-Craftsmans-Softwar...
This is useful because, for example, you stop having the "unmanaged references" you were talking about because now since you are pointing to a version of the data and not the data itself you can be sure of what you are talking about.
It is not a solution for change in time it is a solution for taking change in time out of the equation when we don't need to talk about that. With immutable ds when we talk about data we are just talking about exactly that and time is taken out of the picture. Its called immutable because now you are talking about facts and not the representation in time of those facts. Because now we are talking in versions so it does not make sense. The thing is this data is a snapshot so you are not guaranteed to have the most up to date snapshot. And that's ok because precisely here we take change in time out of the question in order to be able to talk more precisely about the data. Tracking change in time is another story.
So for example you could have things like react. There you have snapshots of the world updating. When you talk about the data it is immutable but then you change it and update the mutable variable where you are keeping change in time.
This is the basic principle behind how sane database transactions work.
ORMs in some languages can be especially dangerous if they overload the getter/setters of the object in such a way that the checkins and checkouts are obscured; you could be passing your object to a function or method that expects to mutate a polymorphic class[0] that is usually a traditional "shared memory" form of objects, well hopefully you can imagine the chaos, redundant database transactions, consistency problems, failure modes, uncaught exceptions, etc. that are going to be a nightmare to debug.
[0] worse yet, imagine if it's someone else's code and they change the api from not mutating to mutating for performance reasons. Will you notice the documentation change or the changelog? It's bad enough in the case when it's not an ORM and just a mutable object.
Mutability is just a hack to save some memory.
.. and time. If we needed to compute an account balance by summing all the debits and credits since the account was opened...
All of these ideas (and I think they are good ones) are inspired heavily by Rich Hickey's talks and rational behind developing the Clojure language (the author of the post states as much). And while you can use these techniques in other languages/paradigms/problem domains, they are really intended to work well inside the constructs of Clojure, and when applied to "information-driven situated programs" [0] (read business applications with dynamic requirements).
As for some of the short-comings you mentioned:
"But you still need a mechanism to manage mutating data"
Clojure supports this through the use of locking constructs like atoms. [1]
"I think what it's getting at is that you don't really know the precise type of your data, over time, in a distributed system, so it's good to include the flexibility to handle that. That makes sense to me. but generic data structures aren't necessarily always the right way to handle that."
Clojure attempts to bridge the gap between generic data-structures and strongly-typed constructs using run-time specifications. [2]
I mean, the ideas presented here can be generally useful, but your mileage may vary if the principals take you too far out of the idiomatic for your particular language/paradigm/problem domain. If that's the case, you could find yourself wasting energy swimming up stream.
[0] - https://www.youtube.com/watch?v=2V1FtfBDsLU [1] - https://clojure.org/reference/atoms [2] - https://clojure.org/about/spec
The rise and fall of paradigms that present themselves as panaceas is instructive. You have "structured programming", "object oriented programming", "functional programming" and now "data oriented programming".
What I'd like to see is paradigms paired with "where this works well" rather than paradigms sold based on "this will solve the software crisis", "this works better(unqualified by when)" and "if you're not doing this, you're doing it wrong". The later two claims leave a bad taste in the mouth of the casual users/observers, who gradually morph into active critics and sink the paradigm, wasting the good and useful parts of each (OOP has a vast universe of haters because it's most successful of the panacea-paradigms so far and as a panacea, there's much to hate here but still).
You still use if/then/else and repetition and subfunctions in OOP. But you restrict and replace some of their previous usage.
Same for functional programming: you take OOP but remove all the sideeffects. You still have objects which encapsulate data which can only be accessed in a certain way and can behave polymorphic. Mind that inheritance was never really part of OOP in general.
Now about DOP - not sure what exactly it adds on top of FP or if it is even an established and well defined term. Doesn't look to me like it, looks more of a bundle of recommendations right now.
SQL is about as data-oriented as it gets, you have a programming model that's constrained and focused on data layout, structure and performance over being a general purpose language. There's a good article a while back about ECS that I can't find which talks about how most performant ECS start to mirror SQL and other row based data engines.
So the parent is correct. It's not enough to say data orientation is a good thing. It needs to be compared with previous approaches (encapsulation) and then explain what the tradeoffs are.
These are are part of a book that is being written at the moment as far as I can tell:
"This article is an excerpt from my upcoming book about Data Oriented Programming. The book will be published by Manning, once it is completed (hopefully in 2021). "
Traditionally people use mutable struct or objects for non-primitive data structures, which implicitly allocate addresses in computer memories - that is a very implementation-wised approach, more specifically, a very Von Neumann approach.
For example, there are many systems having entities in the databases, like Person { id, name }. But if there's no database, in many programming languages people would write Person { name } only. If people write the in-memory version first, they have to re-write their models. Systems with mutable references inconsistently force the referencing system on people by default, instead of just giving people only they ask for.
In modern days, a lot of computing goes beyond a single computer, there are many distributed systems, a lot of serialization is going on, and the reference in one computer's memory doesn't mean anything to another.
If we try to loosen the definition of the computer, we can think of many systems as bizarre computers. If we take a look at data processing systems like Spark or Flink, conceptually they also look like computers - bunches of hardware and 'operating systems' sit on top of them, but in the form of clusters instead of single computers. In such cases, there are no shared memories, and the memories are implementation details - users won't aware of them like they have to realize there are underlying memory systems in Von Neumann computers. In such kind of systems, people only care about computing which is more substitution based instead of mutable references. While at most, they only need to aware of nodes as a whole in the cluster.
It's close to the 'substitution model' vs the 'environment model', or the 'functional' vs the 'object-oriented' described in the SICP. Where I found conceptually the 'substitution model' is more fundamental, and simpler - it's basically elementary or middle school maths.
> The problem here is that data isn't actually immutable (generally) and mutability isn't actually the problem. The problem is unmanaged references or other dependencies on the mutable data.
So another novel way to look at it, is if you don't have mutable references, you don't have to manage it, then the problem is eliminated as a whole. Like Haskell, Erlang, or Spark as we just mentioned. At the end of the day, you can have references in outer worlds, be it ST monads, processes, database, or just our real world, sometimes the processing part of the program don't have to care about them.
Indeed he (or she) would have made use of traits/protocols/categories/whatever, to separate behavior from data, while keeping the design extensible (via polymorphism).
This is something I usually find in OOP critics, too much focus on class driven implementations, without spending too much on the other parts of the toolbox.
Which breeds data oriented "anti-patterns" when i/o performance becomes the bottleneck. Focus on hardware. It's almost like you need to work backwards to build scalable algorithms for modern data loads ;)
Scalable Machine Learning & Graph Mining via Virtual Memory
My latest approach to get enough tweakability is what I tentatively call "fractal rendering events" or "staged rendering". When rendering HTML or SQL, you need event "hooks" for the different stages. Level 1 events may override/alter field attributes. Level 2 events may override/alter the HTML (or sql) generated for the field based on the Level 1 values. Level 3 events may override/alter the HTML of page sections (or entire SQL clauses). Level 4 is overriding/altering the entire page (or final sql statement).
In other words, the schema provides drafts, which can then be adjusted along the way through event hooks. The granularity of what's tweaked goes up with each stage.
But managing that many potential events needs something more powerful than a file-based system. It may be better to manage such source-code in an RDBMS so you can search, sort, and group by different factors at different times rather than hard-wire in one viewpoint as file systems do.
But current IDE's are not ready for this. I do believe it's the future, though. File trees are too limiting.
Consider this: it's common for a non-coding analyst to want to change a field label, page title, max field length, or "required" status. If they could do it in the schema info (data dictionary), then they don't have to involve the coders. Whether the data dictionary is referenced directly or generates scaffolded code is a stack-specific or shop-specific choice. Minor things like this shouldn't involve a lot of effort.
The biggest annoyance I can foresee is discoverability. Object-orinented classes make it pretty clear what a given object's interface is (just look at its public methods). This is not true of actors in the languages I've used (not many), but in theory it should be a fairly trivial question of syntax.
But for dealing with the historical data from an IoT device it may make sense to use a data oriented/functional approach. That time series data is not stateful, it's an immutable history of something stateful. Functions/transformations work best there usually.
Which I'd highly recommend. It's aimed at a less experienced audience, so, as someone who's mid-career, I admit I did skim some sections. But, all-in-all, I enjoyed reading his take on how things should be done.
Perhaps overly so. I'd have liked a bit more focus on data. That could be a by-product of his Clojurist roots. Data-oriented programming is so integral to Clojure's culture that I'm not sure Clojurists even realize they're doing it half the time.
I totally agree with you. I am a Clojurist and my hope is that my DOP book will spread the "light" to non Clojurist.
The author acknowledges this as being incompatible with static typing, but I'm not so sure. Is it that, or is it that it's incompatible with contemporary static languages?
In FP, we already have a concept of type-safe hetergeneous lists and maps, and even some clever implementations in languages like Java[1]. The ergonomics are often less-than-stellar, but I'm pretty sure that's something a new language could fix with some syntactic sugar.
There is also the data frame abstraction (like, you see in Pandas), which is typically implemented on top of dynamic typing, but major implementations often rely on static typing behind the scenes to achieve efficiency. There are also projects like Frameless[2], which implements a statically typed interface over a dynamically typed dataframe package.[3] I'm guessing, again, that careful language design could get us something similar, but with better ergonomics.
And I'd be happy with that. I've been pulling away from static languages lately, and a big part of that is that I really like how some of the dynamic languages let me model my data as data. It's enough of a complexity saver to feel like a net win, even at the cost of some performance and static verification.
[1] For example: https://github.com/palatable/lambda#hlist
[2] https://github.com/typelevel/frameless
[3] Which is itself, notably, implemented in a static language. Spark also has a statically typed version of the API, but its usage is not recommended for several reasons, one of which is performance. That's something that all us static typing fans should really stop to think about for a bit.
There's a bit of a terminology collision here. What the article is talking about is not Data-Oriented Design, the practice of organizing your data for efficient processing. It's proposing a separate (but not incompatible) concept of organizing your code for easier maintainability. For example, the sample code (which appears to be JavaScript) is not doing ECS at all. It's almost exclusively doing the data modeling by creating one heterogeneous map per entity.
So:
{ a: 6, b: 7 } is different to { b: 7, a: 6 } and { a: "six", b: "seven" } is different to both.
This matters if you are using structural typing as you are still creating different shapes to be passed into the same function(s).
1) The author map has a "books" entry that is a list of map containing the book data. But multiple authors might have written the same book, so does the book data get copied there?
2) The author contains a "books" entry that is a list of Book IDs, like foreign keys in a database. But how about immutability? If you want to get the book addressed by the ID, you need to look it up somewhere and are you guaranteed the object you get from the lookup is still the same one as it was earlier?
3) There is a "Author -> Book" map somewhere that you can pass an author and it gives you the list of books they wrote. Not sure about this one.
Btw. we program everything in Clojure + ClojureScript so immutability and the other points is like preaching to the choir.
Not related, I thought Manning will not publish the book. At least that is the last information I have seen a few days ago. I thought about buying that book.
It's a shame the code examples are just the fp and oop solutions not how it could look in a data oriented way
I mean in this case, he's transformed the program from running with a constant amount of memory (only need as much memory as it takes to run a turn) to having to store the entire history in memory. So this would be really bad if there were many turns, or if the program was going to run indefinitely, responding to user input, etc.
You can bridge that gap by taking periodic snapshots of the reduced state, which is a useful pattern in distributed game development where you sometimes have to back-track and re-simulate when input arrives over the network.
The isuee are arrays/lists.
Because in C# an int[] array or List<int> is a reference.
So even if you put a int[] in a struct this int[] will NOT be copied when you assign an instance of this struct to another instance: you will get a reference share which makes this annoying since you cannot do a proper deep copy with the language itself: you are forced to use reference copy instead of deep copy.
In C++ this is easy because the types differentiate between pointer and non-pointer explicitly + you can overload the assignment operator.
https://docs.microsoft.com/en-us/dotnet/csharp/programming-g...
https://docs.microsoft.com/en-us/dotnet/api/system.memory-1?...
Storing fields in a map leads me to believe this is not Data Oriented Design (DOD). And I completely reject this idea (fields in maps). The "flexibility" there is hardly useful, and could be achieved with defined shapes (types) in modern statically typed languages without all the dowsides.
"Separate code from data" is a big core belief I share with this article, but the rest doesn't seem good idea / novel / important.
Well, it depends on what you mean by data. To avoid ambiguity it is better to talk about data values and data objects which have different properties. This can be formalized as follows [1]:
o data values are modelled via mathematical tuples – tuples are immutable
o data objects are modelled via mathematical functions (one field is a function from this reference to the field value) - functions are supposed to be mutable
(In reality of course we meet quite different situations, for example, struct is mutable and objects can be immutable.)
[1] Concept-oriented model: Modeling and processing data using functions https://www.researchgate.net/publication/337336089_Concept-o...
Perhaps you are just pointing out that the output of the function (and therefore the value of the field...?) will change as the input changes?
If mathematical tuples are immutable, then surely mathematical functions are immutable as well ;)
If I have a function:
int Add1(int x) => x + 1
Assume f: X -> Y. We can now map x_1 to y_1 f(x_1)=y_1. And then change this same function by mapping x_1 to y_2: f(x_1)=y_2. Thus we can easily modify functions. Moreover, we do it constantly when we modify object fields in OOP. It is probably easier to comprehend if a function is represented as a table which we modify.
In contrast, we cannot modify data values (mathematical tuples). Say, x=42+1 means that a new value 43 is created rather than the existing value 42 is modified.
> I would expect the domain and codomain to be immutable;
No. Domains, codomains and any set can well be modified by adding or removing tuples. What is immutable are values (in the sets).
f(x_1)=y_2
x=42+2
To be clear, I'm not suggesting you are incorrect. I just don't fully understand what you are getting at.
They would be different functions, the first being the identity function: x => x, the second being: x => x + 1
> Thus we can easily modify functions. Moreover, we do it constantly when we modify object fields in OOP
This isn't the case. A field with a different value in it just means the object is a different value. If the object is passed to a static function, then the domain is the full set of possible values that the object can hold (this is known as a product-type, you multiply the total possible values of each of its component parts to find out the size of the domain).
If it's passed to a method then there's an additional implicit argument: `this`, which is the same as a static function with an additional argument that takes the object. The function is the same.
Global (or even free variables) should also be considered part of the domain: i.e. it's akin to implicit arguments that are being passed to the function.
> No. Domains, codomains and any set can well be modified by adding or removing tuples.
This also isn't the case. If a function is defined that takes an integer and returns a boolean value: Int → Bool then the domain is the set of integers, the co-domain is True and False. You can't pass a tuple to a function that takes an Int and therefore dynamically increase the size of the domain. Even in dynamic languages the codomain is effectively `top`, the type that holds all values, and therefore the domain is all values and the codomain is all values, which makes them immutable still.
Now maybe I am misunderstanding you, but this is how all of the mainstream statically and dynamically typed languages work. Perhaps there's some edge-case language that I'm missing here that allows types to be extended, which would be interesting in its own right.
https://github.com/prostodata/prosto - Functions matter!
``` function isProlific (data) { return data.books > 100; } ```
Isn’t the behavior of a system more critical than its data elements?
I don't think they're inherently contradictory. You can have plain data objects representing the domain, and functions that act on these objects representing the behaviors/actions in the domain. You could include these functions as part of the "class" for these objects, and have them return new instances of the class to maintain immutability.
Data accuracy is critical. Data storage is critical. Analytics and warehousing are critical. But none of these things reflect the nature of a business, which are the behaviors of its domain and sub-domains.
i have implemented ES into few projects and am now writing a ES library, since I think it can be done much simpler than my previous implementations that felt too verbose, and I will take some pointers from this into account.
Through I would argue that #5 isn't part of DO. Especially given that e.g. `{ "a" : "b" }` is not necessary a literal but potentially an expression (depending on arbitrary language definition aspects). On the other hand e.g. `vec![ "a" ]` in rust is definitionally not an literal but wrt. to the idea behind principle #5 as good as `[ "a" ]` in JavaScript.
EDIT: Lastly sometimes contexts using something like a builder pattern can be the better way to "not verbose creation" and "data is exploreable in any context".
Often times literals are only things like `"string"`, `0` and so one.
But thinks like `[ 1,2]` would be a array expression where each "entry" is syntax wise an expression (and any literal is an expression itself).
I don't know if JavaScript specifically does define object literals or object expressions, in the end it depends on what you define a literal as.
Lastly depending on the language something like `[ 1,2]` might literally de-sugar to something like following pseudo code `var tmp = Array.new(capacity=2); tmp.push(1); temp.push(2)`.
So a better way would be that a formulation like "data should be creatable without explicitly doing any function calls, variable assignments or similar. Creation must not depend on implicitly captured data".
Could you explain what you mean by "Creation must not depend on implicitly captured data"?
E.g. instead of `a = [1,2]` you have `a = vec![1,2]` which de-sugars to `a = Vec::with_capacity(2); a.push(1); a.push(2);`.
Now custom data structures can define their own macros like that, e.g. `skip_list![1,2]`.
Which would be all fine. But what if now `bad_skip_list![1,2]` accesses a thread local variable (or other implicit provided data) and adds that, too?
Now `bad_skip_list![1,2]` might be not equal to `bad_skip_list![1,2]` defined somewhere else. Which is against the ideas behind the rule #5.
You should be able to copy-past the literal-like creation of data to any place (e.g. a unit-test) and get the same result.
EDIT: If I remember correctly you could override parts of `Array.prototype` and array construction in JavaScript to brake the #5 for JavaScript for thinks like `[1,2,3]` but I'm not to sure about that anymore.
E.g. ECS is a direct consequence of seperating data from code.
Furthermore for it to work well you normally also want #2, #3 and #4.
Sure you can build a ECS without #2,#3 and #4 but it makes it more complex.
Lastly in a ECS you split up components into many parts each having their own data and you normally want the idea behind #5 to apply to each of the parts.
EDIT: Well ok, weather #2 makes any sense at all depends on the language you use. And using a language where #2 makes no sense can be a as reasonable choice. I only would apply #2 IF it makes sense for you language of choice.