Gamedev in Lisp. Part 1: ECS and Metalinguistic Abstraction | Hacker News Reader

Gamedev in Lisp. Part 1: ECS and Metalinguistic Abstraction

awkravchuk.itch.io

178 pointsmyth_drannon30 comments

A recent, notable game made in Lisp is Kandria: https://store.steampowered.com/app/1261430/Kandria/ / https://github.com/Shirakumo/kandria

I've been working on a MUD in Clojure a bit lately. Initially I started with ECS using immutable data because I wanted automatic concurrency. Systems declared what components they needed to work, and the core of the system would examine what components each system used and automatically parallelize them. Systems could say if they only read it, or updated it, or appended it (two systems appending but not reading or updating the same data can run concurrently), if they could use stale data, etc.

Components were just a key in my game state hash map, which used immutable data, so if something said it only read (but didn't write) a component, but it was lying, there would be no bug - the core is responsible for actual updates returned from systems, not the systems themselves.

But then... where do you stick "globals"? Sometimes systems need access to common non-component data. You could hack it and stick it in components, which felt weird.

So I kinda changed my design. Now, at the core, I have systems. And systems declare what keys in game state it needs, similar to how they did with components. But it doesn't have to be only for components. It can be any key or "key path" (nested maps). And systems are automatically ran concurrently based upon this.

I happen to arrange entity data as components, because that increases the concurrency of the systems. But it doesn't have to. Sometimes I split things up for the sake of concurrency.

This is also really easy to test. You just have functions that take data spitting out data.

These globals are sometimes calles ‘Resources’ instead of components, and are basically singleton components or dependency injection

Enjoyed this post a lot. As a former professional game developer I spent some time once exploring writing games using the SDL and related libraries. Ultimately it did not progress pass proof of concept but we got some hobby users. Likely it will still mostly work today as Common Lisp is quite resilient to bit rot https://github.com/lispbuilder/lispbuilder

That is interesting! I worked on game and VR development on the late 1990s for Nintendo’s and Disney but I always used C++.

Common Lisp is probably my favorite language so I bookmarked this to use when I get home from traveling.

If you don't mind me asking.

What did you do for Nintendo and Disney?

Because doing VR in theate 90s sounds super interesting!

Main things, worked on Ken Griffey Major League Baseball, and I was lead developer on a dinosaur river ride VR project.

That is really cool!

Cool use of macros. That simulation at the end was very interesting.

One thing I've always wondered about ecs is if you just focused on the array and looping part and instead ignored the component part. Seems a lot simpler over all amd is probably faster too because you're not doing a bunch of branches on components.

This is a good spot, ECS really shines when you have a lot of homogeneous entities. Which is why demos are all about enormous cities, thousands of troops or basic particle systems.

If your entities are more heterogenous or don't need to be running all the time the benefit for gameplay code disappears quite quickly. In particular the acronym conception of ECS architecture isn't the most performant way of organizing things. Which is why you end up with concepts like archetypes to base memory organization around.

Nothing wrong if you're writing a general-purpose engine around ECS as an organizational principle but for game makers you still need to think about memory access patterns and how to organize around that.

If you're just trying to make a game then make the game and organize the game data around it's unique access patterns. If that's just a flat array of tagged unions then who cares.

ECS is a fairly "late" idea in game development history, "just stuff widgets in arrays" works just fine in small games. But when you have

- commercial games getting too big for any one single person to keep track of all those widgets getting developed in parallel

- engines being developed as standalone generic projects to be used in many, many games that need solid abstractions

- enough RAM to justify the overhead (whole megabytes! Scandalous!)

...then ECS starts making a lot of sense. That was in the late 90s/early 00s, and these days the (performance) overhead is negligible even for indie game dev. The only potential downside is the cognitive overhead, but that very much depends on what you're doing.

"This concept is also known as DSL (Domain Specific Languages), but only Lisp dialects have it incredibly tightly integrated into their core."

Historic reminder: Forth was making games using DSLs long ago. In the days of Kilo-byte memory spaces, it was a strategic advantage, IF you created a good DSL.

Lisp in gamedev: many of Harmonix's rhythm games are partially powered by a bespoke Lisp-ish scripting language called DataArray, that is serialized into a tree of nodes and arrays of nodes.

Whenever I think of Entity Component Systems I think it as the furtherance of basing your game systems around singletons.

I prefer testing and so typically avoid singletons, but I do acknowledge the pros of thinking of games as a series of “managers” telling things how to behave. Enough to appreciate that thinking of a game as centralized set of manager systems is kinda the basis of ECS.

Throwing this out there in case anyone in this community has interesting thoughts on the subject!

I agree that having a pure function is better than one with internal state or which uses global variable. But if your problem requires global state, then there’s nothing you can do.

Games generally are inherently a complex system all interacting with a global state.

So whether you’re making a singleton class, a regular global variable, or averting your gaze while passing a massive state variable to every single function, you can’t escape it.

Ignoring the side effects of io, a main loop can be replaced by a recursive call that passes the new world state resulting from the time step back to the beginning. That world state can be expressed as an ecs, and the operations performed on a world state to produce a new one can be the systems, and still be pure functions. There is nothing in an ecs that requires imperative programming. Some of the benefits of an ECS are related to allowing granular copies to be as shallow as possible for such a world state, to allow for immutability. I'm not sure where a singleton would get used even if you were wanting to go imperative with an ecs.

You don’t even have to ignore io: input is a coeffect which is passed to your otherwise pure function as an argument and output is directly derived from your returned works state. That is, yes there’s some side effects happening before and after each iteration but it can be relatively cleanly separated. In my toy engine, I do this: input is fed in and rendering is driven by the output. My game logic isn’t a pure function (it’s a task graph that reads and writes from event queues and internally maintained task state, but input is fed into the start of the task graph and rendering is driven by tasks at the end), but it conceptually could be as all the statefulness is encapsulated and internal.

Sure you can express a game loop as a pure, recursive function but the overhead of doing so is typically too much for stable performance.

Interesting stuff. Saw some steam games in lisp and heard about lisp game engines as well, Need to try this

When I read an intro to game dev the first thing I read should be how to draw something to the screen not a discussion of design patterns. I get that lispers might like that but for anyone else it's probably less than informative.

For editor I never like emacs. The vim is very ok with this starting point : https://susam.net/lisp-in-vim.html

I don't know if you did it on purpose, but this is arriving some 40 years late into the flame war and then casually throwing in some bait like you've been living in a cavern. Also, it's entirely unrelated to the post.

> Also, it's entirely unrelated to the post.

Entirely unrelated to the post? Entirely? The post talks:

"To write Common Lisp code with comfort and still have the ability to interact with REPL, you can choose your favorite IDE"

And then it goes on to provide some recommendations for major editors out there. So the GP is at least a little related to the post?

I did not get the impression of any flame war in the GP. As a Emacs user myself I am well aware that everyone does not like to use Emacs. Many like Vim. I don't see the problem with recommending a Vim setup for Lisp. We need more recommendations like that for different editors to make Lisp programming more attractive to people of all types of technical background.

Reply on news.ycombinator.com