Show HN: I built a multiplayer voxel browser game engine | Hacker News Reader

Show HN: I built a multiplayer voxel browser game engine

kevzettler.com

236 pointsVekz63 comments

This is amazing! I read everything. I completely agree about the state of the 'big three' engines.

> My focus is still on browser multiplayer experiences.

Here's mine: earth.suncapped.com (https://github.com/Suncapped/babs)

It's been ongoing for 12 years, with several burnouts and rewrites, hah.

If you want, I would love to talk offline (email in profile) about player-freedom, MMO projects, and browser games. I feel like some of us have a shared history going back to older MMOs [0] and Flash browser games [1].

On my project, I sneaked past the hard networking problems by settling on tile-based movement. It worked for Ultima Online and I still believe in it. It scales really well. Networking zones measure 1000 x 1000, and tiles are 4 units, so X and Z can each be stored in 1 byte. This also helps with storing and transmitting tons of objects. See the page load time.

Actually, I sneaked past a lot of difficult things, which is what strikes me about your work - you solved hard stuff directly. I haven't tried to integrate skeletal animations into my Blender pipeline yet. Yikes. I barely know how to write cameras, whereas you made a cross-platform one and blogged about it.

> shared client & server game code

How did this one go? I ended up literally doing a `ln` for shared frontend/backend files.

> React+Redux setup was too much performance overhead for the real-time gameplay section.

This is such an interesting aspect of web games: HTML-based UIs. I went with Svelte because I was curious about it. It is SO much easier than doing a game UI yes. But the "line" between web UI and game UI keeps being a problem. For example, when a player drags an object into a UI backpack: That object's graphic needs to change from 3D mesh -> mouse cursor -> png icon. Craziness. I might scupper the whole thing.

A bit of a ramble, but in conclusion, wow!! Thanks for sharing this.

0: UO, Lineage, you said Exteel which I'd like to learn about.

1: (First Earth had a Flash version lol. Also a Babylonjs version.)

Thanks for reading and sharing your work!

I host an indie gamedev coworking group in the bay area and on discord that you might find interesting. Will DM you.

> How did this one go? I ended up literally doing a `ln` for shared frontend/backend files.

Very easy with module imports. Same scene graph and entity component system shared between client and server.

Greetings Vekz! Great to read your article!

(Conrad from the coworking group)

> This is such an interesting aspect of web games: HTML-based UIs.

I am neither a web dev nor (especially) game dev expert , but I recently came across this crossroads when building a small js canvas game. Should I use html elements styled with css for my game UI?

After a very short time thinking about it I decided not to implement the UI this way. It seems like a good idea because html and css are easy to use. Much easier to style a button and assign it a js function than draw buttons on the canvas and check if they’ve been clicked. But I felt like it spanned a boundary that might prove hard to reconcile in some scenarios, but that was more of an instinct than a well reasoned conclusion. I’m curious what others think about it.

Personally I went with HTML for my web game UI and I've not had any reason to regret it. I mean game UIs are thorny and annoying, but it would be a lot worse working inside the canvas.

My biggest surprise was, before I jumped in I figured there would probably be some existing "RPG inventory" libraries I could build from, that handled dragging icons around between icon slots etc. But I never found any game-oriented UI projects I could use, and ended up building entirely from scratch.

Not a game but I once worked on a Miro clone. We used canvas for the... canvas part... but plain old HTML for all the UI elements as those don't need to be redrawn every time you blink.

I'm still reading this, but I just wanted to say this bit of your analysis was SO cathartic for me to read:

> React+Redux setup was too much performance overhead for the real-time gameplay section. The state updates through the Redux action reducer pipeline, and the minimal React render updates were enough to cause noticeable hiccups in the gameplay frame rate. Performance in the browser environment is susceptible to garbage collector management. To minimize garbage collector hits, you need to use object pooling. Object pooling is a mutable state management pattern in which you pre-allocate a pool of objects. The collection of allocated objects gets mutated and reused during the program's life to minimize runtime memory allocations. This object pooling pattern conflicts with the immutable update patterns of React and Redux. Hitting these performance issues was a significant roadblock and essentially became a 'rewrite' in which I had to rewrite the game state management to be performance optimized. This rewrite was costly and took a lot of time.

This is 100% the conclusion I came to when trying to build a web-first game from a React/JS background and it's wildly reassuring to read someone else coming to similar conclusions.

Damn, it took me years to accept that dropping C-style or C++-style explicit memory management in favor of garbage collection was an overall improvement. And now we have to reuse and mutate objects instead of creating new ones because of it.

I can see the need for memory management for a game. There are other things, also graphics related like map points or point clouds.

You are more likely to a struct of arrays than an array of structs/objects as well. Very handy in C but also JS.

I suppose you can use a sliding window object (or “fly” object) that pretends to be the object when the data is really split across various arrays, just don’t actually loop over them. ;) I bet someone has done a library for something this way and used proxies.

The React style forces a lot more object creation and copying then you're probably imagining. If you want to update an array, you generally have to make a new array, and copy over all the data except for your mutation: https://react.dev/learn/updating-arrays-in-state

If you, for example, hold your voxel level data in a big array, that means copying that data every time the level is modified.

"The React style forces a lot more object creation and copying then you're probably imagining."

Its not react style its literally the core of how react works(immutablity). To rerender a component you need the new state to point towards a new reference to an object/array (hence the copying data to a new array) because JS compares objects/arrays through reference and not value. Not exactly react's fault but how JS works.

I said "react style" because they are not using React in a conventional way: https://github.com/kevzettler/react-regl

> I'm writing this post to put closure on the project before moving on to new work and in the hopes that this experience may be helpful to any others interested in this space.

Thank you, as someone that has struggled trying things in new spaces I'm sure I've missed out on learning from others who came before me, hit a wall, and then walked away taking their knowledge with them.

I appreciate you sharing your story and work for others.

Wow, as someone who's been maintaining a JS voxel engine ("noa") for some years now it's surprising to find a bunch of influences here I hadn't seen before. I also got started from the algorithms on Mikola Lysenko's blog, but I'd never come across VoxLords at all. Great stuff OP!

I just checked out noa - I'm shocked that Minecraft Classic just went and used it without much contact? That's a really huge feather in your hat. Would you have done anything differently about that? (I just switched from MIT to AGPL, so this has been on my mind.)

Babylon.js, cool. How did it do with voxels, or how did you work with it? If you feel like answering. I used it previously on my project.

I have to check out all the other games on that list [0].

0: https://github.com/fenomas/noa

Thanks! I know basically nothing about the Minecraft thing - I didn't hear about it until a week after it launched, and by that time the multiplayer servers were dead and the game was abandoned. Apparently the Babylon.js folks had some backchannel dialog with Mojang about it, but they declined to include me or tell me what was discussed.

As for what I'd do differently - I should have ignored it. I invested a fair bit of emotional energy into trying get info, doing some work that the Babylon team asked for in hopes it would lead somewhere, etc. But nothing came of it and it was pretty depressing; I'd have been happier if I'd just tweeted once and moved on.

(I guess technically I should also have bugged Mojang to comply with the MIT license by adding my code's copyright notice, but... life is short.)

Re: Babylon.js, it doesn't have any voxel-related features, so to be honest all the babylon-specific code in my engine would probably look very similar if I'd gone with three.js. But Babylon's real killer feature is definitely its forum, which is extremely helpful and active. I'd probably never have been able to build my engine in three, just because I don't know any way of getting answers to thorny "oh you need to set mesh._dirty before calling that API" type questions for it.

Wow, that's really shitty of both of them. They should be ashamed.

> "When I started progressing with Stackgl, I took a year off and worked on it full-time throughout 2017. I lived off some savings and unemployment checks until late 2018"

Impressive dedication, congratulations!

I did something similar when I was 23. (Now two decades ago - wow...) That project wasn't a runaway success on its own, but it was a fundamental learning experience. Those years spent building a software project of my own, yet one meant for other people to use and extend, completely changed how I think about my career and life and its external dependencies.

Not sure if they still are, but the folks over at DotBigBang were looking for a graphics engineer recently. I don't know how much experience overlap you have with that role, but given your interest here, they might want to hear from you. They're great people :)

Thank you. Great idea.

Cool project, when you add multiplayer to a simple game without any game engine and want to actually do it right (CSP, interpolation, lag compensation) it gets technical really fast

I spent months trying to make a CS like in Three.js, then I was wondering myself if it was actually worth it

I could just have used a random game engine that can export to WebGL with wasm even if the first loading was bigger, people wouldn't even notice nowadays

WebGPU is coming out if I'm correct, I wonder how good will web games run now

Could check this out https://github.com/hexops/mach-gpu

That's WASM based which puts it at a disadvantage to a javascript based renderer.

Really? My understanding was that WASM support was good these days. And performance is a key issue with games

JavaScript will typically have better performance than WASM. WASM is good for porting existing software to the web. So if you have tens thousands of engineering hours dumped into some game engine that engine can be brought to the web instead of having to duplicate that work with a rewrite. Using javascript apis from WASM isn't possible and requires you to do extra copies. Yes, there are optimizations that browsers can do, but it's game of catch up with the performance of javascript. Or if you want to share code between a native version of the game and the web version WASM may be good. Performance should not be the reason you pick WASM.

“Preliminary results show that WebAssembly, while still in its infancy, is starting to already outperform JavaScript, with much more room to grow,”

https://thenewstack.io/javascript-vs-wasm-which-is-more-ener...

I would like to see this done with JavaScript that is an actual game or application. A gameboy emulator is a virtual machine and is not heavy on IPC with the DOM. Measuring a C++ program compiled into JavaScript (C virtual machine) doesn't seem representative either.

The paper more shows that if you are building a virtual machine in the browser you should use WASM. I agree that is a sensible choice.

The calling overhead from WASM into JS in order to talk to browser APIs is usually negligible compared to the actual cost of the called function.

Also see this article from 2018: https://hacks.mozilla.org/2018/10/calls-between-javascript-a...

Quote: "calls between JS and WebAssembly are faster than non-inlined JS to JS function calls"

The problem is that these function only work with numbers. If you are trying to use functions that take or return objects or strings you start running into trouble.

So either you end up making a JavaScript engine that does all of the heavy work in regards to interfacing with the browser (including APIs like webgpu) or you come up with some serialization scheme to allow for wasm to work with these functions.

My point is that if you are making a big chunk of JavaScript already there is a benefit to just doing most of everything else with JavaScript if you can get away with it. I'm not trying to claim that in all circumstances WASM will be slower and I will encourage people to check for themselves, but I will stand behind the claim that performance should not be the reason you choose to make your web application with WASM.

> performance should not be the reason you choose to make your web application with WASM

This I can mostly agree with, but it's a slightly different take than "JS is faster than WASM" though ;)

The marshalling for 'non-trivial types' (like strings or data structures) can indeed be a performance problem, but quite often one can get away with just passing numbers between the JS and WASM side. I'm actually a bit disappointed that WebGPU turned out so 'object-oriented' (e.g. most WebGPU objects are actual Javascript objects instead of 'number-handles'), which means that a lookup-table needs to be maintained on the JS side (I think that's what the externref proposal is supposed to solve).

> JavaScript will typically have better performance than WASM.

That's a very - erm - interesting opinion which I haven't seen elsewhere yet ;)

IME, asm.js can achieve similar performance as WASM, but not 'idiomatic' JS.

Calls from WASM into JS are fast nowadays, but they are still an optimization barrier, doing this frequently (e.g. hundreds-of-thousands times per frame) is neither recommended, nor needed (and that's about the only situation where I can imagine that pure JS might outperform WASM - and IIRC I've seen some poorly built benchmarks which showed exactly this and which then claimed that "JS is faster than WASM").

(to be clear: JS and WASM performance is closer than most people think, but JS consistently outperforming WASM is the same sort of myth like "Java is faster than C++ because a JIT has more optimization opportunities" - that might be true in theory, but not in the real world outside some fabricated benchmarks).

Super cool! I love all multiplayer browser game efforts.

Ahh I want to share my browser 3d (tiny) mmo project!

It doesn’t work on mobile:

http://rotb.io

I totally agree with the issue with browser build sizes. It is not nice with bigger engines! I was even hesitant to use React because I wanted to keep things minimal but it is too nice. Lol

This is cool! Thanks for sharing! I love how there's already gameplay.

Thank you so much! :D

I also tried to make a browser game with WebRTC. This is a bit annoying to do IMO, but I am surprised that no really famous game leverages it (to my knowledge ? I mean I know that some games like openarena.live exists, but it's niche), even though it's essentially UDP in the browser, so it offers a lot more networking options than the games we see currently.

The dependent work to setup a signaling server and ICE/STUN/TURN NAT punch through is a big barrier to using WebRTC.

Very nice!

I'm the author of VoxLords, one of my early Voxel engines written for the web. Super fun to see/read about other projects like this!

My last webgl voxel-engine was this one: https://github.com/Lallassu/voxelengine3 (a bit more optimized than the VoxLords one) :)

> The game dev UI complaint is because game engines use geriatric 2-way mutable state management UI code that is error-prone and brittle. Web developers have solved these UI complaints and built tools like React that make creating UIs easy

Borderline satire.

Probably React is easy for someone who already knows it. Is it "simple"? No, not by many measures. Just like game dev UI is easy for someone who is used to it. Is it "simple" though? Nope, also not simple.

UI, simple as it seems to some and complex to others, seems to still be a problem that there is no simple solutions to, they all carry a lot of accidental complexity, web UIs or not.

I don’t know if that comment was mainly regarding web-based game UI libraries, but it seems on many other platforms there are many stateless immediate mode UI libraries available. Many based on (or inspired by) Dear ImGui [0].

By the way, I tried to use an immediate mode game UI in the past (Lua-based, SUIT), but somehow I feel more comfortable working with object-based UIs. Perhaps years of object-oriented programming has damaged my brain a bit :)

I think in the past the object-oriented was seen as a big improvement regarding GUI dev, but I guess with regards to game dev, it can be problematic wrt performance.

———

[0]: https://github.com/ocornut/imgui

It was an interesting read. I don’t really have anything to add, but wanted to kick off a pedantry argument here.

> I wanted to recreate a canceled MMO called Exteel in an .io style.

I think it should be “a .io style,” since the next letter pronounced is d.

Pedantry activated: in my game dev circles, I've only ever heard .io pronounced "dot i-o" when talking about itch.io. The style, however, I usually hear as "i-o style" or an "i-o game".

Interesting -- if I see "open a .jpg file" I never hear people pronounce the dot, it's "open a jay-peg file". You'd definitely say "open an .exe file", again because no "dot".

On the other hand, "Microsoft .NET" includes the "dot", as well as "bought five .com domains". Yet in contrast, "bought five .edu domains" would usually be said without the dot.

Hypothesis: we don't pronounce leading dots unless for some reason they've become part of well-known "branding", like .NET or .com.

I wonder if it just became “itch dot io” because itchdottyo sounds kinda cool, while itchio sounds more like “itchy, yo.”

It was funny when Sun proudly and unilaterally proclaimed that Sun put the "dot" into "dot com", leaving it wide open for Microsoft to slyly counter that oh yeah, well Microsoft put the "COM" into "dot com" -- i.e. ActiveX, IE, MSJVM, IIS, OLE, Visual Basic, Excel, Word, etc!

And then IBM mocked "When they put the dot into dot-com, they forgot how they were going to connect the dots," after sassily rolling out Eclipse just to cast a dark shadow on Java. Badoom psssh!

https://www.itbusiness.ca/news/ibm-brings-on-demand-computin...

Every gamedev's dream, to make their MMO, then they start to make an MMO and they realize it was a mistake.. :p

Too Real.

They probably give up when they realize they have to make a MMO, because the first M comes from Massive (a consonant sound).

Or don't ever realize it was a mistake...ahem.

Then it's like Alice in Wonderland.

What a great journey man! Open sourced your hard work will definitely save a ton of times for later curious developers, kudos!

Of my browser games, I've only ever finished 2D stuff. I've tried 3D a handful of times, different engines (or rolling my own) and backends. The javascript ecosystem and entropy as you said is a big stumbling block. It just moves too fast for the levels of dependencies needed when you make progress at the speed of a hobbyist.

This write up is great. Frankly I’m impressed that you were motivated to develop this over such a long period of time. Where did the motivation come from?

Also, I’m impressed that you can remember the obstacles you overcame several years ago. Landon entropy, m1 conversion, etc.

> Where did the motivation come from?

I host a bay area indie gamedev coworking group. The coworking group's regular recurring meetup interval kept me in a peer pressure and positive feedback loop. I also felt the need in the market had not been addressed until this new wave of engines started appearing.

> I’m impressed that you can remember the obstacles you overcame several years ago.

I kept a daily journal in org-mode and separate dev log file. Can search through this and my commit history for lots of info.

Makes sense, thanks for the feedback

Impressive! Having already tried the simpler experience (development of a 2D web-browser game with multiplayer), things can get complicated very quickly and the way the project was carried out shows the quality of the work.

What an awesome project. How well does it perform in the browser, though? I've never worked with them but I've always heard of voxels as kind of being a nightmare.

Enjoyed reading this, I came across your game engine when I found react-regl so I was psyched to see a retrospective.

Awesome work!

Thanks!

this looks great! I am also building one (https://joinplayroom.com) for casual 2d/party games. It's still a WIP but there is overlap.

This is awesome! I too would like to one day build a 3D game myself.

works for me with node 16 on osx brave/chrome

thanks for this :D I am interested in game engines.

Reply on news.ycombinator.com