Hacker News Reader

Those are some very good points, but I think some of them are wrong or do not apply for most uses cases.

1) "Do not serialize JSON, or use msgpack or protobuf - write your own protocol." - Why should you write your own protocol insead of protobuf? If your game has more than 10-20 types of messages/objects than it becomes very hard to implement your own binary encoding for each object. Plus with protobuf is easy to use and the packets generated are really smaller (most likely smaller than your own protocol).

2) "Measure everything with realistic load - i.e. I've found out that it's more costly to calculate world entity deltas to send to client, than to send everything in his view." - Profiling is something that you should always do, with both synthetic and real loads. In your case, my guess is that your diffing code was inefficient or that the load was so small that both computing the delta and sending the message were almost instant, so comparing them would give inaccurate results.

3) "Prefer fixed entity speeds." - I think this is a game design decision and you shouldn't limit your game to this if you don't have too. Also, I don't think there's any issue with entities being able to have a different speed each tick.

And, a question. What VPS are you using? Do you recommend any specific VPS for hosting Node.js game servers?

1) Demonstrably not so, in my case. Devising and implementing a protocol flexible enough for me was, along with test-driven implementation of that particular aspect, a joy and more performant solution than any other out of the box. Yes, I've tested against msgpack/protobuf (and I use protobuf in production on other projects) and found it to be less optimal, for both implementation (you're never as flexible as with blank board) and runtime resource usage, across board.

2) Sure, I might go the way of the delta one of these days - it does still smell like good design. That's why I was underwhelmed to see it underperform first time around.

3) Yeah, design and implementation are never separate. Neither is content and presentation. :)

DigitalOcean works for me now. Wouldn't mind slightly better CPUs, as even High Compute droplets are not all they're cracked to be.

Just curious about your own binary protocol. If your object has an int32 value, but the value could actually be represented with only 12 bits, do you send the entire int value (32 bits) or only the 12 bits needed to reconstruct the number?

I also used DO for a low-tickrate game (2ticks/second), but it doesn't seem like the servers are able to handle multiple game servers running at 60FPS.

If you were to implement the game again, would you rethink your networking so you don't have 2TB of data transfer for only 200 players?

I pick smallest needed value type, and some values encode multiple fields (i.e. type/state/frame are all in uint8). Node.js uses cast to unsigned 32-bit integers for bitwise operations on numbers, so bundling doesn't pay out as much in CPU as does in bandwidth savings.

2TB of data is really not that much. Say you run at 30fps, streaming only 2D position (say, 2 x uint32) of 100 entities to 200 clients...

(64 * 100 * 200 * 30 * 60 * 60 * 24 * 30 ) / 8 / 1024^4 ~= 11TB per month.

You bring some sanity to that with less frames, less entities, naive delta not sending updates for entities, etc. It's really not surprisingly big number.

I'll definitely explore world state deltas in next game, but this works good enough for now.

>>> A question to all reading - I'm considering adding an option to mine Monero in my next game. It'd probably be a flip-switch between ads and mining right there on the home screen, with some intelligent throttle/thread decisions depending on the kind of device game is running on. I'd appreciate any feedback regarding the sentiment on that.

Mining => You WILL murder the battery of your users. You WILL cause overheating of the devices. You will also grind the browser to a halt on all hardware of moderate speed or already busy for reading a youtube video in the background.

Personally. I'd consider that any website that start mining when opened should go right into an adblock list, if not blocked as malicious entirely.

This is the reason why I'm considering only enabling that feature on Chrome (because of WASM and battery API) and Desktop PC (because of the battery/heat issues).

For experiment, I've turned on a miner in one tab of my brother's 7-year old modest PC setup. He went for several hours without noticing it, while doing blender rendering as well as somewhat intensive photoshoping.

Just because the user doesn't notice or doesn't understand what happens doesn't make it acceptable.

Chrome is going toward more and more aggressive CPU throttling of tabs in the background. If mining becomes a thing, we're going to a world where background tabs will stop being rendered at all.

> Just because the user doesn't notice or doesn't understand what happens doesn't make it acceptable.

I did this on my brother, for data gathering. It'd most certainly not be implemented non-transparently in the end-product.

> Chrome is going toward more and more aggressive CPU throttling of tabs in the background.

That's why you mine during active game. The more people love it, the more they mine it. It can certainly be made mostly win-win scenario, which is why I aim on, again, transparently experimenting with it.

Is mining in a browser remotely cost efficient? Doesn't it end up costing the user more in electricity than just buying the coins directly?

There are many tradeoffs one makes taking any route. Electricity-wise? I'm not sure. We'll see.

> ...just buying the coins directly?

Realistically, there's never such a thing such as "just buying coins". You need payment processor / security / refunds / authorization / ...

There is, however, something close to "just mining coins". This is one of the fundamental reasons cryptocurrency is attractive in the first place.

Is this not already in the BTC situation, where you burn more in cost of electricity than you gain from mining? Granted, in this case it would be somebody else's electricity, but that feels a bit ethically grey, as well as perhaps environmentally irresponsible.

Don't get me wrong, I've considered similar things, but I'm pretty uncomfortable with the idea for the above reasons. Plus, whatever people might tell you, even the absolute best JavaScript engine is slow by comparison with other languages.

Example: in the last few weeks due to #reasons I've ported a measure calculation engine for market research data from JavaScript to C#. At first, in many ways, it was a like for like conversion, in that I preserved the data model and general mechanism and flow of calculations, albeit that I applied a much stronger separation of concerns. Nevertheless, with no optimisation, it was roughly 2x the performance of the JS version (V8, of course) running on the same machine. I then started to take advantage of .NET's strong typing for numbers and strings, switched out dictionaries for strongly typed arrays using (obviously) integer lookup, rather than using integers as keys in dictionaries. I also fixed a couple of issues where I was inadvertently using exceptions for flow control, and recalculating strings to generate hashcodes (doh!).

What's surprising is that those changes get me to about 15x faster, and that's really just taking advantage of strong typing and more appropriate data structures that are available in .NET. (JS has arrays, and engines like V8 do optimise for them, even though they're strictly just objects with special properties. Whereas in C, C++, C# arrays are the absolute fastest data structure to access, bar none.) Also worth pointing out that judicious switching from dictionaries to arrays absolutely slashed memory use.

I also implemented the kicker - which happens to be language agnostic - which was to change the moving average calculations to preserve and reuse intermediate results. This got me a 10x performance gain on top of the rest, taking me to about 150x. Like I say, that would probably have been the case in JS as well, so it's not really a fair comparison.

Nevertheless, bear in mind that whilst JavaScript is certainly a lot faster than it used to be, it is not fast, even by the standards of other managed runtime languages. And it's absolutely not fast by the standards of well-written C++ or C compiled down to native code.

I'm not sure how much benefit you'd get from a WebAssembly approach, although possibly something worth looking at.

Go check it out with this miner implementation I intend using and customizing. [1]

[1] https://coin-hive.com

Can you/will you run that on a web worker? No way that would work on the foreground thread if you're using that for your game. That obviously depends on how much work your game does but this, for example, would suck with anything else going on in the foreground, especially on a phone: https://arcade.ly/games/asteroids/.

I'll first do whatever's most sensible with current readily available tech and measure. Coin-hive success case [?] (or so I've read), used secondary tab, which contained details about mining data itself.

[?] I think they've removed it from their homepage, most likely because they thought it's not fitting to include it there. :/

Given their test and API documentation both refer to threads I'm guessing it must be using web workers. I might start offering it to desktop users, along with something like https://www.pollfish.com/, as ways to get rid of adverts on the site (I make sweet f.a. from ads, although I get pretty minimal levels of traffic, tbh - need to spend a lot of time on SEO, which is of course not the fun bit). I'm definitely steering clear of offering it on mobile though.

Feel free to hit me with any data you gather, if you choose to do so, of course.

I’m actually super curious to know now the total resource consumption of one video ad versus one hash computation for mining purposes (including bandwidth and battery consumption). The thing with mining is that, since it’s basically a lottery system, you can do as much or as little as you want. With sufficient scale, you could do a few computations and stop when you feel the user has “paid” for access to your page.

> Is this not already in the BTC situation, where you burn more in cost of electricity than you gain from mining? Granted, in this case it would be somebody else's electricity, but that feels a bit ethically grey, as well as perhaps environmentally irresponsible.

Probably. But the point is that it's not an alternative to mining for yourself, but an alternative to showing people ads or having people explicitly pay, which is likely to be even less efficient for very small transactions.

If you do it behind the users back, then sure, it might be ethically grey (though how many users care about how much/little electricity the game they play cause their computer to consume?), but the alternative is to clearly show a message saying you do this instead of showing them ads and/or offering them a button to switch to ads instead.

I'd suggest finding out, in the real world, if it is ads or mining that gets you the greater income. Then, you optimize for that.

In either case, when you're having them doing the mining, I'd recommend full disclosure AND I'd recommend some explanatory text that goes into the details of what you're doing and why.

Something like, "Click here to learn more about mining." That should do the trick. I'd just put said link below the quick blurb/switch/invite to use mining.

If mining generates more income, then you may wish to offer some sort of in-game bonus for those who enable it. You'll have to figure out some way to ensure they really have it enabled, of course. Giving them a reward may entice more users to choose to do mining.

Another post mentioned that this would be a battery killer. You responded that you'd only enable it for certain devices/browsers. If you want, you could probably enable it (as a choice) for all devices and browsers BUT I'd absolutely ensure that you made it very clear that doing so would mean a rapid depletion of stored energy on battery powered devices.

The reason I'd suggest you do allow it is because many people only use mobile devices and, in some cases, will operate them while they are plugged in. For example, I'm typing this on a tablet that is plugged into an outlet. If mining makes you income, I'd be inclined to want to do mining for you - even while on a mobile device. It'd be even more tempting if there were an in-game reward for it.

Sort of related: That might make an interesting funding option. Free games but mandatory mining on behalf of the creator. Obviously, full disclosure is required if one is concerned with ethics. Inside this idea, they might be able to get rewards directly related to how much they mine on the creator's behalf.

Responding to all your suggestions would take too much time for me right now, but I can tell you that all are very on-point. Thanks.

One interesting trivia though - for few undisclosed games I've checked the data, with following conclusion - they would've made order of magnitude less with mining than with ads. However, if you include currency trend for those periods, they would've made almost the same. :)

I think this might be indication of a possible close-future tipping point in favour of this model. We'll see.

>>> I'm doing similar, it's so much fun

Yes, basically this. I think most of us solve the same sort of problems day in and day out with our jobs; it's unbelievably fun to be dropped into a totally new problem space and have that feeling of complete ignorance again.

Nice list! These are pretty useful. I found the experience of using protobuf for frontend <-> Go communication to be good, since it forced me to do the work of 'writing my own protocol' but without all the encoding/decoding. There was some issue with naming in protobuf and Go's conventions though.

To add my 2 cents: I think mining Monero is actually very reasonable (you seem to have thought through the impact on your users), and you've nailed the most important thing: making it opt-in.

I'm fine contributing CPU cycles when playing a game, as long as it's in the foreground and I agreed to it.

> I have Node.js serving 200 clients and 10k entities with vision (can run at or from you) at 20 frames per second that keeps under 100MB RAM with no hiccups.

What's in the 100MB?

Doing some napkin math, take 10k hypothetical game objects that each have as state: pos, vel, acc, rot, ang vel, ang acc, name (32 chars), color, health, mana, armor, level, type, energy, kills, timers, inventory (256 bytes)... etc etc, all together comes in at around 512 bytes each. Even if that's double buffered it's still just 10MB.

Curious where all that memory is going!

game devs... do not understand the web.

well I'm not sure how it could only be 100MB considering that our Angular2/4 Web Application (a big enterprise CRM/ERP) uses 400 MB (no joke).

To be perfectly honest, I haven't profiled and have been fairly cavalier about it. Seemed good, and most importantly, stable enough - it's not bottleneck. Like you've noticed, it definitely has low-hanging fruits to pick up there, yeah.

Please tell me what your game is. Even considering abusing my hardware for mining is enough reason for me to avoid anything you create.

You say "Ignore socket.io, do pure binary websocket" and that sort of makes sense to me, along with your advice about not using JSON, but what are you using to do "pure binary websocket"?

Do you have any code samples, please? (Don't need to run or compile, just an example would be handy.)

Also, can you post a link to your game, please? I'd like to give it a try.

You can find links to both (and then some) in my comments on this submission.

Well, I've found munchies.io so I can have a look in the network tab in Chrome Dev Tools and see where the requests are coming from. Thanks. :)

I don't see any links in your comment.

> comments

There are so many msgpack and protobuff implantations on npm, does anyone have recommendations on which to use / what the tradeoffs are?

https://npmcharts.com/compare/avsc,thrift,msgpack-lite,msgpa...

> I've found out that it's more costly to calculate world entity deltas to send to client, than to send everything in his view.

If you don't mind me asking, Why is this the case? I would have assumed world entity deltas is probably how I'd approach things first...

>Write game bots to test your load non-synthetically.

Can you expand a bit more on that? Cool tips, for sure. Personally I'd appreciate a flip-switch between ads and mining, as long as it defaulted to ads.

In my particular game, for core gameplay, I'm just sending the angle and whether the action button is down or up. So I just slapped together a Node.js script to spawn N clients that, according to some simple logic send that input to server. They don't even interpret what server sends back to them, because I'm testing the server-side with them. Bonus is that you can test your algos in extreme circumstances, like send all clients to particular spot on the map, so you can see how it performs with different distributions of locations.

This was invaluable to almost all multiplayer games I did.

Wrt mining/ads, I'm considering to default to mining if I detect ad-blocker, or you're on Chrome + Desktop PC (not laptops).

> Wrt mining/ads, I'm considering to default to mining if I detect ad-blocker, or you're on Chrome + Desktop PC (not laptops).

Do what you've gotta do to get paid for your work, but if I found out this was happening it would strongly discourage me from remaining on the website.

Thank you, that's great feedback and exactly what I'm after. Nag popup after every few sessions to turn on mining or whitelist the site in adblocker would be better compromise then?

That would work for me.

Honestly this would be a great use for attention grabbing full screen notification (gasp).

"We're cool with adblock, but to continue on this site click here to enable mining, here for paid accounts, or just disable adblock and report inappropriate ads so we can forward that to our syndication."

Yeah, actually I think that'd be great place for full disclosure and help user get opportunity to make as informed decision as possible.

"how dare they consume my compute resources to make money?" I mean, what the hell is internet advertising if it's not consuming your compute resources to make money?

The amount of resources required to load a blob of json, some images and do a shitty js callback matroska dance is relatively well understood and accepted as a necessary evil by those that don't/can't block such things. Yes there are bad actors that will churn through your mobile data while your phone is locked, endlessly reloading shitty video ads, but who knows how well miners will be coded or perform.

I'm betting on hearing some horror stories fairly soon as the first gen is rolled out.

Are you sure about no JSON? My experience is JSON is fairly small, and faster than anything else because it has browser support.

For my use case, replacing JSON with custom binary protocol, some old version being here [1], had savings of about 2 orders of magnitude in bandwidth and at least 1 order of magnitude improvements in server CPU usage. Some significant memory savings too. There's also space for more optimizations to be done, but I consider it viable enough for now.

ArrayBuffer and DataView are fairly fast for my standards.

[1] https://gist.github.com/synthmeat/d23595db732d366f658a1981e9...

Not OP but yes, absolutely. JSON and MsgPack are schemaless, which means they put the schema, including object keys etc on the wire. On every transfer.

The problem is less the client, you're right with that. The problem is that server bandwidth and CPU scale up really quickly if you try to do 30fps, no matter what method you use to keep state in sync. If you want any significant amount of players in a realtime game, this will be one of your first bottlenecks, probably before any encoding and decoding.

For less realtime-y games, JSON and also socket.io can be perfectly adequate.

Protobuf is just insanely slow to encode/decode client side. So that's out too if you want fast paced games. And for slower games it is not worth the effort.

It's not for client. It's for server and bandwidth.