Hacker News Reader

Comment by henning | Hacker News Reader

They built a large codebase on a language that doesn't let you control memory, because that makes you "more productive". So just having Rails allocate a per-request arena that is asynchronously freed which would force the programmer not to have any objects that outlive the request, or just pre-allocating memory for a fixed amount of request handling per server instance, or whatever allocation behavior you want to do that is generally possible in C/C++/Zig/Rust/Odin/etc, requires hacking on the language itself. Which means your changes have to go through the Ruby team first. Any additional changes would also need to go through them, which increases the cost of change. Then there is a permanent layer of indirection between your GC callbacks and the semantics of what those callbacks do. Instead of just writing out the custom allocators you want, because that's impossible. How depressing.

View on HN

> Which means your changes have to go through the Ruby team first. Any additional changes would also need to go through them …

I do want to pick on this specifically - people can and should be patching open source projects they depend on and deploying them to production (exactly as described in the article). Something being in the language vs in “user” code should be no barrier to improving it.

There's a pretty huge difference between implementing a performance optimization that works in your use case, and upstreaming that optimization to be generally usable.

The latter is often orders of magnitude more work, and the existing solution is probably chosen to be well suited in general.

Patching a dependency comes with significant downstream costs. You need to carry the patch forward to new upstream versions . This implies remembering that the dependency was patched, extracting a patch from the existing changed code, and reapplying the patch, fixing comflicts, recompiling the now special version of that dependency and running tests, checking/updating required license notices accordingly.

This is in essence another form of technical dept.

I'm guessing that Zig, Rust, Oden, and "etc." didn't exist when they started the codebase. Now they need to keep moving in their imperfect state. I don't think anyone would start a large company on Ruby today. (They would on Python, though, which is equally unfortunate.)

Startups are not large companies in the beginning.

Although I'm not sure what the preferred language for quickly getting a startup up and running would be these days.

Whatever works and you can find enough developers for.

The language (or the rest of the stack even) is rarely a barrier to success. What matters are a good idea, good motivation, and decent availability of competence.

JMHO.

I don't see how it is imperfect.

Per request arenas sound super cool on paper, and work very well on system with clear constraints. But if suddenly a request start allocating more than the arena can accommodate you're in a bit of a pickle. They're absolutely not a panacea.

Setting aside the challenge of refactoring the Ruby VM to allow this sort of arenas, they'd be a terrible fit for Shopify's monolith.

Ultimately, while it's a bit counter intuitive, GCs can perform extremely well in term of throughput. Ruby's GC isn't quite there yet, but still perform quite well and is improving every versions.

> allocating more than the arena can accommodate

In Zig, at least, this isn't how arenas work. They're a wrapper around a backing allocator, so if the arena runs out of memory, then that means the process is out of memory, something no allocation strategy can fix (ignoring the fact that Zig returns a specific error when that happens, and maybe you can trigger some cache eviction or something like that).

It's easy to set them to retain a 'reasonable' allocated capacity when they get reset, for whatever value of reasonable, so big allocation spikes get actually freed, but normal use just moves a pointer back and reuses that memory.

I don't see Shopify harvesting a lot of value from a complete Zig rewrite, no. But arenas are basically ideal for the sort of memory use which web servers typically exhibit.

And when the default arena size is often outgrown, you'll known from whatever diagnostics/logging/dashboard solution you are using. Which is incidentally also a great tool when optimizing per-request memory usage.

Being explicit about memory has many advantages, and is a strict requirement when scaling.

> something no allocation strategy can fix

Well, yes, with a GC when your heap is full, you make space by getting rid of the garbage.

Also, with a good GC, allocating is most of the time just bumping a pointer, exactly like an arena, and the collection time is proportional to the number of live objects, which when triggered out of band is basically 0.

Hence why I think a well tuned GC really isn't that far off.

I really like Zig, and I’ve been thinking about using it for developing servers offering APIs over HTTP, using arenas bound to request lifetime. I think I would be comfortable developing like this myself. But all the devs in my org have only ever used managed-memory languages such as Java, C#, Python or JavaScript, which makes me hesitant, as I’m wondering about the learning curve, and of course the risk of use-after-free. Not something I would do anyway before Zig reaches 1.0.

I wrote substantial amounts of C, and Pascal/Delphi before that, before learning Zig, so you and I wouldn't see the same learning curve. That said, I found it straightforward to take up. Andrew Kelly places a great emphasis on simplicity in the sense Rich Hickey uses the term, so Zig has a small collection of complete solutions which compose well.

Now is a great time to pick up the language, but I would say that production is not the right place to do that for a programmer learning memory management for the first time. Right now we're late in the release cycle, so I'd download a nightly rather than use 0.13, if you wanted to try it out. Advent of Code is coming up, so that's an option.

Using a memory-managed language means you need to design a memory policy for the code. Zig's GeneralPurposeAllocator will catch use after free and double free in debug mode, but that can only create confidence in memory handling code if and when you can be sure that there aren't latent bugs waiting to trigger in production.

Arenas help with that a lot, because they reduce N allocations and frees to 1, for any given set of allocations. But one still has to make sure that the lifetime of allocations within the arena doesn't outlast the round, and you can only get that by design in Zig, lifetimes and ownership aren't part of the type system like they are in Rust. In practice, or I should say with practice, this is readily achievable.

At current levels of language maturity, small teams of experienced Zig developers can and do put servers into production with good results. But it's probably not time for larger teams to learn as they go and try the same thing.

I started programming in Pascal, C and C++, so personally I’m fine with manual memory management, especially with a language like Zig. I actually find it quite refreshing. I’m just wondering if it’s possible to “scale” this approach to a team of developers who may not have that past experience (having only worked with GCed languages) without ending in a code base littered with use-after-free errors.

I don't think using Zig over Python is gonna have the biggest impact in making your next big company successful. It's a drop in the ocean compared to the quality of people you have to actually design and build it.

It doesn't matter for a startup trying to get acquired.

It does matter for a company trying to scale its user base while keeping costs down.

That is nonsense. If you can run your code on 10 servers instead of 1k servers, that is an insane time and money saver that could make or break a company.

For most startups the difference between Rust and Ruby is not 10 servers vs 1000 but rather between using 0.1% of a CPU or 1% of a CPU. A single server running Rails will easily scale to hundreds of thousands of daily users. Most companies never get that many users in the first place, and those that do will have the funds to afford rewriting the hottest paths in a more performant language.

I certainly don't claim to be an expert, but I have a hunch that getting to the point where performance becomes a significant factor (in the success or failure of a product) isn't going to be about the choice of language. I also think you're vastly underestimating the performance that good architects can get out of ANY (primary) language through good system design. Good design vs bad design makes the biggest difference in my experience, at least from a technical standpoint. Probably just nonsense though as you say.

Reply on news.ycombinator.com