Hacker News Reader

> it's pretty absurd to only have 1GB for a web application in 2016

While I don't for a minute doubt what you say, as an old guy, the absurdity of 1GB being insufficient for serving up web pages hits me pretty hard.

Our main proxies (that do the majority of the heavy lifting over here) run nginx and use, what, 50MB tops? If even? I throw them on $5/mo VPS instances and they just blast out IO without any problems.

But nginx was carefully written in C over the course of many years, with a general goal of hauling IO and static data around from other systems. Which is all you really need for web pages, in the end. Solved one thing and holy crap they did it well.

It's the business logic side of the web development world where things get nasty. The tradeoff is you get a giant chest of libraries for solving any stupid problem you want. I didn't have to for example figure out how to do IDN encoding for domain validation. You throw enough of that crap together in a single process and things start to get nasty. But how long would I have spent carefully writing all of it in C? Let's just say I'd be out looking for another job.

Worth the tradeoff, IMHO, but I think with projects like Crystal (https://crystal-lang.org) in the future we will discover we can get pretty close to both in the end. That combined with Moore's law and I think we'll be good to go.

Assuming, of course, that cloud providers don't artificially restrict available RAM through economic constraint (know the true costs - demand better!).

I realised (thanks to the other poster) that my comment may have seemed dismissive, which was not my intent. Just marvelling at it myself :-) But I do wonder about this kind of stuff. I have been wrestling with some legacy systems chewing up gobs and gobs of memory lately. Then wrestling with my own code that was chewing up gobs and gobs of memory :-)

It's pretty easy to make mistakes because we don't really get punished, as you say. But I wonder if we might be doing ourselves a disservice by not restricting ourselves. Generally speaking, better code is better and while it often feels slower to write, my experience has been that it doesn't always work out that way.

I might try working in a memory restricted container to see if it improves my code...

Another thing to look into is to write things in Rust. It will be as fast as C code, but a little nicer to work with. Right now the problem with that is that you don't have the nice libraries for whatever you're doing.

But if people actually wrote fast Rust libraries for everything, they're as easy to embed into your apps as C.

But one thing that people keep forgetting is that even 50 MB would be a dream to any user on the old Xerox PARC workstations.

So it is mostly an issue of sloppy coding and bad GC implementations than anything else.

If you don't want to wait for Crystal to mature there's a long line of high-level languages with similar performance characteristics - check out OCaml or Haskell, or perhaps F# or Scala (if it's throughput you need rather than fast startup).

Interpreted languages can have pretty high memory overhead. Let's say we have a page in python or php or ruby or whatever, and it needs to fetch 1,000 rows from a database (100 things here, 20 things there, whatever). Each row in the database can be turned into an object several times, one from the model, one from the controller, one for the template/view. On top of that, each object is a composite key/value object of let's make up 10 fields, so that's 20 PyObjects or whatever. In Python an empty PyDict uses up 280 bytes. sys.getsizeof({}). so 1,000321*280 = 17.64MB. That's just a back of the envelope theoretical minimum, include bad coding (for another few multiples), pages that require more data (analytics/stats/reporting pages), and concurrency and you can get up there!

Needing to fetch 1k rows from a database is different from needing to keep all that data in memory, however. Several years ago I had to return 1M rows from a database to the client with a PHP script, and it needed less than the 16MB (yes, sixteen megabytes) that the environment of the time was configured with as a memory limit. 1G rows would not have consumed any more memory either, because at any one time the script never needed to keep more than around a dozen cells in memory, each not more than a few KB. Read a cell from the database, process it, output the result and move onto the next one. Essentially O(1) space regardless of how much data there is.

Or are you saying that web applications are now written to retain all that data in memory despite the fact that it's not actually needed, and such stupid algorithms are actually considered acceptable? I think that's pretty ridiculous. The whole idea of database concepts like cursors is that you shouldn't be required to hold the entire resultset in memory, because it might not fit.

This should be even easier these days. Async I/O platforms like Node.js mean you don't have to worry about a process sticking around while it trickles data to a client. And web frameworks make it easier to deliver datasets asynchronously as JSON to the browser. Generally, there's less excuse to buffer everything server-side, e.g. to generate a full page.

I think the reliance on large frameworks and external libraries is the problem here because they don't generally compose in a way that permits optimizing data flow. That and the usual reduction in average skill required to work in this area.[1] People will spend an eternity arguing over which is the faster hash table algorithm, but have no eye or concern for optimizing the large details even though the relative improvements in performance and scale could be orders of magnitude better and conceptually much easier to implement.

This is why higher-level languages need stackful coroutines, not just promises or async/await. Stackful coroutines make it much easier to implement and compose space-efficient iterators, for example, in the context of uncooperative libraries, and to do so performantly. People are so obsessed with async I/O they've forgotten why it's so darned useful and how you want to best leverage it. They keep going down the rabbit hole of callbacks and actors, but the fact of the matter is that those things are too complex to use pervasively in the pipeline of individual requests. They're the modern equivalent of gotos--easy to use and abuse, but not what you want to use as an abstraction for tying together different pieces of code because they have poor locality of context, require leaking too much detail at the interface boundaries, are more difficult to refactor, and generally require a high cognitive burden. We want to be lowering cognitive load and other constraints!

[1] To be fair, I was one of those less-skilled newcomers back in the late 90s, abusing the newer technologies and frustrating the grey beards.

> Or are you saying that web applications are now written to retain all that data in memory despite the fact that it's not actually needed

Even the standard PHP MySQL API (mysqli) defaults to MYSQLI_STORE_RESULT which buffers all the results in memory. You have to specify MYSQLI_USE_RESULT to get streaming results.

And what do you do if an error happen in the middle of your iteration while the server already answered with 200 OK and sent half of the response?

Your only option in those case with PHP is to truncate the response which IMO is the ridiculous choice here...

It's about return on time invested. As an IoT company we got super excited after we decided to bump our chip from one that has 64kB instead of 32kB. However, I wouldn't blink an eye if we would need a few more GB for our cloud servers. Of course if it's just a static website I run it of Github pages.

For static sites, surely just load all of the content into memory and serve it based on key/index? eg. load into a C++ map of <const string,const string>, where first is URL and second is content. Even lookups would be fast (and would negate the need to hit the disk too).

For dynamic stuff, adjust accordingly.

You don't even have to bother loading it in memory. Just use `sendfile(2)` and let the kernel do all the work for you.

You can't really do faster than that.

My favorite Ruby memory leak was in a text processing app that was extracting via regex short strings (< 50 chars) out of 10MB+ text files. The short strings were apparently substring-like objects that held on the original 10MB text and indexed into it at a particular offset. Solution? Return the short_string + "" and then the GC did its thing.

I work on the project discussed in the article. We're running puma in clustered mode, with multiple workers per dyno and multiple threads per worker. 1GB is plenty in this context - each worker is pretty lightweight. At the time this was written we were running ~10 2x dynos, but we've since switched to 3 performance-m dynos with more puma threads/workers.

> I don't think it's any less crazy than being forced to chase a GC white whale for two weeks on a tiny memory leak to avoid a huge rate hike on your hosting bill.

The main issue we were facing wasn't the hosting bill, but high (for us) traffic, that was leaking memory on every request. Under low traffic it was unnoticeable, but at peak load the leak would cause dyno memory to max out pretty quickly, which would cause timeouts and increase the traffic to other dynos, causing cascading failures. Having more memory available would definitely have made things a lot easier, but we would have run out eventually either way.

1GB is an obscene amount of ram for a webapp, it's just that Ruby is a memory hog. You can run an Elixir webapp with much much less memory.

See: http://www.littlelines.com/blog/2014/07/08/elixir-vs-ruby-sh...

tl:dr;

Ruby/Rails: 235.37MB

Elixir/Phoenix: 34.69MB

    > I did want to chime in quickly and say that
    > it's pretty absurd to only have 1GB for a
    > web application in 2016

For Rails and Java apps, I can see that.

But if the 512MB dyno isn't enough for any of the stacks I use, including Node, then I'm doing something very wrong (or weird).

>Interesting post, but I did want to chime in quickly and say that it's pretty absurd to only have 1GB for a web application in 2016

That really depends on a lot of things. For a simple web app with a couple of users, 1GB should be overkill.

Out of curiosity, albiet slightly off topic can you share an example server that you'd buy? I'm totally in the aws kool aid soup and would love to see some reasonable examples of how to buy server alternatives

> it's pretty absurd to only have 1GB for a web application in 2016, even for a small one

what is pretty absurd is the amount of memory a ruby web application uses. 1GB should be enough in 2016 to serve a small web app. Ruby might be a pleasant language to write it was never designed to be efficient memory wise. And frameworks like Rails that have 0 concerns for real performances make things even worse. That's why people are ditching Ruby for statically typed alternatives, constantly. It's just not a good language for the task in 2016.