How I spent two weeks hunting a memory leak in Ruby (2015)
be9.io
be9.io
I get that it means you have to "run a server", and insert arguments for expensive cloud providers vs DIY servers here but I don't think it's any less crazy than being forced to chase a GC white whale for two weeks on a tiny memory leak to avoid a huge rate hike on your hosting bill.
On an aside, I've only had one problematic memory leak with ruby ever (the infamously leaky RMagick), I threw this in every time I used the lib and it solved it for me:
GC.start(full_mark: true, immediate_sweep: true)
While I don't for a minute doubt what you say, as an old guy, the absurdity of 1GB being insufficient for serving up web pages hits me pretty hard.
But nginx was carefully written in C over the course of many years, with a general goal of hauling IO and static data around from other systems. Which is all you really need for web pages, in the end. Solved one thing and holy crap they did it well.
It's the business logic side of the web development world where things get nasty. The tradeoff is you get a giant chest of libraries for solving any stupid problem you want. I didn't have to for example figure out how to do IDN encoding for domain validation. You throw enough of that crap together in a single process and things start to get nasty. But how long would I have spent carefully writing all of it in C? Let's just say I'd be out looking for another job.
Worth the tradeoff, IMHO, but I think with projects like Crystal (https://crystal-lang.org) in the future we will discover we can get pretty close to both in the end. That combined with Moore's law and I think we'll be good to go.
Assuming, of course, that cloud providers don't artificially restrict available RAM through economic constraint (know the true costs - demand better!).
It's pretty easy to make mistakes because we don't really get punished, as you say. But I wonder if we might be doing ourselves a disservice by not restricting ourselves. Generally speaking, better code is better and while it often feels slower to write, my experience has been that it doesn't always work out that way.
I might try working in a memory restricted container to see if it improves my code...
But if people actually wrote fast Rust libraries for everything, they're as easy to embed into your apps as C.
So it is mostly an issue of sloppy coding and bad GC implementations than anything else.
Or are you saying that web applications are now written to retain all that data in memory despite the fact that it's not actually needed, and such stupid algorithms are actually considered acceptable? I think that's pretty ridiculous. The whole idea of database concepts like cursors is that you shouldn't be required to hold the entire resultset in memory, because it might not fit.
I think the reliance on large frameworks and external libraries is the problem here because they don't generally compose in a way that permits optimizing data flow. That and the usual reduction in average skill required to work in this area.[1] People will spend an eternity arguing over which is the faster hash table algorithm, but have no eye or concern for optimizing the large details even though the relative improvements in performance and scale could be orders of magnitude better and conceptually much easier to implement.
This is why higher-level languages need stackful coroutines, not just promises or async/await. Stackful coroutines make it much easier to implement and compose space-efficient iterators, for example, in the context of uncooperative libraries, and to do so performantly. People are so obsessed with async I/O they've forgotten why it's so darned useful and how you want to best leverage it. They keep going down the rabbit hole of callbacks and actors, but the fact of the matter is that those things are too complex to use pervasively in the pipeline of individual requests. They're the modern equivalent of gotos--easy to use and abuse, but not what you want to use as an abstraction for tying together different pieces of code because they have poor locality of context, require leaking too much detail at the interface boundaries, are more difficult to refactor, and generally require a high cognitive burden. We want to be lowering cognitive load and other constraints!
[1] To be fair, I was one of those less-skilled newcomers back in the late 90s, abusing the newer technologies and frustrating the grey beards.
Even the standard PHP MySQL API (mysqli) defaults to MYSQLI_STORE_RESULT which buffers all the results in memory. You have to specify MYSQLI_USE_RESULT to get streaming results.
Your only option in those case with PHP is to truncate the response which IMO is the ridiculous choice here...
For dynamic stuff, adjust accordingly.
You can't really do faster than that.
> I don't think it's any less crazy than being forced to chase a GC white whale for two weeks on a tiny memory leak to avoid a huge rate hike on your hosting bill.
The main issue we were facing wasn't the hosting bill, but high (for us) traffic, that was leaking memory on every request. Under low traffic it was unnoticeable, but at peak load the leak would cause dyno memory to max out pretty quickly, which would cause timeouts and increase the traffic to other dynos, causing cascading failures. Having more memory available would definitely have made things a lot easier, but we would have run out eventually either way.
See: http://www.littlelines.com/blog/2014/07/08/elixir-vs-ruby-sh...
tl:dr;
Ruby/Rails: 235.37MB
Elixir/Phoenix: 34.69MB
> I did want to chime in quickly and say that
> it's pretty absurd to only have 1GB for a
> web application in 2016
But if the 512MB dyno isn't enough for any of the stacks I use, including Node, then I'm doing something very wrong (or weird).
That really depends on a lot of things. For a simple web app with a couple of users, 1GB should be overkill.
what is pretty absurd is the amount of memory a ruby web application uses. 1GB should be enough in 2016 to serve a small web app. Ruby might be a pleasant language to write it was never designed to be efficient memory wise. And frameworks like Rails that have 0 concerns for real performances make things even worse. That's why people are ditching Ruby for statically typed alternatives, constantly. It's just not a good language for the task in 2016.
- return Data_Wrap_Struct(klass, rb_redcarpet_rbase_mark, NULL, rndr);
+ return Data_Wrap_Struct(klass, rb_redcarpet_rbase_mark, xfree, rndr);I would have been even more explicit and added a cast to the expected structure pointer type. Entirely useless, except to the human reader.
I'd say use a separate free function when it needs one, which is not the case yet (and might never be.)
I don't mind adding 1 useless function per complex type if it saves me those headaches even a small minority of the time, or for the next maintainer. Opinions may differ, but that's me.
[1] http://clalance.blogspot.com/2011/01/writing-ruby-extensions... [2] http://inferior-products.com/docs/userdocs/ruby19/html/d8/d1...
Publib then? I'm finding this when I search: http://man.cx/xfree(3) http://man.cx/publib(3)
Also, are you sure? I see xmalloc in glibc all over the internet...
#define xmalloc ruby_xmalloc #define xfree ruby_xfree
I had a similar issue that I was tracing last week that did end up being my Ruby code...and it turns out I was modifying a constant like in the example.
What a fun read! I've been learning more about Ruby's GC since 2.1 and this got me looking even deeper -- definitely picked up a couple of new tricks/tools from this. Thank you be9!
ENUM = [0, 1, 2].freeze
LOOKUP = {
:A => [1, 2, 3].freeze,
:B => [4, 5, 6].freeze
}.freeze
# or:
LOOKUP = {
:A => [1, 2, 3],
:B => [4, 5, 6]
}.tap { |s| s.values.each(&:freeze) }.freezeGreat insight, though. Author described this experience as it was a great venture... in hindsight I suppose :-)
Necessity is the mother of invention, and when you're the first person to notice a memory leak, you're on your own.
Both ecosystems tout the low barrier to entry as a great benefit, but memory leaks, mostly benign inefficiencies, and poor algorithmic efficiency becomes at least par for the course if not a crippling liability when reality hits your application like a freight train (or say a 2 order-of-magnitude spike).
Stick with pure-language extensions (they're so much easier to work with!) until you can measure and quantify the cost and benefit of switching.
The leak was in a gem.
2. With over 10M downloads, I assume at least some of them spend large percentages of time using it.
3. It's Markdown; replicating the idiosyncrasies of SmartyPants (https://daringfireball.net/projects/smartypants/) in a different language is not something many people are going to want to do.
I'm not saying that people should or should not use Redcarpet; just that it's unlikely that anyone engaged in "premature optimization" here.
This experience made me question myself: is it wrong to create a lot of objects in memory and let them reference each other, and hope Garbage Collector to magically work?
If instead of object, we just put data into memory more organized as a (column) database, then we don't need GC anymore.
OOP made programming easier, by modeling real world as object. At the same time, it made memory management harder, since it's not nature to the hardware's memory model: a linear array of memory cells.
I wish in future, there is a place for this paradigm of in memory database model of programming.
The English grammar is mostly just simplified German, but there are some constructs, which I know of, but do not naturally use (e.g. "your every …", "to name but two") - they somehow seem twisted to me. Another one are compound words, I have given up on distinguishing between spaces, hypens and actual compounds and just use spaces almost everywhere.
I tried to add ASan to the travis config for this project but I couldn't quite figure out how to change CFLAGS and/or CC. Never used ruby but interwebs hinted that the bundle config/install commands might accept "--cc" and "--with-cflags" commands. It's ignored when I tried it though [1].
[1] https://travis-ci.org/androm3da/redcarpet/jobs/159711520
To test all of the ones we use individually is prohibitive. But thousands of people could do test one.
"Update on 2015/09/29. Redcarpet fix has been released."
Love it when reading HN pays off immediately like that :)
Ideally much less tooling would have been needed.
But no, a PHP extension in C can have a memory leak like any other C code. It's not at all uncommon actually, since there are fewer consequences; CGI mode PHP cleans up after every request, FastCGI mode PHP by default restarts workers after a number of requests because of the class of effects (unwanted state) that includes memory leaks.
However, you can still create memory leaks just like you could in Ruby - dynamic allocation in a c extension that isn't freed up (like in this story), or creating objects that can't be garbage collected (like a slowing expanding array in a global variable).