Ruby Garbage Collection: Still Not Ready for Production

From the article, the issue is that the Ruby GC is triggered on total number of objects, and not total amount of used memory.

The article claims this got worse with the new generational GC algorithm, which sought to minimize the amount of work the GC has to do during the collection pause. By marking long lived objects as probably OK, you end up with fewer objects to free per GC cycle.

The problem then, is if you allocate some massive strings, and they get marked as "old" then they might never get collected by the GC. It's not clear to me if the article author sees it this way, but apparently there is some kind of bugs where some class of objects get marked "old" by accident, and it sounds like it's exacerbated by existing interpreter architecture.

Work on the Ruby interpreter is weirdly silo'ed off and mostly done by Japanese developers, so there's a significant barrier to entry for any enterprising C developer to roll her sleeves up and get hacking.

Anyhow, so, my understanding of your question is no, it would not help outside of really specific optimizations? It depends on what this Buffer class would do. Would they let you mark them as being young? Or avoid extra malloc calls because you have special information on the size of your strings? Kinda depends on how the generational algorithm was implemented? At this stage my knowledge grows thin.

There's nothing really stopping you from doing this with a cext or just using a String with all its encodings set to BINARY. I don't really see why it would be helpful here, though. The Ruby VM is essentially abdicating the difficulties of managing a large block heap to the platform's implementation, this wouldn't really change with a buffer class that could be similarly large.

The project appears to be called gctools: https://github.com/tmm1/gctools

And the known bug is probably "Allow dry-running the OOBGC": https://github.com/tmm1/gctools/pull/5

Without knowing anything about the project, it seems like the bug is particular to Passenger's use case.

Yes, exactly. You're changing the slope of the growth, but the growth still happens.

Being multi-process is the "waste". Each process has its own heap, its own copy of the AST and bytecode, etc. Even with preforking and copy-on-write, the overhead introduced by processes is still significantly bigger than running many threads within 1 process.

Of course, this doesn't have to be a problem for everyone. Whether threads are advantageous depends a lot on the workload.

I think Passenger only offers multi-threading in the enterprise edition.

> There should be no need for "memory doubling" to run this code -- you're generating throwaway strings of identical size.

How do you know they are throwaway?

Your code involves two method calls. While it would be unlikely that someone has overridden them given that they are core String and Fixnum methods, it is perfectly possible. E.g:

  class String;

    alias :old :*

    def * right
      $store ||= []
      $store << self
      old(right)
    end
  end


  "foo" * 5
  "bar" * 3

  p $store

In other words, even with seemingly innocent calls like that, it takes extra work to be able to reuse that memory without a full GC pass. It's certainly not impossible, but it's not there yet.

So I agree with you in principle, but this is one of those areas where the malleability of Ruby objects makes it tricky to optimize.

(one possible example approach is to set aside a bit to indicate "has at least once been stored somewhere where it can escape" as "poor mans escape analysis" - if your object is only ever stored in local variables that have no been captured by a lambda, or passed as arguments, then it can't escape higher than where it was created, and so you can take shortcuts, otherwise you'd still need a full gc pass)

That sir, is a bug that should be reported on bugs.ruby-lang

   while true do 
    "a" * (1024 ** 2)
   end

it is to do with the dynamic malloc limit growing too much, should be fairly trivial to patch in the next 2.1 release

I admire your recent work with threads in Ruby with the Puma libraries and the threaded in memory queue (I was working on something similar myself, I keep forgetting to get back to it though). Most companies I see using Heroku are always wondering if their bill could be lower and most of the time they are using preforking servers and background job processors.

I look forward to your talk at Ancient City Ruby next week!

Any idea why the docs[1] recommend using Unicorn instead?

On Cedar, we recommend Unicorn as the webserver. Regardless of the webserver you choose, production apps should always specify the webserver explicitly in the Procfile.

[1]: https://devcenter.heroku.com/articles/ruby-support#rails-4-x...

Wow, this is awesome. You should add a link to the Puma guide [1] as I've been wanting to make use of Puma workers for a while but have always been worried about running out of memory.

[1] https://devcenter.heroku.com/articles/deploying-rails-applic...

> the default Heroku server

There is no "default Heroku server" you get exactly what you specify. If you specify nothing you get what Rails runs by default which is webrick and happens to be awful in production. It's not that "Heroku has a bad default" here but rather Rails' default web server is not intended to run in Production. Most of our docs try to emphasize that you should be using another server other than webrick. Some people want to just bang out a proof of concept and not worry about configuring Puma or Unicorn, for them webrick is fine.

There is no 'default' Heroku server on Cedar, it runs what your app asks to run. If that's Thin you'll be running on Thin. However, Heroku recommend Unicorn.

These benchmarks are uselessly bad:

- The client connections are run without concurrency, serially, on the same machine: https://github.com/brianhempel/isrubyfastyet/blob/master/run...

- The JIT is only given 3 seconds to warm up: https://github.com/brianhempel/isrubyfastyet/blob/master/run... -- HotSpot kicks ass, but not after only 3 seconds of serially issued requests.

- By running the tests single-threaded, they're throwing away one of the major wins of the JVM -- low overhead concurrency.

I think this would be a better benchmark than that one as it runs a more apples-apples comparison:

http://www.techempower.com/benchmarks/#section=data-r8&hw=i7...

> mutable state on the ApplicationController and the models

Well, mutable state on the models seems pretty fair, unless I'm misunderstanding you. It's rather unforgivable on ApplicationController though.

Edit: and the slow startup time doing TDD with JRuby was my primary complaint when I developed in it. Although iirc you can use nailgun or something to keep the JVM active and mitigate that pain.

Is the JRuby environment sufficiently different that you can't TDD against MRI and then save JRuby for final validation testing and deployment? They shouldn't be that different, unless you're not writing pure ruby code.

Alternatively, Nailgun should reduce a lot of that pain. Effectively you have a running server instance of JVM and run some clients on it. That gives you JVM speed advantages without the slow startup. JRuby natively supports it: https://github.com/jruby/jruby/wiki/JRubyWithNailgun

Reference counting is also the reason why Python requires a GIL. Incrementing a reference count is an opportunity for a race condition (another thread could read the refcount in between your read and write), which means that every reference count needs to be protected by a lock. Either you do fine-grained locking for each object, or you add a GIL for the whole interpreter. The former will absolutely kill performance (not only do you need to increment a refcount with every assignment or function call, you need to take a lock). The latter makes the whole interpreter thread-hostile.

Python also has weakrefs to avoid creating cycles

http://docs.python.org/3.4/library/weakref.html

> The downside is that you can leak memory if you have circular references. The programmer needs to take care to not let this happen.

Not entirely true[1]:

> CPython currently uses a reference-counting scheme with (optional) delayed detection of cyclically linked garbage, which collects most objects as soon as they become unreachable, but is not guaranteed to collect garbage containing circular references.

The "most object" and "not guaranteed" refer to[2]:

> Objects that have __del__() methods and are part of a reference cycle cause the entire reference cycle to be uncollectable, including objects not necessarily in the cycle but reachable only from it. Python doesn’t collect such cycles automatically because, in general, it isn’t possible for Python to guess a safe order in which to run the __del__() methods. If you know a safe order, you can force the issue by examining the garbage list, and explicitly breaking cycles due to your objects within the list.

Note that this is Python 2 thing; in Python 3, the behavior has changed, and cycles with __del__ methods get collected. I include it because many people still use Python 2. As for Python 2, however, I believe this case is rare, and requires both an object to be involved in a cycle (a bit rare) and have a __del__ method (I've never written such a method in many years of Python).

> Objective C offers weak pointers

Python also offers weak pointers.

> Finally, I believe it has been shown that reference counting can be slower that a well-implemented garbage collector

My understanding is that this is true, but workload dependent. I think this mostly comes from being able to quickly allocate memory, because your heap is just allocating by moving a pointer the required number of bytes through the heap. (Allocation is just an addition, mostly.) When a collection happens, objects not collected move to a different heap (the next generation), and the heap used for allocation is emptied out. The amount of work and moving depends on the number of objects remaining, which tends to be low, since objects are short lived.

The advantage to pure refcounting, in languages such as C++, is that collection is deterministic. In the rare case that you have a cycle, then you do have to handle it manually (that's the tradeoff), but I find these are extremely rare. Memory management in modern C++ is a non-issue most of the time. And prevents bugs like:

  data = open(filename, 'rb').read()

I see this bug far too often. One could argue that in Python, it'll get collected via refcount, but this isn't guaranteed by the language. I've seen the same bug in Java/C# code, and there are no refcounts to save you there. (Of course, these languages have a with/using/etc. block, and it's an easy fix, yet nonetheless, these bugs are frequent.)

  [1]: http://docs.python.org/2.7/reference/datamodel.html#objects-values-and-types
  [2]: http://docs.python.org/2/library/gc.html

Well RubyMotion is using ARC.

Yep. https://www.digitalocean.com/community/articles/how-to-set-u...

>Also, don't forget that everything is an object.

This is false. What point are you trying to make here?

Could you provide an example of those attacks/injections?