Hacker News Reader

I mean... 1 million requests an hour isn't that many.

That's around 300 requests per second. Assuming 1000ms upper bound on requests, you need 300 workers to handle that load. Assuming 150MB per worker, that's 45 GB of memory required to handle the load. So like... 5 m4.xlarge instances on EC2 (to give redundancy and allow loss of 2 hosts). That's $700/month.

That's not that much. We've got a Rails app that pushes 5000 qps. And to be fair, we just dial up the number of instances and it handles it fine. It runs on over 100 instances. It costs us around $10k/month. Not the end of the world, cost wise, but we have multiple Go services that handle similar levels of traffic and run on a dozen instances. Additionally, deploys take a long time (rolling restarts plus health checks on 100 machines takes time).

Moving to Go (or Elixir) allows us to handle far more requests per unit of hardware. While latency would indeed improve, it's not the primary motivator for us moving away from Rails.

I haven't even mentioned the websocket story on Rails. That's a whole new can of worms.

* 1 million requests an hour isn't that many.*

What percentage of websites serve 1 mil or more requests per hour, .01%? .001%? Meaning Rails is going to be performant enough for 99.9+% of projects, and for those projects it would have been a mistake to trade dev time for performance you’ll never need.

1M request per hour at peak?

A SPA backed by rails is probably going to make at least 10 requests on page load. So in terms of actual traffic, 100k page loads during a peak hour. Assume a roughly linear peak increase/decrease and we've got roughly 1M page loads per day. 30M page loads per month.

How many websites have 30M (non-unique) page loads per month? After some rough scouting on Alexa ranks, I'd put the over-under at probably 10k US sites, and 50k worldwide. Assuming Alexa has 50M sites, then 0.1% of publicly facing sites serve that much traffic.

Rails is used often on private, internal sites and tooling. Those sites wouldn't come up. That would definitely skew the 0.1% number.

Not making a huge argument here, I just started down that analysis path out of curiosity and figured I'd share it.

Great analysis! According to Netcraft, there are > 600million websites, so assuming Alexa ignores the 550 million with near zero traffic, you can get to the .01% pretty easily. Another area I'd tweak is the skew towards SPAs. Most sites aren't SPAs, especially as you slide down the traffic rankings.

Yeah, the skew towards SPA is more Rails focused. I haven't worked on a non SPA Rails app in over 5 years. P(SPA | Rails) is higher than P(SPA).

Do rails apps actually use 150MB per worker or was that just a lofty ceiling estimate? Or is that with an app server that doesn't utilize process forking, and loses the benefit of shared libraries/heap/etc?

Yes, but you only need one process per core, just like NodeJS.

Since 1.9 you can use real OS threads to achieve parallel IO, and certain parallel computations which can proceed without holding the Global Interpreter Lock.

JRuby offers completely unrestricted threading with a single process, plus a 3x performance boost, plus the advantage of an incredible amount of work put into their VM and GC. It's a really underrated option these days.

The problem with forking, is MRI Ruby is unaware what memory is actually inherited by forking, so eventually it causes the entire heap inherited from the parent to copy into the child.

I'm actually working on a patch to fix this. The solution is simple, just don't mark, collect, or allocate into inherited pages, but the implementation itself is fiddly.

What's really exacerbated this, is that most Linux distros now have Transparent Huge Pages turned on by default, and flipping a single bit in inherited causes a 2MB copy instead of a 4kb copy!

I'm fairly certain this was fixed in 2.0? https://medium.com/@rcdexta/whats-the-deal-with-ruby-gc-and-...

Nah, unfortunately simply moving the GC bits from the object itself to bitmap in the page header made Ruby CoW friendlier but not CoW friendly! Each Ruby page is 4x OS pages on Linux, so marking into the header of each Ruby page still causes 1/4 of the parent heap to copy into the child process.

The bigger issue is heap fragmentation. Lots of inherited pages have a couple of free slots that Ruby will happily consume, effectively causing you to pay a 4kb copy for a 40 byte object.

This means allocating a few hundred objects can cause basically the entire heap to copy. Combine this with Transparent Huge Pages, where flipping one bit causes a 2MB copy rather than 4kb, and a few hundred object allocations can cause the entire parent process memory to be duplicated into the child.

Aaron Patterson is doing some great work to bring GC compaction to MRI, which will help reduce fragmentation, but it's a huge task.

Yes, they often do :/

> It costs us around $10k/month.

As someone who develops web apps for startups not in SV, that seems incredibly expensive to me.

It's the difference between 30 developers and 31 developers on payroll.

Haha, yeah. It's always interesting when you put it in that context. I often joke that AWS is our hardest working employee.

That's amazing. I make less than 1/5th of that and it is considered among the top wages over here.

Well, it's a sliding scale isn't it? If you have less traffic/paying customers you can pull that back down.

Have you ever considered JRuby on Rails? We've had 10.000 students choosing lectures online at the same time using torquebox back in 2012 on just two server instances with multimaster replicated MySQL.

I share your opinion, if you have any bottlenecks which are unsolvable because of Rails or Ruby being slow your app probably scaled up to a point where you have the resources to make a creative solution that is not Phoenix/Elixir.

Having said that, while I don't find any major reasons to opt for Phoenix instead of Rails, my main reason to work with Elixir/Phoenix right now on side projects is that it won't make me unlearn Rails which I can always go back to, in fact, using Phoenix for a while made me miss a lot of the available libraries, plug-in solutions and wide range of community knowledge.

So ATM I have more reasons to choose Rails over Phoenix, but I suppose that's why these sorts of articles are important, publicity to increase a community and fulfill Elixir's biggest need right now.

> in fact, using Phoenix for a while made me miss a lot of the available libraries, plug-in solutions and wide range of community knowledge.

This is my main reason for focusing on Rails rather than Elixir at present. The sheer breadth of gems available for rails to quickly build sophisticated applications is hard to beat for any up and coming language/framework.

That said, my gut feel is that many of the developers who build and maintain these gems have moved on and that Elixir is the new hotness and will probably catch up in the next few years.

I wonder if you might extrapolate on which libraries, plug-in solutions you missed in Phoenix.

My bottleneck with Rails was always memory usage. Our concurrency was always limited by how many web and task workers we could run in the amount of memory we could afford, which was not very many. It was not straightforward to optimize this and ignoring it didn't work for us, so we put lots of time into it and it changed the whole productivity argument for Rails.

You should really only need one process per core, plus a few thread per process. Obviously, this is a problem on Heroku where they give you 8 "cores" but only 512mb of RAM per 1x dyno but on a DigitalOcean or AWS server you shouldn't be running out of RAM before maxing all cores.

Agreed. I have a sidekiq job that imports data into elasticsearch from a mysql database using activerecord, and it uses hundreds of megs of memory, which is ridiculous considering how little data is actually being imported.

This is basically my job at ChartMogul and we've pretty much solved this problem. The two biggest issues for us were: Ruby prefers to grow the heap really quickly rather than spend much time in garbage collection. You can turn this growth factor down at runtime using an enviroment variable.

The second problem is importing a huge chunk of rows at once means they have to exist in RAM at the same time. Use batched iterators to reduce peak memory usage. All GCed languages have this problem, Go included.

You'd think Go's GC was somehow revolutionary from the way they talk about it, but it's basically the same as the Ruby GC, plus a little more parallelism. What helps Go is that the compiler can optimise and use stack allocation and re-use temprory variables. If it fails, it causes a nightmare, and the Go standard library is full of tricks to convince the compiler to do stack allocation.

Java, OTOH, has compacting garbace collection, so after high peak memory usage, it can release the memory back to the OS. Aaron Patterson has been working on doing the same for Ruby. If you use JRuby, you'll get this right now, plus it's about 3x faster for shuffling data around.

Another difference (which, as I recall, mattered for us) is that class objects in Ruby take up quite a bit of permanent space in memory.

Ruby is actually pretty good in this regard. If you define your module or class anonymously, but give it a name using a constant, Ruby will GC it when possible. The standard way of defining modules and classes obviously means they can never be GCed.

Java doesn't, or at least didn't, collect anonymous classes without an additional GC flag being enabled, which could bite you quite hard using JRuby with gems which made heavy use of anonymous classes.

In practice, modules and classes are not defined that way - likely not in your own application, and certainly not in all the gems you depend on, or in the Rails framework itself. That entire set of transitive dependencies can take up a lot of memory.

The class definitions will be a tiny fraction of memory usage. A template Rails app memory usage only has about 20% managed by Ruby. The rest is the VM, C libraries, maybe long strings etc. Definitely not class definitions pulled in from gems.

Yeah we struggled to keep our memory usage under 500M per worker. Sidekiq helped because it was able to use threads effectively, but especially when we were running resque workers, we could only handle tens of tasks per second, and often fell behind. I think Rails' autoload-the-world philosophy was a big part of the problem and we spent some time trying to untangle dependencies, but it was swimming against the stream.

I'm not sure if these same problems do or don't come up with Phoenix, but when I briefly used it, it did seem to have a smaller memory footprint.

500MB per worker is totally standard. What happens is a job causes a huge array or hash to be allocated, and after it‘s finished the memory can’t be returned to the OS due to heap fragmentation. Java does some crazy stuff with compaction. C programs typically try and internally allocate into arenas to avoid it.

The thing is we're not even using rails, it's a simple Sinatra app with ActiveRecord, so there's not much being loaded that's not being used. Could be the ActiveRecord itself is the problem though.

Tbh, 200ms for the db is massive. I have a 100ms limit at which my mongodb writes into the slow-log. It's an almost empty log. My point being, apps can have all sorts of bottlenecks and you only have to throw the right brain at the problem to fix it, no need to replace the techstack.

Yes, gems. The ecosphere in ruby is superb. I'm started working with node a lot more these days and it's a) confusing and b) small packages for all those things whatevertheycallitthesedaysscript is missing. 243 gems in my 7yr old Ruby vs 918 node modules in an app we started this month. What's Elexir/Erlang like in this respect?

even for "old databases" (mysql, postgresql) 200ms is huge or to it differently say probably his database is not the bottleneck. He probably fetches big lists, and even lists with over 100 entries are extremly slow on python/rails, whatever. on java/c++/go/rust these things are way faster.

The database doesn't need to be an issue - it just currently is. We have a fairly complex query on a several terabyte large PostgreSQL database and didn't want to spend more money. Basically, we just haven't sharded it yet, and don't want to pay for a larger machine. The point is - typically the issue isn't the web app. Almost all web frameworks are designed to be scalable from the start. Just launch another instance.

Also, we do about 99 writes for every 1 read, so query times don't matter as much. For the given app I was using as an example.

Network I/O and large databases are usually the bottlenecks from what I've seen consulting and developing.

This was probably true 10 years ago, but these days it's basically FUD. Since then Ruby improved performance something like 5-80x from 1.8 to 2.5 and moved from an interpreted language to a VM nearly as fast as LuaJIT. Go only has 3-5x the throughput for tasks like: grab 100 rows from PostgreSQL, serialize to JSON, respond via HTTP. Lots of the hot code in Ruby around fetching rows from the DB, serializing to JSON, and HTTP parsing have also since been implemented as native functions in C, massively improving performance with zero effort required by the developer.

yeah this is cool, until you make a single map...

Do you mean: "take a few thousand rows and map them to a different data structure"? Because I benchmarked that recently and mapping 16,000 rows of GPS points using haversine distance in pure Ruby takes about 12ms, and about 5ms in Go. It's not that much slower. There are tasks where Ruby can be ~100x slower than Go, but a simple map isn't one of them.

MySQL and PostgreSQL are considered “old” now? I’d guess that’s much like Rails being considered “old” and “legacy.”

Well there is always question of $ if running on Phoenix would let you save $100/month it's one thing if it's 100k/month things might look a bit differently and when it is 1 mil/ month things that will change equation even more. (This is not going into areas were Ruby is not a usable option such as massive real time apps etc.)

> Waiting 200ms for the database + an additional 20ms for rendering

Those are pretty awful numbers. https://blog.gigaspaces.com/amazon-found-every-100ms-of-late...