Can you really blame Ruby when the project didn't have the right resources?
The team could have booted up more processes. Isn't that the answer?
Or have I misinterpreted your comment?
The team could have booted up more processes. Isn't that the answer?
Or have I misinterpreted your comment?
Not 200,000. A single process wasn't able to handle 20.
You'll be hard pressed intentionally writing code that performs that slowly in a framework in another language.
Ruby by default has an incredibly long list of "you can but you shouldn't" and things you need to be aware of, especially around concurrency.
Other languages are mostly fast by default. When you follow some nodejs/express/whatever introduction tutorial and implement your first application in the most naive way possible, you'll end up with something that will easily scale to hundreds if not thousands of users out of the box.
Do the same with ruby (or even with some python frameworks, to be fair), and stuff can break at less than a hundred users.
And that's why yes, you can blame that on ruby. Or at least ruby how it is commonly taught.
That aside, a ruby script that can't handle 20 req/s on 2 vcpus seems rather suspect. I'd be really curious what that code looked like. I'm doubtful that it has much to do with the language and not the code design.
I have never ever been concerned about concurrency in Ruby, or Python or PHP. Just deployed the number of workers which would be able to serve our peaks.
I do write high concurrency application though, but those are not web apps and are not written in these languages.
The example said there was 0.5 seconds of IO per request so each process could serve 2 requests per second. 2 processes would serve 4 requests per second. The mistake was running only one process per core. 10 processes would server 40 requests per second, and the cores would still be mostly idle. With rails you always run as many processes as memory allows, not one per core.
Now that rails is thread safe you can have each process run a few threads. If you ran 10 processes and 4 threads per process you are up to 120 requests per second.
If you really need massive concurrency that isn't anything special. But that is more than enough for 90% of the web apps out there.
Serving only 20 users only has to do with how many workers were assigned.
I actually implemented a high traffic RoR payment provider on minimal infra ( 1 small VPS ), and benchmarked a couple of Python CMSes and compared them to WP.
I'd like to see your results if you still have them.
- overprovision resources for big demos by a significant amount
- test your demo before you give it
- deploying pets (servers you ssh into and lovingly configure) instead of cattle (instance abc123 that's created automatically) means you can't scale them as easily
- if you have concerns about a big demo, particularly ones that could be fixed by spending $10 on EC2 instances, you should fix them yourself instead of emailing the tech lead and assuming he'll handle it
- spinning up processes with phusion passenger is awful[0] and we should have been using something threaded like puma
That being said, the same footguns apply at various points as you scale, because Ruby's GVL still exists, and because even though threads are lighter weight than processes, they're still relatively expensive to switch between and to maintain in memory compared to e.g. Go/Elixir lightweight threads or evented IO.
Unfortunately (or perhaps fortunately) the problems with threaded IO start applying well above 20 concurrent users, so I don't have any fun stories about failed demos involving Puma instead of Passenger.
[0] Specifically, Rails processes take up a lot of memory, none of it is shared, context switching between processes is pretty expensive, and so you're limited to a smaller number of processes that can become IO-bound very easily
Another optimization is using jemalloc. Going to try this soon with Sidekiq workers.
This is a really common misconception. Both switching between threads and processes is _extremely_ fast compared to what web applications do.
An exceptionally fast web response is generally <1ms. Linux thread context switching takes ~2 microseconds. Forking a new processes takes ~20 microseconds.
It's also not a fundamental restriction on threads vs Goroutines either. switchto is very nearly as fast a Goroutine context switching https://www.phoronix.com/scan.php?page=news_item&px=Google-U...
Not true. Context switching pthreads is orders of magnitude faster than any Go/Elixir concurrency mechanism.
So-called "lightweight" threads exist to fix the concurrency problem with garbage collection in scripting languages. Without garbage collection pthreads are fast enough for any problem.
That's the original advantage over OS threads.
Everyone dropped the concurrency ball hoping that was someone else's problem. Its the dev team's fault because everything is ultimately the dev team's fault. But it would also be great if it was fixed in ruby + ruby's ecosystem.
Cause running more than 2 processes on a 2-core server just sounds weird.
I thought Node's doctrine was "One process per core", and (async) Rust and Go seem to do fine with managing their own thread pools. I haven't _tried_ to do a realistic benchmark of Hyper, but I've never seen it struggle with concurrency either, even on long-polling applications where concurrency was the norm for all clients.
More processes would work, but it doesn't seem like the right layer of abstraction for 2015, now that Nginx and Node have made the event model so popular.
The example given was a Rails app. I don't think one process per core is common for Rails deployments but I'm not sure.