Hacker News Reader

Comment by dorianmariefr | Hacker News Reader

Could somebody tell me what's wrong with threads, e.g.

    require "open-uri"
    require "json"

    results = []

    (0..100).each_slice(10) do |slice|
      slice.map do |i|
        Thread.new do
          results << JSON.parse(URI.open("https://httpbin.org/get?i=#{i}").read)["args"]
        end
      end.each(&:join)
    end

    p results

View on HN

There’s nothing wrong with threads.

But the promise of Fibers + a fiber scheduler is that you won’t have to think about the performance impact of blocking IO as much as you might normally.

For example, in a rails app (or any ruby web server) I am very scared to make slow external http requests because that might tie up a thread for a while, and request threads are a pretty limited resource in typical configurations. (16 threads/process) * (1 process/cpu core) might be a reasonable default, giving you at most 128 concurrent requests in flight on an 8 core server.

Suppose you make a really slow 3rd party api call that takes 10 seconds to service a request. Your server can handle no more than 128reqs/10s = 12.8 of these requests per second, even though it will spend most of its time doing nothing but waiting.

If we bring fibers in to the mix and do one fiber per request, then they can yield to another fiber while waiting on the api response - you can then easily have thousands of requests in flight instead of 128.

Granted, if you are entirely IO bound then you can probably get away with a much larger thread pool, or do thread-per-request. But if you have some other endpoints that are more compute intensive then bumping up your thread count might be dangerous... then what do you do? Fibers might be the answer.

Correct me if I'm wrong, but this code would block if (for example) one single request in a batch of 10 takes significantly longer to complete. The 9 other threads are then doing nothing while they wait.

Of course, you could implement a more robust algorithm for scheduling threads, but it sounds like an implementation of FiberScheduler would make the best use of a single thread's resources (and you could scale that to multiple threads as needed using something like a shared queue). Since fibers are lighter-weight, it's presumably more efficient to run parallel IO operations in a single thread, and only use more threads if your work requires it.

The article example waits for all the fibers to complete as well.

I'm not sure about Ruby's specific implementation, but the threads need not wait around. When the thread entry returns pthread_exit is called. I believe the OS thread would go away regardless of how you are waiting on them.

Well the idea is that thread creation itself is not really cheap. For most software I would say it is not really a problem, but if the "work" to be done is short enough, that creation of thread becomes the most costly factor. True, that can be solved by thread pool usually.

So I guess as with most things in software engineering, if there is something wrong with threads or not depends on your use case.

With 10 threads at a time - not much, but threads are pretty heavyweight when all you need is waiting on IO. Systems where waiting IO doesn't map 1:1 to threads scale to more concurrent users better (Erlang, Node, Go all do this). With evented IO you can service hundreds of thousands of connections (if you have a good way to round-robin between them or some other way to schedule them well). Having the same number of threads will be difficult.

You can’t have a thread per http request.

Of course you can. In the days of CGI scripts, we forked a full process per request, never mind a thread.

I'd wager that the great majority of Web apps (i.e., the not-terribly-busy ones) would scale reasonably well on thread-per-request.

Reply on news.ycombinator.com