Hacker News Reader

Comment by thepumpkin1979 | Hacker News Reader

thepumpkin1979

I just tried to compile folly in OSX and it fails to compile with homebrew. https://github.com/facebook/folly/issues/400

Also, Once I manage to compile folly and then wangle, what kind of database drivers can I use in conjunction with this? Would I shoot myself in the foot if I use this with libpq under CPUThreadPoolExecutor?

Same with Rust and dlang, I'm not sure how to apply the freedom to have pluggable executor into something useful in real life, I think this is the main reason Golang is killing it and it keeps getting better.

For now I have my money on Go but can't wait for http://www.ponylang.org/ to become stable.

Edit: Issue reported has been closed, I can now proceed to install wangle https://github.com/facebook/folly/issues/400

Edit 2: Found another issue, this time installing wangle https://github.com/facebook/wangle/issues/37

View on HN

The problem is the exact opposite. The question you should be asking, is: how can I use my go code in Rust/D/C++? The answer is, not very easy, given that go's runtime is largely incompatible with C.

At least with C++, you can easily write C shim layers and your code becomes compatible with most runtimes in existence. The opposite is not true with Go.

Go uses synchronous I/O backed with an M:N implementation. Rust and D use synchronous I/O backed with a 1:1 implementation. Semantically, there is no difference between the two. The difference is in the implementation, and 1:1 threading is not as slow as you think on Linux.

Why are we talking about Go, Rust, and D on a thread about a new C++ library? I mean, I know the answer, but I wish we weren't.

I ordinarily wouldn't care, but I usually feel like I have something to learn from ambitious networking libraries even when they're written in languages I prefer not to use.

For what it's worth, I try to not bring up Rust in threads about other languages; it just happens that most threads about Rust appear in threads about other languages.

I would rather talk about the implementation of async I/O too.

I just wanted to point out the barriers a developer faces when trying to use C++. Sorry, I can't resist compare it to what I know, which happens to be other programming languages different to C++.

Does that mean that one can use system threads as frivolously as goroutines (e.g. spawning new thread for each incoming request)? Conventional C++ wisdom advices against it. But then it may be informed by some old and bad pthreads implementations.

A reasonable compromise is to use a threadpool (which is a big part of what comes with wangle) with a fixed number of threads and you queue the tasks to be run by the threads. It works quite well in practice, and you can use coroutines (I'm most familiar with the implementation in boost coroutine and the Ewan webserver personally) to go the whole hog and get something as lightweight as Erlang green threads or goroutines. I would only bother doing that of course if in your particular use case you find the threadpool doesn't perform well. With proper sizing and broken up units of work, it really works quite well in practice.

Usually memory usage ends up being the limiting factor, not thread spawning performance. Memory usage is independent of 1:1 vs. M:N, though, as pthread stack sizes are configurable.

> Does that mean that one can use system threads as frivolously as goroutines (e.g. spawning new thread for each incoming request)?

Yes. In fact, under Linux oftentimes spawning a thread and then running a system call can be faster than just running a system call.

> Conventional C++ wisdom advices against it.

With 32 bit CPU's you might run out of virtual memory quickly, since each thread allocates virtual memory for its stack.

With 64 bit CPU's there are no issues. (Also never forget that virtual memory isn't real memory, it's just a page counter.)

> Yes. In fact, under Linux oftentimes spawning a thread and then running a system call can be faster than just running a system call.

Interesting, do you have any further info (references, cites) on this?

Sign me up for wanting more info as well. I'm curious how that could work out, since spawning a thread is a system call.

> what kind of database drivers can I use in conjunction with this?

Usually synchronous database drivers have to be reimplemented in an asynchronous fashion to work with event loops. It's not very hard though, could be as simple as adding jump tables and context structures into synchronous functions.

Emulating asynchronous execution with a thread is ok too. You do that to access filesystem anyway.

But if your first thought is about accessing a database server - you probably don't need to go asynchronous at all.

> Emulating asynchronous execution with a thread is ok too

Straying away from the topic, but I used to support a file streaming library for the game studio I worked with, and while this approach worked, some things were not gracefully handled - for example canceling reading in the middle, or somehow giving all your I/O requests to the underlying system, so it can decide what's the best way to read them (aio?).

The wangle native way of doing this is just to use an IOThreadPoolExecutor: https://github.com/facebook/wangle/blob/master/wangle/concur... Just make a block of threads to run your db tasks, and you can use a promise/future to return the results back to the async code.

There's https://github.com/facebook/squangle for MySQL, though it's lacking documentation, and requires a https://github.com/facebook/mysql-5.6 or WebScaleSQL client library

MariaDB has an asynchronous C API.

libpq (Postgres's client library) has an asynchronous interface as well.

Reply on news.ycombinator.com