Hacker News Reader

Comment by onlypositive | Hacker News Reader

> I have run SQLite as a web application database with thousands concurrent writes every second, coming from different HTTP requests, without any delays or issues.

Is this with nodejs or something single threaded as the webserver?

I would kind of assume you'd run into issues with something like PHP.

View on HN

In production, Node is generally run in clustered mode (multiple threads per instance). SQLite works fine with multithreaded applications, if you set a longish timeout for acquiring a file lock.

It does work better when all of your writes are on a single thread, though. I used Node’s IPC to accomplish this, and was able to get it up to 10k or so writes per second while still having it do tens of thousands of simple reads per second.

Blocking can become an issue pretty quickly with SQLite unless you do what this guy does and make a different database for essentially every process writing to the db. This can crop up in all sorts of situations.

For example today I was writing tests for a python application I'm working on that uses SQLite. SQLite is the only option I have for this program as I have literally no way to setup a client server db in it's environment.

I had the tests all configured to write to the same test db file. The program I wrote the tests for had no issues with locking because it doesn't run anything concurrently.

However, when I did the same in my test suite, they ran in parallel, which caused my tests to hang. Of course, I figured this out quickly and made the tests write to their own individual DB. This is a really small program, so cleaning up afterwards is no big deal. (I could also just use an in memory db instance instead of even dealing with files). But, for a program with a LOT of tests, I could see this being an issue.

Why do you believe that PHP would cause issues that Nodejs would not?

PHP is usually run multi-process, so may have more parallel activity than a single Node instance running single threaded

The answer here is non-trivial. I have since long stopped developing with PHP, but PHP has several run modes (And multiple runtimes like HHVM), that may bring different performance and concurrency characteristics.

Last I remember, PHP still runs all inside of a single process, so it still all share the same memory space, and it no longer has the overhead of starting a new process with every request.

The piece of engineering that made node.js so fast back in the day was Libuv, which allowed for non-blocking IO, greatly reducing the number of system calls/context switches.

But I am also going to guess that PHP developers have since caught on to the performance optimizations of non-blocking IO, and integrated the improvements into the runtime. Doing a quick search for "php vs nodejs benchmark" on Google [1], it seems like the performance of Nodejs is comparable to that of HHVM.

So as usual, use the right tool for the job. This is more of a problem of using your tooling correctly, more than choosing the correct language.

[1] https://yuiltripathee.medium.com/node-js-vs-php-comparison-g...

I used PHP but probably should have specified modphp or phpfpm.

Your mental model of concurrency is not correct.

Node.js uses non-blocking IO to achieve massive amounts of concurrency on a single thread, while one-thread-per-request models such as PHP rely on OS threading to do the exact same thing.

If anything, the node.js model is often capable of more concurrency because of the inefficiencies involved in OS threading and context switching.

EDIT: of course, event driven PHP exist. I don't know much about the current state of it.

Maybe very small, quick writes, or a workload where modest blocking isn't a problem. Curious what the IOPs look like.

Reply on news.ycombinator.com