Hacker News Reader

Comment by brlewis | Hacker News Reader

An even better secret to making things easy: only try to solve problems that exist.

This is a nonexistent problem: "The problem? Most databases are still storing my name using the same techniques that they did 15 or more years ago -- they seek the disk head to some specific location and then read or write my name to that location. These databases rely on the one operation that DID NOT dramatically improve in the past 15 years! They still perform like it's 1991."

I don't think PostgreSQL is the only database that uses write-ahead logging.

View on HN

Post some numbers. How many updates per second can you do?

167 small transactions per second on MySQL 4.1 / InnoDB / FreeBSD / FFS / SCSI, which looks like one per rotation. Presumably the time required to write a block at the end of the write-ahead log.

5000 per second with --innodb_flush_log_at_trx_commit=0, which only writes the log to disk once per second.

The MySQL documentation claims that this configuration does crash-recovery correctly, though you may lose the last second's worth of transactions.

BTW, in case anyone is thinking, "why would I need more than 167 transactions per second", imagine that you have a web site that displays everyone's "last active" time (for chat maybe). If users are performing some activity once every 30 sec, then you will be pretty much be at your limit with about 5000 simultaneous users.

Robert, did you test MySQL with a parallel load? To give it the best chance at maximizing tps, it will need to be able to combine multiple pending transactions in a single write. Also, can you tell if it's updating it's b-trees? (if not, it may not be able to sustain these rates)

I ran the test with 1,000,000 INSERTs (taking 200 seconds), with the innodb cache size set to only 200,000 bytes. So there's a fair chance it updated the b-trees on disk.

I don't know. If that becomes a problem I'll put the write-ahead log on a faster disk to buy time to look into it.

Why can't you just succumb to peer pressure and get rid of that database? :)

I'm not sure. First I'm going to get rid of that operating system. It has lots of features that aren't relevant to my web app.

It's somewhat amusing, but also mildly depressing, to see these remarks despite the fact that I explicitly stated (in bold, even), that "I'm not advising you to ditch your database!"

And yet somehow that gets translated into "get rid of that database."

Sorry it's mildly depressing. Unfortunately, any thoughts you bring up related to a polarizing issue are going to end up being framed in terms of the polarization. That's just the way it is; some things will never change.

On the up side, you are going to learn a lot about essay writing, and about how concepts do/don't get through to people. These are very valuable skills.

Don't be depressed. You started something that's spawned (as of now) 28 comments. Great thread.

Fast disks are only faster by a factor of two.

I think I could address any such performance issues within a factor of two growth of updates per second.

But wait, didn't you say it was a non-existent problem?

Someday updates per second might be a problem. The nonexistent problem is "they seek the disk head to some specific location and then read or write my name to that location." I may have missed it, but I haven't seen you admit yet that you were wrong. You've been informed now that reads use a memory cache and writes use a write-ahead log, right?

I'm probably at least partially wrong about everything..

It's true that PostgreSQL uses a write-ahead log, but it's still going to do that seek when it checkpoints, correct? I don't think that it will be able to sustain high transaction rates (100,000 tps for small transactions should be very doable). If you find that this isn't true and that someone has demonstrated these rates, let me know -- it would certainly be good news.

Yes, it will still do that seek when it checkpoints, but I seriously doubt it will do one seek pert transaction at that checkpoint. Still, I doubt 100,000 tps can be achieved with PostgreSQL.

As for being wrong, that's one of the privileges you lose with celebrity. Everything you write in your blog is going to be preceded by "The guy who wrote gmail says..." When you say something that looks like it might become a popular misconception, guys like me will jump up and down, scream, and tear at our hair. Or maybe we'll just post terse, cold, unfriendly corrections. We still like you, though.

My main issue with your post is the way it comes across as saying databases are overengineering for 99% of web sites, but mainly deals with a performance issue that won't matter for 99% of web sites. I would have no problem if you framed it as high tps vs low tps for certain classes of application, not as simplicity vs overengineering. Using a relational database is a fine default choice for web apps today, just as you expect to have an OS on your server by default.

Reply on news.ycombinator.com