Hacker News Reader

Comment by cperciva | Hacker News Reader

This article was mis-titled: The title should be "how to get poor performance by using a completely inappropriate tool".

Quoth the article: "Several million records a day need to be written to a table. These records are then read out once at the end of the day, summarised and then very rarely touched again."

This isn't a job for a database. This is a job for a log file. You might want to use a database to store the summarized data, of course; but bulk data which is written, never updated, and quite likely only ever read once? That's practically the definition of a log file.

View on HN

Agreed. They did stipulate, however, that the data needed replication between multiple datacenters for redundancy, which would make me think about putting a message queue in there that replicates across DCs, and having logging daemons at each DC pull data out of the queue.

Surely you could just use logs with rsync?

Or just get your storage to do it transparently. Every vendor has something like this these days http://www.emc.com/products/detail/software/srdf.htm

You're right, I guess I was thinking both data centers needed to be in as tight of sync as possible, but looking back that does not seem to be the case.

that is the case. all databases have to be within seconds of each other - it's required for BCP.

Facebook Scribe supports multiple stores

http://wiki.github.com/facebook/scribe/scribe-configuration

some hackers hack simply to find out what's on the other side

Well... that's fun and all, but surely it's better to use the optimal solution and see how far you can take things? I personally find building meaningful products that my customers love hard enough without handycapping myself for my own amusement!

No, it's just another case of RDBMS blinding people for all other possible solutions.

Have you ever noticed how when this happens, it's practically always MySQL? (And when people complain about how badly relational databases perform and it turns out they don't know about indexes, transactions, or stored procedures, it's always MySQL...)

This.

The non-MySQL RDBMS people that I know, know when their RDBMS of choice isn't the right tool for the job. Most of the MySQL people I know, seem to think that you just shove everything into the DB, and let Eris sort it out.

Because no other member of the pantheon is crazy enough to even look at that mess.

I suspect a network effect is in place where the vast majority of people whose entire understating of databases came from "PHP + MySQL For Dummies"-caliber books, blog posts, etc. use MySQL by default. A critical mass of them end up perpetuating bad advice, griping about how since MySQL hasn't worked well for them, all RDBMSs are junk, etc. Then, people looking for advice starting with databases read their tutorials. To some degree, MySQL probably brought this upon themselves[1], but now there's a feedback loop in place[2][3][4].

   [1]: http://www.reddit.com/r/programming/comments/764fp/mysql_vs_postgresql/c05sayb
   [2]: http://sqlanywhere.blogspot.com/2008/03/unpublished-mysql-faq.html
   [3]: http://news.ycombinator.com/item?id=561277
   [4]: http://ask.metafilter.com/117908/Theres-got-to-be-a-faster-way-to-update

There seems to be a similar effect between Linux and BSD. I'm not going to claim that BSD is objectively superior in every regard, but on average, the BSD community seems to be quite a bit better informed than the Linux community. It may just be because Linux is so much more visible. People using BSD, Postgres, etc. probably already knew enough to evaluate their options, while the path of least resistance stays overrun with clueless newbies.

No. And it doesn't have anything to do with any deficiency MySQL may have - RDBMS is not an OK replacement for a log file.

Of course, but that's orthogonal to my point.

It's not that hard to understand.

MySQL is a popular RDBMS starting point. Therefore, almost by definition, if you run into someone using a RDBMS but not really knowing WTF they are doing, they're quite often using MySQL.

If it wasn't MySQL, the same people would flock to, and poorly use, some other tool.

> If it wasn't MySQL, the same people would flock to, and poorly use, some other tool.

That's what I said, deeper in the thread. I don't think it's because MySQL is terrible, but almost all the people who don't really know databases start with MySQL by default, so the community surrounding it consistently has more people who are grossly misinformed than (for example) Postgres's.

"How long will people continue believing in the myth of the 15-year-old hacker genius while simultaneously decrying the unreliability of software before the cognitive dissonance finally cracks?" - Dave Herman (http://calculist.blogspot.com/2005/12/12-weeks-with-geeks.ht...)

Reply on news.ycombinator.com