Hacker News Reader

Comment by dillydally | Hacker News Reader

Erm, that solution is a little extreme, unless there are a few steps missing.

The first things I'd do: 1. Aggregate those inserts into a single SQL command. You never want to be issuing O(n) SQL queries per request.

2. Use InnoDB, not MyISAM, for the database engine. I assume it's MyISAM because he's talking about table locking. InnoDB has row-level locking, so you'll be able to INSERT and SELECT from a table concurrently.

I had a Facebook app with 20MM MAUs, and it worked fine with two machines. A component of the app was voting in polls, and I was processing ~10k votes per second. Each "vote" is an insert into a votes table.

Soo...not really sure how you go from "We're running too many INSERT queries and locking the tables!" to "Async queues and MongoDB!"

View on HN

I could not agree with you more here. And they are using InnoDB since it's Rails-based. Wrapping up all the INSERTs into a single query or transaction would really help. I'd also say consolidating the validations into a single SELECT would help, but it seems they threw the validations out when moving to MongoDB.

And on the server level, Passenger isn't optimized for running a single site. I use Unicorn for my Rails app.

Yes, yes, yes.

0. Make the right kind of table. create table user_apps (user_id integer, app_id integer, primary key (user_id, app_id)); -- this does the validation for you, and (should) index the table on both user_id and (user_id,app_id).

1. Make the right kind of insert statement! insert into user_apps select 5, 6 union all select 5, 7 union all select 5, 8; -- etc

2. How do you do recommendations? That sounds like it'd be tough to do well, especially within a web request.

I didn't read the part about validations. That's silly.

Even if you have to do it in the application layer, why would you need more than one SELECT? Just get the installed apps from the DB in a single query and do a set difference operation to only get the apps not already installed.

But I agree that putting the constraint in the DB layer is the correct solution. With MySQL you can just do INSERT IGNORE to only add apps not yet marked as being installed.

Yea I agree. From what I can tell from the blog post, they aren't even close to the limits of what mySQL can handle if properly configured unless they are running it on some kind of super tiny virtual machine. Seems like they just jumped the gun and over engineered it.

Reply on news.ycombinator.com