300,000 requests / second | Hacker News Reader

300,000 requests / second

qos.doubleclick.net

62 pointsdm_mongodb21 comments

If their speed goes over 600,000 impressions per second, does their data center blow up? Where's the red zone?

It is cool, I suppose, but also vapid. It's a meaningless boast. Reminds me of the McDonald's how many billions served signs.

Never heard that no manager has ever been fired for hiring IBM?

When you go with the market leader, or a big player, the perception is that there's a smaller chance of things going wrong, and when they do, they have more resources than the smaller player.

It's Marketing 101.

I thought that had more to do with the fact that if something goes horribly wrong no one will fault you for going with the market leader - where as if you took a risk on something unproven you may have also risked your job.

Yes, and the reason none will fault you is because the perception I spoke of, which is molded by marketing campaigns like we're seeing here. So I was talking about the reasoning of _why_ being perceived as the leader, or as the best tech supplier, is important.

Vapid in the sense that it makes no difference? Or that it has no context?

Ok, so right now I see ~300,000 (315, but let's round down - because 15,000 hps is nothing, right?). Let's assume DoubleClick has some very big customers who all do lots of traffic. Let's say they have at least 10 big clients like Amazon that could bring their peak traffic up to a theoretical 10x more than the current average of 300K. That's 3 million hits per second, minus adsense adwords etc.

That is enormous. Astounding. And from my point of view, impossible to do without an extremely well laid out infrastructure. This number is therefore not just a vapid boast but a telling example of how awesome their network and applications are for being able to handle load of such magnitude. Even if it were only 5x the current load, 1.5 million hits per second is no walk in the park. You can't handle traffic like that without a truly robust design and several really scalable, really reliable layers, from appliances to handle incoming dns traffic all the way back to the app and database servers.

According to this document (http://docs.google.com/viewer?a=v&q=cache:nqfZBBPVbikJ:w...) in 2006-2007 DoubleClick's peak ad delivery rate was 250,000 with 900 servers (before they switched to using some . Right now it's current average is more than that.

I agree it would be nice to have some comparison to their peak capacity right now, but if you consider how tough it is to handle their current load and that they have to account for significantly increased load, the result is a network which is gigantic. To me this means I can rely on them to properly handle whatever meager site I throw at them to serve ads for.

That's just incredible. I wonder what Google's doing with AdSense on top of that.

Google owns DoubleClick. (http://en.wikipedia.org/wiki/DoubleClick)

Grandparent's question implies to me he's aware of that, and is wondering how much traffic AdSense drives on top of the DART stuff.

You can traffic AdSense through DART, so it is possible that the number has already been inflated with some AdSense numbers.

I think it's fair to count those though. If you have a DART tag on your page, the request needs to be handled by DART's ad server regardless of if it returns an image or a redirect to another network.

The off-the-record # I've heard from Google people is 1M/s

On what structure? A single IP? A single hostname? A hundred servers? Same datacenter? From a single SPARCstation 5 under the floor?

Without context, a number like this is meaningless.

Since they name it "Real time DART data", I suppose it's about DART, which is the name of their adserving platform.

One can only assume it's about their entire adserving platform.

It's ads served per second by DART. Doesn't count adwords, adsense. It is served from several data centers and this is the total.

I (think) know that mongo was created by doubleclickers.. am I right? Is doubleclick actually using it?

What I'd love to know is.. what kind of sharding is in place accomplish that?

Correct - MongoDB project was started by ex-DoubleClickers.

DoubleClick doesn't use MongoDB (afaik). My understanding is that it is switching over gradually to internal google infrastructure services - so presumably bigtable like things, and its internal successors, whatever those are. That said, MongoDB is basically what I wish I had had when we began building DART long ago.

In the early days of DART we build a proprietary distributed key/value store called UIS. It was super simple and super fast and worked well. The request volume, both read and write to it, was huge: it was hit in realtime by the ad servers.

These numbers just make me sad. That much advertising trying to warp and control us. We've come a long way, baby.

Unfortunately it's hard to get a web job today that isn't ultimately about advertising. We're all advertisers now.

? Huh. Not necessary. Don't equal programing = server side = free only sites. Some of us are actually building products that people are paying money for, with no ads. supported model whatsoever.

But how is that, as guelo said, a "web job?"

Reply on news.ycombinator.com