Hacker News Reader

Comment by jeffbee | Hacker News Reader

It is fairly surprising to me that FB would pay a roughly 5% throughput penalty to get this.

In a car, you get the best speed when you press the pedal to the metal and close your eyes. Yet we pay performance penalty by driving it, instead.

Car analogy is not applicable - think more instead of lowering speed of the entire high-speed roadway by 5mph (e.g. by road material/quality) - has a qualitative difference at that scale.

Point is slowing down a bit, allows us to see what's happening, so we can make course corrections, and crash less.

That's an understandable tradeoff for car driving.

It's an understandable tradeoff for Facebook debugging.

Ergo, we do it.

Where does the 5% number come from? I didn't see it in the article or on the linked page for Intel Processor Trace.

Pulled from my experience. How much stuff needs to be logged depends on the structure of the program, and different programs are more or less sensitive to having some of their memory store bandwidth stolen.

Thank you for sharing!

It is not that surprising. Facebook is complex ecosystem, and therefore experiences lots of outages/problems.

Looking at their status page (https://developers.facebook.com/status/dashboard/), they seem to have a problems every week. These are only the public ones!

I guess they figured they are losing more money due to these problems than the additional 5% they have to spend on infrastructure.

I can see why an org would choose to do this, but the number is still frightening. At Google, we were held to a limit of at most 0.01% cost for sampling LBR events. 5% for debug-ability just seems really high.

They must have a reason. Probably helps them resolve otherwise costly failures in good time.

Reply on news.ycombinator.com