Hacker News Reader

"As of 2011 Facebook is in the second group, and spends several milliseconds of CPU time sanitizing every display string on its way to the browser, which multiplies out to hundreds of servers worth of CPUs sitting in a datacenter paying the price for the invalid UTF-8 in the databases."

I can imagine how companies will be taxed extra for this somewhere in the (probably not so near) future.

View on HN

Why not sanitize it once when accessed, write the sanitized result back to the database and set a flag on the record. Use only presanitized strings if the flag is set. After some time, when enough of your strings are flagged, run a sanitizer over the rest that hasn't been touched. That way you don't waste those milliseconds. However, I can imagine that this would waste quite some storage bandwidth because such a read is now a write as well.

Doing that at the scale of facebook does not seem like a problem you can solve in a paragraph of explanations, which is the point of the blog:

The longer you wait, the more difficult it is to solve.

Reply on news.ycombinator.com