http://www.reddit.com/r/news/comments/1co395/live_updates_of...
buro9 : This is just a complete conjecture. I do not know Reddit's internal hardware infrastructure. I just know that real-time police radio transcription is the best news source, bar none. And it's getting hammered.
Are those are by-products of the threaded display? As a flat display of a thread would just result in new pages at the end, all of which would be incredibly cacheable.
During the Boston Marathon bombing, there were, if I remember correctly, over a dozen threads that continued the original conversation, simply because each of the previous threads kept getting hammered by people posting comments.
I am mostly wondering whether it's the threaded nature of a thread on Reddit that makes this a difficult problem, or whether this is reflective of some limitation in the underlying datastore or something.
It's easier to cache and serve a fixed piece of content than to constantly insert and re-compress it (which kills cached versions too)