Hacker News Reader

HNHacker News

Top New Best Ask Show Jobs

Comment by nitred | Hacker News Reader

Full thread

nitred·

Could someone tell me the disadvantages of solving the problem in the article using S3? I am actually curious.

View on HN

Metrics on S3 at scale? It's about the worst tool you could use for this task.

The cost of puts alone would kill you.

Off topic, doesn't Twitter use their own data centers and GCP? When did they start using AWS?

They're inserting 5 billion events per second (for some reason), and AWS throttles everything (the following link mentions a limit of 3,500/second):

https://aws.amazon.com/about-aws/whats-new/2018/07/amazon-s3...

Also, if they're planning to do alerting, S3 would not help with that.

PUTs are 1 cent per 2000:

https://aws.amazon.com/s3/pricing/

5 billion events per second? That seems to be too much for monitoring data even at Twitter's scale. The article refers that they store the raw data and use rollup tables for aggregate queries, I wonder how much the ELT process costs in total.

> 5 billion events per second?

When you work for a large company, you have 2 problems:

1) Every has-been manager bikesheds over metrics retention

2) you can't say no. See #1.

Indeed, the article mentions 5b metrics per minute

You write to S3 in chunks for long term storage and keep hot data live. You don’t write every metric directly. Along with compression It shouldn’t be that 5 billion/s.

Even with chunks it's prohibitively expensive at this scale, and you then need some intermediate system batching the inbound data either to disk (then why use S3 instead of a database) or in memory (more expensive, more risk of data loss).

Reply on news.ycombinator.com