Hacker News Reader

Midway through I realized they're reinventing solved problems already.

Mixpanel should take a look at Apache Iceberg for your writing and Apache Pulsar to keep your cost lower by not needing to keep 7 day retention in your pipeline once the messages are ack'ed by all consumers.

For replaying you can use Trino to just read from your Iceberg and insert back into your stream.

View on HN

It's a lot harder to make such drastic changes to something that's already serving production traffic at scale. It's okay to reinvent solved problems sometimes because the alternative is be a multi-year project with unclear returns.

Also, the 7 day retention is a feature - it helps us recover data quickly in case of bugs in our storage code.

I had written a novel of counter arguments and realized anyone who tries to tell someone that their bug is a feature isn't going to see the but for what it is.

Best of luck, and God help the poor souls who have to maintain this when the original authors have left.

Reply on news.ycombinator.com