yashdotrv

Oh I see, what you say! ...but that’s intentional... we start the calculator at 1 TB/day because beyond this scale it's usually where observability pricing starts to hurt and ROI discussions kicks in. So, the point we’re trying to show is that, even at 1 TB/day, Parseable will still cost you lesser than others, and easier to operate too. Also the best part is that you will have the complete ownership of your data.

Hi HN,

This is Yash, founding team at Parseable (https://github.com/parseablehq).

We've built an open source observability data lake using Rust, that handles high-cardinality data at around 100M time series in production (https://www.parseable.com/blog/how-parseable-handles-100-mil...)

Our architecture is built around columnar design, and we use Apache Arrow for in-memory columnar processing and Apache Parquet for durable columnar storage on S3-compatible object storage. In Parseable, every labels stay as columns in the data instead of becoming a large long-lived per-series index like many TSDBs.

Also, one thing we’ve been thinking about a lot is how observability changes as agents become part of day-to-day engineering workflows. They're not just another service, they produce traces, tool calls, prompts, intermediate decisions, errors, costs, and sometimes sensitive business context.

Observing them matters just as much as observing any other system. But it is equally important to decide where that telemetry data should reside. Our view is that teams should be able to keep these observability data close to them: in their own object storage, under their own retention, access, and compliance controls.