Hacker News Reader

Comment by btown | Hacker News Reader

Did you build on any low-level libraries like RocksDB for data persistence etc.? Or did you fully hand-roll the database? Curious about the tradeoffs there nowadays.

View on HN

The core time series engine, called cesium (https://github.com/synnaxlabs/synnax/tree/main/cesium), is written completely from scratch. It's kind of designed as a "time-series S3", where we store blobs of samples in a columnar fashion and index by time. We can actually get 5x the write performance of something like InfluxDB in certain cases.

For other meta-data, such as channels, users, permissions, etc. we rely on CockroachDB's Pebble, which is a RocksDB compatible KV store implemented in pure go.

One thing that might keep me from using this is how well it integrates with other tools I might want to use for data analysis or historical lookback. For example, I currently use grafana as a simple, easy way to review sensor data from our r&d tests. Grafana has solid support for postgres, timescale, influxdb and a number of other data sources. With a custom database, I'd imagine the availability of tools outside of the synnax ecosystem would be rather limited.

That's a valid concern! As we're currently a team of 3 building this out - we still are working on building out our integrations with other tools, hardware, etc. We have been prioritizing building direct integrations to systems that our current users are interested in.

We also value enabling developers to build off of Synnax or integrate the parts they are most interested in into their existing systems. We've tried to service that end by building out Python, Typescript, & C++ SDKs and device integrations. We're continuing to look into how we can better support developers build/expand their systems with Synnax, so if there are any integrations you think are important, I would appreciate your take.

Reply on news.ycombinator.com