377 karma · joined August 23, 2012
FMR Founder at HYDRA. Hydra.so
YC Badge: 0xf0aaca65fc7940aaaf31b089afd502043f807415
Events, time-series data, user sessions, click, logs, IOT sensor readings, etc. generate a lot of data over time. While on-disk storage works well for Postgres’ rowstore, it’s a poor choice for fast growing data that requires analysis. To avoid the scale limit of on-disk storage, Hydra separates compute and storage. Also, we're not charging separately for bandwidth since it's been factored into the overall plan price.
While storage volume can be a good proxy, many people see the limits of Postgres with a complex join and filtering on relatively small data volumes. With decoupled columnstore and serverless processing, Hydra can be used in big (and small data) use-cases. Company size is a little less relevant since medium and large-scale companies have use-cases where efficient 'small data' is needed too.
Ideally, you can easily switch over to Hydra. Or Hydra can work as a fast, external analytics database too. It's Postgres-native so no changes are needed to use it in a traditional architecture if you wanted to.
Feel free to DM me on X (@JoeSciarrino) or email founders@ so we can coordinate on the SJC region.
One of the downsides of serverless is that it can be difficult to predict the overall monthly cost when the granularity of billing (per invocation, memory usage, or execution time) is complex. For developers this might be totally fine (even preferred), but we think that giving a single, predictable price: Hydra $100 / month is better for businesses to plan around.
Usage caps per plan are purely soft limits so users don't actually encounter them. Yes, we want people to upgrade to higher plans. In the words of Maya Angelou "Be careful when a naked person offers you a shirt" - meaning, we believe these are the best prices we can offer today to build a sustainable project on. That said, I appreciate your point about our # of users limit. If we removed that limit would you try out Hydra?
We initiall set the rowstore as default, but people wouldn't create columnstore tables and were confused on why performance wasn't improving. So, figured this was cleaner, but you always have the option to switch the default table type back.
Features: Serverless Processing
- Parallel, vectorized excution
- Compute Autoscale
Bottomless Storage
- 10X data compression
- Automatic caching
- zero-copy snapshots & forks
Our collaboration with Hydra revolves around pg_duckdb, an open-source (MIT licensed) program that embeds DuckDB’s state-of-the-art analytics engine and features within Postgres. pg_duckdb is meant for developing high-performance applications and analytics with any new or existing Postgres database. We’ve observed software engineers increasingly embedding powerful analytics directly into their applications. These applications tend to require both greater access to disparate data sources and sub-second response times. We believe pg_duckdb will serve these use-cases nicely by overcoming Postgres’ known limitations in analytical processing. ... continues in article
Upsert is useful in several ways:
1. Data Consistency: It manages and keeps the data clean and free of redundancies. Any attempt to insert a new row of data into an existing table is modified into update commands when a conflict occurs.
2. Simplified Queries: Instead of writing separate INSERT and UPDATE queries, with Upsert, you can both cases in a single query thereby reducing the complexity.
3. Efficiency: Since a single statement can handle the process of inserting a new row or updating an existing one, the number of queries processed by the server can be reduced.
4. Atomicity: Upsert operations are atomic, meaning they will either fully complete or fully fail, ensuring the data integrity is maintained.
5. Error Handling: It prevents the processing from being halted due to error thrown if data already exists. In a large batch operation, this is especially useful.
Sharing our design inspiration for Hydra. Hope you enjoy!