Hacker News Reader

Comment by satuke | Hacker News Reader

how is it different than s3?

Sounds like they provide a caching layer in between

"Every blob uploaded is stored in-memory to provide fast access — and then blobs that are not read frequently are eventually automatically moved to disk for low-cost, long-term storage. When you make a new read request to a blob in disk, it is reloaded back to memory, providing you the fastest possible access again."

For a developer-focused tool, it strikes me as odd that they wouldn't just use the term "caching" though?

Yes, I agree with your point. It reminds me a little of the ongoing debate over CDN vs Edge. However, one difference we offer is that eventually our 'cache' will flush to a persistent storage layer - so you don't have to think about managing your memory or disk resources. Data not used for weeks ends up in low-cost object storage - further saving you from high-cost memory storage costs and the repetitive task of performing this archiving operation yourself.

> eventually our 'cache' will flush to a persistent storage layer

What happens if the server goes down before the flush?

When a blob is saved, first I write it to Postgres (https://neon.tech), so that ensures there's a persisting backup. However, it's typically a waste of money to store infrequently accessed blobs in disk with Postgres over months and years. After 4-6 weeks data is offloaded to object storage - so that you benefit from low long term storage costs.

The lifecycle of a blob works out roughly something like as follows, based on last read date:

< 30 mins ago

- In-Memory: Cloudflare CDN

< 30 days ago

- In-Disk: Redis Auto-Tiering Memory/Disk combination

- In-Disk: Postgres

> 30 days ago

- Object Storage: Backblaze B2

Reply on news.ycombinator.com