Show HN: Turn any data into a fast analytical API | Hacker News Reader

Show HN: Turn any data into a fast analytical API

columns.ai

26 pointscaoxhua9 comments

Open articleView on HN

What query engine are you using?

It's worth checking out Tinybird (https://www.tinybird.co/) if you haven't seen them. Might give you some ideas.

we use our in-house baked engine - open sourced here https://github.com/varchar-io/nebula

Yeah, Tinybird has lots of similarities, I will do more research on it, thanks for the reference.

Cool.

"All data will be loaded into our cloud cluster in distributed memory, encrypted under your credential."

So before uploading I encrypt my data? Then how does the queries work?

Or you encrypt it for me on your end?

Hey, thanks for your comment!

You don't encrypt, just leave your data where it is, what we need is a live connection, after loading your data in our distributed cluster in cloud, we have option to encrypt the data for further security if needed. So yes, we encrypt at our end if asked.

Our API service doesn't store data, only load them into distributed memory for computing and API serving.

Let me know if you're interested in this API service, would like to see if we can help.

Interesting concept. Small typo: "will be splitted into multiple tasks" s/b "split"

Thank you for the correction, appreciate it! I'll update it.

Our API service doesn't store data, only load them into distributed memory for computing and API serving.

I think you're suggesting that a queryable in-memory cache isn't storage. I don't understand that. What is it if it's not short term, non-canonical storage?

How is this different from Postgrest?

Fundamentally they both need to do computation to answer query, but they are very different, a couple of highlights: 1. API service invest a lot on reliable data ingestion/loading for users, you specify data spec, data will be served with lease term reliably. It takes care of all the data sync maintenance for you. 2. It works seamlessly with real-time streaming such as Kafka for RTA. PG is not suitable for this. 3. It's distributed nature + In-memory store guarantees low-latency API service.

Hope this describes a reasonable overview to think of it.

Reply on news.ycombinator.com