Hacker News Reader

Comment by corford | Hacker News Reader

We're hitting some perf issues with Snowflake at work (not necessarily due to Snowflake itself but possibly more what we're trying to do with it: data warehouse storage needs but also a need for close to real-time analytical querying over that data). Has anyone here had any good/bad experiences with MemSQL?

View on HN

This use case sounds like a good match for MemSQL at a high level (analytics with an SLA is our bread and butter).

We'll also have a lot of elasticity features of snowflake shortly without sacrificing our performance advantages. (https://www.memsql.com/blog/the-future-is-bottomless)

(disclosure: MemSQL CTO).

Will "bottomless" have the ability to spin up multiple compute clusters on a centralized storage layer?

Yes

Orthogonal question, what happened to you guys? At one point you were the hottest start up on the block every ICPC competitive brogrammer wanted to get into and then you just fell off the radar..

Spark/Databricks + Delta lake would be a good solution for combining streaming and batch analytics. MemSQL is good for low latency streaming.

Have you considered using change streams to keep a "view" up to date? Do you have a good cluster-key defined?

Very curious about your performance issues?

We make heavy use of views within Snowflake, have sensible cluster keying and, further downstream, also leverage things like Looker's PDTs and Elastic Search.

The issue is we have billions of rows and very varied analytical requirements, so there are quite a few "pathalogical" queries.

Change streams are something we're looking at (as well as MemSQL, Clickhouse etc.)

What are your performance requirements? I've found Snowflake to be fast enough for most workloads, but if you're talking in the sub-500ms range, then MemSQL's in-memory capabilities will help. Though I must warn you that the system is a pain to manage and their managed service is unproven.

Haven't used MemSQL, but check out ClickHouse.

Reply on news.ycombinator.com