Hacker News Reader

Fundamentally, any enterprise today has to deal with OLTP data, OLAP data (transactional plus other sources), Streaming and finally machine learning. Our premise is that you can choose to use 4 different platforms for each one or move to a unified platform. Spark offers streaming, it offers Spark SQL and there are a bunch of machine learning libraries available on Spark. Also, everyone who has anything to do with data has a connector to Spark so it becomes a good data integration platform. The API is uniform across these. So it forms a good substrate for what we are trying to do. As for bugs, it is a platform that is growing rapidly , going through some growing pains, and over a period of time, it will mature. We believe that the core capabilities will become powerful over time (SparkSQL, Streaming etc.) It is a somewhat opinionated choice but one that we think will pan out over time.

Also (correct me if I'm wrong), the stability of SnappyData will depend more on GemFireXD (related to Apache Geode), the in-memory database that has been integrated with Spark to form SnappyData, then it will depend on Spark. GemFire has been in development for over a decade and has a multitude production use cases.

LDAP and OLAP are different things.

In what way is LDAP related to this project?

(We run Spark in production and don't have the issues you are seeing at all)