Hacker News Reader

Comment by dreyfan | Hacker News Reader

Databricks is a rapidly approaching IPO. Trying to justify their valuation with their overpriced in-memory hadoop.

Databricks is way more than hadoop or spark. A great analogy - Spark is a great engine but you need to design and build all of the other subsystems.

Databricks is an F1 car - everything is built out. You get in and drive - FAST.

Databricks is a shit platform that encourages terrible data practices and accretion of technical debt.

Seems pretty good to us! Can you give more information?

As people noted elsewhere, you have to be VERY careful with using databricks for a full data warehouse due to the fact that it drives you to notebook driven development and scheduling of those notebooks when data pipelines should follow similar development practices as other software projects.

Great for proof of concepts, but when you start to build out complete pipelines please look into how to make the pipelines more sustainable and maintainable.

As with other offerings in this space, the key to managing technical debt is to get functions out of notebooks ASAP, stage intermediate results where appropriate, and turn everything into jobs.

Finally somebody that has used Databricks! I can't believe all the praise I read elsewhere in the comments here. Databricks is broken in so many ways, it is beyond me how anyone can like using this.

> Databricks is an F1 car

F1 cars really unreliable and need a lot of engineers to keep running, are very expensive, and completely impractical in normal use. They are fast but only on very specific roads, they couldn't survive on normal roads.

What do you know, you might be right! :D

You nailed it. Meanwhile the rest of the world just needs a camry.

> Databricks is an F1 car - everything is built out. You get in and drive - FAST.

found the databricks employee

Reply on news.ycombinator.com