Show HN: OpenMetadata – OSS platform for data discovery observability governance | Hacker News Reader

Show HN: OpenMetadata – OSS platform for data discovery observability governance

open-metadata.org

19 pointsjpnguyen11 comments

Hi HN,

OpenMetadata is an open-source metadata platform built to solve complex data management challenges and deliver high quality data assets. It’s based on the experiences of solving the data problems encountered at companies like Uber and Hortonworks. Our team includes the creators of Hadoop, Atlas, and Uber Databook, as well as committers/PMC of projects like Apache Kafka and Storm.

Unified Metadata Platform

* Unified: A single platform for data discovery, governance, and data observability.

* API-first: Standardized metadata specifications using well-designed REST APIs, crucial for anyone consuming or pushing metadata for modern data teams.

* Metadata platform: The data catalog is just one of the many applications of metadata built on the platform, vs different siloed data catalog tools, quality tools, governance tools, etc. that each have to be set up and maintained.

Please check out more details about our project at https://open-metadata.org and join our community in Slack https://slack.open-metadata.org. You can also try out our live product sandbox with demo data at https://sandbox.open-metadata.org/.

Open articleView on HN

How does this compare to DataHub? I have been comparing the two and I don't have any pros/cons that really help me pick. What I've noticed:

* It seems like DataHub has an async Kafka ingestion approach while OpenMetadata is API

* It seems that the data model of OpenMetadata can be extended/modified by API calls while DataHub only allows for changes to be made in code (in a forked version, no less) and deployed.

* It seems that DataHub comes with more data types out-of-the-box such as JSON Schema

* It seems like DataHub has an async Kafka ingestion approach while OpenMetadata is API

We do not use Kafka by default. If someone needs kafka they can add it. However for Metadata APIs, we do not feel like Kafka is needed. Lot of projects are getting dependent on Kafka and calling it as real-time. Its unnecessary burden on users who are going to operate in production for 99% of use-cases Kafka is not needed, coming from a Kafka committer :)

2. Yes all of our APIs and Entity definitions are generated using JsonSchema. For us, Json Schema has been awesome, all of our backend / ingestion and UI is generated from JsonSchema and its easy to extend and add new models when needed

3. IMO, we have much more coverage , you can look at the types available here https://github.com/open-metadata/OpenMetadata/tree/main/open... and we are support JsonSchema as a type from a long time

Fast growing project with a supportive community. A lot of connectors available and an UI that is smooth. Deployment is easy with a lot of possibilities to bundle with managed services (elastic, Postgres, external orchestrate like Kestra in my case).

Easy to build on top of it thanks a large range of APIs endpoints making it easy to query what's needed.

+ a lot of great contributors such as yourself :)

Glad to have you in the community!

Fastest growing OSS project in data cataloging space! Love the Slack community support and the business challenges the project aims to solve for Organisations!

wish I knew about this project sooner. so much easier to manage our data ecosystem with OpenMetadata. We used to have everything scattered all over the place. catalog here, docs on a siloed website, conversations in slack and for data quality we had a legacy process that was only accessible to engineers. With OpenMetadata everything is one place which means better collaboration and faster deliveries!

It is one of the fastest-growing OSS projects focusing on a great user experience. Not all open-source projects focus on UI, as the OpenMetadata team does.

Thanks for sharing the details!

Indeed fastest growing OSS project. I love the support and active slack community.

What is "data discovery, governance, and data observability"?

We explained the need for a centralized metadata repository here https://blog.open-metadata.org/announcing-openmetadata-20399... . Please give it a read.

Reply on news.ycombinator.com