You should take a look at https://github.com/openobserve/openobserve - Extremely performant and simple full-stack observability solution.
266 karma · joined November 12, 2018
handle? What does it mean? Be able to ingest data? Be able to query?
ingest data - Using kafka helps only during ingestion for handling the spike. query data - Kafka has no role to play in it. Querying performantly at scale is a hard problem. I do not doubt Mimir's capability in being able to query high volumes of data, but other systems can do it too and OpenObserve's internal benchmarks show that it's querying is much faster at scale than Mimir and we will publish it at the right time (We don't just publish benchmarks to satisfy plain curiosity of people on internet), but this is not about OpenObserve so let's push it aside for a while.
About - how many active timeseries We've built OpenObserve with a fundamentally different architecture. We don't have the "active timeseries" constraint that Prometheus-based systems do. High cardinality isn't an issue by design It's a topic for another day though.
The primary function of a message broker is to decouple producer and consumer so writes can happen efficiently (consumers do not get bogged down by high incoming volume). Something like Kafka allows that very, very well, and it is one of the best systems designed to do it. It allows massive volumes of ingestion reliably without dropping packets. It's a beast on it's own though.
Kafka was also built in an era when autoscaling was not available (Still very relevant though and will be for a very long time). Autoscaling to a great degree can allow you to handle write spikes (It's not the same thing but can attack the same problem from a different angle) and extreme spikes will still require a message broker. Horizontally scalable does cut it to a great degree though.
Having architected massive systems for multiple large companies, I can argue about technology for a long time, but the only point I want to drive is to avoid the use of words like "period". Mimir's architecture makes sense but it's not the only solution that works at scale, and the operational complexity has real costs. There are no absolutes in tech as in life.
I love it when people take a hard stand like this, using the words "period"
BTW, Cortex is used as Amazon Managed Prometheus (Probably at a much larger scale) than Mimir by AWS.
OpenObserve too, is already being used at a multi-petabyte scale.
Disclosure: I am a maintainer of OpenObserve
This should cover all companies with 10 developers.
OpenObserve clusters can ingest PBs of data every day. While more can be discussed - I would rather focus on what are your needs when it comes to observability. Let's talk about them. I will be more excited to answer those questions.
We run benchmarks internally and will publish them once we are ready for it, but are unlikely to benchmark someone else's product. Benchmarking someone else's product always leads to a conversation similar to - You ran your benchmark in the most optimized way but used our non optimized settings or a version of that.
People who use OpenObserve like it for it's ease of setup, ease of management, high performance and rich feature set.
Especially for logs, grafana and loki is no match in terms of features and performance when it comes to OpenObserve. I will let you test it if you are curious and have some spare time.
OpenObserve is used by people ingesting MBs of data in their basement to PBs of data in large clusters in AWS, Azure, GCP and other cloud environments.
BTW, here is a story of a large EV company who moved to OpenObserve for traces and increased performance by a factor of 10x and reduced their cost at the same time - https://openobserve.ai/blog/jidu-journey-to-100-tracing-fide...
Not all managers are averse to paying, but many are. I have had discussions with Director/Sr. Director and VP level folks in these companies. I have been paid and I have been denied.
Our biggest customer is a fortune 10 company and we are able to offer the kind of support that they need. It indeed takes a lot to provide that kind of support, though, and would be difficult for most small open source projects to do.
For those looking at much more simplicity, and much higher performance OpenObserve is the way to go. Many folks have moved from Loki to OpenObserve due to performance issues with Loki. Many have moved from LGTM stack completely to OpenObserve. Many have chosen to use Grafana as a front end for OpenObserve too.
Take a look at how easy it can be to build dashboards in OpenObserve.
It takes time for community and ecosystem to build for great products. Grafana started in 2014. OpenObserve started in 2022.
It's not "only 28 MB/Sec/Core". Try doing same with Splunk/Elasticsearch - You won't go past 5 MB/Sec/Core (Typically it will be lower) on their best day.
On our part OpenObserve offers free SSO on our cloud service to anyone and Free SSO for anyone using enterprise version if they ingest under 200 GB/Day (6 TB/Month).
How many open source log systems work at PB scale given any number of resources? Also FWIW, OpenObserve can ingest data at 28 MB/Sec/Core (We are working on optimizing it even more) and ingesting 1 PB of data would cost just $435 based on on-demand prices (AWS m7g family).
We have spent over 2 years building OpenObserve into a simple, highly usable and efficient observability tool. You could run it using a single binary that provides all the functionality of logs, metrics, traces, front end monitoring, dashboards (18 different chart types), alerts and pipelines.
OpenObserve is being used by startups, mid tier enterprises and fortune 100 companies. There are thousands of active installations of OpenObserve globally.
Folks have replaced Elasticsearch, Splunk, Graylog, Datadog , Newrelic and more for OpenObserve.
Comment from a user -
We moved from 5 node OpenSearch cluster to single node OpenObserve and measured using our actual everyday queries, which are reasonably complex queries (1 to 5 conditions applied) over our real logging data. We see that typically they complete in about the same time. OpenObserve costs us 10 times less though (instances + storage)
Also, we are currently working on replacing one of the world's largest splunk installations.
p.s. I am one of the maintainers of OpenObserve. Feel free to ask questions. I will be happy to answer them. You can also visit our slack workspace at https://short.openobserve.ai/community for discussions.
Pinot is cool.
OpenObserve is similar to Pinot but built in rust with Apache arrow Datafusion as the underlying technology and for a different, targeted and tailored use case of Observability.
Going back to the basics - 12 factor app principles must also be adhered to in scenarios where opentelemtry might not be an option for observability. e.g. Logging is not very mature in Opentlemetry for all the languages as of now. Sending logs to stdout provides a good way to allow the infrastructure to capture logs in a vendor neutral way using standard log forwarders of your choice like fluentbit and otel-collector. Refer - https://12factor.net/logs
OTLP is a great leveler in terms of choice that allows people to switch backends seamlessly and will force vendors to be nice to customers and ensure that enough value is provided for the price.
For those who are using kubernetes you should check the opentelemtry operator, which allows you to autoinstrument your applications written in Java, NodeJS, Python, PHP and Go by adding a single annotation to your manifest file. Check an example here of sutoinstrumentation -
/-> review (python)
/Check for complete code - https://github.com/openobserve/hotcommerce
p.s. An OpenObserve maintainer here.