Pyroscope and Grafana Phlare join together | Hacker News Reader

Pyroscope and Grafana Phlare join together

grafana.com

187 pointsburo921 comments

Hi! Tom from Grafana Labs here, super excited to welcome the Pyroscope team and can’t wait to see what they achieve.

Continuous profiling is a next big thing IMO - easier to get started with than distributed tracing and delivers immediate value.

This is exciting news Tom! Looking forward to seeing some posts in the future on how using Pyroscope reduced spend by finding perf improvements.

I’m really, really liking this.

I’ve been testing the whole stack on a local server, finding kinks, documenting workflows, because I hope I move us to this soon. Now with Pyroscope I would love to try it out even more. I kid you not, we were just testing Datadog Continuous Profiling for our legacy Ruby application not two days ago and it was quite lackluster.

Not saying this will be better (have yet to try), but I’d prefer to support and report feedback back on this for a brighter future for our community.

Keep it up!

Ryan here from Pyroscope -- Awesome to hear you're interested in migrating over. Would love for you to try out our Ruby integration and let us know what you think: https://pyroscope.io/docs/ruby/.

We've added a lot of features like tags/labels, integration with tracing: https://github.com/pyroscope-io/otel-profiling-ruby, integration with CI/CD (in rails), etc.

Feedback very welcome!

> we were just testing Datadog Continuous Profiling

Felix from Datadog here :). We'd love to hear your thoughts on our profiler if you're willing to share them. My e-mail is in my profile.

PS: Congrats to Ryan, Dmitry and team :).

Hey! It was alright, but at least for Ruby it didn’t report on the memory allocation of the calls, which is what we were looking for in the first place. We do not use it anywhere else, we only tried it for this pesky legacy system.

This is great news.

At the company I work at, OpsVerse, we offer a single-click, packaged, pre-configured OSS-driven observability stack [1] that makes heavy use of Grafana (UI + Loki) as well as Pyroscope so it's great to see them take the route of merging projects rather than building a whole new project. Continuous application profiling is such a game-changer and as eBPF becomes more popular, I'm sure tools like pyroscope will become bread and butter for devs.

As an addon, we recently published a blog on how we ended us dogfooding pyroscope to debug a pesky memory leak [2].

---

[1]: https://opsverse.io/observenow-observability/

[2]: https://opsverse.io/2023/02/09/pyroscope-to-the-rescue-debug...

Interesting!

What's the performance impact of continuous profiling in production for Node.js?

We've heard of 2-4% performance hit in terms of performance using pyroscope on Node, but in practice, we've seen ~1% hit in terms of perf (Though we're not running high-performance systems where we're trying to eke out every bit of perf from each pod, so YMMV)

Dmitry here (Pyroscope cofounder)

This sounds about right.

My favorite little anecdote that I like to tell is that the first thing people often see when they add Pyroscope to their apps is that it takes way less CPU than other signals like tracing or logging. It's pretty common to see logging taking 5-10% of overall CPU utilization.

The other 90% is usually spent doing serialization / deserialization (half-joking).

Pyroscope's license has already been converted from Apache to AGPL v3.

Two commits on LICENSE, second was today a647634d6c05db86cd6b066d31323456528f9bf0: https://github.com/grafana/pyroscope/blob/a647634d6c05db86cd...

Previous: https://github.com/grafana/pyroscope/commit/a539a5ff69a4a390...

Previous discussions when they relicensed Grafana, Loki, and Tempo to AGPL back in 2021: https://news.ycombinator.com/item?id=26877528

I've worked with Dmitry and Ryan - both are fantastic engineers. So happy to see this. Congratulations!

This is the last piece of the puzzle for migrating away from New Relic. Interested to see where this goes.

I wonder if Grafana is going to rebrand at some point. They have been aggressively diversifying, but keeping the Grafana name makes people immediately associate the company with the Grafana product and consider the rest as side-projects.

As an engineer, I care about the other products of monitoring: tickets, alerts, reports. My management? The people with the money? All they think about from that stack is the graphs they can view on Grafana.

Curious to know how much code is kept from Phlare? How did you merged two similar projects.

This is being worked out, but the gist is likely that we end up merging things rather than choosing between them.

Pyroscope have driven more of the clients and have a deeper understanding of the UI — so we'll probably bias towards Pyroscope for those — and Phlare has driven more of the scalable storage and database — so we'll probably bias towards Phlare for the database.

Both teams bring different perspectives of the same problem space, and the two products and their teams complement each other well, so we'll figure out how to take the best from each.

The biggest win is the people, the Pyroscope people are incredible and they join a few of our best people at Grafana Labs who were already working on Phlare — in doing so they create this high talent density with a strong purpose to make profiling be an essential, easy-to-use and scalable 4th pillar of observability. For me that is the real potential, to amplify the possibilities of both teams and products by bringing them together.

(I work at Grafana Labs, check my profile)

This is so good, congrats Ryan.

I've heard "new" Grafana is running something like chromium for server-side rendering under the hood. Is it true? (it's freaking me out a bit) Sorry if it's a silly question.

What is "new" Grafana in this context? The only thing that sounds similar to this is the Grafana Image Renderer plugin, which uses Chromium. But that's hardly new (it's been around for about 4 years), and it's not part of core Grafana...

Reply on news.ycombinator.com