Hacker News Reader

Comment by burke | Hacker News Reader

As a non-expert, I still find this exciting. While eBPF may be more powerful, I find it far less approachable than DTrace. Perhaps this is just a failure to discover the right documentation. I use DTrace now and then on macOS and have resented that I can't really use it on Linux as well.

View on HN

eBPF lacks a high level front end, in part because that's optional. Engineering effort has gone into the essentials first: kernel development of eBPF, basic user-level frameworks to use it (bcc), and lots and lots of testing and bug fixes to make that rock solid. bcc is not as easy to code in, but I've been able to port over many of my DTraceToolkit scripts already.

We're only more recently looking at higher level languages, as a nice to have. Here's one project:

https://github.com/ajor/bpftrace

We can also look at taking the /usr/sbin/dtrace code (licenses permitting), and calling into libbcc and emitting BPF bytecode. That'd give us a D language front-end to BPF, so I can run my old DTrace scripts directly (plus sell more copies of my DTrace book -- I joke ;-)

Help on either of those projects would be appreciated (bpftrace or DTrace-BPF).

Am I reading too much into the last sentence, or are they actually trying to use existing probes with dtrace frontend:

> So with all those techniques now available in the linux kernel it will be exciting to see if dtrace for linux can unite them all.

With regards to tracing eBPF has actually moved at a fairly brisk pace the last year or two and it will probably take a while for the documentation and knowledge to permeate.

The usability side you mention is what Brendan's suggesting is worth porting from DTrace, just the front-end that the users sees. That would just back onto eBPF and the user would know little different.

IIRC one of the big things about dtrace vs systemtap is that dtrace could trace a process from userspace to kernel space and then back again. Can eBPF do this? How else do they compare?

That would not be recalling correctly: SystemTap does that, and so does eBPF (via uprobes and kprobes).

A large problem with SystemTap is that it was developed in the open, and in the early days people were scared off with its kernel panics. SystemTap built a negative brand: first impressions matter. DTrace, however, was largely built before its public release, and with the help from many other engineers and test teams. People's first impressions with DTrace was something that was already awesome. Of course, there are other problems with SystemTap: its process of compiling code and loading kernel modules was always risker than an interpreter.

As for how eBPF compares: I summarized it last year http://www.brendangregg.com/blog/2016-10-27/dtrace-for-linux...

> and in the early days people were scared off with its kernel panics.

It is ironic that to this day those kernel panics are associated with systemtap (the visible user interface) rather than the kernel (where the vast majority of the responsible bugs actually lay). But perceptions indeed matter, even erroneous ones.

Note for freshness that systemtap can also emit eBPF directly now. So kernel modules are the fastest & most powerful way, but not the only one.

Thanks. How does the stap language being compiled into kernel modules differ from an interpreter? Both seem to have a limited surface (unless you're doing C directly in systemtap)

An option for correlated kernel and userspace tracing on Linux is LTTng. [0] For a comparison of LTTng and other tracers, check out the LTTng docs. [1]

[0] http://lttng.org/

[1] http://lttng.org/docs/v2.10/#doc-lttng-alternatives

Reply on news.ycombinator.com