Application Traffic with eBPF

I have to say I find projects that talk about generic concepts (observability, tracing, eBPF), but then when you dig in the docs it's 100% Kubernetes-specific, to be highly misleading. Not everyone uses Kubernetes, and distributed tracing is a good thing to have regardless of the underlying platform.

For distributed tracing, how is deepflow able to correlate an inbound request (eg. client call) with an outbound request (eg. 3rd party API call required to service client call) without being inside the business logic?

The copy is way too buzzword dense imo

tcpdump only uses BPF, not eBPF. BPF is a simpler language that, among other things, is guaranteed to run in finite time because it doesn't have backward jumps, and has limitations on program size (4096 instructions). (The "e" in eBPF stands for "extended", as it extends BPF to remove those limitations, among other changes.)

It compiles your filter expression into a series of instructions, using libpcap. For instance, the output of `tcpdump -d -y EN10MB 'ip and tcp port 80' (which is rather similar to the use case in the OP, but not identical, since it doesn't strip headers), on my machine, is:

    # Load the 2-byte ethernet protocol into A (the accumulator register).
    (000) ldh      [12]
    # If IPv4, continue to line 2. Else, jump to line 12.
    (001) jeq      #0x800           jt 2 jf 12
    # Load the one-byte IP protocol into A.
    (002) ldb      [23]
    # If TCP, continue. Else, jump to line 12.
    (003) jeq      #0x6             jt 4 jf 12
    # Load the 2 bytes corresponding to IP flags / fragment offset into A.
    (004) ldh      [20]
    # If the fragment offset is nonzero, jump to line 12. Else, continue.
    (005) jset     #0x1fff          jt 12 jf 6
    # Load the internet header length into X. (Note that this is the bottom 4
    # bits of the first byte of the IPv4 header, expressed in 4-byte words)
    (006) ldxb     4*([14]&0xf)
    # Load the source port into A.
    (007) ldh      [x + 14]
    # If 80, jump to 11. Else, continue.
    (008) jeq      #0x50            jt 11 jf 9
    # Load the dest port into A.
    (009) ldh      [x + 16]
    # If 80, jump to 11. Else, jump to 12.
    (010) jeq      #0x50            jt 11 jf 12
    # Accept. Return 262144 bytes, the default snaplen.
    (011) ret      #262144
    # Reject. Literally, return 0 bytes.
    (012) ret      #0

If you know how to read assembly, it should be fairly straightforward to follow a typical program (you'll need various protocol header wire formats handy if you haven't memorized the offsets).

I'm not too familiar with eBPF (yet, keep meaning to dig into it).

But I believe tcpdump just opens a AF_PACKET socket, and it will add a BPF filter if one is specified.

I don't know enough to say how this relates to the eBPF stuff, but I think internally the kernel may convert the BPF program to eBPF.

Edit: netpeek looks cool, thanks for sharing!