Show HN: Credentials dumper for Linux using eBPF | Hacker News Reader

Show HN: Credentials dumper for Linux using eBPF

github.com

235 pointscitronneur50 comments

I love the animated GIF you included in the readme that shows two sessions at once. It made it perfectly clear in moments what it does, and how easy it is to use.

Thanks!

Related: TripleCross - A Linux eBPF rootkit with a backdoor, C2, library injection, execution hijacking, persistence and stealth capabilities.

https://github.com/h3xduck/TripleCross

You have also https://github.com/pathtofile/bad-bpf or https://github.com/Gui774ume/ebpfkit which are good references also

Embrace the red also has a couple of good write ups to learn Offensive eBPF:

https://embracethered.com/blog/posts/2021/offensive-bpf/

Great clean example of using libbpf.

Latest of libbpf (which seems like you vendored) comes with ability to calculate symbol offset for you. Thoughts on using that instead of your custom logic?

Thanks I will check that!

> built as a static binary without any dependencies

> As pamspy rely on libpam, we have to set the path where libpam is installed on your distribution.

Confusing text in the readme. Does it have dependencies or not?

I read this as, due to pamspy setting an eBPF probe, pamspy needs to know where libpam.so lives. Not that the pamspy needs libpam to be built

Exactly, we have to found the address to hook on the system, so we need the path of the currently use of libpam by other process

Oh, makes sense, thanks!

It is still quite confusing.

> built as a static binary without any dependencies

Static binaries are explicitly used for removing the need for specific dynamic runtime dependencies. It does not refer to build dependencies, which are not interesting here.

Based on the terms, I would except that libpam is included for the final binary.

The entire point of this program is that it hooks the func inside the libpam.so actively being used by the system for auth...

If libpam was compiled in, then this tool would do nothing. libpam is not a library for this tool, it's a target, like an input file. libpam is a library for the kernel of the target system. this tool hooks into it to do its work.

Exactly, it is the target. The later phrase pointed out in the original comment it to be some sort of dependency for runtime use, making the confusion. While it is not related to runtime code functionality at all.

When the author says it “has no dependencies” they are referring to build time dependencies (i.e. development headers) and runtime library dependencies (dynamic libraries that will be linked and used at runtime).

In this case the function of the program is to hook a library function in `libpam` using eBPF so it has libpam as a “dependency” in roughly the same way that a program which converts wav to mp3 depends on “the input wav file”.

Given that this is a somewhat unusual way to depend on a `.so` file it’s reasonable for there to be some ambiguity in the language here.

You could say libpam is the "target".

Like pointing a disassembler at a shared library, it's not needed to run the disassembler, it's the thing you're disassembling.

Am I the only one for whom it doesn't work? the reason being that using PARMx registers on uretprobe has no guarantee that the registers still hold the argument values, as these registers can be clobbered at will (and are for me).

The solution for this is usually to track both probes and remember arguments, here's what it'd look like with bpftrace -- that does the same as his program, I've just hardcoded the offset for username in pam_handle struct but the repo hardcoded the struct (it's also possible to include a .h in bpftrace to stay up to date)

  bpftrace -e 'BEGIN { printf("pid,comm,user,pass\n"); }
    uprobe:/lib/x86_64-linux-gnu/libpam.so.0:pam_get_authtok {
      @user[tid] = arg0;
      @pass[tid] = arg2;
    }
    uretprobe:/lib/x86_64-linux-gnu/libpam.so.0:pam_get_authtok /@user[tid]/ {
      printf("%d,%s,%s,%s\n", tid, comm,
             str(*((uint64*)@user[tid]+6)),
             str(uptr(*@pass[tid])));
      // just illustrating arg2 (rdx on x86_64) changed:
      printf("%lx, %lx\n", reg("dx"), @pass[tid]);
      delete(@user[tid]);
      delete(@pass[tid]);
    }'

I made a little fix, if you want to retry

Thanks! This happens to work for me, but the root of the problem is the same: there's no guarantee that the handle will still be in that register when the function returns; my compiler just happens to not be clobbering that register.

Anyway, this is probably good enough as a bpf demonstration and it definitely has made its impact looking at other comments here. That's probably all that matters.

Can't you achieve same thing with uprobe[1]?

[1]: https://brendangregg.com/blog/2015-06-28/linux-ftrace-uprobe...

You need uretprobe but also need to read an arg by ref so no I don't think so... But thanks for the tip

eBPF is one of those things that I feel like I ought to get in to but havent found the time yet. Great app, was it your first venture in to eBPF?

Yes we also use for https://github.com/airbus-cert/dirtypipe-ebpf_detection which is a dirtypipe detection program!

So is this an exploit? Or are root privileges on the local machine needed to run it?

Root is still needed, so not an exploit. Still a simple straightforward example on how to use eBPF/libbpf to grab returned data from a userspace function call

It appears that you need root, at a minimum the demo gif uses sudo to run the program. At an absolute minimum you would need CAP_BPF[0] to execute the eBPF.

[0] https://man7.org/linux/man-pages/man7/capabilities.7.html#:~...

Pretty much the same as loading an unsigned or untrusted kernel module, someone would have to get it loaded from a privileged account.

I've heard `eBPF` described as "like JavaScript for your kernel" if the kernel itself was being related to a web browser that runs embedded scripts, so, that should give an idea of how much and what type of power it brings, as well as the expected access level to be able to take advantage of it.

Other uses I've seen for eBPF are inspectors that tell what is happening on encrypted connections and the request headers for any connection, including authentication details that you would expect to be protected. It's great to have this kind of capability on systems that you own!

This is not an exploit in itself, but could be very useful for pivoting and privilege escalation (across the network). You have to have already achieved root on the target machine, but once you have obtained that you want to start pivoting to other machines which may not have vulnerabilities you can exploit.

The first thing I usually do is dump the /etc/shadow file and start up hashcat on it. However this is a very slow and often unsuccessful approach. With a tool like this, I would still dump the /etc/shadow file but I would also fire this thing up so I can obtain passwords as people log in.

The reason this is useful is because most people reuse passwords across other systems. If I can get the password they use for this system, chances are I just gained access to other systems. The mitigation/defense against this is to always use unique passwords. I'm already root on this box so getting your password benefits me nothing if it's a unique password that you haven't used elsewhere.

Whoa, awesome work OP this is super neat!

Not only is it an actually useful tool for pen testers, and a remarkable PoC for abusing eBPF, but it's a sweet and simple example for how to write an eBPF module.

Back in the days at some point I realized how powerful strace was, and ran sshd with it. Fun times. But you can also just sniff (or snoop it was called back then) all input done on a tty.

Well done!

thanks !

Does it grab ssh passwords? (Not sshd password) when a user runs ssh from the target server itself to other servers

Doubt it, but only for lack of trying. As the readme says this hooks into libpam, which wouldn't be used when ssh client makes an outgoing connection, unless the outgoing connection is to localhost. However, the method used in this should also be usable for hooking into an ssh client.

Usecases? What was it conceived for?

Lateral movement for example

My ignorance, I had no idea eBPF tracing would make grabbing people's passwords so easy .. that's quite scary to me. I thought it was mostly good for telemetry and deep kernel metrics, but this seems like a serious security flaw to me.

Anyone know of any tools to check for abuse?

Unless you enable unprivileged eBPF, root is required to load a module. If a user has root there are plenty of ways to get passwords.

Understood, and agreed once you have root access you can get passwords but I've not seen many that are this easy, and I'm now thinking there's something I need to understand about how to detect if certain traces are happening so I can detect a potential breach.

Also seems prudent to get rid of passwords and move to Kerberos and SSH keys + 2FA. Anything else I'm missing?

> Also seems prudent to get rid of passwords and move to Kerberos and SSH keys + 2FA. Anything else I'm missing?

This is a good path to go down anyway, despite the fact that Kerberos, for instance, is totally susceptible to 'pass the hash'[1] type attacks. Concentrate on things like Yubikey-based authentication. You can do SAML/OIDC2/mTLS and SSH with Yubikeys.

Eliminate passwords.

[1] - https://www.beyondtrust.com/resources/glossary/pass-the-hash...

Looking for specific traces like this is a difficult (but still worthwhile) way of detecting a breach, as there’s potentially infinite things a root user could do once they have that level of access.

Another approach is focusing on detecting the privilege escalation in the first place. You can use normal auth logs in Linux alongside things like auditd, or more complicated EDR tools that look for suspicious system calls etc to identify root logins that are suspicious, or when a process might have been exploited and elevated to root. Make sure you’re shipping your logs somewhere remotely so they are protected from tampering.

If you use GitHub, you have your public keys available: https://GitHub.com/withinboredom.keys. Replace my username with yours or whoever.

There’s an option in sshd to run a program that should output the contents of an authorized keys file: AuthorizedKeysCommand

So you write a simple bash script or program to output authorized keys based on your own rules. If you want stronger auth, check out libnss-ato which can allow you to masquerade as root if the user is authorized. (In your authorized key script, check if the user is in your org and/or part of a certain team, if so, output their public keys, otherwise, output nothing).

I really should open source my code, but it’s literally only 5-6 lines of code, and 3 lines of configuration.

Anyone know what the status is for enforcing signed eBPF programs?

Why? eBPF is usually compiled at runtime (so there’s no binary to sign) and running it inside your kernel requires root.

Running eBPF programs doesn't necessarily require compilation at runtime nor root privileges. Look into bpftool's skeleton generation as well as CAP_BPF.

With that being said, because eBPF programs can be compiled at runtime, it makes signing eBPF programs trickier. The kernel team doesn't want efforts such as bpftrace to be stifled.

It seems like the conversation on signing eBPF programs is still ongoing with an eye at looking at fsverity to help with the use cases here.

Hmm I see. I’m still not sure what’s the use case and threat model.

Is this all for Secure Boot just like signed kernel modules?

The threat model is that I want to deploy ebpf programs to my base amis and let devs load them as-needed without root, basically.

Does loading the eBPF programs and then letting devs attach them later work for your use case?

I wrote about the possibility of this with fd passing in a recent blog post: https://mdaverde.com/posts/cap-bpf/

I'm also working on agent that allows for this at https://bpfdeploy.io/

Reply on news.ycombinator.com