Hacker News Reader

Comment by gizmo | Hacker News Reader

Do you have more info on this? iDRAC doesn't have the best security track record either, but people don't really seem to care.

View on HN

In a nutshell, the problem is this. I don't know whether this has changed, but this was true as of 2018 / 2019.

Most of their motherboards have IPMI with a separate a management port. A good number of them share IPMI management with the motherboard's primary ethernet port by default if nothing is plugged in to the management port. The motherboards have no way to configure them to NOT share the primary ethernet port beyond having the full stack of software needed to configure their IPMI.

What this means is that there're no jumpers one can change and no settings accessible in the BIOS that can force IPMI to stay on its own port, so if a BIOS gets reset, the battery dies or even just temporarily fails to provide power (like if it's being shipped by air and gets very cold), or you want to ship servers directly to a datacenter, the machine is 100% ownable on the public interface BY DEFAULT unless the management port is connected (and even then sometimes it decides to share the primary port - probably a function of link negotiation speed with the switch).

Sure, it's not a common occurrence, but it happens.

The solution for all the servers we already had deployed? We got ethernet loopback plugs for every one of them where the IPMI port wasn't already connected to a switch we administered.

A reasonable response: "Sure, that could be a problem sometimes. We can't change motherboards we already sold, but we'll bring this up with our design team so there'll be a jumper you can change so sharing will never happen, even with a reset BIOS."

Their response: "This isn't a security issue."

This! And their bmc is trash and openbmc only ships on few boards (arm ones iirc)

This is an interesting attack surface. Can you extend the risk out a bit? Assume that you have a vulnerable supermicro IPMI now exposed on a public interface. It has no IP address, and is presumably issuing DHCP DISCOVERs in an effort to get an IP. How do you reach the IPMI device to exploit it? What additional access do you need to get there?

Root on another device on that public network would do, you could forge the necessary DHCP responses to get it configured with an IP address of your choosing. Non-root on another device on that network might also work, if it fails DHCP and self-configures on a 169.254 address, assuming it does that.

Is there an obvious way to exploit such an issue from beyond the public subnet?Every attack I can imagine would be blocked by either inbound firewalls, or a failure to reach the IPMI as an unexpected device on the public subnet. I suppose that it would be a possible risk if you have a DHCP server on that public subnet issuing IP addresses to all devices, but that seems like a larger risk anyways. Server networks should be static assigned or static DHCP in all cases.

> This is an interesting attack surface. Can you extend the risk out a bit? Assume that you have a vulnerable supermicro IPMI now exposed on a public interface. It has no IP address, and is presumably issuing DHCP DISCOVERs in an effort to get an IP. How do you reach the IPMI device to exploit it? What additional access do you need to get there?

I think you misunderstood the level of fucked up this really was. The BMC device sits on the north bridge and literally scoops up packets from the main NIC which means it can even be accessible from the internets (if you didn't firewall port 623). See [0] for an example how variation of this unfolded.

[0] - https://www.zdnet.com/article/over-47000-supermicro-servers-...

I unknowingly did this, I found a random ip exposing the interface, and used admin/admin to compromise it - I was very confused as I explicitly did not plug in the ipmi interface as I do not want it.

I ended up using a PCIE nic, which ipmi does not auto bridge to.

There are plenty of cheap colos that do no filtering on their public networks. Some are saving money by putting a number of machines on a single ethernet segment, some are saving IPs by not having a /31 (or, much more often, a /30) for each client, and some both, so a compromised machine could easily run a DHCP server and scan any takers. You're right that no sensible network would forward packets to a misconfigured IPMI, though.

That still leaves very real things that've happened - the IPMI switches to the public interface and can no longer be reached on the managed local interface, and then you're rebooting several times in hopes it'll switch back and making aliases on a public interface to see if you can talk to it on the public segment. It's not professional at all.

Assume your network is not-entirely-trusted - like a college network where some curious students might be poking around and running network scans, and some others have malware on their computers.

The expected network port is plugged in, so even if your switch is checking MAC addresses, everything is in order. Most networks use DHCP, so the IPMI picks up an IP address. A student finds it with their port scan.

Interesting. I never use DHCP pools / random allocation on server networks; everything there is static assignment on the host, or static DHCP with whitelisted MAC.

> The motherboards have no way to configure them to NOT blah blah blah....

Most of your claims are false.

Super Micro has a utility to write the correct bits into EEPROM to disable this behaviour and stop the failover as default.

The utility was available years ago, prior to the time frame you state.

Any competent sysadmin would just build this into the deployment task sequence.

First, do you have a link to documentation for this ability?

Second, "any competent sysadmin" would have to know that this exists. Super Micro's security team didn't know this existed, or if they did, they failed to mention it in their response.

In the normal run of things, I'd tell you to do your own research.

But we're all Irish today and I'm in a particularly giving mood.

https://www.supermicro.com/Bios/sw_download/645/IPMICFG_User...

IPMICFG -lani 0

You're welcome.

(I do recall the syntax being a bit more cryptic, passing hex values, perhaps they've improved things since I last did this. Nevertheless, the capability has always been there.)

SuperMicro themselves not knowing this exists isn't surprising in the least.

according to that doc the functionality was only added late 2022.

Impossible as I was doing this nearly 10 years ago.

See my comment about remembering the process to be rather cryptic (writing hex values to address offsets) but the capability WAS there.

Perhaps they added that switch recently to make it more user friendly.

...but IPMI configuration isn't stored in EEPROM. It's stored in NVRAM.

And I believe you that you configured this pre-2022, but anyone could use the IPMI tools to configure this pre-2022 and pre- -lani option. You're trying to say it's in EEPROM, meaning it's invulnerable to battery loss. It definitely isn't.

Supermicro does offer board variants without the IPMI feature. I'd argue that most people who are buying the variants with IPMI are planning to utilize the feature..

The sideband feature also tends to be associated with an interface on the board that is considered the non dedicated IPMI "management" interface. Use one of the other onboard NIC ports or an PCI-E NIC like x550-T2, etc.

Indeed. An OOB interface is something you should always handle like radioactive material. It's volatile, powerful, and should be handled with extreme care and caution.

Reply on news.ycombinator.com