OpenPOWER Foundation announces LibreBMC, a POWER-based, fully open-source BMC | Hacker News Reader

OpenPOWER Foundation announces LibreBMC, a POWER-based, fully open-source BMC

openpowerfoundation.org

211 pointsrbanffy55 comments

Is it written in FORTH?

https://en.wikipedia.org/wiki/Open_Firmware

If you look at one of the earlier x-series servers- x3650-M1 I think, the BMC (IBM calls it an IMM) is daughter board with a PPC 405 chip. The artwork has "Proudly made in North Carolina" with a map of North Carolina in etch. You can just see it in the picture, under the "IBM":

https://admirestore.top/other/ibm-x3650-rsa-telecharger-pilo...

Anyway, I wonder if the code is related. I would have thought this all went to Lenovo.

What you really need for an open-source BMC, is one that works with the common Aspeed KVM chips (ARM-based I think, look at AST2300 and above). This would have the nice effect of avoiding having to pay AMI for their BMC code.

Edit: Actually here is the code, it supports Apseed:

https://github.com/openbmc/openbmc

What's new in LibreBMC is the hardware toolchain. Frankly, they should target RISC-V for this. Edit again: it does, using LiteX:

https://github.com/enjoy-digital/litex

I'm not sure where Power fits in..

I don't know why FORTH would come into it. OpenBMC does a different job to Open Firmware (Sun Openboot v. ILOM). OpenPOWER boots through Linux -- or at least some of it does.

What I really need from a BMC is properly-working and secure IPMI. I don't admin them, so I don't know how how the implementation on the AC922s I use holds up, but I've only had poor experience with proprietary BMC software in the past, and there's considerable appeal to being able to fix it.

Why should POWER systems use RISC-V rather than their own free cores?

They're replacing openbmc, thankfully - the build system is absolutely ridiculous. The motivation is to escape binary blob ARM world. You know that POWER and RISC-V are competing solutions, right?

All I remember about OpenEmbedded/Yocto was trying to wrap my then teenage brain around it a decade or so ago when it was the build system / distribution of choice for Beagleboards (via Angstrom).

Ended up going with buildroot instead at the time...

Seriously, nothing like spending a day building an image in a way documented only to see it blow up and fail, resulting in having to redo the whole thing.

Wish Nix were more popular building embedded system images, the tooling is capable of doing everything bitbake and yocto do, with far saner usage.

> They're replacing openbmc

The press release says they're planning to use openbmc. The "LibreBMC" part seems to be about an open HW platform (incl. created with FOSS tools) for running openbmc.

Or are you saying openbmc is planning on some major refactoring?

The statement was "run software from OpenBMC"... so that could mean almost anything - anything except the conclusion to seem to have drawn. My guess would be that it means they'll grab some of the python code and carve up the systemd scripts. I've been living with an openbmc equipped system for a year or so - there is a lot about it that I won't miss, especially the way it handles serial communication.

> there is a lot about it that I won't miss, especially the way it handles serial communication

Out of curiosity, what were the issues you had with OpenBMC's serial communication?

It was eating characters, which is the second worst thing it could do. Digging into the source, I found a really wonky software buffer... which kind of blew my mind - since hardware buffers and flow control exist. I don't remember if that was a case where python was somehow in the pipeline, but that was unfortunately very common in OpenBMC.

I also had a lot of trouble getting the thing to honor the most basic settings, like baud rate. But it is harder to confidently place the blame on that one. Is it OpenBMC's fault that the software provides no clue as to where things are going wrong in the serial plumbing? Is it the AST2500's fault that designers are routing every serial line through it and making it impossible to eliminate as the cause while troubleshooting? Is it IBM's fault, because the design of OPAL (OpenPower Abstraction Layer) encourages construction of this kind of Rube Goldberg machine?

If anyone from IBM is listening: don't route all your serial lines through the BMC on your reference designs... system implementers will just copy it.

> It was eating characters, which is the second worst thing it could do.

Okay, that's interesting. Did you file a bug with your system's firmware provider or with upstream?

You can file it against obmc-console at https://github.com/openbmc/obmc-console/issues

> Digging into the source, I found a really wonky software buffer... which kind of blew my mind - since hardware buffers and flow control exist.

Yep, it's how OpenBMC provides the console via IPMI, Redfish, SSH and on the commandline, as well as routing the console out to the connector on the rear of the chassis.

> I don't remember if that was a case where python was somehow in the pipeline, but that was unfortunately very common in OpenBMC.

Generally there hasn't been any python in the console handling pipeline. That said, python, while especially slow on a BMC, was pretty important for getting the project off the ground.

At this point OpenBMC has been python-free for several years.

> I also had a lot of trouble getting the thing to honor the most basic settings, like baud rate

There's some nuance to this, as depending on your system design the console may be coming from the host to the BMC via Aspeed's Virtual UARTs (VUARTs). The way the VUARTs work is the BMC and host are connected to either end of the two FIFOs between the UARTs' APB and LPC/eSPI interfaces. As such there's no baud rate as no data is clocked out in RS-232 fashion - the data is transferred as quickly as either side can access their register interfaces.

This has caused issues in the past with control flow that lead to data integrity issues:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...

> But it is harder to confidently place the blame on that one. Is it OpenBMC's fault that the software provides no clue as to where things are going wrong in the serial plumbing? Is it the AST2500's fault that designers are routing every serial line through it and making it impossible to eliminate as the cause while troubleshooting?

I agree it can be difficult to debug where things are going wrong in the console pipeline.

In concept a serial console should be simple, but when you bring in requirements like various kinds of SoL, it starts to get more complex.

> Did you file a bug with your system's firmware provider or with upstream?

Nope, I patched my code to mainline the bits directly to where they were going.

> ...it's how OpenBMC provides the console via IPMI, Redfish, SSH...

Which is why I didn't bother with upstreaming :) I'm all for it generally, but in this specific case I saved myself and others time (distilling a reproducible test case, crafting code for public consumption, arguing my case, etc); I'm not the target market - I'd cut the BMC traces if I could without wrecking the machine.

> ...was pretty important for getting the project off the ground.

Sure, but far too close to the metal for my liking. But then I've always bristled at the sight of it being used to replace far more constrained tools - where constraint is desirable (like build systems).

> There's some nuance to this...

That was mercifully high-level, but touches on an major concern of mine that is more to do with Aspeed than OpenBMC: backdoored blobs. The AST2500 is constantly sniffing a software defined uart for a magic sequence to drop it into the debug shell... this is the opposite of what I'm looking for in a secure or performant system.

> I agree it can be difficult to debug where things are going wrong in the console pipeline.

Yup, without even considering the client hardware/OS/application stack, or the getty process on the host you actually want to talk to. I've actually been researching a bug that made it into BSD4, the fountain head. Nobody seems to have noticed that a bunch of terminal attributes, when set from gettytab, stopped having any effect 30 years ago (aside from a single potentially related unanswered complaint on Usenet in 91).

http://students.engr.scu.edu/~sschaeck/misc/aspeed-edac.html

Which soft-core Power are they using? They mention Lattice ECP5, so a fairly small one I assume.

An OpenPOWER one, likely this with modification: https://github.com/antonblanchard/microwatt

Microwatt looks nice (though I wish it was Verilog), it has a floating point unit which most of the low-end RISC-Vs do not.

Also: "Anton Blanchard", Distinguished Engineer: IBM Total Duration 20 yrs 3 mos

Well the same guy (yes, he is a madman) did another one in Chisel, which is one step removed from Verilog: https://github.com/antonblanchard/chiselwatt

I don't spend enough time with FPGAs to even pretend to have an educated opinion on HDLs.

Yeah, they are semantically equivalent, but I'm more comfortable in Verilog. It used to be that the open source situation for VHDL was not as good, but it's clearly improving.

Probably entirely unrelated, but Sun T series (definitely T5xxx, not sure about T1/2 series) ILOMs were Linux on PowerPC. Always found that amusing.

Sun Fire V20z (AMD Opteron) servers are the same.

Technically unrelated, but it does beg the question of why OpenPOWER and friends haven't used any of the embedded PPC cores...

Before OpenPOWER, IBM Power servers used a PowerPC-based don't-call-it-a-BMC. Either IBM decided not to release that chip to the outside or maybe Google didn't want it.

All the existing embedded PPC cores are targeted at network or storage applications; emulating a terminal / console / graphics card needs a bit of a different arrangement (at least if you want it to be compatible with some existing stuff.)

It does, it is used for soft realtime related stuff (power, fan, errorlog, etc). Throwing BMC related activities in there is certainly doable, but it would complicated things to the point where another chip would be tempting.

I mean, the OCC and some of the other on-chip cores are based on the 400 series cores, but what I mean is there were/are tons of SoCs from NXP almost purpose built to serve as BMC-style control chips, with reasonably powerful cores and all the peripheral I/O it'd need. These were used (AFAIK) in Sun servers, so I wonder why not in OpenPOWER.

I'm guessing cost/availability is really what it boils down to, since the ASRock chips are probably just that much cheaper and better understood.

I imagine Google was already using ASPEED BMCs in their x86 servers so they kept using it in OpenPOWER. There are some really specific things like PECI and VGA redirection that x86 BMCs need and AFAIK nothing besides ASPEED has those features.

I doubt IBM gave Google any thought in this matter. I'm pretty confident that it has a lot more to do with the way IBM does their market segmentation, and how that is related to hardware + software bundling and firmware. The ARM BMC fits in a HMC shaped hole.

https://en.wikipedia.org/wiki/IBM_Hardware_Management_Consol...

Are there fully open ones?

How about Pi-KVM (https://pikvm.org/)? Secure, flexible and extendible.

A KVM system is not a BMC. Running a cluster (even a fairly small one) you probably want power control, e.g. with powerman, serial console availability, e.g. with conman, metrics, e.g. via freeipmi and some monitoring solution, and probably logs for alerts. Control and monitoring need to be out of band.

(Powerman, conman, and freeipmi come from Livermore for use with serious HPC systems.)

I might be completely missing something here, but I don't seem to spot any relation to the OpenBMC software payload that is currently available for certain existing BMCs. Perhaps they are not looking to integrate the two, but it would seem like a lot of duplicate work if you create new hardware but then not use existing open software to power it (aside from the FPGA bitstream of course).

The press release notes that they expect to run the OpenBMC software on this LibreBMC hardware

Ah yes, I see it now, right at the end of the page (and it's even in the tags). I think I got too excited and immediately started tabbing to their hardware repos for the two FPGA designs and that LiteX tooling before reading the last two sentences.

I hate this name. “Libre” anything makes me think it’s some FSF/GNU project. OpenBMC was taken, I get that, but there has to be a better name for this.

GNU software is more likely to be "free" than "libre", like FreeIPMI in this context. (As far as I remember, that was originally part of Gluster, the GNU cluster system with existed for a while before the filesystem took over. At least we have good free software for cluster management now, apart from needing to interact with proprietary stuff like Nvidia's, sigh.)

This must get points for Respects Your Freedom anyway.

I don't think so. LibreOffice should be most popular /^Libre/ OSS.

The page takes over scrolling. What is this called? It is quite annoying.

Hope this is already influence of RedHat on IBM.

IBM began this initiative in 2013.

It seems mostly OCP-based, but it's possible that the choice for POWER cores is indeed an IBM thing. On the other hand: it doesn't matter as much since the design assumes swappable management cards.

wow this reminds me of metal gear solid for some reason

I wish they disclosed a country of origin for each organisation participating in the project. I had to click through a lot to find out some of them are based in countries with appalling human rights track record. Now I am thinking whether the agenda to bring open computing is genuine or is it a front to make dictatorships independent from western technology?

> Now I am thinking whether the agenda to bring open computing is genuine or is it a front to make dictatorships independent from western technology?

I mean, technological independence from the US should be a desire of every country that _isn't_ the US, as the US gives no rights to foreign nationals or their data, has a history of backdooring products and is suspected of making some of the most sophisticated malware the world has ever known.

Well, in its history IBM was aiding the Nazis with concentration camp infrastructure - making the whole bureaucracy much easier. I'd rather have technology restricted for certain countries at risk so anything like this won't happen in the future.

You've positioned it now so that anything I say will sound sympathetic to Nazis.

But what if the USA was WW2 Germany, and they had dominion over your entire technological infrastructure.

I'm not necessarily saying that's the case, but I'm not so quick to throw in pure unadulterated trust to any foreign country, and ultimately, USA could decide the rest of us are bad for nearly any reason, just look at how much pressure the US exerts on Sweden, which is hardly known for human rights abuses.

I am sorry, I didn't want it to sound like that. I just wanted to express my distrust for IBM as they have not exactly addressed their past and such initiative is potentially sinister.

I understand that the US has its fair share of atrocities, however for me they are the "lesser evil" if that makes sense.

How is comment not whataboutism?

Why not both?

If you have reason to believe that some state-equivalent espionage group implants malware in machines headed in your direction, it is to your advantage to defend against that.

It seems to be the case that human-rights advocates as well as dictatorships have been such targets.

It's not so much about spying on western infrastructure, but about using such technology for genocide, racial profiling (for example restricting certain ethnic groups movements) and the whole police state infrastructure to keep population "under control".

How does an open standard remote management card enable genocide or whatever? Non-open standard management chips are cheap and off the shelf and work ok; or you know, just have a tech in the datacenter with a kvm cart; it's not as nice, but not really a big deal.

I also don't see a huge difference in genoicde capability between 2010 servers and 2020 servers, but 2010 servers are eWaste. Restricting genocidal organizations to ten year old technology would raise their power bills, but not diminish genocidal capabilities; and restricting much more would require immense effort on a global scale. (And, of course, you would need to get near-global consensus about the country, which is quite difficult)

I do get why you were downvoted; because it's easy for readers to assume that open source logically HAS to mean open to every dictator, too. But it doesn't. You're right about that.

We could have "friendly nation" or "no pirate nation" open source licenses that do not give North Korea, Iran, Russia, etc massive free software gifts to help them oppress their citizenry. Logically, there didn't have to be a North Korean Linux (which there is.)

Such licenses wouldn't extend any rights to citizens of rogue countries, or to non-citizens while in such countries, nor entities (including companies) with control or ownership ties to rogue countries, or, say, countries which jail or execute LGBTQ. (At the very least, we don't have to extend a license without payment and contracts.) There are risks to friendly-nation or no-rogue-nation licenses, granted; but there are nasty consequences to handing more and sharper knives every year to rogue countries, too.

Whether one likes it or not, an executive order from Biden could perhaps make that happen tomorrow.

Seems rather pointless since it would be totally unenforcable. As if North Korea, the #1 producer of counterfeit US currency worldwide, is going to respect IP rights. Likewise with Russia and China.

The same logic applies to the US too, by the way. I don't think the NSA thought too hard about e.g. whether backdooring Cisco routers was in violation of any IP laws - I doubt they'd care much about the contents of LICENSE.txt. Complete exercise in futility.

Much more enforceable than you'd think, since trade treaties don't allow countries to widely violate licenses. Not to mention that products can't be shipped from rogue countries to the rest of the world with such software, say an embedded OS. What doesn't have a chip in it, now?

How many treaties like this is North Korea a signatory of? And if they are a member of any, how much do they care about breaching them considering they‘re already under comprehensive sanctions?

Russia, China, Iran - all still rely heavily on international trade treaties. North Korea not so much, true.

This idea has been strongly rejected by the FSF/GNU group. Now, they don't have magical sovereignty over the word "open source", but as commonly used, especially in "these parts of the woods", most people will think of Open Source == FOSS, which with no uncertainty intentionally allows bad people to use it. This is one of the things that has gotten RMS in hot water, and when push comes to shove is a pretty unpopular opinion, but is deeply embedded in the GNU GPL and BSD sense of "open source".

There are multiple "don't be evil" licenses, with varying types of stipulations. Some literally say "don't be evil". There was one license that made the rounds here on HN a while back that said something to the effect of "you must accept the authority of the Christian Bible". These are controversial at best, and disliked in part because they're often extremely vague, and very questionably enforceable.

Trying to spell out the specific moral or legal stipulations that would prevent one from using software is ... more doable from a legal perspective, but still opposed by FSF/GNU types, because it feels contrarian to the free software movement, and a general distrust for governments and government regulations. Taking people's software freedom away because of where they're born rubs some people the wrong way.

Not sure exactly how I feel about the issue as a whole, I don't buy wholesale into Stallman's ideology, but it's something worth considering.

The OpenPOWER Foundation GitHub organization appears to be here, but it doesn't look like there's a public repo for the openpowerfoundation.org site that would allow submitting pull requests to remove the hostile scrolling behavior on this page.

https://github.com/OpenPOWERFoundation

(Please do not upvote this comment.)

Reply on news.ycombinator.com