Hacker News Reader

Comment by mbreese | Hacker News Reader

At this level, I don’t understand the PoE requirement. If you’re already making “blades”, why not also make a proper backplane that includes power and network connections as part of the backplane. You’d probably be more power efficient and remove the need for so many network ports.

This would also increase the costs for the chassis (because you’d have to add a network switch), but you could probably also pack in more blades… But even if you kept the ethernet PHY ports, you’d probably be more power efficient with even just power on a backplane.

View on HN

A backplane is nontrivial to design electrically/mechanically and expensive to manufacture (even just some high quality connectors will quickly run up your BOM). If you also want to carry networking there you'll end up having to do some semi-custom network switch design, which locks you into a particular switch/ASIC vendor 'forever', or at least vastly increases friction when wanting to upgrade. Yes, they already have their own PCBA and some mechanical design, but it's very simple compared to what it takes to design a reliable backplane with an integrated power supply and network switch.

At the end of the day this a low-cost system for low-cost devices. IMO the little benefit from having a backplane is not worth the R&D cost and the downsides of fully backplaned blade systems.

And this is not just about this project: from what I see, the industry seems to have rejected fully integrated blade systems. Dell's M1000e is dying, and I don't think I've seen HPE bladecenters in years. Instead, semi-integrated systems like Supermicro's high-density offering is king. No proprietary chassis management system, no proprietary network switches, no locking yourself into whatever the backplane can carry.

At work we have oodles of 2U4N (8P) systems. Just going by eye I'd guess these have the same or even higher density than a blade center, while allowing more granular scaling, and being a standard form factor, and you get both front and back access to each node, so you can have whatever I/O you want, and you don't need a lift to handle the chassis.

totally agree, thanks

> the industry seems to have rejected fully integrated blade systems

In my probably too simplistic mental model, the blade systems approach got overrun by the advent of multicore processors combined with virtualization and containers. While not 100% equivalent, it’s close enough and the price difference ends up making the decision in most scenarios.

> And this is not just about this project: from what I see, the industry seems to have rejected fully integrated blade systems. Dell's M1000e is dying, and I don't think I've seen HPE bladecenters in years.

No offense but this is just wrong. HPe is all in with their synergy platform (the replacement to bladecenter that has been out for several years now). Cisco just released their next gen UCS blade platform as well. Dell has never been a dominant player in blades.

https://www.marketwatch.com/press-release/at-a-47-cagr-data-...

Not offended. This is a good reminder that this industry is far and wide enough that different people will see different subsets of it, and just because I observe something doesn't make it universal :).

I think it's mostly convention and convenience—in the Pi ecosystem, most people using these things headless or as small servers are used to powering with PoE already.

Networking gear is often powered by PoE, but most servers have higher power requirements so a custom backplane or separate power method would be required.

Boards like the Turing Pi v2 (so far just prototypes and a marketing page on a website) would have a built-in network switch and power backplane (direct to each CM4), and I'm guessing another board or two like it will appear someday. I hope.

Is there somewhere after the POE that power could be tapped in? Even just a pair of pins for 5v sounds like it might make for significant power savings.

No, I get that... and there is a different level between hobbyist projects and larger production projects. Using the existing PoE "infrastructure" makes a lot of sense in the prototyping stages.

But if you're already going through the process to build custom boards for a cluster, then a backplane makes much more sense. We're not talking about a small 4-5 RPi cluster where everything can work off of a single PoE gigabit switch with a rat's nest of cat6. This project is talking about 16 of these blades in a 1U chassis. If you're going to go that far, a backplane isn't a big leap and would save power, space, and make cabling much easier.

The trick is, if your backplane is providing ethernet switching, then you need to provide for different levels of network needs.

Some people would be fine with a dumb N + 1 port gigE switch, which would be inexpensive, others will need 10G uplink, some 2x 10G with LACP so there's no bottleneck, some are going to want vlans or other managed switch style offerings, etc.

If the PoE power conversion really eats 6W per board though as mentioned in the article, that seems like a lot of power and an alternate power arangement would be a no-brainer.

Given the amount of these that will probably be made (I imagine it will be a low number), I could certainly see having at least two backplanes:

1) network passthru cables with power 2) 1G network switch with 1G or 10G uplink

I just don't see there being much of a market for these where you'd have to have many more options. But, I think the power option is really a no-brainer... if you have to put passthru RJ45 jacks on the back of it, then that would be fine. At least they'd be in the back and could be pre-wired.

I'm tempted to get a CM4 and try to put together something just to see if it would work. I really don't have a problem with much else on the blades as-is (aside from the TPM, I'm not sure what that gets you). I do like the ide of putting an SSD on the blade for storage.

Yeah, I could see a passthrough thing, where you had an edge connector (or something), that was the 8-pins for networking and two for +5V and the backplane was just a +5 rail, and the rj45 jack right next to each slot. As a zero experience with PCB design kind of person, this seems relatively simple. You could make a 'one blade backplane' with a PoE kit for the single/small number of devices case where PoE makes more sense.

I disagree that a true "blade" approach would be better, and believe PoE is a better alternative to PSU and backplane. My hope is Uptime Labs use the same approach as their current half-rack chassis - zero electronics, just hold the PoE-powered pseudo-blades.

Jeff Geerling made an extensive video series on the comparable $189 Turing Pi v1. In the final analysis of it, he breaks down how the networking part of the Turing Pi on it's own is $100 worth of chips. With the Turing Pi v1 holding 7 CM3 modules and the current Uptime Labs half-rack chassis holding 8 CM4 pseudo-blades, you might expect a fullsize chassis backplane supporting 16 blades requiring $200 worth of networking chips. Mind you, that $200 is for an unmanaged network with no VLANS or anything nice like that. Who knows how much more that would cost? $250? $300? More?

I've got two Turing Pi v1 boards. One is a dud because the network switch chip didn't work. It's a common problem if you read through the Turing Pi tech support Discord channel. These things are hard to get right. Leave it to a vertically-integrated network gear company to come up with something reliable.

I imagine two target consumer groups for these:

Group A will chose this over some Ampere Altra server, so the cost of a fancy new enterprise PoE switch is likely well within their budget.

Group B are Raspberry Pi hobbyists / "homelab" enthusiasts, and this is a group that's probably not averse to purchasing either a low-cost PoE switch on Amazon or a used enterprise PoE switch on eBay. There are many used gigabit PoE switches for sale at approximately the same price as a "managed" backplane (backplane parts only, ignoring R&D and QA overhead). Difference is, Group B gets both power and network for that price.

For both customer groups, the PoE switch is also reusable if they decide not to keep their Uptime Labs chassis, whereas a backplane wouldn't be. If only a fraction of the chassis capacity were used, then with a backplane, you've wasted money on idle backplane space. With the PoE approach, you have switch ports that can be connected to other things.

No one is going to choose this over an Altra server. If you need a server, you get a server, you don't go for a hacked together RPi solution.

For the other group -- I'm in that group. I have half a dozen RPis hooked together with various bits of PoE and duct tape. Most of them are doing something, but aren't very mission critical (unless you count streaming music around my house as mission critical!).

As much as I like the RPi for various projects, I don't think I could rationalize using something like this for work.

I'd like to have the networking built in, but realistically, I suspect it would be an easier sell to have a passthru backplane where you still don't have the physical RJ45 ports on the blade, but on the backplane. But, you don't provide any switching on the backplane... it would be a 1:1 blade:port setup on the chassis. You could always add a switching backplane if you needed. For me, the backplane idea is primarily about power. The PoE numbers suggested in the power don't scream efficiency.

Really, the chassis is still going to need it's own power supply anyway (for fans), so why not just expand that out to supply the RPis from one power supply?

Yup, this is only really practical for a rack you manage yourself. The colocation provider is going to charge you like $5 per network connection.

I like this idea. Can we do something similar to how Frame.work laptops work and just provide the connection to/from the chassis as USB-C?

It'd be great if the next version of the compute module would provide thunderbolt 4 instead. I think we'd be able to provide both power + 40GB nic support over one connection.

Reply on news.ycombinator.com