Would anybody recommend that sort of setup over a Mikrotik Hex or Ubiquiti Edgerouter X?
Anything MARVELL should mostly do, usually. But that's not the cheapest segment anymore, so the people are hurting themselves by buying the common plastic trash. Or use DD-WRT instead of OpenWRT, because they give a shit about binary-blobs and integrate that however they see fit, because they prefer to be able to use everything the hardware supports, instead of having an almost brick running open-source.
It is not always clear, but enabling QoS usually disables hardware offloading, for example with Ubiquiti devices[1] "Traffic to which QoS policies have been applied cannot also take advantage of the EdgeRouter's Offloading feature". The same is true for open source even with binary blobs. The chips simply can't do it.
Using QoS and implicitly disabling hardware offloading significantly drops the throughput as it becomes CPU bound. "Without offloading enabled, IPv4 traffic will be routed via the CPU and will be limited to around 300Mbps on the EdgeRouter Lite (ERLite-3). With offloading enabled, the throughput will be about 950Mbps." [1]
[2] https://en.wikipedia.org/wiki/Turris_Omnia
[3] https://docs.turris.cz/hw/omnia/omnia/
These two and similar were what I had in mind.
Though I don't have them. But considered them a few years ago. But I've chosen another path, of really dark art ;-> (Don't ask)
edit:
Oh, and
The smaller, and more affordable option from the makers of [1]
But this is all from three, or even four years ago?
I don't even know if that Turris Omnia thing is EOL meanwhile?
They wanted to make something even more modular.
Anyways, all MARVELL based and thus IIRC with working offloading under Linux, though not mainlined at the time.
[5] https://en.wikipedia.org/wiki/Marvell_Technology,_Inc.
edit: Funny, Amazon.com says they have 9 Turris in stock
[6] https://www.amazon.com/Turris-hi-Performance-printserver-Vir...
While amazon.de has 19.
So they should be available. But about 400, instead of maybe 40 to 200 for some cheap plastic trash which melts or resets if you look at it the wrong way.
I recently bought a GL.iNet Brume. It's a nice tiny ARM machine I use as a wired-only router. /proc/cpuinfo confirms it supports hardware crypto...
Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid
I might have tried the FriendlyELEC NanoPI R4S instead, which actually has two separate NICs. But if I'd ordered that I'd still be waiting for it...
One thing I don't care about: hardware NAT/routing offload. I think that's often unsupported by OpenWRT, and it's incompatible with SQM.
They look very nice (although unavailable till January)
I haven't read into the specifics yet, but it sounds like nft-tables configuration will be translated to a eBPF program, that will then hook into network flows, and tell the switching HW/NPE over some side channel which accelerations to use and shortcuts to take.
But do I really want to trust a "Black Box Network Processing Engine" bolted onto my CPU, that can route packets entirely outside the realm of control of the main CPU?
I always wonder about intentional or unintentional "in-band debug features", when people mention, that they run their home-lan on the same switch as the WAN-Uplink. I mean, is there really a HW vendor, that we trust to get the switching logic 100% correct/secure/performant?
My GL.iNet Brume supports DSA on its switch with OpenWRT 21.02, as does my older Netgear WNDR3800.
I'm not sure there's any hardware/software combination out there that supports offloading NAT/routing with proper QoS. So I'm puzzled when people talk about the importance of offloading or suggest using machines with very low-power CPUs. The Brume is Dual-Core ARM Cortex-A53 @1.0GHz, which gets the job done for me, but I wouldn't want to go any slower than that.
[1] https://www.kernel.org/doc/Documentation/networking/dsa/dsa....
To me, DSA (as a "convenient" means to access/name/treat individual switch ports) is a prerequisite to enable usage of whatever Switch-Chip-Level-Offloads the switch chip supports. Without DSA, vendors had to use nasty hacks to make use of e.g. HWNAT, which is why there never was mainline support for these acceleration features.
HWNAT is a routing function, and many/most? switches of the last decade support that. But, OK, true, most of them will probably not be able to HWNAT "with proper QoS". But newer devices are not "just" a switch-chip, they contain a "Network processing engine", which is not a "fixed-function" device, but rather a "programmable" accelerator, that you could perhaps even "teach" how to do PPP, Encrption/VPN wrapping, and/or "offloading NAT/routing with proper QoS".
And now that DSA and eBPF are in mainline, using eBPF to generate a program/set_of_rules for the Ethernet acceleration device from your nftable-rules seems like the natural next step for me.
Or where do you see my reasoning go off the rails?
No, I don't know of any problem in your reasoning. I'll look forward to that happening! I wasn't aware newer hardware was so flexible.
I just hope, "flexible" doesn't come with a large "Difficult-to-audit-caveat" side-dish. NPEs "governing" traffic could potentially substantially reduce the trust the user can have towards the platform.
At the time it had much better hardware at a similar price to more popular brands like Protectli and Qotom. Not sure what the scene is currently, but there a bunch of similar offerings on AliExpress.
Like you, I run a single x86 box with a handful of guests for random stuff - including Home Assistant!
I do want to ask though, has there been any downsides to using OpenWRT on x86?
I haven't really run into notable downsides. Installation was somewhat of a pain since I had to resize the partition (all of the prebuilt images assume small hardware), and all of the default packages are as minimal as possible - like busybox for userspace/shell. Upgrading on X86 is I guess possible (I haven't tried yet), but most of OpenWRT's upgrading instructions/support are geared towards reflashing firmware, not updating packages on ext4.
But, there are a bunch of available packages - coreutils, zsh/bash, etc., that can be installed as desired. In general OpenWRT is pretty set-it/forget-it for me.
https://forum.openwrt.org/t/adding-openwrt-support-for-xiaom...
Thanks to full duplex, clients are able to use the fibre uplink without slowdown. VPN is managed by a separate system, though we had some successful experiments with WireGuard. Both a colleague and I are running the exact same setup at home (with fibre uplink and VDSL).
It's a very capable and power efficient setup. I'd definitely go with a CM4+dual NIC board today.
If you're looking for a network connected server that happens to do a lot of packet processing you can go the rpi route but I'd still front the internet and rest of the home internet off a small cheap hardware box like the Hex.
[1]: Ubiquiti has had packet processing issues in the 2.x firmware series for what seems like forever, and lack of updates lately has prompted me to move to something that is arguably better anyway.
I only found some quick&dirty patch, that seems to make MTU up to 2000 functional. But since the patch aims for 9018, and only gets 2000, that does not seem like the "right" solution.