Show HN: Router7 – A pure-Go implementation of a small home internet router
github.com
github.com
In case anyone wants to use this on router7, where CGO_ENABLED=0, you’d need something like https://github.com/google/gopacket/pull/470
Turris Omnia: 209 € (-), built-in SFP (+), UART (-), GOARCH=arm (-)
apu2: 105 € (+), no SFP (-), DB9 serial (+), GOARCH=amd64 (+)
Basically, the apu2 costs half (I now have 3), has a real serial port (now permanently attached to my workstation) and an architecture (amd64) which is closer to what gokrazy already supports (arm64) and more likely to be useful to others — I could find way more suitable (≥ 2 ethernet ports) amd64 boards/mini-PCs than 32-bit arm boards.
Hope that makes sense!
Edit: I do agree that the Turris Omnia is a pretty good platform, though! See https://michael.stapelberg.de/posts/2017-03-25-turris-omnia/ for more thoughts about the device.
The apu2 has all the Omnia has (aside from the built-in SFP), and is well-supported by PC Engines and their vendors. They run coreboot and even sell you recovery SPI flash chips for a few bucks if you want to change coreboot yourself :).
The currently supported platforms are the Raspberry Pi 3 B, Raspberry Pi 3 B+, the PC Engines apu2c4, and qemu x86-64 (I’ll update gokrazy.org with an overview about this in a minute). As long as your hardware is similar enough to one of these, chances are things just work.
Edit: https://gokrazy.org/platforms.html now describes the support level of the various targets.
Edit: I was wrong, the Turris has docs and firmware to play with.
However, I had previously run dnsmasq instead of the default kresd so that the Omnia would resolve hostnames in my local network from DHCP leases, and it was such a pain! Updates would frequently break the dnsmasq setup, so I was glad when they finally integrated DHCP lease-based name resolution in the Omnia.
For me, running custom software on the Omnia is a frustrating experience. Working on router7 was a pleasant experience.
Lastly, from my experience with gokrazy.org, I expect the total maintenance/debugging time for the router to be really low, and not in C and obscure build systems, but in Go, about which you can read my thoughts at https://michael.stapelberg.de/posts/2017-08-19-golang_favori... :)
I'd say the Turris Omnia will slowly die in favour of the Mox [1] so it's probably a good time to jump.
That's surprising to hear. Dnsmasq has been my go-to solution for almost 10 years. Dhcp hostnames via DNS just works out of the box. I have two subnets at home, serving dhcp 4/6 on both. The only thing that was a pain (mostly because I had to understand it first) was setting up dhclient6 for prefix delegation and get hooks in place so the dnsmasq config would be updated when the prefix changes, but even that addition already survived a dist-upgrade already.
But should that stuff crash and burn next time I dist-upgrade I might take a look at router7, even though I personally really dislike go, but along as I don't have to write code in it I'm fine. Considering how happy I'm with i3 I'm pretty sure you did a good job here too. :)
And thanks for the praise, but if Go is not your thing, maybe this is not the best project for you :).
The lower-level is very visible through Wireshark, with which you can capture all network traffic.
Does that help?
I don’t see any stability issues with the apu at all.
The external rebooter isn’t used as part of a watchdog setup. Instead, it is programmatically used by rtr7-recovery, with which you can recover from a faulty update, to trigger a PXE boot. This way, you can update your router each day in an unattended fashion, with automated rollback in case connectivity is lost with the new software.
The first milestone of obtaining a DHCPv4 and DHCPv6 lease from my ISP and successfully pinging google from my laptop through router7 was reached on 2018-06-02.
I switched my home network to run on router7 on 2018-06-15.
Today is 2018-07-14, so it took about a month to polish before actually being able to publish it. But it was good enough to use less than a month after starting work on it. I only have a little bit of time in the mornings and evenings before/after my day job, and on some weekends, i.e. it could have been done in less time when working on it full-time.
I learnt about PXE booting, MBR boot loader code, details about DHCPv4, DHCPv6 (had previously not looked at DHCPv6 at all), how wireshark does ssh remote capture, and a bunch of other things I’m blanking on right now :). Lots of interesting details to be learnt in this space!
Meanwhile, some programmers (myself included) spend months dithering over the decision whether or not to start that simple web project that's been hanging around in their head for a year. Lots of respect to you. I'll be coming back to this comment again and again.
(Oh, I guess linux is handling all the actual routing, of course. So really the question is performance of dhcp/dns/etc. at peak on the hardware, vs. alternatives.)
Regarding DHCP, DNS and router solicitations: don’t worry about it. In my network I get barely 1 request per minute (with 31 devices). With the 1 GHz quad-core in the apu2, many orders of magnitude more traffic should easily be in the cards ;)
This only requires [1,000,000,000 b/s / (1,538 B * 8 b/B)] == 81,274 f/s for full-sized (1500 byte) frames.
The 'fun' starts as the frame size drops. At the far other end, for minimum-sized frames [1,000,000,000 b/s / (84 B * 8 b/B)] == 1,488,096 f/s.
For an APU2, the first is relatively easy, and the second, nearly impossible with kernel-based networking.
Do you know if it's possible to pack arbitrary files in a gokrazy image ? Like configuration files, data files, or any other native binary ?
gokrazy images currently only contain Go packages, so depending on what you want to do, the best way might be to bundle the file(s) into a Go package.
For router7 specifically, rtr7-recover’s -backup flag accepts a tar.gz archive, which will be unpacked to the persistent data partition /perm. This mechanism is used to restore state and configuration when reverting faulty updates. You can use this mechanism, or https://github.com/gokrazy/breakglass if you prefer an interactive shell, to place files in /perm.
That said, I was thinking about an -overlay flag for the gokr-packer, too, so that you could indeed place additional files in the static root file system. I’ll see if it could be done any other way, but this might be a good escape hatch for when the pure-Go model isn’t applicable (e.g. large legacy applications which cannot easily be ported to Go).
How would one go about implementing this? Call the tc binary or reimplement tc in go?
Looks great! I’m impressed.
I’m actually about to be out of touch for a week, so I’ve cloned the code for Router7 to read while I’m offline.
If we're casually mentioning alternatives to PC Engines (who provide, AFAIK, the cheapest solution and are using Coreboot):
https://opnsense.org/about/about-opnsense/ developed & supported by Deciso BV. Still supports x86-32, unlike pfSense. Their official hardware is also expensive though, just like Netgate.
If you want a cheap PC Engines alternative running Linux I can recommend Ubiquiti gear instead. The entry level products are cheap (ER-L, for example, has 3 ports and costs ~100 EUR), and the Unifi products are user-friendly, while the EdgeMAX allow more freedom/control at the price of user-friendliness. Although it isn't DIY like the other solutions, that could be an advantage. But if you're going the DIY route anyway, PC Engines gives the best bang for the buck unless you need 4G modem support.
or, better, the board based on the espresso.bin design.
Try and do any kind of meaningful traffic shaping, firewalling, with an ERX. Or, just make some configuration change which knocks traffic handling out of silicon and into the CPU: you'll quickly see where the extra money would've been nice.
My only serious complaint about pfSense is that BSD only has single-threaded PPPoE, which really does not work well at anything approaching gigabit speeds.
These are x86-64 (AMD64) machines, running a 64-bit fork of FreeBSD. pfSense doesn't run on x86-32 anymore (even though x86-32 works perfectly fine as router). The argument is that its "ancient" hardware; yet here we are discussing single-threaded PPPoE...
EdgeRouter X is a bit weak, sure (though it also does not use a lot of power), EdgeRouter Lite (which I got) is already much better performance, sports a MIPS64 and more RAM.
> [..] you'll quickly see where the extra money would've been nice.
If I'd want the highest performance + bang of buck + got the time to maintain (including the hardware) I'd go for PC Engines plus OPNsense or Router-7. But I don't want to put a lot of effort/time in maintaining my home router.
I'm positive we'll have loads of fun with RISV-V routers in the near future. If not the big corps, perhaps smaller ones. Amazon also appears to have plans to sell switches with AWS support.
There's also MicroTik/RouterOS.
Mikrotik hardware suffers the same problems as Ubiquiti (and most other manufacturers): it's very easy to lose gigabit routing performance with even the most basic of home firewall rulesets.
The author is based in Switzerland. He has some other pretty awesome projects, such as the I3 Window Manager
I wish the law was worded such that it just had to be unrelated to the work that employee does for the employer, but it was not written with that forsight.
Edit: forgot to mention, you can use static leases, too, permanently assigning an IP/hostname to a MAC address: just remove the expiration field in /perm/dhcp4d/leases.json :)
I have a live weather station [1] that pulls data from a Go-based service that I wrote and run on a server on my home network. When I'm outside of the house, mhkweather.com resolves to a Digital Ocean IP, where I run a proxy to forward requests back to home IP. However, when I'm at home, mhkweather.com resolves to a local 10.x.x.x IP and goes straight the the local service, no proxy. It faster this way. I want to be able to set DNS overrides for local things. I can do this with unbound but it would be nice if dnsd could do this for me.
For the time being, you’d need to modify dnsd to install your custom handler. It should be as simple as adding another server.mux.HandleFunc call in internal/dns/dns.go.
If this turns out to be a feature which many people would like, I’ll think about how to best structure the code to make this easier.
The "best way" is to use a real domain for your local network. But this can be annoying, costs money and has other issues (renaming/-branding, mergers/splits, domain squatting, accidental record publication and so on ---- it's a lot like renting an internet-routable IP network for LAN use, it's kinda silly and error prone).
I just learned this however: https://www.icann.org/resources/board-material/resolutions-2...
So .corp and .home will not become gTLDs and should be relatively safe to use.
For instance, I've got a tinc VPN network setup between many of my hosts. What IP range should I set that to such that it will not clash with whatever access network my laptop roams onto? The proper IP-philosophy answer is to apply for a block, even if it's never announced by BGP.
The pragmatic hack is to attempt to choose the most obscure block that isn't likely to conflict with the public Internet or with how local networks are generally setup. In my case, I chose something NOT from RFC1918 (so shoot me).
It's of course less of a big deal if you do realm / split-horizon based routing. But that is not straightforward to setup, as it's getting away from how IP was designed to be used.
What I'd like to see is a home router that lets me put arbitrary rules in. Currently I use dnsmasq as a resolver and it basically does what I need, but I'd rather have it running on the router instead of needing another computer.
My current rule set is roughly:
* Resolve from local hosts file - this is where I put my statically assigned home servers/devices.
* If the domain request matches .domain.tld, forward it to a.b.c.d for resolution - this is a dnsmasq running on a remote box and covers off VMs that I run there. There is an IPSec tunnel (running on a small VM) and static routes on my router that handle getting to that box.
Finally, pass the request onto my router which will pass it onto my ISP or use its local DHCP-driven hosts table to resolve it.
I'd love it if an off-the-shelf router could do this. Currently I'm using a UniFi Security Gateway and it seems like this feature set would be a good fit. The router is professional-enough that advanced features like this make sense.
I’ve done Go cross-compilation for running on Ubiquiti gear before, and it worked. So, if you’re willing to spend the effort, you could do it :).
I run 2 dns servers usually on the same box on different bind addresses. You'd run a caching resolver on the gw ip and forward requests for my.tld to the second resolver which is configured to be authoritative for my.tld. the hosts can then be configured either through DHCP or statically.
This is a pretty common technique to make enterprises Intranets.
Added benefit is that when you did a file transfer between Lan hosts with the globally accessable DNS, the transfer stays on the LAN. Also ssh host works in you ssh config regardless of location.
I guess Go's cross compilation makes it easier.
PS: Without hardware offload, the 8 port POE switch only manages 12 megabits / second.
dnsmasq should be able to do what you want to do and it can be enabled on the USG - at least from the controller.
One last comment, you might check out Dragonfly BSDs network stack if you haven't yet for ideas, the last benchmarks I ran were surprising.
To anyone listening, does Bpf enable us to get past the 10gbs dma issues on a Linux router?
It can work with any ISP (not just fiber7) as long as you have an ethernet interface and the ISP uses DHCPv4 and DHCPv6 for configuration. If that’s not the case, you’d need to develop e.g. PPPoE support yourself.
With some effort, it could become the new Inferno, just with Go instead of Limbo and the Dis VM.
That said, your guess is likely correct, this gokr-packer invocation should do the trick:
gokr-packer \ -hostname=router7 \ -overwrite=/dev/sdb \ -gokrazy_pkgs=github.com/gokrazy/gokrazy/cmd/ntp \ github.com/rtr7/router7/cmd/...
But note that you might need to rebuild https://github.com/gokrazy/kernel with some additional kernel options from https://github.com/rtr7/kernel, plus options for whichever USB ethernet adapter you’re going to use.
Note that you won’t have automated recovery (rtr7-recover), as that uses PXE boot, which Raspberry Pi’s don’t do by default (and rtr7-recovery-init has some hard-coded values which won’t work on the Raspberry Pi, at least not when using an SD card for storage).
I would consider a router software stack, systems level coding.
Also, 4G is a shared medium, so I prefer my dedicated 1 Gbps line. We should handle internet load on dedicated lines where possible, so that the shared medium keeps working well for people who have no other choice.