How to turn anything into a router

Windows PCs had (have?) that Internet connection sharing feature for a long time. It was really just a checkbox to enable NAT too.

Sometimes I think combining a firewall/router/switch/AP/file server/etc into a device called a "router" really confuses people. Even people who should know better.

That snowballed into “we want a website do you know how to do that?” and. Well, no, but it had Apache available and I … figured things out enough to take the skills elsewhere.

Repeated the same trick with a place in Wisconsin, who initially shared a 56k dialup connection with all their dispatchers and were impressed the thing had stayed up for 900 days without even redialing. 90% of their work was done in an on-prem wyse terminal anyway, dialup used to do the job for email or googling an address.

27, 28 years later I’m still dragged in front of them once in a while to ask how they can accomplish something cheaply with Linux, bubble gum, paper clips, or whatever . The times and technology have changed, but not how cheap they are!

I was doing the same. Router and firewall on old Pentium CPUs. I don't have these machines anymore but I still have HDDs from back then with post-it notes on them saying stuff like: "Linux firewall / HDD 120 GB". For whatever reason my HDDs adapter that can read just about everything doesn't have the correct pin out for those HDDs. Would be a blast if they were to still boot: at some point I'll just buy a compatible adapter and see what I can find on those HDDs. I was very likely also saving some backups there.

But really my best memory was years (I think) before 120 GB HDDs became an affordable thing, in the super early Slackware days, on a dial-up connection: I had a 486 desktop computer and I'd share the Internet connection to a very old laptop (!) using... PLIP. A printer cable and the Parallel Line Internet Protocol. Amazing hack: my brother and I could then both use Netscape at the same time and to us this felt like a glimpse into the future.

The old one is getting really old now, nearly 25 years ago [2].

[1] Book Review: Linux Routers - A Primer for Network Administrators, 2nd Ed:

https://www.linuxjournal.com/article/6314

I did routing duties for my LAN with my primary desktop for about a decade, variously with Linux, OS/2 (anyone remember InJoy?), and FreeBSD -- starting with 486 hardware. Most of that decade was with dial-up.

The first iteration involved keying in ipfwadm commands from, IIRC, Matt Welsh's very fine Running Linux book.

WAN speeds were low; doing routing with my desktop box wasn't a burden for it at all. And household LANs weren't stuffed full of always-on connected devices as they are today; if the Internet dipped out for a few minutes for a reboot, that wasn't a big deal at all.

I stayed away from dedicated hardware until two things happened: I started getting more devices on the LAN, and I saw that Linksys WRT54G boxes were getting properly, maturely hackable.

So around 2004 I bought a WRT54GS (for the extra RAM and flash) and immediately put OpenWRT on it. This lead to a long rabbit hole of hacks (just find some GPIO lines and an edge connector for a floppy drive, and zang! ye olde Linksys box now has an SD card slot for some crazy-expensive local storage!).

I goofed around with different consumer router-boxes and custom firmware for a long number of years, and it all worked great. Bufferbloat was a solved problem in my world before the term entered the vernacular.

And I was happy with that kind of thing at home, with old Linksys or Asus boxes doing routing+wifi or sometimes acting as extra access points... until the grade of cheap routers I was playing with started getting relatively slower (because my internet was getting relatively faster) and newer ones were becoming less-hackable (thanks, binary blob wifi drivers).

---

I decided to solve that problem early in 2020. Part of the roadmap involved divorcing the routing from the wifi completely -- to treat the steering of packets and the wireless transmission of data as two completely distinct problems.

I used a cheap Raspberry Pi 4 kit to get this done. The Pi4 just does router/DNS/NTP/etc duties like it's 1996 again. Dedicated access points (currently inexpensive Mikrotik devices) handle all wifi duties.

That still works very well. Pi4 is fast enough for me with the WAN connections available here (which top out at 400Mbps) even while using SQM CAKE for managing buffers, and power consumption of the whole kit is too low to care about.

The whole OpenWRT stack just plods along using right around 64MB of RAM. VLANs are used to multiply the Ethernet interface into more physical ports (VLANs were used to do this inside the OG WRT54G, too).

It's sleepy, reliable, and performant.

---

And it'll keep being fine until I get a substantially-faster WAN connection. For that, maybe one of the China-sourced N150 boxes, with 10gb SFP+ ports, will be appropriate -- after all, OpenWRT runs on almost anything including AMD64 and the UI is friendly-enough.

But there's no need to upgrade the router hardware until that time. Right now, all of my routing problems are still completely solved.

Eventually I got all the devices marked from origin to their patch cables in the server room, and I started looking into the Squid cache. It turns out that they were caching everything, as well as blocking websites. I figured out what websites they needed to do their job, and turned off caching, while also learning the ACLs for blocking websites. Anything else was allowed, but the Squid cache would hold a copy for some set amount of time (I think it was 24 hours, so if it was legitimate they only had to wait a day, but it also saved on bandwidth by quite a bit - although think this was used more to monitor user activity).

It was frustrating as someone new to large LANs, as well as to in-house caching, but had been using Linux since an early version of Slackware in the later 1990's. Even to this day, as someone that writes software and does DevOps, that knowledge has helped my debugging skills tremendously. Dealing with caching is a skill I feel you need to be burned by in order to finally understand it, and recognize when it's occurring. I cut my teeth on Linux through a teacher that set up a web server in 1997, and not only gave students access to upload their web files, but also a terminal to message each other and see who was online.

IIRC, there were some Macs that were confused if there was a bridge in the network, so had to change the segmentation and run masquerade, but that was still better than not having internet. And no need to allocate those precious public IPs, though you could still get them.

Masq was one of the first killer features for Linux.

I used a lower power Intel Atom mini PC with an additional NIC as a router for years. I tested it and found it could route around 300Mb/s which was plenty.

But then I got gigabit internet. So I bought an Intel 4 port GigE card from eBay and now run OPNSense as a VM. If you get the right Intel card you can pass through ports to VM individually, which is nice for playing (don't know the exact details but look for cards with virtualisation support, mine is an 82575GB I think).

To be fair, my setup still probably has too much to go wrong, due to the VM thing, but I just haven't got round to getting dedicated hardware, and it's worked fine for a couple of years now.

I was recently introduced to a Barracuda router, and bashed my head against the wall long enough to discover it had an ssh interface, and linux userland, and was able to solve my immediate problem by directly entering the commands to get it to [temporarily] do what I needed. (Of course, using the GUI to reapply settings wiped my manual configuration...)

I've used pfsense, OpenWRT, Barracuda, Verizon's OEM router (Actiontec) and they all represent the same functionality wildly differently.

Is there something like that?

Mikrotik sells the CCR2004-1G-2XS-PCIe, which is a fascinating device:

https://mikrotik.com/product/ccr2004_1g_2xs_pcie

It is a full Mikrotik router stripped down to just a board and hung off a PCIe interface. Iirc by default it exposes a virtual gigabit interface to the host and otherwise acts exactly like a CCR2004 running RouterOS.

Doesn't really buy you anything vs a RB5009 unless you can use the pair of 25Gbps ports, but it sure is neat.

I've got a 10G fiber connection, and I swapped out a Fortigate 100F for a server running VyOS. I had performance problems, because the 10G to 1G transition caused dropped packets at the switch. I was able to solve it by shaping the traffic to the 1G devices to handle queuing in the router, which is something this particular Fortigate can't do. (High end routers have algorithms like WRED designed to get TCP to behave nicely on 10G to 1G drops, but I don't want the noise of a Cisco in my basement.)

I am not really sure about it. My ISP provided AP can do a gigabit over wifi.

I need to change it because the ISP hardcodes the dns for spying reasons.

But sadly to match that performance I need to spend like $180 to get an AP with that performance

Imagine everyone had their routers disabled simultaneously. I don't know if the cell networks could function with the surge in standard traffic that would happen, and then you've effectively plunged all or part of the country into a communication blackout.

I think "turn it off permanently by bricking it" is almost as bad as "leverage for DDoS".

I worked on Bot Mitigation at Amazon, and we once saw a ton of traffic that was heavily distributed amongst consumer devices world-wide, but surprisingly in the US too. We suspected compromised routers that were using the home page as a health check. There was a lot of investigation I did, and the short realization after talking with the network engineers is that the amount of traffic, and distribution of sources, would be impossible to stop. There merely isn't enough bandwidth in the world to stop so many residential device if it hits a specific target. To be clear, this was coming from less than half of active Amazon customers, not everyone in the US.

Anyway, it wasn't routers, but it was a consumer device, and it wasn't nefarious, it was incompetence (in code), as usual.

That should probably be the technical concern. Even if you have traffic protected by TLS, you still typically have enough metadata to cause some problems for users individually, but the assumption that foreign equipment is back-doored by some security service or other is probably safe.

- Access to data (dns/ips, domain names (if not using ESNI), amount of traffic, etc) of sites you are visiting

- Access to the inside of your network where it can attack machines that may not be secure

- DDoS

- The ability to shut down your internet

I'm sure there are more.

These are networks of controlled devices. They're hard to eradicate, as shown by the fact that they haven't been eradicated: they're still active and being used to compromise systems, including defense and intelligence systems, power systems, financial systems, identity systems, etc.

Is banning foreign gear going to fix this? No. Security isn't a product. It is, however, a process, and in a process you take steps. I think this: we (individuals and institutions) enjoy tremendous liberty in the use of communications equipment in the US and most of the West. Taking that for granted is a mistake. If part of keeping this means the US has to spin up a domestic supply of network gear, or carefully modulate where such gear comes from, then lets do that. Otherwise, The Powers That Be will leverage its concerns into far worse steps.

If this were really about computer security they would follow California’s example of requiring unique passwords. Maybe make manufacturers liable for not patching known remote exploitable security vulnerabilities. It doesn’t matter if the source of a DDoS is a Huawei box or a Netgear box.

In additon, compared to PF/OPNsense or OpenWRT (Linux based), you have more control and exposure to the underlying network concepts with VyOS. You're not configuring the kernel manually, but you still learn quite a bit.

I've always found fq_codel to be good enough for gaming through my router. But I have a 10gbps uplink.

I kind of feel like that's cheating though; I've outsourced the hardest part of the project to someone else. Maybe one of these days I'll take an old NUC or something and buy a decent wifi antenna for it and try and do it properly.

[1] Initially pfsense, then OpnSense, then ClearOS, and now some custom firewall rules in NixOS.

Version control is in the GUI, you can adapt for your needs the number of changes you need. automatic config.xml backup also possible.

After I upgraded to a 10GbE ethernet card in my previous router, my card didn't work correctly with FreeBSD-based stuff anymore. I changed to ClearOS and that was actually comparably easy to Pfsense...maybe even easier? I recommend checking that one out.

https://openwrt.org/docs/guide-user/installation/openwrt_x86

Ideally the PI also should to what the extra DSL Modem does… but I guess that's where the dram must stop. :D

But you might want VLANs anyway, so it's an interesting thing to consider.

The "router on a stick" paradigm using VLANs to a share a single physical port is perfectly valid. You're creating a "now you have two problems" scenario in which you need a VLAN-capable switch and have VLAN configuration to make.

I typically like the ISP router on a dedicated router port to make monitoring the physical link and/or cycling the physical link easier.

Unless your ISP is >1Gbps adding a second port to most devices is as easy as adding a USB NIC.

Ran openbsd for a few years like that, the base OS included everything needed. I recall it used 24MB of ram and closer to 30MB if ssh'd in. It was very handy to have a local login when playing with firewall rules.

Before Thunderbolt was common, people attempted to use external GPUs with this sort of expander, but it worked really poorly.

As an added bonus, you get atomic updates of all chains for free.

Granted, for simple usecases, ufw or firewalld may be simpler though.

Then there's the roaming issue. This is largely what the commercial "mesh" systems try to solve: deciding / helping inform when clients should switch APs. There are many solutions and none of them are without issues, including the commercial ones. Here's a starting point: https://openwrt.org/docs/guide-user/network/wifi/roaming

* WAN connection comes in by coax, into my cheapo cable modem (off screen), and then by Ethernet into the franken-NIC sitting on top of the laptop.

* The NIC on top is a normal PCIe card, but with the bracket missing. The ExpressCard riser [1] is connected by a mini-HDMI cable, the flat black cable, which curves up, around, and back in from the left side into the laptop

* Then, the blue cable on the side of the laptop is a VLAN trunk going into the Cisco switch on port 23/24, outside the picture.

* From there, another port on the switch is setup as an access/untagged port going into one of the LAN ports on the D-Link acting as the access switch

I don't think it was set up here, but at one point I also had a dock under the ThinkPad, with the serial adapter wired up to the switch's console port so I could manage everything by ssh'ing into the router.

[1] https://www.ebay.com/itm/115721630079

Also note that all the cables were hand-crimped because I was too cheap to buy new patch cables at the time.

I was in college, and truly had more time than money back then. it's the kind of doohickey made by only somebody very young, very crazy, or a bit of both. ;)

It seems like you weren't really asking, but I'll answer anyway.

It's bad security practice, and opens up your network to attack and/or compromise, you're massively increasing the attack surface, and a compromise of one of those components leaves the attacker sat on your edge router, at which point your entire network is fair game.

Generally speaking you shouldn't expose anything on your edge router / firewall, it's a safety barrier.

You can sit things behind it in a "DMZ" and port-forward and isolate them etc so that there's no packets terminating on the actual edge device itself.m, that lowers the risk of a full network level compromise.

Chances are you might be fine and never have a problem, but it's still recommended against.

E.g. is your pf-based load balancer running its rules before or after the global filtering rules? And if they're running first are they SNATing incoming traffic so the LAN rules allow the traffic through or does it need explicit exceptions for external IPs to traverse to a LAN endpoint?

If you're comfortable with more advanced networking then it's fine to run it all on one box. If you just want to open ports for internal LAN services then that is a very canned and well-supported feature for a gateway firewall.

E.g. see AirSnitch which resulted in large part from mixing too many complex networking rules in single devices.

https://docs.docker.com/engine/network/#published-ports

A year or two back, I was able to get a brand-new fanless Intel N150 with 4x2.5G ports with 16 GB memory for about $150 from AliExpress. I run Proxmox on it, with OpnSense and a couple other things in virtual machines. These days, due to tariffs and the memory shortage, that is more like $440 now, unfortunately. I am kicking myself for not buying two, not so much because of the price increase, but because it would have come in handy multiple times to have a second one on-hand for random experiments.

Given that CPU performance does _not_ tend to be critical for firewall/NAS use cases, if I had to replace it tomorrow, I would go onto eBay and get the highest-spec'd used Dell or HP mini workstation I could find for $120 and plug in a USB3 1gig ethernet dongle for the WAN side.

If you want maximum speed a Lenovo Thinkcentre m720q has a desktop Intel CPU and a PCIe slot. You can add a 2x SFP+ NIC and PCIe riser to get 10G.

- Soekris net4501 (x86, 486-class CPU) (discontinued)

- PCEngines alix2d3 (x86, AMD Geode LX800) (discontinued)

- PCEngines APU (x86, AMD T40E) (my current router/firewall) (discontinued)

I'm also currently using an APU2 as one of my wireless access points (with hostapd).

All of these have been solid machines that have given me zero problems.

The next system I plan to use is going to be a Banana Pi R4 (ARM Cortex A73), it's a solid choice for a simple router/firewall/DNS/DHCP box. It has a built-in 4-port gigabit switch where each interface can be used as normal Linux interfaces, as well as 2 SFP+ ports that are capable of supporting up to 10 gig ethernet.

It's also one of the few systems that offers true hardware offloading for connection tracking, so things like netfilter flowtables don't have to use any main CPU processing.

I'm currently experimenting with a Banana Pi R4 as a Wifi7 access point (running Debian with hostapd), however the current state of the wifi7 module for it (BPI-R4-NIC-BE14) and Linux driver (mt7996e) is still pretty young and a bit buggy (i.e., limiting transmit power to 6 dBm without patching the driver to override it, and there's apparently a lack of RF shielding which can contribute to low SNR on the receiving end). With the proper patches in place it makes a decent Wifi 6 access point. I'm hoping these issues get ironed out in the future and I can use it as a true Wifi7 AP. frank-w is doing outstanding work to help support the open source community with this new hardware.

Configuring FreeBSD is extremely straightforward.

Run it in what miniupnpd calls "secure mode" (which prevents clients from adding rules for IPs they can't talk from), put the daemons's rules after your manually-managed ones and -because of today's world of NAT hole-punching and "just tunnel it over HTTPS, it's the universal firewall bypass protocol" techniques- you're exactly as secure as if you had it off.

The ready-mades are brutally under-powered, have snowflake UIs that have to be re-learned on each revision, and have very short firmware update windows, so have to be replaced (and the UI re-learned) every few years. While with my OpenBSD thing, it's just `sysupgrade` then `pkg_add -u` -- for almost a decade now.

But yeah, if you're just copy-pasting without understanding, OPNSense is far better.

I'd be willing to bet, though, that the overwhelming majority of people who use consumer routers aren't doing anything remotely advanced. A how-to that covers the majority of use cases is valuable even when it excludes advanced use cases.