Hacker News Reader

Comment by syntaxing | Hacker News Reader

Recently transitioned from Ubiquit stuff to a OPNSense setup. It was such a good decision. The firewall rules make much more sense. Better functions than the dream machine series. You can also get a lot more for the same price. Ubiquti hardware is very under spec for the money you pay for. Highly recommend this guide to setup your own. It’s very dense and more verbose than you need so skip the irrelevant sections.

[1] https://homenetworkguy.com/how-to/set-up-a-fully-functioning...

View on HN

I switched to pfSense from EdgeRouter a few years back, and find the firewall rules make _less_ sense. The reason is likely that I understand IPTables pretty well, where as the approach used in pfSense seems "abstract" in comparison.

I'd certainly recommend grabbing something like a Protectli box (if power draw is a concern) or building a small server with NICs to install OPNSense on over the Ubiquiti stuff.

For me, the router graveyard was getting out of hand, buying everything from mid-range to high-end consumer routers only to have them left behind software-security wise within 3 years, I needed something open.

The promise of the EdgeRouter range was the hardware offload and Debian based OS, but Ubiquiti has fallen out of favour with me in that space, their software has gotten worse rather than better over time.

I went with Protectli because of the ability to use Coreboot meaning I could get as much of the stack Open Source as possible. Unfortunately, at the time I set it up, OPNSense didn't work for my requirements and I had to use pfSense; pfSense is now too falling out of favour, but I don't have the time to swap over a fairly large home network without pissing off the family.

I recently switched to RouterOS. The learning curve was a bit high to me. I'm still tinkering with it but I got my main things working - vlans, default internet access out via vpn, one port (internet accessible server) routed without vpn.

I learned a lot in the last 3 weeks it also helped me understand networking a lot more. Sure I had to do a lot of trial and error or figure out why things don't work but in the end it was worth it for me.

I came from 30€ OpenWRT routers with only 100mbit links which is why I upgraded.

The hAP ax³ were 140€ a piece (I use two) and the one 2.5Gb PoE port is actually nice since it powers the second router. It was a pricey upgrade compared to what was there before. I tried to buy another OpenWRT router but RouterOS seems to offer 10 times more compared to what OpenWRT can do.

I've looked at a lot of things even one of the more looked at super cheap thin client for this stuff (Fujitsu Futro S920).

At home I can't really justify a dedicated firewall.

RouterOS has a steeper learning curve, but I find it very comforting that it is predictable/boring and I run it on hardware that is easily replacable. I have a cheap Mikrotik 1U unit.

May I ask which VPN you are using? I have tried several and they are always blocked somewhere. I tried: Mullvad, IVPN, ProtonVPN.

So far the only one which is never blocked is cloudflare’s WARP but I suspect it’s because it’s newish (the VPN functionality of WARP is at least, originally it was mostly about DNS). But it has severe performance problems when running on a router as far as I can tell, I suspect it has to do with MTU, but I couldn’t solve it so far.

Next I was going to Google One’s VPN, but I don’t think it’s a candidate for me, I want to use a VPN for privacy, so google is probably not the best choice.

The sites which are blocked are mostly enterprise login pages. As I work from home, it’s a no go for me. Secondly, I need to connect via VPN to some company network a couple of times a day and the performance of “VPN over VPN” is currently catastophic, it’s just unusable. Not sure if it’s also related to MTU…

I use mullvad with the specific one being de-dus-wg-001 (Germany Düsseldorf Wireguard 001) most of the time.

Sounds like an issue that could be solved by routing specific traffic/domains not over the VPN?

That's possible??

I'm the other way, I find IPTables less intuitive and featureful than PF. The way Netfilter/IPTables works under the hood seems much more over-complicated[1] than it needs to be (which is how I feel about a lot of Linux system stuff these days). The architecture of PF[2] (and thus how the rules get processed) just makes more sense to me.

pfSense/OPNsense is just a stripped-down FreeBSD under the hood. If there are things that you want it to do that you can't with the base install, you can look into installing FreeBSD packages.[3][4] I personally just use vanilla OpenBSD for most edge router/firewall tasks and then if I need mesh wifi or some modern gear that is not well supported, I just delegate the BSD box to run a strong PF firewall (which is what pfSense uses) in front of it all. Ubiquiti mesh wifi and RTSP stream play really nicely with it.

It's so easy to overcomplicate a routing setup which is probably why these off-the-shelf solutions are so popular, but I agree, I prefer to have less of a magic box approach to routing/firewalls whenever possible.

[1]: https://upload.wikimedia.org/wikipedia/commons/3/37/Netfilte... [2]: https://mailing.openbsd.misc.narkive.com/jtIB9W3w/pf-packet-... [3]: https://docs.netgate.com/pfsense/en/latest/recipes/freebsd-p... [4]: https://forum.opnsense.org/index.php?topic=21739.0

I use exactly the same thing after I got tired of pfSense: straight OpenBSD, PF, unbound, and a couple of other native obsd things (starting to play with OpenBGPD for AS level filtering) on NUC-alikes (currently liking Protectli and Seeed). No pretty web interface, but the documentation is so good it was pretty trivial to set up. This has worked really well for several years now.

Note: like vogon said above, it's easy to overcomplicate things like this. If you go this route, start with the most basic configuration that works, and only go looking for 'tuning' or 'optimizing' if you run into a specific issue and can measure before-and-after improvements. Obsd does a very good job of working correctly out of the box, and if you follow the advice of some 'popular' PF tuning sites, you'll end up with tons of stuff you probably don't need, probably will interact in weird ways and will get you laughed at when you go looking for help. KISS.

One caveat: Don't expect to go to the obsd mailing lists or IRC channels for beginner support.

I definitely believe PF/Netfilter is the way forward, I just don't have the experience with it yet.

I'd love to build a BSD based router/firewall in a declarative manner/source controlled configuration on top of a vanilla OS, I just don't have the time (specifically network downtime for the home, family) to play with it.

I like graphs and charts, but otherwise not a massive fine of GUIs that hide functionality and complexity, would rather know/understand exactly what's going on at the CLI.

Thanks for the links, no time like the present to learn more!

I've had great success with an old Dell Optiplex computer running xcp-ng with PCI passthrough of the NIC to pfSense. I was considering a protectli box, but for $300 on ebay + an old NIC I had laying around I got something much more powerful (Intel I5-8600 w/ 16GB of RAM) than even the most expensive protectli offering, which was ~$1500 last time I priced it out. I calculated out the power draw and the Optiplex ends up with only a cost of about an extra $1.10 on my power bill per month at high usage, which was the deciding factor.

As a bonus I have xcp-ng spin up a ubuntu server with Portainer running things like pihole / unifi controller / plex / a terraria server. I also decided to randomly throw in a cheap nvidia GPU for some extra oomph. Highly recommend it for tinkerers, and the only reason I would spend more for the protectli is if the fan noise of a mini PC might be a nuisance.

How does the link aggregation work?

I get that OPNSense can do this, but do you need a switch with a capability to make sense of this?

I'm considering moving from Unifi USG to OPNSense and have two Cat6a runs from one end of the house to the other (through the loft and it's not possible to add more runs without building/decorating work). Presently the two cables do WAN and LAN, but I've been curious about putting something closer to the modem and to somehow use both cables for the LAN.

LAGG looks like it can do this and isn't something I knew about for the home.

Would I need a special switch on the other end? I've currently got Ubiquiti switches but as I'm already looking at binning the USG I'm fairly open to reconsidering a lot of the network.

PS: The reason to abandon the USG is heat issues. Packet loss when the ambient room temperature exceeds 30'c, and serious packet loss when the room temp is 35'c. This is no longer rare, and the USG is only rated to ambient temp of 40'c and there are many Reddit threads of people ripping the case apart and fitting fans. I'd rather just have stable internet with better hardware.

I think there are multiple ways to set up a lagg, but the one I've tried and works fine is LACP. This requires an equipment that can handle this at the other end, though. I'm not familiar with Ubiquiti kit, so you'd have to check if it's supported.

if you want to have two links grouped together so that a single transfer uses both links together (2gbps) then the switch has to support it. default is for two separate transfers to each get one gbit, which is good enough for most applications.

packet loss at summer temps is indicative of faulty hardware (maybe just the thermal paste or other parts of the heat management)

> packet loss at summer temps is indicative of faulty hardware (maybe just the thermal paste or other parts of the heat management)

there is no heat management in the Ubnt USG. no thermal paste on the hot network ports, no cooling design beyond passive cooling (little air holes in the side of the case) that doesn't work when it's laid flat (need to vertically mount to encourage airflow).

most of the hacks are people fitting fans in a case that isn't designed for it: i.e. https://old.reddit.com/r/Ubiquiti/comments/cr88fw/cooling_th...

if the USG is in a cupboard or somewhere with poor airflow, and it's the Summer, then it's packet loss city.

would agree that this is a faulty hardware, faulty by design.

You need a switch that can handle LAGG but I don’t think you need a particularly expensive or fancy switch. I run a TP-Link jetstream switch that supports LACP and was only $110 (same price as a 8 port lite ubiquti switch). It has 8 port PoE+ AND 2 port SFP. You have to run a omada controller (similar to a unifi controller) but you can buy a box (OC200) or just run a Docker image which is what I do. I find tbe TP-link APs much better performing at a reasonable price too. Ubiquti with their no free shipping and price increases has gone completely out of hand since the pandemic. Made me switch out all their hardware because of it. Ubiquti hardware was simply not worth their price and OPNsense + omada had blown my Ubiquti setup out of the water.

Thanks.

That was enough for me to find https://help.ui.com/hc/en-us/articles/360007279753-UniFi-Net... and know that I can mix OPNSense and my existing Unifi switches just fine as as it's all LACP.

Alternatively, you can pick up a used switch from Cisco, Dell, etc. that have excellent support for LAGG and other useful things for significantly less than the TP-Link (as long as you don't need 10G). I think the last 2960Gs I picked up were less than US$30/pc. Down side is 1) they use more power, 2) bigger switches (24- or 48-ports) are much louder, and 3) not going to see any new OS updates for most of them. So you have lots of options.

Old Brocade switches are the sweet spot. Cheap and OS is still updatable!

I shall check those out; thanks for the tip. I will say I checked and it looks like my elderly Dell switches have gotten an update in the 2020s, so perhaps it's just Cisco who said "nope...we're done with IOS 12".

I've been looking at this as well. I'm currently running my gigabit fiber connection through a GPON-ethernet media converter and from there over Cat 6 to a ubiquiti Edgerouter-X. It does okay with hardware offloading enabled for things like ipsec and NAT, but it's taxing the CPU and I'd like to move to something with a little more headroom.

Got any recommendations for something that can route beyond gigabit, when NATing and DPIing and other things?

> Got any recommendations for something that can route beyond gigabit, when NATing and DPIing and other things?

goto recommendation for the last decade is a used small formfactor enterprise desktop or a laptop. the former allows for a pcie nic and more performance while the latter usually requires a usb-eth but has a builtin keyboard and screen for debugging.

just go for something x86, avoid the ultra low-end cpu's and your usually good for soho stuff.

You should checkout the new R862 mini pcs. You can get 3X 2.5 NIC and 2 SFP port with 10G support. The homenetworkguy did a great review on it. But overall, 10G works but only with nothing turned on. With IPS/IDS and proper MTU, you can get about 3 Gbps. But all this for sub $400 in a computer about the size of your hand is insane to me.

I think you mean R86S, I wasn’t able to find something relevant by R862. They’re pretty interesting. I am sort of looking to replace my R210ii and this fits the bill but downgrading cpu performance (for a great deal of efficiency gain) kind of feels bad.

Aha yeah, I’m on mobile so it was a typo. I currently run a fanless mini pc with N100 and 4X 2.5G which has been great. If you don’t need SFP and can live with LAGG + 2.5G, you can probably get pretty similar real world speeds compared to a 10G SFP line.

I’ve been thinking about swapping out my UDMP for opnsense, but keeping the ubiquiti APs, they’re the best I’ve ever known and I think they play reasonably well with non-ubiquiti stuff. Thanks for the link to the guide!

You can actually run the controller with a plugin too! Worked great, didn’t have any issues when I was still using my ubiquiti switches and AP.

Reply on news.ycombinator.com