OPNsense: Open-source security platform
opnsense.org
opnsense.org
[1] https://homenetworkguy.com/how-to/set-up-a-fully-functioning...
I'd certainly recommend grabbing something like a Protectli box (if power draw is a concern) or building a small server with NICs to install OPNSense on over the Ubiquiti stuff.
For me, the router graveyard was getting out of hand, buying everything from mid-range to high-end consumer routers only to have them left behind software-security wise within 3 years, I needed something open.
The promise of the EdgeRouter range was the hardware offload and Debian based OS, but Ubiquiti has fallen out of favour with me in that space, their software has gotten worse rather than better over time.
I went with Protectli because of the ability to use Coreboot meaning I could get as much of the stack Open Source as possible. Unfortunately, at the time I set it up, OPNSense didn't work for my requirements and I had to use pfSense; pfSense is now too falling out of favour, but I don't have the time to swap over a fairly large home network without pissing off the family.
As a bonus I have xcp-ng spin up a ubuntu server with Portainer running things like pihole / unifi controller / plex / a terraria server. I also decided to randomly throw in a cheap nvidia GPU for some extra oomph. Highly recommend it for tinkerers, and the only reason I would spend more for the protectli is if the fan noise of a mini PC might be a nuisance.
I learned a lot in the last 3 weeks it also helped me understand networking a lot more. Sure I had to do a lot of trial and error or figure out why things don't work but in the end it was worth it for me.
I came from 30€ OpenWRT routers with only 100mbit links which is why I upgraded.
The hAP ax³ were 140€ a piece (I use two) and the one 2.5Gb PoE port is actually nice since it powers the second router. It was a pricey upgrade compared to what was there before. I tried to buy another OpenWRT router but RouterOS seems to offer 10 times more compared to what OpenWRT can do.
I've looked at a lot of things even one of the more looked at super cheap thin client for this stuff (Fujitsu Futro S920).
At home I can't really justify a dedicated firewall.
So far the only one which is never blocked is cloudflare’s WARP but I suspect it’s because it’s newish (the VPN functionality of WARP is at least, originally it was mostly about DNS). But it has severe performance problems when running on a router as far as I can tell, I suspect it has to do with MTU, but I couldn’t solve it so far.
Next I was going to Google One’s VPN, but I don’t think it’s a candidate for me, I want to use a VPN for privacy, so google is probably not the best choice.
The sites which are blocked are mostly enterprise login pages. As I work from home, it’s a no go for me. Secondly, I need to connect via VPN to some company network a couple of times a day and the performance of “VPN over VPN” is currently catastophic, it’s just unusable. Not sure if it’s also related to MTU…
Sounds like an issue that could be solved by routing specific traffic/domains not over the VPN?
pfSense/OPNsense is just a stripped-down FreeBSD under the hood. If there are things that you want it to do that you can't with the base install, you can look into installing FreeBSD packages.[3][4] I personally just use vanilla OpenBSD for most edge router/firewall tasks and then if I need mesh wifi or some modern gear that is not well supported, I just delegate the BSD box to run a strong PF firewall (which is what pfSense uses) in front of it all. Ubiquiti mesh wifi and RTSP stream play really nicely with it.
It's so easy to overcomplicate a routing setup which is probably why these off-the-shelf solutions are so popular, but I agree, I prefer to have less of a magic box approach to routing/firewalls whenever possible.
[1]: https://upload.wikimedia.org/wikipedia/commons/3/37/Netfilte... [2]: https://mailing.openbsd.misc.narkive.com/jtIB9W3w/pf-packet-... [3]: https://docs.netgate.com/pfsense/en/latest/recipes/freebsd-p... [4]: https://forum.opnsense.org/index.php?topic=21739.0
Note: like vogon said above, it's easy to overcomplicate things like this. If you go this route, start with the most basic configuration that works, and only go looking for 'tuning' or 'optimizing' if you run into a specific issue and can measure before-and-after improvements. Obsd does a very good job of working correctly out of the box, and if you follow the advice of some 'popular' PF tuning sites, you'll end up with tons of stuff you probably don't need, probably will interact in weird ways and will get you laughed at when you go looking for help. KISS.
One caveat: Don't expect to go to the obsd mailing lists or IRC channels for beginner support.
I'd love to build a BSD based router/firewall in a declarative manner/source controlled configuration on top of a vanilla OS, I just don't have the time (specifically network downtime for the home, family) to play with it.
I like graphs and charts, but otherwise not a massive fine of GUIs that hide functionality and complexity, would rather know/understand exactly what's going on at the CLI.
Thanks for the links, no time like the present to learn more!
I get that OPNSense can do this, but do you need a switch with a capability to make sense of this?
I'm considering moving from Unifi USG to OPNSense and have two Cat6a runs from one end of the house to the other (through the loft and it's not possible to add more runs without building/decorating work). Presently the two cables do WAN and LAN, but I've been curious about putting something closer to the modem and to somehow use both cables for the LAN.
LAGG looks like it can do this and isn't something I knew about for the home.
Would I need a special switch on the other end? I've currently got Ubiquiti switches but as I'm already looking at binning the USG I'm fairly open to reconsidering a lot of the network.
PS: The reason to abandon the USG is heat issues. Packet loss when the ambient room temperature exceeds 30'c, and serious packet loss when the room temp is 35'c. This is no longer rare, and the USG is only rated to ambient temp of 40'c and there are many Reddit threads of people ripping the case apart and fitting fans. I'd rather just have stable internet with better hardware.
That was enough for me to find https://help.ui.com/hc/en-us/articles/360007279753-UniFi-Net... and know that I can mix OPNSense and my existing Unifi switches just fine as as it's all LACP.
packet loss at summer temps is indicative of faulty hardware (maybe just the thermal paste or other parts of the heat management)
there is no heat management in the Ubnt USG. no thermal paste on the hot network ports, no cooling design beyond passive cooling (little air holes in the side of the case) that doesn't work when it's laid flat (need to vertically mount to encourage airflow).
most of the hacks are people fitting fans in a case that isn't designed for it: i.e. https://old.reddit.com/r/Ubiquiti/comments/cr88fw/cooling_th...
if the USG is in a cupboard or somewhere with poor airflow, and it's the Summer, then it's packet loss city.
would agree that this is a faulty hardware, faulty by design.
Got any recommendations for something that can route beyond gigabit, when NATing and DPIing and other things?
goto recommendation for the last decade is a used small formfactor enterprise desktop or a laptop. the former allows for a pcie nic and more performance while the latter usually requires a usb-eth but has a builtin keyboard and screen for debugging.
just go for something x86, avoid the ultra low-end cpu's and your usually good for soho stuff.
Do you (or anyone else reading) have any suggestions?
Right now the software update function dies half the time I try to run a check, with a long sqlite query string / error being dumped to the console. This has been going on for at least the last couple of months worth of releases.
About a year or two after install, reboots and power-offs stopped working. The system just hangs instead after printing out a message about USB, and I cannot figure out for the life of me what's wrong. It's a standard Dell SFF PC, nothing exotic, and had been working fine until a major release broke it. FreeBSD's documentation about ACPI is impenetrable, so I can't figure out what's going on.
Startups and reboots used to be lightning quick, with maybe a minute or less between the bootloader kicking off loading the kernel and interfaces/routing/firewall up and it giving its happy chime. These days the system spins its wheels for ages doing...something, not sure what.
I find the project pretty outdated and behind the times. The UI purposefully obtuse with terrible organization and field names and a lot of missing help text to keep their support/consulting biz strong.
They're really far behind on features. There's no application blocking, monitoring/diag is rudimentary, it has extremely limited backup functions (Google Drive and that's it, I believe), and even the DNS blocklist functionality is extremely rudimentary, with only a fixed list of really trashy, unreliably lists available to pick from (one of the groups they pull lists from has demonstrated extensive issues with QA, routinely including things like certificate validation servers in their blacklists.) They've also gone out of their way to make the Adguard Home plugin annoying and confusing to get working if you want to configure it as a proxy to unbound, which is needed if you want DHCP hostname records to work (speaking of which, DHCP leases are needlessly obtuse to mange.)
Their release process is wildly unsuitable for production network equipment. A 'major' release is immediately EOL'd as soon as the next major release comes out. Running 20.1 and need to stay on it because 20.2 breaks something or you want to wait for the dust to settle? Too bad. There's no security releases for older major revisions. And it wouldn't be so bad if each major release was followed by a number of "oops we fucked up...." point-point releases because their QA isn't very good.
The devs are sticks in the mud, too - mostly "franco." They bitched and moaned up a storm for YEARS about wireguard being "insecure" despite no evidence to back their claims, citing that as the reason for refusing numerous requests for integration, and even refusing code contributions from the community for it. They eventually caved. The wireguard plugin is still pretty meh and difficult to navigate unless you know wireguard well.
ARM support? Zero interest in even assisting community efforts, which have gotten impressively far with it, especially now that ARM support in FreeBSD got appreciably better in the last release or two. I suspect it's because they see it as a threat to their (grossly overpriced) hardware offerings.
The list goes on.
They forked pfSense (a good thing, the pfSense devs were being massive dicks) but seem to now be largely on "cruise control" and leveraging community goodwill.
My initial read here would be that this is where serial port / console redirection is happening. There should be settings in the BiOS you can look at however I don't know how limited PC's are in their options / functionality.
>I find the project pretty outdated and behind the times. The UI purposefully obtuse with terrible organization and field names and a lot of missing help text to keep their support/consulting biz strong.
pfSense was the same way as well as most projects if you understand the underlying configurations. You can find people saying the same about ubiquiti's interface in this thread as well. In my experience the GUI is to capture the 80% of mostly default configurations.
What other firewall/routing software have you looked at in comparison to opnsense? I'm interested in what other features they have? The API interface and IDS functionality was one of the draws for me.
>Their release process is wildly unsuitable for production network equipment. A 'major' release is immediately EOL'd as soon as the next major release comes out. Running 20.1 and need to stay on it because 20.2 breaks something or you want to wait for the dust to settle? Too bad. There's no security releases for older major revisions.
I mean are you saying this as a paying customer? Free always has its risks and costs.
> The devs are sticks in the mud, too - mostly "franco." > wireguard being "insecure" > ARM support? Zero interest in even assisting community efforts
I observe this with projects overtime and this usually just adds to the bloat and disorganization because everyone is looking for "their" one-stop solution. I think its useful to consider things from other view points and complexities you may not have insight into. Not that I have any specific insight into this project however, there are other companies that make a lot money off networking gear, firewalls, etc and provide what you are asking for but the price isn't free. I assume most open source projects are "best-effort" unless they have a formal revenue stream or foundation behind them and even then I wouldn't expect any claim to expect features or support (not saying you are, just generally).
They could likely do well selling one of them as a supported solution.
What if thats not their business model? What if it requires hiring or finding a dedicated ARM developer? What about security? If a zero day comes out now what you have (3) different architectures to support and test. Even if they offered it as a supported solution would most of the people complaining fund the work through support. Probably not, because they expect the software to just "work" for them, for free.
I looked through the forum on one of the first ARM[1] posts and as expected (2) pages in and it becomes a tech support thread for people who want to try the latest but be handed the answers. For a project where they document[2] the development workflow, architecture, and environment its a bit difficult to understand the complaints when its opensource. Clone the repo and get to work.
[1] https://forum.opnsense.org/index.php?topic=12186.0 [2] https://opnsense.org/developers-invitation/
Well, you don’t have to wonder, the owner of the domain was revealed by court action to be Jamie Thompson, one of the two founder of Netgate, which sells the commercial version of pfSense. Surprise…
not to dispute your points but i rekon this goes along dealing with semi-noobs and prosumers that don't necessarily bring any business but demand attention for the better part of two decades.
With PCEngines shutting down it’s almost impossible to find reliable, cost effective hardware. I hope pfSense gets back on track because their hardware seems ok.
I’m currently running Mikrotik x86 on a NUC, but their hardware support on x86 isn’t great. I’d rather switch to Vyos, but I’m too lazy to learn their CLI.
Semi-related: OpenWRT is also genuinely not bad on x86, but it’s slightly too simplified compared to say Mikrotik or VyOS. It also runs great in a VM.
Can you elaborate on this?
> It seems to have a similar architecture as a Ubiquiti controller.
That’s a hell of an insult to be tossing around for an unreleased product. Lmao.
And btw: > Everyone can build an LTS release image from the stable branch too. For 1.2.x, the branch is named “crux”. The image built from the branch is equivalent to the latest official LTS image.
By far the biggest problem is that they don't give enough consideration to recover-ability of offsite devices. If something causes a device to disconnect from the controller their solution is to SSH in and re-adopt it. That doesn't work once you're dealing with hundreds of devices across dozens of sites.
A good example of where that becomes problematic is to look at the controller hostname override setting they have. You could change it to 'unifi.invalid' and it would happily push that change out to a thousand devices and leave them in a state where you'd need to be hands on with every device to recover. If you can do it on purpose, they can do it accidentally with a buggy update, so, IMHO, there's always a risk that an update could break things pretty badly.
That's not a hypothetical either. They (purposely) pushed an update that did something like that when they started supporting HTTPS for the inform URL several years ago.
The second issue I have with it is more of a design issue. Sites should be somewhat sharded and I should be able to update the controller version on a per-site basis. I think that does a lot to reduce the risk of an update breaking things.
I also dislike the default settings and prompts for auto-updates. I like the scheduled updates and think they're great, but the push to "update everything daily at 3:00 AM" is too much. I have a controller with 100+ sites and need to schedule updates to ensure any breakage is fixable via manual intervention up to the point of physically visiting a site. Edit: To clarify this, I'm sure I've been prompted to enable 3:00 AM auto-updates on the newer controller versions and accidentally clicking "yes" would be a huge headache for me.
The "rolling update" was also pretty trashy when I tried it. That was years ago, but I think it simply updated APs sequentially and happily continued if the previous one didn't come back up. How hard is it really to implement a rolling update that stops and waits for intervention if even one device doesn't come back online?
And the UI. I can't even use it without setting my browser to 80% zoom and it gets worse every time they push out an update. Everything is stuck into tiny little scrollable boxes. I have multiple 27" monitors and get stuck scrolling around in a 1" x 2" box that can't display more than 2 or 3 lines of config. Why?
And then TP-Link copied them with Omahda. It's almost funny. I wonder if they even realize they're copying off the dumbest kid in the class. Lol.
That said, is there anything better? Mikrotik, while more configurable, is so much worse and feels straight out of the 90s design-wise.
Beyond those options I’m not aware of anything remotely better.
I haven’t found anything better for switches and APs. I use pfSense for firewalls.
I run OPNsense (very happy with it) and had a B, so I followed this guide (https://docs.ibracorp.io/opnsense/) and I now have A+.
Do you have any information on fq_codel vs cake?
https://www.bufferbloat.net/projects/codel/wiki/
https://www.bufferbloat.net/projects/codel/wiki/CakeTechnica...
[1]: https://www.phoronix.com/news/FreeBSD-WireGuard-Lands-2022
[2]: https://arstechnica.com/gadgets/2021/03/buffer-overruns-lice...
Also no problems with wireguard... Using it to vpn in and also out for some routes to mullvad.
A few rough edges in the UI, but I got the basic routing functionality running within minutes, and got Wireguard going with the help of a guide from Home Network Guy pretty soon after.
Since then I've bought a second box and will be setting it up soon, too!
It’s something tech influencers should mention more often
They don't just give you a fish, they teach you to fish, then it's in your hands.
no updates, means you are sure to collect known deficiencies.
> Frequency of zero day RCE fixes should not be taken as indication of better security.
true, but lacking an alternative metric, it's at least helpful to gauge the willingness to not ignore problems.
Migrated from pfsense because OPNsense actually had a usable API, so we can do things like add people into the Captive Portal programatically from our event check-in system.
Has anyone attempted to run OPNsense on Netgate hardware?
some tried, but no one is left to tell </s>
I'm currently upgrading my box to add an NVMe M2 SSD, so I'm moving the VM to a spare box, doing the upgrade, then moving it back. The VM itself knows nothing, no reinstall and reconfigure, minimal downtime. Easy af.
Was using pfSense for a few years until a couple of years ago, upgraded to OPNSense. The plan is to keep OPNSense for much longer than a handful hardware cycles.
Even SIP calls don't get disconnected
Is it relying mostly on switches for the physical connectivity, including upstream?
If my PON is connected into one of the nodes, how do I allow the router to roam if a node fails? Will look into openvswitch.
Not really ever done virtualized networking so it doesn't immediately click for me.
I am actually interested in OPNsense but I can’t actually find out if it meets my needs or if I can contribute because the website sucks so much.
NixOS is a Linux distribution managed by nix-the-package-manager.
but you should make sure you didn't turn dumb things on like WebUI/SSH acces from WAN etc..
this is a good thread on the subject https://forum.opnsense.org/index.php?topic=22117.0
Edit: I checked the source and they are already utilizing it [1].
0. https://github.com/opnsense/dhcp6c
1. https://github.com/search?q=repo%3Aopnsense%2Fcore+dhcp6&typ...
I’ve been running on a small Protectli box and mainly using it for normal firewall stuff, Snort and as a Wireguard server.
A big thread on serve the home https://forums.servethehome.com/index.php?threads/cwwk-topto...
Alternatively the Protectli’s https://protectli.com/vault-4-port/ . Protectli gets you support and coreboot, next day Amazon shipping but more expensive and older hardware.
These are great devices for running Opensense bare metal or virtualised in Proxmox. I use Proxmox on a protectli so can run unifi controller and ad guard home in lxc containers with Opensense in a VM. I’d buy a cwwk if buying again, I just didn’t know about them at the time.
[1] https://sschueller.github.io/posts/wiring-a-home-with-fiber/...
Here is a post with some more details. https://forums.servethehome.com/index.php?threads/fujitsu-fu...
Protectli was nice to buy from, but I do think there's a lot of similar options.
The jest part about Protectli is the Coreboot and they provide instructions to compile your own, so it even feels open.
That said, fully configured with RAM and storage mine cost roughly £600.
Given electricity prices in the UK, and my requirement for an as-open-as-possible software stack, it was well worth it. If it outlasts the EdgeRouter it replaced (7yrs before power circuitry died) I'll be very happy.
That said, my main concern, as a reply to you pointed out, is that I'll want more than Gigabit in that time; my street is getting 10Gb/10Gb in the next couple of years.
I think they should upgrade their NICs to 2.5G and 10G though. 1G is no longer future proof.
I had one availability blip a few months ago, don't know what it was but a full hardware reboot fixed it.
I have it running proxmox, with a debian pivpn/pihole + op lnsense VMs. I believe I have 12 gb ram in it.
[1] https://shop.opnsense.com/product/dec3840-opnsense-rack-secu...
I've had it running on a dell SFF desktop for years.
https://shop.opnsense.com/product-categorie/hardware-applian...
I spent $43.97 (which included shipping) for the Optiplex, added a 2 port NIC for $16.37, and it's been running my house great since then.
At our current electricity prices, if that machine ran at 100W, you'd be spending the equivalent of what you paid for it EVERY month in electricity.
I've always seen Americans, who typically have more space and cheaper energy and fuel suggest people grab 1U servers for $40 etc, and respond just like you did when people ask them _not_ to.
My car also does 50mpg, and I still pay more pay mile for fuel.
In Europe, we need to spend more to buy efficient tech because the running costs require it.
I think the misunderstanding is the assumption about form factor. The 7050 series is available in a case the size of a paperback book.
Despite DELLs willingness to slap the same series number on everything from an ATX tower to a "paperback book", they are far from the same thing.
I'm willing to accept I may be wrong, so let's see.
First page, the one on the right. I don't see why you'd doubt an 8-watt idle, since it's basically a laptop processor and chipset. Please note that these models are available with both 35W and 65W TDP processors; I'm talking about the (generally cheaper) 35W models, of course.
I doubt the original poster is talking about the small "book" sides micro form factor at ~$40.
I expect that one to cost at least double, if not more (a quick eBay search outs the micro form factor at closer to a £200 average), and the tower form factor, with it's 240W PSU is likely to be drawing much more than ~8W in a typical configuration.
My point stands; if you want to spend £35 on an x86 machine, youll pay in electricity costs, spend ~£200+ and overall youll likely save money overall if it lasts a decent period.
if you want to be sure, use an old (skylake and above) laptop and give it a usb3-eth plug. my latitude e7470 idles at around 3W (powertop)
The reason why I don't expect it to draw as much is because I have an 8-core xeon with 2x10k RPM + 4*7200 RPM drives, a dedicated raid card, integrated BMC, and it reports ~100W power draw when booting up. When sitting around doing nothing but with the drives spinning, it reports a draw of about 80W.
edit: The BMC alone draws 11W, judging by the reported power consumption when the server is off, but plugged in.
However, if you want to have a really small appliance-like pc, then yeah, the NUC-sized ones are much better.
Also, the AMD processor in those systems is getting extremely long in the tooth.
They’re very close though. You can follow Pfsense guides on opnaense basically
But the actual heavy lifting of these router/firewall systems is done by code that ships with FreeBSD.