I had a similar experience and ended up doing the same. OPNsense does include a userspace implementation of Wireguard, but is obviously inferior to what Linux already has. It ended up being that, how poorly the initial kernel space implementation was handled for FreeBSD and that I wanted Cake for QoS, which made me move back to OpenWRT.
I do want to ask though, has there been any downsides to using OpenWRT on x86?