No, the I/O passthroughs are fine. Proxmox and HA are fairly great at keeping them stable. I have passthrough for WiFi/BT/USB zigbee/USB thread Of course it pins you to a singular proxmox host, not benefitting from proxmox HA, but that's the way the cookie crumbles with h/w.

pfsense and home-assistant both claim to be declarative configs, which is technically true. However the config files are not well or effectively documented, and where there is documentation it typically relates to the GUI which diverges significantly in arrangement. Their configs are declarative in that they declare the way their internal processes are configured, not in the way that they should interact and appear to other services (networking people will find that statement very confusing).

Both are effectively "Operating Systems" within operating systems, starting/stopping/configuring/managing other programs, home assistant is doing this to the nth degree. When you start them it is very hard to determine when they have actually started - particularly the bits you care about. Getting errors and logs out of them is painful. Updating configs and restarting has multiple routes, the longest of which is very long.

Both are reasonable ways to get to grips with the problem areas they solve for; they are not optimal however.