Sadly my bug is still present if you attempt to create a Virtual Machine with bHyve in a jail.
Appears to only effect limited servers: in this case an Cisco UCS 220 M5.
I was engaged with the early stages of a project where one of the founders wanted us to base a payment platform around FreeBSD. His reasoning boiled down to "Linux is too popular". The consequences of this choice would be significant, and that just isn't a justified position so I bowed out. The project, unsurprisingly, was a failure.
We've already had issues around compatibility with third party software and OS upgrades (which surprised me in particular, likely a mistake on the admin's part but I'm not privy to the details).
FreeBSD seems like a wonderful OS for purpose-built hardware (eg pfsense, PlaystationOS, Netflix's distribution servers), but I'd much prefer to be on something common for the general compute. I don't even want to know what hiring another/replacement admin will be like in the future (we're entirely on-prem in a city that's away from a major population centre, niche enough as it is without adding FreeBSD on top).
One major consequence is that your software and setup scripts are likely to continue to work over many major releases. Basic system tools don't often get replaced, so you don't need to worry about migrating to a new tool to configure interfaces, or a new firewall (although you do have three firewalls to choose from, they all have been there for a long time) or a drastically new init system etc.
Otoh, there are many projects that are made for Linux only. Sometimes intentionally, sometimes unintentionally. Lots of assumptions creep in, people write Makefiles that need GNU make, and call make recursively with 'make' instead of $(MAKE), but FreeBSD has BSD make as make, and GNU make, if you install it, as gmake. FreeBSD sets up cc to be llvm's clang, and I don't think that's common in Linux. Many projects compile with either, but some don't. Specialized linking is tricky. Compiler and linker changes are probably an exception to my rule earlier about not replacing tools, but there's a case to be made that clang is at least as good as gcc, and the license is a better fit; there's no requirement that the base be all BSD licensed, but it's nice to move in that direction where possible.
System monitoring is different, but capabilities are roughly similar. FreeBSD has some nice things, and Linux has some nice things, I've got my preferences, but whatever.
I don't think it's hard to take a reasonable systems aware person and sit them down with a FreeBSD book and a FreeBSD system and get acceptable results. Yes, mastery takes time, but consultancies are available if you need mastery right now, and don't have time to develop it in house.
It's up for you to take my recommendation and try for what you want the OS for rather than take my reasons to why.
I may recommend a Honda for that it's a Japanese manufacturer but the reason may be because my type of drinking cups fit the holders.
Why FreeBSD:
bHyve (Virtual Machines) within Jails. I have individual applications running in Jails, with proper ZFS integration including encryption.
The kernel is rock solid, its a fresh breath of air. Especially when to compiling your own.
Server stability is excellent minus this issue.
Everything just works, it feels less clunky than Linux in my opinion. That then turns in to a flame.
My personal opinion is that Systemd infuriates me. Microsoft has its grubby hands over it and many others. Political and non.
I don't have time to debate all this nor do people seem to accept that's how I feel about the Linux community. Both have their flaws and disadvantages.
Justifying my opinions is tiring and time consuming.
Besides full-on panics, I have regularly gotten live locks with Linux. Never with FreeBSD (or Solaris when I was admining that).
You made the right choice. That founder demonstrated domain incompetence alongside a demonstration that they don't know when to delegate, with a side of shallow reasoning.
Anecdote: I found a null pointer dereference in the OpenBSD httpd last week that allows an attacker to crash the server with a single crafted request. It's now patched.
I tried for a while to get a backtrace of the bug on OpenBSD, but their ancient gdb port doesn't support follow-fork-mode, and their clang port generates DWARF that their gdb port doesn't understand. Every time I reproduced the crash, the backtrace was just useless ASLR addresses. FreeBSD had none of these problems, and I was able to collect the backtrace in minutes.
Is it just that you already have a jail and need to add Bhyve to it?
[0] https://mysteriouscode.com/blog/sandboxing-the-hypervisor/
As a desktop OS?
I run very basic GPU (GTX 750), use ZFS for everything, run lxde, and use linux firefox as my main web browser (via the LinuxJails project at https://wiki.freebsd.org/LinuxJails).
For day-to-day use its nearly the same as linux, and runs nearly the same apps as linux. The biggest frustration I have is "linux recipes" for doing things often don't work (naturally, since the OS is different), and electron apps are a crapshoot, since the electron is chrome, and the emulation needed for chrome is incomplete. For example, I'm listening to spotify right now and I can listen to playlists, but can't just play a song..
EDIT: One the things I loved about Ubuntu LTS is the packages being essentially frozen, aside from security fixes. FreeBSD is a "rolling release" OS for 3rd party packages, even on stable branches. That's one of the reasons that I use the ubuntu LTS firefox ... I can apt-get update & apt-get upgrade just the web browser and its dependencies, rather than the 1700 unrelated packages that I'd need to update just to get the newest firefox.
For Spotify I use spotify-qt which is ok (not great) though with the latest version from ports it refuses to store my password in the KDE keychain any longer :(
Ps I actually love the rolling packages on FreeBSD while keeping a stable OS. On Linux these things are usually coupled. You can however change FreeBSD to quarterly updated packages if you so desire. Just set it to the quarterly branch instead of latest.
I mostly don't care if I have the latest version of libfoo, but I do care that I have the latest version of anything exposed to the internet (eg, firefox), and any security updates required to libfoo.
What I hate is when I do a pkg update; pkg upgrade firefox and it pulls in 1000+ pkg updates via dependency chains, some of which change how things work. That's why I like the linux jail approach. Back in the old days (early/mid 2000s) when things were simpler, I still ran linux web browsers for the same reason. I could just download the tarball from mozilla or opera or whatever and update to the latest browser binary without taking a chance on fubar'ing my system.
I use Xfce and Nvidia drivers on a 3060 but AMD worked too when I was using an 5700 XT.
Processor: Ryzen 5950X - no issues
Sound, I'm using a PreSonus via USB
Ethernet (realtek) too works nicely.
WiFi I've not attempted but probably would. Can't fault it.
That stuff happens a lot less on FreeBSD.
- http://www.slackware.com/info/
A person could reasonably argue that Slackware was always a BSD that happened to use the Linux kernel:)
If someone did the opposite of Debian/kFreeBSD I would be interested. FreeBSD/kLinux?
The kernel feels solid and bHyve is a goddess.
Jails are great for running isolated browsers with sound.
Does it work well with graphics?
I just started using FreeBSD and one of the first thing I did was to follow the Handbook instructions to set up a minimal Debian in Bhyve that I then used to access my old Linux partitions to copy files. Works very well for that and I have no complaints, but I also can't say I noticed any major improvement over Qemu?
But I made a quick attempt to also install something with graphics, probably Lubuntu, and I did not get that to boot. Is the graphics good enough to play games and run some Linux-only applications with reasonable performance?
You can pass though hardware devices including graphic cards but you need to have it as a secondary and unused. bHyve is unable to use the primary card as well as a shared device for the guest.
I'm not even talking about 3D passthrough. Just a fake VGA adapter you can see in a window so you can run an installer.