I suspect interest in FreeBSD will rise with the demise of CentOS. What are the DevOps best practices for working with and deploying to jails instead of Docker?
OS package mismatches will bite you in the ass years down the road if you try to maintain thin-jails.
I manage alot of sites that Management refuses to invest any dev-time to upgrade old Python2.7 infrastructure. Having fat-Jails allows me to keep everything else upgraded.
pf is a fantastic firewall, but the syntax in OpenBSD is different then in FreeBSD.
VNET is worth it's weight in gold. Each jails has it's own network stack. That let's you avoid funky localhost workarounds.
iocage is a great framework for jails management (but it isn't necessary) you can do everything from jail.conf. Personally I prefer it for it's ZFS integration.
My (personal) unbound jail has 95 files in it. The reason it has so many files is because it is simultaneously my 'nsd' jail.
Which is to say, I actually point two different 'jail' commands at the same directory - each with a different IP and command line:
/usr/sbin/jail /jails/dns ns1 1.2.3.4 /nsd/nsd -c /nsd/nsd.confSo /jails/dns is a holder for two different dns servers and they run, jailed, from the same root. It's a little unorthodox but in this case I find it to be elegant.
For some of my jails that is how I run them too. Especially where I am being especially paranoid about security. I was trying to answer about 'general use/unexpected gotchas' scenario.
We deal with the hands we are given. I wish I could run it as tight as that on all of my boxes, but the bosses I deal with do not want to waste money on upgrading old infrastructure but still get mad when it isn't running. So fat jails are my best compromise. My other option was a virtualbox/vmware setup.
There are very very few things that cannot be told to bind to the real jail IP address instead of localhost. Most stuff will just work. And that's it. But there are also added benefits here:
* Parts of the filesystem cannot be altered, period. You update your base from your jailhost and every jail takes the change. That was an added security benefit.
* Pkg software can be individually installed per jail, goes into /usr/local. FreeBSD has already been brilliant in separating the base system from the third party software.
* For minor and major version bumps, we developed a simple set of steps and tools to upgrade a jailhost but leave the jails in their previous versions, maintaining multiple versions of base, like /var/jails/basejail104, /var/jails/basejail113, and so on. So the downtime is minimized, and each jail can be upgraded later in its own pace. This was a huge relief compared to having to upgrade everything only to find that some piece of software doesn't quite work correctly in the next version. And rollback is possible, by just copying the jail.
* Jails can likewise be very easily copied between jailhosts with a couple of rsync commands and some simple script.
Have used iocage and zfs and it is brilliant, but even without the benefits of zfs clone and snapshot, jails were (and I believe still are) a very pleasant experience to work with.
Why dont they do a freebsd container runtime based on the jail semantics? They sound much more effective than the linux ones. Ok, the problem would be on how to run linux executables on freebsd.
That problem is mostly solved. Has been for years except for marginal sysctl() calls unlikely in datacenter usage of Linux.
If you have 5+ years investment in docker, the missing, parts here are going to hurt.
Mainly, what's missing is helpful evidence of transition.
E.g. I have a docker image i got from git which works as a one-shot execution chain to run xml2rfc and I got a version for cramdown which worked mostly the same way to do rfc/drafts from markdown. Where's the BastilleBSD equivalent? How do I translate a Dockerfile to a Bastille script?
Indeed, the first thing I thought when seeing this post was "I wonder how long the support for each FreeBSD release is". Looking at Wikipedia (https://en.wikipedia.org/wiki/FreeBSD#Version_history), it seems to be around 5 years. Still no match for the 10 year support CentOS users expected; IMO, 5 years is too short when installing out of cycle (for instance, since the current stable 12.x was released two years ago, if I installed it today on a new computer I'd have only 3 years of support left).
If indeed 5 years suffices, Debian LTS may also be a candidate.