We tried for multiple years to make Nomad work because it's simple. We're already enterprise customers of Hashicorp, too. We love Hashicorp! Nomad is a great scheduler and the UI is wonderful, but there is no story around network security. Example: You have postgres running on dedicated metal, and you want to restrict which nomad services have access to postgres. Consul Connect allows you to enforce access policies, but these work more like haproxy with automated service discovery. There is no batteries-included way to prevent traffic from leaving your container (or identify which container the traffic is originating from). You can use a custom CNI plugin to give each container an IP address from a unique subnet per service (and restrict traffic at your firewall/ selectively add subnets to iptabels on your postgres server), but now we're adding bespoke complexity. We brought this problem up to the Hashicorp internal teams over many calls, but ultimately they said we'd need to pay for a 3rd party overseas consultant to advise us on implementation. They weren't sure what to do. K8s is certainly more complex out the gate, but you don't feel like you're the first person in the world to implement.

That said, I think Nomad is a few years away from being something truly amazing. If they can solve the network security problem (beyond optimistic documentation), I think it'll be amazing. For now, it's just a good scheduler.