More philosophically, are containers a necessary component needed to run software at scale (such as replication and partitioning)? Or are they the result of the accidental complexity the modern OS has?
VMs on one hand, tools like Guix and Nix on the other.
> More philosophically, are containers a necessary component needed to run software at scale (such as replication and partitioning)?
No. They're one tool in the box among others. They're common for reasons, though.
> Or are they the result of the accidental complexity the modern OS has?
What does this even mean?
The namespacing and layered filesystem work in the OS that make containers possible itself arose through other motivations within kernel scope to actually reduce accidental complexity and make various workloads, like suspend and resume, more sane.
Now that containers exist and have been successful, related but alternate tech like unikernels have seen interest, to reduce the security surface and further optimize deployments of large fleets.