> as a small developer
Do you need kubernetes?
I know the hype cycle is mad for copying big tech, but if stackoverflow can operate on a couple of IIS instances I’d argue that you almost never need kubernetes.
Do you need kubernetes?
I know the hype cycle is mad for copying big tech, but if stackoverflow can operate on a couple of IIS instances I’d argue that you almost never need kubernetes.
Ten years ago you started with PHP + memcache + MySQL running on just a physical box running linux. No Docker or Kubernetes. No virtual machine for what it is worth. Then you split it in multiple PHP frontends with a load balancer or MySQL master and slaves as the traffic demanded it.
I think a lot of systems these days are over-engineered and over-paid from day one. It may cost you 10-20 times what you would if you keep the "old day" approach.
That said Docker and Kubernetes are nice. They give you a lot of flexibility. But the most important part, I think, has been that they consolidated the shift from "pet" to "cattle" servers that simplifies a lot the sysOp and sysAdm work.
I’m not attacking docker, however, I can see why you would want containers. We still haven’t found an efficient usage for them at my place, but we never need to spin up more than one instance of our software. I think docker can sometimes be a way to cheat unsafe software around operations, but that’s more of an anti-pattern than an issue with docker.
Then again, I’m probably old and grumpy, but that sometimes has the advantage of not adopting techs before they are easy to use.
From my perspective k8s is really good for a company who has a team of experienced SRE who can manage k8s well and the company has a marketing team driving new development that needs to get to market fast
It is not cheaper than simple autoscale groups, it is not easier than rebuilding packages with jenkins ( rpm spec files or debian src rebuilds )
It is however the current popular framework so if you want to ride the wave learn it. Also learn how to migrate away from it as that will be a future role as several early adopters are pulling back out of it.
The larger issue is the simple fact automating cloud infrastructure is not trivial so abstract layers lets more people implement things without understanding the lower levels giving folks the appearance of having a complete framework.
It works great until it doesn't. Then it is interesting watching people try to figure out how to fix it.
Learn the lower levels of Linux and k8s will come to you organically
> the company has a marketing team driving new development that needs to get to market fast
At what point in the growth curve? Are your SRE shipping the new features, or running like a red queen to keep the product developers able to keep shipping?
What development process or cycle has overheated with friction that requires k8s? Where is that friction which k8s relieves?
Successful utilization of k8s, interesting metric, what is your criteria ?
>At what point in the growth curve?
I have seen properly staffed startups (in silicon valley) leverage the platform to pivot direction fast but overall I would say you need a large enterprise to support it. Ironically the large enterprise which could benefit the most wants to lay ITIL on top of k8s, call it agile and the methodologies conflict creating more issues than they had.
> Are your SRE shipping the new features, or running like a red queen to keep the product developers able to keep shipping?
Yes to both
>What development process or cycle has overheated with friction that requires k8s?
Upgrading k8s :)
>Where is that friction which k8s relieves?
What k8s relieves is once you get a pipeline getting product to production fast, the approval process from legacy ITIL groups being the problem ( I have yet to see an enterprise company use k8s properly though I have been to presentations where some claim to have done this )
If someone attempts to use ITIL and k8s you are going to have problems. Training people to NOT do this is the #1 issue in my experience.
> Successful utilization of k8s, interesting metric, what is your criteria ?
Sorry, what pain motivated using k8s, and using k8s relieved that pain.
> I have seen properly staffed startups (in silicon valley)
What is 'proper' for staffing?
>> Are your SRE shipping the new features, or running like a red queen to keep the product developers able to keep shipping? > Yes to both
Is it the appropriate use of a Reliability engineer's skills to develop or change arbitrary features? Or are you saying they are at least Engineers so they should be able to pitch in everywhere... CSS accessibility features or k8s config.
>> What development process or cycle has overheated with friction that requires k8s? > Upgrading k8s :)
This tautology keeps me away from k8s. Before k8s what pain in the process required using k8s to solve?
> a pipeline getting product to production fast
Does k8s make it fast? Does k8s make it possible? I think I'm deploying "fast", without k8s... But because you keep mentioning ITIL I suppose it's more about the infrastructure changes? Or are you talking about ITIL tension because k8s requires a more liberal policy than ITIL allows?
There is so much assumed context when talking about k8s, the only comprehensible part of these discussions is k8s is a rabbit hole to end all rabbit holes.
CTO / CIO / $SOME_C reads a blog and decides they want k8s is how it usually is introduced
> What is 'proper' for staffing?
Experienced C Developers who can do operations ( like google level SRE )
Summary of k8s : if you use the proper methodology to implement k8s and you are in a cloud framework you likely dont need k8s :)
What shines about k8s is it leverages the current container fad to ship code faster because devs like it. The current container fad pushes the burden of supporting buggy code to operations teams. IF the operations teams have issues they had better be highly skilled to figure them out
I have yet to see a company let requirements drive the choice to use k8s.
For cloud services this is probably a good idea, even for users to a degree if the provider doesn't already give you a fitting box.
But otherwise it is not a must have in my opinion. Maybe that is a mistake and the apps I develop today are not going to work in 10 years. Well, worst case: I have to be paid again.
You can get 99% of the way using a stable distribution and a configuration management system (ansible, chef and the like). It's much much simpler than running an orchestration service. I feel most people don't need containers and orchestration, just config management running redundant system designs.
True, but you can do that with plain system containers such as with lxd, rather than having that bundled with the huge paradigm shift that Docker comes with.
The paradigm of lxd is pretty much exactly the same as the paradigm of a regular distribution installed on bare metal or inside a VM. If you can operate a regularly installed distribution, then you can operate inside a lxd container. The commands to create and destroy lxd containers are trivial ("lxc launch ubuntu:bionic" for example).
> Kernel minutiae is not and the technical skill requirements is much higher.
I'm not sure why you think you need to know kernel minutiae, cgroups or kernel namespaces. Operating lxd needs none of that.
> I am pretty familiar with my application and the distro of the container in a user perspective.
That's all you need.
Dockerfiles give you a fairly easy and consistent way to express, "The runtime environment needs to have Python 3.5 and these packages," in a format that doesn't introduce too many concepts over and above the basic command line junk you'd use to manage your environment without Docker. If your stack requires multiple services, docker-compose gives you another fairly easy way to describe what all goes into that. And then it gives you a _super_ easy interface for starting and stopping all those services, keeping track of what you have running, all of that.
(And it's all fairly disposable, which is nice, since, as devs, we tend to break things. TBH, if Docker has done nothing else for me, it's that it's turned nuking PostgreSQL to get back to a clean install a 10 second process instead of a 30 minute one.)
It's not really that simple, and I've spent my fair share of time screaming at Docker for being flaky and having confusing under-documented configuration. (And I don't think I'd use it at all if I were working on a platform that weren't so annoyingly susceptible to systemic dependency hell. But worse is better, so the unix philosophy won, so here we are.) But eventually you get over that hump, and it starts feeling fairly easy to understand.
I don't know Chef, but I've seen Ansible used in production, and it just doesn't seem nearly so attractive. It could just be how it's being used, but it felt like there was this infinite regress of complexity where everything was tied to something else and you have to have been the person who built it to understand it, kind of like the bad old days when people were trying to put too much smarts into the database itself so they'd just become this rat's nest of triggers and whatnot. I'm sure it's not that bad. . . but my initial impression was that Docker is great for scratching a developer's itches, but slightly sucks for ops, but maybe is still worthwhile there if you're dealing with microservices or elastic scaling or something like that and you can use Kubernetes to smooth over some of the flakier bits. Ansible is much more for ops, and does a great job there, but I don't see it scratching many dev itches at all.
It starts with a Dockerfile, which is a limited shell script, and it does not get any better beyond that. Shell scripts are simple, I'll give you that, but please don't sell them as some magic bullet. Dockerfiles are no configuration management system.
I've seen my fair share of hairy chef, puppet and ansible in the wild. Don't read into config management from those. I've also seen beautiful ansible installs, which deploy from dev setups all the way up to full infrastructure setup and deployment with blue-green deployment.
Y'know, we might violently agree. That is a much more concise statement than my rambling attempt to explain why I think Docker is so much more palatable for development workflows.
You're right, it is no magic bullet. And I misspoke when I said "configuration management"; I forgot that that's a term of art in operations. By "management" I really just meant "stick it all in one or two files so I can get my checklist down to one step, and manage shared packages in a way that's at least a little bit less kludgey than simply abusing environment variables." So I find that it save some yak shaving, and for that I can deal with it under certain circumstances.
I actually hate using it for deployment or production config management, because IMO it seems to do a crap job at it. And it does a crap job at it precisely because of the features (or lack of features) that make it convenient for development. Even using it to manage our integration tests' runtime dependencies is kind of a hot mess. But I'm willing to concede that, together with Kubernetes, it might be nice for cloud-native elastic scaling microservice-y stuff, insofar as it seems to be popular for that. I don't actually know firsthand; I'm allergic to complexity, so try to avoid building things that way.
This! I come from the embedded world with a bit of webui and having yocto for the embedded reproducibility and docker for the infrastructure, I am so happy. No more uncertainty when moving to another machine or upgrading my DEV machine. Nope Everything running right everywhere. Some scriptology and I had a full boot up from tftp for kernel and an ephemeral nfs from an ext4 master, and I could reliably make full system component tests all the way to web browser experience validation.I even had an autopilot simulator spawning ephemerally in a docker. Restarting the containers gave me a clean env again. Pure piece of mind and productivity. The initial investment was big though.
Regarding the investment it is an embedded industry problem. There is very little re-use. My experience is that we are an industry of wheel reinvention, where all of us are deep experts so we roll everything on our own. Maybe against myself I speak as I indeed developed this harnesses even though other solutions exist. My quip with what exists is that Intel absolutely owns, or used to own yocto and embedded tooling open source projects. This lead to very crappy code that was Intel specific being half merged into upstream. Mind you their featues are really not working,and were accepted because Intel is a gold yocto project sponsor. When you try to remove the broken code to a more sane one, you cannot because then the actual good rules of making small changes will bite you back and your changes will not be accepted. So you need to wait for Intel to cave in to remove their broken features. I digress sorry :). Even so Yocto project is a great step towards embedded projects productivity and knowledge re-use. https://www.reddit.com/r/embeddedlinux/comments/bk8a8k/yocto...
Builds use the dind images on gitlab's runners to build an image and push to their container registry.
For deployments I have a host with a personal CI runner instance on Linode's smallest instance type which can access a user on the "production" host when SSHing over a private network, and has the docker-compose command allowed in the sudoers file. Then it can run docker-compose up to deploy. The key for this is passed to the job via gitlab's secrets UI so someone getting read access to either of my hosts wouldn't be able to do anything.
While people will rightly point out that this does mean the CI builder effectively has root on the "prod" host, for a side project it's enough for me. I might investigate podman/buildah some weekend when I have time as apparently that allows for rootless container launches.
Compose and Swarm can take you pretty far, but TBH, it felt like Docker gave up on them years ago, even before k8s "won" the container orchestration war. A real shame :(