I understand this can be done. And this post explains how it's done. I still don't get why it should be done?
What's the advantage of running your dev environment in a container?
What's the advantage of running your dev environment in a container?
You could reconfigure your laptop to look exactly like the production target, but then you have to keep doing that every time you change projects.
Before Docker, I used VMs for this, but VMs have certain disadvantages that Docker addresses. Like size, and documentability. Every time someone wanted me to look at a project, we had to figure out a way to transfer and store a copy of a 20+ GB VM. And they couldn't tell me everything they did to create that VM from scratch, because VMware doesn't do that and neither does Hyper-V. With Docker it is just a small text file that describes everything it takes to create what was previously a massive, undocumented VM image. It forces you to document how to create the environment, and it saves on the space and time of transferring VMs around.
Until Docker came along, it was a royal PITA. I always dreaded getting a new laptop or something breaking, as it took forever to set everything back up again, and it was never quite the same.
Docker changed all that. It forces you to configure everything in a reproducible way in a Dockerfile - and it's much simpler than trying to come up with scripts to install and configure everything in Windows, and I'd say it's also quicker than trying to come up with scripts for a Linux VM, just because you can spin containers up and down so quickly.
Docker has been a game changer for dev/test.
If you mean why use dev environments in containers:
1) reproducibility,
2) re-use of container creation scripts for different environments,
3) isolation from your actual OS,
4) ability to run the same OS/libs/etc as the final deployment,
5) tons of base images with different environments already configured - from LAMP to data science,
6) easy sharing with others, team
7) ability to work with 1-2-5 or 100 different environments, with different OSes, versions, libs, python versions, whatever.
As for why have your IDE/editor work from inside a container (as described in the article).
Well, because you get all the benefits of containers (as above) PLUS get to use the editor as if you were programming directly on the target machine (including having visibility of installed libraries for autocomplete, running code directly there, and so on).
It's the same thing people have been doing with running Emacs in "server" mode inside another host, and programming with Emacs client on their machines as if they were locally at the machine.
For me, this is the major benefit. I don't have to worry about installing new tools or libraries and how they interact with my primary OS. While this isn't a huge issue for many, I don't want to have to worry about how Go, Python, Java, etc... are installed on my Mac. I like being able to pull in a Docker container with everything already setup (or a customized one). Then when I throw away a project, I don't have orphaned installations on my Mac.
Don't get me wrong, I am a fan of containers (in particular LXC), but I wouldn't list those benefits as if they are unique or novel to container based workflows.
Edit: To be clear, there _are_ benefits to containers over VMs, just not the things you listed above from my perspective.
Even the hurdle of SSH'ing to a VM is more cumbersome than `docker run`.
Certainly this could be automated and scripted, but the Docker solution is so... streamlined.
And I say this as someone that used to use Vagrant. With smaller installs available (such as Alpine), maybe more modern VMs would be just as easy as containers...
Also -- for me, VirtualBox was just "meh". It worked, but really wasn't that great. It always seemed like it took too many resources to run. That was another issue with Vagrant. (And yes, I did also use Vagrant with VMWare, but again -- that's a lot of overhead).
But I'd say VMs and containers solve different problems. In the case of dev environments, VMs are too "persistent" and accumulate personal cruft very quickly. Container tooling can be built to be noninteractive.
I also find the workflow of creating Dockerfiles to be much smoother than cobbling together scripts for a VM.
Plus Vagrant was a real PITA to get working on Windows (at least it used to be - I think I eventually gave up trying to get something running on Windows 7).
No, docker server runs directly on top of the OS as a native program.
As for Docker containers managed by the Docker server, they are runing on top of a supervisor - not in a full VM.
"With the latest version of Windows 10 (or 10 Server) and the beta of Docker for Windows, there's native Linux Container support on Windows. That means there's no Virtual Machine or Hyper-V involved (unless you want), so Linux Containers run on Windows itself using Windows 10's built in container support".
In any case, as Linux and macOS prove, there's no need for docker to have to run on a VM. And it seems there's no need on Windows either since 10.
I don’t know what happened with that but I was not wrong and Docker for Windows does still run in a VM: https://docs.docker.com/docker-for-windows/install/
I assume you don’t use Docker on Windows and just pasted the first google result?
Well, VMs are like overweight containers. Containers make "all of those benefits" easier, more performant, and more ligherweight.
Super easy for new team members to get started on a project. No need to manually install dependencies.
Environment versioning in git and docker. Your local environment gets automatically updated with a git pull.
I see this brought up a lot as an argument. So why do we want this? How often do people switch companies? Once every 3 years on average or something? Getting your development env setup takes what, a few hours max on 3 years?
Its not unusual for large organizations to have internally hosted registries (Artifactory), source control and network proxies. This usually requires setting up different config files (.npmrc for Node.js/NPM), installation of custom root certificates, ssh keys, etc. None of that includes project/team specific configurations and workflows.
Take all that and multiply it by thousands of developers and you have a recipe for an endless stream of Slack chats, email chains, and Teams messages repeating the same config questions and answers.
If you can reduce all that down to a single docker pull, while making sure everyone's development environment is consistent, it can be a big win.
* You want means to keep all of the dev environments in sync so you don't get "works on my machine" problems.
* If you update something, then you need a way for everyone to have their environment reconfigured.
* As the number of projects/stacks/developers scales this becomes a bigger and bigger issue.
I've used some Anisble in combination with a shell script wrapper to handle some of this kind of stuff. Even still, it takes a lot of hands on support to make it all work. So, if you can get something like this to scale, it might be a big win ... if...
(edit formatting.)
In docker world, I create an MR that updates the dev container and deploy container dockerfiles at the same time, check that it runs tests, and merge it in. I push a new version of the dev dockerfile, and have the .vscode/devcontainer.json reference that new tag. Next time all the devs open up this repo, they'll get notified they need an update. You just updated a dev dependency across the whole group in a source-controlled way.
What's your way to do it? Email everybody?
Updating a postgres version comes to mind as one of the possible differences and that usually only is an issue when working with pg_dump and pg_restore with different versions.
Good point nonetheless. I am not sure whether it is worth the work to maintain dev containers and the performance hit you get vs running a database directly for example.
Just yesterday, I ran into an issue where a set of node unit tests were failing. My college and I were both getting failures, but different failures. The reason was: Different versions of Chrome, and thus different versions of the chrome integration plugin.
Given that we have effectively no control over Chrome's auto-updates, we'll never have truly identical development environments. A container with headless chrome would have resolved this for us.
On larger teams, setting up a consistent environment like that also makes it easier for developers to collaborate. I've had experiences where attempts to pair program or share utility scripts generally stumbles and fails due to everyone's environment being a special snowflake.
[1] syncing the changes is left as an exercise to the reader, but I use a common git repository.
It's not about you. The people coming in generally need confirmation and help when setting up their environments. Someone there would have to take time out of their day to help you. A few hours, a few days, a few weeks, a few deleted companies (https://news.ycombinator.com/item?id=11496947).
Here's an even more fun one: https://news.ycombinator.com/item?id=14476421
Its easy to load a container with some stored test state. Its easy to load a completely fresh environment. Its easy to run multiple instances of things (with docker compose).
Its easy to totally shutdown the environment.
Its easy to work on different branches with different/conflicting dependencies and juggle containers.
- Clone project
- Build container
- Develop
If you're in a polyglot shop there are HUGE productivity gains in not needing to setup your environment manually, or worse, risk that vital information about it is distributed as tribal knowledge.
Plus, if your project has external dependencies like DBs, S3, etc... you can use docker-compose with VS Code as well.
Here's our current base go template, you only need Docker+VSCode on your system to get started: https://github.com/allaboutapps/go-starter
Bonus points:
* As all IDE operations solely run within the local Docker container, all developers can expect that their IDE will work the same without manual configuration steps.
* We can easily support local development in all three major OSes (MacOS, Windows, Linux) and even support developing directly in your Browser through GitHub Codespaces.
* Developing directly inside a Docker container guarantees that you use the very same toolset, which our CI will use to build these images. There are no more excuses why your code builds differently locally versus in our CI.
Edit: format/typos
I, personally, hope that JetBrains comes up with something similar which will allow devs to use the same workflow with the JetBrains IDEs.
Just launch the respective container and you are good to go.
If you are full stack or work with multiple programming languages there is no need to learn the "equivalent" of virtualenv everywhere else.
Also with docker there is no setup/installation involved. You just pull the image and that is it.
Also virtualenv requires that you already have pip/python installed. Docker requires nothing (apart from itself). So you can instantly launch Java/Node/Erlang/Haskell whatever without any SDK/libs installed.
There’s a “pets vs cattle” angle here, too. Something goes wrong, just pave over it and start again.
YMMV, of course, but I can’t imagine going back.
If I use 4 programming languages why learn 4 tools instead of one (Docker)?
With docker its a single command run after installing docker.
And let's be honest, virtualenvs and all the various ways they're managed and updated and such aren't really bulletproof either.
It's really, really refreshing how much "yeah I managed to break my dev environment" or "I followed the wiki for how to start developing your project but it 'didn't work'" can be avoided if it's just "run docker(-compose)?".
And this is especially true for junior developers, who are probably fresh out of college and won't be familiar with lots of the tooling that exists in the world. Not that docker is a simple tool, but it can hide so much complexity that it is easier to just show people how to docker run and docker build and such.
And if you're working with multiple projects in multiple languages, why bother learning each language's equivalent of virtualenv (assuming it has any), when there's an universal method available?
The TL;DR is there's a lot of things to set up yourself without Docker in order to run a typical web application and it's different depending on what OS / version you use. Some of these things are unrelated to Python too, such as running PostgreSQL, Redis, etc. but these are very important to your overall application.
Docker unifies that entire set up and the barrier of entry is installing Docker once and then learning a bit about it.
It saves so much time to just pull down the repo, run "docker-compose up" and have everything running, almost exactly the way it's running in production. With the right node or php version, databases, Elasticsearch, Redis etc.
Just deploying changes affecting both assets carefully in production can be quite awkward relative to a code-only deployment. You might need to do this in several stages, updating your code to allow for but not require the DB changes, then updating the DB schema, then maybe updating existing DB records, then at least one more round of code updates so everything is running on the new version of the DB. And you might need to make sure no-one else on your team is doing anything conflicting in between.
Doing the same in a staging environment isn't so bad because you're running essentially the same process. However, for a development environment where you want the shortest possible feedback loops for efficiency, you need to be able to spin up a DB with the correct schema, and possibly also with some preloaded data that may be wholly, partially or not at all related to controlled data you use to initialise parts of your database in production or staging environments.
It is not always an easy task to keep the code you use to access the database, the current schema in the database, and any pre-configured data to be installed in your database all in sync, and to ensure that your production, staging/testing/CI facilities, and local developer test environment are also synchronised where they should be.
It kind of amazes me that there doesn't yet seem to be a way of handling this in the web development community that has become a de facto standard in the way most of us look at tools like Docker or Git these days.
If you're an individual, then it would benefit someone who has multiple devices and/or multiple operating systems and doesn't want to manage their environment across all those devices and operating systems. For example, I personally have OSX, Windows, and several different Linux distributions on my laptop itself. My desktop also runs several operating systems.
Managing software across all of those is a pain. With Docker, I only have to manage the containers, and just have Docker installed on all the operating systems. Instead of managing like 50 different dependencies across 7 systems (think 7x50), I only have to manage Docker inside each system.
If you're working for a company, they will have their own dev environment. Instead of setting up and troubleshooting all their dependencies on your computer, you can just use their containers.
It's frustrating at best and catatrophic at worst when you have code working on your machine, deploy to prod, and then discover incompatibility.