Cloud development environments tame complexity by reducing state
medium.com
medium.com
This is ignoring the removal of control from your developers. Autonomy of tool choice is an amazing boon to job satisfaction.
Which is not to say that a cloud environment should not be used. Just don't homogenize the setup.
> This is ignoring the removal of control from your developers. Autonomy of tool choice is an amazing boon to job satisfaction.
> Which is not to say that a cloud environment should not be used. Just don't homogenize the setup.
IMO, there is a good middle ground.
I agree with you on stuff like editor/ide and base OS. But I think there can be great advantages to having everyone do development in docker/VM such that the environment matches prod.
In particular, I think local deployment is important. And having freedom in all local deploys leads to finding the small details that matter.
Especially since you probably do not own the full prod tech stack, you are behooved by having proof that you can change it.
And, by all means, make the best attempt you can. I am arguing that somewhat purposely breaking it in a regular basis with each new developer makes you more resilient if you have to do a change in prod someday. And you will, someday.
There is a hidden issue with not standardizing. Most people either don't care about local vs container (matching prod) development OR are terrible not equipped to make an informed choice. Thus, as a devops, I have to support both methods. This is a huge drain on my resources, especially since devopsing is just part of what I need to do.
The root cause of this is the myth that all software engineers will understand source control, will understand databases, will understand how systems interact with each other and on top of it, will understand the business so they can actually do the work required. This is how you end up with "it works on my machine". This is how you end up with "but it works in repl, therefore your prod environment is broken".
Standardizing doesn't mean this is the only way of doing things. It means, this is the standard way of doing things and if you get stuck, you get support. It means this is the way we consider to be most efficient of doing this stuff. You are still free to do things your way if you chose to, but if you get stuck, it's on you to get unstuck.
And agreed that conforming to what others are doing will, by definition, get you more support. I just think there is hidden value in that diversity. It sucks, as it can't be forced/codified.
I will further agree that too many will chase the "long game" of "if we support diverse environments, we will have a more robust product," at the expense of short term health. And often that is a false dichotomy, in that often the best long term choice for a team is also the best short term tradeoff. Surviving is important.
Containerised workspaces can be version controlled, which gives you the flexibility to fork a copy mess around but still go back to a known good state. And the new branch can easily be cloned for others to use.
This is a public company and I have compliance requirements to meet or I am personally liable or face jail time. And with thousands/tens-of-thousands of engineers, your tool choice and lack of streamlined repeatable systems is costing us millions of dollars in profits every year and keeping your pay and the value of your equity down.
If you're going to depend on collectivist systems to provide you a stable livelihood, don't be surprised that the collective doesn't reward your individuality and entreprenureal spirit. There are other organizations that value that. Go work for them.
It sounds like you think that because you can't measure it, you are legally obligated to assert that it is zero.
You are bad at your job.
As a matter of fact, many listed companies reward an entrepreneurial spirit and value employee satisfaction, because that’s usually linked to sustainable growth.
I would also say that dev environments can be “streamlined” and “repeatable” while allowing for personal choice of editor and whatever other ancillary tools the developer is productive with. They’re not mutually exclusive.
My observation has been that teams that don't have tool choice almost always lost it because they had an overly specified build environment. Often under the guise of "streamlining" the setup. And this is why they have that particular build server still sitting around that only the head developer is allowed to modify.
In what should be a straw man argument, I've seen this go so far as folks not knowing how to even try to build their software on anything other than the sanctioned IDE with them having to use the mouse to click the "now build" button.
Which is all to say, my argument is intended to be that you do need a repeatable and streamlined build system that is tested and run in a variety of environments. The more diversity in where/how it is run, the more trust I have that the team actually understands how it works. Especially as a whole.
For the personal control, letting the devs accomplish onboarding in a way that they control empowers them to make the changes to the setup that help make that happen, and gives more autonomy to them. Giving them autonomy means they can fix things they see. And you want that, as they will see far far more of what is happening than board level management will.
Does this mean you can't/don't meet compliance? Of course not. Most compliance can be tested. And it should be tested. Again, this testing benefits from diversity of execution, as well, such that a test that can pass on a variety of environments and toolsets is far more trustworthy than one that can only pass on a particularly sanctioned setup.
Now, pulling this back from a straw man argument. I accept there are situations where this level of freedom is harder to achieve. If you take my argument as absolute requiring that the entire team supports every possible variation of tool that could be thrown at them, please walk it back some. That isn't my intent. I just get scared by efforts to unify it all down to one, as well.
Cloud dev environments are a tool like any other. How they're designed matters. A lot of home grown tooling ends up being limited for ease of build out and maintenance cost/effort.
Nix offers a solution to being able to do a "cloud development environment", but on your local computer. It also allows you to easily setup the cloud development environment if you need one. While Docker works for setting up a new instance in a cloud provider, on a mac is a major pain so I don't really see it as a proper solution to the things Nix does for me.
But I can't help shake the feeling that Nix and Docker are just the best we have for now. I think the real solution to this involves far more changes before it's not just a pile of hacks. Eventually things will get bad enough that something will appear and it'll be adopted.
I don't think a VM per developer is the final version of this idea.
A ways off, but some colleagues in the hardware world are working out the requirements on paper. Maybe right as I type this, not really tracking their calendar.
There will be some wiggle room for general programming, patches, hands on customization by the user.
Software is a decades long bubble to help us discover optimal machine state models we can etch into chips.
So long as we can avoid social collapse or species extinction between now than then anyway :)
Solve local development first and then maybe cloud dev. Reasons, just off the top of my head:
- always-on internet connection is not always viable
- intermittent and bad connections are a thing
- which local environment do you support for accessing your cloud? An ever changing combination of "latest only" browsers, apps, and select IDE integrations?
- how stable is your cloud environment? which features and versions of which tools will get deprecated tomorrow?
I think this is it. Progressive enhancement beats graceful degradation.
In my experience, that can be a way to deal with some resource requirement and licensing constraints, but can also be pretty horrible due to either bandwidth requirements or latency that adds up for many smaller queries that follow one another (e.g. when your app needs to shuffle around lots of data to load a page).
Therefore, I'm tempted to agree that you should be able to relatively easily run as much as possible locally: especially a database with automated schema migrations and either data import or seeding (generating believable test data).
Then again, one can see why something like that would be problematic in situations where you can have dozens of microservices, running all of which locally isn't entirely viable, yet re-architecting everything isn't either. It'd still be nice to be able to launch and test everything you need for a given scenario locally, even while using certain dependencies from a cloud dev environment.
You can't even test serverless functions properly because... you need the Amazon/Google server to run them on :)
This is unfortunate. I know that OpenFaaS and IronFunctions attempted to create a more vendor neutral offering that can run anywhere and still be usable, but neither really got the critical mass of attention/adoption for the large vendors to look in their direction.
I'd say it's rather unfortunate, when you compare it with the success of S3, which has many self-hostable (and locally launchable) compatible solutions like MinIO and Zenko.
Even managed databases are still often MySQL or PostgreSQL compatible, whereas FaaS doesn't seem to have quite gotten the sort of standardization.
We saw the cloud environment as an essential tool to provide the developers with, but not a tool that it was essential for developers to use. In fact, we had a preference for keeping dev working, so initial work on large changes should be done locally and deployed to dev once they're working. Dev was there to protect the testing environment and make sure that developers had a "real" cloud environment.
In other words, you don't have to deploy to dev, but if you break the testing environment because you didn't deploy to dev, there would be raised eyebrows.
No, Alex has made this bed, and now he's lying in it.
There comes a point where you have to spend some of that 'productivity' on things that keep you from being preempted in the middle of something else. Things like a stable development environment, self-cleaning scripts, etc. It's not necessary to make things bulletproof. It is necessary to make things self-service.
This has been immense not only for my own development (nothing on my local machine, everything can be destroyed and rebuilt in a few minutes) but also for onboarding people onto projects. No mucking about with IDE settings, they are free to use whatever of course but having the basics all there makes for a great system all around.
Probably ok for certain needs, but not something I would use. My productivity comes from a localised and personalised setup. Issues with the local or remote environments IMHO, indicate a problems that
1) others may also encounter.
2) should at least, if possible be understood/ documented.
3) represent opportunities for greater understanding of the environments in question (assuming you have time to dig).
Sort of. In certain stacks, you essentially set up remote debugging like you would for an app running in a remote environment (which is your local container with an exposed port) and your IDE just works.
It's relatively carefree when it works as expected, but a bit of a pain to set up sometimes. Admittedly, something like CPU flame graphs or tools like VisualVM that let you easily select from locally running Java processes to instrument might be harder to work with.
But even then, you can have issues with file system permissions and any bind mounts that you might need (e.g. files in a PHP container, where you want to keep developing and testing your app after page reloads, without rebuilding the entire container).
I wrote a bit more about it here: https://blog.kronis.dev/everything%20is%20broken/containers-...
I'm still a proponent of using containers for making applications more consistently managed (configuration, resource limits, port bindings, storage), self-contained (dependencies, running different/multiple versions in parallel) and easier to launch (e.g. Docker Compose file or a fancier variation of YAML instead of Ansible + systemd services), but they definitely can be a leaky abstraction if you don't have *nix as your development machine OS.
The essay is probably interesting, but I'll never know.
I opened it with an incognito window, no problem.
I opened it with a logged in window, no problem.
Some authors try to monetize their posts. Not this one.
I'd love to know if your experience was somehow different.
https://gitlab.com/magnolia1234/bypass-paywalls-chrome-clean
Someone has to bootstrap a DC of machines before you can run that containerized business logic.
All the old barnacles are there it’s just hidden behind the wizards curtain.