Hacker News Reader

Comment by why5s | Hacker News Reader

For work-related stuff: I disagree. We're not even at Uber's scale at my company and I hate having to manage the ever-changing set of dependencies that I have no control over.

View on HN

The thing is, it doesn’t have to be ever-changing, and creating a reproducible working setup locally can be achieved by appropriate setup scripts.

Easy in theory, but always breaks down for a non-trivial project with >5 developers on it.

Setup scripts that mutate an operating system are fragile. Break something? Unless you understand how the scripts work (which will become stupidly complex at a scale like Uber’s) you will have to reinstall your machine. Or you will have to staff a ton of support to help users when they do break their configuration.

Spinning up a VM with an image containing all the development tools is a much smoother experience most of the time. The only reason why I don’t use it where I work is because I use vim and network adds too much latency for me.

That's a false dichotomy. Just because you don't have a VM doesn't mean that the alternative is a build environment and setup that mutates an operating system.

Using a VM is fine. One can use it locally.

In practice setup scripts are brittle. New person joins the team and their script fails because it turns out everyone else’s dev environment was only working due to something left behind by some older script. Hotfix requires checking out an old branch but now I need to run an old script but my setup is from the future - the old script doesn’t know what needs to be un-done to get the system to a consistent state. And what about data? My old data has gone, the data I have is from the future. Never mind simple stuff like the script author assuming node is in /usr/bin but that’s not true for me because I use nvm.

I'm sorry, but this is the task of proper configuration management. Yes, don't depend on local stuff that isn't configuration-managed. Don't have a workflow where you check out an old branch in a newer environment. Of course you need a way to establish the older environment in that case. I'm assuming that Devpod does a similar thing on the server side. My point is, the ability to reproduce a working setup doesn't imply a requirement of having to work remotely.

> I'm assuming that Devpod does a similar thing on the server side

That's why they use devpods. You moved entire configuration to the cloud. There is absolutely no reason why it should be run in your local environment.

We’ve had good luck using nix for this. Same dependencies for the local dev environment as for the built containers. Same config. All deterministic. And not just major deps like programming languages, but sed, bash, grep, and all the other shell tools, so no more worrying about people running scripts on mac’s ancient version of bash.

One of the reasons we developed devpods was because our setup scripts were unmanageable.

Instructions to new employees would say things like "run this thing, scroll up past dozens of pages of stdout noise and manually deal w/ the errors buried therein by looking up relevant FAQs in some doc somewhere"

The scripts would touch every technology imaginable, from brew to npm to arc (phabricator's cli) to proprietary tools and no single person understands how setup scripts work in their entirety.

One exercise we'd get new employees to run through was get them to brainstorm about how some system ought to work. The lesson was that just about any idea they could come up with would have already been tried (and failed).

I'm told that devpods aren't even the first time we tried cloud dev envs. Presumably lots of lessons were learned from previous attempts at improving dev envs.

Long before devpods (circa 2014) we had boxer, which was a command line tool that abstracted away the nitty gritty details of setting up a dev environment by orchestrating calls to AWS, vagrant, puppet, rsync, etc for you. However this was also a time when there were only a handful of services you would need to run, and because remote editing tools at the time weren’t as nice as they are now, it only worked really well if your preferred editor was vim/emacs so you didn’t have to deal with the (occasionally buggy) remote file syncing.

The devpod flow is a lot smoother. I had my laptop replaced recently and was up it running again in an amount of time that felt like cheating.

> run this thing, scroll up past dozens of pages of stdout noise and manually deal w/ the errors buried therein by looking up relevant FAQs in some doc somewhere

Okay, but if you can define/script your environment enough to run in a pod, couldn't you just run that locally? You already have to solve the manual steps either way...

Re: if a computer can run scripts in a pod, can’t you just run them locally?

It’s a construct of the order of operations.

One core issue with local is the variety of OS’ and local build tools that would fundamentally mess with the centralized scripts. Getting company-wide setup scripts to work on top of existing laptop config was a continuous challenge. Hence, having a consistent baseline (OS flavor, system-level packages) on top of which the company-wide “setup script” is added followed by “developer-customizations” seems to work great.

Central teams can manage the first couple of steps and individual user-specific configuration can be managed much better in a decentralized manner.

It's the same convo as docker: can't one just use setup scripts for reproducible envs? In theory yes, but in practice, theory and practice aren't the same thing.

It's easier to apply best practices to a greenfield project written in a modern language (hence devpods) than trying to comb through a decade+ of tech debt written in bash.

> it doesn’t have to be ever-changing

It IS ever-changing. One library among the zillions of local dependencies that you need to build something changes, and you have to go through the dependency hell.

If entire software world valued backwards compatibility and vigilantly guarded it, that wouldnt be a problem. But in the package hell that we are living in today, every other day a package update brings some incompatibility or breaking change for this or that other thing.

But now you're saying "we shouldn't change dependencies". Pypy turns out to work nicer than CPython? now you have a rollout project to your team. Swapping out some libraries? Another rollout project.

Now, it's great if you can avoid complex setups in general cuz complex is harder no matter what! But if you're starting from a complex setup, having easy ways to roll out changes is an important step in actually doing the simplification work to get to where you want to be!

Reply on news.ycombinator.com