Hacker News Reader

Comment by jameshart | Hacker News Reader

* edit: I am an idiot who can’t recognize golang code (facepalm)

So it’s a Ruby* app, which means code you put in the container but never use doesn’t do you any harm - it doesn’t bloat a binary or take up space in memory.

but…

… whenever you make a change in one of your pub-sub handlers, you still have to redeploy your web workers.

… code and dependencies for your pub-sub handlers is sitting idle in your web workers, increasing your security risk and attack surface

Why not just turn this into a monorepo that publishes three artifacts? Having just one container image seems like a weird thing to optimize for

View on HN

I think that publishing 3 artifacts causes a number of other changes which expand the difficulty of management of the backend of the app. While they do allow, hopefully, independent scaling and seperate recovery processes. Separate artifacts also make it a bit more difficult to do local development, and deployments become more complex so it's not a completely clean tradeoff.

I know systems kubernetes / dockerswarm which handle the management of multiple artifacts but that is now another tool needed in order to do development.

So I guess what I'm saying is that "just" publishing multiple artifacts has more knock-on effects which need to be considered. Eventhough I do agree with you that it's probably for the best in the long term if these processes are in seperate containers.

In this model there are already three different command lines to run the thing in three different modes. Deployment complexity of moving that decision one notch left to buildtime instead of startup time seems like it doesn’t increase the accidental complexity at all.

In fact now you can eliminate variation in how your three different processes are started up and health checked and monitored, so instead of code that knows how to operate three different things, you have code that knows how to start and operate an arbitrary thing, and you parameterize it with your three different containers.

Oh hey - that’s what kubernetes is for.

Maybe all this ‘complexity’ of tooling isn’t dumb after all.

> … whenever you make a change in one of your pub-sub handlers, you still have to redeploy your web workers.

We have a similar model with Django. We deploy at least four times a day, so it's much easier to just redeploy everything than reason about what needs and what doesn't need to be redeployed.

It’s a Go app, not a Ruby one. Hence the Go code examples.

And the rationale behind one binary rather than several is to reduce build time and deployment artefact size, along with benefits in development from one binary. All of which mean deploying is much quicker/easier, which means it’s less of a big deal that we deploy everything whenever any part changes (we deploy anywhere up to 30 times a day).

Hope that helps it make more sense.

Apologies - read the ‘Ruby monolith’ up top and then glossed over the code without noticing the language mismatch. Mondays, am I right?

I don’t understand how bundling three sets of functionality into one binary reduces build time or artifact size.

Surely the binary containing all the web and pubsub and scheduled tasks is strictly bigger than the binary for just the web would be, and takes longer to build and test that the binary for just the web?

Maybe the three binaries have shared deps, by bundling them on one you get to not duplicate those shared deps?

I dunno as I have never used Go, but this would make sense with node

Also thinking how all the related tasks a la CI/CD might be total time faster with one process than three, since they're doing less total work, both during tasks and to wake_up the process etc dunno just speculation on my part

No problem, easy to skim by it!

In terms of how this reduces artifact size: Go statically compiles everything, and the majority of that weight is from dependencies. Compiling into three different binaries would:

1. Increase build time, because even while the dependencies are cached from each subsequent build, linking isn't free. Takes about 20s to link each binary, so it adds about 40s+ to the build to create them twice.

2. Increases build size, because each binary is about 150MB, so we go from 150 -> 450MB, which increases time to upload to container registry and download into the infrastructure that runs things.

In answer to the security surface, the code between cron/worker/web is very interlinked. There would be little practical reduction in surface area from splitting the binary, though perhaps for some applications there might.

You only have to rebuild the artifacts that are affected by each change.

And each change (assuming CI/CD) only causes the changes build artifacts to be pushed to prod.

So the amortized build/push cost depends on how often you are making changes that affect multiple artifacts versus fewer artifacts.

If an individual change only affects either the web binary or one of the cron job binaries then those build times will be shorter than they would be to build the big binary, and the amount of data pushed to prod will be smaller.

If an individual change affects multiple binaries then building all of them will take longer, and more data will need to be pushed (although - maybe it nets out the same in terms of fa out to multiple servers - your cron servers only need the cron binary, your web servers only need the web binary)

So, if the vast majority of the time you are only impacting a single component build, you save build time and deploy bytes by building smaller artifacts.

At the cost that making cross-system changes requires a slightly longer build time.

> reduce build time and deployment artefact size

I'm confused... doesn't this plan _maximize_ your build time and deployment size? You can't do worse than building and deploying every line of code you own on every build and deploy; it's the worst case scenario. Go is fast enough in compilation that it doesn't matter, right? But then what's the point?

Reply on news.ycombinator.com