Comment by jacobwg | Hacker News Reader

Comment by jacobwg | Hacker News Reader

Parent Full thread

So we have the concept of a "project", which in retrospect isn't the best name and is way too vague. :) But on our end, a "project" equates to one or two cache SSDs + one or two EC2 instances we're running, depending on whether you've asked for a single platform build or an Intel+Arm multi-platform build.

We do charge $0.05 per minute of build time used, but in theory that $49/mo plan gives you access to up to 20 build machines, if you're building 10 projects at once.

That said, if you already have your own dedicated build cluster / CI setup, you may prefer to just use that! Depot is effectively doing that kind of thing for you if you don't already have your own hosted CI system or would prefer not to orchestrate Docker layer cache.

We will be expanding to more things like SBOMs, container signing, insights and analytics about what's happening inside the builds, but hopefully in more integrated ways, since we control the execution environment itself.

As someone who is not a user yet 49 sounds like a rounding error in our dev budget and if all I have to do is change a line in my CI script that does sound very tempting. Standing up another machine and registering it as runner sounds like way more work. Also incurs a recurring task to run updates there or cycle the VM to get a new base image. An hour of dev time is really very expensive

I see. So it's $50/month plus .05 per minute of build time after 60 minutes is up in that month. Let's say you need 10 hours of building per week, so that'd be another $120/month on top. I'm not sure how fast these builds would go with your setup, maybe 10 hours a week is a lot. But we're still talking like $170/month for something with user limits and fairly restricted resources. For less than that I can get a 16 core AMD Rome machine with nearly 8 TB of flash split across two drives, which should eat image builds for breakfast. The extra cost is a bit of Linux sys admin which can be fully automated (apt-get install unattended-upgrades and a little more on first install).

Clearly from other responses in this thread there are people who feel this is a good deal, so best of luck to you. But I'm kinda reminded here of 37signals saying they can save $7M over 5 years by leaving the cloud. It seems the goal here is to dig people out of performance problems they get by using one type of cloud service, by selling them another type of cloud service!

>But we're still talking like $170/month

Rounding error for most companies.

>The extra cost is a bit of Linux sys admin which can be fully automated

You are overweighting hard dollar costs and underweighting the value of engineering time. Maybe you're the worlds greatest devops/platform engineer/sysadmin and once you wire up everything in under 5 minutes it will never need maintenance ever again but for most everyone else speeding up image builds by using a service that someone else thinks about and does maintenance on is absolutely worth it for $170/mo.

Yes, maybe. I do know Linux pretty well and don't consider sysadmin costs a big drain on my own company or time. I can see that it'd be much more expensive if you hire people who don't have much UNIX experience.

On the other hand, I've experienced first hand how cloud costs can explode uncontrollably in absurd ways. One company I worked at had a cloud cost crisis and they weren't even serving online services, just shovelling money into Azure for generic dev services like VMs for load tests, DBs for testing, super-slow CI agents, etc. They never managed to properly fix this because of the mentality you express here: a few hundred bucks a month here, a few hundred there, everyone gets access to spin up resources and it's all worth it because we're all soooo valuable. Then one day you realize you're inexplicably burning millions on subscription services and cloud spend, yet nobody can identify quite why or on what, or how to push costs down. Death by a thousand cuts, it was quite the revelation. Free cloud credits are murder, because they embed a culture of profligacy and "my time is too valuable to optimize this". By the time the startup credits run out it's too late.

I think the problem is that once you get in that mindset, you start forgetting that you need to optimize/fix your application.

Having network issues due to slow async calls? Just increase instance size until the machine is so fast it completes everything before it becomes a problem. Now you are paying 10x more for something that’s a few hours of dev time.

I think you're giving the worst-case scenario when citing large-scale cloud spend. A couple of hundred dollars a month for very fast builds is a good deal, and they can build on Intel and ARM, which is useful.

Worst case scenario, I mean, maybe? I don't have many data points. It does feel like cloud costs come up more often lately. I don't think there was anything particularly special about that company though. It felt like almost the default outcome of using modern development practices and giving everyone who "needed" it access to the Azure console.

I have accidentally spent far too much on Azure logging before, so I am a firm believer in giving as much (read-only) access as possible!

You should probably include the cost of your own salary in your calculations

I'm running a bootstrapped startup so a special case, my salary is nearly nothing :) :( :)

At a bigger company you'd just ask a junior to set it up or maybe a sysadmin. You can probably contract to get part timers too. I guess I spent half a day or so setting up the CI cluster at the start and have barely touched it since. That wasn't much cost even if I was earning a big salary. Our builds can use caches to speed them up (not docker, other types of cache) and when we turned that on it was like a 3x speed win, so having persistent disks is definitely worth it for many types of program. Especially if you're brave and trust your build system to cache unit test results between builds!

Reply on news.ycombinator.com