How would you characterize the quality of CircleCI? I've spent quite a few hours trying to work out mysterious bugs in a build process that runs fine locally. This morning the web ui was full of 'client error' messages.
Circle however I've used with some fairly lengthy and complex build processes involving multiple parallel builds of Rails apps during upgrades and it's handled the workflow well.
I think they've improved the UI a bit recently, but it's still pretty complex though.
edit: I'll also add that in terms of time, I feel like most of the time in build pipelines is actually spent in tweaking, adding, and modifying the build configs, as opposed to the servers themselves (and you might also value developer time differently from devops time). I have found Buildkite easier to work with than CircleCI in this regard, and thus saved much more time overall.
Coupled with Terraform for us.
The motivation for this was exactly this problem. With Box CI your local build is your CI build because it runs jobs via agents on your own hardware so you can very easily debug / understand what is going on.
The thing I'm concerned about that doesn't seem addressed on your home page is security : by installing your agent on my machine, I'm basically giving you access to my machine non-stop, no ? What guarantees do I have that your agent can't be used as a backdoor ?
Firstly, and very importantly, the agent is open source (https://github.com/boxci/boxci) so you can see exactly what it does and even build it from source if you want. I should mention Box CI isn't launched yet, everything is still in beta including the agent.
The Box CI service doesn't have any way to send requests to or send commands to agents - the requests are one-way, agent to service only, with agents polling the service for build jobs to run, running them, and sending logs and results to the service.
Build scripts and configuration are all in your source code. This is a tradeoff, because it definitely means you have to trust your source (and everyone committing to it) if you're going to be running agents on your dev machine for instance.
There are obviously mitigations to this though, like running the agents on a dedicated Cloud VM with nothing on the filesystem and internet access locked down to package manager and cloud provider hosts. A future plan is to provide pre-configured VMs for various cloud providers so this can be set up in a few clicks for low cost ($20 a month or whatever a reasonable build machine VM would cost).
The real security point of the the agents though, the other side of these tradeoffs, is that your code and your production secrets never leave hardware you control - the CI service never has access to them. That's a massive security advantage Vs centralised CI services which do have full read access to your source and effectively full write access to your production infrastructure via keys you give them - compared to what I've described above this is a massively bigger and more dangerous potential security hole.
You can probably tell by the length of this comment I'm super interested in this aspect (along with dev experience I see it as the key advantage of the model and one of the reasons I'm building it) there's a lot more depth to this and if you'd like to discuss further, I'd love to :-) Email's in my profile.
It's quite spartan, but it gets the job done, and I love that a) it's fast, b) I can see exactly what happened, down to ssh'ing into the build directory and poking around, and c) If there's a feature I want, I can just add it!
Here's my bot in action: https://github.com/lukechampine/us/pull/37#issuecomment-5936...
When something fails it makes the debug and retry loop just as simple and fast as when you're developing, and there's really nothing like watching a production build run and ship in literally like a minute, just as fast as if you're building and starting your local dev server :-)