Launching Today: CircleCI 2.0 Reaches General Availability | Hacker News Reader

Launching Today: CircleCI 2.0 Reaches General Availability

circleci.com

98 pointsssemmaprise72 comments

Do you recommend using the same Docker image that passes tests on CircleCI in production?

I have an open source Rails app <https://github.com/coreinfrastructure/best-practices-badge> that self-certifies that open source projects are following best practices. It currently uses CircleCI 1.0 and then deploys to Heroku <https://circleci.com/gh/coreinfrastructure/best-practices-ba....

If I configure it for development and test, it will contain all the test and development gems in addition to the production ones <https://github.com/coreinfrastructure/best-practices-badge/b.... I would also need to include gcc and other build tools. But I would have immutability in that my dev, test and production images would be identical.

Or, should I use one image for dev and test, and then build a different image for production, in order to have the smallest possible size? CircleCI supports building <https://circleci.com/docs/2.0/building-docker-images/> further Docker images, but then there's a chance of losing dev/prod parity if, for example, a software dependency gets updated in the meantime.

I'm not sure if Circle supports it yet but you should definitely look into docker multistage builds as that should solve this problem for you.

https://docs.docker.com/engine/userguide/eng-image/multistag...

I've been experimenting with multi-stage builds, and they do let me build a leaner image with clearer syntax by using one the first image to build my gems, and then copying the gems to the final image. <https://github.com/coreinfrastructure/best-practices-badge/b...

However, that doesn't fix the problem of having lots of development and test gems installed in test that I don't want for production. I don't know of a way of copying just my production gems to a production image.

Well, you could load only the gems required for the environment: http://bundler.io/groups.html

Or maybe have more than one path for gems and change load paths between environments?

Or keep tests in a separate image from the production image under test? Similarly, build tools could live in a different image/layer.

It's not a crazy idea to first install production only gems, then copy them (cache them), and then install the remaining gems. But I'm asking for info on best practices.

You can bundle into vendor folders, one for each env and one for the rest.

Do you have a URL demonstrating this, please?

Here’s a node example that demonstrates the same concept.

https://codefresh.io/blog/node_docker_multistage/

No URL right this second (maybe I should write a blog post–), but I would set GEM_HOME or BUNDLE_PATH env variables as I ran bundler to have it install in a different folder, and add the folders as necessary to GEM_PATH at runtime. I would also consider using a local cache folders for the source of the gems bundler uses. This is the purpose of http://bundler.io/v1.6/bundle_package.html which assumes that gems are installed separately rather than loaded directly by path because of platform differences (packaging on Mac to run on Linux, for example). But you don’t need the overhead of bundler’s packaging and install process if your binaries are built in the same environment they run in, as could be the case with Docker or VMs.

You could also point BUNDLE_PATH at the last place bundler ran for that environment (don’t delete the old gems) and that would speed up the install, but a better approach might be to package to a shared folder and then install to env specific bundle path folders in your docker image/build workspace, if you want bundler to manage the shared caching while keeping gem output separate and unique to each build.

It’s important to remember that gems are just folders of ruby files placed in the load path, sometimes with compiled bits to load at runtime, that could in turn be dynamically compiled to rely on system libraries or non-Ruby packages. So you have similar constraints with Ruby that you would have with Node.js – the potential for issues from the portability or lack thereof for native code dependencies. That’s why there’s a distinction between “install” and “package” – install will make sure the binary output can run on the current system unless flagged not to.

You can optionally use a newer version of Docker CE that supports multistage builds.

Typically it's a good idea to separate build and testing phases strictly, exactly for these types of reasons. You deploy the same image to dev, testing and production, and it becomes easy to reason about an artifact without confusing it with the environment it runs in.

I hope this helps.

Right, but the point of my question is that deploying the same image to test, staging and production means that I'm necessarily using a less lean image, since I'm including gems only needed in development and test.

Why do you have gems only needed in those envs?

I'm trying to balance two conflicting goals in Docker: identical images in different stages and the leanest possible production images.

My question is whether development and testing gems should be included in production or not.

I think you might be missing an environment: staging. I would install whatever you want for development. Then once you're happy, push that to CircleCI and have it build the image. Push that image to staging for final testing. Once you're happy with that, push it to production.

I currently push all passing builds on master to master.bestpractices.coreinfrastructure.org, and also run staging and production environments. See https://github.com/coreinfrastructure/best-practices-badge/b...

My question is when switching to a Docker workflow, does it make sense to use the same image for test on CircleCI as for deployment on staging and production.

The "on-the-knee" setup i have been using for my (Django) side project is:

- base image that installs OS Updates + OS package requirements + production dependencies

- dev image inherits from base image, installs dev dependencies + source code volume mounted on '/app/src'

- test image inherits from base image, installs test dependencies + source code volume mounted on '/app/src' (image used both for local tests and CI)

- prod image inherits from base image, adds source code to '/app/src' (this image will eventually be tagged and pushed to a production registry after builds)

Only adding source code to the image in prod image for not breaking cache during development.

That makes sense, thanks. Wim not clear how to break up the gems for different environmenta into different directories.

If we're honest we have to admit that any difference between dev/test (where you're running tests) and prod (what you're deploying) can introduce subtle bugs either due to configuration errors or something like a missing dependency that you accidentally put in the development group but it was actually needed during production. I've seen this happen and to a large extent it's unavoidable unless you're actually able to run your integration tests with production environment settings (I don't know anyone who really does this.)

On one hand, having dev/test gems in the image should be innocuous, if I recall correctly bundler will not activate them when the environment is production. So other than some additional image size (is this really a problem?) it doesn't have any material effect. But I empathize with wanting to have a completely minimalist prod image, regardless of whether it has measurable benefit.

iron.io has for quite some time advocated having two parallel base images, one dev which has the build tools, and then copying the built production gems to a prod image. [1] If you do this, then the build tools (gcc and the like) are never in the docker image which is used for test or prod. But yes you still need test gems in an image that runs rspec/capybara/etc.

Maybe the real solution then is to take your prod image, and deploy it to some staging environment which then gets treated to a functional test suite. The approach would necessitate running the suite in one container, targeting the application running in another container however. (This is naturally how you would do it if you were using webdriver/selenium-based testing tools.)

[1] https://github.com/iron-io/dockers/tree/master/ruby

Ideally, yes you would want a single Docker image. That's somewhat the point of Docker, to isolate your app and have it run the same, everywhere. If you use multiple images, then you're circumventing that benefit.

You could install all the dev gems, build tools, etc in the CircleCI build container and run unit testing. Once everything looks good and you can build your file app that passes, then take the app and throw it in a new Docker image. Then, integration test that Docker image itself.

That final piece could be done on some sort of staging server of in a new build environment using CircleCI 2.0 Workflows.

Use Circle daily, signed up for the beta 2.0 release which has been fine in production for the last few months.

When I've needed support (mainly around installing custom versions of various apps) the staff have been very responsive.

Circle folk: for me it still displays 'Leave 2.0 beta' and 'Join 2.0 beta' - do I need to anything to get 2.0 final?

That label on your dashboard refers to a UI change, which, now that you bring it up, does sound super confusing. You don't need to do anything to go from beta 2.0 to GA 2.0 - if you're using 2.0 config in your yml file, you're in.

Thanks for pointing that out, let me see if we can make that less head-scratchy.

Ha maybe I haven't been using 2.0 then - I thought Circle 2.0 meant Circle 2.0, UI and backend.

More on going from 1.0 to 2.0 here: https://circleci.com/docs/2.0/migrating-from-1-2/

You should also see a little icon in your dashboard that says 1.0 or 2.0 to the right of each of your builds - that will tell you what version you're on :)

We pushed a change this morning to make it slightly less head-scratchy :)

Oh no, we tried switching over to it for a test project and it was basically a disaster. Something as simple as running Python tests and node tests on the same instance just didn't seem to be possible. We _really_ don't want to have to maintain our own Docker images. Their support forums are running Discourse too, so it's basically impossible to find anything you're looking for.

I always found Discourse sites to be quite easy to navigate. Do you suggest anything better?

Unfortunately not. I spent many years on Proboards / VBulletin and their ilk, I don't feel like any of them really nailed it.

The problem with any community forum support is that without representatives from the company constantly on there, a question might:

- go unanswered

- get answered by the community with "here is how you do it," without the "here is why" part.

- the question can't be answered by the community and all you see are messages along the lines of "same issue here."

Rarely do I see the "here is how you do it and here is why" answer I am looking for.

Those are all valid, but none of those are inherent with Discourse, so not sure why the guy mentioned brought it up.

we were kind of in the same boat, but realized you can still use the "machine VM" setup from before and just install the required dependencies.

Not great, but works.

You can also start with a base Docker image and then install any extra dependencies. Should be faster than the machine VM.

I don't know how possible this actually is. I was having some issues with this, personally

Like isn't the point of docker to not be able to install a bunch of stuff after the image is built?

For shipping an image yes, but that's not what you'll be doing.

For example, if you've used CircleCI 1.0 before, those builds run in LXC which is Docker's underlying technology. My suggestion is more down that path rather than 2.0's bring your own Dockerfile approach.

The base image you choose just means you don't need to install that specific language/toolset manually, which saves part of the build time and work writing the config.

You don't have to maintain your own Docker image if you don't want to, though that's probably the best route for the fastest, most efficient build.

You can choose a base image (or machine, a VM), and then go from there. No custom Docker image.

Also, what's wrong with Discourse? It's not perfect but surely the best forum system around I think.

Aside from CircleCI Discuss, we also have actual Support and Docs.

I wrote a binary for interacting with CircleCI from the command line that you might like. Notably you can type "circle wait" and get friendly output showing how long your test steps took to run and which (if any) containers of a multi-container test failed.

We use it a ton and you might like it too.

https://github.com/Shyp/go-circle

Re: https://circleci.com/docs/2.0/migrating-from-1-2/

Rather than have every customer repeat these steps to transform their file, why don't you do it for them, and make a circle.yml to .circleci/config.yml converter?

I'm currently trying to work out what

    dependencies:
      pre:

Turns into for 2.0. There's instructions for

    dependencies:
      override:

But I'm not sure if the same logic applies to pre, and the replacement shown there is indented in such a way that it looks like there's some missing information about what the parent container should be. All I can do is guess what the replacement for

    dependencies:
      pre:

should be.

Thanks for the feedback, we have a tool in the works to translate the config. Sorry it's not available right now.

For the 'dependencies: pre' step, you would replace that with a 'run' step, the same as outlined for 'dependencies: override'. We'll update the docs - thanks for pointing it out.

Is there something above 'steps:'? The indentation looks like there's a missing parent key - eg, it starts with a bunch of spaces at the top level - but it's hard to work out what it is.

Edit: looking at another doc, https://circleci.com/docs/2.0/configuration-reference/#deplo..., the missing parent key is 'jobs:'

Also shouldn't "Search and Replace Deprecated 2.0 Keys" be "Search and Replace Deprecated 1.0 Keys" (or better "Search and Replace Deprecated Keys")?

Also

https://circleci.com/docs/2.0/configuration-reference/#table...

Implies 'steps' is directly under 'jobs', whereas:

https://circleci.com/docs/2.0/configuration-reference/#full-...

shows that 'steps' is under 'jobs > build'

Some feedback:

- Re:

    * missing executor type
    *  is not supported executor type

Include https://circleci.com/docs/2.0/executor-types in the error.

- This should be one error, not two.

- Why is 'working_directory' mandatory now? It wasn't before. Why not provide a default?

- 'working_directory: ~' fails. But 'working_directory: ~/' works (but then fails later). This is silly. Do path normalisation.

- https://circleci.com/docs/2.0/executor-types should include a node 8 image already. it's the current version of node and '@latest' won't be great when 9.x comes out.

- Re: https://discuss.circleci.com/t/directory-tmp-you-are-trying-.... The error is missing a word. Also checking out to a directory with existing contents should be fine.

Edit: finally got it:

https://gist.github.com/mikemaccana/b6a7be0351d769a9099f5fdb...

`~` is YAML for null - http://yaml.org/type/null.html

Oh god.

Wow. I've always thought I understood `$HOME` was generally considered safer/more portable, but I've never actually had a problem when (interactively) using `~`.

Always use `$HOME`!

Thanks for the feedback, the nesting of keys in the configuration-reference doc is fixed.

Basically, there is no longer any concept of `pre` or `post`. The reason being, they were `pre` CircleCI inferred commands and `post` CircleCI inferred commands. `override` meant instead of inferred commands.

In CircleCI 2.0, there is no longer inferred commands. You just run the commands you want.

After years of using CircleCI 1.0 (loved that they let me install custom packages) for pre-Docker apps, I found 2.0 required a ton of scripting to test and deploy. Moved back over to Codeship - love the workflow. Setup used our docker-compose.yml, and testing/deploy config using a simple yml file worked great. Love the command line tool that lets us encrypt ENV files, as well as run and deubg the build locally.

Is there a way to cache build results for follow up builds? So the next build only needs to rebuild the files that changed?

This may be what you are looking for: https://circleci.com/docs/2.0/artifacts/

Yes: https://circleci.com/docs/2.0/caching/

CircleCI is great!

I use it almost everywhere.

Unfortunately here in Berlin mostly everyone switches to self-hosted version of Jenkins at one point.

What do you think happens at that point to make them switch toJenkins?

(In case it's not super obvious, I work at CircleCI)

I've heard some reasons like :

- Privacy issues - Owning critical service component

Unfortunately I've never heard the reason that I think is the right one ( at least from what I can see is happening here ), which sounds like :

"Hey, we don't need this, because at Zalando / Nokia Maps, etc. we used Jenkins, so I don't wanna spend time maintaining a new tool ( CircleCI )."

Same stuff happens with project management ( dominated by JIRA around Berlin ).

At one point a company trashed all my CD-setup ( which was with the free plan anyway ) to run Jenkins with Docker ( I don't know how well supported docker is with CircleCI, but I doubt that is so much different than Jenkins ).

EDIT: I'm freelancing in Berlin, so I usually do a hand-over when the team of devs is hired.

Yeah, CircleCI really needs to allow self-hosting. Jenkins is basically the only player in the space and Jenkins is not good.

We do -- but let me know if for some reason this isn't what you're thinking/wouldn't meet your needs: https://circleci.com/enterprise/

> $35/user/month

That's ~$40k/year for 100 devs, plus the infrastructure costs. A Jenkins license is $0 (or $100k+ for enterprise support). If it was my budget, I'd have a hard time fronting up a large fraction of a developer's salary to a third party to provide a fuzzy level of support, versus spending zero money and putting the support burden on an in-house team I can trust and directly influence.

TeamCity has great support, I trust them from JetBrains's great work with IDEs, and their top-shelf license is ~$22k (one-off with 1-year of support & upgrade; iirc the yearly support fee is discounted 50% of original license).

If you have 100 devs, you need at least 1 FTE supporting Jenkins and that's $60-120k/year alone.

Nah, just have everyone spend 1% of their time on CI.

We have more devs than that, and we spend way less than 1 FTE (or equivalent) per year on keeping it in good working order. Our biggest issue is when we want to do something that's not well-supported or hit bugs.

That's impressive. We run into not well supported features and bugs frequently. It's probably a side effect of our workloads.

Right now we're evaluating GitLab CI and it looks like the way forward (for us).

Don’t get me wrong, we are owned daily by Jenkins bugs. We just push as much as possible out of job config and into shared workflows & Makefiles, and treat Jenkins as a glorified cron with a web UI.

Like falsedan said, this doesn't cut it if you don't need an enterprise contract. My team for example: We're a 4-man self-funded startup and paying $140/month just for a license is super expensive, given that we can use Jenkins for free.

I appreciate that there's the option, but I'm thinking more along the lines of what Sentry offers: Open source, self-hostable for free, optional paid enterprise support, optional paid hosting for teams of all sizes. In my book that's the perfect lineup.

Travis doesn't offer self-hosting, Jenkins doesn't offer (non-enteprise) paid hosting.

Try Shippable's Startup edition @ $499/year. Though Jenkins software is free, you still pay for master and slave nodes. With Shippable Server SE, you can spin up nodes dynamically when builds are triggered, which will work out cheaper since you probably utilize your slave nodes less than 50% of the time.

https://aws.amazon.com/marketplace/pp/B072K2F5KF

To be fair, there's also Concourse, TeamCity and GoCD which all enable self hosting for free.

And Drone CI which is the most similar to Circle CI

Thanks for the reply. Yeah, of course there is always the "devil you know" argument :)

CircleCI 2.0 was built with first-class Docker support, so if that's something you're interested in, definitely take a look.

Half-OT: How do you do CI with iOS applications? Is there some good cloud service? Or do you simply set up a local macOS server with Fastlane, for example, and be done with it?

iOS/macOS also supported on CircleCI. Unfortunately not on 2.0 yet, but coming soon! https://circleci.com/mobile/

We use buddybuild. It can release to iTC. https://www.buddybuild.com

Is this fundamentally a more Enterprise version from 1.0?

We're looking for something with security, control, support etc that we didn't see when we looked at 1.0.

Like what specifically? There's tons more flexibility which would help enterprise companies. Separately, there's also CircleCI Enterprise.

Check out http://docs.shippable.com/getting-started/byon-overview/

All the security and control of an on-premises solution with all the benefits of SaaS.

Reply on news.ycombinator.com