CoreOS Introduces Clair: Open Source Vulnerability Analysis for Your Containers | Hacker News Reader

CoreOS Introduces Clair: Open Source Vulnerability Analysis for Your Containers

coreos.com

142 pointsxfiler26 comments

Does this use the OVAL data published by various companies? (eg. Red Hat's here: https://www.redhat.com/security/data/oval/ )

Yes it does:

> For now, Clair uses three databases, one for each supported operating system: Ubuntu CVE Tracker, Debian Security Bug Tracker, Red Hat Security Data

https://github.com/coreos/clair/blob/master/README.md#detect...

Hey CoreOS, quick question for my own curiosity. You have released/are releasing tons of cool stuff, I'm just wondering what you plans are to maintain them? How much to you expect the community to update them? Is your team growing as fast as you release software?

Thanks for all your great work!

Great question. Our projects are lead by a set of maintainers that may change overtime. We love to have folks from the wider community help there too and have had good success with that. Two examples of long term involvement: Ben Darnell from CockroachDB helping maintain the raft library or Simone Gotti helping build out rkt. So, community is one way to sustain and we welcome new folks to join.

We also continue to grow the team at a good pace for the stuff we are building too. We have a diverse set of challenging technology we are building to make everyone successful with distributed systems and containers. If you want to learn more see https://coreos.com/careers

It's exciting to me that virtualization and containers raise some new security issues (cotenancy, memory dump, etc.), but also allow new security capabilities (reading memory from outside the protected system, separation kernels, etc.). It would be foolish to ignore them from a security perspective, on top of all the other management/usability improvements they bring.

Cool project! Detecting installed packages and checking against known vulnerabilities is definitely useful. A similar feature is on the todo list for DockerSlim[1], so it'll be nice to reuse relevant bits and pieces.

[1]DockerSlim was a recent Docker Hack Day project. The goal is to make Docker containers lean and mean :) It analyzes existing images and then creates much smaller images throwing away stuff you don't need.

That's really cool. It sounds similar to Squashed Image support on Quay.io[0]. Is it open source?

[0]: https://blog.quay.io/New-quay-features/

Yes, it is open sourced. I think it's different though. It looks like the "squashed image" feature flattens an image. DockerSlim tries to extract only the artifacts the application needs, so it can turn a 431 MB image into a 14 MB image (just an example, not always possible with every application type). DockerSlim relies on runtime analysis to shrink images that much.

Neat.

Reminds me a little of this: https://github.com/mafintosh/torrent-docker

Is this a pivot?

On a related note, what do you guys think the future holds for CoreOS now that Kelsey Hightower has quit and moved to $GOOG?

I'm not sure why you think this is a pivot.

CoreOS, the company's name, is a bit of misnomer now. Their original product, CoreOS, is their namesake and a good foundation for the rest of their platform.

As a company, they've been putting out a number of products, such as etcd, fleet, flannel, and more. It seems that Clair is another product under their umbrella.

So it's not a pivot, it's just another product as a part of their larger offering.

CoreOS's products are all open-source and openly developed, which is awesome. They make their money from their managed service offerings, such as Quay.io container registry (which Clair seems to bolster), CoreOS managed linux, and Tectonic (Kubernetes as a service).

So each product announcement you see is another facet of CoreOS's larger offering.

Quay.io has been outstanding for us so far. Much better overall than Docker Hub for our organization. Particularly in ergonomics and team/permissions granularity.

100% agree. I just tried Docker Hub for a separate open-source project, and I'll probably just move it back to Quay.io instead.

Please don't use Quay for official open source images if you care about international users, or at least offer a Docker Hub option as well. Quay is super slow compared to Docker Hub. When I contacted support back in July, they were very polite and professional, but in the end "everything is being served from AWS's US East region". Peak time performance is intolerable. It was so bad that our systemd units were timing out even with a massive 5min TimeoutStartSec.

To worsen the issue, Quay still doesn't seem to support parallel layer downloads, and Docker 1.9 even complains that "this image was pulled from a legacy registry. Important: This registry version will not be supported in future versions of docker."

I just ran a quick test (way off peak time) and Quay was 2.5x slower than Docker Hub for an image built from the same Dockerfile.

I'm looking forward to more usable international service at some point, but right now it just isn't really worth it.

It makes little difference. He's still doing exactly what he did at CoreOS: going to conferences and meetups giving Kubernetes demos.

Kelsey was a leader in product at CoreOS. He did a lot of what you say, yes, but there was a lot more he did out of the spotlight. He spent the most time talking to users and could have been considered their best advocate, for example.

I enjoyed arguing with him because he and I were cut of the same cloth and have largely the same goals for the operations industry. He brought some great perspective and experience to product, so I'd suggest against shoehorning him as a pitchman. I was kind of bummed out to read that people think the demos are all he did, given how much respect I have for him and the work I've observed.

Perhaps I was too blunt. I didn't intend a negative tone. I'm a big fan of Kelsey and very appreciative of his work. My comment was about what practical effect I think him being employed by Google vs CoreOS will have, which I think is very little. His recent change was just to better align him with the work he had already been doing.

I've heard him talk. He's a futurist and a technologist. He needs to be out pumping the future of operations...for all of us. :)

CoreOS was CoreOS before I got there.

I came to CoreOS because of the open source nature of the company and the awesome collection of projects such as etcd, CoreOS Linux, and their work around containers. The release of Clair proves this will continue without me.

While much of my work at CoreOS was seen through the eyes of people attending conferences and readers of my blog entires, my true contribution was helping grow a community around containers, and more importantly exposing the benefits of distributed computing to a much larger audience.

Based on feedback, I consider my community efforts largely successful for many reasons including the following:

* I've been willing to help anyone learn this stuff by doing 1:1 hangouts, engaging in social media, or writing a book on Kubernetes. Whatever it takes.

* Building and sharing open source projects like confd[1], and my collection of prototypes such as motorboat[2], or exploring new integrations between tools like Docker Compose and Kubernetes[3].

I also spent a fair amount of time hacking on open source projects at CoreOS including rkt, etcd, and working with the community to ensure they could adopt our technologies, then incorporating feedback from users when we (CoreOS) made it hard too. As a result I gained Product Management responsibilities to go along with my advocacy work.

During my time at CoreOS Kubernetes came out, which represented a turning point for what I consider the majority of the industry (well the part that cares about application containers). Kubernetes represented many of the ideas we had been working towards at CoreOS, and in many ways was the perfect addition to the CoreOS stack.

I was an early code contributor of Kubernetes, which resulted in gaining commit access. But once the number of outside contributors grew, I felt I would have a larger impact building tools to fill in the gaps between Kubernetes and people adopting the platform. One of those tools being kube-register[4], which made it easy for people to automate the scaling out of a Kubernetes cluster using fleet.

Overtime I started to shift focus towards education, because what use is a platform for next generation infrastructure if no one knew how to use it? So came the workshops, book, and more conference talks.

In many ways I'm doing the exact same things -- working to improve the future of computing based on containers and core concepts of distributed computing, but at a different company, Google, which has many of the same core values regarding open source, community, and vision.

What happens to CoreOS?

I expect CoreOS to keep shipping stuff based on the same core values that attracted me there in the first place, and regardless of who provides my paycheck I'll always be a member of the CoreOS family. This is the power of community. I never had to leave.

[1] https://github.com/kelseyhightower/confd

[2] https://github.com/kelseyhightower/motorboat

[3] https://github.com/kelseyhightower/compose2kube

[4] https://github.com/kelseyhightower/kube-register

The evolution in tasks a Developer Advocate takes on are still very much misunderstood by many people who think of Advocates/Evangelists/Relations as primarily "pitch & demo" people.

Thanks Kelsey for distilling your perspective on your time at CoreOS -> Google here.

My impression is that he was spending so much time working on a non-coreos product (kubernetes) that it won't impact coreos much at all.

This is pretty interesting. It would be great if there was a step-by-step guide to check my own Docker containers. Instead of looking like a piece to promote Quay.

You could use contrib/analyze-local-images - see discussion at https://github.com/coreos/clair/issues/5

Unfortunately using dpkg and yum to detect installed software means this isn't a secure audit. Anyone can add known bad binaries after the fact and pretend to provide a "secure" base image. I'd be skeptical of trusting anything quay.io says is secure based on this scan.

It's intended to identify known security vulnerabilities, not identify malicious actors. Someone could add known bad binaries after the fact, but they could also add unknown bad binaries.

We have been doing vulnerability and malware scanning of images at FlawCheck.com for quite some time. If you are interested in scanning everything inside the image, would be happy to demo it.

Reply on news.ycombinator.com