The Man Who Would Build a Computer the Size of the Entire Internet | Hacker News Reader

The Man Who Would Build a Computer the Size of the Entire Internet

wired.com

51 pointsjulien42128 comments

Hi everyone, I'm the guy featured in this article. Now that the dust is settling I thought I might answer a few of the most frequent questions.

> So what does Docker actually do?

Thanks for asking!

From the project homepage: Docker is an open-source project to easily create lightweight, portable, self-sufficient containers from any application. The same container that a developer builds and tests on a laptop can run at scale, in production, on VMs, bare metal, OpenStack clusters, public clouds and more.

There's a nice high-level overview on http://www.docker.io/learn_more/ , and a really cool interactive tutorial on http://www.docker.io/gettingstarted/

I also encourage you to join the IRC channel (#docker on freenode). It's extremely active and the people there are super nice. They will get you up to speed in no time!

> Why is the press talking about Docker when the kernel developers did all the real work?

Press coverage is not a zero-sum game. Does it hurt Linux and its developers when Docker gets attention? Or does it further establish Linux as the ubiquitous software infrastructure that it is?

There's no question that Docker wouldn't be possible (or would be much less useful anyway) without the work of countless kernel hackers who implemented and hardened namespacing and control groups. We also owe a lot to Junjiro Okajima, the author of AUFS. I don't believe we're doing any of these people a disservice by building cool stuff on top of their code, open-sourcing all of it, and getting a lot more people excited about the combined result.

> How do I buy an article like this on Wired?

That's not how press works. Based on my experience, here's how to get quality press coverage.

First build something people want. As more people use it and express their excitement, journalists will take notice. They will then contact you for details and if they like what they hear they will write a story.

> I was going to try Docker, but it's so hyped now that it doesn't seem legit anymore

Really? This reaction makes me so, so sad.

Speaking on behalf of everyone who has contributed to Docker in one way or the other. ALL WE ASK is that you evaluate the project based on its merits, instead of how fashionable it is.

Tell me you hit a bug. Tell me it doesn't solve a real problem for you. Tell me the documentation isn't clear, or the code is not clean. But for the love of god, don't tell me your criticism of a project is that it's too popular, and therefore no longer a trendy thing to use. Please tell me that is not what the hacker community has come to.

Can someone explain how a little wrapper around LXC gets major magazine articles and hype, whereas the people who've been doing the real hard work for years get no attention at all?

They really should offer an online course on "Over The Top Hype Generation with Advanced Hyperbole"

Really? "Build a Computer the Size of the Entire Internet" based off LXC containers they're creating an abstraction over?

It's like saying virtualenv can "Build a Computer the Size of the Entire Internet" because you can relocate an entire python program easily.

I love how rabid they are, but self-promotional attention is getting out of hand.

[SELF-CHECK: If you find this article annoying to read, is it because you're jealous of their attention? Are there things you've done that have mattered (or could have mattered) if only other people paid attention? How are you going to get attention next time? Yeah, getting attention this way can seem slimy, icky, unseemly, and feel like outright lies, but they'll be the millionaires and you won't. Gotta get slimy sometime.]

Have you ever been in the news? I'd bet that most or all of the crazy hyperbole was added by Wired. The Docker guys take a much humbler tone on their web site. Please don't criticize them for over-the-top claims that they had no hand in making.

Old quote: "All of the press hype actually only made convincing them [prospective employees to join] harder to do... there's some sort of weird anti-correlation between press hype and ease of convincing an A-level engineer that your technology is real."

Wired is the Cosmo of tech journalism.

makes a great mousepad.

I read this comment when this story was on the front page, and it's occurred to me ask, what would you consider the antithesis of Wired? You know, something you'd recommend someone read...

Reminded me of the "Linux Hackers Rebuild Internet From Silicon Valley Garage" [1] piece that went around about CoreOS, which was discussed on HN too [2]. In fact, they are both written by the same guy. Personally, these are pretty BOLD headlines, like "Rebuild Internet" and "Build a Computer the Size of the Entire Internet", which I do not think either of them claim and sound ridiculous to me. Docker is not going to build a computer the size of the entire internet, and CoreOS is not rebuilding the internet!

I like Docker, because it automates much of the LXC bits, and adds cool filesystem features. So, do not take my comments the wrong way, but I think articles like this do a disservice since they over hype!

[1] http://www.wired.com/wiredenterprise/2013/08/coreos-the-new-...

[2] https://news.ycombinator.com/item?id=6249724

And we got the Russian software that is taking over the internet.

http://www.wired.com/wiredenterprise/2013/09/nginx/

So the internet endgame seems to be One Huge Reinvented Internet in a huge computer running Nginx. It could be worse.

I'm not being facetious when I say this: that question is a bit like asking why Ruby on Rails got all the hype when Ruby had been in development since the 1990's.

AFAIK Docker is LXC + union filesystem layer management + a build system + the repository system + community, so it's not that little. But the real answer is marketing. DotCloud and CoreOS are companies that directly benefit from exposure, so they spend the effort to go out and get that exposure.

Because articles and magazines don't magically come and talk about you. You have to work to get them. The people behind LXC don't have or need a PR team. The people behind Docker, on the other hand, do.

Marketing. Docker demonstrates a clearer value proposition.

Here is the home page of LXC: http://lxc.sourceforge.net/

Here is the home page of Docker: https://www.docker.io/

Reading both should answer this question.

It's also sales; basically putting information in a way that's consumable not just to techies that are knee deep into work, but both non-techies and former techies who are now managers.

I don't think it's fair to marginalize the work that the Docker community has put in to it so far by saying another tool is doing all of the hard work. You could go down the stack and always say that about technology. That's what makes it so powerful.

I also would not categorize Docker as a 'little wrapper' around LXC. Sure, Docker uses LXC as a default container provider but it could have just as well have been raw cgroups/network namespaces, libvirt, openvz, or any other abstraction that suits the core abstractions of Docker.

> Hykes wants to create a standard means of moving software applications across the internet and the world’s private company networks, from machine to machine to machine.

Nobody's paying for advertisement of scp scripts.

How do startups cope with pressure to perform? The pressure must be immense on the dotCloud/Docker folks. They have been active on HN responding to questions and comments, but having such publicity, Wired and HN driving traffic and questions, on non-production/beta software (docker) must be hard. I guess you just keep going like you always do.

In several of my personal project, I have noticed that once you have a following, you try to keep the standards up and only release quality, things start to take longer, and longer, because you do not want to let anyone down. I have felt real pressure to perform, and my projects are not anywhere near their level!

I can only speak for myself, but I feel that if you aren't in a place where there's tremendous pressure to perform your focus should be on getting there as soon as possible. Part of that pressure comes from the vibrant and active user and business community surrounding Docker.

The amount of work ahead of us is minuscule in comparison to the amount of work we'll have ahead of us in a year, and so on. However, scaling appropriately to become an organization that can handle those demands is really, really hard.

Can someone explain, briefly, how Docker works? It seems to be getting an increasing amount of buzz lately and I'd like to understand what's going on. All I hear is how you can wrap software in a container that will run anywhere -- not a very helpful explanation. Docker's own website is surprisingly light on specifics... all I know after reading it is that it will, indeed, run almost anywhere! (but how?)

https://www.docker.io/gettingstarted/#

Try the tutorial, if it still doesn't make much sense to you, then it's probably not something you need to worry about.

Docker is a couple different things in one, so in that respect, it works in different ways.

The main thing Docker does is run arbitrary commands with (or like) LXC, which is basically a way of running a program inside a chroot environment, plus providing a more low-level separation for various parts of the kernel. The end result is your host is kept insulated from the program being run. Here's a great HOWTO on setting up LXC: http://lxc.teegra.net/ And here's a great StackOverflow answer on what Docker/LXC does: http://stackoverflow.com/questions/16047306/how-is-docker-io...

This page from Docker's blog discusses not only the security aspects of a container, but a comparison to traditional virtual machines: http://blog.docker.io/2013/08/containers-docker-how-secure-a...

And the best technical summary from Docker's documentation on what a container is:

"Once you start a process in Docker from an Image, Docker fetches the image and its Parent Image, and repeats the process until it reaches the Base Image. Then the Union File System adds a read-write layer on top. That read-write layer, plus the information about its Parent Image and some additional information like its unique id, networking configuration, and resource limits is called a container."

On top of that it adds a Registry, an Index, and a Remote API. So really, it has remote deployment features, it builds chroot environments, and it combines them and runs applications in a container. Oh, and here is the FAQ: http://docs.docker.io/en/latest/faq/ Summary of their points de-marketingized:

  Portable deployment across machines

You can copy a big image of your chroot environment, application, etc to a remote machine and run it. Docker makes the configuration configurable.

  Application-centric

They added features to make it easier to deploy a program and run it how you want it to run.

  Automatic build

Instead of using a shell script to build your container's chroot environment, you use a Dockerfile, which is like a less useful shell script.

  Versioning

Docker assigns a SHA hash to each image you deploy so you can roll back and do other things on the hash instead of the file name. It also transfers deltas of changes, apparently.

  Component re-use

Due to using a unionfs, you can mount multiple images on top of one another, so you can have smaller separate images and pick the ones you want to use for your app.

  Sharing

You can upload your images to a public repository or download them for your own use.

  Tool ecosystem

Docker has an API. (Actually three APIs)

I don't get it. I don't see how Docker makes service management any easier than using a plain ol' VM. The steps involved feel about the same to me.

See: Daedalus, Icarus and Helios.

[1] http://en.wikipedia.org/wiki/Deus_Ex_characters#Artificial_i...

way to GO.

seems someone didn't get it.

We get it, but it's just not that funny. Also, I think I would consider Go truly successful when people stop feeling the need to point out every place that it's used.

well, is not that successful yet. that's why it's nice to see a big project using it.

Reply on news.ycombinator.com