Hacker News Reader

Comment by tedmiston | Hacker News Reader

Do you really find k8s useful for a single node deployment or were you just making an example?

I haven't used but I have used DC/OS, Mesos, and Marathon extensively, which is not a setup I'd do for a small number of nodes personally.

View on HN

Yes. Though I should emphasize that I'm using GKE. So it's zero config, zero ops. I am running a single n1-standard-2 node on GKE for a smallish app right now. Comes to 30-40 dollars a month all in (egress traffic, cloud storage, other services). I am working hard on getting people to use this app, and it's great to know that scaling will be best-in-class if I succeed.

But I stress it's not about the scaling. It's about the features even at a single node. I wouldn't be spending time writing this stuff had it not been revelatory for me.

PS. If you're not able to try it on GKE, there's also https://github.com/kubernetes/kops. GKE is great for trying, at least, though. Just get a cluster running and get rolling with the concepts. Then you'll know what it's all about. There isn't that much to it, and what there is to it is great and well documented with a great community.

Yet with a Docker Swarm setup, you could get away with probably 10 dollars/month. K8s likely taking more resources than your app containers is why k8s may not be the best option for single node deployments.

I've personally tried both orchestration options and much prefer Swarm to smaller deployments. Sure k8s could prepare you better for the future if your app takes off to the moon but that's just premature optimization.

This might all very well be true, but Swarm doesn’t let you connect cloud disks directly to your containers, to my knowledge.

Also I challenge the $10 because what cloud instances plus ancillary services add up to $10? I’m talking 2 CPUs and 7.5gb RAM.

A Digital Ocean droplet @ 1 GB runs at $10/mo which I have Docker Swarm + my app containers running on.

It does. Check out Docker volume plugins like: https://docs.docker.com/docker-for-aws/persistent-data-volum... https://github.com/thecodeteam/rexray

I've found that both Swarm and k8s - but _especially_ the latter - makes the day-to-day application deployment really smooth compared to almost anything I used before (git-deploy, fabric, rsyncing things around). Docker containers are a large part of that, of course, but the tooling and configuration/secret handling help as well.

As a discloser, I work at Mesosphere who engineers DC/OS.

I would like to note that DC/OS now has Kubernetes [Beta]. You can manage Kubernetes resources on your DC/OS cluster now.

Here are the instructions on Github to get started: https://github.com/mesosphere/dcos-kubernetes-quickstart You can also join the Slack to help you on you out: http://chat.dcos.io/ (the #kubernetes channel)

>I have used DC/OS, Mesos, and Marathon extensively

Would you use DC/OS-marathon (vs k8s) now if you were to make that decision now.

I've heard that mesos stack is good for machine learning/big data stacks but how does marathon compare for deploying webapps.

I've run 2 Mesos stacks in production and have experience setting up a k8s stack (on prem). First off in my experience k8s ops is way more complex that the DC/OS stack. I recently setup a new DC/OS deployment (80% of the cluster resources was Spark, which works natively with Mesos and I'd rather run the ancillary services on Marathon, then spend another 80% of my time on k8s). If I didn't have the Spark requirement I would have went k8s.

Despite going with mesos I really had to contend with the fact that k8s just has way, way more developer support - there are so many rich applications in the k8s sphere. Meanwhile I can probably name all the well supported Mesos frameworks offhand. Next, marathon "feels" dead. They recently killed their UI interface as I imagine that they are having trouble giving resources to marathon. 3 years ago I wanted a reverse proxy solution that integrated with mesos as well as non-mesos services so I hacked Caddy to make that work [1]. 3 years later, I was looking for a similar solution and found traefik. It claimed to work with mesos/marathon, but the marathon integration was broken and the mesos integration required polling even though mesos had an events API, so I hacked traefik to make that work [2]. On the other side of the fence, you have companies like Buoyant who rewrote a core piece of their tech (Linkerd) just to support K8s (and only K8s). This has a compounding effect, where over the years things will just become more reliant on assuming you are running k8s.

That "cost" you pay to setup Mesos/k8s is usually a one time cost on the order of a month. I feel however, that k8s is going to give you a better ROI (Unless, you are managing 100s of nodes with Spark/YARN/HDFS, then Mesos continues to be the clear winner).

[1] https://github.com/mholt/caddy/pull/40 [1] https://github.com/containous/traefik/pull/2617

From me, an eternal thank-you for upgrading Caddy's proxy middleware!

I think I need to try k8s in prod to give a good answer.

The DC/OS stack is nice for web apps, APIs, scheduled tasks, etc once you get it up and running. If I were just deploying small APIs and web apps, I would use something more managed like Lambda or Heroku/similar myself depending on the use case. That said, I'm more of an app dev than an ops person.

Reply on news.ycombinator.com