HNHacker News
TopNewBestAskShowJobs

BenTheElder

242 karma · joined November 21, 2014

https://elder.dev
submissionscomments
BenTheElder··on Canonical introduces high-availability Micro-Kubernetes
[I work on kind amongst other things...]

If you want to turn your host into a node you can do this with --vm-driver=none in minikube or better yet just use `kubeadm init` directly. In KIND we point people to the latter -- the main thing we're doing is running inside a disposable container "node" of which you can have many.

Assuming you don't actually want to turn your host machine into a node managed by Kubernetes, you'll want to stick Kubernetes in a VM or container. If the rest of Kubernetes is in this container or VM, it doesn't make sense to be running containers out on the host, things like mounting volumes won't work, you need a consistent filesystem between kubelet and the container runtime.

With kind it's also important that we simulate multi-node and multi-cluster, which is not viable with a single container runtime instance.

Without actually running Kubernetes against the hosts's runtime you can't share storage. The way docker desktop does this is to run Kubernetes with docker as the node's container runtime while only supporting a single node/cluster in a VM and expose the same runtime for building.

For our test workloads it's important to have different clean clusters constantly for different tests / projects.

KIND and microk8s have made a bet on containerd, as kubernetes is actively moving away from dockershim towards CRI, so even if we exposed a node runtime you can't build with it.

It's indeed space-inefficient, but it's a tradeoff in isolation between projects etc. For multi-node you're going to wind up with multiple copies anyhow, and a lot of projects we work with wind up needing some multi-node testing.

BenTheElder··on Open-Source Virtual Background
The article links to [1] and comments on this near the beginning -- as others have mentioned this is not always the case. The Linux client does not offer this functionality, only green screen.

I'm also using this with other apps for fun though (duo, hangouts, etc.)

[1]: https://support.zoom.us/hc/en-us/articles/210707503-Virtual-...

BenTheElder··on Open-Source Virtual Background
Amusingly the current bottleneck is actually reading from the webcam with the suboptimal ~default capture config. Without doing anything else that's ≤ 10fps. Low hanging fruit still :-)
BenTheElder··on Open-Source Virtual Background
BodyPix does downsample before masking OOTB, the article is doing 'medium' (50%) (though for this script we ought to move that over to the python side), it's still not 30fps though without egregiously sacrificing quality, at least on my (fairly powerful) machine unless I've missed something.

Amusingly I did some hacking on this and the current bottleneck is actually reading from the webcam which is capped at <10fps without doing anything else. Switching the capture to MJPG helps.

BenTheElder··on Open-Source Virtual Background
I haven't tried yet but there's some work related to this going on in OBS https://github.com/obsproject/rfcs/pull/15#issuecomment-6054...
BenTheElder··on Open-Source Virtual Background
I need to profile it more closely, I actually don't remember the exact FPS etc. but I don't expect that to be the limiting factor.

The inference / ml is expensive (which I did profile initially...), and I suspect not really optimized on this backend. It appears to be faster with webGL in the browser.

I sorta stopped worrying about it once it was "good enough" to show up to a few meetings with, but with all the attention I'll probably take another look.

It does look like someone ported bodypix to python, I'll probably try that next.

https://github.com/ajaichemmanam/simple_bodypix_python

BenTheElder··on Open-Source Virtual Background
I'm in a 1 bedroom apartment with my fiance who is also work from home now, there isn't really space to put a physical green screen behind me.

If you have the space, and good lighting it's a much simpler approach.

It's also less fun though, and I could do this with what I had on hand pretty quickly.

BenTheElder··on Open-Source Virtual Background
Docker made it easier to package up the dependencies (especially aligning cuda etc.), and containers are my dayjob :-)

The web requests are just an easy mode of IPC to pass around some bags of bytes, "high frame rate" is at most 30 qps ... that part isn't really interesting performance wise and this isn't a production tool :-)

I'm not sure I'd be so confident about tensorflow.js being so fast on the CPU ... you can see a marked difference in the backends httpss://www.tensorflow.org/js/guide/platform_environment

BenTheElder··on Open-Source Virtual Background
Thanks for sharing -- With very similar settings too! (Not far from the defaults unsurprisingly).

Injecting this into a web client seems like the sweet spot effort wise.

BenTheElder··on Open-Source Virtual Background
The demo at the end of the page is a video (webm), but there's not a ton of motion to reference besides the blinking.

IIRC it's something like 10FPS currently which is sufficient enough for meetings so far (about 1/3 what you might get with sufficient bandwidth in most video conference tools).

There's definitely room to improve it.

BenTheElder··on Open-Source Virtual Background
This

I did actually tinker with this and other approaches a little.

Also the things in the background get moved around pretty frequently, e.g. what is on the couch. I don't have a dedicated office room

BenTheElder··on Open-Source Virtual Background
You could actually pipe in a video clip with just ffmpeg and v4l2loopback pretty easily without writing any code I think :-)

The virtualvideo readme has an example of looping a single frame. https://github.com/Flashs/virtualvideo#errorhandling

BenTheElder··on Open-Source Virtual Background
:-) Great minds think alike!

I'm iterating on it again tonight, and tentatively virtualvideo [1][2] gives better results by just piping frames into ffmpeg vs pyfakewebcam

[1]: https://github.com/Flashs/virtualvideo/ [2]: https://pypi.org/project/virtualvideo/

I'll have to update the post or do a follow-up at some point.

BenTheElder··on Kind - run local Kubernetes clusters using Docker
loses all the data because you have to start a new cluster? The data should be persisted...

If this refers to https://github.com/kubernetes-sigs/kind/issues/148, the good news is that we're most of the way there and I'm going back to work on this now, ideally out in a v0.8 in the next week or so.

BenTheElder··on Kind - run local Kubernetes clusters using Docker
unfortunately not.

figuring this out would make a lot of people happy, but it doesn't rank highly for our current use cases versus other work.

https://github.com/kubernetes-sigs/kind/issues/763

BenTheElder··on Kind - run local Kubernetes clusters using Docker
Appreciate the feedback :-)

I think the "best" depends on what you're doing to be honest, (e.g. if you only develop on ubuntu, check out microk8s too! they have some good ideas, eg focusing on straight-forward support for a local registry instead of side-loading) and there's a _lot_ of room to improve kind, but the vote of confidence is still very nice to see :-)

BenTheElder··on Kind - run local Kubernetes clusters using Docker
author here, some background:

kind was originally built for developing kubernetes itself, as a cheaper option for testing changes to the core components.

it wasn't really meant to compete with minikube et. al, but complement for differing usage, but you may now find it useful as a lightweight option with a slightly different feature set.

it's also the only local cluster that is fully conformant as far as I know, because conformance tests involve verifying multi-node behavior, at the time minikube did not support - building kubernetes from a checkout and running it - docker based nodes - multiple nodes per cluster

These days they've gotten more similar, we're both shipping docker and podman based nodes.

I think one of the most interesting things about kind is that the entire kubernetes distro is packed into a single "node" docker image, it's very easy to work with fully offline.

BenTheElder··on Kind - run local Kubernetes clusters using Docker
actually we run containerd and I think you will find it is comparatively lightweight and fast compared to most options :-)

it's certainly heavier than _not_ using Kubernetes

BenTheElder··on Kind - run local Kubernetes clusters using Docker
This shouldn't happen, can you please file a bug with more more information about your environment and specific usage?

I run kind in the minimum docker for mac spec which is one core / 1gb and it performs just fine. We've worked hard to make it lightweight, including a KEP upstream for slimming down the binaries.

BenTheElder··on Kind - run local Kubernetes clusters using Docker
Hi, kind author :-)

microk8s is really cool! We wanted kind for development of kubernetes itself and I don't think microk8s was around at the time.

One difference besides being able to build & run arbitrary Kubernetes versions is being able to run on mac, windows, and Linux instead of only where snap is supported.

We're paying more attention to local development of applications now, expect some major improvements soon :-)

BenTheElder··on Apple Is Sending URLs to Tencent?
Looks like the twitter post you referenced was deleted. That user's only recent post is about super mario maker 2...
BenTheElder··on Mozilla is giving up on their IRC server
Right, most of the APIs used are documented, but https://slack.com/api/users.admin.setInactive is not. There are some unofficial docs: https://github.com/ErikKalkoken/slackApiDoc/blob/master/user...

Otherwise APIs used are documented: https://api.slack.com/methods/files.delete https://api.slack.com/methods/chat.delete https://api.slack.com/methods/search.files https://api.slack.com/methods/search.messages

To be clear: NOT recommending Slack. Just mentioning that we also have issues and some tooling has helped, the tools could use more docs but they're pretty simple and open source FWIW.

BenTheElder··on Mozilla is giving up on their IRC server
Kubernetes also had (has?) pretty major issues with these, but a community member built some tools using what APIs they do have to make the moderation aspects easier at least: https://github.com/kubernetes-sigs/slack-infra

E.G. Reporting: https://twitter.com/KatharineBerry/status/110594147072323584...

Bulk-Deletion / banning: https://twitter.com/KatharineBerry/status/111310119103288934...

They've been really helpful, but Slack still seems _far_ from ideal for this.

BenTheElder··on Google donates money to help run the Kubernetes infrastructure
Echoing what Tim said, it seems to be standard practice on HN, I'm just trying to be transparent rather than somehow get accused of something(?)

I normally lurk more than I comment, but I'm disappointed to see how it's been represented. My teammates work hard _with_ the community.

BenTheElder··on Google donates money to help run the Kubernetes infrastructure
Disclosure: I work at Google on Kubernetes CI / Infra etc.

The tooling, configs, etc are all open source [1] (and largely run on Kubernetes!), but the actual deployments have been Google operated. This is about opening up access to operation of the deployments. I'm not about to stop working on the automation etc. :-)

[1] https://github.com/kubernetes/test-infra

Edit: if anyone is interested in more, we have a post out on the Kubernetes blog today about some of the infrastructure: https://kubernetes.io/blog/2018/08/29/the-machines-can-do-th...

BenTheElder··on Georgia Tech's free math textbook collective
Huh, I cannot reproduce that. I asked a friend who's never been to GT to check as well and they can't either.
BenTheElder··on Georgia Tech's free math textbook collective
This is one I remember using that I didn't see on there:

Professors William T. Trotter [1] and Mitchel T. Keller [2] Applied Combinatorics [3,4]

[1] http://people.math.gatech.edu/~trotter/

[2] http://rellek.net/home/

[3] http://rellek.net/book/app-comb.html

[4] https://people.math.gatech.edu/~trotter/book.pdf

BenTheElder··on Skip Lists Done Right
We had to implement them when I took it with Dr. Waters at least (Summer 2015). CS 1332 is a pretty good intro course.
BenTheElder··on [dead]
Yes, the previous projects preceding Kubernetes have a history of being Star Trek references, specifically to the Borg [1].

As to Heptio’s name, Beda explained: “When we were helping create Kubernetes, we pitched it as ‘Seven of Nine,’ a ‘Star Trek: Voyager’ character who’s a former Borg drone. That was a reference to the Borg, a code name for Google’s internal version of Kubernetes, the thing that runs its search and apps and ads. It’s total geek culture. We wanted a friendlier Borg. That name turned into ‘Project Seven.’ When we went public with Kubernetes, we didn’t want to lose track of the ‘seven.’ The Kubernetes logo has seven sides. ‘Hept’ is the Greek prefix for ‘seven,’ and it’s a way to pull the ‘seven’ through.”

[1] http://www.geekwire.com/2016/ever-come-kooky-kubernetes-name...

BenTheElder··on Cargo: predictable dependency management
I actually do this with one project, with a very small script that wraps cargo and sets $CARGO_HOME to a project local path before actually calling cargo. I keep a very short cargo.py and cargo.sh with my project.

I've experimented with using this to "vendor" dependencies in a local .cargo and so far that seems to work. An early example with discussion of the python version is here: https://www.reddit.com/r/rust/comments/3ea6je/is_there_a_bet...

← PreviousPage 2 of 4Next →