1,215 karma · joined June 16, 2015
I use this today to keep very “wide” modules synced
Define everything in like cdk (... I use ruby to generate the tf.json), generate the code, import everything you can without error, and apply the rest.
Performance will be _bad_ but that will completely eliminate state problems.
I did this with Nomad by writing a small API to "place" host volumes which Nomad can use for job placement constraints automatically. It works shockingly well for my use-case which I think is somewhat similar to Fly's? A smallish number of containers that require host persistent storage, and don't expect anything to move -- just disappear forever (persistent til' it ain't).
They show this in the video! I actually almost wish the initial Steam Controller had these flat pads instead of the concave circles, it looks like it's going to be a straight up improvement.
I'd imagine if anything it makes databases _easier_ to reason with, because you could shard by workflow steps or something.
eBPF can (basically) monitor anything at a low level! check out ebpf_exporter for some other great examples of real-world use.
If you don’t have a way to do immutable infrastructure, Ansible is still extremely alive.
From what I can tell is this just Jack shilling for BTC?
We run a Raft cluster of 5 voting members with 13,000 physical Nomad nodes mixed across OS's with a bunch of random workloads using docker, exec, raw_exec, java, and some in-house drivers. I'll clarify that (and I can't... because I can't edit my post anymore :( ).
Even better if you're running Mac + BSD as well, it just runs everywhere.
Source: I used and helped maintain one of the “don’t federate with these instances” list.
> Interesting, so JWT proxy (or any other auth mechanism that is viable over HTTPS) -> ghostunnel machine w/ public VIP -> Target machine ? Or ghostunnel directly running on the Target machine which holds the public VIP? Or does the JWT proxy take the public IP and the ghostunnel machine keep the private VIP?
jwt proxy takes in the iap jwt, they give you the audience and it's just parsed, this lives in the same pod as ghostunnel. ghostunnel goes through a NAT to a public dest, where ghostunnel is _also_ running. it has extremely strict TLS requirements (Forced valid CN to be sent/accepted, strict DNS, along with a single purpose CA, cert, and key).
It's like
IAP =inside-gcp=> (JWT proxy -> ghostunnel) =public=> (ghostunnel -> thing)
There's a more modern way to do this, but this works really well and gives absolutely fantastic introspection, is super easy to use it as a public proxy, and allows you to only use minimal APIs in GCP (IAP + GKE, you don't need GKE but you also don't wanna manage all the things it does for you with annotations. :D)
I use a JWT proxy + ghostunnel within GKE with a VIP so it's not quite their reference setup but it's extremely "just works" outside GKE being weird and eating its own routes.
BTW, side-note but try out ghostunnel over stunnel! I've really enjoyed using it and it's been fantastic to debug and work with.
Most of the problems I've had with Vault have been around it's Terraform provider which they've improved enough that it's not an issue anymore.
I think the only thing about Raft that folks don't realize is how disk hungry it gets, if you want fast write performance you gotta make dang sure all those fsyncs can keep up. Our largest Consul cluster today runs on storage-heavy boxes as it does ~500Mb/s of writes pretty much 24/7.
I use Consul w/ Vault today instead of the internal storage for Vault just cause Consul has really nice monitoring around some stuff that Vault doesn't (path-based stuff for the most part), I think the internal storage is a really good option for 90% of use-cases.