escardin

Do not provide backups and don't have backups aren't the same thing. If you have triply redundant local disks, you can probably afford take half of them and use them as backups for other systems and achieve better availability results (I'm assuming it's not triply redundant for performance).

While I did say S3, what I was really thinking about was Ceph. I don't think it's a silver bullet (almost certainly way more maintenance than a bunch of ZFS nodes), but if you're big enough to have multiple storage nodes with 100's of customers each (and again, triply redundant disks), then you could have built around the eventual failure of a node with what you already have. I'm not expecting them to hit S3's 11 9 availability, just taking a glance at what they have said about their design and proposing that basic changes to how they allocate what they already have would have avoided their problem in the first place.

I don't know what their exact situation looks like, or how they got into this situation. I see a post-mortem that says they spent 5 days trying desperately to recover customer data because they don't have backups, and they're not going to change anything about how they do things to eliminate the problem, even though it appears they have the raw storage capacity to have a backup. A sister comment says that brand damage, customer costs and recovery costs are just hypotheticals. They were, right up until this incident. Hopefully their internal postmortem has more details about what the costs were.

Clearly if they're trying to recover the customer data, it was important enough to the business to do so, and maybe it's time to re-evaluate 'no backups'.

There was one, obvious thing they could have done. Backups. ZFS even makes it easy. They state they have triple redundancy on their servers. Pull 1/2 of the drives and have backups. ZFS supports streaming snapshots (maybe not on this particularly old system). It sounds like they have multiple ZFS servers per datacenter, so given 2 servers, they could use 50% of the storage on each nodes as a backup of the other node.

It's not like the backups have to be customer available, use them to increase availability and decrease MTTR. In this situation, even with a daily snapshot they could have had customers up and running with yesterday's data while they took their time recovering the old system and not moving boxes around and bypassing safeties for speed. How much did five days of panic cost them? Their customers? Their brand?

I feel like they read something about how S3 has at least three copies of everything, and then did that locally with ZFS, instead of accounting for all the other failures that can happen that the S3 design accounts for.

You are right, there isn't a whole lot that could have been done without radically changing their infrastructure, but they're clearly at the scale and have the hardware available to make better choices than they have.

The difference between kubeadm and 'The Hard Way' is setting up a ton of certificates you then have to manage and manifests/config for etcd, kubelet, kube-proxy, kube-apiserver, kube-controller-manager, and core-dns. If you diff the default manifests of kubeadm and 'The Hard Way' you'll find they're not especially different.

You should still take the time to read the docs on the components and decide if you want to make changes. 'The Hard Way' doesn't explain what's going on any more than kubeadm does. It's just a set of instructions to follow (that are GCP specific).

I thought that too. Dials were more irritating because you had to move it juuuuusst right to get the time you wanted. It didn't help that it had acceleration, and that as you got higher it used bigger increments.

The best UX I've ever experienced on microwaves are restaurant microwaves and Danby microwaves.

Restaurant microwaves have programmable buttons and a space beside them to write what time is programmed in (or what recipe it is for). If you want a different time than the preprogrammed ones, then you press a time button and then punch in your time.

Danby's work similarly, you press a digit and you get that many minutes and it starts immediately. You press the +30sec and it adds 30 seconds. Simple, efficient, discoverable. Admittedly anything beyond the basics is a bit of a train wreck, but I'm okay pressing 1 instead of punching in 50 or 55 and then start when I want a bit less than one minute.

My current microwave only starts when you press start. The +30sec button will save you at MOST 1 button press ever, unless you want to add 30s to an already running microwave, which is more than I can say for my last rotary, where +1min would only work if you used it to start the microwave. If you used the dial, it didn't work.

Appliances have TERRIBLE UX, and I could go on about the good and bad things I have seen and been frustrated by.

That's already too complicated. You want a normally closed switch (i.e. on when not pressed). When the door is closed, it's pressed, and the light is off. When the door is open, the light is on.

that sounds like your pod-cidr, service-cidr, or network plugin are misconfigured. As another person said, going through the hard way could help you understand, but realistically you need to take some time to understand your network plugins config and the kubeadm flags for configuring cidrs.

I went through your problem before on a practice cluster and ended up giving up on that one.

Realistically the hard way doesn't actually explain much of the commands you're running, and much of it covers the tedium of getting the many many certs you need to bootstrap everything (which will then expire and not rotate automatically). It's much better to just use kubeadm, and take some time to read all the args to kubelet, kubeadm, kubeproxy, etc...

Trying to realtime encode 4k into AVC using pure software encoders takes about 80 cores/hyperthreads. Then you need to meet the tight 70ns? timings of the SMPTE 2022-6/7 protocols (i.e. 12gbps without using jumbo frames because reasons).

It's also necessary for clip switching. If you want frame accurate clip switching (i.e. show->ad->ad->show) you need consistent and precise pointers into your files.

The reason you don't see flicks used all that much is because media encoding is incredibly proprietary in the space where it matters most, live broadcast. Selling into the broadcast space, every broadcaster has petabytes of video in their own chosen format. You have to support EVERY format to sell broadly, and they're not going to let you transcode everything into your format. Using flicks gives you the ability to support as many combinations as possible with the least effort.

It matters very little when users are trained to tolerate slow transitions between videos, formats, etc...

It also doesn't matter a whole lot when doing offline transcoding either, as you can afford to do the more expensive calculation.

I have the same concerns about power. In the expanse they spend pretty much a whole book griping about how little sunlight there is to grow plants on a moon of Jupiter, and yet they have fusion powered star ships that can maintain 1g+ indefinitely without running out of fuel for a trip basically anywhere in the solar system.

Target Process is worse for agile. It is at once too flexible and too rigid. For someone who doesn't take the time to learn the tool, they end up with frustrating and not very useful defaults. For someone who does, you run into limitations like your backlog is always on the left or top because it won't let you move it around. It also ends up that as a power user you end up with a wildly different view from what everyone else sees, which causes additional problems.

Having a few extra levels of issue types in jira would be nice, but if you treat epics like TP features, you can use portfolio to add however many layers the PM and management team want on top.

I would say it is far less acceptable for boys to talk about their gender dysphoria than is it for girls, if only because even if a girl wears boys clothes and doesn't 'pass' they'll receive much less attention than the opposite situation.

That's entirely without considering the stigma society imposes on men who want to talk about their mental health at all.

It's much harder to automatically remove if it isn't static.

Have you tried the Personal Blocklist extension? https://chrome.google.com/webstore/detail/personal-blocklist...

Use your gas cap.