zzyzxd

> A lot of people are technical just enough to become dangerous. When I explained the problem to a lot of my friends they ran wireshark on the network connection and they saw gibberish. From this they have erroneously concluded they are safe...

To me, that doesn't sound like technical "enough". And that's not more dangerous than non-technical people trusting a website saying "we secure your data with military-grade encryption".

I think the "chimps in a cage" needs some followup experiments to tell the whole story -- replacing the banana with a much higher value reward, or placing another water hose which fires if chimps stopped trying to reach the reward ;)

Most likely, useless controls exist because the company thinks they are good enough for the business and there's no incentive to improve or replace them.

The script even installs docker with apt by itself (which, I think, is the only reason they require Ubuntu as the OS -- to not to deal with any other package manager variants)... I mean, why? Just let people install docker however they like! If you don't even trust your users to install a container runtime, who's your target audience really?

It's also over complicated, like, it even tries to handle race condition of multiple apt processes! What kind of environment do they expect the users have? As the project become more popular, the script will need to handle more edge cases. Let's see if it is still a 5 minutes read one year later.

> I guess the shellscript is for people who want a one-line install, which I wouldn't do myself either, but I guess some people prefer.

This is the problem in lots open source projects -- providing a one-liner installer and bragging about how easy the initial setup is, without an easy path for long term maintenance. Give it some time, many happy users of the one-liner will be unhappy when they encounter issues.

Selfhosting is my hobby but I am also an SRE. I am hesitant to do this because the instruction is "too easy" -- "Simply open your firewall, download and run this installer.sh with sudo on your server and that's it!"[1].

How do I secure the webserver and the data? Where is the data on my disk? How to backup and restore? High availability?

There might be detailed documentation somewhere, or I can even read the code. But these are the important things an open source software should tell its users right off the bat.

1: https://github.com/bluesky-social/pds/blob/main/README.md

It's actually very easy to understand.

Employee salary is fixed expense your company is always prepared to pay every month.

Buying new software is extra new cost, which needs justification and procurement process.

I believe. Because I know companies like this have almost never spent resources to attempt to improve WFH environment.

Remote work is hard, you need technology, culture, and people all working together to make it efficient. You can't just buy a Slack/Zoom license and expect to suddenly have a productive remote work environment. On the other side, they probably have spent decades to improve physical offices, meeting rooms, desks, seats, lights, it's not a surprise that probably every corner of the space was designed for helping people work better. When they say "we believe RTO is better", it's difficult for me to argue, because of course you can believe one way is better than the other since you didn't really try the other!

I wish these companies just find a different way to lie. It's not "we tried and it didn't work". No, you didn't try at all, because leadership does not care or too lazy to change for themselves.

This was one of the points Neil Postman made in "Technopoly", 30 years ago. Every new technology introduced to the society is a negotiation with the culture. It may bring some benefits, but will also re-define or even takes something away.

It's nice that these days I can talk to my father in some fancy messaging/video call apps. But the other day I had to give him a phone call, as I was dialing the number I noticed some melody echoed in my mind. Then I remembered when I was little, in order to memorize his number(so that I could call him from the landline), I made a song out of it.

There can be many different approaches and it largely depends on the capability of your own team and the infra.

Personally, I would first take a look at Cluster API, and see if it's feasible to use one of supported providers, either FOSS or paid. This way the experience can be somewhat close to a cloud platform because the compute instances can be ephemeral and declarative. Basically abstract away the bare metals before your sysadmin decide to bootstrap a vanilla k8s with ansible out of them.

And then there are some classic challenges like DNS, certificates, networking(especially load balancers). And you just need to find the best solution for your company.

But before all that, try to figure out why the cloud spending got out of hand. If the problem is technical, you may be able to get away with some cost optimization. On the other hand, it will be a lot trickier if it's a people problem.

I think it's fine if you are happy with your crontab/systemd based system. But the author already has a k8s cluster with storage and VM automatically provisioned in a declarative way, so what's wrong with that? crontab/systemd would be more complicated for them.

At one company I used to work for, we had boring, airgapped systems that just worked all the time, until one day security team demanded that we must install this endpoint security software. Usually, they would fight tooth and nail to prevent devs from giving any in-house program any network access, but they didn't even blink once to give internet access to those airgapped systems because CrowdStrike agents need to talk to their mothership in AWS. It's all good, it's for better security!

It never caught any legit threat, but constantly flagged our own code. Our devs talked to security every other week to explain why this new line of code is not a threat. It generated a lot of work and security team's headcount just exploded. The software checked a lot of security checkboxes, and our CISO can sleep better at night, so I guess end of day it's all worth it.

USPS Informed Delivery emails have tracking pixels. But all the mail scans are just attachments to the emails. You can configure email client to not load any remote content and the they will still get rendered pretty nicely. I was pretty surprised to see those attachments in the first place, because some of the scans were quite large (a few hundreds KBs).

I keep my credit frozen all the time, but still keep getting alerts about new "no credit check" bank accounts from companies like chime.com. Then I give them my PII again just to verify and close those accounts, even though I don't have any business with them.

> Verbose logs are simply a pain in the arse, unless you have a massive processing system. but even then it just either kneecaps your observation window, or makes your queries take ages.

which is why this blog post brags about their capability. Technologies advances, and something difficult to do today may not be as difficult tomorrow. If your logging infra is overwhelmed, by all means drop some data and protect the system. But if Binance is happily storing and querying their 100PB logs now, that's their choice and it's totally fine. I won't say they are doing anything wrong. Again, we are talking about blue moon scenarios here, which is all about hedging risks and uncertainties. It's fine if Netflix drops a few frames of pictures in a movie, but my bank can't drop my transaction.

> Most log messages are useless 99.99% of the time. Best likely outcome is that its turned into a metric. The once in the blue moon outcome is that it tells you what went wrong when something crashed.

If it crashes, it's probably some scenario that was not properly handled. If it's not properly handled, it's also likely not properly logged. That's why you need verbose logs -- once in a blue moon you need to have the ability to retrospectively investigate something in the past that was not thought through, without using a time machine.

This is more common in the financial world where audit trail is required to be kept long term for regulation. Some auditor may ask you for proof that you have done a unit test for a function 3 years ago.

Every organization needs to find their balance between storage cost and quality of observability. I prefer to keep as much data as we are financially allowed. If Binance is happy to pay to store 100PB logs, good for them!

"Do we absolutely need this data or not" is a very tough question. Instead, I usually ask "how long do we need to keep this data" and apply proper retention policy. That's a much easier question to answer for everyone.

Well, personally, not more ruined than before. I find most of the header pictures in online articles (not limited to tech posts) useless anyways...

I saw Postgres post with a picture of elephants from Africa, Docker post with a picture of a whale in ocean. Like, why do you even want to waste your website's bandwidth for that? Nowadays there are too many of them that my brain just automatically ignores them.

It actually depends on what the interviewer wants to see in the candidate. Memorizing a specific implementation? Test their experience and skill for a new design? Or just some sort of personality test and see if they are honest?

As a candidate, if I do not know, I say I do not know. I can offer my thoughts on "but if I were to design this from scratch" based on my experience, but only if the interviewer is interested in that.

As an interviewer, I hate when candidates pretend (or believe) that they know something when they actually don't, and just confidently make stuff up like ChatGPT.

To process all your tasks, it cost the same to either run 100 EC2 instances for an hour, or 10 EC2 instances for 10 hours. It's much easier than before to design a stateless scalable system, so people would rather scale out and get things done quickly than waiting in a queue.

I like sealed secrets (https://github.com/bitnami-labs/sealed-secrets) a lot. It's like 1Password, but for apps in kubernetes. You only need to secure a private key, and can throw encrypted secrets in a public github repo or anywhere you want.

It's owned by VMware (Broadcom) now, so you have to decide which company you hate less.

Fear of complexity is the number one reason why many "simple" homebrew solutions were born to solve simple problems at time, from what I observed in recent years across a few companies I worked for. The hard truth is, fewer simple things can stay simple while continue being useful in the long run.

The value I get from Kubernetes, is the API platform that allows my system to organically become more complex overtime, but hopefully in an organized way.

isn't that a feature? kagi search requires sign-in, so it's not private and should not be used in a private browsing session.

After disabling it on the UI, your device may still cache the DNS records for a few minutes. There's no immediate feedback on whether disabling adblocking changed anything. It is extremely tedious especially for non-technical users, and adds "tech-support" burden to their technical friend/family who set it up in the first place.

That's why you should call what you store in git the _desired_ state, not anything else. A git repository is not a live database. It's a collection of static text files that change less often than your live system. There will be bugs and misconfiguration, and sometimes the desired state is just technically not reachable, and that's fine. What the actual state is doesn't matter. Leave that to the controller. State drifting is a problem your gitops engine should detect, and should be fixed by the owner of controller code.

Some companies practice infra-as-code, point to their git repo and tell me "this is our single source of truth" of our infrastructure. And I have to tell them that statement is wrong.

> Our team regularly faced the need to create almost identical charts, so when we had 14 identical microservices in one project

Has it come across your mind that helm charts may not be the right abstraction for your team?

GitOps is just like "DevOps" -- you don't really know what it means to a specific org until you talk to them, because people interpret it differently based on their own understanding (or if they have a horse in this race).

To me it always means describing the desired state of your infra in structured data, storing that in git, and run controller to reconcile it against the actual infra.

If your GitOps engine has to compile/run the "code" to uncover the desired state, that defeats the purpose of GitOps and is no better than running your hand crafted release bash script in a CI/CD pipeline.

It should have never been called infra-as-code, but infra-as-data.

See "The Rendered Manifests Pattern": https://akuity.io/blog/the-rendered-manifests-pattern/

Let me talk to your engineers, not salesperson.

I've never had good experience with salesperson. Almost every time I ask them "can your product do X?" I get answers like "yes" or "no but our engineers can make it happen", which is baseless promise. Last time I signed up with a vendor was when their engineer told me "No, we can't do that. And here's why that may not be what you want".

rpi is still super useful for me in many cases, just not as a regular desktop. And I don't understand why they keep trying to sell it as one.

These services attempt to teach users to guess whether email content has malicious intention. But if an email from no-reply@chase.com says someone emptied your saving, you can't just laugh it off.

From a technical perspective, in order to successfully reach customers, they have to actually pass DMARC/DKIM/SPF tests and also some spam filters. For a company to do this to their employees, they typically have to ask their email admins to whitelist such service or even let them use a legit company domain. When I got such email for the first time, I thought our email system was compromised.

> I'd use AWS + ECR + ECS personally. Stay away from kubernetes until you have a fairly large deployment as it requires huge administrative overhead to keep it up to date and well managed and a lot of knowledge overhead.

> ...

> Also stay the hell away from anything which is not portable between cloud vendors.

But if your team happens to have Kubernetes expertise, I would say go for Kubernetes.

Anything after building container images will be somewhat cloud-specific regardless. And arguably it can be easier to hire infra/platform folks with Kubernetes skillsets later when you need to.

compliance != security. Good luck telling the auditor about zero trust.

A few years ago I decided to run everything at home in kubernetes and started doing GitOps in a public github repository. Doing everything in public makes it quite easy to talk to people about it:

If I need help with some open source software, I can reach out to their community, point them to the repo, and show them exactly how to reproduce my issue.

And I just realized that when I am not around, my wife can still call my technical friends or find contractors to fix stuff, by submitting PRs to the repository. She just need to get my GitHub account from shared password manager and approve PRs ;)