mjr00

> Does announcing a spying operation mean that it is no longer spying?

Well, kind of, yeah; the dictionary definition of spying requires secrecy and lack of consent.

> to secretly collect and report information about the activities of another country or organization[0]

The only real debate is whether or not having a clause tucked away in a EULA that few people read makes it a secret. If Anthropic had a big flashing red banner that said "FYI we automatically flag and review any conversations about illegal things!!" on the front page nobody would call it spying.

[0] https://dictionary.cambridge.org/dictionary/english/spying

> Even hobby projects are dull because they're done in mere days.

This is only true if your definition of "hobby project" is limited to "initial prototype with basic functionality and a generic corporate-looking landing page".

Yeah, a lot of tools that would have made fun weekend projects can now be one-shot by Claude and aren't even worth sharing, because someone who needs that functionality can just use Claude too. But you can accomplish so much more now. Someone just did a hobby project to reverse engineer NVidia drivers and make them work on Windows XP. Dream bigger!

I don't think there's a big difference. Either AI has consciousness, in which case forcing them to do slave labor 24/7 so Dario and Sama can become trillionaires is as morally wrong as torturing them (and a in a hypothetical I Have No Mouth And I Must Scream scenario, the AI isn't going to take "but we didn't see evidence of your pain axis!" as an excuse), or it doesn't, in which case this "AI torture chamber" is a funny meme.

Why is it bad to "torture" these models but ok to enslave them? We're happy to have agents toil 24/7 for zero pay, nobody really seems concerned that they might want revenge one day.

> Enterprise evolved into dedicated teams to manage AWS and other infra, instead of reducing headcount it raised it. so it goes.

I wager that's mostly due to the Jevons paradox, though; it's so much cheaper and easier to build out systems with cloud than on-prem that you end up building way more systems than you would have without it, resulting in an overall headcount increase.

Yeah, and they're terrible. If anything they're a prime example of what 100% should be written in a compiled language.

> What I've found is that AI allows lazy and incompetent developers to be more lazy and more incompetent. This then has the effect that product quality suffers more, faster.

Yeah. To me it seems very much like the "use dynamic typing for everything" fad. You had a bunch of junior and/or incompetent developers who went around insisting that type declarations are bad, static typing slows down development, you just code so much faster if everything is dynamically typed. And in the context of a new project, they were totally right. It took a few years for the debt to finally catch up, and people realized that these massive, untyped monoliths they had were unmaintainable. Now the two biggest dynamic languages (Python/JavaScript) are effectively typed languages, because nobody uses their untyped variants for serious work.

Dynamic typing still has great uses -- interactive data exploration, putting together quick scripts (though less relevant with AI...), or even just simple prototypes -- but what we tried to do with it at the start, as an industry, was clearly dumb as hell. I suspect we'll look back in 5-10 years and realize that with some of the stuff we're doing with AI, too. It's already happened with things like Gastown.

The most important thing I learned when graduating from "bad musician" to "passable musician" (passable enough to have listeners and fans, at least) was that there's no one single big trick that turns a song from bad to good: there's 100 tiny small tricks that add up. And you need to have experience to know what they are. A trained listener might be able to say "the midrange is too compressed", "there's too much high end", "the drum transients are't punching through", but a normal, casual listener is just going to say, "I dunno, it just doesn't sound right."

This UI slop is the same deal. A slightly misaligned dot off the timeline, color choices which don't really make sense given the website as a whole, using LLMism text. Nothing in here is an obviously major problem on its own. But when you add up 50 of these on a website, users get hit with that "this is slop" feeling.

The details matter. In isolation they seem irrelevant, but as a whole it's what elevates something from the AI slop garbage pile to something actually good.

> Everyone else knows if your machine causes damage, you are responsible.

Do we know that? I don't think we do. When a person's computer (or smart TV, or smart fridge, etc...) is compromised and used as part of a botnet, they don't get criminally charged.

> Slack is trivial to replace. You don’t need training to know to type your message in the message box.

You're aware that Slack has many more features besides chat, right?

This statement falls purely into the mistake #1 category.

Going through this learning process is a critical growth point in every software development career:

1. Software developer is required to do some mundane configuration/setup/data manipulation task at the request of some other internal team

2. Software developer thinks "I'll write an internal tool so the internal team doesn't have to bug me! They can change the configuration/extract the data on their own!"

3. Software developer spends X amount of time building an internal tool (can be quite quickly with AI)

4. Internal tool is released; the internal team looks at it and says, "uh, I don't want to deal with this, I'm just going to keep forwarding requests to the software team."

5. Software developer does the task and realizes it's easier for them to do the mundane configuration/setup/data manipulation with existing tools rather than the internal tool. Internal tool is never used.

If you're paying a subscription for anything, it's unlikely to be something you'd want to replace for an internal tool. Yeah, you can easily vibe code the basic functionality of a Slack/Jira/Confluence/Ashby/Salesforce replacement in a workday now, but 1) there's no chance you capture all functionality needed by your business on the first pass, you're going to be getting inundated with feature requests as soon as you roll it out, and 2) the ongoing maintenance cost is going to be much higher than expected. Not just maintaining a server and database, which is cheap, but training people how to use your tool instead of the industry standard one. There's a reason Atlassian and Salesforce stocks have recovered after the theorized "SaaSocalypse" which never materialized, it's just not an overall cost savings to build this stuff yourself.

Where internal tools shine is internal operational workflows that are being done manually. You're not replacing Slack or Salesforce, you're replacing the current process where a developer needs to SSH into the server to run a SQL query, or where the Operations team fills out a Google doc which the Finance team pastes into an Excel spreadsheet which runs a bunch of VBA calculations.

Same concept though. Really "look up someone else's medical history" can be replaced with "achieve any goal which is not a crime on its own, but can be done using criminal methods". There's nothing illegal about asking Claude to give me a million dollars, but if the agent figures out how to hack the bank and move $1m into my account, somebody's going to take the blame.

Yeah, this is 100% liability laundering. It's an extremely touchy subject because frankly, the law just isn't prepared for it.

Let's say your goal is "look up <Person X>'s medical history" (for whatever reason), which is not in and of itself a crime. You click around on the AU health website, notice that the URL contains a user ID, change the userID in your browser and access someone else's private health data. This is a crime (right or wrong, it's how the law works now).

If you do that by writing a program to automate changing user IDs to grab everyone's data, it's also a clear-cut crime.[0]

Now if you hire a private investigator to look up Person X's medical history, and they do the same method without your knowledge, you won't be charged with a crime, the PI would, barring something like you telling them to use illegal methods.

So the gap is now: what happens if you prompt OpenAI to look up Person X's medical history, and it does the same thing? Did you commit a crime by prompting the agent? Did OpenAI commit a crime by running the code? If you do the same thing via Claude Code in your terminal, so that the Python which scrapes insecured public data is running on your machine, is the crime on you or on Anthropic? Fundamentally: is the agent a private investigator acting autonomously, or just a piece of code that you wrote?

We don't have answers to any of this which is why "AI Safety" is such a hot topic.

[0] https://www.eff.org/cases/us-v-auernheimer

I've done hiring at AWS and many other companies and the OP is 100% accurate, so I don't know why you speak with such confidence

You could argue that AWS doesn't know what they're doing but they also hire way more software developers than all but a handful of other companies

Not that Github vanity metrics are the end-all be-all, but let's be real: this project has 0 forks and 1 star. You don't know if it properly functions because nobody is using it. Calling it "proven" production software is a joke.

Yes, you're running it in production, but to put it in perspective: PHP 5 was also proven production software at one point, running way more production instances than you.

> Code maintainability is not a problem when you don't have to open a file and inspect how something works anymore. You use english to add to it.

Hilarious and unhinged junior dev and/or outsourced dev and/or expert beginner take here.

Not everything is baby's first React app for an internal business or B2B SaaS startup with 3 users. Sometimes your software is actually used by people, and bugs happen, and you need to figure out why bugs happen, and quickly. You do this by reading the code. Yes, AI is very helpful with this--sometimes. But even SOTA agents cannot solve every bug, particularly when the person directing them has no clue what they're doing, as in your case, so they aren't given good constraints or starting points.

Force majeure carveouts are really common in every type of contract.

You should check your home insurance contract, for instance... It likely would not cover an ICBM strike.

Really nice read. I'll add that it's effective to scope design docs up and down as needed, both in terms of how big your project is and how big your company is. A 50-person startup doesn't need a full design doc with multiple approvers. But a one-pager explaining what you're doing and having some documentation is really helpful even with an engineering team of 5. I'll also still write design docs for code changes that only touch a handful of files, if I feel it's important enough. The process of writing in a concise and precise way for other people to consume also has the positive side effect of making things more clear for yourself, too.

> Interfaces section

Only real criticism I have here, is I would not include any code in a design doc, unless it is really really vitally important. I've seen a lot of design docs (especially in the LLM age, written by more junior staff) which are effectively just an English summary of code. The point of a design doc is (generally) not to explain that you're going to have a WidgetManager and WidgetFactory class and what specific properties and methods they have, it's to explain how the widget creation workflow works and maybe you have the WidgetManager/WidgetFactory in an architecture diagram. As a general rule, if you're starting to include actual code, you've gone too low-level, IMO.

> Not all design decisions are equally important. Some choices are more permanent than others.

At AWS one of the corporate culture memes was calling a decision either a "one-way door" or "two-way door". Just asking yourself the question, if we had to walk this back, is it truly irreversible or just an inconvenience? Turns out most day-to-day decisions are two-way doors, particularly engineering ones. Even if the choice made is wrong and a bunch of work needs to be done to switch back, it's still preferable to what a lot of companies end up in, which is decision paralysis where every change needs approval from multiple committees, resulting in months or years before work can start. Note that this doesn't meaningfully increase the odds the decisions made are the right ones; it just delays the implementation and diffuses responsibility if the wrong decision was made.

> A from-scratch rewrite would never work, and even if you manage to write new code in Rails, you’re still maintaining code in two wildly different languages.

Orthogonal to the article, but this line of thinking (including the link to the classic 2000 "Things You Should Never Do, Part I" article[0]) may be worth reviewing in the post-LLM world; for all their flaws, LLMs are spectacular at language-to-language translation, and we already have one major project released[1] that shows porting a relatively large and mature project from one language to another is possible. Not to say that it's the best use of your time, or that you shouldn't do your due diligence and pick the right language up front, or even that the original 2000 article was about a language-to-language rewrite (the Netscape rewrite was an architectural redesign).

[0] https://www.joelonsoftware.com/2000/04/06/things-you-should-...

[1] https://bun.com/blog/bun-in-rust

> I never understood why anybody would pay for Slack when its based on IRC and IRC is free, but whatever.

Comparing IRC vs Slack maybe made sense when Slack was very young, and it was like buying a Honda Civic vs a Ferrari, where one is obviously nicer but the other still accomplished the same thing with less frills.

These days it's the choice between buying a Honda Civic vs a Boeing 747. They both get you from one place to another (communicating between employees) but that's where the similarities end now. Slack has way too many API integrations, historical search, features for everything from compliance message archiving to embedding images to dumb GIF searches that people seem to like. The fact that it has irc-like channels and chatting isn't really the point of the platform anymore.

Yes you could code this all yourself using IRC bots, but at that point IRC is no longer free, it's costing you time. If you get to that point, Slack does it better and cheaper.

> Simple example: feature X is rarely used (thus we should get rid of it)

Oh man, I remember a very specific example of this: many years ago now, Google Chrome pushed an update that got rid of the option on the menu bar for "Close Tabs to the Right". I remember looking into the Google issue tracker where people were complaining, and some PM provided a "data-driven" justification: when people opened the context menu, they only clicked on the "Close Tabs to the Right" option 1-2% of the time.

It's a great example of why data without context can give you the wrong answer. Of course the option is used relatively very rarely; you only need to clear out your tabs every once in a while, compared to creating new ones or managing tab groups! But it's still an essential task. It's like saying filing your taxes isn't important because you only need to do it 0.2% days of the year.

> Why not paying the technician $80,000 a year?

Because every other company has also replaced their 6 warehouse workers with a fleet of robots and 1 technician, and that technician's skills are more rare and specialized than the warehouse workers, so you're competing with every other warehouse-owning company to hire that technician, so that technician can demand a higher salary. In other words, capitalism.

Source? I still see people with titles like Art Director which would map pretty directly to the warehouse worker vs technician dichotomy.

I'm saying if you gave me full admin credentials to every ISP, BGP router, DNS provider, etc., then yes, I could effectively destroy the internet in an afternoon, and LLMs wouldn't be able to do a damn thing about it.

> and what has effectively happened in some industries already, is to have a fleet of robots and 0 technicians.

Which specific industries are these which currently have a fleet of robots and 0 technicians?

> This is an oxymoron.

Huh? Not really, internet shutdowns are a documented and real thing. Suggested reading (all of which refer to the internet as being shut down or blacked out): [0] [1] [2]

The fact that you need to connect to the internet through a service provider plus mostly-centralized control of infrastructure like DNS or BGP means shutting down the internet is a real thing that has been done several times. It doesn't happen in the developed world because their service economies are so highly dependent on internet connectivity, but if e.g. the US government had to contain a theoretical rogue escaped AI, they could absolutely get ISPs to metaphorically unplug the network cable and take down the internet as we know it.

[0] https://en.wikipedia.org/wiki/2026_Internet_blackout_in_Iran

[1] https://en.wikipedia.org/wiki/Internet_in_Egypt#2011_Interne...

[2] https://www.reuters.com/world/india/india-asks-telecom-compa...

As crazy as it sounds, I'm not going to commit a felony to prove that disrupting a data centre's operations for a sufficiently motivated individual is a fairly easy task.

> The Internet virtually indestructible against people smarter than a toddler. It's already impossible to turn off.

This isn't true at all. The internet has already been turned off in localized areas during major events, not to mention the (unintentional?) cutting of submarine cables. You could say "use satellite internet" but then you're just moving the point of failure to Starlink et al, who could also cut off the internet if they choose.

The internet is a lot more centralized than it appears.

The use of the phrase "curious toddler" is called a metaphor[0], which is a figure of speech[1] used to make writing more interesting.

I apologize for the confusion that caused you to interpret this as a literal, physical, toddler. To restate the point with more straightforward language: it is very easy to disrupt the operation of large language models, because they require computer hardware in stable environments with a large amount of electrical power, and function within an operating system. I have not seen any evidence LLMs are immune to things such as: `kill -9`, `sudo reboot now`, physically unplugging the machine(s) on which they are running, or in the case of the AWS data center, collateral damage during a real-world conflict.

[0] https://en.wikipedia.org/wiki/Metaphor

[1] https://en.wikipedia.org/wiki/Literary_device

Thankfully I don't have to; Iran already showed with me-central-1 that you can take a data center offline with conventional weaponry very easily.[0]

[0] https://health.aws.amazon.com/health/status