8,471 karma · joined May 25, 2019
Let's say your goal is "look up <Person X>'s medical history" (for whatever reason), which is not in and of itself a crime. You click around on the AU health website, notice that the URL contains a user ID, change the userID in your browser and access someone else's private health data. This is a crime (right or wrong, it's how the law works now).
If you do that by writing a program to automate changing user IDs to grab everyone's data, it's also a clear-cut crime.[0]
Now if you hire a private investigator to look up Person X's medical history, and they do the same method without your knowledge, you won't be charged with a crime, the PI would, barring something like you telling them to use illegal methods.
So the gap is now: what happens if you prompt OpenAI to look up Person X's medical history, and it does the same thing? Did you commit a crime by prompting the agent? Did OpenAI commit a crime by running the code? If you do the same thing via Claude Code in your terminal, so that the Python which scrapes insecured public data is running on your machine, is the crime on you or on Anthropic? Fundamentally: is the agent a private investigator acting autonomously, or just a piece of code that you wrote?
We don't have answers to any of this which is why "AI Safety" is such a hot topic.
You could argue that AWS doesn't know what they're doing but they also hire way more software developers than all but a handful of other companies
Yes, you're running it in production, but to put it in perspective: PHP 5 was also proven production software at one point, running way more production instances than you.
Hilarious and unhinged junior dev and/or outsourced dev and/or expert beginner take here.
Not everything is baby's first React app for an internal business or B2B SaaS startup with 3 users. Sometimes your software is actually used by people, and bugs happen, and you need to figure out why bugs happen, and quickly. You do this by reading the code. Yes, AI is very helpful with this--sometimes. But even SOTA agents cannot solve every bug, particularly when the person directing them has no clue what they're doing, as in your case, so they aren't given good constraints or starting points.
You should check your home insurance contract, for instance... It likely would not cover an ICBM strike.
> Interfaces section
Only real criticism I have here, is I would not include any code in a design doc, unless it is really really vitally important. I've seen a lot of design docs (especially in the LLM age, written by more junior staff) which are effectively just an English summary of code. The point of a design doc is (generally) not to explain that you're going to have a WidgetManager and WidgetFactory class and what specific properties and methods they have, it's to explain how the widget creation workflow works and maybe you have the WidgetManager/WidgetFactory in an architecture diagram. As a general rule, if you're starting to include actual code, you've gone too low-level, IMO.
> Not all design decisions are equally important. Some choices are more permanent than others.
At AWS one of the corporate culture memes was calling a decision either a "one-way door" or "two-way door". Just asking yourself the question, if we had to walk this back, is it truly irreversible or just an inconvenience? Turns out most day-to-day decisions are two-way doors, particularly engineering ones. Even if the choice made is wrong and a bunch of work needs to be done to switch back, it's still preferable to what a lot of companies end up in, which is decision paralysis where every change needs approval from multiple committees, resulting in months or years before work can start. Note that this doesn't meaningfully increase the odds the decisions made are the right ones; it just delays the implementation and diffuses responsibility if the wrong decision was made.
> A from-scratch rewrite would never work, and even if you manage to write new code in Rails, you’re still maintaining code in two wildly different languages.
Orthogonal to the article, but this line of thinking (including the link to the classic 2000 "Things You Should Never Do, Part I" article[0]) may be worth reviewing in the post-LLM world; for all their flaws, LLMs are spectacular at language-to-language translation, and we already have one major project released[1] that shows porting a relatively large and mature project from one language to another is possible. Not to say that it's the best use of your time, or that you shouldn't do your due diligence and pick the right language up front, or even that the original 2000 article was about a language-to-language rewrite (the Netscape rewrite was an architectural redesign).
[0] https://www.joelonsoftware.com/2000/04/06/things-you-should-...
Comparing IRC vs Slack maybe made sense when Slack was very young, and it was like buying a Honda Civic vs a Ferrari, where one is obviously nicer but the other still accomplished the same thing with less frills.
These days it's the choice between buying a Honda Civic vs a Boeing 747. They both get you from one place to another (communicating between employees) but that's where the similarities end now. Slack has way too many API integrations, historical search, features for everything from compliance message archiving to embedding images to dumb GIF searches that people seem to like. The fact that it has irc-like channels and chatting isn't really the point of the platform anymore.
Yes you could code this all yourself using IRC bots, but at that point IRC is no longer free, it's costing you time. If you get to that point, Slack does it better and cheaper.
Oh man, I remember a very specific example of this: many years ago now, Google Chrome pushed an update that got rid of the option on the menu bar for "Close Tabs to the Right". I remember looking into the Google issue tracker where people were complaining, and some PM provided a "data-driven" justification: when people opened the context menu, they only clicked on the "Close Tabs to the Right" option 1-2% of the time.
It's a great example of why data without context can give you the wrong answer. Of course the option is used relatively very rarely; you only need to clear out your tabs every once in a while, compared to creating new ones or managing tab groups! But it's still an essential task. It's like saying filing your taxes isn't important because you only need to do it 0.2% days of the year.
Because every other company has also replaced their 6 warehouse workers with a fleet of robots and 1 technician, and that technician's skills are more rare and specialized than the warehouse workers, so you're competing with every other warehouse-owning company to hire that technician, so that technician can demand a higher salary. In other words, capitalism.
Which specific industries are these which currently have a fleet of robots and 0 technicians?
Huh? Not really, internet shutdowns are a documented and real thing. Suggested reading (all of which refer to the internet as being shut down or blacked out): [0] [1] [2]
The fact that you need to connect to the internet through a service provider plus mostly-centralized control of infrastructure like DNS or BGP means shutting down the internet is a real thing that has been done several times. It doesn't happen in the developed world because their service economies are so highly dependent on internet connectivity, but if e.g. the US government had to contain a theoretical rogue escaped AI, they could absolutely get ISPs to metaphorically unplug the network cable and take down the internet as we know it.
[0] https://en.wikipedia.org/wiki/2026_Internet_blackout_in_Iran
[1] https://en.wikipedia.org/wiki/Internet_in_Egypt#2011_Interne...
[2] https://www.reuters.com/world/india/india-asks-telecom-compa...
This isn't true at all. The internet has already been turned off in localized areas during major events, not to mention the (unintentional?) cutting of submarine cables. You could say "use satellite internet" but then you're just moving the point of failure to Starlink et al, who could also cut off the internet if they choose.
The internet is a lot more centralized than it appears.
I apologize for the confusion that caused you to interpret this as a literal, physical, toddler. To restate the point with more straightforward language: it is very easy to disrupt the operation of large language models, because they require computer hardware in stable environments with a large amount of electrical power, and function within an operating system. I have not seen any evidence LLMs are immune to things such as: `kill -9`, `sudo reboot now`, physically unplugging the machine(s) on which they are running, or in the case of the AWS data center, collateral damage during a real-world conflict.
It's a great question. I've yet to see any research into how a humanity-destroying LLM defends itself against a curious toddler who pulls the power cord out of the wall.
This would be true if you're talking about randomly picking people from around the world to live in the US for their formative years and go through the US education system (at which point they barely count as "foreigners"). This isn't reality, though, and if you're talking about hiring a worker today, a large swath of that 95% won't speak English, or won't be literate, or will have never received a formal education, all of which are disqualifying for pretty much every job in the US.
s/software engineering/running a business.
I'd much prefer that the people building the bridges and planes I use, or the staff at the hospital I visit, have procedures in place to prevent them from "making a whoopsie" than the space to be creative or interesting. Even if they enjoy their job less.
The good news with software engineering is that unlike building a plane or running a hospital, nobody is stopping you from making your own projects and being creative and interesting as you want!
How long after that moment did the ChatGPT robots need to invent the time travel machine which let you come back to 2026 to confidently claim this?
Reflective of the times. Most people on the internet don't know how to install a torrent client, or even download a file. (Because most people on the internet are on a phone. I don't even know if it's possible to torrent on a phone.) And the stuff people download is mostly "free" anyway via Netflix, Spotify and other content streaming services. A download no longer represents a lost $50 Blu-ray or $20 CD sale to the bean counters, it represents a lost $0.01 streaming royalty. It seems like the only time copyright enforcement bares its teeth these days is for live sports matches.
> I often work with data scientists and have to productionize their jupyter notebooks
I'm not a huge Python fan, despite working with it fulltime, but this feels like mixing correlation and causation. Data scientists would not be writing good, optimized code in any language.
I assume the "gatekeeping" decision to not implement a feature request is coming from someone responsible for the product, not from a developer.
> Workflows like jira_close.yml use deprecated atlassian JIRA actions and have a dependency on the gh-actions repo. This is not ideal and unecessarily complex. PR updates jira_close workflow to use direct API calls via curl. It preserves custom fields used too.
I won't speak to this projects' management and how they prioritize things, but from my own experience, pre-AI, this type of change would have been firmly in the "this is a minor annoyance, put it in the Tech Debt Backlog alongside the 50000 other tickets" and never actually done. The cost of a human investing the time understanding how to fix the problem, doing code changes, testing them, and deploying them is just way too high for what actual value this change brings, which is close to nothing.
Now with AI, it's as simple as firing up an agent and telling them to make a change; as much effort as writing that backlog Jira ticket in the first place.
Similar to the problem open source is having with low-value PRs, companies are going to have to start realizing that code is not free to review or maintain, even when it's generated for ~free, in their internal processes. Just because an agent can fix a minor tech debt annoyance with a few lines of instructions doesn't mean it should.
[0] https://github.com/snowflakedb/snowflake-connector-net/pull/...
I specifically remember a prolific HN commenter working on an AI video startup claiming that by the end of 2025, children would be able to create Hollywood-quality, feature-length movies using available AI video tools.
I believe this person's startup has (wisely) pivoted to building AI-powered tools for the movie industry.
Too true. This isn't limited to AI, either. The most obvious example in my lifetime was during peak blockchain hype, when people who had never worked in finance convinced themselves that blockchain was going to act as the backbone for how money gets moved around. As if the problem that needing solving was Bank of America doesn't trust Capital One to update a number in their database.
The nice thing about AI, at least, is I can always push back and tell people, "Sure, we can do this with AI. I just need you to use Claude or ChatGPT manually to prototype how it would work." This normally results in the requestor realizing that there's human judgment calls involved in the inputs, process, or outputs that require meatbag intelligence.
Maybe it wasn't clear in my comment, but this was already happening in many large organizations pre-AI: you had a small contingent of competent senior developers whose job was to review and correct outputs from junior and/or outsourced staff. From people I've spoken to in these types of situations, they're very happy with getting rid of their offshore teams and using AI instead.