01100011

Reducing which risk? The risk that the models of big American companies cause harm? What if they abide by some rules while models of others remain unfettered? That also carries risk. I appreciate the wish to have tame AI which only does good things but I'm afraid we're already beyond that.

Again you're shifting your arguments and not addressing your initial response.

It's frequent for SWEs to make blanket statements with considering the vast space of issues other people face that they don't have experience with or awareness of. Anyway, I'm not going to tell you what you do or don't need, only what worked and didn't for me.

In my codebase it is difficult to get agreement on comments and documentation so rather than rely on it I adapted. One of the first things I did when I succumbed to agentic development was to point codex at the code and ask it to generate a high level description of where important files, such as our public API, reside, what the hierarchy is, what the code does, etc. In my case, this level of documentation is fairly static if I avoid implementation details. So now I have a handful of agent files in my tree and it seems to save quite a few tokens and improve my results. I frequently have other devs ask me how I get such good results when doing agentic reviews of their changes(always my first step now before I start my human review). I also include instructions in the agents files instructing the agent to maintain the agent files if any relevant changes are made. It seems to work quite well for me.

So you want to regulate the sale of GPUs? How are you going to do that and why would it be more effective than currently failing measures to regulate nearly everything else?

Good examples. We are currently failing at all of that.

They very likely will via shell companies in jurisdictions outside the west. Are you suggesting we go to war to stop them? The massive datacenters are needed to serve models to millions. Criminal orgs don't need massive data centers anyway.

As Bruce Schneier recently discussed, law and tax law are code, just like source code. LLMs are great at finding holes in them. Illicit organizations looking to launder funds are most certainly interested in what AI can do for them.

Ok but this response has nothing to do with your other comment. What exactly were you trying to say?

So you cede the frontier of human progress to illicit organizations and other nations?

Nuclear weapons take a bit more work than doing math.

So you are contending that individuals cannot run advanced models? What brought you to that conclusion?

Secondly, are you contending that progress in model efficiency and hardware just stops at whatever level you think is sufficient to prevent individuals or organizations from acquiring sufficient resources to run advanced models?

Humanity has discovered a way to create a form of intelligence using math. This knowledge is not going back in the box.

When did chip demand skyrocket?

How long does it take to ramp up capacity?

Eventually... But it is materially relevant if that happens in 2 years or 10.

While I think SWEs(yeah, not HW/chip but that's not my field) are cooked in 5 years, I think we'll be quite busy in the meantime fixing all the bugs that AI finds.

Ask chatgpt or Gemini with reasoning enabled. It fixes a lot of issues.

Until Google wants to pay for reasoning for everyone their AI results will continue to be terrible. In my experience, reasoning fixes a great many of the sorts of hallucinations and falsehoods I've received from googling.

Lol sure dude.

We're uh.. we're a well known and widely used piece of software. There is an insanely large amount of testing that needs to go in to each change across many platforms and configurations and we have a lot of devs hammering that system with changes to test. Even worse now because of AI...

Code debt.

Software engineering is possible but largely a myth in practice.

Sure and sometimes that is what happens. Sometimes it's not. We should be doing a lot of things, but have to triage issues and act pragmatically.

In the case of what I'm currently working on, filing bugs for every issue I found, and then factoring out each fix, and then running each change through the 8 hour ci/cd system, and hoping an unrelated issue doesn't get misattributed to me... No, I'd rather just wrap it up into one coherent refactoring change and be done with it because when I'm done there are several more like it waiting for my attention.

FWIW, our codebase is growing, and the size of each change is also growing, but it's because AI is making us fix all the bugs we'd previously check in because our code long ago surpassed what even our best developers can reason about.

The funny thing is that the AI adopters are in the middle of the bell curve. Our worst devs continue to perform worse than AI yet refuse to use it and our best devs continue to insist AI sucks despite it finding issues in their code and the reviews and designs they've approved.

I tried Astra w/ high reasoning on a design document project and it was horrible. It started duplicating output lines, made document edits without permission, and basically did a poor job writing clear prose. I went back to 5.6-sol and it's great. I'm an OpenAI fanboy and was severely disappointed. I hope Astra is better for coding.

I think a lot of the success of the valley comes from disguising the thing the people are working on. If the end goals of our work were apparent to us we'd have a lot harder time showing up for work everyday.

That said, I don't buy this article. I'm AI neutral. It's going to break a lot of things and fix many others. Good for you if you think you know what the net outcome will be but I think it's delusional to believe you do. Also I don't think you can resign and stop it without ceding the world to bad people. Pandora's box has already been opened. We've discovered a new thing math can do and now it's yet another arms race between various factions seeking to exploit it. Help the faction you want to win. It's not the world I want it to be, but it is what it is.

Single shot or with reasoning enabled? My experience is that reasoning dramatically reduces hallucinations and improves output quality. I don't trust models without it.

Also, unlike SW engineers, lawyers control their industry and will quickly circle the wagons to protect themselves. Paralegals are probably screwed, but lawyers will do fine. I'm considering law as my post-ai-SWE-apocolypse career path.

Don't take my comment as a value judgement on the way things are or will soon be. Yes, I think new grads and younger engineers are screwed. Not my choice, not your fault. It is what it is. Sorry.

If a company thinks they are within reach of superhuman intelligence they probably aren't going to be concerned with compliance.

It's scary how much your comment aligns with my thinking. Yes to all of that.

Hah, I think I agree with this, but from a cynical, dark perspective.

I'm an old systems programmer (embedded, c/c++, linux, etc) with an EE background (but not one of them if you know what I mean. I enjoyed software and appreciate the art). But early on I came to believe software engineering at a workplace is not your personal art project. If you want that, great, go write code on your own time(as many of us do!). But on the job I often get annoyed by folks who don't want to do the obviously correct thing, or want to fight against good refactoring or design feedback (usually not from me.. I'm like a 'B' tier programmer and in awe of the 'A' level I work with). People treat software as an art and while you want to be artful in execution, if you are a paid artist, it is a commission work. Honor the requirements of the company (business needs) and not just your personal feelings and desire to only make art.

I've had some form of this conversation several times in the past few months:

Me: "Holy cow, LLMs are getting good. You should take another look at how good recent tools are because without them we're going to get left behind."

Them: "But I like writing code!"

Which is cool, but completely irrelevant in the context of a career. Write all the software you want. Paint all the pictures you want. Carve wood. Build a boat. Great! But in 3 years absolutely no one is going to pay you $500k+/year to do any of that and if you can't get your head around it you're going to have a bad time.

Idk folks, maybe it's because I actually don't like writing software as much as I thought I did. I'm realizing I actually like making products and creatively solving problems, and software is a mildly enjoyable means to and end for that. I also like many aspects of the job of programming (money, sitting on my ass, dictating much of my workday, feeling like a boss, also money, etc). I was lucky to be alive in a time when that role was in demand but I can now see that is coming to an end for at least 80% of us. Yes, probably you, and probably me.

Now is the time a lot of us should be contemplating our next move in life. Not easy given we're about to enter a period of utter economic chaos that makes the industrial revolution look tame. I'm saving money, building other skills and renewing my social connections.

None of this is a value judgement on AI. I'm not saying it's good or bad. It's probably both. I'm not making this statement based on anticipated future gains of AI. LLM advances could stall, token prices could quadruple. None of that changes the economics and the already existing technology that is sufficient to effectively end the role of a software engineer.

Idk, seems like overconfidence outside your domain really got going in the '00s and is largely perpetrated by SWEs.

Something about making a lot of money and living in a world of abstractions really seems to fuel our sense of overconfidence in our abilities in other areas.