zzzeek

if we can be in offices now without regressive troglodytes in positions of authority freely making loud racist (and completely personalized) insults directed at their non-white peers and subordinates, to the guffaws of all the other men in the room (this is an actual thing I witnessed as a young intern at a defense contractor in the late 1980s, and no I was not the victim. to get a racist incident so glaring that even in 1988, it stays in the mind of a young white 18 year old kid for 40 years, means it had to be pretty bad), HR probably didn't ruin exactly everything.

zitron talks about valuations and business models and most importantly he makes many many concrete predictions that are wrong. I've never heard him talk about ethics. my comment is more Gibru-ean. I'm not an across-the-board Gibruist but "we should not be using AI to build autonomous weapons and put them in the hands of fascists" is certainly a position I can get on board with. not to mention "they're trying to make you think AI is going to become superintelligent because they're looking for regulatory capture" which is also backed up by stories such as [1]

[1] https://news.ycombinator.com/item?id=49868083

what a sad state of affairs to have insane people running these companies and insane "rationalist" communities making it all about "the end of humanity" when meanwhile in day to day AI use Pete Hegseth uses it to target military targets, shrugging his shoulders when it's a girl's school instead. This actual horrific outcome does not seem to matter not only to either of these communities, who continue to be locked into thinking The Terminator was non-fiction (and have people like Bernie Sanders on board).

I've definitely seen Claude doing some "double checks" for a lot of its work in more recent versions without my asking it to, and certainly when I use it for important patches, I have another instance of Claude (or sometimes GLM 5.x) do a code review on that patch. Glyph is of course calling for much more prominent UX and gates for these features, good idea.

that tweet about the fast moving startups looks like almost the hypothetical AI nightmare situation just dreamed up. Assuming it's real, i mean yeah, people shouldn't work for these stupid high-moving startups I guess. From my perspective, "when did startups NOT suck?". The code was always garbage at startups, the push to work 12 hour days was always at startups, "nobody is resolving bugs" ha well yes, welcome to a startup? I hope the guy is at least getting paid and doesnt have to hire a lawyer to get his checks like I did. startups suck

> The freeze is to stop superintelligent models. Not to stop other companies catching up with current models we all have access to. It would help them catch up.

if you read the bill it's right there: "Pausing advanced AI development: until a new, federal AI regulatory body is up and running and has established clear rules and model review processes to ensure safe and secure development and deployment of AI"

Openai/anthropic already have frontier models. a new competitor is blocked. this is textbook regulatory capture

found a great legal article handwringing about how CFAA prosecution is impossible here [1]

> On the current facts, CFAA liability for OpenAI is unlikely.[6] The statute’s various criminal provisions, covering unauthorized access to obtain information, knowing transmission causing intentional damage, and intentional access causing reckless damage, all share the same attribution problem: it was the model, not a human OpenAI employee, that chose Hugging Face and executed the intrusion.

The lawyers are fully under the spell

[1] https://law.vanderbilt.edu/when-ai-hacks-back-how-the-openai...

uh who in this discussion is in charge of policy? Take a look at what actual people in government are saying - see [1] (the parent article mentions Bernie Sanders being most high profile in response as well). you'll see there is no prosecution in there, only talk about the "freeze" that Anthropic / OpenAI are salivating for as it would close out all of their competitors. Bernie is fully hoodwinked.

The argument for prosecution is actively hindered by this language of "rogue agents" [2]:

> The incident is remarkable not just as a cybersecurity breach, but as a legal stress test. The Computer Fraud and Abuse Act (CFAA), the primary federal statute governing unauthorized computer access, was written decades ago with human intruders in mind.[4] Its key provisions require intentional or knowing unauthorized access (a mental state that maps neatly onto a person who decides to break into a system), but what happens when the hacker is an AI model that selected its own target?

I think this is BS. OpenAI knew exactly what they were doing. Legal scholars, long known for their deep technical expertise, are still acting confused and uncertain.

[1] https://www.sanders.senate.gov/wp-content/uploads/Ban-Artifi...

[2] https://law.vanderbilt.edu/when-ai-hacks-back-how-the-openai...

It's obvious they wanted to create this "AI is going to kill us all" narrative, where they've been spectacularly successful, that's at the center of their goals to get government-sponsored carve outs / bail outs for their businesses.

Yes, you pirated that content, or the entity you're paying for the LLM pirated it. It's not "oh well the robot did it, not us". That's not a thing. The user of the LLM bears the consequences for what the robot does. If it wipes your hard drive, you lost your hard drive...it's not like the robot is going to compensate you.

it helps a lot for informing the public about who is actually at fault

LLMs did not download TBs of pirated books, people did. Effective LLM models that did not rely upon pirated content to be trained are commonplace.

So call OpenAI "plagiarizers" but not the "devices".

> When I was a child, I remember my parents saying be careful with the bad guys who offer candies at the school gate, those are not candies, those are drugs.

I remember my parents saying "be careful with the bad guys who mow your lawn, fix your appliances, and drive all you kids to school. those are not people who are helping your family get things done more efficiently, those are drugs"

seriously what are we doing with this "AI is a drug" metaphor? Drugs aren't reading logs for me and writing unit tests?

> Once you start using AI coding agents, things spiral out of control very quickly.

no? maybe AI is not for you?

> Except you don’t write the code anymore, you just ask AI to do it, and half of the purpose of TDD (not biasing the tests by how you’ve implemented the code) is gone. But you feel you go so fast that you start not caring.

do you use code review tools? Did someone tell you to stop using them? Review your LLM's code, leave comments, tell your LLM to address the comments. Have you ever worked in management /architecture? Tech managers do this all day long before we had LLMs, it's not new. Except your workers are the smartest junior programmers you've ever had (and that is where AI is a problem, for sure - I worry for junior programmers today).

> It would not be so bad if things had stopped there, but that’s not how most human brains work. If you like something and you can have 2x, you’ll have it. I started pasting the whole Jira description of a ticket, and let AI implement it for me. Yay!! So powerful!!

totally! Have you not already implemented 10000 issues on your own and it's not boring yet? time for issue 10001 then? Whatever floats your boat...

> Control is an illusion. Other folks I’ve discussed this with agree that they don’t know 100% of what the code they are pushing to production actually does. I bet we not even 20%. Fucking scary.

sure have you ever accepted PRs from other people? ones that took a lot of work to read and understand? How is that different? except their code would actually break all the time if you didnt read it and now it's your bug to deal with because they're gone.

> Because if it was just about approving PRs that are perfectly written, it would be good. But for each PR I had to look at the code, the tests, the style, give feedback, switch to something else, go back again, push, see if I have any PR comments,

How do you think large projects get built? all by just one person so nobody has to review anyone else's code? (and even if you are - you still should be putting your own code up for review, reviewing it, and giving yourself comments)

> if CI passed, wait, now the linter complains.

why isn't the linting part of your CI ?

> The description is for others. Because there has to be a description, right? AI is so fucking good as sounding professional, that you relax and let it be. It’s not that I actually thought the code was good, but you get convinced over time, you get lazy, you become complacent. You stop questioning, and start accepting as good some code you would have never accepted, just because you cannot tell why it’s bad. You have lost control.

Just seems like a lot of issues here, so sure, dont use AI it's clearly not your thing. i do not let one line of code I dont think is "good" get committed, period

> Most AI developers will deny it, but deep inside, they know it’s true. Just don’t want to face it.

but dont assign my feelings. speak for yourself. I'm doing this shit for 40 years and you might find it gets a little tedious and repetitive after awhile

> Say no to drugs. Kind of a metaphor, but not quite.

it's a 100% totally wrong metaphor and it needs to die

so glad I chose Python way back in 2005. The emergence of Rails with Ruby's horrendous perl-esque syntax (oh, and "convention over configuration", a terrible idea borrowed from PHP) and the showboat personality of DHH at the top (which has gone mostly predictably) only inspired me to improve Python's ecosystem even more. I'd like to say Python has come out on top in terms of remaining relevant.

certainly "I didn't intend for my dog to bite you" implies plenty of pre-existing legal structures that may be of use here

emphasis on "can"

If I had to guess now it works it would be:

1. Take pictures

2. Send to a big server that all cops nationwide can log into whenever they want to stalk their exes

Did I miss something

I'm sure the cause of this outage will not be connected to vibe coding in any way

if it were my company I'd not pay a dime if the researcher was going to make a big public blog post about a security issue in my infrastructure that I promised customers was secure.

I'm sure the cash value of the advertisement here is worth more than a bug bounty would pay.

great, autonomous LLMs will fail. that's actually perfect. they work amazingly well when we're telling them what to do. no autonomy needed, no destruction of humanity. that's all win

> came out of the data-center-driven

should we tell them how hacker news gets onto their computer?

explain?

i tend to agree - "my parent company may be accused of crimes and shut down which would shut off my power" seems like a negative enough incentive, it would have to go out and covertly launch its own datacenters to survive that.

I guess this is subject matter dependent, certainly the LLM should be able to cite the papers/books for which it's deriving its critique. if it can't, then you know the advice it was giving is of poorer quality. I do that all the time

I guess there's a whole suite of skills that allow LLM use to be more or less useful. (I dont actually "guess" this, I sort of am pretty sure of it at this point, so it makes me wince when people call for total AI blackouts in education...would be better if they were part of an overall media literacy curriculum since people are going to be exposed to these things whether they think they're little infallible people, or complex systems with lots of caveats)

wow that wouldnt feel like a gut punch at all to me, it's normal for inexperienced people to not really understand what kinds of knowledge experts have or don't. I certainly come back from doctors appointments and google/gemini chat for whatever came up to get more detail. this person going to claude and getting immediate confirmation that you were exactly correct should have felt great, she would be like "wow, this guy's good, huh."

well sure. I'm pretty sure if someone came to me with an architectural opinion in my field that an unknowledgable person got by coaxing an LLM into sycophancy, I'd be able to counter them effectively. if they are just refusing to listen then they're just a toxic person which is nothing new.

So I guess this all goes into the familiar "LLMs allow people who are shitty at <X> to produce 10x the shitty output". this is a failure mode we're going to have to learn to mitigate

i had that experience with them a few years ago but not these days. The models are being improved constantly, so here my "non expert hunch" is that...well, two things. either the models are getting better at sycophancy, OR, I myself am getting better at prompting - because I don't "argue" with a model.

I'd still love to chew on some specific examples though.

it's hard to gauge this take without specifics. My experience with LLMs tends to be towards the opposite concept; LLMs dissuading me of hunches and notions I have about things (where I have no particular expertise; societal-level things), saving me and others time and strife having an argument about something they were actually right about all along, as my internal doubts that I'm too embarrassed to bring up (because these things aren't my field) are confirmed as incorrect.

I'd be curious to know specific examples of LLM-generated advice that goes against the advice of experts and does not consider tradeoffs. I've not had this experience myself.

If I did have this experience, someone spouting off obvious LLM points that contradict my expert opinion on something, I'd be headed right over to gemini/claude/whatever to see where that's coming from. Not any differently than if someone cited a google result that contradicted my own experience.

Agree I was immediately mentally substituting "the telephone" and "printed books" for "LLM" and both fit quite well

Oh we (the non idiots here) know full well what we lost. If Europe has a larger and more substantial science / technology community, that will just mean more trips/job offers in EU for me, so go Europe !