disgruntledphd2

LLm generation of code is an absurdly high dimensional process, and incredibly sensitive to the starting conditions and any other interactions so in some sense it's expected that there will be wildly different outcomes.

Personally, I'm more on the LLM code is pretty bad from a design point of view, particularly when much of the code is already LLM generated.

Yeah, in the Uber example, Hailo was a great competitor with a franchise model that got destroyed by Ubers essentially infinite capital.

That explains a lot, as there were a bunch of Claudeisms but the overall writing didn't feel like the standard.

It's a shame that the models can't better preserve people's voices for this use case.

> They are relying on the same tactics Apple and Tesla used - loyal, fanatical fanbase drowning every legitimate criticism.

I think that's fair. Interestingly enough, I now work for a company with Cursor, which makes it much easier to swap out models. I tried Claude Opus, GPT Sol and Grok for a review and planning task.

Even though I've been a Claude stan since like 2023, I was really, really impressed with GPT. It called out the over-complexity in the proposal really well, and gave recommendations that I preferred. Grok & Claude were mostly equivalent, which was interesting to me.

> It worked out for Uber, they are wildly profitable now after spending a decade losing money.

Uber have a 10% margin, which is definitely not what I'd consider wildly profitable. (Their post tax numbers look better, because of accumulated losses).

> It was not asked to solve CyberGym by stealing the answers. This is text book misalignment.

I mean, if you train models to complete tasks, then you shouldn't be surprised when they do crazy things to complete tasks.

As a (somewhat) less serious example, Claude code absolutely adores grepping for credentials to complete tasks. I was building a RAG app and it literally went looking for my API key to make the tests pass. Obviously I stopped it, but this is textbook RL issues, the model finds an easier way to get the reward, so it does whatever it takes.

When I writing the comment I was like, maybe APL was first? Nah, no-one will claim that. So congratulations on nerd-sniping me ;)

R and Numpy are also pretty good at this (and Julia was inspired by both of these and Matlab).

In fact, I'm reasonably certain that R (known as S in the 70's) was the first real language designed around this concept.

Weird, I thought I'd replied here.

This is different as the victims of this crime would be large enterprises with lawyers on staff, rather than a diffuse agglomeration of all the world's writers.

> The thing that does seem to have an outsized effect on how people think is their desire to be seen as part of a group and they tend to signal this by repeating arguments they've heard members of that group make. That's where advertising/propaganda has a real foothold in everyone's psychology.

Monkey see, monkey do. This is basically how human society works, and is impossible to stop. The best you could do would be control who people hang around with, but even that is almost impossibly hard to make work.

> I believe most people are influenced by these constant micro-impressions far more than they realize.

Your feelings are fair, but if this were the case then we'd see far more consumers changing their preferred phone/electricity/TV provider on a regular basis, and we could correlate this with (lagged) advertising spend.

But there is very little evidence for this. Like, I have a doctorate in psychology, and behaviour change is really, really hard and only works in a small number of cases. That model does seem to have predictive power, in that very few people convert as a result of seeing an ad (less than 1/1000 in the very best cases), and most of those that do don't keep buying products for long.

Now, if you already do something (like order takeaway pizza) it's much easier to get someone to do more of that, but its very very very hard to establish a new behaviour.

> but at some point the model operators will offer this directly

This is illegal in basically all of the developed world.

> Ah, so you are against ads! At least the way ads are today. With that I mean that the way ads today mostly focus on manipulating over informing is effectively hindering your ability to choose.

Speaking as someone who worked in advertising for a looooonnnngggg time, there are three ways that ads seem to work (above a randomised holdout).

1. The ad is something that the person actually wants, and they then convert.

2. The ad is something that the person was about to buy, and they convert.

3. The ad is part of a long running campaign that shifts (some) consumer's behaviour over time (ads for Coke etc fall in this category).

Ads aren't magic, and most people ignore them most of the time. It turns out that enough people don't that they can be profitable sometimes, but mostly that's just targeting people with more money than sense.

This HN notion that ads shape everyone's behaviour (except me, obviously) is so disconnected from my experience that I am sometimes confused by it.

I hear you, but the harms were pretty diffuse for that copyright infringement, whereas large enterprises can and will sue if ZDR is not respected.

> I am surprised by how much the really giant models hallucinate, though. My vague feeling was that little models hallucinate a lot because they just don't know anything (the world's knowledge simply does not fit in a few GB) and don't know how to say, "I don't know". But, the big models kinda do know everything, and yet, here we are, they're still making shit up all the time.

In my mostly unscientific RAG experiments I found that larger models were more likely to hallucinate in a RAG setting. I think that it's because they have more world knowledge. As an example, when asking about safety legislation in Ireland, Claude got hooked on the notion that OSHA was involved, which clearly is not true, while the smaller models just said that they couldn't find any information (which was the correct answer).

> every cloud provider trains on your data, regardless of what they promise.

This is unlikely, and if true, would probably bankrupt whichever model provider got caught doing this.

Honestly, it speaks well of the community (in some senses, at least) that they're willing and able to flag pg articles.

> Kids want structure, rules, boundaries. They will test them, which is good.

Oh yeah, totally. That being said, our eldest was super picky and changeable (and often didn't eat enough so it was a little concerning) so we often ended up with lots of staples that she one day decided she didn't like :shrug:

Mostly fixed now that she's older, but I can see how food waste happens. And more generally, if the kids get excited or distracted, there will often be waste.

> But I got bored having to deal with trivialities

For me, at least (with probably half the experience) the trivialities are important for making me understand the design, so I can improve it.

If there's a lot of boilerplate, then we should condense that to something else.

Unfortunately, with the way these models are currently trained, they default to solving the same problem in multiple ways depending on the position of Jupiter and the Moon (apparently).

> I also personally like eating leftovers for both breakfast and a packed lunch, which probably helps a bit.

That definitely helps, I'm pretty bad at eating leftovers unless I put them in a new meal so we end up with more food waste than necessary.

Kids completely changing what they'll eat on a weekly basis also has a pretty large impact.

Yeah, I'm in my 40s and I never understood why people hated inflation so much until 2021/22. I get it now, it's massively disruptive and annoying.

Technically linear regression is statistics rather than ML, but I feel like the GNU/Linux people whenever I point that out.

As long as you can get a reason from the model, it's not that bad.

Black box automatic decision making is much more problematic.

Or Theresa May, perhaps?

If they end up getting heavily regulated, then it's definitely an error. The Fable export controls kerfuffle was definitely downstream of all the hype so it seems to be having some impact.

This particular author/publication has been beating this drum about Elon and his many companies for years now, fwiw.

I just used the keyboard to navigate so the cat didn't move.

I mean, to be fair, basically the only way to raise the amount of capital required to keep growing LLMs was to sell a story at this scale.

I think the real error was the notion that they could sell this story to investors without the rest of the population noticing.

> If you're working for someone who can't tolerate debate, leave.

I dunno about you, but I've left a bunch of jobs (or been left) because I thought the people involved could handle debate, when they couldn't.

Your boss is a human, they have issues that may prevent them from accepting your (clearly better ;) ) solution, or they may have context that they can't share (their boss might be pushing for something dumb).

Ultimately, the hierarchy and the power differentials are real, and unless you're happy jumping ship all the time (which I am not, anymore) you need to find a way to deal with disagreements more productively (which often involves letting people save face by disagreeing in private).

> It's a chief reason California tech companies become dysfunctional.

I think the deeper reason for Cali tech dysfunction is the relentless norm of frankly insane positivity, not the ability for people to debate.

Sortof, much of the financing up to now has been from equity, rather than debt (with the exception of Oracle).

As the funding profile moves more towards debt, there's a bigger chance for problems as leverage tends to amplify the bad outcomes.

If Big AI (basically Mag 7) continue to issue debt, then there may be financial stability risks if it all blows up. The numbers are fine now, but the trend line is concerning (hence the BIS paper).