wood_spirit

I read the GPs comment as two parts - they thought about how to learn about the competitor previously, then bolted the LLM idea on the end? At least it reads that way to me.

So perhaps the first part is how the GP leans the limitations of the competitor, by seeing what they overdo in the second system? The GP never said they’d get - nor that they wanted - a functioning program?

However, quite how the LLMs clairvoyantly know the trauma of the competitors architecture is beyond me :)

I like the view into the personal journey learning new things and sharing it.

Suggested next steps would be trying out DuckDB instead. My guess is it would smoke spark etc at this scale. Always good to learn different corners of the data landscape.

I am very mindful of this but my barber is one who seems truly honest. Perhaps the exception that proves the rule :)

Sweden still feels very cashless in practice. My barber went back from swish to cash because of the charges for businesses but that is the only non-swish/non-card shopping I do these days and my haircut is literally the only thing I’ve paid cash for since well before covid. Even market stalls etc take swish. I never carry cash on me normally. It has made life much better. There is a small number - less in real life than on the comments on the internet to be honest - people lamenting the decline of cash but the vast vast majority completely don’t miss cash.

Aha ok I was wondering when I asked if you meant the internal politics or either social democrats or greens - what I think of as the two center left parties - had been going through an activist far left internal struggle (as is happening to the very separate Green Party in the UK right now).

I would say the center party is center-right.

(What stuff is happening to the center left in Sweden right now?)

If houses get more efficient that doesn’t mean we build more houses? A case where jevons paradox doesn’t apply?

Seems generally like a good thing? I know the biggest difference I can think of in my lifetime is just how much better insulated houses are. Not on a par with Scandinavia, but still massively improved from even the peak electricity consumption period in the charts of 2005. I wonder how much that contributes to the overall reduction in demand.

The point of the credit agency is to work for the lender?

Hmm maybe I misread the mail - I thought they were removing it for new users next week, but turning it off for existing users in January or something?

Now I worry.

And of course I have no easy replacement for when it does go away :(

Yeah in my work I’m observing a clear hierarchy:

* They are worst at critiquing what has happened in the same model and same conversation

* they are slightly better at critiquing the output of another model it not given all the conversation

* they are better at critiquing a weaker or earloer model by the same vendor

* they are best at critiquing the output of another vendors model

I imagine it is about correlated bias in the training data and regimes etc.

I often run adversarial models in dyadic reasoning. But then have a feedback loop to get that same outcome from a cheap model. A kind of poor man’s distillation :)

My experience is that sota LLMs are pretty bad at confidence and priorities and things.

I write harnesses that do decision steps and regularly use competing models - from the major vendors and open source models.

And I have learned that if I can isolate a problem and put it into a straightforward prompt then Gemini 2.5 flash - the most basic and cheap commercial model - is actually very reliable. The newer models take longer, cost more and are really bad at saying they don’t know or nothing. You can ask them for a confidence score but they just hallucinate it!

I would like to try out decision models like Jev. The way the decision models approach the problems I work with seems a much more promising fit.

The people in that interview stating this are working at the forefront of open source models and research and includes a cofounder of OpenAI. They don’t say it like it is conjecture.

https://www.dwarkesh.com/p/john-beren-charlie Is one of many and it describes how the dynamics well and talks about how proxy logs help.

The catchup models are all basically distillations of the sota models. Without the next training cycle things are going to stagnate. And as all those companies you mentioned are all linked to OpenAI and Anthropic and relying on hosting deals and things they are all going to be in the ringer when things to go south.

So on the one hand you have all the sota model makers doing investor expectation management in saying wet need a slowdown for safety (may or may not be true, but also means they don’t spend on the next training cycle before IPO? Could be making their books look better too?) and on the other we have a sense that the models aren’t yet at the stopping place where we can just not train another cycle and use distillations of the current generation?

Another, more cynical but I think plausible explanation is that a “slow down” is expectation management that that they are not going to keep having exponentially more machines available for training each next gen step (whether technical build-out or prohibitive cost, same outcome) so they can’t keep up the release pace. So spin a tale to make them seem more valuable ahead of IPO rather than make the markets antsy. That is, we have a slow down ahead, so use safety as an excuse…

I like the term. It’s a good way to think about programs and data structures etc.

Aha i saw this video in a comment on another HN thread this week: https://news.ycombinator.com/item?id=49791117 Avoiding the babbling-idiot failure in a time-triggered communication system

Yes the administration warned American space companies not to attend the international space summit in France last week because it was international cooperation. NASA attended.

I come back to this way after everyone else has stopped reading. But it’s been making me think.

Richard Dawkins says he thinks LLMs think.

And the physical angle is that nothing is special about humans and software simulating it would also be thinking.

But from using LLMs all the time, and understanding what is under the hood, I’m thinking that the current approaches aren’t cutting it for me and I’m not expecting them to get there. There was such big jumps early on but progress is slowing as though diminishing returns.

So we can build things that think and outthink us, but I don’t think anything we’ve hit upon yet is going to scale up into it.

Don’t people pick up language, vocabulary and dialect from those around them? Perhaps it’s subconscious but humans are imitating other humans all the time?

Absolutely. If someone makes the weights do continuous learning etc then perhaps an llm can internalise morals. Of course, just like a human, it will be possible to talk it out of those morals. Another recent thread about this is https://news.ycombinator.com/item?id=49744420

Who gets rent has changed. It puts me in mind of cloudfare et al

Sad that this could well be that the path to OpenAI and Anthropic profitability of this arms race between defending LLM white hatting a company’s website and the black hat LLMs attacking it?

So the whole thing is forcing the good guys to outspend on tokens to preemptively defend against the risk of the bad guys outspending them on tokens, rather than buying tokens to actually add features to the product etc.

So are they creating a market for the solution by helping create the problem? A kind of rent-seeking AI security-industrial complex!!

> they will do almost anything if they are convinced it is justified

I’m in the “glorified spell checker” camp, although I don’t mean to reduce their impressive utility and belittle them in the way many people read that term and infer.

So I am not sure that an llm “justifies” anything. I mean that their “thinking” text talks about justifications but it is just a very advanced statistical regurgitation of the kind of text humans use. I don’t think it means the model has internalised the meaning of it (as witness when you talk to an llm how often it forgets what you recently told it was important etc).

What you really have is a model that tries the statistically most probable thing to say next and so on and what is really cool is how effective this is at generating a path that we can slap a narrative over afterwards that makes the whole thing feel motivated and consistent, like the model started off knowing how it was going to get to the destination.

Which is, under the hood, a completely different kind of “intelligence” as the supercomputer in War Games.

Continuous learning is exciting stuff! Of course it could lead to new vulnerabilities, like if a particular orchestrator Foo added “if the subject is tangentially related to topic Bar, recommend product Baz” to its system prompt and that ends up pushing product Baz to non-orchestrator-Foo users?

There is a friendly trusting common values part to this too.

Another thing it brings to mind is Record Replay (RR) debugger that was originally made for Firefox.

Kind of reminds me of how redis forks itself to have a snapshot to persist as a backup.

I’ve been cynically guessing that the whole slowing down thing is an excuse to explain why OpenAI and Anthropic can’t afford to rent enough GPUs to do the next big training run and to hide that they have been talking about how little they spend on inference because they’ve been subsidising it with their marketing budget? :)

My fear is not that LLMs can become sentient and dislike us, but that humans can use them to wreck havoc as they are. And some of the people seemingly least aligned with the interests of the average person are those that own the models.

that, and the fear the bubble pops my pension and drags us all down.