Hacker News Reader

Comment by pvab3 | Hacker News Reader

Those people have a lot of overlap with the LessWrong crowd. They do not have RSI now and probably never will

They absolutely do, unless you believe they are lying about the fact they're using current generation models extensively to develop the next generation of their models.

The letter S stands for "Self" and word "using" is for sure not a superset of the word "self". Basically LLMs are assisting someone who does improving of said LLM, while RSI is a carpal... ahem, RSI is "self" improvement, meaning no intemediary in a human form. PS: it's also not recursive but iterative improvement, even if it ever happens.

At which point is the human using the model as a tool, and at which point is the model using human as a tool?

I can already see the border shift even for mundane tasks I have Claude working on. Increasingly, I'm just setting a high-level goal, and then checking progress and occasionally answering questions or doing something like configuring a system Claude can't easily reach itself (e.g. recording a bunch of traces through my normal use of a system that Claude deemed too fragile to risk operating on its own). Of course, I get detailed instructions to help me - "go there, do this and that, then press this to capture recording, run through this script here to process, attach result to next message". In those cases, Claude is effectively using me as a tool to call.

"improvement" is the word Id put the most emphasis on.

weve seen some improvement from the LLMs unattended, maybe, but will it actually keep improving vs needing a human to bring it back on track?

the recursive part is that it keeps improving on itself, but we really have no example of that. if it does it 30 times with improvements, then maybe, but even then, to actually be relevant it has to do better than paying scientists to do the work for the same cost, consistently.

RSI still means nothing if it costs 1000x the cost to get the same improvements as a human researcher

The physical constraint is money - which could be said anything. Could vehicle factories be more automated if we threw a gazillion dollars at it? Sure. Would it make economic sense? No. ROIC would be disatrous. No investor wants part of that.

This is the nuance that poster doesn’t understand. Given how much money thrown at it - we’re not even close. Who has the appetite to keep throwing more given they continually need to keep raising fresh money?

> The physical constraint is money - which could be said anything.

Money is fake and not a constraint, that's literally the point of capital investments you guys are overindexing on so badly.

> Could vehicle factories be more automated if we threw a gazillion dollars at it?

They already are automated as much as it makes sense. Some of the automation is silicon-and-steel based, some of it is protein-based. Car manufacturers aren't in the business of pushing robotics and nanotechnology, so they prefer to hire protein automatons instead of developing and building their own, so yeah, they "pay people", but think what exactly they are paying them for.

Then consider that this is very much the work the AI is gradually getting as good as, or better, than us.

> This is the nuance that poster doesn’t understand. Given how much money thrown at it - we’re not even close.

What you seem to be missing is that "investing in AI" isn't investing in a chatbot, it's investing in technology that will (and already partially is) sit upstream of every other industry, of everything humans do. Like electricity or the Internet itself.

(The other thing you seem to not understand, in contrast with some of the investors, is that RSI is not a linear walk, it's an exponential curve. X-risk notwithstanding, by the time it's obvious to everyone, it's too late to make money investing in it.)

> keep throwing more given they continually need to keep raising fresh money

Have you heard about R&D?

I'm starting to think that "investors first" thinking that's so common here, that makes people feel they're smart, is actually quite backwards, especially at this scale. Or maybe it's simply people starting with a conclusion and trying to fit reality to match it, no matter how clear of a nonsense that conclusion is?

> if it does it 30 times with improvements, then maybe, but even then, to actually be relevant it has to do better than paying scientists to do the work for the same cost, consistently.

Not necessary. Scientists are capacity limited and supply limited.

> RSI still means nothing if it costs 1000x the cost to get the same improvements as a human researcher

It means you can replace a human researcher with 1000x their salary burned on electricity. It also mean you can get two of them for 2000x of salary of one,

That's a bargain, actually, even with the anomalous, absurdly-overinflated salaries in top-tier ML.

I you knew it was consistent, then these companies would immediately fire their scientists, and burn 10 000x as much as mean researcher salary this month to be able to 10x their virtual headcount overnight, and then use that to make the 1000x be 500x, then 250x, then 125x, then ... and at that point they'd had all the money in the world, because even more skeptical investors would notice what's going on there.

TL;DR: what you all seem to miss is that electricity scales better than people.

If I'm learning a human language I can do it in many different ways. But the key distinction would be, if for example I'm using only books to study by myself, or using only some recorded course on the internet - it would be called self-study. But if for example I will go to a class teaching that language, or to a private teacher, or to summer camp, or join online platform pairing students teachers etc. then neither of those will count as self-study.

Reply on news.ycombinator.com