Comment by natureiskino | Hacker News Reader

Comment by natureiskino | Hacker News Reader

Parent Full thread

>Whatever world we construct for LLMs, no matter how detailed we make it, will always be a blocky projection of the real domain onto a virtual one.

I don't know exactly why but I never really understood this argument. Might be some kind of control thing? Because for me it's pretty simple, it's basically free to give access to reality. Just add "sensory organs" as it were. I can argue you can make them perceive reality even better than we (humans) do, just enlarge the audio/video spectrums. Bam...more reality. The whole point of the argument is we're missing information.

Again, I get the need for controlling the environment for what LLM/AI/AGI/whatever will be, but that will always cost more than giving them access to like...reality. Same reason I don't really believe in the whole simulation argument, it's just more expensive all around, loses resolution, let alone control. I don't doubt there will be some people that would indulge in neverending hedonism but not all people. You need to give up control for that.

> Because for me it's pretty simple, it's basically free to give access to reality. Just add "sensory organs" as it were.

I dunno what you mean by "free". The model is trained on text. To "give" the model sensory organs it would need to be trained on those sensory organs.

Current models can predict text, because that's what the weights represent. Models with sensory organs will need to be trained on the output of those sensory organs.

That sounds close to impossible in the foreseeable future.

Vision and audio is already in use in multimodal LLMs. So it's possible in the past.

Who said anything about vision and audio?

Those are like the sensory organs.

> Those are like the sensory organs.

Sure, in much the same way that veggie burgers are "like" beef burgers, and NES Duck Hunt is "like" clay pigeon shooting, or Dirt Rally is "like" driving a real rally race!

But no one is going to claim that being good at Duck Hunt means you know anything about firing a gun, nor winning all the races in Dirt Rally is anything close to driving a real car.

>I dunno what you mean by "free".

Reality is free. You don't have to waste any resources to model it, you just need to capture it.

>The model is trained on text.

See in my previous reply:

>LLM/AI/AGI/whatever will be

LLMs don't even have a sense of time because they work differently to a human brain.

Sense of time: just positional encodings on the tokens. Solved issue.

There are two reasons.

First, reality is continuous whereas the digital world is discrete.

Second, data in the real world is many orders of magnitude more detailed than what we're able to model with today's computers.

Reply on news.ycombinator.com