Hacker News Reader

Comment by famouswaffles | Hacker News Reader

NLP is solved, more or less. Either way, Bespoke NLP is on its way out. It's pretty funny how buried this is in the original paper.

View on HN

Parts of NLP have made great progress. There are still parts of NLP that could still be improved, such as the truthiness of generated answers.

It is true, however, that the problem has been solved completely in the human-to-machine direction. The output of the current-generation LLMs is completely off base in many cases, but they certainly understand what they are being asked, for any useful definition of 'understand.'

I'm much more impressed by GPT's ability to handle input than I am in its ability to generate output. It's arguably as good at reading comprehension as most humans.

But that's not an NLP problem at heart. Language is just a collections of tokens (words, letter) that are tied together by certain rules to convey some meaning. There is no concept of reality per se.

For example, consider filling the blank:

A giant ______ flew over my head!

It can be a plane. Or a dragon. Or an UFO. Or a balloon. The thing is all of those are correct answers language-wise and the model works correctly as long as what gets filled in conforms to the rules of the given language.

The language that we generate encodes reality to some extent and the model picks up those correlations but there is no concept of reasoning or reality behind it. Maybe it is emergent at some point (as to effectively compress it needs to encode some subset of rules governing our reality) but it is not an agent that optimizes for understanding our reality. Something like Dreamer would be much closer to that.

Sorry to get heavy here: truth is not an NLP problem, it's an alignment problem. We want truth, but we don't have a reliable way to train an AI to provide the truth, only to provide things that are either true, or sound true enough that they fool the reward function. And even then, that may not be exactly what the AI learns to do, because of there's another level of alignment problem, the "inner alignment" or "mesa-optimizer alignment" problem!

With an AI like GPT, it is quirky and amusing. Once AIs get really powerful, it becomes scary, and a lot of people who understand this field much better than I do are worried it has a good chance of being deadly. Like, potentially kill-everyone-on-earth deadly.

Hard agree. I'm really trying to figure out how to inject this idea into my friends' heads effectively. The main struggle I'm facing is how to convey the danger behind it. Why can it be deadly exactly? What can a program actually do to harm people, to the level where it's a risk of extinction or societal collapse?

Personally I didn't need to imagine a specific scenario to understand that there's risk, but I think it would help me convince other folks if I did.

> risk of [...] or societal collapse

If you want society to collapse all you need to do is succeed in having AI automate all jobs.

Every single country where money comes from somewhere other than people (oil, diamonds...) is an authoritarian nightmare simply because keeping people happy is not necessary.

Once AI can do everything and robots that can do any physical labor are developed the population will shrink dramatically as people with killer robots kill each other for resources. There is no need for AI rebellion or AI failure to get there.

“Just happened”

Do we get hoverboards now or is that later ?

Reply on news.ycombinator.com