Comment by rvense | Hacker News Reader

Comment by rvense | Hacker News Reader

I think the point about language being a model of reality was interesting. I have an MA in linguistics including some NLP from about a decade ago and was looking at a career in academic NLP. I ultimately left to become a programmer because (of life circumstances and the fact that) I didn't see much of a future for the field, precisely because it was ignoring the (to me) obvious issues of written language bias, ignorance of multi-modality and situatedness etc. that are brought up in this post.

All of these results are very interesting, but I'm not really feeling like we've been proved wrong yet. There is a big question of scalability here, at least as far as the goal of AGI goes, which the author also admits:

> Of course everyday language stands in a woolier relation to sheep, pine cones, desire and quarks than the formal language of chess moves stands in relation to chess moves, and the patterns are far more complex. Modality, uncertainty, vagueness and other complexities enter but the isomorphism between world and language is there, even if inexact.

This woolly relation between language and reality is well-known. It has been studied in various ways in linguistics and the philosophy of language, for instance by Frege and not least Foucault and everything after. I also think many modern linguistic schools take a very different view of "uncertainty and vagueness" than I sense in the author here, but they are obviously writing for non-specialist audience and trying not to dwell on this subject here.

My point is, when making and evaluating these NLP methods and the tools they are used to construct, it is extremely important to understand that language models social realities rather than any single physical one. It seems to me all too easy, coming from formal grammar or pure stats or computer science, to rush into these things with naive assumptions about what words are or how they mean things to people. I dread to think what will happen if we base our future society on tools made in that way.

My point is, when making and evaluating these NLP methods and the tools they are used to construct, it is extremely important to understand that language models social realities rather than any single physical one.

I'd claim actual involves language both a general shared reality and quite concrete and specific discussions of single physical and logic facts/models. Some portion of language certainl looks mostly like a stream of associations. But within it is also are references to physical reality and world model and the two are complexly intertwined (making logical sense is akin to a "parity check" - you can for a while without it but then you have to look at the whole to get it). I believe one can see this in a GPT paragraph, where the first two sentences seem intelligence, well written but the third sentence contradicts the first two sentences sufficiently one's mind isn't sure "what's being said" (and even here, our "need for logic" might be loose enough that we only notice the "senselessness" after the third logical error).

In human language I think physical reality is always a few layers out. Language is social, first and foremost, and naming something is not neutral. We can hardly refer to single objects directly, we mostly do it through their class membership, which always, always include a whole range of associations, reductions, metaphor, etc. that are cultural.

In human language I think physical reality is always a few layers out.

Yes, the point is you can neglect physical (and logic) reality for a while in a stream of sentences. But not forever and that's where current NLP's output has it's limits. Just a simple level, a stream of glittering adjectives can be added to a thing and just add up to "desirable" unless those adjectives "go over threshold" and contradict each other and then the description can get tagged by the brain as a bit senseless.

Are you a NLP bot?

Oh Gawd, I suppose that quip's topical.

It seems like at this point, there's no way to distinguish the coherent comments of a bot from a person. The bot could have written sentence X for just about any X. It's just that bots can't sustain stream of logical claims that are consistent with each other.

So it's easier to demonstrate a paragraph is done by a bot, ie, that a paragraph makes no sense, than it is demonstrate that paragraph is not done by a bot - since both humans and bots write some sensible paragraphs.

Still, I'm egotistic enough to think a bot couldn't come up with that argument though I could be wrong.

Much as I'm assuming the person you're responding to was joking, I've encountered a number of comments/commenters where I felt the same way I felt about GPT output.

The best way I can describe the feeling is that it reminds me of conversations and friendships I've had with schizophrenics, people in the process of having a psychotic breakdown, and people with alzheimers.

There's a feeling that what they're saying is not entirely non-sensical, a feeling of 'catching up' to what they're trying to say (akin to translating in a language one isn't too proficient in). But reflecting on the conversation, I find myself wondering how much I managed to understand what they were trying to convey, and how much it was just my brain trying to make sense of something that ultimately doesn't.

'Understanding' or 'communication' aside, I've often valued these kinds of conversations because they tickle the more free-associative side of my own thinking, and the results, however I/we got there, were useful to me.

As a result, I'm much more interested in how these developments in 'AI' might augment this creative process than I am in how they might convincingly appear human. Not that the latter isn't interesting too, though.

These tools seem to be getting pretty good at fiction. In particular, playing around with AI Dungeon, it doesn't believe anything, or alternately you could say it believes everything at once. It's similar to a library that contains books by different authors, some fact, some fiction. Contradictions don't matter. Only local consistency matters, and even then, not that much.

Unfortunately, many people want to believe that they are being understood. But, on the bright side, this stuff seems artistically useful? Entertainment is a big business.

> many people want to believe that they are being understood

Not only that, they also want to understand. I read some computer poetry a few years ago that worked. The computer did not intend any meaning, but I found it anyway.

And of course I did. This is how language works. We assume that people (and by extension, the texts they write) follow a set of rules which structures how we engage with them in a successful manner. Paul Grice formulated this as the cooperative principle in his study of everyday language, but I believe it is the exact thing that is at play when people meet NLP:

https://en.wikipedia.org/wiki/Cooperative_principle

It's probably worse than that.

It's a model of reality as filtered through the human brain, with all of its neuroses, most of which we have only begun to model.

Aren't we stuck on things like sarcasm? How are you going to model everything from confirmation bias to undisclosed PTSD, in order to have a prayer of filtering the noise from actionable information?

> it is extremely important to understand that language models social realities rather than any single physical one...

Oh gosh yes. Gorge the ML tool with the right streams of text and you'd have a glib anti-vax, white supremacist, flat-earth generator pouring out endless logorrheic twaddle.

Oh wait... could that already... it would explain so much...

Reply on news.ycombinator.com