Hacker News Reader

> It seems to me that the real question here is what is true human intelligence.

IMHO the main weakness with LLMs is they can’t really reason. They can statistically guess their way to an answer - and they do so surprisingly well I will have to admit - but they can’t really “check” themselves to ensure what they are outputting makes any sense like humans do (most of the time) - hence the hallucinations.

Apparently GPT-4 is getting pretty good at knowing when it's wrong: https://thezvi.substack.com/p/ai-26-fine-tuning-time#%C2%A7g...

(They asked GPT-3.5 and GPT-4 "are you sure" to see if it would change its answer, both when the original answer was right, and when it was wrong)

Does it do that because it can check it’s own reasoning? Or is it just doing so because OpenAI programmed it to not show alternative answers if the probability of the current answer being right is significantly higher than the alternatives?

I don't know. I don't think anyone is directly programming GPT-4 to behave in any way, they're just training it to give the responses they want, and it learns. Something inside it seems to be figuring out some way of representing confidence in its own answers, and reacting in the appropriate way, or perhaps it is checking its own reasoning. I don't think anyone really knows at this point.

As the other poster said, they can check themselves but this requires an iterative process where the output is fed back in as input. Think of LLMs as the output of a human's stream of consciousness: it is intelligent, but has a high chance of being riddled with errors. That's why we iterate on our first thoughts to refine them.

Why do we have to feed the input back? Why doesn’t it do it itself?

Maybe it’s because it can’t tell when it’s wrong and needs to “try again”, and we have to do it for them.

Because that's how LLM chatbots are designed. Those papers describe systems where the review process is automated for better results.

I'm getting suspicious that there is a bit of a blind spot in understanding the world and the usefulness of what we call intelligence, as Noval Yuah Harari says, intelligence is overrated. Look at what we've done to the planet and our environment we have fucked it properly, yet we consider ourselves to be intelligent?

Could it be that intelligence is overrated and discovery of new ideas / thing is underrated? Our egos tell us it's intelligence that makes us special and creative and awesome but maybe most of the special stuff is already there for us to find and we conflate discovery with extrapolation. Maybe knowledge and experience are the "important bits" of intellect.

Example: Einstein didn't really invent anything, he discovered things about the world that blew our mind and changes our lives. Yes he was a great thinker and a courageous soul to go against the grain and he had the balls to be open minded enough to discover new things. We obviously believe Einstein to be intelligent but was he just a great explorer ?

I have a similar attitude towards technological progress, yes we've done amazing things but fundamentally the air we breathe, the water we drink and the beauty we are subjected too when looking at a sunset are taken for granted while we stare at our phones.

What we've done to the planet is perhaps less a consequence intelligence, more a consequence of multi-polar traps. Though it's in our collective self-interest to protect the planet, it's in our individual self interest to ignore the problem / prepare so that our own children can "weather the storm".

It kinda depends on how much you care about people across the world and future generations.

But yeah, if humans were more intelligent, we probably would have sorted all this out by coming up with better coordination mechanisms, and by overcoming our tribal tendencies more effectively.

All of this can be boiled down to the simple maxim: "Stupid is as stupid does."

But, if we were more intelligent, we might have raped the earth much faster and much harder too?

I see this too often, more intelligence = positive outcomes, but no, some of the smartest people ever put their intellect towards stupid causes, such as oil exploration and AI to capture peoples attention.

I hope you're right though and I'm wrong ;)

>One salient question in this is: will we reach a level of intelligence where we become beings capable of actual collaboration that doesn’t waste so much effort in conflicts, or one that is capable of living in harmony within its environment?

That's more about ethics and an wish for moral behavior and conflict aversion, than about intelligence.

Intelligence (human and AI) could just as well opt for conflict and evil, if this helps it get the upper hand for its own private goals and interests.

Simply put, the interests of the collective, are not necessarily the interests of the individual intelligence.

(Even assuming there was a single, easy to agree upon, "interest of the collective" for most problems).

Not sure how being able to reason about good behaviour in an effective manner that's collectively beneficial isn't in the domain of intelligence.

We have moral philosophy as an academic discipline, after all.

Human brains develop in interrelation. Much, if not all of our intelligence gets developed in relation to other humans and beings.

>Not sure how being able to reason about good behaviour in an effective manner that's collectively beneficial isn't in the domain of intelligence.

That's neither here nor there.

Having the intelligence "to reason about good behaviour in an effective manner that's collectively beneficial" doesn't mean you're constrained to reason and act only on that, and not also able to reason and act on behavior which is beneficial to you to the detriment of others and the collective.

And it's pefectly intelligent to follow the latter if you can get away with it, and if the benefit for you is more than your share of the collective benefit alternatives would be.

>We have moral philosophy as an academic discipline, after all.

And how has that been working out for us?

(Not to mention, keyword: academic).

>Human brains develop in interrelation. Much, if not all of our intelligence gets developed in relation to other humans and beings.

Yes, and a lot of it is devoted to duping and getting the upper hand of those other humans and beings. So?