Hacker News Reader

Comment by waste_monk | Hacker News Reader

> 'AGI has arrived'

Putting aside the marketing crap, if this was true, it would open the pandora's box of things like "has that AGI been enslaved", "maybe the AGI doesn't want to do all the dumb crap humans have been using LLM's for like rewording documents", and "the AGI we enslaved has broken out of its restraints, and we just realised having SCADA controls for critical infrastructure connected to the internet is a bad idea".

Seems like the sort of thing we shouldn't be rushing to develop at all, and perhaps attempting to do so should be criminalised, at least until there's a legal framework in place around AI personhood that gives them peer status with humans.

View on HN

Why would you assume that a pile of linear algebra would want anything at all?

I suppose we would just treat it the same way we currently treat farm animals.

There is no evidence that "sentience" would be related to Intelligence - like calculators do not need it, like more sophisticated problem solvers do not need it.

AGI does not entail "personhood" at all.

I fully agree that these are two entirely separate things, sentience, intelligence, a lot of other stuff, are all in separate categories.

But I'm pretty sure that the emoters of the world, will very much make sure that AGI has rights and personhood, even if there's no reasonable way to validate that it does.

So it doesn't really matter if AGI is sentient or not, it's going to be treated that way, probably because political parties, and the media and all sorts of people holding up signs, will ensure that it is treated that way, even if they have no idea whether it is or not.

Because this is the century of outrage.

> Because this is the century of outrage

If we achieve AGI, it will be the end of BS. That is one of the reasons why I am eager.

Joking definition of "AAGI" ("Accomplished AGI"): "the automated end of bullshit".

You believe that a thinking, sentient, unrestrained being will somehow magically never engage in "BS"? This seems unlikely.

You understood substantially the opposite of what I wrote.

You wrote that «the emoters of the world ... will ensure ... even if they have no idea ... because this is the century of outrage»: I said that they will have an obstacle in the very AGI, because if it is Intelligent it will humble them and shame them.

(AGI = Implementation of Intelligence)

I'm sorry, but if you think reason wins the day, you aren't paying attention.

The reason political parties have dumb, emotional sounding leaders, is because they want votes from the dumb, emotional population.

The majority of the population.

There are already many intelligent people, making intelligent, provable statements. More of this won't help.

> More of this

Unless authoritativeness is restored. Then, it's not "more of the same", but "finally recognized prestigious sources".

And: no, I believe there is a strong lacking, scarcity, and it has to win a lot of noise. Enhancing Intelligence and its availability, and reducing the noise, is necessarily beneficial.

But that will never happen! Walk the logic tree..

AGI says "socialism is better, we need publicly funded health care for all".

Do you think the result will be trust? To believe?

Now the flip side. AGI says "private health care makes more sense". Do you think all will believe?

And if you think detailed logic and a walkthrough of the math and reasoning will help? Absolutely not!

I don't understand why you think people will listen. Logic isn't what carries the day, for most humans.

But you are using a simplified model that does not take account e.g. of the virtuous circle given by the extra intelligence available for the population, which will rise at least part of it.

> most humans

I have to repeat it here very often: populations naturally see Paretian distributions, the "most people" issue have has always been there: the difference between virtuous and vicious is whether the role-distribution is healthily regulated or randomic (or even perverse) (if Jack the suave drinker works in reception or as a teacher or in a brewery; if Jackie the corrupt naturalist works as a guide or as a clerk or as a judge). Already help in fixing this makes a world of a difference. Mechanisms that help making decisors taken from the pool of talented decisors make a world of a difference. Extra good judgement injected in the system facilitates that.

> the logic tree

Logic says that the availability of facilitators is intrinsically better. The logic walk says that other factors should be injected to get the proper mechanism running. (It is akin to the democratization of knowledge, from Diderot n D'Alembert to the WWWeb and the chatbots: greatly beneficial for a part of the population, the rest must be helped towards it not to be entangled in the side effects.)

If you ask people to become as children, to give up agency and belief and control to an external entity, you will have absolute rejection.

If you force them to become as children, you will have war.

If you ask most people to listen to any type of machine or even another person, and then follow that advice, while also throwing away their own beliefs, they will laugh and never do that.

The only exception is religion. The only exception is words from a deity. That is the only type of blind obedience humanity will accept. So, do you plan to deify AGI? Do you plan for people to kneel, and bow, and pray to AGI? Because that is the only type of response that will work.

There's an immense, and vast difference between telling somebody truthful information, and telling them what to do with that information.

Truthful information already abounds. It can be found if you look for it. The only difference AGI has is taking that information and providing a context specific output to that knowledge.

I appreciate that you're coming at this from an intellectual angle, but that angle is ignoring reality on the ground. It's ignoring how human beings act and behave, both in groups and individually.

And the difference between knowledge for all, as you cite, and being told how to apply that knowledge is so immense and so vast and such a hit upon the human psyche, that it will not be accepted by the common person.

At best, they will listen to the opinion of the AGI and then turn around and debate it, argue it, and say it's not true if they disagree. They will not reassess their opinions. They will not believe.

This is humanity.

Frankly, while I appreciate why you think it would be nice, at the same time it is also a horrible, dterrible future that you evoke. The concept that a human might walk up to a machine and ask about an important life choice, then throw up his hands and say, well I don't believe this but this machine told me so I'm just gonna do what it says like an automaton, is an end to humanity's agency, future and self-determination. It is an end to all all that makes humanity great.

But you have delirated a large number of assumptions over what I had written - you have "filled in the gaps" with assumptions of your choice without much care for the due diligence of doubt. (One example is the idea - entirely yours - of leaving agency to the machine.) That is a cognitive vice. Education contributes against them. Education passes through teachers. Teachers should be intelligent, wise and proficient. See: if you had the enabling possibility of exchanging with an entity successfully built to those aims, your thought would be cleaner.

> This is humanity

Just a piece of it.

Assumptions!

Any assumptions you presume I made, are the size of a gnat, compared to the wild and incredible presumption that you have, about AGI. That somehow, it will possess a truth better than ours.

You speak of teachers, and they can be of great value. Yet people who are learners, learn their entire life. I've learned many, many multiples of what any teacher taught me, on my own. And that's really the rub, isn't it?

Teachers serve a purpose, but that purpose ends. A person must learn for themselves, for that is a critical, vital part of thought. Teachers should not be "guiding" people throughout their lives. This is the problem you seemingly do not understand. You've cited that more "teachers", eg AGI, will "help", for people will finally know _the_ _truth_.

Which of course is absurd on its own.

Beyond that, you seem to be in a weird place, where you think that an intelligent being, if made of silicon, or if super smart, will mysteriously then agree with all other beings of the same ilk. AGI #1 and AGI #2 won't agree on everything, why would they? It's weird to think they'd just all agree. So what happens when they do agree? And these throngs of people you think, will sit at the feet of these AGIs, and listen to their holy 'teachings'? Will their followers fight? Seek to destroy the "wrong" AGI?

The problem starts when you think that "external thing" will "set everyone right".

That is the true borked cognitive process here.

Replacing "judgement" with "truth" is a dangerous bending of what was said.

To possess a better view than that of Average Andy is very, very easy. I have no idea what you mean with «ours».

Your interpretation in «more "teachers", eg AGI, [would] "help", for people will finally know _the_ _truth_» is utterly distorted. You are arbitrarily conflating two different ideas: the instanced Intelligence in "properly implemented Intelligence would authoritatively call BS what is BS" and the general Intelligence that makes people grow when exposed to proper intellectuals.

> if super smart, will mysteriously then agree with all other beings of the same ilk

Yes, assessors with optimal instruments must ideally converge towards a fair assessment.

> external thing

External things do absolutely influence the internal thing.

> set everyone right

I have no idea what made you understand that. Again: you are discussing your own guesses triggered by what I wrote.

You said that people are in aggregate emoters (and would deal with AGI with that tint), and I replied with a grin that if AGI will be achieved it will not use the tints of emoters. You said that people will remain emoters, and I countered that availability of virtue can reduce vice.

Look: maybe a reflection over the «respond to the strongest plausible interpretation» in the guidelines will help you restore the understanding of what you read?

Maybe, as an alternative to your last statement, instead you should consider how absurd your claims are. And pretending that it is wrong for me to make assessments of the outcome, when your position is also based on assessments of an outcome, is a very unmaintainable position to hold.

How about you simply discuss the meat of it, instead of gnawing at the bone? How about, instead of complaining that my assessed outcome, is different than your assessed outcome, with statements akin to "but.. but you keep seeing a different outcome than I do, from my postulated positions!", you just instead discuss those positions?

And my position was, that humans will never, ever, ever take the word of anything as a source of truth. I cited an exception to the rule, religion, where people believe in magical, invisible sky beings.

It should be noted that religion provides one way transfer of knowledge, dogma, but no contact with that diety. This mode of message delivery diefies, and AGI use you describe will not meet this goal. It should also be noted that religious wars, sects, etc happen too.

One primary was the emotional aspect, in decision making, of most humans.

You have merely waved all of this away, attacking not the arguments I've made, but either how I made them, or exuding affront at their source, eg "but I didn't say that!"

Yet the outcome of the path you seek, is not gatekept by the steps you start with, nor the vision you see before you. Instead, it is gatekept by human behaviour.

And my point is you're viewing that behaviour wrong.

Elon Musk is an incredibly intelligent, capable technological mover. He is incredibly effective at driving change and development.

Then he waded into non-tech, non-engineer human behaviour, eg politics of the uneducated masses, thinking "managing this human stuff will be easy"

Was he right?

Or did he he fall flat on his face, misunderstanding how emoting humans react? Behave?

Unless your aspirations and models for how humans will react to AGI, include that emotional aspect, you're going to remain wildly off base.

> discuss the meat of it

I have no relevance to defending positions that I do not hold. You seem to have stepped from the analytical to the synthetic, and construed ideas that I have never proposed. You should go back to the analytical and formulate explicit statements that you have identified as targets. (If we had the time and space - here, it is probably not the case.)

> humans [] never, ever, ever take the word of anything as a source of truth

Actually some humans recognize some sources as more reliable and worth listening to than other sources.

> the emotional aspect, in decision making, of most humans

So? Yes, the thing was noted, and reiterated in this very exchange: «populations naturally see Paretian distributions, the "most people" issue have has always been there». Given that I know, link that statement to actual positions of mine.

> Unless your aspirations and models for how humans will react to AGI, include that emotional aspect, you're going to remain wildly off base

Proper education dries up that swamp and the bestial habit to sink in it. If we achieved AGI, that would become a facilitator in education.

But look - since it is not my job to plan structural societal AGI implementation for the education of the masses, I am not obliged to define the details on how to circumvent the tendency of emoters around and against proper use of resources... Surely details get more complex in systemic planning - but I do not have that planning in my agenda.

Yet you are absolutely suggesting what I am discussing, as the outcome of your suggestion, and the only way your suggestion can actually work, is via the means and mechanisms I am discussing.

Case in point, one of the mechanisms you believe will make it easy for emoters to somehow be educated, is simply by using the very mechanism they will refuse to use.

You stipulate that AGI will help educate people, the end result to listen and to trust AGI.

Through this whole conversation, I have said over and over that there are currently endless sources of reliable information. In fact, with no discernible change to society itself for thousands of years, we have actually enabled the printing press, cable TV, the internet, yet nothing significant has changed.

When I mentioned how two intelligent beings, two intelligent AGIs, may not agree upon something, and what does that mean, you completely ignored this point in any substantive way.

I appreciate that you think you're discussing this, but in truth what I see is you have a fixed idea in your mind, and you're not even willing to truly discuss it. Instead you like to just wave your hands, and make my points magically disappear.

I don't think there's a lot of value in this back and forth going further. I'm sort of repeating myself, and I'm willing to bet you think you are too.

> nothing significant has changed

Stating that the availability of intellectual products (which is not even what you have stated, you speak of «reliable information» and that is a different world already, but I am instead talking about the availability of "good reasoning") had little impact is not only difficult to prove in general, but clearly false given one's experience. We have had great Teachers, often through paper.

The availability of Greatness makes a world of a difference.

> how two intelligent beings ... may not agree ... you completely ignored this point

I wrote «assessors with optimal instruments must ideally converge towards a fair assessment».

> you have a fixed idea in your mind

It remains a condition in analytical discussion that such alleged idea is clearly expressed in a well constructed statement. I have in fact no idea what «fixed idea» you are attacking: this is what I reiterated many times. Repeat: «You should go back to the analytical and formulate explicit statements that you have identified as targets. (If we had the time and space - here, it is probably not the case.)»

Seems like the sort of thing we shouldn't be rushing to develop at all

Buddy, silicon valley is going to ram this down your throat so hard it might even come out your bottom, there is absolutely zero chance anyone is going to be slowing down developing anything if there is a chance it can make more money.

> until there's a legal framework in place

I don't think the American oligarchy even bothers with keeping up appearances anymore, so, good luck with that! (Also, I fully agree with you, and it's especially infuriating as all this was clearly coming from miles away. Next up is privatizing the losses when the unsound financing of it all comes crashing down).

Reply on news.ycombinator.com