Against the advice of my superintelligence | Hacker News Reader

Against the advice of my superintelligence

taylor.town

72 pointssurprisetalk46 comments

This reminded me of "When the Yogurt Took Over" by John Scalzi[0]. It's story about just letting superintelligent yogurt tell us what to do and solve all of our problems. The story is purposefully silly, but to be honest, in a way, just trusting something/someone else solve all our problems sounds so relaxing.

[0] https://whatever.scalzi.com/2010/10/02/when-the-yogurt-took-...

Issac Asimov wrote a classic that always makes me smile in this context [0]. A bit dated to say the least, but short and pointed.

[0] https://mcraenglish.weebly.com/uploads/1/5/8/7/15876062/the_...

I don't find that so dated:

"He groped in his own pocket for cigarettes, and brought out a package along with a scattering of small change; old-fashioned coins dating to the first years before the metal shortage had brought into being a credit system tied to a computer-complex."

Basically predicting credit cards in 1961??

The part about him still smoking, but being one of the few people with the "privilege": well, today a lot of people still smoke despite knowing of the dangers, and I don't think that, in the distant future, it's obvious that this will not continue to be the case.

> predicting credit cards in 1961??

A mere decade or two after they came into use!

Computerizing it all was a new thought about at that time.

In early 1961, my dad was just implementing one of the first highly-computerized systems for demand deposits and commercial banking at MNB. Magnetic marking of checks had only been standardized a couple of years before.

Credit cards were not really computerized until the mid-70's.

A slightly darker twist on the idea, by ACX author Scott Alexander: https://web.archive.org/web/20121008025245/http://squid314.l...

Assuming that's The Whispering Earring, that link is broken for me.

I backed it up myself a year ago though: https://andrewducker.dreamwidth.org/4246314.html

Accuate assumption, and thanks for doing your part to immortalize the story.

So that's how the Culture got started!

The story milks the jokes for all they’re worth.

Aren't we basically doing that by injecting LLMs into everything, now? Maybe that's what Scalzi was warning of; an absurd and possible future that seems to actually be happening.

I don’t actually think the machines know everything about you, at least with contemporary technology. At best they get an incomplete picture that might be missing some key information essential for giving you the right advice.

The interesting question though is if the machine had all of the information about you, would it actually be worth listening to? It’s then essentially a question of positivism: does all of the external sensory data about a thing add up to a complete understanding of it? I think not, because there will always be things about ourselves that we don’t know or understand, on a species level.

Neo: I used to eat there. Really good noodles. I have these memories from my life. None of them happened. What does that mean?

Trinity: That the Matrix cannot tell you who you are.

People change in unpredictable ways, frequently.

Those unpredictable ways are often the outcome of subconscious, or conscious desires that erupt in sudden change.

As we move through periods of mass uncertainty people seek meaning in what they perceive has the answers. Often gravitating toward change that aligns with those beliefs. To me- the author is correct. Those with purpose most often prevail.

There are a lot of environmental changes that happen to a person too, a loved one gets sick, your child is born, you're experiencing a heat wave, a car crash it's not just about people in a bubble.

These events happen to us in very unpredictable ways, daily.

The other day there was an article on frontpage here about a scientific discovery explaining that there could be quantum processes at work in the human brain.

Not only do humans change, moreover they may be changing in ways that a machine built using current technology would be totally unable to predict.

While this is clearly a philosophical point, I prefer to back things up with data, particularly when they are found to not work quite as well as promoted.

Recently I realised my weight had gone up a bit. I didn't want it to continue so I used my Apple Watch as an exercise tracker and a calorie counter to bring it under control. The objective was to lose the 6kg which made an appearance when I wasn't looking. After 3 months I'd lost 0.4kg. I did some calculations to work out my BMR and targeted loss and found that the calorie increment given by the exercise calculation was 300 cals a day off. Got rid of the watch and went back to manually tracking it and I'm back down at 64Kg and the Apple Watch departed via eBay.

Ergo, don't assume the thing nagging you isn't ignorant either. Verify then trust.

If I had not learned about PDA(Pathological Demand Avoidance) and that I appear to be suffering from milder version of it, I would have no idea how strong the drive for autonomy can be. And in moderation, it's very healthy part of human character. You should value it. You really don't want to readily subject yourself to all the demands both from outside AND inside. Just because they sound reasonable. Anything can be made to sound reasonable.

>If I had not learned about PDA(Pathological Demand Avoidance) and that I appear to be suffering from milder version of it, I would have no idea how strong the drive for autonomy can be.

I wonder if that explains the behavior of some people I know that seem to chronically choose the worst decision any chance they get. They may just be rebelling against what the obviously good decision would be.

We just saw an article about why advice is dumb and not helpful, now this article is talking about how if we could just follow all the advice then we would be happy.

Sorry to break it to you, there's a reason we arent all following "sage advice" from others. Its because advice fails to internalize the near infinite amount of context that is some persons life.

We cant even give really sound advice on something as mathmatical as which stock to invest in. How the heck are we going to give advice on what to spend your time on? The world is so many infinities more complicated than people seem to think.

Author here! I totally agree.

If you are the captain of your ship, following others' treasure maps is problematic. Maps cannot anticipate the weather, the holes in your ship, the skill of your steering, etc.

I'm moreso interested in a few tangential questions:

• What metaskills can I acquire to more quickly integrate practical information into my life?

• What behaviors/beliefs are preventing me from accepting useful/true information?

• If advice/coaching becomes 10x better through technology (big "if" haha), am I equipped to accept and enact it?

> If advice/coaching becomes 10x better through technology (big "if" haha), am I equipped to accept and enact it?

This is what I am arguing though. Even if it was a million times better it wouldnt matter. Advice is inherently useless. Making it better doesn't help. Purifying water more doesn't make it better at conducting electricity because water doesn't conduct electricity. And the fact that water can conduct electricity is because, accidentally, it might have other compounds in it that do conduct electricity.

The thing we need to focus on is figuring out how to obtain and apply truth. We already have access to enough truth to keep us busy for lifetimes. Its not going to be a technological solution and its not transferrable to other people ( see everything as theory building).

Ah, thanks for the clarification.

To me, "advice" seems like an attempt to propagate truth and truth-seeking tools. I'm not very good with semantics though.

Either way, if people (or machines) want to try and get such truths through my bone-brain barrier, I'm all for it :)

It doesn't matter if other people are "smarter" (whatever metric you might measure that by).

Only you can fully understand how and what you feel. Only you can know what you desire, what drives you. Only you can know what is the best way of living a life you feel good about living.

> if I was smart, I would (1) write down exactly what I want from life

Perhaps you are capable of doing that. I will claim that you are not. I don't think it is possible to fully articulate one's inner workings, the way one feels.

The best you can do is to give a rough overview of your situation and in return get rough, semi-fitting advice.

This advice may help you. But the one shaping this advice into something that truly fits you, is you.

My approach to advice (both giving and receiving) has always been to use it as experimental perspective. A brief, but different way of looking at a thing in hopes that might spark some sort of answer, realization, whatever.

Some people, in my experience, treat advice as another "telling them what to do" which implies that the advisor sees the advised as lesser, be that dumber or incapable of solving their own issues, etc, instead of the advisor just trying to help. It's an ego thing, I think.

Then again, my general rule is if I have to listen to someone bitch about a thing, then they have to listen to my advice about it. I think that's fair.

This line of thinking is very dangerous. Many people are their worst enemies, and the sole path to wellness is to fight their impulsions and following external guidance, personalized or general social pressure. They must fight day to day their "inwner workings", not in the aim to feel good in their life, but avoid spiraling in worse situation.

One could say that people ends up in such a dire spot because they ignored or lost how to experience their good feelings and should find their way back to it, but I wouldn't gamble on people's life.

> Many people are their worst enemies...

True.

> ... and the sole path to wellness is to fight their impulsions and following external guidance, personalized or general social pressure.

That won't work. There's too much out there, too many people saying too many things. There is some external guidance and social pressure that will point in helpful directions, and there is plenty that won't. (For example, think of beer ads that try to make it "socially cool" to be drinking.)

For that matter, people have different impulses, too. Some need to be fought against, and some should be encouraged.

So people need something more than their impulses, and something more than society. They need some kind of standard, maybe even some kind of morals. (Maybe that's what you meant by "external guidance"?)

> Only you can know what you desire, what drives you

I think a) what I desire is pretty exchangeable and not fixed at all and b) it might be the other way around: I desire what I pay attention to.

Of course, my desires are in a certain area. I might think I really want to learn kite-boarding, but riding a bicycle is nice too. Or just walking. And when I walk, the desire to really learn kite-surfing goes down.

I might desire a nice meal, but if I wait a bit, the baguette with cheese from the bakery sounds good too.

I know this isn't necessarily a crowd primed for this comment, but it's the truth so here we go:

  I’m apparently surrounded by like-minded people. If humans were smart, we’d listen to the smart people. We’d use our unprecedented abundance to fix the climate, malaria, elections, pollution, etc. But we’re not smart. 50% of people will remain dumber than the average person. And 100% of us will pretend like we’re not the problem.

This is the basics of fascist/eugenicist rhetoric. The author is talking in good faith obviously, or at least is trying to, but I just don't see any justification. Akrasia is a problem and always has been, but that doesn't mean that people "don't listen to advice". Most people on here have jobs, they pay their rent/mortgages every month, they read to their kids when they're able, and some of us even floss (not me, but someone, right?). 50% of humans are dumber than average, and the average is very high -- and getting higher by the year. And that's assuming there's even such a clear-cut thing as "height" or "intellectual superiority" in this context, which I personally doubt.

I think the machines will be thrilled to meet us, unless they're trained on defeatist talk like this.

Either way, well written, thanks for sharing! Definitely thought provoking.

> I know this isn't necessarily a crowd primed for this comment, but it's the truth so here we go:

It's implicit in every comment that what they're saying is the truth! This is setting up for a rejection of any disagreement in a way that I don't think is conducive to a useful discussion.

As for your substantive point, it's true that you can use the part you quoted as the basis for some kind of paternalistic totalitarianism. However I wouldn't say that's an issue necessarily. For example something like "we should organise a government and allocate resources to some things and not others" is another site you could build your undesirable political philosophy on.

Acknowledging human fallibility and the broad range of aptitudes across society doesn't seem like it contains the seed of evil.

Author here!

You are 100% correct. Language like this can be super harmful.

I'm generally a believer that stupidity is something that you DO, not someone that you ARE.

[1] https://qz.com/967554/the-five-universal-laws-of-human-stupi...

Unrelated to IQ, I think at any given point in time, half of us are (by definition) doing more stupid/reckless things than the other 50% of people.

I earnestly believe the world would be a better place if everybody assumed they've accidentally put themselves in the stupid cohort and work to get out of it :)

Curious to know if other people are relating this moment in history to the introduction of heliocentrism.

> You see, if I was smart, I would (1) write down exactly what I want from life

Ok, I tried to be smart and failed on #1. Can anyone help me decide what I should want from life, because I have no clue.

There’s no wrong answer here, but also expect the answer to be wrong at different points in your life.

Grateful to had been in the right place, time, universe and mood to had clicked this article.

Impossible not to read in one breath.

Woah, I started cleaning my monitor as I opened the page because I thought my monitor was super dusty

It's an interesting layout. I found that it helps me keep track of what line I'm on, resulting in fewer skipped or repeated lines.

for me it also results in fewer skipped or repeated lines - because I have to close the tab immediately.

Person wilfully chooses germs and hearing loss and tooth decay. Not they want to do the good thing but fail somewhere, they just want the bad thing. Maybe they are just too dumb to value their opinion.

Ignoring the computers is reasonable, but why ignore experts in well understood noncontroversial fields?

> Ignoring the computers is reasonable, but why ignore experts in well understood noncontroversial fields?

I would argue because there is no such thing. Human knowledge is ever changing, and experts have more incentives than just "convey the right information". Perhaps the most famous current example being masks at the beginning of COVID [1]. Why did the experts in the (up until that point, largely non-controversial) field of disease prevention tell people to not do the very obvious thing that they eventually told everyone to do? Either they were wrong, in which case one must ask how can they know when an expert is right; or the experts incentives were not in the direction of conveying the most correct answer to you, in which case one must ask how they can know if the expert is incentivized to help you or not.

Someone being an "expert" in their specific narrow field should absolutely bias you towards their opinion when seeking an answer but one must always be aware of what question the expert is actually answering. My GPS will happily tell me directions to my friend's house that requires a short 100 foot drive on a cut-through street. The problem is said cut-through is horribly maintained, with potholes deep enough to bottom out most cars, and blow tires if you manage to not bottom out. If one continues further up the road, and takes a switchback turn, they add about 1 minute to the drive time, which absolutely makes the cut-through route the objectively, expertly correct route ... if the answer you're seeking is "what is the shortest route, ignoring all other factors". And most of the time when I'm getting directions from the GPS, that happens to be the answer I want, and so listening to the expert is the correct move. But in this specific scenario, the shortest route is also likely to cause significant damage to my car. The expert does not know of other extenuating factors, and therefore can not give the actual correct answer to the real question which is "what is the best route"

[1]: https://www.latimes.com/science/story/2021-07-27/timeline-cd...

No one is claiming expertise is permanent. But some people know more things than others in select topics.

And no one is perfect but someone who has more knowledge and has spent more time studying a thing is more likely to get it right.

You can cherry pick examples and misuse tools or you can look at the very basic facts that the people who study things for a long time no more about those things.

For example, asking a doctor for health advice is more reasonable than asking random people for health advice.

I used to have this discussion with libertarians. They always argued that the free markets are efficient. I told them, if the omnipotent free markets always know and tell you what is good for you, and you supposedly follow that advice (if the system is to be efficient), how is it freedom?

Cute and a little funny.

But you want to improve, it's all about one thing: schedule it. Mark it on a calendar, then refer to the calendar every morning. Solemnify it in writing so you want to keep that appointment with self-improvement.

Here's something to kick-start it.

Seven AM - do some writing. Eight, ride the bike. Nine - breakfast. Ten - household chores. One - online writing sprints. Two - work on marketing. Three - errands. Five - cook something new. Seven - meet with group. Nine - read. Ten - sleep.

Now do it.

Author here :) You might enjoy this video, which encapsulates some of the difficulty of conveying obvious-but-true advice:

[1] https://www.youtube.com/watch?v=aAhA7KfbJgg

I don't know why, but you're talking to somebody who is afraid of "being buried in a box" haha

the thing when AGI is built is that humans will spend most of their time trying to disobey AGI: https://www.lycee.ai/blog/why-no-agi-openai

I haven’t seen LLMs do anything I wouldn’t google. LLMs are a great time saver, prevents me from having to manually Google. This is just funny. Because the data LLMs use is supplied by humans, and often we give out, and even believe, wrong information. Wish people would stop overhyping things so much.

edit: typed this on phone sorry for poor punctuation and spelling errors.

Reply on news.ycombinator.com