Comment by avgcorrection | Hacker News Reader

Comment by avgcorrection | Hacker News Reader

> Over 1,000 researchers, technologists, and public figures have already signed the letter.

Embarrassing.

> > Should we let machines flood our information channels with propaganda and untruth?

Already been a problem for a century. The root problem is letting the same people turn the volume to 11.

The problem is who owns the means of technology (and therefore AI).

> > Should we automate away all the jobs, including the fulfilling ones?

This is silly on its face. Automating jobs is a problem? More leisure is a problem?

If you want to still do those former jobs (now hobbies) then you are free to. Just like you can ride a horse, still.

Of course I’m being obtuse on purpose. The real root fear is about other people (the people who own the means of technology) making most of us obsolete and then most-of-us not being able to support ourselves. But why can’t we support ourselves with AI, the thing that automated us away…? because the technology has been hoarded by the owners of technology.

So you can’t say that the root issue is about automation if you end up starving (or else the robots could feed you)… clearly that’s not it.

What’s rich is that technologists and other professionals have been the handmaidens of advanced alienation, automation, and (indirectly) concentration of power and wealth. Politically uncritically. And when they fear that all of this is cranked to 11 they are unable to formulate a critique in political terms because they are political idiots.

> > Should we develop nonhuman minds

The embarrassing part. Preaching a religion under professional pretenses.

> > > Should we develop nonhuman minds

> The embarrassing part. Preaching a religion under professional pretenses.

There are quite a few companies with the explicitly stated goal of developing AGI. You can debate whether or not it's possible as that is an open question but it certainly seems relevant to ask "should we even try?", especially in light of recent developments.

There could be quite-a-few companies whose mission statement involved summoning the Anti-Christ and it would still just be religion.

That analogy is nonsense. We're actually making measurable gains in AI and the pace of progress is only increasing. Belief in this isn't based on faith, it is based on empirical evidence.

Elevator- and construction-technology has been improving for centuries. We will surely make a Tower of Babel that reaches the top of the Sky—it’s only a question of when.

Are you suggesting that artificial minds are unattainable in principle, or that the much-hyped language models are elevators for a task that requires rockets?

I suspect that avgcorrection is suggesting that we don't have enough evidence to know whether either is the case.

More the latter.

I’m agnostic on the whether artificial minds can be made in principle and very much against people calling their chatbots for “minds”.

Another hyperbolic analogy. What a great argument, I'm convinced.

Not as hyperbolic as the GAI true believers.

The Anti-Christ is not real. OPT social engineering humans and GPT4 reverse engineering an assembly binary are.

>"But why can’t we support ourselves with AI, the thing that automated us away…? because the technology has been hoarded by the owners of technology."

The problem I think is that if for some reason AI "decides" not to support us any longer we will have lost the ability to do it ourselves. We are getting dumb I think. Obviously I have no data but I feel that back in say 70-90s wide public's ability to think and analyze was much better then now.

So yes do my job and feed me but make sure I do not loose some vital skills in the process.

That was addressed in my last point (“religion”).

To me it does not look addressed at all.

No it wasn't. You have the misconception that an AI system has to achieve anthropomorphic qualities to become dangerous. ML algorithms are goal-oriented, they are optimized to maximize a specific goal. If that goal is not aligned with what a group of people want then it can become a danger to them.

You seem to be dismissing the entire problem of AI alignment due to some people's belief/desire for LLMs to assume a human persona. Those people are uninformed and not the ones seriously thinking about this problem so you shouldn't use their position as a straw man.

> You have the misconception that an AI system has to achieve anthropomorphic qualities to become dangerous.

More than that. It has to achieve super-human abilities.[1] And it might also need to become self-improving, because how else could it spiral out of control, as the hysterics often complain about? (Remember: human intelligence is not self-improving.) It also needs to gain an embodiment that reaches further than server rooms. How does it do that? I guess it social-engineers the humans and then… Some extra steps? And then it gets more and more spooky.

This is more science fiction than technology.

[1] Or else it is no more dangerous than a human, and thus irrational to worry about (relative to what humans do to each other on a daily basis).

> ML algorithms are goal-oriented, they are optimized to maximize a specific goal.

The Paper Clip Machine. I’m more worried about the Profit Making Machine, Capitalism, since it is doing real harm today and not potentially next month/next year/next decade. (Oh yeah, those Paper Clip Machine philosophers kind of missed the last two hundred years of the real-life Paper Clip Machine… but what would the handmaidens of such a monstrosity have against that, to be fair.)

> Already been a problem for a century

Propaganda thrives on scale. It is only recently, as much as around 2014 that SM tools were available to scale propaganda cheaply and by non-state actors.

Reply on news.ycombinator.com