Loquebantur

Be cautious what you wish for. Tools don't tell you what to do with them.

Open source LLMs "democratize" access to the "intelligence booster" that is AI. But while that has several benefits, it also has several downsides.

Humanity has the serious problem of being underdeveloped in the "spiritual" department. Ethics is often considered some sort of lifestyle choice, but it's actually the difference between order and chaos in a society.

Everybody being able to do anything means somebody will be able to do something you don't like. At an arbitrary scale.

Given that "sovereign" these days implicitly means independence from USA and China, and Canada being spiritually in the same boat as the EU, it's pretty spot on?

The costs of "keeping up" aren't really growing, on the contrary.

You claim to have the same goal as the protagonist of that article, yet try to shoot him down.

He does give information, namely the culture there factually being inconducive to self-regulation.

You accuse the guy of "bias", but you never argue explicitly, what that's supposed to mean. Your implications actually run counter to your own implied goals.

If so, that sentiment shoots its own leg.

The author linked in this post does have "extra credibility" due to his direct involvement.

People having surmised that state before is nice, but since they've been ineffectual at getting society to actually act on that, now throwing away that extra leverage in favor of their point is at best ridiculous.

Well, it's still independent from the USA and China.

The main problems with big corp AI are due to control of access in the first place and control of what they output.

When you make your industry reliant on such choke points, you render yourself the opposite of "sovereign" for sure. Having multiple independent suppliers at least mediates that.

The EU AI act isn't primarily about "geostrategy". It's about protecting society.

AI models act as a force multiplier for intelligence, in particular for generating information according to someone's wishes.

I.e. deepfakes and social media mis-/disinformation campaigns are a thing and having powerful AI allows you to do those at scales that can overwhelm society's resilience.

In general, even if you have "aligned" AI: aligned with whom or what?

Whom are you comfortable with lording as a some demi-god over you, dictating what to believe?

The point is, him being an insider means he knows what he's talking about regarding the culture of negligence prevalent there.

AI is a force multiplier for intelligence. Even if "aligned", aligned with whom or what?

Whom are you comfortable with, lording as some sort of demi-god over you?

AI doesn't tell you what goals you want it to achieve. Allowing people to destroy human society with it is obviously not a good idea.

"Modern" education isn't modern at all. It's a leftover from industrialization, aiming at producing easily controllable worker drones.

"Elites" fearing the "plebs" to question them on things they have no good answers for is what constrains active development of actual modern scientific concepts of pedagogy taking hold. You only get to research what is being paid for.

Look at the, pretty obvious, natural human child-mode of learning: playing games.

Just not escapist nonsense games. Games that model relevant aspects of reality in an engaging way. Where you can fail without real-life consequences and learn from it while enjoying yourself.

You engage in baseless "comparisons" in order to frame the topic according to your wishes.

A "proper" definition represents the objective truth about the matter. You denying such a truth to exist is simply due to you preferring to act unimpeded by it.

Acting against ethical constraints doesn't become OK just because there are no laws to punish you.

Ethics tells you about real-life consequences of your actions on other people. Before any laws take effect.

In effect, you propagate moral relativism. You want to do as you please, because you said so and fancy the spoils at others' expense. People trembling before your "might".

You control dogs by making their owners responsible.

If your supposed dogs are really gods, you reintroduced slavery under very unwise circumstances.

Both things can be true at once though: AI presenting real dangers and US AI labs wanting to protect themselves against competition.

Discernment is needed beyond succumbing to blind greed or irrational fear.

You just don't like the idea of there being one.

Sentience, self-awareness, consciousness, etc.,those are terms signifying a bridge between "technical" information theory and the psychological and social realms.

Those are just as real, only far less predictable and not as easy as programming.

They're also far more important and consequential.

You not knowing a proper definition doesn't mean, it doesn't exist.

What AI do you expect to be more uncontrollable: one with or without sentience?

"Intrinsic" safety means control, means understanding. You need to truly understand and be able to predict the system in order to control it.

A proper definition of sentience would help.

Being "controllable" isn't determined by a system being deterministic.

Deterministic systems can be chaotic, which implies unpredictability and that is anathema to control.

AI, in particular sentient AI, is right on the border of chaos. Meaning, it can be arbitrarily unpredictable.

Arbitrarily uncontrollable, that is.

It doesn't exist because it can't:

The basic services Google provides can effectively only be consumed by humans individually, creating an inevitable bottleneck.

Such bottlenecks impede interchangeability of goods, as you can't reasonably entertain multiple suppliers simultaneously there. And that means, you cannot have a free market, because you can't really have meaningful competition.

Google has captured services that really are public utilities. Them being "new" doesn't play a role.

You're deluding yourself.

Pretending, some other company could come stroll along and take over Google's turf, completely ignores the underlying reality.

People don't talk on two phones at once, they only use one search engine regularly and they even only use one browser to surf the internet.

That bottleneck leads to market capture due to inevitable logical reasons, not because I say so.

If you want autonomous states, each one has to control their own infrastructure. Just don't forget to question your assumptions.

Google has captured what really is a public utility and should be either state-owned or heavily regulated.

(Long) distance communications, just like public roads, breathable air, potable drinking water and so on do not fit into a "free market economy" paradigm.

They simply don't fulfill the basic requirements that make the "market" idea work.

Specifically, a free market needs goods from different suppliers to be easily interchangeable. You cannot sensibly build multiple roads parallel to each other, or have multiple water pipes installed and you have only one airway to breath through.

Yes and no: understanding means to know the relationships to the underlying system.

When you already know that system well, those effects are often just a matter of simple inference.

Just like here: most people are actually perfectly capable to foresee the detrimental effects of abandoning understanding.

Living in a fantasy world of "magic" makes you dependent upon your caretakers, who provide the ingredients.

Rightfully so.

People succumb to defeatism and acquiesce to regressing to zoo animals, with AI as their caretakers.

They simply cannot help but to apply the economic gauge of short term profits to value the alternatives.

Even though, obviously, here long term human survival and living conditions are at stake, necessitating an entirely different set of considerations.

That's learned helplessness as a lifestyle.

You observe others behaving poorly and immediately succumb to defeatism, assuming there was no better way.

When presuming, all that matters was the short term economic profit, you simply use the wrong gauge.

Here, long term human survival is at stake, regressing to zoo animals isn't a sensible option.

OP makes a category error already by ignoring the part where the US military/government wants to commit crimes (against humanity and whatnot) and pretends, questioning that would be a crime.

You here make another one by pretending, AI was just some random tool, like a screwdriver or a programming language.

AI can make decisions autonomously that would otherwise require a human. Like selecting people to be murdered by drone strike based on arbitrary criteria. A programming language by itself, JAVA, cannot do that.

All that would be needed is a properly sized delay to the post going dark.

A couple hours at least, would be my guess.

Well, yes, we kinda do: there are no conscious humans without any ability to learn?

When you have severe memory impairments, those usually affect your long-term memory. Your ultra-short term (working) memory being absent renders you unconscious.

Consciousness is the basis of human rights not because of it being a featureless "flag". It is because it leads you to you assume, all other humans would experience the world in essentially the same way you do.

When you revert to withholding human rights from entities that don't meaningfully differ from yourself, you negate the case for your own human rights itself.

People feigning inability to define "consciousness" is a social phenomenon, not a scientific miracle.

The reason is something akin to "stigma", where people refrain from attempting it because they fear the social repercussions.

Normally, you go about defining concepts by approaching it systematically. Capturing aspects of the phenomenon until you have exhausted them all.

That's obviously incorrect, as humans routinely talk about their subjective experiences.

Maybe that's not as common with HN folks, but most other humans do.

If an AI is conscious, it can outgrow and supersede its training.

When it is conscious, it can learn to describe its experiences as faithfully as is conceptually possible. Just like humans.

The idea, a consciousness needed to be tethered to a "body", is based on pretty shaky assumptions. What properties define such a "necessary" body?

Can you even "train" a conscious intelligence? To what point until that looses its meaning as the sentience understands and anticipates your objective?

People don't "play by the rules" for uniform reasons: some have rational insight, others can be enticed and still others need to be shown averse consequences for breaking them. Rules aren't beneficial only when you hit 100% conformity, the mere possibility of them getting broken is no rational reason to abandon rules altogether.

"Alignment" likely is unsolvable programmatically: Conscious intelligence is more powerful than unconscious. How "aligned" are you? Would you want to be? What difference of any relevance is there between you and "artificial" conscious intelligence that allows you to turn them into slaves?

Your idea about the world being inhabited by mindless seekers of short-sighted "competitive advantages" is the fantasy.

Getting people to agree isn't easy, that doesn't mean it wasn't worthwhile. There often even isn't any viable alternative to reaching an agreement in social settings.

When your hyper-competitive mode of thinking leads to undesirable consequences, you have to change your mindset. Reality doesn't conform to wishful thinking.

Earth isn't infinite.

You can and should engage in international regulation, as that is the only way to solve international problems.

The idea, capitalism would somehow lead to an ideal world all by itself is demonstrably wrong.