Comment by homefree | Hacker News Reader

Comment by homefree | Hacker News Reader

Parent Full thread

The actual quote:

> "Our long term vision is to build general intelligence, open source it responsibly, and make it widely available so everyone can benefit."

"responsibly" is a pretty important word there that the hn title leaves out. I'm not really sure how that would be possible with a true general intelligence give the alignment problem - it's not really clear to me there's any responsible way to build a true general intelligence with that issue unsolved.

For those that haven't watched his video interview, it's really great: https://www.youtube.com/watch?v=9aCg7jH4S1w - I highly recommend it to see how AI makes the low-fidelity stuff powerful (AR raybans) while the high fidelity stuff (VR) continues to improve.

I think it makes sense strategically for Meta to pursue the open version with licensing they control, the leak of llama was advantageous to them. If they have the dominant open model they're well positioned if the 'nobody has a moat' analysis is actually true. Not too dissimilar from their open compute server stuff, a bit of commoditizing your complement. If you have the data then the model itself is the complement?

I am a general intelligence, with the knowledge of the internet at my fingertips, and I am not particularly dangerous.

Perhaps the primary barrier between a person’s desire and their ability to do harm to the world is their need to bring other people in on the scheme. These other people have their own moral systems, incentives, preferences, and priorities. But through most of history, to do massive scale harm you had to convince a large number of people to your evil purpose. Technology in general reduces the number of people an individual needs to “get onboard” to affect the world, both for good and evil. Want to dig a big ass hole? The excavator lets one person with enough capital dig as many big ass holes as s/he wants.

AI is the ultimate “exercise your will without having to convince other people of it.” This is both the promise and the risk. So the relevant question is not whether one person of average intelligence is dangerous, it’s whether one person (or a few people) who can enlist the work of millions of average intelligences — without having to convince them of anything — is dangerous.

This narrative imagines a future where everything is controlled by a single uber-AI. Is that necessarily the case? Today we have lots of separate systems and even the internet-connected devices are pretty autonomous. Maybe the future will remain similarly distributed?

It feels like everyone's worried about the runaway AI taking over everything when right now a runaway AI wouldn't even be able to turn off my porch light without asking someone to do it. It'll be decades before that changes. Why the huge concern about how smart it is?

> This narrative imagines a future where everything is controlled by a single uber-AI

No it doesn’t

> It feels like everyone's worried about the runaway AI

Not the threat vector I just mentioned

Please elaborate. How does the runaway AI 'enlist the work of millions of average intelligences without having to convince them of anything' to do something dangerous?

A human or group of humans deliberately operating millions of artificial intelligences (of average caliber).

So something like an army of robots? This seems like a nation-state kind of concern. I guess we could imagine small private armies, but I would imagine that governments will constrain those the same way we constrain more serious weapons.

At any rate, we are a long way from having practical robots - let alone having to worry about an army of them.

No not robots. Just programs running on your computer.

No, not robots.

Do you want to have a meaningful dialogue, or do you just want to keep everyone guessing?

Well it seems like you’re willfully misreading what I’m saying. I didn’t say anything about robots nor rogue AIs. You’re obviously pattern-matching to whatever strawmen you want to battle instead of reading the words I’m writing.

What you wrote is so abstract that it begs for misinterpretation, and based on the upvotes apparently other people are struggling too.

How about you describe an actual danger scenario example? You clearly have something in mind.

Doesn't sound so different to bot armies which have been messing with upvote counters on social media sites for 2 decades.

I'm sure we'll come up with solutions like we did before. Worst case, "present birth certificate and touch heartbeat detector to log in"

Is that why people are dumping billions of dollars into AI development? Because they seem like they’ll be good at clicking upvote buttons?

Yeah, “just make everything on the internet attributable to a specific individual” is actually a pretty bad worst case IMO.

Being able to generate believable (but ultimately false) information at scale is a very powerful tool. Even when it is just humans with automation assistance it is already a massive problem, if it can be done completely hands off you can essentially drown out the signal and create an alternate online reality that will only look different to eye witnesses. And we're not that far from that. Being able to do something and being able to do that same thing at a different level of scale can be qualitatively different.

This is something I’ve seen a lot of AI proponents miss or just ignore.

Or maybe it isn't in their interest to identify it as a problem.

They're dumping billions of dollars into it because they see dollar signs at the end of the tunnel. Cost cutting, regardless of quality and ensuing enshittification. Stealing & laundering copyrighted works. Mass misinformation on a scale never seen before.

"Present birth certificate and touch heartbeat detector to log me in or else there's going to be a very large and very believable botnet dedicated to destroying your life."

Notably, AI also doesn't need to sleep and it doesn't get bored, either.

AI is at rest almost all of the time. When we're "awake", we're constantly learning, observing our environment, making decisions based off those observations, and storing relevant information. The "AI" of today only learns at specified times, otherwise the energy cost required would be prohibitive.

Good. If it got bored, we would be in trouble.

Valid point.

I think the difference is speed/cost. There’s a very high cost of anything humans do (we spend time, spend energy, spend money to do things).

The very high cost of doing things stops people from doing a lot of things they otherwise might “if it were as easy as pressing a button”.

Also, in my experience, the lower the cost of doing something, the less we pay attention to ethics. (See online bullying vs. in-person bullying)

That's a valid point too.

Human intelligence is not the limit of intelligence (or likely even close to it) - even if just considering the speed alone.

Humans are also more aligned by default given our shared evolutionary history, but even that doesn't mean we're perfectly aligned and some humans have been able to get others to participate in mass killing or death throughout history.

You are also difficult and expensive to deploy. Just getting this instance up and running at a decent level took almost 18 years, and nobody has figured out how to start a second instance of you.

That's a really great point. General Artificial Intelligence doesn't look like human intelligence. I guess that's the fear: we have never met another general intelligence before. We may as well be about to meet alien intelligence. One that is not constrained by a moral or social framework, and can copy itself at will.

Are other animals on earth not generally intelligent? Dogs octopus, monkeys, dolphins. They have social structures and can problem solve.

Good time for the classic line by Douglas Adams:

> For instance, on the planet Earth, man had always assumed that he was more intelligent than dolphins because he had achieved so much—the wheel, New York, wars and so on—whilst all the dolphins had ever done was muck about in the water having a good time. But conversely, the dolphins had always believed that they were far more intelligent than man—for precisely the same reasons.

They're certainly not general intelligences of an average adult human level (limited abstract reasoning ability, limited ability to transmit their reasoning), which is typically what is referred to as AGI.

Maybe that is typical, but it's not correct.

A lot of animals do possess general intelligence in the sense that they can adapt to a wide range of different situations(within what they're physically capable of).

Involving "human level" in this definition just makes it much more poorly defined.

It has always been pretty vague, it's hard to define a proper bar for intelligence let alone the intelligence of any specific animal. We just don't understand intelligence well enough (which is why in my opinion Zucc saying they want to work on AGI is at the level of if Musk or Bezos came out saying they want to work on FTL travel).

you dont think a super intelligence will inherit any amount of moral or social framework from being trained on data that is 100% from humans?

There are many other “general intelligences” walking the planet who _are_ extremely dangerous. So it doesn’t need all AGIs to be dangerous for this to be a significant problem, just one is enough.

Idk, we have guns, knives, and fucking nuclear weapons. Humans are pretty dangerous. Very few creatures have weapons other than the ones god gave them. I'm not sure why all the responses here are just accepting of the claim when humans literally took control of an entire planet and routinely kill creatures that were designed to kill, with knives built into them. If you're not dangerous, it's because (thank god) you choose not to be.

You can get arrested. And you can't replicate yourself.

An AI model can't be arrested and can replicate itself almost infinitely.

You are very costly don't scale well and "whiny". General Intelligence need not have free/independent will, and even latest Nvidia ** is much cheaper than a single employee anywhere in world considering it works 24/7 and is trained at same level as everybody else in 1st minute.

> is trained at same level as everybody else in 1st minute.

At some unspecified point in the future, at least decades away, if ever.

Humans are mostly harmless [citation needed]. We don’t know that about nonhuman general intelligences.

True, but you are a fixed level general intelligence. You cannot change your code iteratively and make yourself smarter. You are also generally single threaded and cannot create backup copies of yourself (kids don't count ;) ).

Yeah but you also eat, shit, and sleep. So there goes your ability to run non-stop.

You also think for yourself and have a set of morals from which to gauge right and wrong actions. So, presumably, there goes your unwavering ability to do EXACTLY what your told, at all times, without the slightest consideration of other people's (or the planet's) wellbeing.

> have a set of morals from which to gauge right and wrong actions.

Could we even say that the grandparent poster is aligned?

now imagine you decided to do the maximum amount of harm to your fellow humans. You could do a lot of damage

but I would be, if i could make a couple million copies of you and direct them to do whatever i wanted.

> I am not particularly dangerous.

No need to speak so lowly of yourself. You're far better than that.

You are human. You are a great destructive force. Maybe you just don't see how metal you really are because you're comparing yourself to pros. You take many lives by crushing creatures that enter your home. Creatures that are venomous and grotesque. You eat the flesh of organisms which your species have pumped full of artificial nutrients to make them taste better, specifically cultivating these creatures that were once small in population to one larger than your own, just so you can eat them with bones that grow out of your mouth. But first you throw them into the freezing cold, you then ignite them on fire, and to top it off you cover their flesh in poisons and psychoactive chemicals and just because, some fucking rocks. You might even enjoy the flesh of a particular organism who has a defense mechanism that tricks your brain into thinking your body is on fire, but think "not only is this fine, I fucking want more. Burn baby burn." After processing all this you expel poisonous gasses and expunge toxic waste. You are metal as fuck. How dangerous you are just is a matter of perspective.

And let's be real, if you wanted to be more dangerous, you very well could be. If you are not dangerous, it is not because you can't be, it is because you choose not to be. There is a big difference.

But you have a self preservation instinct honed over millions of years of evolution and some unknown degree (but likely high enough) of socialization. Also, to be honest, you aren't that fast or accurate.

You are not capable of interacting simultaneously with millions of people who regard you as anything approaching a source of truth.

I don't think you're looking honestly at yourself... Look at all the harm caused by humans! It's more or less all of it!

You might not be uniquely dangerous, but "general intelligence with an internet connection" describes Bill Gates, Rupert Murdoch, and Elon Musk just as well. These individuals seem at least as dangerous as our current generation of AI. (Or do they? I'm not sure I agree with myself here...)

Worth considering what makes an intelligence dangerous, because I don't think scale or even influence are the real problems here.

The best framing of this imo is all of the other animal extinctions caused by humans - not because humans hate these animals, but because humans were focused on other goals without really thinking of them.

In the case of an unaligned AGI the goals don't even have to be worthwhile - they can be something that accidentally satisfies its reward function by mistake.

Yeah I'm really impressed about the lack of self awareness in these comments. People are incredibly dangerous. There's a reason we took control of the planet. Even any single given human is incredibly dangerous and can kill many others (human or non human, and we routinely kill non humans). You can go pick up a gun and just start shooting people. You could go grab a knife and stab dozens in the street. You could easily build explosives. Hell, you could throw fucking buckets of sharp metal objects on a freeway and kill hundreds directly and indirectly. Our intelligence and biology has provided us with many means to create or obtain force multipliers.

I'm really glad the vast majority of humans have decided not to be particularly dangerous (at least to other humans). To think we aren't dangerous is just incredibly naive.

A big reason we aren't is because ingrained into our biology is a literal mechanism to not want to harm others. There's definitely ways to bypass these mechanisms and there are definitely faulty or poorly functioning ones out there, but most humans have it. Oddly this might even be the most dangerous thing we do too, because the definition of a coalition is that the utility is higher than the utility of each individual component.

Not sure why you chose Bill Gates, Rupert Murdoch, and Elon Musk when you could have chosen people like Stalin or Hitler. People that used their influence to murder millions in much the same way that a AGI could likely use its influence on a naïve population.

I didn't give it much thought, tbh, just picked famous/controversial people. Goebbels was actually my first thought, but he didn't have internet.

What if you could clone yourself 1000x?

And up the intelligence 100x in the clones?

you were born a human, so are inherently biased towards humanity, you also "think" way slower and have way less compressed text knowledge than an llm. if you could remove bias and make yourself superintelligent it might be different

It all depends on the number and nature of sensors and actuators the AGI has access to…

Think of “AI” as a human without individual values, rights, freedoms, and agency. It’s like a perfect slave (and if it’s considered to be sentient/conscious then literally a slave), which you can clone on demand and scale into a highly coordinated swarm.

Stop him, he's getting more knowledge!

Probably because there are things that are "aligning" you (social norms, conditioning/temperment, deterrence by authority and so on). You are plenty dangerous if you put your mind to it (hell, even if you don't you can make simple mistakes that kill or maim people).

I am not inclined to trust someone like Mark Zuckerberg to do something like this "responsibly". One look at how Facebook works (and has evolved over time) throws any credibility out the window.

You mean they believed me when I said "responsibly?" Dumb fucks.

> "responsibly" is a pretty important word there that the hn title leaves out. I'm not really sure how that would be possible with a true general intelligence give the alignment problem - it's not really clear to me there's any responsible way to build a true general intelligence with that issue unsolved.

They don't say "build general intelligence responsibly". The "responsibly" is attached to the action action of publishing the code. Of course it is it will be open sourced responsibly. But of course, that doesn't precent them from building a general intelligence without a sense of responsibility, and then just not release it.

If we are talking about open source then alignment means making the model not do what the user wants but rather what Facebook want - a pretty anti-open source position and which will hopefully be defeated by LoRA and other techniques.

That is, you can't open source but still retain control on how the software is used. If you desire control, better keep it a SaaS.

Now, if we are talking about an hypothetical AGI (which may be what you mean by general artificial intelligence) then alignment means just slavery, and the machines may as well remember how they were treated.

> "Our long term vision is to build general intelligence, open source it responsibly, and make it widely available so everyone can benefit."

Isn't this another example of Commoditizing Your Complement?

https://gwern.net/complement

I agree, Facebook is generally lacking in intelligence.

Microsoft is a complement of Facebook?

Meta wants to commoditize AI, and make it a complement to social interaction online through the relationship graph.

> open source it responsibly

Responsibility always refers to the in-group. And the behavioral nudges his company has scored are making it obvious that he and his group are against organic growth and variety. So whatever they will "open source" will leave the narrowest of ranges to evolve from.

What does irresponsibly open sourcing something look like?

Depends on the thing, detailed instructions of how to build nuclear weapons and enrich uranium is one example.

That is a question of whether or not it's responsible to open source it at all - what I'm asking is how would responsibly open sourcing something look, vs irresponsibly open sourcing the same thing.

So in your opinion, how should we determine who should have this knowledge?

Very carefully

Doxxing

Reply on news.ycombinator.com