Hacker News Reader

Comment by alex_young | Hacker News Reader

Paper says it's focus is on all ET communications, but instead focuses on malicious AI code.

Secure isolation review of code is dismissed because AIs can trick people into freeing them.

Finding states: "[M]essage cannot be decontaminated with certainty, and technical risks remain which can pose an existential threat. Complex messages would need to be destroyed in the risk averse case."

Couldn't we simply agree not to execute un-audited code from unknown third parties? Seems like the same threat vector as unknown executable code delivered over email to me.

View on HN

I immediately thought of Vernor Vinge's "A Fire Upon The Deep" (https://en.wikipedia.org/wiki/A_Fire_Upon_the_Deep). Half of the plot is an AI tricking people into freeing it and the subsequent havoc it wreaks.

It is indeed a devious AI.

> Couldn't we simply agree not to execute un-audited code from unknown third parties? Seems like the same threat vector as unknown executable code delivered over email to me.

There are a couple of problems with this.

First, limiting the spread and ensuring that nobody executes the message is non-trivial (especially given that currently the protocols dictate "These recordings should be made available to the international institutions listed above and to members of the scientific community for further objective analysis and interpretation". Also, unless all instances of the message were destroyed AND transmission ceased permanently, humanity is left with a literal Pandora's Box. It's hard to imagine that this box wouldn't be opened eventually by someone with more curiosity that caution.

Second, there are messages that cannot be safely audited without running. A self-bootstrapping decompressor/compiler that modifies it's own code as it parses itself into existence could be impossible to evaluate. Committing ourselves to never executing un-audited code means accepting there are certain types of messages we would never understand.

So while your solution is a possibility, it is not an easy one and it is not one without costs. Maybe I'm not risk averse enough but I (like authors) think the potential gains of executing ET code outweigh the risks associated

I think the epidemics attributed to European explorers encountering the Americas are literally an example of unleashing unaudited alien codes on people.

I used to read old Analog magazines in high school, and I distinctly recall a short story about people picking up a transmission with a radio telescope, which told them how to build a computer - basically a form of alien invasion based on the fact that radio transmissions are the only practical way of interstellar travel.

I have something urgent I need to tell you. Your life could be in danger. You need to contact me immediately.

I mean, that's an example. There are many ways they could try to trick us. Presumably we are talking about hyper-intelligent beings.

Or more effective and sinister, appeal to one person in a posotion to run the code. “Want be rich/immortal/powerful beyond human limits?”

Furthermore, that strategy doesn't have to be a one shot deal, where the message says "believe me" before it is proven trustworthy. You can have little pieces of information that demonstrate neat tricks, leading to progressively increasing trust until the full payload is unveiled, for better or worse.

This civilization colonized half the galaxy thanks to this one weird trick. Execute the rest of the message to find out more.

"Hello mister sentient being! I am the informational treasurer for the dying civilization of Nigerium, with access to great technological and cultural wealth that is about to be lost forever. I wish your help in establishing a vast database for our future descendents to recover our advancements, and would like to send you the entirety of our knowledge in a format that will be accessible to you and our great-great-grandspawn. All I need is an example of the informational structures used by your civilization, with a complete and robust list of all known vulnerabilities that may have an impact on the safety and integrity of our heritage."

The simplest of tricks have been proven to work again and again.

"They're following this signal at 0.98c. You might be able to defeat them, but you must act immediately or face utter annihilation."

If they’re following at that velocity, then we’re already dead, and “they” are a relativistic bombardment we won’t even see coming.

Depends on how far away the signal came from. 10^9 light years away, and your .98c attackers won't be here for 20 million years. Even 10000 light years away, you have 200 years to play with.

Though it's likely pretty hard to get a signal 10000 light years, especially if you don't know the direction to send it.

Then the promise of safety is all the more tempting, isn't it?

Maybe, but if they’re sending us EM signals then we know we’re doomed. Physics as we know it, with c as a limit, strongly implies that it is simply impossible to defend against a relativistic strike.

>Couldn't we simply agree not to execute un-audited code from unknown third parties?

How exactly would we achieve that? How can we prevent every single human being on the planet from executing some malicious code coming from open space?

All ten of them that have the requisite equipment to pick up the message, yes.

> Couldn't we simply agree not to execute un-audited code from unknown third parties?

The ability of humans to abide by such a restriction is highly doubtful. Destroying the message would be much safer.

Reply on news.ycombinator.com