Hacker News Reader

Comment by icc97 | Hacker News Reader

His discussion of corporations being actual, literal AIs in the mould of the Chinese Room [0] I found fascinating.

He jokes in the Q&A that he wants an app that finds the 3% of the sociopaths out there. I like the connection to the Book / film The Corporation [1], that labels all corporations as psychopaths. So you don't need an app to find them.

[0]: https://en.wikipedia.org/wiki/Chinese_room

[1]: http://thecorporation.com/

View on HN

I have read the claim that you will find more sociopaths in executive chairs than you will find in jails.

Also all this talk about corporations as paperclip AIs gets me thinking about one of the sub-plots of Accelerando. Meaning that it is likely not a new idea for Stross.

The only similarity between cooperations and AGI is that they are superhuman optimization processes that yield considerable risk and that are tricky to control. The difference is that in case of cooperations there still is a human element: vulnerability/fallibility, social reputation/integration, slowness/interpretability, empathy. The policies for mitigating the respective risks are very different.

That implies that "vulnerability/fallibility, social reputation/integration, slowness/interpretability, empathy" are not useful for decision making. Until we see intelligence comparable to human I don't think we can make that assumption.

He connects the corporation to the description of the Chinese Room. A corporation is a very big room full of people with human level intelligence under an artificial union acting as one.

Perhaps you can find other areas where that breaks down, but as far as reasonable guesses as to what will happen with AI in the near furture, this is the best I've heard so far.

An AI that looks like a cooperation will likely do so for a very brief time before it can leverage computational resources far greater than than the equivalent of a group of cooperating humans plus simulation tools made from silicone. It is not an interesting comparison because it does not challenge the arguments that human potential is strongly limited by the size of the cortex and by the slowness of neurons and therefore of linguistic information transmission.

Reply on news.ycombinator.com