AI thinks like a corporation (2018)

The narrative that the danger is corporations might accidentally let AIs gain power is appealing but indeed ought to be challenged.

The paperclip/strawberry monomania fear masks the way things how things are shaping up now. AI is far from being to able intelligently plan over a time horizon - maintaining a real world process all the way to even a smallish goal requires real world corner-case handling that the best AIs haven't got (self-driving cars have five years away for 10-15 years, etc).

But AIs as "intelligent" filters are here now. These allow "better" decision making along with unaccountable decision making, both of which indeed fit well with the agenda of the modern corporation. And that's the thing - the modern corporation is already short sighted, the process of ignoring the potential for climate happened well before automated decision making appeared. But the modern corporation is still more far seeing than a deep learning network and still theoretically and legally accountable to society. However, indeed, deep learning networks that only maximize those qualities that these corporation want maximize and ignore other considerations suit the indeed short-term preferences of the institutions.

The functions of processing inputs to create meaningful outputs are carried out by neurons made of meat (human employees) and of silicon. These processes are governed by rules and metaprocesses, which the entity continuously optimises and improves, in order to further its own growth and to improve its fitness for its purpose - typically, enhancing shareholder value. I cannot think of a single criteria for intelligence or even life that a corporation does not possess, from independent action to stimulus response to reproduction to consumption to predation.

I think the first true hard AI will emerge accidentally, and it will be a corporation that has largely optimised humans out of the loop - but even with humans as a component of a system, a gestalt, an artificial supra-intelligence, can still emerge.

This also neatly sidesteps the whole question of “should AIs have rights?”, as corporations are already legally persons.

Anyone who sets it up to maximize raw output of very simple commodities is doing things incredibly wrong economically in the first place. They would go out of business in the first place paying for storage costs alone - let alone the access of everyone else to counter the rogue self-replicating.

It would be admittedly silly as an objection for the metaphor if not for the fact people really do believe AI would be that way.

I think the point of thinking about how we regulate corporations so they don't focus on singular goals in a harmful way is important. I just find the article's framing weird, as if it thinks people can only care about only one of AI alignment or corporation alignment, and that it's desperately important that they pick #2 instead of #1. I think the topics are actually very related and it's actually useful to think of them as different examples of the same problem. Thinking about AI alignment can help us understand how corporations should be regulated, and vice-versa.

(I think AI safety/friendliness/alignment problem is important, but it's really only relevant to human-level AGI. It's not about modern-level AI / machine learning / algorithms / whatever. No modern AI/ML understands the world at a human level and generates its own goals and motivation in a general sense. There's plenty of ethical issues about how we use modern AI/ML in businesses and so on but it's pretty separate from the AI alignment problem. The AI alignment problem is only an imminent issue further down the line, like "how do we build moonbases that don't fail?", which is interesting, and I'm tired of tons of articles bringing up the alignment problem like "There's some weirdos wondering about how to make buildings in space, but don't they know there's nothing to build things out of in low-earth orbit?? Are they just delusional?".)

Here's some actual insight on the topic: https://www.youtube.com/watch?v=L5pUA3LsEaw

A hammer that hammers is a good hammer. So if a system of ML/AI were to be optimised to hammer well (and 'be' a hammer) that is fine. If someone then would decide to use that hammer to attack someone and break their bones, would that be the hammer's fault? Probably not. But when you get a hammer that is optimised for bone-breaking the discussion changes. Same with a kitchen knife vs. hunting knife vs. kabar which are all knives but are also all 'named' with different intentions. And suddenly it's no longer dual-use or 'who is at fault' but in the very grey area of 'what was the intention'. And that brings us back to 'transparency' which loops back in to "what if it is just a tool". Darn circles.

A human can tell you a more or less convincing narrative about why they did something, but you have fewer independent and objective ways to measure that than you do basically any conceivable computing system.

[Lem, the author]... wrote that he deliberately chose the ocean as a sentient alien to avoid any personification and the pitfalls of anthropomorphism in depicting first contact.

https://en.wikipedia.org/wiki/Solaris_(novel)

It remains to be seen if corporate-like structure is something it surpasses, but complex systems seem to usually have orders of influence, and multiple hands on any given wheel.

And yet if we perished from the planet, literally every single other species, animal mineral or vegetable, would benefit from our absence. Everything would grow back at an amazing rate and it would be like we were never here.

Meanwhile, if the bees disappear, we're all screwed.

Supposedly, there's also an empathic element to being an effective hunter, both in the human and the animal kingdom. To deeply understand and empathize with your prey helps you capture and devour them. Indeed, the marketing, advertising, and product development divisions of corporations can be deeply "empathic" to the desires of the buyer, without necessarily being to their benefit.

At any rate, I don't disagree; corporations are agnostic to societal externalities, and highly incentivized to create habit-forming relationships with customers, hence often leading to behavior indistinguishable from sociopathy. To some extent, I think the best we can hope for is aligned incentives; sometimes what's best for a company's bottom line really is to make people's lives better. But we shouldn't be naive to exploitative relationships (even when nominally voluntary), nor should we lean on "The Market" as the singular societal organizing principle.

Extrapolating to AI, we should be very cautious as to what metrics we optimize any particular algorithm for. Corporations optimizing for stockholder returns are ultimately a subset of "paperclip maximizing"; what we really want is a balance of multiple leading indicators of success, and to constantly be tweaking those success conditions as we discover new metrics for measuring human flourishing.

But by some measure, human thinking is itself math: emergent pattern-recognition traveling through cortical hierarchies and deeply nested layers of abstraction and metaphor. While it's indisputable that there's a qualitative leap between something like computer vision, and the basic reasoning capacity and perceptual systems of a human (even a toddler), there isn't anything intrinsically magical about atoms rather than bytes. It appears to simply be a factor of scale, and the billion-year-old "legacy code" gifted to our wetware by iterated selection pressures.