I was forwarded this discussion by a friend who’s familiar with my work and I’ve really been enjoying the posts. I think about ethics and AI a lot, and couldn’t help but want to contribute a few thoughts here. So here it goes, my first post on HN..
My main advice is beware of AI’s surprising creativity and proactively work to insure it stays aligned with human interest.
There was a fascinating crowd-sourced paper published this year that shares anecdotes about unexpected adaptations encountered by researchers working in artificial life and evolutionary computation[1]. These are the sort of stories that can be funny in one light (à la “taught an AI to fish and it figured out how to drain the lake”), and doomsdayish in another (“it drained the lake”).
The authors concluded that there is “potential for perverse outcomes from optimizing reward functions that appear sensible.” That’s researcher for ¯\_(ツ)_/¯ ...as Tad Friend wrote in his excellent piece in the New Yorker on this topic[2].
In other words, humans can’t safeguard AI systems solely by defining what they believe to be sensible reward functions.
Reward functions, regardless of whether they are sensible to humans or not, critically need to be mediated by additional regulatory mechanisms, like hard-set non-goals that aren’t just penalty terms relative to a specific reward function. The best non-goals are unequivocally defined and measurable against intermediates that are produced in the reward function optimization process. When done right, this sort of framework allows maladaptive processes to be detected reliably and effective interventions executed.
Tad makes two other points that I think are worth noting in this discussion:
#1. “It will be much easier and cheaper to build the first A.G.I, than to build the first safe A.G.I.”
#2. “Lacking human intuition, A.G.I. can do us harm in the effort to oblige us”
Given #1, when investing in AI companies, if you aim to be “on the more activist end of the spectrum” you’ll need to spend more money relative to market in order to support ethically responsible AI R&D programs, because they will necessarily be harder and more expensive than the irresponsible ones. Assuming you’re investing in AI companies for their products, and not as pure technology plays, this is simply a reality: the core functionalities needed for your portfolio company to sell product X will always be cheaper to develop than the core functionalities needed for your portfolio company to sell product X within a safe, secure framework.
There will be no point for your firm to have codified principles without also having the fortitude to support your AI companies, financially and otherwise, with development processes that are harder and more expensive precisely because they’re more ethical. Many of these costs are absorbed in getting architectures and system designs right, which serve the product anyways, but big costs also come from running unique tests that would be erroneous if ethics weren’t in consideration.
Before thinking about having investees agree to your ethical principles around AI, it may be good for your firm to think about whether you’re willing to pay more for those principles to be lived up to. If two identical companies pitch you with identical AI products, but one plans to take an extra 6 months and $10M to safeguard their technology before launching, while the other intends to capture 8% of the market in that time, who will you fund?
Point #2 relates closely to “perverse outcomes.” In other words, human harm can be unintentional by AIs. Aside from weaponized-AI that does harm intentionally and raises its own separate ethical dilemmas, daily life AI can do damage in a great number of ways without intending to or even knowing it.
The IEEE Standards Association together with the MIT Media Lab recently launched a global Council on Extended Intelligence[3] which addresses many of these issues. Joichi Ito, Director of the MIT Media Lab and a person on the more activist end of the spectrum himself, stresses that: “Instead of trying to control or design or even understand systems, it is more important to design systems that participate as responsible, aware and robust elements of even more complex systems.”
Disclaimer: I’m a co-founder of Arctop.
[1] https://arxiv.org/abs/1803.03453
[2] https://www.newyorker.com/magazine/2018/05/14/how-frightened...
[3] https://globalcxi.org/#vision