Hacker News Reader

Comment by visarga | Hacker News Reader

On the other hand you can test a neural network a million times, exhaustingly mapping its biases and failures. You can't do that with a human, you got to take a chance.

View on HN

Yes you can do a million tests but the biases and failures of the neural net are chosen by human developers, a miniscule subset of citizens who themselves have some form of bias depending on their educational background, places where they grew up, family and friends etc. While you can replace a human in a powerful position with another one, since we do not know how to surgically correct individual weights in order to remove a specific decisional bias from a neural net, we can only retrain it and hope for the best. Because we are humans ourselves, we can understand the incentives of other humans and create adequate mechanisms to correct for their selfishness and their bias but what would be the incentive of an all powerful neural net? Which loss is it minimizing? And how does it know the citizens' preferences in the present and in the future? If the citizens stop feeding it (accurate) information at one point in time, will it still be the benevolent dictator that it was supposed to be?

*Edit: Another counterargument that generally applies to neural nets making decisions for human activities is the argument of accountability. While you can put on trial a bad politician for their harmful-to-society decision making, who is to blame when the neural net will inevitably spew the wrong output on an issue that it has not encountered before and a policy decision will be made based on that? Will we put the developers on trial?

Apparently GPT-3 has learned a large number of personality types and their fine grained biases, well enough to simulate a poll with good accuracy. This means you can poll it any time to check how a population of choice would have reacted had something happened. The language model does not carry the just biases of its builders, it learns all biases equally, you just need to specify the desired bias (personality type) when you call it. The responsibility falls with the user, builders are not to blame this time unless they didn't cover every bias equally well.

> GPT3 can simulate real people, Jack Clark

https://jack-clark.net/2022/10/11/import-ai-305-gpt3-can-sim...

Ok so GPT-3 can model biases well. This still doesn't solve problems such as the optimal aggregation of citizens' preferences, which is the actual optimization problem of policy making. Just to give an idea of how complex a field this is, there is a subdomain of economic theory and information theory called social choice https://en.wikipedia.org/wiki/Social_choice_theory that works with these issues and itself does not have many theories about policy formation, mostly choice between already formed policies. If a neural net finds the way to write policies that are Nash equilibrium collective decisions in all possible democracy systems then we will be close to solving the problem.

> you can test a neural network a million time [...] You can't do that with a human, you got to take a chance.

Funny, as a biochemist, I always took it that minds had essentially been through unfathomable numbers and levels of iteration through cultural and biological evolution cycles, and that ANNs were the untrustworthy new kids on the block :)

In aggregate, yes, but in particular, it's the opposite.

Reply on news.ycombinator.com