Sometimes, it makes more sense being cautiously optimistic (pro-active) rather than reactive. We have already gone down that reactive slope and it's better to act now before it's all too late [0].
[0] https://blogs.scientificamerican.com/roots-of-unity/review-w...
I'm a bit afraid that this will happen with self driving cars and AI. That politicians will create draconian policies and laws to protect against the threat of AGI etc, without understanding or knowing what the real threats even are (just look att the trolley dilemma debate...). This could make it economically prohibitive to develop many technologies which has the potential to save many lives as well as improve life quality overall.
* It seems to be more about how rules and policies can be unfair and just to a small extent about how policies can be made opaque by being internal to some ML system.
There's a lot more money going into making plants resistant to pesticide than into making plants better adjusted for harsh conditions or more nutritious, things that could potentially have a huge effect for poor people.
(Just venting here, not even primarily at you.)
360k babies are born each day. Clearly it is possible to reproduce intelligent machines. The only way it would be impossible to artificially do the same is literally if life was a magic, non-physical thing. I wish people who state things like this would also state any religious beliefs that lead them to think so.
An example might use crypto... you observe random information flying through the air, you may recognize it as an encrypted channel and you may see a machine acting in response to this encrypted signal.
With enough observation you may be able to mimic the encrypted signal to get the machine to act in a certain way, but you haven't decrypted the actual signal (and can't ever, if you believe in strong crypto) and can't ever say with any certainty you know the full scope of communication taking place or the capabilities of the machine you've been observing.
At any point you can make your own version of the machine mimicking the language tuning it to be an exact replica of the original and even responding to the original signal. Yet, is it truly a copy of the original?
It I send a message using a one-time pad, the other person knows what I sent, you can ask questions and see that communication is actually happening... so if we're not using magic to communicate, it must be possible to communicate in the same way, right?
Yet it's mathmatically impossible to do so without access to hidden information (shared key)... no laws says that you can access that information, even with all the computational ability available in the universe.
The mechanisms and communication patterns of consciousness are similar, if it takes till the sun explodes to train a true AGI then aren't we just getting pedantic about what is possible/magic?
(I figure I should be allowed to ask such a normally speaking quite loaded question because of my previous statements, up the thread.)
I don't really see how religion factors into this though... I feel like I'm talking about a simple concept too. If I show you an encrypted message and show you that other people can read the contents with a key, then ask you to read it without the key, why can't you? It's not magic, it's math.
The question is, will it happen in less than 50-100 years, or would we be like medieval alchemists rushing to outline the first nuclear weapons treaties, right after they have just invented black gunpowder.
If this is the basis of future AGI, I have to wonder if which flavor of dystopia we'll get to enjoy. Will it be a child-selling dystopia where we all raise a dozen kids hoping that some of them will pay off? Or more like silk-farming, where some capitalized breeder sells kits to all the villagers, and buys back the developed products if and only if the villager was lucky enough to raise them to fruition?
Also, if a human baby is our only basis for assuming AGI, then we ought to think about it like genetic-engineering or human augmentation. We'd better anticipate providing schools, hospitals, psychiatrists, courts, and prisons to deal with the wide variety of behaviors and misbehaviors which will come with these new products which have so little determinism as a baby's lifecycle.