The appealing thing about this is the programmer doesn't have to understand anything. If you have little data, approximation just isn't going to capture the situation. Either the programmer gets an understanding of the system (extremely costs and time-consuming) or we create systems that are themselves capable of this understanding. But no one knows how to do this, all the "artificial intelligence" victories anyone has observed have come from throwing computing power at a problem. Maybe someone will figure out how to throw computing power at the general problem of understanding but I'm doubtful.
I mean, there are a few situations that can stay pretty constant but just think when a bunch of AI system are established, all thinking the world won't change, then a cascade of causes changes and the systems no longer working becomes part of that. Of course, we already saw fragility of modern systems with Covid, so let's add even more!
But if we're talking present day deep-learning/machine-learning/etc, these system are not doing much than pattern recognition in an impressive, sometimes useful but never reliable way. They'll add to the capital of those on top but not be a fundamental game changer unless something changes (and their seem clearer despite more and more processing power being thrown at them).
Too bad we (as a country and somewhat as a species) don’t really do ‘big’ things anymore.
Human mental health is a very difficult & intractable problem to solve in part because we have a very poor understanding of the closest thing we currently have to AGI-- ourselves. I wouldn't expect the human race to be capable of producing AGI that isn't batshit insane before we fill in some gaps of knowledge there first.
With things like either the flight to the moon or the atomic bomb, the basic principles involved were established and the engineering process to accomplish the task needed to be worked out (which isn't to deny these were huge, difficult projects).
Unlike moon shots and Manhattan Projects, large bureaucracies have poor record for automating discovery. Add to that, that nearly everyone is lost on the question of how to get to AGI (if it's not by brute-force training). And those few people who have an opinion on the subject don't necessarily agree with each other and speak in generalities that can't really be implemented. Having 1000 or 10,000 people brain storming seems as plausible as anything but the problem keeping such pure research at all motivated and directed is itself very hard.
The process actually is kind of like ... a university. In fact, universities are already where creative research tends to happen. But our universities, universities across the world, have evolved to keep researchers on a shorter and shorter leash. To get this unlimited speculation, you'd have to undo this - and lots people have an interest in keeping things this way.
And it's worth considering that wingnuts and crazy people are very interested in AGI along with very smart people - sometimes you get someone who's all three things, even.
So managing 1000 people trying to get to AGI would a serious challenge no matter how you cut it. Plus, the results of AGI might not be what anyone but it's creator(s) want, if that. So there are some hurdles.
What you can do is measure to some degree which direction certain publications or articles are leaning, such as left/right or establishment/anti-establishment. And whether they are more provocative, or more nuanced. That way you can be aware of biases and try to integrate information coming from different perspectives.
The problem here is that frequently there isn't an objective truth. There's often a substantial amount of nuance involved too (something ML can't really process yet). You'll notice that fact checkers often limit what they fact check to subjects that are well documented and not contentious in the realm of experts in the field. Unfortunately many problems don't fit under this umbrella.
Of course, you could do this in a limited sense, but there's a can of worms that comes with it. Unfortunately we don't know how big that can is, just that if we open it that there's worms inside. So we should be cautious.