763 karma · joined December 7, 2020
Give up too early -> users will get annoyed because the task would have been solvable if the model pushed harder.
Give up too late -> collateral damage while completing the task A.K.A. misalignment.
If these 10 problems were solved by humans, it would be pretty impressive, even if it took a large number of researchers! Yet when AI does it, HN commenters suddenly feel the urge to play accountant.
The hard problem is that there is such a feeling at all.
Funny seeing this repeated again in response to Siri which is just... not very good.
All I'm saying is that blind enumeration of mutations seems combinatorially infeasible due to the vastness of the search space. It is already known that mutation bias exists, so what I'm saying shouldn't be that controversial.
My argument doesn't depend on the existence of an intelligent species on the planet. The problem already arises when there are multiple species on ONE planet. If you calculate the pure combinatorial distance between the DNA of 2 species, you must find that you can't just brute force your way from one to the other before the heat-death of the universe. This is why mutation bias exists: not all mutations are equally likely, evolution favours some kinds over others.
I'm not trying to suggest woo here, but there has to be some mechanisms to constrain the search space somewhat.
That said, this shouldn't be a competition of who is "more self-serving". Just because your neighbour murdered two people, doesn't mean that you get to murder one.