https://www.fastcompany.com/3059089/it-turns-out-its-almost-...
https://www.fastcompany.com/3059089/it-turns-out-its-almost-...
It's the same for "draw a house". You may do a square facade, triangular roof, windows, chimney, door. It is equally unrealistic, because people don't have a clue how tall floors should be, have no sense of proportion etc..
It's just that in the bike case it's more obviously wrong.
The largest ML systems of today have roughly the same complexity as human brains, and evolve in much the same way. The brain has 100 billion neurons, and GPT-3 has 175 billion parameters. Neurons and parameters aren't comparable, but there isn't an obvious advantage in either direction. Neurons have more parameters than ML parameters, but also operate at around 10Hz, versus many, many MHz.
That doesn't mean machine sentience will be anything like human sentience. Brain disorders are helpful to look at here -- there are people who don't experience specific emotions (e.g. pain, fear, etc.). Even a minor tweak can have a major impact. That's far less than, for example, evolving without evolutionary pressure for self-preservation, for pro-social behavior, or with the sort of ephemeral nature of ML systems.