Right, it rings too much like "You can't teach Chinese." It's more likely a combination of it being really difficult past childhood to begin with, and also that we haven't figured out how to teach it yet.

From personal experience, long ago I spent some time trying to acquire perfect pitch using a synesthetic method from somewhere, and eventually convinced myself that I was making some progress, but that the effort required to take it to the point of being at all useful wasn't going to be worth my practice time.