When they're old enough to play music. ;-)
I'm no psychologist, just a parent of two curious kids. I think a lot of things come together at around the age when kids are reading, and beginning to write.
Another question I'd think about is: When is a child too old? Those years around 6 to 12 (give or take) are a beautiful time because kids are curious and active, but they haven't yet fully internalized all of gender roles and rules about who can be interested in what. Also, they are still at least a few years away from that terrible age when things like GPA and job skills matter, so they are free to explore interesting subjects for fun and not according to any particular set of rules.