It's no surprise that I didn't make any real progress, since I wasn't learning an instrument and my school teachers made us sing but didn't bother trying to teach us to sing -- but by late primary school I was a notably bad singer relative to (almost literally) everyone else in my class. My home was by no means full of music, but we did have a record player and my sister liked to sing with my mother sometimes. And although it took me a while to get interested in pop music, by late primary school I had a CD player and some bands that I liked.
So I doubt the difference in ability was solely environmental rather than innate; surely the other kids hadn't all been taught how to sing, or grown up in very musical households. (I'm still very weak musically, and still can't sing, but I'm not literally tone deaf.)