In theory you can do that with any language that has a written form (eg. English). Just that it's counter intuitive and nobody does it that way.
I doubt anyone _actually_ learns Mandarin without learning how the common characters are pronounced... Unless you don't have to verbally communicate with a teacher / fellow classmates?