It's possible to have a gentle ramp-up on the math content while learning quantum mechanics. "The Theoretical Minimum" by Susskind and Hrabovsky is structured like that: They teach you the math as part of, and motivated by, teaching the physics, and the books don't lag or resort to hand-waving.