Because physicalism implies that the mind is an emergent phenomenon, and because quantum physics is a linear theory, there's simply no place for free will unless you go down some weird/unfalsifiable rabbit holes like the MWI.
So a lot of people prefer to latch onto the idea that there is a soul, because if there wasn't one, then they wouldn't be special anymore.
I don't follow. Quantum mechanics means that physics is stochastic, and only approximates classical determinism in aggregate. Meanwhile, brains seem to operate in a critical state that can be tipped to different phase transitions by small perturbations, so there is potential for a quantum event to influence the brain as a whole in a nondeterministic fashion.
This means that strict physicalism still leaves the door open to our behavior being stochastic rather than deterministic, doesn't it?
So in this view consciousness is better explained as an illusion than as the seat of your will. Because you can't really have a will if every interaction that led to your existence can, in principle, be computed forward in time from the big bang. Physicalists see it as a complex emergent phenomenon akin to a run of Conway's Game of Life that started from a random seed.
Whether that stochasticity amounts to free will is a separate discussion, but it is fairly clear to me that many of our decisions are consequential and AREN'T predetermined. Which at least sounds like free will, or close enough to warrant being treated as such, anyway.