I think the problem with arguments regarding free will is that it's very hard to define (like consciousness for that matter). I am my brain. My brain controls what I do. Therefore, I control what I do. Only if we lift up our consciousness or free will as something higher than our brain does this become a problem.
The fact that my free will is deterministic, that is, a consequence of how the brain is structured and how the environment looks, does not change the fact that I, that is, my brain, controls what my body does. If anything, randomness makes this worse - then it is not my brain controlling my body, but some random result of a wave function collapse.