It sounds like we agree on the problem but disagree on the solution. I would argue most of the impediment to communication on this subject is exactly because people are trying much too hard to brand a certain conceptualization "free will" and trying to delineate different contexts and scenarios far too little. It's absolutely worthwhile to clarify what context one is referring to when using the term. It's absolutely a waste of time to try and assert this context is the only correct context to do so.