But, it doesn't. The reason dualism and free-will are so deeply tied up is because “you are within physics” does not provide a means by which “you” can be the root cause of anything; if you are merely within physics, then you are simply a product of material causes and can only be (to the extent physics is deterministic) a fully-externally-determined phenomenon between the original cause and other fully-determined phenomena.
People are evolved to consider the decision process as a basic element in the world partly to recognize the decision-making ability in others, so we can instinctively predict other people's decisions, and know that we may be blamed by others for our decisions, and do that we can blame others for their decisions. So the ability to make decisions feels like a basic element of the world in ourselves and others, but that feeling is evolved into us to make us effective socially, it's not an element of the universe.
If you made an AI and an entirely self-contained virtual world for it to survive in, and you peeked into its memory, would you expect to find it actually caring (beyond mere curiosity) about whether the physics/software it existed in was deterministic (with pseudo-randomness, though still completely unpredictable to the virtual inhabitants), even though it wouldn't result in any possibly noticeable changes for it? If you traced the AI's beliefs and found that it believed that the correctness or free-ness of its decisions were linked to this question, you would assume the AI was very confused about something.
That is a common assumption, but I am not aware of any evidence that it is the case. If someone knows of any, then please share.
So, just make a decision and let the universe worry about providing initial conditions, which physically lead to the chosen action.
edit: If everything is "within physics," saying that the preceding history isn't relevant to the process of choice is a declaration that cause isn't relevant to effect. It's an arbitrary assignment for culpability's sake, which is a moral judgment, not a physical or logical argument.
In other words, you can't peer into the universe itself at any individual moment to observe all properties because you have access to partial information, the world could be markov, but the lack of your perceptive abilities requires that you treat it as non markov.
For example, given a moving object such as a ball, at any particular instant, you do not have access to enough information to estimate velocity, but if you have access to more frames, you can do some algebra and figure it out.
In the vein of cause of actions, since you have limited access to information, especially about you and your brain, you can not decide whether certain events such as having a good day affect your decision making abilities. Your state is part of the environment you do not really have access to it affects your behavior.
A choice is conditioned on a myriad of things, from whether you had your coffee in the morning 8 hours ago and the milk had gone bad, to whether your parents paid attention to you 20 years ago when you were a child.
Literally everything matters because experiences shape our brain's structure and that is precisely one of the factors that need to be accounted for in some cases. Not necessarily what one's own brain structure is like, but other people's as well, so as to gauge and expect their reactions to our decisions.
If the universe is Markov (which also means hidden variables as a means of predicting quantum processes which iirc was disproven except when said hidden variables were traced all the way to the beginning), then the current frame has enough information to fully describe all future states.
However, regardless of whether it is Markov or not, we, humans, need history because it enables us to generalize and approximate the future because we do not have the capacity to peak into the exact state of the universe at any given point.
While the process of making decision needs a short amount of time, the decision itself is conditioned on history and we need history to make predictions because we lack compute.
How is it not relevant to the process of choice when it fully determines all aspects of it including the outcome? It is as relevant to the process of choice as the choice itself is to the action it produces.