This is what I've been worried about. I have lens implants so I already have a fixed focus as well. The combination of the two would likely be a problem.
It's not crazy to think you could move the microdisplay position and get a virtual display at 6". There might be other optical consequences (aberrations, change in viewable area) but in principle it can work.
Even if you have absolutely no intention of ever buying one it would give you a free and easy way to find out if a headset type device would work well with your vision or just be totally incompatible.