There is a Windows-native SSHFS client[1]. I have it installed on a Win10 VM so can confirm it "works" albeit I haven't needed to use it for anything so cannot confirm how well it works.
[1] https://github.com/Foreveryone-cz/win-sshfs I think is the currently maintained version but you're welcome to confirm that independently.