This is the actual port which has the CI and Installer for Swift on Windows: [0]
If a Windows dev can target other platforms using Linux subsystem or containers, the only downside becomes an inability to target Windows desktops and servers, which are not hugely important targets outside of enterprise IT.
But there is of course a sense in which Swift is a general purpose language as opposed to something like SQL if that's what you mean.
Unfortunately, right now Swift is not (yet) a pragmatic choice for anything other than iOS/macOS apps.
I disagree that this is common. Most people are on web + browser + Office these days.