Its not terribly difficult to use swift on linux, theres a a working group trying make it easier for developers to use swift on the server [0]
I think the reason you'd pick it for a server language is because the strengths of the language and ecosystem meet your server requirements.
”The goal of the Swift project is to create the best available language for uses ranging from systems programming, to mobile and desktop apps, scaling up to cloud services”
And if “doesn’t have rich libraries and vibrant communities” were a decisive argument to not start using a new language, there never would be a new language again. I can see that, for some, using Swift on the server could be the right choice (for example because it makes it easier to share code between server and iOS apps)