They used to have an auto organize plugin which was doing that very well - but sadly it is no longer maintained (and no longer working on current versions). I did have quite a few things not correctly recognized with Plex which went away after going to Jellyfin and letting auto organize do its thing.

On the plus side, lack of auto organize made me properly setup the complete arr stack, which throws things into correctly named directories, does support batch renaming for existing stuff, and handles a lot of other stuff on top of that.