YT Music has a significantly modified UI. The entire UI is geared towards listening and discovery of music, artists, albums, tracks. If an artist is properly set up in the system, they will have a home page, and a list of their albums, songs, videos displayed. You can touch off a radio station playlist based on that artist, etc.
The difference in content is that many YT videos are simply unavailable inside YT Music, if they are not deemed "music". Therefore you won't be inundated by creators or comedy acts or newscasts, or anything but artists and music.
Music Premium has a feature that allows you to switch seamlessly between an "Audio" and "Video" version of your track. This is not magical but depends on both existing as YouTube entities. They do often match different audio tracks to a corresponding video. And if you're coming through an official "Artist" page then it may be more likely that you discover an official track, but I've never had trouble doing that in YT itself (it's usually a matter of "always only the second link in the search results").
Some artists configure "Playlists" and/or "Releases" tabs in YT proper, but indeed, YT Music is better organized so that you can discover those full albums and use them directly as a play queue.
I am fairly sure that there is no content exclusive to YT Music, only the structured presentation. All their tracks and videos are present and accounted for in YT itself; if you share a link/UUID you'll be able to find it. Conversely, as I mentioned, if it's not "Music" it won't be offered in that app. (This classification may be surprising -- I've found musical tracks and artists that aren't in there, such as the long-form BGM stuff. YT Music is for conventionally short tracks and vids.)