at least on iOS you can permanently delete the music app (permanently meaning system updates won't restore it)
do you remember about when that was?
(I also have to have downloaded music... thankfully I got that U2 album ages ago, ha.)
I actually bought this 10 minute long silent song on iTunes at the time, so my phone wouldn’t automatically play Weezer’s “Across the Sea” every time I connected my phone to the car: https://music.apple.com/us/album/a-a-a-a-a-very-good-song-si...
Now I recommend Plex. I use Prism player on my iPhone and I download my whole library from my MBP host. It's perfect.
The play button defaulting to Apple Music even when another sound source has been used more recently is one of the few things I dislike about iPhones.