Yes, but streaming over cellular is mostly limited to support from Apple apps, not third party apps. Spotify might be the kind exception for that right now.
Apple Music and maybe Spotify will work off line. The cellular Apple Watch has 16GB of storage.
Shockingly, Pandora seems to have offline playback: https://9to5mac.com/2020/04/29/pandora-shows-spotify-how-to-..., so I'll chalk this one up to "Spotify spends all its time complaining instead of actually building the app"