You also have to be online with good connectivity within the app to view the menu for removing an album from the device. Presumably, because offline access means access only if you are really, truly, completely offline.
Just opening the context menu of a song (to add it to the queue for example) now requires connectivity and does at least one round-trip to their servers. Unless you actually cut the connection (e.g. air plane mode), then it suddenly works fine
They should just have set the timeout to half a second and no one would have noticed