I'd be less mad if they were up-front about that. In my current gig, I need to travel in 4-5 times a year, but they were clear about that at the very beginning. I don't think it's _necessary_, but it keeps them happy.
If this was known prior to you taking the job, I can hardly see anything to be mad about...I think this is what compromise might look like for some organizations. What you are looking for is "remote only" not "remote first".
Obviously, it'd be up front. And also, it'd be expected that you were in the same geographic region as the rest of your team. Hell, even the same time zone.
Springing that on unsuspecting people is slimy, anyways.