They feel that way because you know where your local maximum is. But that isn't the maximum for a whole organization.
If the people on the factory floor have to show up for work every day why should you get to stay home? If the business you're in has lots of gray legal risk, working from home means more things in writing. There are good interactions that can happen in bullpens and hallway and (I hate that I'm using this word) synergy from overheard conversations.
I don't think the office as it was (open floor plan) helped any one. I don't think that we all need to go in all the time. There is a happy medium that would give us "the best of all worlds" and we should be looking for that!