I noticed that I'm a lot less aware of random things that go on in my company. It's amazing how much stuff I used to pick up by just overhearing people and random passing conversations.
Another obvious drawback is that my work requires me to use proprietary hardware. I can use it remotely, but it's just not nearly as efficient to do so.