I also work remotely and being able to work remote will definitely be a major factor in taking any other jobs in the future. For me it comes do to avoiding a terrible commute and the open office trend has made it more and more difficult actually get work done in the office.

I'm lucky enough to have a hobby where I can interact with people and get that social outlet. So for me there's not really been a downside. In fact it's more of an upside where my social interactions are with people I choose to be around doing something I want to do rather that with people I'm around by default doing something I have to do.