It sounds like you got burned pretty bad on remote...
From my experiences, remote can be as good as an office for early innovation but just like office it has a unique set of challenges and sets a unique tempo. I think either way you pick one that works for your needs and you will be happy with the results as long as you keep an open mind.