Working in an office and being incidentally exposed to how others work, debug, talk about code, collaborate with other engineers, etc. was incredibly important to my growth as a software engineer at my first job. There are a lot of things you see and learn that you would never think to ask a question about unless you overheard someone else say it or like see someone do a nifty command line trick while sitting next to them.

After that first job, I would say there hasn't been much advantage to non-remote.