I havent used any VR system, so take what Im about to say with a grain of salt, but Ive heard the actual VR headsets are much better in terms of immersiveness and quality than using phone headsets.
Go also has a Snapdragon 821 for ONLY VR. The GearVR was sharing resources with a ton of other garbage background processes that Go doesn't need to worry about. And the thermal performance is also way better.
Finally, you can get Oculus Go to run at 72Hz which is a way more comfortable experience than GearVR.
So you get a lot more than your phone can give you for 25% of the cost.
As you say, it's a good deal given the modest price point.
Sadly the Snapdragon is woefully underspecced in it's video decode capability - tops out at 4K which is less than ideal - 5K HEVC are already popular and a noticeable improvement for VR video.