If the cameras are further apart than your eyes, things will look smaller in size and have an exaggerated stereo effect also. If they are closer, the stereo effect is less pronounced and things appear larger in size.
It's actually the sense of scale I like the most in 3D (and VR). I don't care that much about things popping out at me on a 3D TV. I much prefer 3D that treats the screen like a window you look through, where the 3D effect comes mostly from things receding into the background. As an example, I loved the parts in Star Wars: The Force Awakens where you had characters shooting at things in the distance and the camera sat over their shoulder with them in the foreground.
There's also another issue here - you can't just take any game and 'upscale' it into 3D using a driver or such. The game's geometry needs to be properly modelled to scale for it to really work. There are other more egregious issues too - things like lens flares appearing in the wrong plane, because the game happens to be rendering them to a separate 2D plane.
These and other issues like them are the same reasons that you can't turn a non-VR game into a VR game with a driver and have it be as good as a native made-for-VR experience (although many games do come somewhat close, if you don't have a native VR version of them for comparison).
FWIW I think the 'average' IPD is 64mm.