First there's the expense of the headset. Granted, it might come down a bit, but it's still a large expense on top of a console.
Second, there's the space that you need to play a Motion + VR game. It's not trivial for people to get that. A simple console + TV setup can fit in any tiny apartment around the world. Dedicating a whole room just to play a game? That's like asking people to move house just to play VR...
Even in my house which is fairly spacious, I'd still need to move around furniture everytime I want to play. It's just not going to happen.
Motionless VR, or at least, seated + arm movement VR, has a fair amount of potential however.