Glasses you can wear all day outside are a marvelous way to augment human capabilities. A bulky headset that only works with a desktop or game console is a niche product.
VR gaming is mostly compelling to people who are sexually unfulfilled. Apple builds products that are desired by attractive rich people, which is key to having premium brand pricing power and therefore high profit margins.