Yes, absolutely. That's why the Apollo capsules and the Dragon spacecraft don't have them. The loads and heat involved are enormous. Even the Space Shuttle had pretty stubby wings, and the special tiles those wings needed to protect themselves were a constant source of high-cost maintenance (and ended up destroying one of the Shuttles and killing all its crew when they failed).
The wings on SpaceShipOne were pretty light and flimsy because they didn't need to be any stronger considering how slow SpaceShipOne's top speed was. It topped out at Mach 3, and you need to reach Mach 25 to achieve orbit. Wings strong enough to handle Mach 25 would be too heavy to ever reach Mach 25; Catch 22!