Instead of thinking of this like a missile, instead think of it like a submarine. Submarines don't use star tracking, they use inertial navigation systems to inform dead reckoning. This accumulates error over time, but my impression is they work a lot better than you might expect.