However, it is getting small inroads in the instrument clusters for Automobiles. (http://www.genivi.org/)
http://en.wikipedia.org/wiki/RTLinux
Edit: Wind River is the provider of VxWorks.
For systems which require a guaranteed response time, such as advanced robotics, this is a major deal-breaker.
From what I understand, Linux runs the rover simulator environment - known as RSVP. It consists of a DSL that is used to direct the rover's operations and a visualization environment. This way the ground crew can first plan and test the activities they want the rover to perform in this simulated environment and then upload the action plan to the rover - so overall Linux plays a pretty important part in the the rover tech stack.
Most probably the initial rovers ran VxWorks RTOS, because back then it was the best choice (even now it isn't a bad choice at all) - complex systems are built incrementally.