I seem to recall reading that the original IBM PC was still in use at United States National Weather Service outposts. Why fix it if it isn't broken?
Related: http://www.zdnet.com/article/a-23-year-old-windows-3-1-syste...
The oldest box I've personally seen runs Windows 3.1 it is a specialized measurement instrument (Microwave radar) with a hardware card that runs in an ISA slot.
There are also a fair number of RS/6000 Unix boxes which are getting pretty long in the teeth.
For spare parts I've heard sometimes engineers buy things like Scsi hard disks off of ebay.