They had geniuses working in their graphics side, and when they decided some 15 years ago that "Unix was dead" and WinNT was the future, those geniuses left for Nvidia and other places. SGI never recovered and ended up reselling supercomputers with ATI cards in them.
It wasn't pure suicide, Microsoft helped.
A common theme running through the history of the industry.
Later he got a comfy job at MS. Go figure...
Today their systems almost all use Linux.
Obviously, they didn't switch to NT, they switched to Linux.
Alias/Thomson division of SGI was merged with newly bought Wavefront into Alias|Wavefront - they've spent money on wavefront, and delegated AW to build a new platform, code name Maya, to compete with newly announced Sumatra. So, basically PowerAnimator and Explore products they already had lacked sales suddenly in awaiting of Maya (20,000$ list price + 20K for each of the modules, like artisan for example - really).
Then there was talks with Microsoft regarding OpenGL future - both of them tried to join powers and bring the world Fahrenheit... Microsoft though bought some small company making what was later known as DirectX.
Discreet (makers of high end compositing and editing applications) was bought by Autodesk and joined with their Kinetix group (formed by autodesk and yost group - makers of 3d studio) to consolidate new media division in Autodesk. Future plans were to migrate/and or/ develop new "systems" platform to linux. This materialized some years later, though not in full capacity. It showed its teeth when AVID strongarmed into PC market though.
With all this happening, SGI also introduced PC line of workstations with some innovative architecture designs (Cobalt) - which was ridiculously priced in heavy competition by then Intergraph Z workstations and Sun. There was a nice commercial tie in with movie lost in space - which, ironically was mostly made with 3dsmax (it's first feature credits)
ATI bought ART-X which made custom graphics chip for N64 among other things and Nvidia was hard pressed into buying talent around - mostly from advanced graphics division of SGI. This resulted in suffering in new lines of graphics workstations from SGI - like Octane2, Fuel and Tezro. Monster Reality engine was pretty much dead in the water by then - only brute force doubling of power.
And the cookie crumbled. SGI was almost fully vertically integrated hardware manufacturer - they had financial trouble. So they have ditched Alias|Wavefront as Alias to some investment fund (which refurbished it and sold it to Autodesk), MIPS division went from processor manufacturer to IP core provider only. R&D left to Nvidia, with appropriate patents (sold to Nvidia). PC division was dead. All that was left was HPC which sgi with its cray division focused on - unsuccessfully. Even though Altix platform is/was great - it was in direct collision with companies like Sun, RackSaver and most importantly IBM. IBM was still fully vertically integrated (almost still is) and had lots of money to throw around. It also had several strong HPC platforms (instead of one), and even grabbed a console processor market from MIPS/SGI in a feud before.
SGI is a sad story of bad decisions and bad timing. Had it kept media division and bought Discreet and Softimage instead of Cray, or even along with it. Had it scaled down MIPS to IP core only after lost war with IBM and reallocated resources better to align with Microsoft's vision of Talisman by keeping its R&D lab doing Talisman... Today we would have a company that would be a hybrid of a DCC giant and Nvidia. Now we only have a nice memory of turquoise and crimson wet dreams.