Most of the cost is in CPU/GPU/RAM.
Look at a Dell or HP server configurator website to see why.
If you "only" need GPU, a craigslist search should find a less expensive 2 or 3yo server to plug your latest GPU into.
Also if you want to use big transistors, power dissipation also means you have to stay small or you will start fires.