The N64 also suffered from a margin problem. Game cartridges were sometimes $10-$20 wholesale for producers, while CDs were probably about 10-25 cents. And then they had to pay a hefty licensing fee to Nintendo on top of that.
What's odd to me, is that almost no one on the N64 chose to simply recreate 2D graphics from the SNES era with improved quality. A quality JRPG built with SNES-like graphics would have still sold just fine, but I think the belief back then was that everything had to be 3D and cutting edge to sell.