I absolutely love how Sony then trampled over Nintendo by first using their experience from this collaboration to make their own PS, then basically cloned the N64 architecture in the form of PS2.
Sony didn't want anything to do with this project. This was all Ken Kutaragi's ambition, and after the Nintendo disaster he joined forces with the Sony Music side of the business (!) to form Sony Computer Entertainment, and was basically given full autonomy to launch his console. Sony Music knew retailers, knew how licensing worked, and knew how to take advantage of Nintendo's greed by undercutting their margins.
I don't know whether they had lower margins, but they used CDs, which were cheaper to produce than the ROM chips which were so far common for consoles (N64, SNES, Mega Drive, NES, ...). The price of those did go down a lot over time (early NES cartridges had 4 KB, early N64 cartridges 8 MB, a 2000 fold increase), but not quite quick enough apparently. High cartridge prices were not limited to Nintendo.
Were there really significant similarities between the N64 and the PS2 architectures? My understanding is that all the "first generation 3D consoles" had substantial weaknesses in their architecture, due to lack of experience, which were ironed out in the next generation.
They weren't identical, but they sure stood apart from others. Both used 64-bit MIPS CPUs, both had dedicated 128-bit SIMD cores driving their display list based GPUs. For a 1996 N64 having a SSE2 level SIMD was absolutely amazing. They're even similar from emulation standpoint: only in the last decade we've got enough CPU power to consistently emulate them at full speed.