More cache means slower read/write operations to it, and less transistors for other things.