N64 also happens to be by far the heavier console to emulate in 5th-gen group. The unified memory architecture poses unique challenges for cycle accuracy, given that conflicting accesses by different peripherals are serialized in various ways, causing stalls, and also non deterministic behaviors as the signals cross different clock domains.
So the issue is in part that this level of detail hasn't been fully reverse engineered yet, but that's because there is no rush since the information wouldn't be usable anyway right now in an emulator.