In graphs I saw it slowed somewhat.. but even if you are right, this is still waste of router memory since those same routes could be for much bigger v6 blocks instead of tiny v4 ones.

And also, in absolute terms it's currently tiny, ~ 2 M routes that can be represented in a couple of addrs worth of space each in line card memory when deciding just which interface to forward, and keeping in mind that only core routers that don't outsource transit to other upstream telcos have a need to keep this in memory.