isn't the difference between L1, L2 and L3 also because of the functionality, not just the size+speed? L1 is data and code. L2 is data only, per core. L3 is data, synchronized between cores.
With 32KB L1 at 64B line size, you can only cache 512 lines. Grouping them in larger buckets means less spillage as hot lines randomly end up in the same bucket.
L1 code and L1 data caches are separate.