There's so much more than clock frequency. The place silicon shines isn't so much it's material properties for producing individual switches - they're good, but there's better out there.

It's the raw manufacturability that silicon has. It's by far the easiest material to work with. It's abundant, cheap, and easy to obtain. It's got properties that lend it to easy junction doping and easy annealing. It's easy to deposit new layers on top of etched silicon. It's got a lattice that matches up nicely with the materials you'd want to use for contacts.

In short, the advantages of silicon aren't in it's ability to make transistors. It's in it's ability to make transistors while on a high volume assembly line.

It's going to be a while if we see whether graphene can match silicon for that.