1. Get a through hole supported usb connector instead of a pure smd one, since the latter likes to fall off.
2. I would avoid routing under the switches for durability, this might just be superstition though.
3. You can route the x and y of the matrix on different sides of the board to make things easier.
4. If you want to pick and place or wave solder it is probably easier to have the smd and through hole parts on different sides of the board.
5. No reason not to use the supported switch footprint, especially since you can connect your ground planes with the additional holes. (well, I guess space is a reason but still).
6. If you are going to assemble by hand, make the pads longer as needed. Probably especially on the microcontroller and diodes.
7. Solder in order of cost and success rate.
8. Always add switches, leds and connectors for debugging even if you don't intend to use them.
9. Some traces are quite close to the center hole, which might also be too small? Anyway, some margin is good for reliability/yield. (especially mixing 'technologies' e.g. holes/edges and traces).
Edit: Oh, I forgot. You might consider scripting the placement of the switches/diodes.