Maybe an an extra row above the numbers would work (Get rid of the current F1-F12 keys).
I'd also like to have ABCDEF and x keys next to the number pad, to make typing hexadecimal numbers easier. Maybe comma and space keys would be useful over there too.
For example, I use Colemak on a Kinesis Advantage and QWERTY on all flat keyboards. Since the physicality of they keyboards are so different I don't have any confusion while typing, though sometimes I'll mix up single character presses. That is, if I type a sentence I never have troubles, but if I want to hit just the 't' key I may mistype the first time. Other than that I can seamlessly switch between the two and my speeds are comparable across both layouts.
I occasionally get a few keypresses wrong when I first start typing after I've been using the mouse, but once I'm going I'm fairly accurate.