I've heard a lot of people talk about keeping their fingers on the home row, and I more or less do that when I'm typing text like this, but for general navigation? I've sat next to a number of people who do it, and I get around my editor faster than they do. Based on this anecdotal evidence, I don't think moving my hand a short distance to reach another key is a significant speed cost.
(Each C-u is a power of 4. You can specify any number by typing it after the C-u, for example C-u 100 C-n to move down 100 lines.)
Sometimes I want to make a complicated motion in the horizontal and vertical axes at the same time in a context which is not well-formatted source code and hence has no good paragraph/word landing points. For this I move my hand slightly further and grab the mouse. It always amuses me watching people try to do it "the right way" and then spend 30 seconds trying to work out what sequence of buttons will get them there, then spend another 30 seconds remembering what they wanted to do when they got there.
I'm not trying to minimise the number of logical keystrokes, I'm trying to minimise the time and distraction of navigation. A few simple operations combined with key repeat seems to work really well. Lots of people seem to neglect the power offered by key repeat.
I'd love to meet somebody who could navigate source code faster than me and learn how they do it. When I do, I'll probably change my approach. However, I've worked with a lot of different people who used a lot of different approaches (yes, including that one where you display the line offset from point and type the number of every jump) and in the past decade I have not found any of them to be faster than me - although I have found a few who were roughly the same speed.
I'm sure something like this is possible in Emacs, but in vim I have it set up so that my current line shows the absolute line number and other lines show the relative line number.
If I'm eyeing a particular line that I want to jump to, I just read off its relative line number and hit j or k, according to whether it's down or up from where I'm at -- I'm there. Because I use relative line numbers, the number is never more than two digits.
For more precise navigation (jumping to a particular spot on a particular line), I use a plugin called EasyMotion, which basically lets me jump to any text object on the screen with a few key strokes.
I remap it to Caps Lock of course.
I keep escape as escape as well. Capslock is absolutely worthless to me.
These sort of systems are a great way to remap keys; I wish such remapping was exposed by operating systems in a nice stock way.