I think HJKL is fine because I primarily move my cursor down and I primarily use my index finger for things as opposed to the other three (non-thumbs).

On the other hand (no pun intended), I have countless hours invested into WASD since the Quake days[1] in the 90s. There I press W most often and most definitely with my middle finger. So the middle finger can definitely be trained to a very high level!

[1]: I started my shooter "career" with Wolfenstein 3D but since there was no mouse involved, I was using the arrow keys. On the IBM layout one would press up and down with the middle finger (which I suppose is sort of your point).