I use Colemak, so hjkl is very counter-intuitive for me, whereas the arrow keys make sense and aren't /that/ far away.
EDIT: http://news.ycombinator.com/item?id=3684515 has some good explanations. So does anyone have suggestions for non-QWERTY users? remap the keys that are in the HJKL space, leading to a cascade of keys moved around? Deal with the counter-intuitiveness of actually using HJKL?