If you use vi mode then history can be navigated with hjkl.