As I understand it, vi's author, Bill Joy, used a terminal that placed arrow symbols on hjkl because it did not have dedicated arrow keys.

Wikipedia backs up this story: http://en.wikipedia.org/wiki/HJKL_keys#HJKL_keys