Map keyboard arrows to shortcuts

Not exactly what you're asking for, but nearly...

You can remap X keyboard settings using the xmodmap command. You can see the current settings by running xmodmap -pke

Each keycode (which is a value associated with every key on your keyboard) can have up to 8 keysyms (which is essentially the output) attached to it. They are as follows:

  1. When pressed on its own
  2. When pressed with shift
  3. When pressed with Mode switch enabled (seems to be obsolete)
  4. When pressed with Mode switch and shift (seems to be obsolete)
  5. When pressed with Alt-Gr (aka ISO_Level3_Shift in xmodmap terms)
  6. When pressed with Alt-Gr and shift

So currently my "h" key looks like:

h H h H hstroke Hstroke

where the Hstroke keysym produces Ħ.

Unless you're using Alt-Gr to input non-ascii characters, you could remap h, j, k and l with Alt-Gr to do what you want. This would be:

xmodmap -e 'keycode 43 = h H h H Left Hstroke'
xmodmap -e 'keycode 44 = j J j J Up J'
xmodmap -e 'keycode 45 = k K k K Down ampersand'
xmodmap -e 'keycode 46 = l L l L Right Lstroke'

The Alt-Gr with shift entries aren't strictly necessary of course, I'm just preserving the values that already exist. You can find the keycode values by running the xev command from a terminal. xev displays X events, so you can press a key and find out information about it.

To make this setting permanent, you can add the following to .Xmodmap in your home directory

keycode 43 = h H h H Left Hstroke
keycode 44 = j J j J Up J
keycode 45 = k K k K Down ampersand
keycode 46 = l L l L Right Lstroke

Pressing Alt-Gr certainly isn't as nice as pressing Alt when using hjkl, but you I think you may struggle getting just Alt to work because it is used for menu access. You may be able to use the information in this answer along with the tools mentioned by Fraekkert to bind e.g. Alt-Shift-h instead, which may suit you better.

It's also possibly worth noting that you can set vi mode in bash (or whichever shell you use) by using set -o vi. This gives you vi like behaviour when editing command lines and history. I don't believe this looks at your .vimrc though, so don't expect your key remappings to work.