How can I make my ISO_Level5_Shift arrow keys work in Java Swing GUIs?

Solution 1:

OK I found a solution. It is not really ideal, but it does get the desired behavior.

First, I dumped the complete state of my keyboard configuration using

$ xkbcomp $DISPLAY - > now.xkb

Then I found the lines

interpret Overlay1_Enable+AnyOfOrNone(all) {
    action= LockControls(controls=Overlay1);
};

and changed it to

interpret Overlay1_Enable+AnyOfOrNone(all) {
    action= SetControls(controls=Overlay1);
};

which prevents the modifier from being "sticky" ie it only applies while you are holding the key down.

Then I took the key that used to be my ISO_Level5_Shift:

key  <TAB> {
    type= "ONE_LEVEL",
    symbols[Group1]= [ ISO_Level5_Shift ]
};

and changed it to Overlay1_Enable:

key  <TAB> {
    type= "ONE_LEVEL",
    symbols[Group1]= [ Overlay1_Enable ]
};

Then for every key where I wanted the change to take effect, I added an overlay definition:

key <AD07> {
    type= "EIGHT_LEVEL",
    overlay1= <PGUP>,
    symbols[Group1]= [               g,               G,        asterisk,               G,           Prior,               G,     Greek_gamma,     Greek_GAMMA ]
};

Then a re-applied the whole thing with

$ xkbcomp now.xkb $DISPLAY

Useful documentation:

• XKB Page on ArchLinux Wiki
• Doug Palmer's XKB Guide

Solution 2:

I was having issue with this as well, here's my solution using caps lock key as the overlay switch to enable emacs/vim like navigation, hopefully it will help anyone looking to do something similar.

// Emacs like keys with CAPS as overlay switch.

// Using ISO_Level3_Shift or ISO_Level5_Shift would also make
// most applications work and would have been more flexible,
// however they don't work in Java Swing apps (e.g. IntelliJ IDEA)
// but using overlay works

// To enable, save this file as /usr/share/X11/xkb/symbols/emacs and run:
//
//   setxkbmap -option '' "emacs"
//
// However it may not persist and can get reset back to the default by other things.
// Alternatively, insert the following into /usr/share/X11/xkb/rules/evdev.xml
// ...
// <layoutList>
//   ...
//   <layout>
//     <configItem>
//       <name>emacs</name>
//       <shortDescription>en</shortDescription>
//       <description>English (US, Emacs)</description>
//       <languageList>
//         <iso639Id>eng</iso639Id>
//       </languageList>
//     </configItem>
//   </layout>
//   ...
// </layoutList>
// ...
// Then you should be able to choose 'English (US, Emacs)' in a keyboard preference
// GUI such as fcitx and have it persist.

default partial alphanumeric_keys
xkb_symbols "basic" {

    // Base keyboard functionality, using 'us' here
    // See definitions in /usr/share/X11/xkb/symbols
    // e.g. 'include "us(intl)"' where 'intl' is defined inside the 'us' file
    include "us"

    // Press Shift_L + Shift_R together as an alternative way to toggle Caps_Lock,
    // then turn CAPS key to enable overlay1 mode when and only when holding down.
    include "shift(both_capslock)"
    key <CAPS> {actions[Group1]=[SetControls(controls=overlay1)]};

    // Emacs like navigation keys when holding down <CAPS>
    // e.g. caps + n to go down
    key <AB05> {overlay1=<LEFT>}; // b
    key <AC04> {overlay1=<RGHT>}; // f
    key <AD10> {overlay1=<UP>  }; // p
    key <AB06> {overlay1=<DOWN>}; // n
    key <AC01> {overlay1=<HOME>}; // a
    key <AD03> {overlay1=<END> }; // e
    key <AC05> {overlay1=<ESC> }; // g

    // Emacs like editing keys when holding down <CAPS>
    // Redo/Undo only work with applications that understand the them
    key <AB10> {overlay1=<UNDO>}; // /
    key <UNDO> {[Undo, Redo]};    // Shift + / -> Redo
    key <AC03> {overlay1=<DELE>}; // d

    // VIM like navigation keys when holding down <CAPS>
    key <AC06> {overlay1=<LEFT>}; // h
    key <AC09> {overlay1=<RGHT>}; // l
    key <AC08> {overlay1=<UP>  }; // k
    key <AC07> {overlay1=<DOWN>}; // j

};