Looking for a CLI application that allows me to connect to server over SSH quickly

Solution 1:

okay, CLI means for me, it must be on the command line - no graphical interface wanted. The best linux ssh client still is ssh. You can use .ssh/config to define custom host definitions, which include username, ip, port, path to keyfile:

Host a
    Hostname 10.1.2.3
    Port     3333
    User     root
    IdentityFile /path/to/keyfile

that way, you can just execute ssh a, and you are connected. Those host definitions support all ssh_config options. Modern linux systems even support bash completion on the Alias Name defined in .ssh/config - from my point of view, you cannot get more comfort on the command line.