config for using git via ssh on jump host

Solution 1:

You have to have some trust that bob can't be abused, but the simple answer is to run an ssh-agent, and use agent forwarding.

Start an agent on alice, add your PKCS11Provider using ssh-add -S then configuring agent forwarding in your config.

Host bob
  ForwardAgent yes
  HostName bob.example.org
  ProxyJump jump

See the man pages for ssh-agent, ssh-add, and ssh_config for details about the PKCS11 providers, and agent forwarding.

Minor, I think, complication being a jump host

Because of how ProxyJump works, it is not a complication, and literally doesn't matter at all. From the perspective of the ssh client on Alice, it acts exactly like you made a direct connection to Bob.