how to use ssh agent forwarding

First you have to invoke ssh-agent on your client to make it remember your key

ssh-agent -t 3600 ~/.ssh/private_key_rsa

(assuming that your key is stored in ~/.ssh/private_key_rsa, you can also leave out the -t 3600 if you want infinite lifetime)

then you simply ssh into one of your servers using the -A option

ssh -A server1

from there you will then be able to ssh into server2

ssh server2

If you do not want to specify the -A option everytime you can add the following to your ~/.ssh/config (on the client and optionally both servers)

Host server1
 ForwardAgent yes

Host server2
 ForwardAgent yes

This works for any number of servers. To keep the ~/.ssh/config short you can introduce wildcards e.g.

Host server?
  ForwardAgent yes

Forward server host to localhost :

ssh -L localhost:22:localhost:22 user@host

or

ssh -N -f -L serverhost:22:localhost:22 user@server1

After reading your question again.

You want to ssh into server1 :

ssh user@server1

Then you want to ssh into server2:

Into new terminal from client do:

ssh user@server1
ssh user@server2

Then you have 2 connections:

1. client to server 1
2. client to server 1 ==> server 2

If you want to have:

1. client to server 1
2. client to server 2 (With same key.)

Just do following command.

On client:

Use tmux or open 2 terminals

ssh user@server1

In new terminal:

ssh user@server2