How to retrieve files to local machine when logged in to SSH?

Maybe zssh?

zssh (Zmodem SSH) is a program for interactively transferring files to a remote machine while using the secure shell (ssh). It is intended to be a convenient alternative to scp , allowing to transfer files without having to open another session and re-authenticate oneself.

zssh is an interactive wrapper for ssh

It uses the venerable rz, sz implementations of zmodem file transfer.

You could add the host information to your ~/.ssh/config, something like:

Host            foo
HostName        foo.baz.com
User            bar

And then you can do scp foo:~/path/to/file instead of scp bar@foo:~/path/to/file, or alternatively you can setup an alias or function to do the same thing.

As for your actual question, I don't know. scp creates a new connection to transfer files and doesn't use the existing ssh connection.