Terminal won't ask for passphrase and not stored in keychain

Your passphrase isn't being stored anywhere, but your decrypted private key is stored (in memory) by a process called ssh-agent (man page). This process, which OS X starts when it boots up, stores and manages private keys so they never have to be exposed to other processes that use SSH connections.

When you enter in your password, your computer decrypts your private key and ssh-agent gets a copy to hold on to until it is killed (e.g. on shutdown) or the key is manually removed using ssh-add (man page):

• ssh-add -l lists all currently held keys
• ssh-add -D forces ssh-agent to forget all currently held keys
• ssh-add ~/.ssh/newkey_rsa adds the private key ~/.ssh/newkey_rsa to ssh-agent.
• ssh-add -t 3600 ~/.ssh/newkey_rsa adds a new private key with an expiry time, so ssh-agent will only remember newkey_rsa for (say) 3600 seconds.

It may satisfy your concerns to know that your passphrase isn't stored anywhere. But if you really want your computer to prompt you for your passphrase every time, you could use ssh-add to make ssh-agent forget your key and then re-add it with a short expiry time.

Keep in mind that other solutions — like requiring a password to unlock your workstation when you're away from your desk — may also address your underlying security needs.