Send CSV from Google bucket to SFTP server (With RSA) using GCP Cloud function

Solution 1:

The problem may be, that you're using a Python; it's easier with gcloud compute scp, because one can provision keys on demand and buckets are accessible, too. The sequence of commands is:

gcloud config set ...
gcloud compute config-ssh
gcloud compute scp ...
gcloud compute ssh ...

For example; and this explains how to manage the keys (not store them at all).

While I wonder why you don't just fetch the CSV from bucket, on that server? Such situations are always push vs. pull. This may make sense, but not always. Choosing the wrong environment or building a solution more complex than required, this only adds handicap - and "Send CSV from Google bucket to SFTP server" doesn't permit to determine the actual purpose of doing that.

Solution 2:

Leaving aside everything except where to store a private key, I would suggest to store all secrets (for example, private key, pass phrase, etc) in a Secret Manager. The cloud function should get all those values using the Secret Manager API, and use them to establish a SFTP connection.