How can I copy files from a Google Cloud Compute Server to my local machine?

I'm trying to download all of the server files from Google Cloud Compute (running Linux Debian). I'm extremely unfamiliar with SSH and Google Cloud Compute as a whole. I'd prefer either copying all of the files (similar to a git pull request) or creating a zip and downloading. In short I need to copy all of the files on the remove Google Cloud server to my local machine.

Solution 1:

Try using gcloud compute scp:

gcloud compute scp 
  [[USER@]INSTANCE:]SRC [[[USER@]INSTANCE:]SRC …]
  [[USER@]INSTANCE:]DEST
  [--compress] [--dry-run] [--force-key-file-overwrite]
  [--plain] [--port=PORT] [--recurse] [--scp-flag=SCP_FLAG]
  [--ssh-key-file=SSH_KEY_FILE]
  [--strict-host-key-checking=STRICT_HOST_KEY_CHECKING]
  [--zone=ZONE]
  [GCLOUD_WIDE_FLAG …]
  [-h]

e.g.,

$ gcloud compute scp \
    my-instance-1:~/file-1 \
    my-instance-2:~/file-2 \
    ~/my-destination \
    --zone us-central2-a

As an alternative, you can also use (non-gcloud) sftp or scp; see this SO answer for details on the command line arguments you need to specify, which I'll quote here for simplicity:

sftp -o IdentityFile ~/.ssh/google_compute_engine user@host

To use any other SFTP tool, similarly provide to it the path to the SSH private key and it will be able to connect to the instance. However, in the case of gcloud, it can resolve the hostname to the public IP themselves; for all other tools, you'll need to find the IP manually and specify the IP address directly.

Solution 2:

(gcloud compute copy-files is deprecated now, hence gcloud compute scp is recommended)

Use gcloud compute scp as follows:

gcloud compute scp [[USER@]INSTANCE:]SRC [[[USER@]INSTANCE:]SRC …]
[[USER@]INSTANCE:]DEST [--compress] [--dry-run] [--force-key-file-overwrite]
[--plain] [--port=PORT] [--recurse] [--scp-flag=SCP_FLAG] 
[--ssh-key-file=SSH_KEY_FILE] [--strict-host-key-checking=STRICT_HOST_KEY_CHECKING]
[--zone=ZONE] [GCLOUD_WIDE_FLAG …]

For example:

$ gcloud compute scp example-instance:~/REMOTE-DIR ~/LOCAL-DIR \
      --zone us-central1-a

If the zone is already set (using gcloud config set compute/zone ZONE or through the environment variable, then you dont need to specify the zone above)

Also, gcloud compute scp does not copy recursively by default. We need to use the --recurse flag for it.

See the detailed documentation here.

Solution 3:

Maybe you can use SFTP (google used Filezilla in their docs). I am not too good with CLI and hence like the GUI options Filezilla and other provides.

Here are the steps at a high level:

1. Install and run the gcloud SDK ($ gcloud init)
2. Generate SSH key ($ gcloud compute ssh)
3. Setup the FTP client (install and add the key file)
4. Connect to the VM using the SFTP client

And here is documentation from Google.

Also, I have recorded this, it might be helpful:

https://www.youtube.com/watch?v=9ssfE6ODpak