Copy files from Google Compute Engine instance to Google Cloud Storage bucket

Solution 1:

The best way to do this is to SSH into the instance and use the gsutil command to copy files directly from the GCE instance to a GCS bucket.

Keep in mind the instance needs to have Google Cloud Storage "write scope" which is a setting you need to create when you first create the instance OR you can add later using a service account.

If you're using a machine image that was provided by Google, gsutil is already installed on the VM instance.

Example:

gsutil cp file1 file2 gs://bucket

If you have a lot of files to upload, you can parallelize via -m:

gsutil -m cp file1 file2 gs://bucket

If you want to recursively upload a directory, use -r:

gsutil cp -r dir1 gs://bucket

See the docs for gsutil cp for more information.