Can't write any file into bucket only read after gcfuse
I deployed an instance of Wowza Streaming Engine on Google Cloud thank Made a bucket in Google Cloud Storage and mounted it all with GCFUSE. My bucket connected with success and I can see in it and list but I can't write any file with gsutil nor with any FTP even with RSA Key. I also tried gcsfuse -o allow_other but nothing change. What am I doing wrong please help
By default GCE VM instance has Cloud API access scopes
scope Storage
set to Read Only
. It's not enough to set it to Read Write
. To solve this issue you should set it to Full
by editing VM instance or using a gcloud command when the instance turned off.
Please have a look at my steps below:
- create a new VM instance and bucket (optional):
$ gcloud compute instances create instance-1 --zone=us-central1-a --machine-type=e2-medium --image=ubuntu-1804-bionic-v20201014 --image-project=ubuntu-os-cloud
$ gsutil mb gs://test-prj-fuse-bucket-1
- ssh into the VM instance:
$ gcloud compute ssh instance-1 --zone=us-central1-a
- install
gsfuse
by following this article:
instance-1:~$ export GCSFUSE_REPO=gcsfuse-`lsb_release -c -s`
instance-1:~$ echo "deb http://packages.cloud.google.com/apt $GCSFUSE_REPO main" | sudo tee /etc/apt/sources.list.d/gcsfuse.list
instance-1:~$ curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
instance-1:~$ sudo apt-get update
instance-1:~$ sudo apt-get install gcsfuse
- mount bucket to the VM instance by following this article:
instance-1:~$ mkdir ~/bucket
instance-1:~$ gcsfuse test-prj-fuse-bucket-1 ~/bucket
Using mount point: /home/username/bucket
Opening GCS connection...
Mounting file system...
File system has been successfully mounted.
instance-1:~$ mount | grep gcsfuse
gcsfuse on /home/username/bucket type fuse (rw,nosuid,nodev,relatime,user_id=1001,group_id=1002,default_permissions)
- check access to the bucket:
instance-1:~$ cd ~/bucket
instance-1:~/bucket$ touch test
touch: cannot touch 'test': Input/output error
so, we're not able to write with default Cloud API access scopes
.
- shutdown the VM instance and changes
Cloud API access scopes
with command:
$ gcloud beta compute instances set-scopes instance-1 --scopes=storage-full --zone=us-central1-a
or via Console. 7. start the VM instance, mount the bucket and check again:
$ gcloud compute ssh instance-1 --zone=us-central1-a
instance-1:~$ gcsfuse test-prj-fuse-bucket-1 ~/bucket
instance-1:~/bucket$ touch test
instance-1:~/bucket$ ls
test
everything works now.
Alternatively, you can solve this issue by using service account as it is described below :
By default, GCE instances run as "Compute Engine default service account" which only has read access to GCS objects.
To write to GCS from a GCE instance, do one of the following:
- Create a service account that has the roles/storage.objectAdmin role
- Add the roles/storage.objectAdmin role to your GCE default service account
To add the role to your GCE service account using the gcloud utility, run
gcloud iam service-accounts list
Then find the entry marked "Compute Engine default service account." It will have an email address like [email protected].
To add the role to the GCE service account, run these commands:
PROJECT_ID='' # Enter your GCP project ID SERVICE_ACCOUNT_EMAIL='' # Enter the service account email you found above
gcloud projects add-iam-policy-binding "$PROJECT_ID" \ --member "serviceAccount:${SERVICE_ACCOUNT_EMAIL}" \ --role roles/storage.objectAdmin
Have a look at this Google Group thread, in order to successfully mount the bucket as a file system using Cloud Storage Fuse. For more details about the subject check this document.