Get Download URL from file uploaded with Cloud Functions for Firebase

node.js firebase google-cloud-functions firebase-storage

After uploading a file in Firebase Storage with Functions for Firebase, I'd like to get the download url of the file.

I have this :

...

return bucket
    .upload(fromFilePath, {destination: toFilePath})
    .then((err, file) => {

        // Get the download url of file

    });

The object file has a lot of parameters. Even one named mediaLink. However, if I try to access this link, I get this error :

Anonymous users does not have storage.objects.get access to object ...

Can somebody tell me how to get the public download Url?

Thank you


Solution 1:

You'll need to generate a signed URL using getSignedURL via the @google-cloud/storage NPM module.

Example:

const gcs = require('@google-cloud/storage')({keyFilename: 'service-account.json'});
// ...
const bucket = gcs.bucket(bucket);
const file = bucket.file(fileName);
return file.getSignedUrl({
  action: 'read',
  expires: '03-09-2491'
}).then(signedUrls => {
  // signedUrls[0] contains the file's public URL
});

You'll need to initialize @google-cloud/storage with your service account credentials as the application default credentials will not be sufficient.

UPDATE: The Cloud Storage SDK can now be accessed via the Firebase Admin SDK, which acts as a wrapper around @google-cloud/storage. The only way it will is if you either:

  1. Init the SDK with a special service account, typically through a second, non-default instance.
  2. Or, without a service account, by giving the default App Engine service account the "signBlob" permission.

Solution 2:

This answer will summarize the options for getting the download URL when uploading a file to Google/Firebase Cloud Storage. There are three types of download URLS:

  1. signed download URLs, which are temporary and have security features
  2. token download URLs, which are persistent and have security features
  3. public download URLs, which are persistent and lack security

There are three ways to get a token download URL. The other two download URLs have only one way to get them.

From the Firebase Storage Console

You can get the download URL from Firebase Storage console:

enter image description here

The download URL looks like this:

https://firebasestorage.googleapis.com/v0/b/languagetwo-cd94d.appspot.com/o/Audio%2FEnglish%2FUnited_States-OED-0%2Fabout.mp3?alt=media&token=489c48b3-23fb-4270-bd85-0a328d2808e5

The first part is a standard path to your file. At the end is the token. This download URL is permanent, i.e., it won't expire, although you can revoke it.

getDownloadURL() From the Front End

The documentation tells us to use getDownloadURL():

let url = await firebase.storage().ref('Audio/English/United_States-OED-' + i +'/' + $scope.word.word + ".mp3").getDownloadURL();

This gets the same download URL that you can get from your Firebase Storage console. This method is easy but requires that you know the path to your file, which in my app is about 300 lines of code, for a relatively simple database structure. If your database is complex this would be a nightmare. And you could upload files from the front end, but this would expose your credentials to anyone who downloads your app. So for most projects you'll want to upload your files from your Node back end or Google Cloud Functions, then get the download URL and save it to your database along with other data about your file.

getSignedUrl() for Temporary Download URLs

getSignedUrl() is easy to use from a Node back end or Google Cloud Functions:

  function oedPromise() {
    return new Promise(function(resolve, reject) {
      http.get(oedAudioURL, function(response) {
        response.pipe(file.createWriteStream(options))
        .on('error', function(error) {
          console.error(error);
          reject(error);
        })
        .on('finish', function() {
          file.getSignedUrl(config, function(err, url) {
            if (err) {
              console.error(err);
              return;
            } else {
              resolve(url);
            }
          });
        });
      });
    });
  }

A signed download URL looks like this:

https://storage.googleapis.com/languagetwo-cd94d.appspot.com/Audio%2FSpanish%2FLatin_America-Sofia-Female-IBM%2Faqu%C3%AD.mp3?GoogleAccessId=languagetwo-cd94d%40appspot.gserviceaccount.com&Expires=4711305600&Signature=WUmABCZIlUp6eg7dKaBFycuO%2Baz5vOGTl29Je%2BNpselq8JSl7%2BIGG1LnCl0AlrHpxVZLxhk0iiqIejj4Qa6pSMx%2FhuBfZLT2Z%2FQhIzEAoyiZFn8xy%2FrhtymjDcpbDKGZYjmWNONFezMgYekNYHi05EPMoHtiUDsP47xHm3XwW9BcbuW6DaWh2UKrCxERy6cJTJ01H9NK1wCUZSMT0%2BUeNpwTvbRwc4aIqSD3UbXSMQlFMxxWbPvf%2B8Q0nEcaAB1qMKwNhw1ofAxSSaJvUdXeLFNVxsjm2V9HX4Y7OIuWwAxtGedLhgSleOP4ErByvGQCZsoO4nljjF97veil62ilaQ%3D%3D

The signed URL has an expiration date and long signature. The documentation for the command line gsutil signurl -d says that signed URLs are temporary: the default expiration is one hour and the maximum expiration is seven days.

I'm going to rant here that getSignedUrl never says that your signed URL will expire in a week. The documentation code has 3-17-2025 as the expiration date, suggesting that you can set the expiration years in the future. My app worked perfectly, and then crashed a week later. The error message said that the signatures didn't match, not that the download URL had expired. I made various changes to my code, and everything worked...until it all crashed a week later. This went on for more than a month of frustration.

Make Your File Publicly Available

You can set the permissions on your file to public read, as explained in the documentation. This can be done from the Cloud Storage Browser or from your Node server. You can make one file public or a directory or your entire Storage database. Here's the Node code:

var webmPromise = new Promise(function(resolve, reject) {
      var options = {
        destination: ('Audio/' + longLanguage + '/' + pronunciation + '/' + word + '.mp3'),
        predefinedAcl: 'publicRead',
        contentType: 'audio/' + audioType,
      };

      synthesizeParams.accept = 'audio/webm';
      var file = bucket.file('Audio/' + longLanguage + '/' + pronunciation + '/' + word + '.webm');
      textToSpeech.synthesize(synthesizeParams)
      .then(function(audio) {
        audio.pipe(file.createWriteStream(options));
      })
      .then(function() {
        console.log("webm audio file written.");
        resolve();
      })
      .catch(error => console.error(error));
    });

The result will look like this in your Cloud Storage Browser:

enter image description here

Anyone can then use the standard path to download your file:

https://storage.googleapis.com/languagetwo-cd94d.appspot.com/Audio/English/United_States-OED-0/system.mp3

Another way to make a file public is to use the method makePublic(). I haven't been able to get this to work, it's tricky to get the bucket and file paths right.

An interesting alternative is to use Access Control Lists. You can make a file available only to users whom you put on a list, or use authenticatedRead to make the file available to anyone who is logged in from a Google account. If there were an option "anyone who logged into my app using Firebase Auth" I would use this, as it would limit access to only my users.

Build Your Own Download URL with firebaseStorageDownloadTokens

Several answers describe an undocumented Google Storage object property firebaseStorageDownloadTokens. With this you can tell Storage the token you want to use. You can generate a token with the uuid Node module. Four lines of code and you can build your own download URL, the same download URL you get from the console or getDownloadURL(). The four lines of code are:

const uuidv4 = require('uuid/v4');
const uuid = uuidv4();

metadata: { firebaseStorageDownloadTokens: uuid }

https://firebasestorage.googleapis.com/v0/b/" + bucket.name + "/o/" + encodeURIComponent('Audio/' + longLanguage + '/' + pronunciation + '/' + word + '.webm') + "?alt=media&token=" + uuid);

Here's the code in context:

var webmPromise = new Promise(function(resolve, reject) {
  var options = {
    destination: ('Audio/' + longLanguage + '/' + pronunciation + '/' + word + '.mp3'),
    contentType: 'audio/' + audioType,
    metadata: {
      metadata: {
        firebaseStorageDownloadTokens: uuid,
      }
    }
  };

      synthesizeParams.accept = 'audio/webm';
      var file = bucket.file('Audio/' + longLanguage + '/' + pronunciation + '/' + word + '.webm');
      textToSpeech.synthesize(synthesizeParams)
      .then(function(audio) {
        audio.pipe(file.createWriteStream(options));
      })
      .then(function() {
        resolve("https://firebasestorage.googleapis.com/v0/b/" + bucket.name + "/o/" + encodeURIComponent('Audio/' + longLanguage + '/' + pronunciation + '/' + word + '.webm') + "?alt=media&token=" + uuid);
      })
      .catch(error => console.error(error));
});

That's not a typo--you have to nest firebaseStorageDownloadTokens in double layers of metadata:!

Doug Stevenson pointed out that firebaseStorageDownloadTokens is not an official Google Cloud Storage feature. You won't find it in any Google documentation, and there's no promise it will be in future version of @google-cloud. I like firebaseStorageDownloadTokens because it's the only way to get what I want, but it has a "smell" that it's not safe to use.

Why No getDownloadURL() from Node?

As @Clinton wrote, Google should make a file.getDownloadURL() a method in @google-cloud/storage (i.e., your Node back end). I want to upload a file from Google Cloud Functions and get the token download URL.

Solution 3:

Here's an example on how to specify the download token on upload:

const UUID = require("uuid-v4");

const fbId = "<YOUR APP ID>";
const fbKeyFile = "./YOUR_AUTH_FIlE.json";
const gcs = require('@google-cloud/storage')({keyFilename: fbKeyFile});
const bucket = gcs.bucket(`${fbId}.appspot.com`);

var upload = (localFile, remoteFile) => {

  let uuid = UUID();

  return bucket.upload(localFile, {
        destination: remoteFile,
        uploadType: "media",
        metadata: {
          contentType: 'image/png',
          metadata: {
            firebaseStorageDownloadTokens: uuid
          }
        }
      })
      .then((data) => {

          let file = data[0];

          return Promise.resolve("https://firebasestorage.googleapis.com/v0/b/" + bucket.name + "/o/" + encodeURIComponent(file.name) + "?alt=media&token=" + uuid);
      });
}

then call with

upload(localPath, remotePath).then( downloadURL => {
    console.log(downloadURL);
  });

The key thing here is that there is a metadata object nested within the metadata option property. Setting firebaseStorageDownloadTokens to a uuid-v4 value will tell Cloud Storage to use that as its public auth token.

Many thanks to @martemorfosis

Solution 4:

If you're working on a Firebase project, you can create signed URLs in a Cloud Function without including other libraries or downloading a credentials file. You just need to enable the IAM API and add a role to your existing service account (see below).

Initialize the admin library and get a file reference as your normally would:

import * as functions from 'firebase-functions'
import * as admin from 'firebase-admin'

admin.initializeApp(functions.config().firebase)

const myFile = admin.storage().bucket().file('path/to/my/file')

You then generate a signed URL with

myFile.getSignedUrl({action: 'read', expires: someDateObj}).then(urls => {
    const signedUrl = urls[0]
})

Make sure your Firebase service account has sufficient permissions to run this

  1. Go to the Google API console and enable the IAM API (https://console.developers.google.com/apis/api/iam.googleapis.com/overview)
  2. Still in the API console, go to the main menu, "IAM & admin" -> "IAM"
  3. Click edit for the "App Engine default service account" role
  4. Click "Add another role", and add the one called "Service Account Token Creator"
  5. Save and wait a minute for the changes to propagate

With a vanilla Firebase config, the first time you run the above code you'll get an error Identity and Access Management (IAM) API has not been used in project XXXXXX before or it is disabled.. If you follow the link in the error message and enable the IAM API, you'll get another error: Permission iam.serviceAccounts.signBlob is required to perform this operation on service account my-service-account. Adding the Token Creator role fixes this second permission issue.

Related

Correct format specifier to print pointer or address?
JavaScript: client-side vs. server-side validation
Programmatically get own phone number in iOS
Fastest way to flatten / un-flatten nested JSON objects
How to search and replace text in a file?
How do you clone an array of objects in JavaScript?
Wait until page is loaded with Selenium WebDriver for Python
Why is super.super.method(); not allowed in Java?
How to store custom objects in NSUserDefaults
How to iterate over a JavaScript object?
How to add parameters to HttpURLConnection using POST using NameValuePair
Boolean operators && and ||