How to set maximum storage size limit per user in Google Cloud Storage?

Google Cloud Storage is a highly scalable, durable, and reliable object storage service. It is also cost-efficient as you only pay for what you use. A lot of applications use Google Cloud Storage to store and serve user-generated content, such as photos, videos, audio, and documents. One missing feature in Google Cloud Storage is the ability to set a maximum storage size limit per user or per directory. This blog will show you how to set a maximum storage size limit per user in Google Cloud Storage.

We will use Cloud Firestore to store information of all users and their storage usage in this article, but you can use any other database of your choice. A user's document in Cloud Firestore will look like this:

The storageLimit lets you set a different limit for every user e.g. more storage for premium users.

Before proceeding, make sure you have a Firebase project set up locally. If you don't have one, you can follow the official documentation to set up a Firebase project with Cloud Functions.

A. Create a Cloud Function to generate a PUT signed URL for the files to be uploaded.

1. A PUT signed URL is a URL that allows a user to upload a file to a specific location in Google Cloud Storage. We can set the maximum size of the file to be uploaded by setting the Content-Length header in the PUT signed URL and the file will be rejected if the file size is not equal to the header value.
2. The Cloud Function will be called by the client application along with the details of the files to be uploaded including the file name and the maximum size of the file.
3. Then we will check if user's available storage is greater than the total size of the files to be uploaded.
4. If the user has enough storage, we will generate a PUT signed URL for each file and return it to the client application else we will return an error.
5. The Cloud Function will also update the user's storage usage in Cloud Firestore.

Here is the code for the Cloud Function:

B. Upload the files to Google Cloud Storage.

1. The client application will call the Cloud Function and get the signed URLs.
2. We can then use fetch (or any other HTTP client) to upload the files to Google Cloud Storage using the signed URLs as shown below:

We can then use Cloud Storage Triggers onDelete() update a user's storage usage when a file is deleted.

With this, we have implemented a simple way to set a maximum storage size limit per user in Google Cloud Storage.

You can find the complete code with a VueJS client application on GitHub.

QnA:

1. Firebase recently announced cross-service security rules. Why can't we use that to read user's storage usage from Cloud Firestore directly and use Cloud Storage Triggers to update the storage?

   Security rules are evaluated for each file separately. So if a user has 5 MB of storage left and try to upload 10 files of 5 MB each, all the requests might pass before the Cloud Storage Triggers even manage to update the storage usage.

2. Why not upload the file through the Cloud Function itself?

   Cloud Functions have a 10 MB max HTTP request size so this won't work for large files. This might also increase the execution time of the Cloud Function as it will have to upload the file to Cloud Storage.

3. A user's storage usage is before the files are uploaded. What if the upload fails?

   Although this might be a rare case, we can store the file details in Firestore when updating the usage. Then we can periodically check if the files were uploaded. If not, adjust the storage usage accordingly.

If you have any questions or suggestions, feel free to reach out to me on Twitter.

Thanks for reading!