In this guide we will use django-storages package to manage our static and media files. Let’s walkthrough the whole process on how to store django static and media files in aws s3.

S3 is a cloud based object storage service provided by AWS (Amazon Web Services), it stands for Simple Storage Service (S3).

Using AWS S3 to serve Static and Media files in Django makes sense since Django is not good at serving those and S3 provides a lot more feature around these.

Table of Contents

• S3 Bucket
  • Create a bucket
  • CORS
• IAM User
  • Create IAM User
  • Setting up Permission
  • Get the access key
• Setting Up Django
  • Installing necessary packages
  • Storage Backend
  • Update settings.py

S3 Bucket

Create a bucket

Let’s create a S3 Bucket where our files will be stored. Few things to keep in mind:

1. ACL must be enabled, so that we can control object permission from our Django

2. Uncheck all “Block Public Access settings for this bucket” settings and check the acknowledge box

CORS

After bucket is created successfully, now we need to update the CORS settings of the bucket so that if content is being served to different host then your browser might block it.

• Open the bucket

• Go to the Permissions tab

• At the end of the page there will be CORS settings

• Put the JSON given below, make sure to put the host which will use the S3 files, you can put multiples.

[
  {
    "AllowedHeaders": ["*"],
    "AllowedMethods": ["HEAD", "GET"],
    "AllowedOrigins": ["yourdomain.com"],
    "ExposeHeaders": []
  }
]

IAM User

An IAM user allow us to access AWS resources programmatically. We need IAM credentials to upload the files to S3.

Create IAM User

1. Go to IAM page in your AWS Console.

2. Go to Users then Add users.

3. Fill out the name that you want to give this user.

4. Go ahead and create the user, you can skip the permission.

Setting up Permission

1. Open IAM user which you intend on using.

2. Make sure you are on Permissions (default) tab.

3. From “Add permissions” dropdown select “Create inline policy”.

4. Select JSON policy editor instead of Visual and paste below JSON. Make sure to update the bucket name.

5. Give the policy appropriate name and save the changes.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "s3:*",
      "Resource": ["arn:aws:s3:::your-bucket-name", "arn:aws:s3:::your-bucket-name/*"]
    }
  ]
}

Get the access key

To use the IAM user in our django project we will need to generate access key.

1. Open user page, go to Security credentials tab

2. Scroll down to Access keys section and click on Create access key button

3. Select “Local code” from the use case menu

Setting Up Django

Installing necessary packages

We will be using django-storages package to manage our connection with AWS S3 and to upload our files. We will also need boto3 package to connect to AWS, which django-storages uses.

pip install boto3 django-storages

Storage Backend

django-storages provides S3Boto3Storage class for media storage and S3StaticStorage for static files.

We will subclass S3Boto3Storage to create our custom storage class so that we can customize certain things.

In your_project/custom_storage.py

from django.conf import settings
from storages.backends.s3boto3 import S3Boto3Storage

class StaticStorage(S3Boto3Storage):
    location = settings.STATICFILES_LOCATION
    default_acl = "public-read"
    querystring_auth = False

class MediaStorage(S3Boto3Storage):
    location = settings.MEDIAFILES_LOCATION
    default_acl = "public-read"
    file_overwrite = False

Update settings.py

We also need to update our settings.py file with media and static files settings:

STATICFILES_DIRS = [
    BASE_DIR / "static_assets",
]

AWS_S3_ACCESS_KEY_ID = "your_aws_key"
AWS_S3_SECRET_ACCESS_KEY = "your_aws_secret"
AWS_S3_REGION_NAME = "ap-south-1"  # change to your region
AWS_STORAGE_BUCKET_NAME = "your_bucket_name"
AWS_DEFAULT_ACL = None

AWS_S3_CUSTOM_DOMAIN = f"{AWS_STORAGE_BUCKET_NAME}.s3.amazonaws.com"

# static files
STATICFILES_LOCATION = "static"
STATIC_URL = f"https://{AWS_S3_CUSTOM_DOMAIN}/{STATICFILES_LOCATION}/"

# media files
MEDIAFILES_LOCATION = "media"
MEDIA_URL = f"https://{AWS_S3_CUSTOM_DOMAIN}/{MEDIAFILES_LOCATION}/"

# storage backends
STORAGES = {
    "default": {"BACKEND": "your_project.custom_storage.MediaStorage"},  # for media
    "staticfiles": {"BACKEND": "your_project.custom_storage.StaticStorage"},
}
AWS_S3_OBJECT_PARAMETERS = {
    "CacheControl": "max-age=2592000",
}

We have done all the setup for our static and media files to be uploaded to our S3 bucket. You can test it out in your django project. You can also checkout the tutorial on how to create file upload api using django rest framework.