Media Replication

Description

It’s possible to add S3 replication between two S3 buckets in different regions and configure Shotgun to leverage it for faster access to media.

S3 Replication Diagram

Features

How it works

When S3 bucket replication is activated, users will be able download media files from the replica S3 bucket.

For each user, the S3 replication is activated by the Use S3 Replication field.

Value Behavior
no Never use replica S3 bucket (default)
yes Use replica S3 bucket when delay is over
auto Use replica S3 bucket when delay is over and the client IP is in IP Adresses for S3 replication range

Limitations

  • Only one replica S3 bucket can be configured
  • Only downloading from the replica bucket is supported
  • Configurable delay for new media to be replicated before being made available to users

Setup steps

  • Create the replica S3 bucket in a new AWS region. See Media Isolation
  • Update your existing Shotgun role policy to allow Shotgun to also access the replica bucket
  • Setup the replication rules on the primary S3 bucket. See How do I add a replication rule to an S3 bucket?
  • Setup a VPC + Direct Connect + S3 proxy in the new AWS region. See Media Traffic Isolation
  • Contact Shotgun Support to configure your site to use the new S3 replica bucket, providing the following information:
    • Replica Bucket Name
    • Replica Bucket Region
    • Replica S3 proxy URL

FAQ

Copy existing media

S3 replication only applies to media uploaded after the feature has been enabled. Existing media in the primary bucket must be copied to the replica bucket manually. You can simply use aws-cli sync or use Amazon S3 Batch Operations if you have a lot of media to achieve this.

Next Steps

See Fine Tuning to finalize your setup and optimize costs and security.

Go to Setup for an overview of the possible next steps.


Edit this document