nginxinc/nginx-s3-gateway

NGINX S3 Gateway

Introduction

This project provides a working configuration of NGINX configured to act as an authenticating and caching gateway for to AWS S3 or another S3 compatible service. This allows you to proxy a private S3 bucket without requiring users to authenticate to it. Within the proxy layer, additional functionality can be configured such as:

• Listing the contents of a S3 bucket
• Providing an authentication gateway using an alternative authentication system to S3
• Caching frequently accessed S3 objects for lower latency delivery and protection against S3 outages
• For internal/micro services that can't authenticate against the S3 API (e.g. don't have libraries available) the gateway can provide a means to accessing S3 objects without authentication
• Compressing objects (https://github.com/nginxinc/nginx-s3-gateway/blob/master/examples/gzip-compression, https://github.com/nginxinc/nginx-s3-gateway/blob/master/examples/brotli-compression) from gateway to end user
• Protecting S3 bucket from arbitrary public access and traversal
• Rate limiting S3 objects
• Protecting a S3 bucket with a https://github.com/nginxinc/nginx-s3-gateway/blob/master/examples/modsecurity
• Serving static assets from a S3 bucket alongside a dynamic application endpoints all in a single RESTful directory structure

All such functionality can be enabled within a standard NGINX configuration because this project is nothing other than NGINX with additional configuration that allows for proxying S3. It can be used as-is if the predefined configuration is sufficient, or it can serve as a base example for a more customized configuration.

If the predefined configuration does not meet your needs, it is best to borrow from the patterns in this project and build your own configuration. For example, if you want to enable SSL/TLS and compression in your NGINX S3 gateway configuration, you will need to look at other documentation because this project does not enable those features of NGINX.

Usage

This project can be run as a stand-alone container or as a Systemd service. Both modes use the same NGINX configuration and are functionally equal in terms features. However, in the case of running as a Systemd service, other services can be configured that additional functionality such as https://certbot.eff.org/ for https://letsencrypt.org/ support.

Getting Started

Refer to the https://github.com/nginxinc/nginx-s3-gateway/blob/master/docs/getting_started.md for how to build and run the gateway.

Configuration

The following environment variables are used to configure the gateway when running as a Container or as a Systemd service.

• ALLOW_DIRECTORY_LIST - Enable directory listing - either true or false
• AWS_SIGS_VERSION - AWS Signatures API version - either 2 or 4
• DNS_RESOLVERS - (optional) DNS resolvers (separated by single spaces) to configure NGINX with
• S3_ACCESS_KEY_ID - Access key
• S3_BUCKET_NAME - Name of S3 bucket to proxy requests to
• S3_DEBUG - Flag (true/false) enabling AWS signatures debug output (default: false)
• S3_REGION - Region associated with API
• S3_SECRET_KEY - Secret access key
• S3_SERVER_PORT - SSL/TLS port to connect to
• S3_SERVER_PROTO - Protocol to used connect to S3 server - http or https
• S3_SERVER - S3 host to connect to
• S3_STYLE - The S3 host/path method - virtual, path or default. virtual is the method that that uses DNS-style bucket+hostname:port. This is the default value. path is a method that appends the bucket name as the first directory in the URI's path. This method is used by many S3 compatible services. See this https://aws.amazon.com/blogs/aws/amazon-s3-path-deprecation-plan-the-rest-of-the-story/ for further information.
• PROXY_CACHE_VALID_OK - Sets caching time for response code 200 and 302
• PROXY_CACHE_VALID_NOTFOUND - Sets caching time for response code 404
• PROXY_CACHE_VALID_FORBIDDEN - Sets caching time for response code 403

If you are using https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2.html, you will need to omit the S3_ACCESS_KEY_ID and S3_SECRET_KEY variables from the configuration.

When running with Docker, the above environment variables can be set in a file with the --env-file flag. When running as a Systemd service, the environment variables are specified in the /etc/nginx/environment file. An example of the format of the file can be found in the https://github.com/nginxinc/nginx-s3-gateway/blob/master/settings.example file.

Development

Refer to the https://github.com/nginxinc/nginx-s3-gateway/blob/master/docs/development.md for more information about extending or testing the gateway.

License

All code include is licensed under the https://github.com/nginxinc/nginx-s3-gateway/blob/master/LICENSE.txt.