intel/intel-optimized-nginx

Introduction

Nginx* is a high-performance HTTP/HTTPS and reverse proxy web server based on a BSD-like license. Nginx uses SSL/TLS to enhance web access security. Intel has introduced the ***-NI software solution which is based on 3rd generation Intel® Xeon® Scalable Processors (Codename Ice Lake/Whitley). It can effectively improve the security of web access. Intel_Asynch_Nginx is an Intel optimized version Nginx, used by Intel to support Async hardware and software acceleration for https.

The main software used in this solution are IPP ***graphy Library, Intel Multi-Buffer *** for IPsec Library (intel-ipsec-mb) and Intel® QuickAssist Technology (Intel® QAT), which provide batch submission of multiple SSL requests and parallel asynchronous processing mechanism based on the new instruction set, greatly improving the performance. Intel® QuickAssist Accelerator is a PCIe card that needs to be inserted into the PCIe slot in the server at the start.

Docker Image

• intel/intel-optimized-nginx:qathw:
• intel/intel-optimized-nginx:qatsw:

The docker images use Intel optimized async version of Nginx which can use QAT engine async operations to accelerate https performance from https://github.com/intel/asynch_mode_nginx

How to use this image

Generate certificates:

Create the TLS key and certificate for enabling encryption:

note: Replace <path> for the absolute path where you want to save the file:

openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -keyout <path>/key_rsa2048.key -out <path>/cert_rsa2048.crt -subj "/C=US/ST=State/L=Locality/O=Company/OU=Section/CN=company.com"

Run this image

note: To run qathw image, QAT In-tree Driver setup should be completed first (Detailed instruction is appended below)

note: Replace <path> for the absolute path where the files are saved.

• qathw: sudo docker run -it --cap-add=IPC_LOCK --security-opt seccomp=unconfined $(for i in ls /dev/vfio/*; do echo --device $i; done) -v <path>/key_rsa2048.key:/keys/key_rsa2048.key -v <path>/cert_rsa2048.crt:/certs/cert_rsa2048.crt intel/intel-optimized-nginx:qathw

• qatsw: sudo docker run -it -v <path>/key_rsa2048.key:/keys/key_rsa2048.key -v <path>/cert_rsa2048.crt:/certs/cert_rsa2048.crt intel-optimized-nginx:qatsw

Configure & run nginx

For information on the syntax of the nginx configuration files, see the official documentation (specifically the Beginner's Guide) and https://github.com/intel/asynch_mode_nginx#additional-information. Below is an example conf file for asynch_mode_nginx with qatengine enabled. Edit this file inside the container, then run nginx with nginx -c <location of the conf file>

conf
user root;
worker_processes 4;

# nginx_use_qat_engine=true
#start_qat_engine_block
load_module modules/ngx_ssl_engine_qat_module.so;
ssl_engine {
        use_engine qatengine;
        default_algorithms ALL;
        qat_engine {
                qat_offload_mode async;
                qat_notify_mode poll;
                qat_external_poll_interval 1;
        }
}
#end_qat_engine_block

worker_rlimit_nofile 1000000;

events {
        use epoll;
        worker_connections 8192;
        multi_accept on;
        accept_mutex on;
}

http {
        ssl_buffer_size 65536;
        include /usr/local/share/nginx/conf/mime.types;
        default_type application/octet-stream;
                sendfile on;
                access_log off;

        server {
                listen 4400 ssl reuseport backlog=131072 so_keepalive=off;
                tcp_nopush on;
                tcp_nodelay on;
                ssl_verify_client off;
                ssl_session_tickets off;
                access_log  off;
                lingering_close off;
                lingering_time 1;

                server_name example.com;
                ssl_asynch on;

                ssl_certificate  /certs/cert_rsa2048.crt;
                ssl_certificate_key /keys/key_rsa2048.key;

                ssl_session_timeout 300s;

                ssl_protocols TLSv1.3;

                ssl_conf_command Ciphersuites TLS_AES_256_GCM_SHA384:TLS_AES_128_GCM_SHA256:TLS_AES_128_CCM_SHA256:TLS_CHACHA20_POLY1305_SHA256;
                ssl_prefer_server_ciphers on;

                location / {
                        index index.html index.htm;
                }

        }
}

QAT In-tree Driver Setup For 4xxx Device

Check System Prerequisites

• Platform must have Intel® QuickAssist Technology QAT device such as "4xxx"
• QAT Physical Functions (PF's) can be determined as under:

shell
lspci -d 8086:4942
76:00.0 Co-processor: Intel Corporation Device 4942 (rev 40)
...

or

shell
lspci -d 8086:4940
6b:00.0 Co-processor: Intel Corporation Device 4940 (rev 40)
...

• Firmware must be available.

  check that these files exist:

  /lib/firmware/qat_4xxx.bin or /lib/firmware/qat_4xxx.bin.xz

  /lib/firmware/qat_4xxx_mmp.bin or /lib/firmware/qat_4xxx_mmp.bin.xz

  if not, download form:

  https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/plain/qat_4xxx.bin
  https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/plain/qat_4xxx_mmp.bin
  

  On updating these files run

  shell
  sudo rmmod qat_4xxx
  sudo modprobe qat_4xxx
  sudo dracut --force
  

  to update kernel modules and initramfs.

Required Kernel Information:

• Linux kernel v5.11+ (This is for ***, for compression use v5.17+)
• Fedora 34+ (for compression use 36+)
• RHEL 8.4+ (for compression use 9.0+)

BIOS Settings

• Intel VT-d and SR-IOV must be enabled in the platform (BIOS).

Grub Settings

in Fedora:

• sudo grubby --update-kernel=DEFAULT --args="intel_iommu=on vfio-pci.disable_denylist=1 iommu=pt default_hugepagesz=2M hugepagesz=2M hugepages=4096"
• Reboot system

Ubuntu:

• sudo vim /etc/default/grub
• move to GRUB_CMDLINE_LINUX add intel_iommu=on vfio-pci.disable_denylist=1 iommu=pt default_hugepagesz=2M hugepagesz=2M hugepages=4096
• sudo update-grub
• Reboot system

Install QATLib

Fedora 34+, using software package manager

shell
# Install QATLib
sudo dnf install -y qatlib-devel

# Add your user to qat group and re-login to make the change effective
sudo usermod -a -G qat `whoami`
sudo su -l $USER

# Make sure qat service is started properly and ready for use.
sudo systemctl stop qat.service
sudo systemctl enable qat.service
sudo systemctl restart qat.service
sudo systemctl status qat.service

Other Distributions: Building From Source

Fedora

shell
# Install dependencies
sudo dnf update -y
sudo dnf install -y gcc systemd-devel automake autoconf libtool
sudo dnf install -y openssl-devel zlib-devel yasm

Ubuntu

shell
# Install dependencies
sudo apt update -y
sudo apt install -y build-essential cmake g++ pkg-config wget make yasm nasm libboost-all-dev libnl-genl-3-dev zlib1g zlib1g-dev
apt install -y systemd m4 pkg-config libudev-dev libssl-dev autoconf libtool tar git

Build & install

shell
git clone https://github.com/intel/qatlib
cd qatlib
./autogen.sh
./configure --prefix=/usr --enable-service
make -j
sudo make install

# Make sure qat service is started properly and ready to use
sudo systemctl stop qat.service
sudo systemctl enable qat.service
sudo systemctl restart qat.service
sudo systemctl status qat.service

References

Please refer to the following links for detailed information on QAT In-tree Driver Setup For 4xxx Device

• https://github.com/intel/qatlib/blob/main/INSTALL

• https://github.com/intel/intel-device-plugins-for-kubernetes/blob/main/demo/openssl-qat-engine/Dockerfile

For more information on setting up PFs / VFs for specific QAT devices, please visit

• [***]

Please find 3rd party source codes on this repo

• https://github.com/intel/3rd-party-codes-for-qat-***-base-external-release

For release information

• https://github.com/intel/3rd-party-codes-for-qat-***-base-external-release/blob/main/Release.md