bitnamicharts/postgresql-ha

Bitnami Secure Images Helm chart for PostgreSQL HA

This PostgreSQL cluster solution includes the PostgreSQL replication manager, an open-source tool for managing replication and failover on PostgreSQL clusters.

https://www.postgresql.org/

Trademarks: This software listing is packaged by Bitnami. The respective trademarks mentioned in the offering are owned by the respective companies, and use of them does not imply any affiliation or endorsement.

TL;DR

console
helm install my-release oci://REGISTRY_NAME/REPOSITORY_NAME/postgresql-ha

Note: You need to substitute the placeholders REGISTRY_NAME and REPOSITORY_NAME with a reference to your Helm chart registry and repository.

Introduction

This https://github.com/kubernetes/helm chart installs https://www.postgresql.org/ with HA architecture in a Kubernetes cluster. Welcome to https://github.com/bitnami/charts/blob/main/CONTRIBUTING.md to Helm Chart for PostgreSQL HA.

This Helm chart has been developed based on https://github.com/bitnami/charts/tree/main/bitnami/postgresql chart but including some changes to guarantee high availability such as:

• A new deployment, service have been added to deploy Pgpool-II to act as proxy for PostgreSQL backend. It helps to reduce connection overhead, acts as a load *** for PostgreSQL, and ensures database node failover.
• Replacing bitnami/postgresql with bitnami/postgresql-repmgr which includes and configures repmgr. Repmgr ensures standby nodes assume the primary role when the primary node is unhealthy.

Differences between the PostgreSQL-HA and PostgreSQL Helm charts

There are two different ways to deploy a PostgreSQL cluster, using the PostgreSQL Helm chart or the PostgreSQL High Availability (HA) Helm chart. Both solutions provide a simple and reliable way to run PostgreSQL in a production environment. Keep reading to discover the differences between them and check which one better suits your needs.

• Both the PostgreSQL HA and the PostgreSQL chart configures a cluster with a primary/replica topology. The primary node has writing permissions while replication is on the replica nodes which have read-only permissions.
• The PostgreSQL HA Helm chart deploys a cluster with four nodes by default, one for Pgpool-II, and three PostgreSQL+Repmgr replicas. The PostgreSQL chart configures a cluster with two nodes by default (one primary and one replica).
• The PostgreSQL HA Helm chart uses Pgpool-II to handle the connection to the nodes. Pgpool-II is responsible to spread the queries among nodes.
• The PostgreSQL HA Helm chart includes a Repmgr module that ensures high-availability thanks to automatic membership control. If the primary is down, any of the replica nodes will be promoted as primary to avoid data loss.

The following diagram shows you the options you have for using Bitnami's PostgreSQL solutions in your deployments:

Before you begin

• Kubernetes 1.23+
• Helm 3.8.0+

Installing the Chart

To install the chart with the release name my-release:

console
helm install my-release oci://REGISTRY_NAME/REPOSITORY_NAME/postgresql-ha

Note: You need to substitute the placeholders REGISTRY_NAME and REPOSITORY_NAME with a reference to your Helm chart registry and repository. For example, in the case of Bitnami, you need to use REGISTRY_NAME=registry-1.docker.io and REPOSITORY_NAME=bitnamicharts.

Configuration and installation details

This section describes credentials, configuration, and other installation options.

Resource requests and limits

Bitnami charts allow setting resource requests and limits for all containers inside the chart deployment. These are inside the resources value (check parameter table). Setting requests is essential for production workloads and these should be adapted to your specific use case.

To make this process easier, the chart contains the resourcesPreset values, which automatically sets the resources section according to different presets. Check these presets in https://github.com/bitnami/charts/blob/main/bitnami/common/templates/_resources.tpl#L15. However, in production workloads using resourcesPreset is discouraged as it may not fully adapt to your specific needs. Find more information on container resource management in the https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/.

Prometheus metrics

This chart can be integrated with Prometheus by setting metrics.enabled to true. This will deploy a sidecar container with https://github.com/prometheus-community/postgres_exporter in all pods. It will also create metrics services that can be configured under the metrics.service section. These services will be have the necessary annotations to be automatically scraped by Prometheus.

Prometheus requirements

It is necessary to have a working installation of Prometheus or Prometheus Operator for the integration to work. Install the https://github.com/bitnami/charts/tree/main/bitnami/prometheus or the https://github.com/bitnami/charts/tree/main/bitnami/kube-prometheus to easily have a working Prometheus in your cluster.

Integration with Prometheus Operator

The chart can deploy ServiceMonitor objects for integration with Prometheus Operator installations. To do so, set the value metrics.serviceMonitor.enabled=true. Ensure that the Prometheus Operator CustomResourceDefinitions are installed in the cluster or it will fail with the following error:

text
no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"

Install the https://github.com/bitnami/charts/tree/main/bitnami/kube-prometheus for having the necessary CRDs and the Prometheus Operator.

Update credentials

Bitnami charts configure credentials at first boot. Any further change in the secrets or credentials require manual intervention. Follow these instructions:

• Update the user password following https://www.postgresql.org/docs/current/sql-alteruser.html
• Update the password secret with the new values (replace the SECRET_NAME, POSTGRES_PASSWORD and PASSWORD and REPMGR_PASSWORD placeholders)

shell
kubectl create secret generic SECRET_NAME --from-literal=postgres-password=POSTGRES_PASSWORD --from-literal=password=PASSWORD --from-literal=repmgr-password=REPMGR_PASSWORD --dry-run -o yaml | kubectl apply -f -

https://techdocs.broadcom.com/us/en/vmware-tanzu/bitnami-secure-images/bitnami-secure-images/services/bsi-doc/apps-tutorials-understand-rolling-tags-containers-index.html

It is strongly recommended to use immutable tags in a production environment. This ensures your deployment does not change automatically if the same tag is updated with a different image.

Bitnami will release a new chart updating its containers if a new version of the main container, significant changes, or critical vulnerabilities exist.

Use a different PostgreSQL version

To modify the application version used in this chart, specify a different version of the image using the postgresql.image.tag parameter and/or a different repository using the postgresql.image.repository parameter.

Use a volume for /dev/shm

When working with huge databases, /dev/shm can run out of space. A way to fix this is to use the postgresql.extraVolumes and postgresql.extraVolumeMounts values. In the example below, we set an emptyDir volume with 512Mb:

yaml
postgresql:
  extraVolumes:
    - name: dshm
      emptyDir:
        medium: Memory
        sizeLimit: 512Mi
  extraVolumeMounts:
    - name: dshm
      mountPath: /dev/shm

Configure the way how to expose Pgpool-II

• ClusterIP: Exposes the service on a cluster-internal IP. Choosing this value makes the service only reachable from within the cluster. Set service.type=ClusterIP to choose this service type.
• NodePort: Exposes the service on each Node's IP at a static port (the NodePort). You will be able to contact the NodePort service, from outside the cluster, by requesting NodeIP:NodePort. Set service.type=NodePort to choose this service type.
• Load***: Exposes the service externally using a cloud provider's load ***. Set service.type=LoadBalancer to choose this service type.

Adjust permissions of persistent volume mountpoint

As the images run as non-root by default, it is necessary to adjust the ownership of the persistent volumes so that the containers can write data into it.

By default, the chart is configured to use Kubernetes Security Context to automatically change the ownership of the volume. However, this feature does not work in all Kubernetes distributions. As an alternative, this chart supports using an init-container to change the ownership of the volume before mounting it in the final destination.

You can enable this init-container by setting defaultInitContainers.volumePermissions.enabled to true.

LDAP

LDAP support can be enabled in the chart by specifying the ldap. parameters while creating a release. The following parameters should be configured to properly enable the LDAP support in the chart.

• ldap.enabled: Enable LDAP support. Defaults to false.
• ldap.uri: LDAP URL beginning in the form ldap[s]://<hostname>:<port>. No defaults.
• ldap.basedn: LDAP base DN. No defaults.
• ldap.binddn: LDAP bind DN. No defaults.
• ldap.bindpw: LDAP bind password. No defaults.
• ldap.bslookup: LDAP base lookup. No defaults.
• ldap.nss_initgroups_ignoreusers: LDAP ignored users. root,nslcd.
• ldap.scope: LDAP search scope. No defaults.
• ldap.searchfilter: LDAP search filter. No defaults.
• ldap.searchmap: LDAP search map. No defaults.
• ldap.tls_reqcert: LDAP TLS check on server certificates. No defaults.

For example:

text
ldap.enabled="true"
ldap.uri="ldap://my_ldap_server"
ldap.basedn="dc=example\,dc=org"
ldap.binddn="cn=admin\,dc=example\,dc=org"
ldap.bindpw="admin"
ldap.bslookup="ou=group-ok\,dc=example\,dc=org"
ldap.nss_initgroups_ignoreusers="root\,nslcd"
ldap.scope="sub"
ldap.searchfilter="posixaccount"
ldap.searchmap="uid"
ldap.tls_reqcert="demand"

Next, login to the PostgreSQL server using the psql client and add the PAM authenticated LDAP users.

Note: Parameters including commas must be escaped as shown in the above example.

Securing traffic using TLS

The chart handles two main flows of traffic information:

• Connections between end-clients and Pgpool-II (sometimes referred to as frontend connections).
• Internal connections between Pgpool-II and PostgreSQL nodes (sometimes referred to as backend connections).

The Bitnami postgresql-ha chart allows configuring the securitization of both types of traffic using TLS.

Encrypt traffic between clients and Pgpool-II (frontend)

TLS for end-client connections can be enabled in the chart by specifying the pgpool.tls.* parameters when installing a release. Below you can find detailed information about these parameters:

• pgpool.tls.enabled: Enable TLS support. Defaults to false.
• pgpool.tls.certificatesSecret: Name of an existing secret that contains the certificates. No defaults.
• pgpool.tls.certFilename: Certificate filename. No defaults.
• pgpool.tls.certKeyFilename: Certificate key filename. No defaults.

For example:

• First, create a secret with the certificates files. You will need to generate previously the certificate files:

  console
  kubectl create secret generic pgpool-tls-secret --from-file=./cert.crt --from-file=./cert.key --from-file=./ca.crt
  

Note: Although certificate generation is out of the scope of this guide, bear in mind that PostgreSQL requires that server TLS certificates specify the actual DNS server name in the CN (Common Name) field.

• Then, install the chart using the following parameters:

  console
  pgpool.tls.enabled=true
  pgpool.tls.certificatesSecret="pgpool-tls-secret"
  pgpool.tls.certFilename="cert.crt"
  pgpool.tls.certKeyFilename="cert.key"
  

Note: Certificates permissions: Pgpool-II requires certain permissions on sensitive files (such as certificate keys) to start up. Due to an on-going https://github.com/kubernetes/kubernetes/issues/57923 regarding K8s permissions and the use of containerSecurityContext.runAsUser, an init container will adapt the permissions to ensure everything works as expected.

Enable client certificate authentication

When TLS is configured for frontend connections, the server can be configured to authenticate clients by verifying their provided TLS certificate is valid and trusted. Hence, the client will not be sent a password prompt.

You can enable this authentication feature additionally specifying the following parameter:

• postgresql.tls.certCAFilename: CA Certificate filename. No defaults.

  console
  $ psql --host postgresql-ha-pgpool -d "dbname=XXXXX user=YYYYYY sslcert=client.crt sslkey=client.key sslmode=require"
  psql (14.4)
  SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, bits: 256, compression: off)
  Type "help" for help.
  
  postgres=>
  

Clients using this method to authenticate will be required to provide a certificate with the CN (Common Name) field matching the requested database user name. Please, refer to https://www.postgresql.org/docs/current/auth-cert.html for further information.

Note: As with traditional password-based authentication, database users must exist in both Pgpool-II and PostgreSQL nodes and have the correct privileges to connect to a database. You may use the postgresql.initdbScripts and pgpool.customUsers properties to create them in advance.

Encrypt traffic between Pgpool-II and PostgreSQL nodes (backend)

TLS for backend connections can be enabled in the chart by specifying the postgresql.tls.* parameters while creating a release. Below you can find detailed information about these parameters:

• postgresql.tls.enabled: Enable TLS support. Defaults to false
• postgresql.tls.certificatesSecret: Name of an existing secret that contains the certificates. No defaults.
• postgresql.tls.certFilename: Certificate filename. No defaults.
• postgresql.tls.certKeyFilename: Certificate key filename. No defaults.

For example:

• First, create a secret with the certificates files. You will need to generate previously the certificate files:

  console
  kubectl create secret generic postgresql-tls-secret --from-file=./cert.crt --from-file=./cert.key --from-file=./ca.crt
  

• Then, install the chart using the following parameters:

  console
  postgresql.tls.enabled=true
  postgresql.tls.certificatesSecret="postgresql-tls-secret"
  postgresql.tls.certFilename="cert.crt"
  postgresql.tls.certKeyFilename="cert.key"
  

Note: Certificates permissions: PostgreSQL requires certain permissions on sensitive files (such as certificate keys) to start up. Due to an on-going https://github.com/kubernetes/kubernetes/issues/57923 regarding K8s permissions and the use of containerSecurityContext.runAsUser, an init container will adapt the permissions to ensure everything works as expected.

If you want to encrypt both frontend and backend traffics, you may use the same secret for Pgpool and PostgreSQL TLS configuration.

repmgr.conf / postgresql.conf / pg_hba.conf / pgpool.conf / pool_hba.conf files as configMap

This Helm chart also supports to customize the whole configuration file.

You can specify the Pgpool-II, PostgreSQL and Repmgr configuration using the pgpool.configuration, pgpool.poolHbaConfiguration, postgresql.configuration, postgresql.pgHbaConfiguration, and postgresql.repmgrConfiguration parameters. The corresponding files will be mounted as ConfigMap to the containers and it will be used for configuring Pgpool-II, Repmgr and the PostgreSQL server.

In addition to this option, you can also set an external ConfigMap(s) with all the configuration files. This is done by setting the postgresql.configurationCM and pgpool.configurationCM parameters. Note that this will override the previous options.

Allow settings to be loaded from files other than the default postgresql.conf

If you don't want to provide the whole PostgreSQL configuration file and only specify certain parameters, you can specify the extended configuration using the postgresql.extendedConf parameter. A file will be mounted as configMap to the containers adding/overwriting the default configuration using the include_dir directive that allows settings to be loaded from files other than the default postgresql.conf.

In addition to this option, you can also set an external ConfigMap with all the extra configuration files. This is done by setting the postgresql.extendedConfCM parameter. Note that this will override the previous option.

Initialize a fresh instance

The https://github.com/bitnami/containers/tree/main/bitnami/postgresql-repmgr image allows you to use your custom scripts to initialize a fresh instance. You can specify custom scripts using the initdbScripts parameter as dict so they can be consumed as a ConfigMap.

In addition to this option, you can also set an external ConfigMap with all the initialization scripts. This is done by setting the initdbScriptsCM parameter. Note that this will override the two previous options. If your initialization scripts contain sensitive information such as credentials or passwords, you can use the initdbScriptsSecret parameter.

The above parameters (initdbScripts, initdbScriptsCM, and initdbScriptsSecret) are supported in both StatefulSet by prepending postgresql or pgpool to the parameter, depending on the use case (see above parameters table).

The allowed extensions are .sh, .sql and .sql.gz in the Postgresql container while only .sh in the case of the Pgpool-II one.

+info: https://github.com/bitnami/containers/tree/main/bitnami/postgresql#initializing-a-new-instance and https://github.com/bitnami/containers/tree/main/bitnami/pgpool#initializing-with-custom-scripts

Use of global variables

In more complex scenarios, we may have the following tree of dependencies

text
                     +--------------+
                     |              |
        +------------+   Chart 1    +-----------+
        |            |              |           |
        |            --------+------+           |
        |                    |                  |
        |                    |                  |
        |                    |                  |
        |                    |                  |
        v                    v                  v
+-------+------+    +--------+------+  +--------+------+
|              |    |               |  |               |
| PostgreSQL HA |  | Sub-chart 1 |  | Sub-chart 2 |
|---------------|--|-------------|--|-------------|
+--------------+    +---------------+  +---------------+

The three charts below depend on the parent chart Chart 1. However, subcharts 1 and 2 may need to connect to PostgreSQL HA as well. In order to do so, subcharts 1 and 2 need to know the PostgreSQL HA credentials, so one option for deploying could be deploy Chart 1 with the following parameters:

text
postgresql.postgresqlPassword=testtest
subchart1.postgresql.postgresqlPassword=testtest
subchart2.postgresql.postgresqlPassword=testtest
postgresql.postgresqlDatabase=db1
subchart1.postgresql.postgresqlDatabase=db1
subchart2.postgresql.postgresqlDatabase=db1

If the number of dependent sub-charts increases, installing the chart with parameters can become increasingly difficult. An alternative would be to set the credentials using global variables as follows:

text
global.postgresql.postgresqlPassword=testtest
global.postgresql.postgresqlDatabase=db1

This way, the credentials will be available in all of the sub-charts.

Setting Pod's affinity

This chart allows you to set your custom affinity using the XXX.affinity paremeter(s). Find more information about Pod's affinity in the https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity.

As an alternative, you can use of the preset configurations for pod affinity, pod anti-affinity, and node affinity available at the https://github.com/bitnami/charts/tree/main/bitnami/common#affinities chart. To do so, set the XXX.podAffinityPreset, XXX.podAntiAffinityPreset, or XXX.nodeAffinityPreset parameters.

Backup and restore

To back up and restore Helm chart deployments on Kubernetes, you need to back up the persistent volumes from the source deployment and attach them to a new deployment using https://velero.io/, a Kubernetes backup/restore tool. Find the instructions for using Velero in https://techdocs.broadcom.com/us/en/vmware-tanzu/bitnami-secure-images/bitnami-secure-images/services/bsi-doc/apps-tutorials-backup-restore-deployments-velero-index.html.

FIPS parameters

The FIPS parameters only have effect if you are using images from the https://go-vmware.broadcom.com/contact-us.

For more information on this new support, please refer to the https://techdocs.broadcom.com/us/en/vmware-tanzu/bitnami-secure-images/bitnami-secure-images/services/bsi-doc/security-frameworks-FIPS-compliance.html.

Persistence

The data is persisted by default using PVC templates in the PostgreSQL StatefulSet. You can disable the persistence setting the persistence.enabled parameter to false. A default StorageClass is needed in the Kubernetes cluster to dynamically provision the volumes. Specify another StorageClass in the persistence.storageClass or set persistence.existingClaim if you have already existing persistent volumes to use.

Parameters

The following subsections list global, common, and component-specific parameters.

Global parameters

Note: the README for this chart is longer than the DockerHub length limit of 25000, so it has been trimmed. The full README can be found at https://techdocs.broadcom.com/us/en/vmware-tanzu/bitnami-secure-images/bitnami-secure-images/services/bsi-app-doc/apps-charts-postgresql-ha-index.html