kubernetes mariadb-galera cluster - bitnami helm chart - Readiness probe failed

The issue, as I found out, was related to how the NFS server was accessed. I used the NFS provisioner for Kubernetes here. That provisioner runs as root and that was the issue with the bitnami helm chart. In the securityContext of the chart the user and group are 1001:1001 meaning it does not run as root. My NFS server was also setup with root access. first I needed to change my NFS server share:

/path/to/share *(rw,sync,no_subtree_check,no_root_squash,no_all_squash,insecure,anonuid=1001,anongid=1001)

The important parts are the anonuid and anongid above. Then I created subfolders for each DB node that I intend to run. These sub-folders are the mountpoints in the pv later. I changed the owner of the whole share with chown 1001:1001 -R /path/to/share. That was it for the NFS server.

Next I created 3 pvs on the kubernetes cluster, one for each DB node.

apiVersion: v1
kind: PersistentVolume
metadata:
  name: db-persistent-storage-0
  labels:
    app: mariadb-galera-cluster
  namespace: database
spec:
  capacity:
    storage: 1Gi
  mountOptions:
    - nolock
  accessModes:
    - ReadWriteMany
  nfs:
    server: SERVERIP
    path: /path/to/share/subfolder1
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: db-persistent-storage-1
  labels:
    app: mariadb-galera-cluster
  namespace: database
spec:
  capacity:
    storage: 1Gi
  mountOptions:
    - nolock
  accessModes:
    - ReadWriteMany
  nfs:
    server: SERVERIP
    path: /path/to/share/subfolder2
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: db-persistent-storage-2
  labels:
    app: mariadb-galera-cluster
  namespace: database
spec:
  capacity:
    storage: 1Gi
  mountOptions:
    - nolock
  accessModes:
    - ReadWriteMany
  nfs:
    server: SERVERIP
    path: /path/to/share/subfolder3

Make sure you replace your SERVERIP and /path/to/share to reflect your own setup. Once created in the cluster, I was able to run the bitnami chart successfully. I still got the error message mentioned in my original post above

Readiness probe failed: mysqladmin: connect to server at 'localhost' failed error: 'Can't connect to local MySQL server through socket '/opt/bitnami/mariadb/tmp/mysql.sock' (2)' 
Check that mysqld is running and that the socket: '/opt/bitnami/mariadb/tmp/mysql.sock' exists!

but that must have been an issue with the timeouts I did not change. All 3 pods are up and running now and seem to work fine. If you follow these instructions make sure in the values.yaml that you adjust the persistent volume settings to reflect the label section that is mentioned in the PV above. Otherwise the helm chart won't pick up these existing volumes and not create the nodes. For the examples above that section should look like:

persistence:
  ## If true, use a Persistent Volume Claim, If false, use emptyDir
  ##
  enabled: true
  ## selector can be used to match an existing PersistentVolume
  ##
  selector:
    matchLabels:
      app: mariadb-galera-cluster
  accessModes:
    - ReadWriteMany
  ## Persistent Volume size
  ##
  size: 1Gi

Thanks for all your help.