CentOS openLDAP cert trust issues

# LDAPTLS_CACERTDIR=/etc/ssl/certs/ ldapwhoami -x -ZZ -H ldaps://ldap.domain.tld
ldap_start_tls: Can't contact LDAP server (-1)
      additional info: TLS error -8172:Peer's certificate issuer has been marked as not trusted by the user.

# openssl s_client -connect ldap.domain.tld:636 -CApath /etc/ssl/certs
<... successful tls negotiation stuff ...>
    Compression: 1 (zlib compression)
    Start Time: 1349994779
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---

openssl seems to think the certificate is fine, but openldap's libraries (pam_ldap exhibits similar behavior, which is how I got on to this mess) disagree.
What am I doing wrong?

RHEL does not in fact provide anything that can be used as a 'certificate directory' for CA trust purposes. For OpenSSL, a certificate directory - a 'CApath' - is a directory containing individual certificate files (in PEM format or OpenSSL's extended 'trusted certificate' format), with names in a specific format based on a hash of the certificate's subject name. Usually this is achieved by putting files with human-readable names and .pem extensions in a directory and running c_rehash on it (see man c_rehash). For GnuTLS since 3.3.6 (prior to that GnuTLS had no directory support), it's just a directory with PEM files in it; GnuTLS will try and load every file in the directory and succeed on anything PEM-ish (it can't handle OpenSSL's 'trusted certificate' format). I'm not honestly sure if NSS can actually use a directory full of individual certificate files as a trust root somehow, but OpenLDAP's documentation seems to suggest it can (but if the directory also contains an NSS database it'll give that priority). Regardless, RHEL doesn't have anything like a directory full of individual CA certificate files.

Debian and derivatives provide /etc/ssl/certs in this format; /etc/ssl/certs is the canonical trust store location on Debian, and IMO anything that provides it should basically lay it out like Debian's, as Debian's had that directory laid out in more or less the same way since like 1999. RHEL has a /etc/ssl/certs directory, but it is in not in this format - it doesn't contain any individual certificate files at all. You can't use it as a CApath. Honestly, on RHEL (and Fedora, and derivatives) that directory is basically a trap. Don't use it. (See https://bugzilla.redhat.com/show_bug.cgi?id=572725 and https://bugzilla.redhat.com/show_bug.cgi?id=1053882 for some background on why it exists in the first place, and how I'm trying to get it fixed). So I think you're right about what's going on, but wrong about the reason why. OpenLDAP isn't doing anything wrong, and it's not failing because "ca-bundle.trust.crt...is a Mozilla NSS cert/key database" (those are called cert8/9.db and key3/4.db, and the system-wide ones on RHEL live in /etc/pki/nssdb), it's just failing because /etc/ssl/certs is not usable as a 'certificate directory' at all.

RHEL doesn't provide anything usable as a CApath-style trust store anywhere else, either. RHEL's system trust store is provided as a single PEM bundle file (a 'CAfile' in OpenSSL terms), which can be found at /etc/pki/tls/certs/ca-bundle.crt and /etc/pki/tls/cert.pem. It can also be found at /etc/ssl/certs/ca-bundle.crt as /etc/ssl/certs is actually just a symlink to /etc/pki/tls/certs, but that location is not canonical and really shouldn't be used by anything ever. RHEL also provides a bundle in OpenSSL's 'trusted certificate' format as /etc/pki/tls/certs/ca-bundle.trust.crt.

The correct thing to do, as you figured out, is to use the bundle file the system provides. Your answer will work, but for the reasons mentioned above, I would strongly recommend TLS_CACERT=/etc/pki/tls/certs/ca-bundle.crt or TLS_CACERT=/etc/pki/tls/cert.pem over TLS_CACERT=/etc/ssl/certs/ca-bundle.crt.

(There's nothing remotely new in any of this, btw, but confusion on the interwebs is widespread. RH and derivatives have never provided a directory-full-of-certificates, ever. They have provided a bundle file since the year 2000. It was moved from /usr/share/ssl to /etc/pki/tls in 2005. Debian has had both /etc/ssl/certs as a CApath-style directory and /etc/ssl/certs/ca-certificates.crt as a bundle file more or less since the stone age.)

/etc/ssl/certs/ contains /etc/ssl/certs/ca-bundle.trust.crt as part of ca-certificates-2010.63-3.el6_1.5.noarch, which is a Mozilla NSS cert/key database. Inclusion of this file within TLS_CACERTDIR causes all other files to be ignored.

TLS_CACERTDIR
Specifies the path of a directory that contains Certificate Authority certificates in separate individual files. The TLS_CACERT is always used before TLS_CACERTDIR.` This parameter is ignored with GnuTLS.

When using Mozilla NSS, may contain a Mozilla NSS cert/key database. If contains a Mozilla NSS cert/key database and CA cert files, OpenLDAP will use the cert/key database and will ignore the CA cert files.`

However, openldap-2.4.23-26.el6_3.2.i686 doesn't seem to handle this properly.

Short Answer
Use LDAPTLS_CACERT=/etc/ssl/certs/ca-bundle.crt
(config file TLS_CACERT=/etc/ssl/certs/ca-bundle.crt)
This file is also included provided by ca-certificates-2010.63-3.el6_1.5.noarch.