Centos7 don't trust certificate issued by lets encrypt

I have apache nginx server running with https:// correctly configured with letsencrypt certificate. I can connect with firefox, chrome, ie. All of them reports connection as secure. However centos7 and ubuntu 14.04 reports certificate error:

wget https://gitlab.timeless.cz:8443

Resolving gitlab.timeless.cz (gitlab.timeless.cz)... 82.100.8.23
Connecting to gitlab.timeless.cz (gitlab.timeless.cz)|82.100.8.23|:8443... connected.
ERROR: cannot verify gitlab.timeless.cz's certificate, issued by ‘/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3’:
  Unable to locally verify the issuer's authority.

According https://www.ssllabs.com/ the page is correct.

Output of

openssl s_client -connect gitlab.timeless.cz:8443

is

CONNECTED(00000003)
depth=0 CN = gitlab.timeless.cz
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 CN = gitlab.timeless.cz
verify error:num=27:certificate not trusted
verify return:1
depth=0 CN = gitlab.timeless.cz
verify error:num=21:unable to verify the first certificate
verify return:1
---
Certificate chain
 0 s:/CN=gitlab.timeless.cz
   i:/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
---

but should be like this, which this is working for wget and curl)

CONNECTED(00000003)
depth=1 C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
verify error:num=20:unable to get local issuer certificate
verify return:0
---
Certificate chain
 0 s:/CN=bk1.timeless.cz
   i:/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
 1 s:/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
   i:/O=Digital Signature Trust Co./CN=DST Root CA X3
---

I have also some apache servers, which will work fine with lets-encrypt certificates. I mean for wget...

I don't know why this is working in browsers, but not in cli.

Edit:

I'm using gitlab installed from omnibus package on Ubuntu running bundled nginx on http port 8080 and https 8443. It had self signed certificates installed by default.

Then I installed apache (standard ports 80,443) and configured it using letsencrypt-auto utility. The https works fine, trusted by all.

First I tried to configure apache to terminate https and froward traffic to unencrypted nginx (port 8080). It basically worked, but I had problem to log in and git clone. Which makes it unusable.

Second I tried to link lets encrypt certificates generated for apache to nginx, but theres only .crt and .key files in /etc/gitlab/ssl. So I don't know how to include chain certificate. Strange is that browsers are happy without it but wget, git and curl fails.

Today I found

https://gitlab.com/gitlab-org/gitlab-recipes/blob/master/web-server/apache/gitlab-omnibus-apache24.conf

and managed to fix my first solution. This proffered for me because I get certificates updated automatically and can use standard port for gitlab and finally I can use one ip for multiple services.

Solving the second solution is to include chain into nginx, but it's giltab bundled, so normal config doesn't apply.


In REDHAT 7/ CentOS 7/ Oracle Linux 7:

Install the certificate in your enviroment.

  1. Download Active PEM certificate from: https://letsencrypt.org/certificates/ in /etc/pki/ca-trust/source/anchors
  2. Execute: sudo update-ca-trust

That's all!


I recently had an issue where a C7 system would not upgrade some packages because the remote cert was not trusted. I could verify this using wget. After some searching and head-scratching I decided to reinstall the ca-certificates package

yum reinstall ca-certificates

This solved my problem. Try reinstalling the ca-certificates package on the system you are running wget on.