Jenkins won't serve with CA signed certificate
Solution 1:
I believe you will need a PrivateKeyEntry.
You may have generated the CSR with open SSL instead of keytool.
You can try:
openssl pkcs12 -inkey /var/lib/jenkins/jenkins.key.pem -in /var/lib/jenkins/jenkins.crt.pem -export -out keys.pkcs12
keytool -importkeystore -srckeystore keys.pkcs12 -srcstoretype pkcs12 -destkeystore jenkins.jks
See this for a reference as well:
https://stackoverflow.com/questions/6252045/creating-a-keystore-from-private-key-and-a-public-key
If you look at your old keystore, the entry is Private versus Trusted.