Installing a third party application package on CentOS 6.4 fails due to missing dependencies libcrypto.so.10 and libssl.so.10

This is a Canonical Question about OpenSSL binary compatibility issues between Red Hat Enterprise Linux (and its derivatives) 6.4 and 6.5.

This issue applies to a wide variety of third party packages, not only the ones listed in the original question.

I had Percona 5.5 installed and am trying to upgrade to 5.6 but I'm running into unexpected issues and I'm stuck on how to resolve them.

I followed the instructions at http://www.percona.com/doc/percona-server/5.6/upgrading_guide_55_56.html

And removed the 5.5 packages, then ran the following command to upgrade:

yum install Percona-Server-server-56 Percona-Server-client-56

The errors I received back are:

Loaded plugins: fastestmirror, security
Loading mirror speeds from cached hostfile
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package Percona-Server-client-56.x86_64 0:5.6.15-rel63.0.519.rhel6 will be installed
--> Processing Dependency: libssl.so.10(libssl.so.10)(64bit) for package: Percona-Server-client-56-5.6.15-rel63.0.519.rhel6.x86_64
--> Processing Dependency: libcrypto.so.10(libcrypto.so.10)(64bit) for package: Percona-Server-client-56-5.6.15-rel63.0.519.rhel6.x86_64
--> Processing Dependency: Percona-Server-shared-56 for package: Percona-Server-client-56-5.6.15-rel63.0.519.rhel6.x86_64
---> Package Percona-Server-server-56.x86_64 0:5.6.15-rel63.0.519.rhel6 will be installed
--> Processing Dependency: libssl.so.10(libssl.so.10)(64bit) for package: Percona-Server-server-56-5.6.15-rel63.0.519.rhel6.x86_64
--> Processing Dependency: libcrypto.so.10(libcrypto.so.10)(64bit) for package: Percona-Server-server-56-5.6.15-rel63.0.519.rhel6.x86_64
--> Running transaction check
---> Package Percona-Server-client-56.x86_64 0:5.6.15-rel63.0.519.rhel6 will be installed
--> Processing Dependency: libssl.so.10(libssl.so.10)(64bit) for package: Percona-Server-client-56-5.6.15-rel63.0.519.rhel6.x86_64
--> Processing Dependency: libcrypto.so.10(libcrypto.so.10)(64bit) for package: Percona-Server-client-56-5.6.15-rel63.0.519.rhel6.x86_64
---> Package Percona-Server-server-56.x86_64 0:5.6.15-rel63.0.519.rhel6 will be installed
--> Processing Dependency: libssl.so.10(libssl.so.10)(64bit) for package: Percona-Server-server-56-5.6.15-rel63.0.519.rhel6.x86_64
--> Processing Dependency: libcrypto.so.10(libcrypto.so.10)(64bit) for package: Percona-Server-server-56-5.6.15-rel63.0.519.rhel6.x86_64
---> Package Percona-Server-shared-56.x86_64 0:5.6.15-rel63.0.519.rhel6 will be installed
--> Processing Dependency: libssl.so.10(libssl.so.10)(64bit) for package: Percona-Server-shared-56-5.6.15-rel63.0.519.rhel6.x86_64
--> Processing Dependency: libcrypto.so.10(libcrypto.so.10)(64bit) for package: Percona-Server-shared-56-5.6.15-rel63.0.519.rhel6.x86_64
--> Finished Dependency Resolution
Error: Package: Percona-Server-server-56-5.6.15-rel63.0.519.rhel6.x86_64 (percona)
       Requires: libcrypto.so.10(libcrypto.so.10)(64bit)
Error: Package: Percona-Server-shared-56-5.6.15-rel63.0.519.rhel6.x86_64 (percona)
       Requires: libcrypto.so.10(libcrypto.so.10)(64bit)
Error: Package: Percona-Server-server-56-5.6.15-rel63.0.519.rhel6.x86_64 (percona)
       Requires: libssl.so.10(libssl.so.10)(64bit)
Error: Package: Percona-Server-shared-56-5.6.15-rel63.0.519.rhel6.x86_64 (percona)
       Requires: libssl.so.10(libssl.so.10)(64bit)
Error: Package: Percona-Server-client-56-5.6.15-rel63.0.519.rhel6.x86_64 (percona)
       Requires: libssl.so.10(libssl.so.10)(64bit)
Error: Package: Percona-Server-client-56-5.6.15-rel63.0.519.rhel6.x86_64 (percona)
       Requires: libcrypto.so.10(libcrypto.so.10)(64bit)
 You could try using --skip-broken to work around the problem
** Found 3 pre-existing rpmdb problem(s), 'yum check' output follows:
perl-DBD-MySQL-4.022-1.el6.rfx.x86_64 has missing requires of libmysqlclient.so.16()(64bit)
perl-DBD-MySQL-4.022-1.el6.rfx.x86_64 has missing requires of libmysqlclient.so.16(libmysqlclient_16)(64bit)
perl-DBD-MySQL-4.022-1.el6.rfx.x86_64 has missing requires of mysql

I've tried everything from reinstalling openssl and openssl-devel which is installed correctly but it still doesn't work. Any ideas?

I'm running CentOS 6.4:

root@server01 [/]# cat /proc/version
Linux version 2.6.32-279.5.2.el6.x86_64 ([email protected]) (gcc version 4.4.6 20120305 (Red Hat 4.4.6-4) (GCC) ) #1 SMP Fri Aug 24 01:07:11 UTC 2012
root@server01 [/]#

The root cause of this issue is that Red Hat broke binary compatibility of their OpenSSL packages between 6.4 and 6.5, something that they promised that they would not do.

Resolving this problem is straightforward, but depending on the applications you may have deployed, may take some yelling at your application vendors. Be sure to reserve most of your ire for your Red Hat rep (if you have RHEL).

Cause

Red Hat upgraded the version of OpenSSL in EL6 from 1.0.0 to 1.0.1 in the 6.5 update, in order to resolve a years-old feature request to add elliptic curve cryptograhpy support. This package is no longer binary compatible, and programs that were built against OpenSSL 1.0.0 must be rebuilt from source against 1.0.1.

Unless you're installing third party applications, of course, which almost everyone does. Those have to be recompiled, too, and at this point most third parties have done so, and built new packages against 6.5. It is these packages that third parties are generally shipping today.

Resolution

Identify all of the impacted third party packages and contact the third-party package vendors for updates. Once updates are available for all packages, you can safely update your system to 6.5, installing the third party package updates at the same time, which will complete the resolution.

For packages installed through the package manager and yum repositories, this is trivial; simply attempting to upgrade and being able to do so without dependency problems means that the packages are ready.

For packages manually installed, you will need to check these yourself and apply whatever updates the vendors have provided. You should also pressure these vendors to supply proper RPM packages and yum repositories in these cases.

Most users can update to 6.5 with a command such as:

yum --disableexcludes=all --obsoletes update

RHEL users who have set a specific minor release must first set 6.5 as their release target before running the above update:

subscription-manager release --set=6.5

At this point you should be able to install the third party package(s) you were trying to install.

Other issues

Users of CentOS and other RHEL clones on certain VPS or cloud providers may find that they are unable to update to 6.5. The yum command will state that no packages are marked for update. So far I have seen this on Windows Azure and some low-end VPS providers.

In these cases, the provider of the CentOS image in use has modified /etc/yum.repos.d/CentOS-Base.repo in the image to point to repositories other than the official CentOS mirrors.

This can be resolved by either manually editing the repo file to restore the official CentOS mirrors, or by locating the official centos-release RPM on a CentOS mirror and reinstalling it. For example (this URL is only good today and may go out of date later; check your mirror first):

yum update http://mirror.centos.org/centos/6/os/x86_64/Packages/centos-release-6-5.el6.centos.11.2.x86_64.rpm

You can install PS 5.6 once you upgraded openssl to openssl-1.0.1e-15.el6.x86_64.rpm

For 6.4 we (I work for Percona) also have some custom made packages available: http://www.percona.com/downloads/Percona-Server-5.5-centos-6.4/