Linking problems due to symbols with abi::cxx11?

We recently caught a report because of GCC 5.1, libstdc++ and Dual ABI. It seems Clang is not aware of the GCC inline namespace changes, so it generates code based on one set of namespaces or symbols, while GCC used another set of namespaces or symbols. At link time, there are problems due to missing symbols.

If I am parsing the Dual ABI page correctly, it looks like a matter of pivoting on _GLIBCXX_USE_CXX11_ABI and abi::cxx11 with some additional hardships. More reading is available on Red Hat's blog at GCC5 and the C++11 ABI and The Case of GCC-5.1 and the Two C++ ABIs.

Below is from a Ubuntu 15 machine. The machine provides GCC 5.2.1.

$ cat test.cxx
#include <string>

std::string foo __attribute__ ((visibility ("default")));
std::string bar __attribute__ ((visibility ("default")));

$ g++ -g3 -O2 -shared test.cxx -o test.so

$ nm test.so | grep _Z3
...
0000201c B _Z3barB5cxx11
00002034 B _Z3fooB5cxx11

$ echo _Z3fooB5cxx11 _Z3barB5cxx11 | c++filt 
foo[abi:cxx11] bar[abi:cxx11]

How can I generate a binary with symbols using both decorations ("coexistence" as the Red Hat blog calls it)?

Or, what are the options available to us?

I'm trying to achieve an "it just works" for users. I don't care if there are two weak symbols with two different behaviors (std::string lacks copy-on-write, while std::string[abi:cxx11] provides copy-on-write). Or, one can be an alias for the other.

Debian has a boatload of similar bugs at Debian Bug report logs: Bugs tagged libstdc++-cxx11. Their solution was to rebuild everything under the new ABI, but it did not handle the corner case of mixing/matching compilers modulo the ABI changes.

In the Apple world, I think this is close to a fat binary. But I'm not sure what to do in the Linux/GCC world. Finally, we don't control how the distro's build the library, and we don't control what compilers are used to link an applications with the library.

Solution 1:

Disclaimer, the following is not tested in production, use at your own risk.

You can yourself release your library under dual ABI. This is more or less analogous to OSX "fat binary", but built entirely with C++.

The easiest way to do so would be to compile the library twice: with -D_GLIBCXX_USE_CXX11_ABI=0 and with -D_GLIBCXX_USE_CXX11_ABI=1. Place the entire library under two different namespaces depending on the value of the macro:

#if _GLIBCXX_USE_CXX11_ABI
#  define DUAL_ABI cxx11 __attribute__((abi_tag("cxx11")))
#else
#  define DUAL_ABI cxx03
#endif

namespace CryptoPP {
  inline namespace DUAL_ABI {
    // library goes here
  }
}

Now your users can use CryptoPP::whatever as usual, this maps to either CryptoPP::cxx11::whatever or CryptoPP::cxx03::whatever depending on the ABI selected.

Note, the GCC manual says that this method will change mangled names of everything defined in the tagged inline namespace. In my experience this doesn't happen.

The other method would be tagging every class, function, and variable with __attribute__((abi_tag("cxx11"))) if _GLIBCXX_USE_CXX11_ABI is nonzero. This attribute nicely adds [cxx11] to the output of the demangler. I think that using a namespace works just as well though, and requires less modification to the existing code.

In theory you don't need to duplicate the entire library, only functions and classes that use std::string and std::list, and functions and classes that use these functions and classes, and so on recursively. But in practice it's probably not worth the effort, especially if the library is not very big.

Solution 2:

Here's one way to do it, but its not very elegant. Its also not clear to me how to make GCC automate it so I don't have to do things twice.

First, the example that's going to be turned into a library:

$ cat test.cxx
#include <string>

std::string foo __attribute__ ((visibility ("default")));
std::string bar __attribute__ ((visibility ("default")));

Then:

$ g++ -D_GLIBCXX_USE_CXX11_ABI=0 -c test.cxx -o test-v1.o
$ g++ -D_GLIBCXX_USE_CXX11_ABI=1 -c test.cxx -o test-v2.o

$ ar cr test.a test-v1.o test-v2.o
$ ranlib test.a

$ g++ -shared test-v1.o test-v2.o -o test.so

Finally, see what we got:

$ nm test.a

test-v1.o:
00000004 B bar
         U __cxa_atexit
         U __dso_handle
00000000 B foo
0000006c t _GLOBAL__sub_I_foo
00000000 t _Z41__static_initialization_and_destruction_0ii
         U _ZNSsC1Ev
         U _ZNSsD1Ev

test-v2.o:
         U __cxa_atexit
         U __dso_handle
0000006c t _GLOBAL__sub_I__Z3fooB5cxx11
00000018 B _Z3barB5cxx11
00000000 B _Z3fooB5cxx11
00000000 t _Z41__static_initialization_and_destruction_0ii
         U _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEC1Ev
         U _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEED1Ev

And:

$ nm test.so

00002020 B bar
00002018 B __bss_start
00002018 b completed.7181
         U __cxa_atexit@@GLIBC_2.1.3
         w __cxa_finalize@@GLIBC_2.1.3
00000650 t deregister_tm_clones
000006e0 t __do_global_dtors_aux
00001ef4 t __do_global_dtors_aux_fini_array_entry
00002014 d __dso_handle
00001efc d _DYNAMIC
00002018 D _edata
00002054 B _end
0000087c T _fini
0000201c B foo
00000730 t frame_dummy
00001ee8 t __frame_dummy_init_array_entry
00000980 r __FRAME_END__
00002000 d _GLOBAL_OFFSET_TABLE_
000007dc t _GLOBAL__sub_I_foo
00000862 t _GLOBAL__sub_I__Z3fooB5cxx11
         w __gmon_start__
000005e0 T _init
         w _ITM_deregisterTMCloneTable
         w _ITM_registerTMCloneTable
00001ef8 d __JCR_END__
00001ef8 d __JCR_LIST__
         w _Jv_RegisterClasses
00000690 t register_tm_clones
00002018 d __TMC_END__
00000640 t __x86.get_pc_thunk.bx
0000076c t __x86.get_pc_thunk.dx
0000203c B _Z3barB5cxx11
00002024 B _Z3fooB5cxx11
00000770 t _Z41__static_initialization_and_destruction_0ii
000007f6 t _Z41__static_initialization_and_destruction_0ii
         U _ZNSsC1Ev@@GLIBCXX_3.4
         U _ZNSsD1Ev@@GLIBCXX_3.4
         U _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEC1Ev@@GLIBCXX_3.4.21
         U _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEED1Ev@@GLIBCXX_3.4.21