CMake: how create a single shared library from all static libraries of subprojects?

Solution 1:

OK, I figured it out: this is much more painful than it should be. Until very recently, people at Kitware didn't understand why anyone would ever want to create a DLL from static libs. Their argument is that there should always be source files in the main (e.g. top_project in my case) directory because it is effectively a project of its own. I see things differently & I need to break top_project into smaller subprojects which should not exist independently (i.e. there is no point in creating a full-blown project for them & add them using ExternalProject_Add). Besides, when I ship my shared library (for use, e.g. with a Java Native Interface), I don't want to ship dozens of shared libraries because that would amount to exposing the internal layout of my project. Anyway, having - I think - made a case for creating a shared library from static libraries, I'll proceed to the technical details.

In the CMakeLists.txt of subproject1 and subproject2, you should create your target using the OBJECT library feature (introduced in CMake 2.8.8):

add_library(${PROJECT_NAME} OBJECT ${SRC})

where SRC designates the list of source files (note that these should be set explicitly in the CMakeLists.txt file as it allows make to re-launch CMake when a modification of CMakeLists.txt is detected, e.g. when adding or removing a file)

In the top_project, add the subprojects using:

add_subdirectory(subproject1)
add_subdirectory(subproject2)

In order to see the symbols from the static library, use:

set(CMAKE_SHARED_LINKER_FLAGS "-Wl,--export-all-symbols")

You can then create the shared library using:

add_library(${PROJECT_NAME} SHARED $<TARGET_OBJECTS:subproject1>
                                   $<TARGET_OBJECTS:subproject2>)

I've found that any "normal" library (i.e. not object) needs to be added in a separate add_library command, otherwise it is simply ignored.

For executables, you can use:

add_executable(name_of_executable $<TARGET_OBJECTS:subproject1>
                  $<TARGET_OBJECTS:subproject2>)
set(LINK_FLAGS ${LINK_FLAGS} "-Wl,-whole-archive")
target_link_libraries(name_of_executable ${PROJECT_NAME}

I repeat that this only works as of version 2.8.8 of CMake. Just as well CMake manages the dependencies extremely well & is cross-platform because it's not much less painful than plain old Makefiles & certainly less flexible.

Solution 2:

My solution is simply to add /WHOLEARCHIVE, -all_load, or --whole-archive to the linker flags, so that when your main library is linked, all of the sub libraries are included, including all their symbols (the default behaviour is to only include symbols of the sub libraries that are used by the main library. For example:

Source Files

$ echo "void Func1() { }" > source1.cpp
$ echo "void Func2() { }" > source2.cpp
$ echo "void Func3() { }" > source3.cpp
$ echo "void Func4() { }" > source4.cpp

Naive CMakeLists.txt

cmake_minimum_required(VERSION 3.7)

# The 'sub' libraries, e.g. from an `add_subdirectory()` call.
add_library(sublib_a STATIC source1.cpp source2.cpp)
add_library(sublib_b STATIC source3.cpp source4.cpp)

# The main library that contains all of the sub libraries.
add_library(mainlib SHARED)

target_link_libraries(mainlib sublib_a sublib_b)

Running it (on OSX):

$ make VERBOSE=1
...
[100%] Linking CXX shared library libmainlib.dylib
/usr/local/Cellar/cmake/3.7.1/bin/cmake -E cmake_link_script CMakeFiles/mainlib.dir/link.txt --verbose=1
/Library/Developer/CommandLineTools/usr/bin/c++   -dynamiclib -Wl,-headerpad_max_install_names  -o libmainlib.dylib -install_name @rpath/libmainlib.dylib  libsublib_a.a libsublib_b.a 
[100%] Built target mainlib

$ nm libmainlib.dylib | grep Func
$

Correct CMakeLists.txt

Append this:

# By default, symbols provided by the sublibs that are not used by mainlib (which is all of them in this case)
# are not used. This changes that.
if (WIN32)
    set_target_properties(mainlib PROPERTIES
        LINK_FLAGS "/WHOLEARCHIVE"
    )
elseif (APPLE)
    set_target_properties(mainlib PROPERTIES
        LINK_FLAGS "-Wl,-all_load"
    )
else ()
    set_target_properties(mainlib PROPERTIES
        LINK_FLAGS "-Wl,--whole-archive"
    )
endif ()

Running it (note the extra -all_load):

$ make VERBOSE=1
[100%] Linking CXX shared library libmainlib.dylib
/usr/local/Cellar/cmake/3.7.1/bin/cmake -E cmake_link_script CMakeFiles/mainlib.dir/link.txt --verbose=1
/Library/Developer/CommandLineTools/usr/bin/c++   -dynamiclib -Wl,-headerpad_max_install_names -Wl,-all_load -o libmainlib.dylib -install_name @rpath/libmainlib.dylib  libsublib_a.a libsublib_b.a 
[100%] Built target mainlib

$ nm libmainlib.dylib | grep Func
0000000000001da0 T __Z5Func1v
0000000000001db0 T __Z5Func2v
0000000000001dc0 T __Z5Func3v
0000000000001dd0 T __Z5Func4v

Note that I've only actually tested -all_load so far, and /WHOLEARCHIVE is an MSVC 2015 option.

Solution 3:

Another way of doing it.

This way seems simpler, but I'm not sure how perfect it is:

https://stackoverflow.com/a/14347487/602340

Solution 4:

Another way of doing it is to provide the path of the source files and the header files of all your projects, and build them together to produce the .so . This is usually the recommended way, instead of creating the static libraries and then a shared library out of those.

Basically you should do the following:

FILE(GLOB subproject1_sources
  <sub_project1_lib_sources_dir>/file1.c
  <sub_project1_lib_sources_dir>/file2.c //... etc
)

FILE(GLOB subproject2_sources
  <sub_project2_lib_sources_dir>/file1.c
  <sub_project2_lib_sources_dir>/file2.c //... etc
)

FILE(GLOB topProject_sources
  <top_project_lib_sources_dir>/file1.c
  <top_project_lib_sources_dir>/file2.c //... etc
)

include_directories("<sub_project1_lib_sources_dir>")
include_directories("<sub_project2_lib_sources_dir>")
include_directories("<top_project_lib_sources_dir>") //should be "." if you're building from here

add_library(topProject SHARED ${topProject_sources} ${subproject1_sources} ${subproject2_sources})