Add new options and docs for pybind11_add_module

See the documentation for a description of the options.
This commit is contained in:
Dean Moldovan 2016-12-16 22:58:37 +01:00 committed by Wenzel Jakob
parent b0f3885c95
commit 0cbec5c96e
4 changed files with 144 additions and 66 deletions

View File

@ -39,30 +39,88 @@ extension module can be created with just a few lines of code:
This assumes that the pybind11 repository is located in a subdirectory named This assumes that the pybind11 repository is located in a subdirectory named
:file:`pybind11` and that the code is located in a file named :file:`example.cpp`. :file:`pybind11` and that the code is located in a file named :file:`example.cpp`.
The CMake command ``add_subdirectory`` will import a function with the signature The CMake command ``add_subdirectory`` will import the pybind11 project which
``pybind11_add_module(<name> source1 [source2 ...])``. It will take care of all provides the ``pybind11_add_module`` function. It will take care of all the
the details needed to build a Python extension module on any platform. details needed to build a Python extension module on any platform.
The target Python version can be selected by setting the ``PYBIND11_PYTHON_VERSION``
variable before adding the pybind11 subdirectory. Alternatively, an exact Python
installation can be specified by setting ``PYTHON_EXECUTABLE``.
A working sample project, including a way to invoke CMake from :file:`setup.py` for A working sample project, including a way to invoke CMake from :file:`setup.py` for
PyPI integration, can be found in the [cmake_example]_ repository. PyPI integration, can be found in the [cmake_example]_ repository.
.. [cmake_example] https://github.com/pybind/cmake_example .. [cmake_example] https://github.com/pybind/cmake_example
For CMake-based projects that don't include the pybind11 pybind11_add_module
repository internally, an external installation can be detected -------------------
through `find_package(pybind11 ... CONFIG ...)`. See the `Config file
<https://github.com/pybind/pybind11/blob/master/tools/pybind11Config.cmake.in>`_
docstring for details of relevant CMake variables.
Once detected, and after setting any variables to guide Python and To ease the creation of Python extension modules, pybind11 provides a CMake
C++ standard detection, the aforementioned ``pybind11_add_module`` function with the following signature:
wrapper to ``add_library`` can be employed as described above (after
``include(pybind11Tools)``). This procedure is available when using CMake .. code-block:: cmake
>= 2.8.12. A working example can be found at [test_installed_module]_ .
pybind11_add_module(<name> [MODULE | SHARED] [EXCLUDE_FROM_ALL]
[NO_EXTRAS] [THIN_LTO] source1 [source2 ...])
This function behaves very much like CMake's builtin ``add_library`` (in fact,
it's a wrapper function around that command). It will add a library target
called ``<name>`` to be built from the listed source files. In addition, it
will take care of all the Python-specific compiler and linker flags as well
as the OS- and Python-version-specific file extension. The produced target
``<name>`` can be further manipulated with regular CMake commands.
``MODULE`` or ``SHARED`` may be given to specify the type of library. If no
type is given, ``MODULE`` is used by default which ensures the creation of a
Python-exclusive module. Specifying ``SHARED`` will create a more traditional
dynamic library which can also be linked from elsewhere. ``EXCLUDE_FROM_ALL``
removes this target from the default build (see CMake docs for details).
Since pybind11 is a template library, ``pybind11_add_module`` adds compiler
flags to ensure high quality code generation without bloat arising from long
symbol names and duplication of code in different translation units. The
additional flags enable LTO (Link Time Optimization), set default visibility
to *hidden* and strip unneeded symbols. See the :ref:`FAQ entry <faq:symhidden>`
for a more detailed explanation. These optimizations are never applied in
``Debug`` mode. If ``NO_EXTRAS`` is given, they will always be disabled, even
in ``Release`` mode. However, this will result in code bloat and is generally
not recommended.
As stated above, LTO is enabled by default. Some newer compilers also support
different flavors of LTO such as `ThinLTO`_. Setting ``THIN_LTO`` will cause
the function to prefer this flavor if available. The function falls back to
regular LTO if ``-flto=thin`` is not available.
.. _ThinLTO: http://clang.llvm.org/docs/ThinLTO.html
Configuration variables
-----------------------
By default, pybind11 will compile modules with the latest C++ standard
available on the target compiler. To override this, the standard flag can
be given explicitly in ``PYBIND11_CPP_STANDARD``:
.. code-block:: cmake
set(PYBIND11_CPP_STANDARD -std=c++11)
add_subdirectory(pybind11) # or find_package(pybind11)
Note that this and all other configuration variables must be set **before** the
call to ``add_subdiretory`` or ``find_package``. The variables can also be set
when calling CMake from the command line using the ``-D<variable>=<value>`` flag.
The target Python version can be selected by setting ``PYBIND11_PYTHON_VERSION``
or an exact Python installation can be specified with ``PYTHON_EXECUTABLE``.
For example:
.. code-block:: bash
cmake -DPYBIND11_PYTHON_VERSION=3.6 ..
# or
cmake -DPYTHON_EXECUTABLE=path/to/python ..
find_package vs. add_subdirectory
---------------------------------
For CMake-based projects that don't include the pybind11 repository internally,
an external installation can be detected through ``find_package(pybind11)``.
See the `Config file`_ docstring for details of relevant CMake variables.
.. code-block:: cmake .. code-block:: cmake
@ -72,27 +130,32 @@ wrapper to ``add_library`` can be employed as described above (after
find_package(pybind11 REQUIRED) find_package(pybind11 REQUIRED)
pybind11_add_module(example example.cpp) pybind11_add_module(example example.cpp)
.. [test_installed_module] https://github.com/pybind/pybind11/blob/master/tests/test_installed_module/CMakeLists.txt Once detected, the aforementioned ``pybind11_add_module`` can be employed as
before. The function usage and configuration variables are identical no matter
if pybind11 is added as a subdirectory or found as an installed package. You
can refer to the same [cmake_example]_ repository for a full sample project
-- just swap out ``add_subdirectory`` for ``find_package``.
When using a version of CMake greater than 3.0, pybind11 can .. _Config file: https://github.com/pybind/pybind11/blob/master/tools/pybind11Config.cmake.in
additionally be used as a special *interface library* following the
call to ``find_package``. CMake variables to guide Python and C++ Advanced: interface library target
standard detection should be set *before* ``find_package``. When ----------------------------------
``find_package`` returns, the target ``pybind11::pybind11`` is
available with pybind11 headers, Python headers and libraries as When using a version of CMake greater than 3.0, pybind11 can additionally
needed, and C++ compile definitions attached. This target is suitable be used as a special *interface library* . The target ``pybind11::pybind11``
for linking to an independently constructed (through ``add_library``, is available with pybind11 headers, Python headers and libraries as needed,
not ``pybind11_add_module``) target in the consuming project. A working and C++ compile definitions attached. This target is suitable for linking
example can be found at [test_installed_target]_ . to an independently constructed (through ``add_library``, not
``pybind11_add_module``) target in the consuming project.
.. code-block:: cmake .. code-block:: cmake
cmake_minimum_required(VERSION 3.0) cmake_minimum_required(VERSION 3.0)
project(example) project(example)
add_library(example MODULE main.cpp) find_package(pybind11 REQUIRED) # or add_subdirectory(pybind11)
find_package(pybind11 REQUIRED) add_library(example MODULE main.cpp)
target_link_libraries(example PRIVATE pybind11::pybind11) target_link_libraries(example PRIVATE pybind11::pybind11)
set_target_properties(example PROPERTIES PREFIX "${PYTHON_MODULE_PREFIX}" set_target_properties(example PROPERTIES PREFIX "${PYTHON_MODULE_PREFIX}"
SUFFIX "${PYTHON_MODULE_EXTENSION}") SUFFIX "${PYTHON_MODULE_EXTENSION}")
@ -111,6 +174,3 @@ example can be found at [test_installed_target]_ .
(``-fvisibility=hidden``) and .OBJ files with many sections on Visual Studio (``-fvisibility=hidden``) and .OBJ files with many sections on Visual Studio
(``/bigobj``). The :ref:`FAQ <faq:symhidden>` contains an (``/bigobj``). The :ref:`FAQ <faq:symhidden>` contains an
explanation on why these are needed. explanation on why these are needed.
.. [test_installed_target] https://github.com/pybind/pybind11/blob/master/tests/test_installed_target/CMakeLists.txt

View File

@ -6,7 +6,7 @@ set(CMAKE_MODULE_PATH "")
find_package(pybind11 CONFIG REQUIRED) find_package(pybind11 CONFIG REQUIRED)
message(STATUS "Found pybind11 v${pybind11_VERSION}: ${pybind11_INCLUDE_DIRS}") message(STATUS "Found pybind11 v${pybind11_VERSION}: ${pybind11_INCLUDE_DIRS}")
pybind11_add_module(test_cmake_build SHARED ../main.cpp) pybind11_add_module(test_cmake_build SHARED NO_EXTRAS ../main.cpp)
add_custom_target(check ${CMAKE_COMMAND} -E env PYTHONPATH=$<TARGET_FILE_DIR:test_cmake_build> add_custom_target(check ${CMAKE_COMMAND} -E env PYTHONPATH=$<TARGET_FILE_DIR:test_cmake_build>
${PYTHON_EXECUTABLE} ${PROJECT_SOURCE_DIR}/../test.py ${PROJECT_NAME}) ${PYTHON_EXECUTABLE} ${PROJECT_SOURCE_DIR}/../test.py ${PROJECT_NAME})

View File

@ -2,7 +2,7 @@ cmake_minimum_required(VERSION 2.8.12)
project(test_subdirectory_module CXX) project(test_subdirectory_module CXX)
add_subdirectory(${PYBIND11_PROJECT_DIR} pybind11) add_subdirectory(${PYBIND11_PROJECT_DIR} pybind11)
pybind11_add_module(test_cmake_build ../main.cpp) pybind11_add_module(test_cmake_build THIN_LTO ../main.cpp)
add_custom_target(check ${CMAKE_COMMAND} -E env PYTHONPATH=$<TARGET_FILE_DIR:test_cmake_build> add_custom_target(check ${CMAKE_COMMAND} -E env PYTHONPATH=$<TARGET_FILE_DIR:test_cmake_build>
${PYTHON_EXECUTABLE} ${PROJECT_SOURCE_DIR}/../test.py ${PROJECT_NAME}) ${PYTHON_EXECUTABLE} ${PROJECT_SOURCE_DIR}/../test.py ${PROJECT_NAME})

View File

@ -14,6 +14,7 @@ set(Python_ADDITIONAL_VERSIONS 3.7 3.6 3.5 3.4)
find_package(PythonLibsNew ${PYBIND11_PYTHON_VERSION} REQUIRED) find_package(PythonLibsNew ${PYBIND11_PYTHON_VERSION} REQUIRED)
include(CheckCXXCompilerFlag) include(CheckCXXCompilerFlag)
include(CMakeParseArguments)
function(select_cxx_standard) function(select_cxx_standard)
if(NOT MSVC AND NOT PYBIND11_CPP_STANDARD) if(NOT MSVC AND NOT PYBIND11_CPP_STANDARD)
@ -33,29 +34,48 @@ function(select_cxx_standard)
endif() endif()
endfunction() endfunction()
# Internal: find the appropriate LTO flag for this compiler
macro(_pybind11_find_lto_flag output_var prefer_thin_lto)
if(${prefer_thin_lto})
# Check for ThinLTO support (Clang)
check_cxx_compiler_flag("-flto=thin" HAS_THIN_LTO_FLAG)
set(${output_var} $<${HAS_THIN_LTO_FLAG}:-flto=thin>)
endif()
if(NOT ${prefer_thin_lto} OR NOT HAS_THIN_LTO_FLAG)
if(NOT CMAKE_CXX_COMPILER_ID MATCHES "Intel")
# Check for Link Time Optimization support (GCC/Clang)
check_cxx_compiler_flag("-flto" HAS_LTO_FLAG)
set(${output_var} $<${HAS_LTO_FLAG}:-flto>)
else()
# Intel equivalent to LTO is called IPO
check_cxx_compiler_flag("-ipo" HAS_IPO_FLAG)
set(${output_var} $<${HAS_IPO_FLAG}:-ipo>)
endif()
endif()
endmacro()
# Build a Python extension module: # Build a Python extension module:
# pybind11_add_module(<name> [MODULE | SHARED] [EXCLUDE_FROM_ALL] source1 [source2 ...]) # pybind11_add_module(<name> [MODULE | SHARED] [EXCLUDE_FROM_ALL]
# [NO_EXTRAS] [THIN_LTO] source1 [source2 ...])
# #
function(pybind11_add_module target_name) function(pybind11_add_module target_name)
set(lib_type "MODULE") set(options MODULE SHARED EXCLUDE_FROM_ALL NO_EXTRAS THIN_LTO)
set(do_lto True) cmake_parse_arguments(ARG "${options}" "" "" ${ARGN})
set(exclude_from_all "")
set(sources "")
set(_args_to_try "${ARGN}") if(ARG_MODULE AND ARG_SHARED)
foreach(_ex_arg IN LISTS _args_to_try) message(FATAL_ERROR "Can't be both MODULE and SHARED")
if(${_ex_arg} STREQUAL "MODULE") elseif(ARG_SHARED)
set(lib_type "MODULE") set(lib_type SHARED)
elseif(${_ex_arg} STREQUAL "SHARED")
set(lib_type "SHARED")
elseif(${_ex_arg} STREQUAL "EXCLUDE_FROM_ALL")
set(exclude_from_all "EXCLUDE_FROM_ALL")
else() else()
list(APPEND sources "${_ex_arg}") set(lib_type MODULE)
endif() endif()
endforeach()
add_library(${target_name} ${lib_type} ${exclude_from_all} ${sources}) if(ARG_EXCLUDE_FROM_ALL)
set(exclude_from_all EXCLUDE_FROM_ALL)
endif()
add_library(${target_name} ${lib_type} ${exclude_from_all} ${ARG_UNPARSED_ARGUMENTS})
target_include_directories(${target_name} target_include_directories(${target_name}
PRIVATE ${PYBIND11_INCLUDE_DIR} # from project CMakeLists.txt PRIVATE ${PYBIND11_INCLUDE_DIR} # from project CMakeLists.txt
@ -86,7 +106,7 @@ function(pybind11_add_module target_name)
target_link_libraries(${target_name} PRIVATE "-undefined dynamic_lookup") target_link_libraries(${target_name} PRIVATE "-undefined dynamic_lookup")
if(${lib_type} STREQUAL "SHARED") if(ARG_SHARED)
# Suppress CMake >= 3.0 warning for shared libraries # Suppress CMake >= 3.0 warning for shared libraries
set_target_properties(${target_name} PROPERTIES MACOSX_RPATH ON) set_target_properties(${target_name} PROPERTIES MACOSX_RPATH ON)
endif() endif()
@ -96,23 +116,21 @@ function(pybind11_add_module target_name)
if(NOT MSVC) if(NOT MSVC)
# Make sure C++11/14 are enabled # Make sure C++11/14 are enabled
target_compile_options(${target_name} PUBLIC ${PYBIND11_CPP_STANDARD}) target_compile_options(${target_name} PUBLIC ${PYBIND11_CPP_STANDARD})
endif()
if(ARG_NO_EXTRAS)
return()
endif()
if(NOT MSVC)
# Enable link time optimization and set the default symbol # Enable link time optimization and set the default symbol
# visibility to hidden (very important to obtain small binaries) # visibility to hidden (very important to obtain small binaries)
string(TOUPPER "${CMAKE_BUILD_TYPE}" U_CMAKE_BUILD_TYPE) string(TOUPPER "${CMAKE_BUILD_TYPE}" U_CMAKE_BUILD_TYPE)
if (NOT ${U_CMAKE_BUILD_TYPE} MATCHES DEBUG) if (NOT ${U_CMAKE_BUILD_TYPE} MATCHES DEBUG)
# Check for Link Time Optimization support (GCC/Clang) # Link Time Optimization
check_cxx_compiler_flag("-flto" HAS_LTO_FLAG) if(NOT CYGWIN)
if(HAS_LTO_FLAG AND NOT CYGWIN) _pybind11_find_lto_flag(lto_flag ARG_THIN_LTO)
target_compile_options(${target_name} PRIVATE -flto) target_compile_options(${target_name} PRIVATE ${lto_flag})
endif()
# Intel equivalent to LTO is called IPO
if(CMAKE_CXX_COMPILER_ID MATCHES "Intel")
check_cxx_compiler_flag("-ipo" HAS_IPO_FLAG)
if(HAS_IPO_FLAG)
target_compile_options(${target_name} PRIVATE -ipo)
endif()
endif() endif()
# Default symbol visibility # Default symbol visibility