Compile with hidden visibility always; set via cmake property rather than compiler flag

This updates the compilation to always apply hidden visibility to
resolve the issues with default visibility causing problems under debug
compilations.  Moreover using the cmake property makes it easier for a
caller to override if absolutely needed for some reason.

For `pybind11_add_module` we use cmake to set the property; for the
targets, we append to compilation option to non-MSVC compilers.
This commit is contained in:
Jason Rhinelander 2017-08-10 12:08:42 -04:00
parent a859dd67a2
commit 97aa54fefa
5 changed files with 63 additions and 31 deletions

View File

@ -91,6 +91,9 @@ if(NOT (CMAKE_VERSION VERSION_LESS 3.0)) # CMake >= 3.0
$<BUILD_INTERFACE:${PYTHON_INCLUDE_DIRS}>
$<INSTALL_INTERFACE:${CMAKE_INSTALL_INCLUDEDIR}>)
target_compile_options(pybind11 INTERFACE $<BUILD_INTERFACE:${PYBIND11_CPP_STANDARD}>)
if(NOT MSVC)
target_compile_options(pybind11 INTERFACE -fvisibility=hidden)
endif()
add_library(module INTERFACE)
add_library(pybind11::module ALIAS module)

View File

@ -135,22 +135,16 @@ has been executed:
Naturally, both methods will fail when there are cyclic dependencies.
Note that compiling code which has its default symbol visibility set to
*hidden* (e.g. via the command line flag ``-fvisibility=hidden`` on GCC/Clang) can interfere with the
ability to access types defined in another extension module. Workarounds
include changing the global symbol visibility (not recommended, because it will
lead unnecessarily large binaries) or manually exporting types that are
accessed by multiple extension modules:
Note that pybind11 code compiled with hidden-by-default symbol visibility (e.g.
via the command line flag ``-fvisibility=hidden`` on GCC/Clang), which is
required proper pybind11 functionality, can interfere with the ability to
access types defined in another extension module. Working around this requires
manually exporting types that are accessed by multiple extension modules;
pybind11 provides a macro to do just this:
.. code-block:: cpp
#ifdef _WIN32
# define EXPORT_TYPE __declspec(dllexport)
#else
# define EXPORT_TYPE __attribute__ ((visibility("default")))
#endif
class EXPORT_TYPE Dog : public Animal {
class PYBIND11_EXPORT Dog : public Animal {
...
};

View File

@ -74,13 +74,15 @@ removes this target from the default build (see CMake docs for details).
Since pybind11 is a template library, ``pybind11_add_module`` adds compiler
flags to ensure high quality code generation without bloat arising from long
symbol names and duplication of code in different translation units. The
additional flags enable LTO (Link Time Optimization), set default visibility
to *hidden* and strip unneeded symbols. See the :ref:`FAQ entry <faq:symhidden>`
for a more detailed explanation. These optimizations are never applied in
``Debug`` mode. If ``NO_EXTRAS`` is given, they will always be disabled, even
in ``Release`` mode. However, this will result in code bloat and is generally
not recommended.
symbol names and duplication of code in different translation units. It
sets default visibility to *hidden*, which is required for some pybind11
features and functionality when attempting to load multiple pybind11 modules
compiled under different pybind11 versions. It also adds additional flags
enabling LTO (Link Time Optimization) and strip unneeded symbols. See the
:ref:`FAQ entry <faq:symhidden>` for a more detailed explanation. These
latter optimizations are never applied in ``Debug`` mode. If ``NO_EXTRAS`` is
given, they will always be disabled, even in ``Release`` mode. However, this
will result in code bloat and is generally not recommended.
As stated above, LTO is enabled by default. Some newer compilers also support
different flavors of LTO such as `ThinLTO`_. Setting ``THIN_LTO`` will cause
@ -181,9 +183,8 @@ to an independently constructed (through ``add_library``, not
flags (i.e. this is up to you).
These include Link Time Optimization (``-flto`` on GCC/Clang/ICPC, ``/GL``
and ``/LTCG`` on Visual Studio). Default-hidden symbols on GCC/Clang/ICPC
(``-fvisibility=hidden``) and .OBJ files with many sections on Visual Studio
(``/bigobj``). The :ref:`FAQ <faq:symhidden>` contains an
and ``/LTCG`` on Visual Studio) and .OBJ files with many sections on Visual
Studio (``/bigobj``). The :ref:`FAQ <faq:symhidden>` contains an
explanation on why these are needed.
Embedding the Python interpreter

View File

@ -151,6 +151,33 @@ specifying a larger value, e.g. ``-ftemplate-depth=1024`` on GCC/Clang. The
culprit is generally the generation of function signatures at compile time
using C++14 template metaprogramming.
.. _`faq:hidden_visibility`:
"SomeClass declared with greater visibility than the type of its field SomeClass::member [-Wattributes]"
============================================================================================================
This error typically indicates that you are compiling without the required
``-fvisibility`` flag. pybind11 code internally forces hidden visibility on
all internal code, but if non-hidden (and thus *exported*) code attempts to
include a pybind type (for example, ``py::object`` or ``py::list``) you can run
into this warning.
To avoid it, make sure you are specifying ``-fvisibility=hidden`` when
compiling pybind code.
As to why ``-fvisibility=hidden`` is necessary, because pybind modules could
have been compiled under different versions of pybind itself, it is also
important that the symbols defined in one module do not clash with the
potentially-incompatible symbols defined in another. While Python extension
modules are usually loaded with localized symbols (under POSIX systems
typically using ``dlopen`` with the ``RTLD_LOCAL`` flag), this Python default
can be changed, but even if it isn't it is not always enough to guarantee
complete independence of the symbols involved when not using
``-fvisibility=hidden``.
Additionally, ``-fvisiblity=hidden`` can deliver considerably binary size
savings. (See the following section for more details).
.. _`faq:symhidden`:
@ -192,11 +219,14 @@ world. So we'll generally only want to export symbols for those functions which
are actually called from the outside.
This can be achieved by specifying the parameter ``-fvisibility=hidden`` to GCC
and Clang, which sets the default symbol visibility to *hidden*. It's best to
do this only for release builds, since the symbol names can be helpful in
debugging sessions. On Visual Studio, symbols are already hidden by default, so
nothing needs to be done there. Needless to say, this has a tremendous impact
on the final binary size of the resulting extension library.
and Clang, which sets the default symbol visibility to *hidden*, which has a
tremendous impact on the final binary size of the resulting extension library.
(On Visual Studio, symbols are already hidden by default, so nothing needs to
be done there.)
In addition to decreasing binary size, ``-fvisibility=hidden`` also avoids
potential serious issues when loading multiple modules and is required for
proper pybind operation. See the previous FAQ entry for more details.
Another aspect that can require a fair bit of code are function signature
descriptions. pybind11 automatically generates human-readable function

View File

@ -139,6 +139,13 @@ function(pybind11_add_module target_name)
set_target_properties(${target_name} PROPERTIES PREFIX "${PYTHON_MODULE_PREFIX}")
set_target_properties(${target_name} PROPERTIES SUFFIX "${PYTHON_MODULE_EXTENSION}")
# -fvisibility=hidden is required to allow multiple modules compiled against
# different pybind versions to work properly, and for some features (e.g.
# py::module_local). We force it on everything inside the `pybind11`
# namespace; also turning it on for a pybind module compilation here avoids
# potential warnings or issues from having mixed hidden/non-hidden types.
set_target_properties(${target_name} PROPERTIES CXX_VISIBILITY_PRESET "hidden")
if(WIN32 OR CYGWIN)
# Link against the Python shared library on Windows
target_link_libraries(${target_name} PRIVATE ${PYTHON_LIBRARIES})
@ -175,9 +182,6 @@ function(pybind11_add_module target_name)
_pybind11_add_lto_flags(${target_name} ${ARG_THIN_LTO})
if (NOT MSVC AND NOT ${CMAKE_BUILD_TYPE} MATCHES Debug)
# Set the default symbol visibility to hidden (very important to obtain small binaries)
target_compile_options(${target_name} PRIVATE "-fvisibility=hidden")
# Strip unnecessary sections of the binary on Linux/Mac OS
if(CMAKE_STRIP)
if(APPLE)