From e076c4513dc2c89c1e9b7cb716fb8e4588e8e067 Mon Sep 17 00:00:00 2001 From: Wenzel Jakob Date: Mon, 31 Oct 2022 22:11:16 +0100 Subject: [PATCH] macOS: disable fixup chains when linking extension modules On macOS, extension library builds usually rely on the flag ``-undefined dynamic_lookup`` to get the linker to temporarily accept missing CPython API symbols. As the name implies, those symbols are then dynamically resolved when Python imports such an extension library. Binaries produced in this way are more portable since they don't contain a hardcoded path to the CPython library. This is critical when distributing binary wheels on PyPI, etc.. When targeting macOS>=12, XCode recently started to generate a somewhat ominous warning: ``-undefined dynamic_lookup may not work with chained fixups`` For further detail on what chained fixups are, see the following informative WWDC https://developer.apple.com/videos/play/wwdc2022/110362/ (the relevant part starts about 20mins into the video) The message that the dynamic resolution functionality became broken. I reported this to Apple via feedback request FB11767124. Apple investigated the request and updated the behavior of ``ld`` starting with XCode 14.3b1+: it now disables the fixup chain linker optimization whenever ``-undefined dynamic_lookup`` is specified. The feedback from Apple also stated that the parameter ``-Wl,-no_fixup_chains`` should be specified on pre-14.3 XCode versions to ensure correct behavior. This commit realizes those changes by passing a flag to *always* disable fixup chains. Related discussion is available in the CPython repository (https://github.com/python/cpython/issues/97524). --- docs/compiling.rst | 8 ++++---- tools/pybind11Common.cmake | 6 ++++-- 2 files changed, 8 insertions(+), 6 deletions(-) diff --git a/docs/compiling.rst b/docs/compiling.rst index 2b543be0b..4294fb933 100644 --- a/docs/compiling.rst +++ b/docs/compiling.rst @@ -582,13 +582,13 @@ using ``pip`` or ``conda``. If it hasn't, you can also manually specify ``-I /include`` together with the Python includes path ``python3-config --includes``. -On macOS: the build command is almost the same but it also requires passing -the ``-undefined dynamic_lookup`` flag so as to ignore missing symbols when -building the module: +On macOS: the build command is almost the same but it also requires passing the +``-undefined dynamic_lookup -Wl,no_fixup_chains`` flag to ignore missing symbols +when building the module: .. code-block:: bash - $ c++ -O3 -Wall -shared -std=c++11 -undefined dynamic_lookup $(python3 -m pybind11 --includes) example.cpp -o example$(python3-config --extension-suffix) + $ c++ -O3 -Wall -shared -std=c++11 -undefined dynamic_lookup -Wl,no_fixup_chains $(python3 -m pybind11 --includes) example.cpp -o example$(python3-config --extension-suffix) In general, it is advisable to include several additional build parameters that can considerably reduce the size of the created binary. Refer to section diff --git a/tools/pybind11Common.cmake b/tools/pybind11Common.cmake index 0c985bc8e..f83485881 100644 --- a/tools/pybind11Common.cmake +++ b/tools/pybind11Common.cmake @@ -75,7 +75,8 @@ if(CMAKE_VERSION VERSION_LESS 3.13) set_property( TARGET pybind11::python_link_helper APPEND - PROPERTY INTERFACE_LINK_LIBRARIES "$<$:-undefined dynamic_lookup>") + PROPERTY INTERFACE_LINK_LIBRARIES + "$<$:-undefined dynamic_lookup -no_fixup_chains>") else() # link_options was added in 3.13+ # This is safer, because you are ensured the deduplication pass in CMake will not consider @@ -83,7 +84,8 @@ else() set_property( TARGET pybind11::python_link_helper APPEND - PROPERTY INTERFACE_LINK_OPTIONS "$<$:LINKER:-undefined,dynamic_lookup>") + PROPERTY INTERFACE_LINK_OPTIONS + "$<$:LINKER:-undefined,dynamic_lookup,-no_fixup_chains>") endif() # ------------------------ Windows extras -------------------------