pybind11/docs/compiling.rst

729 lines
28 KiB
ReStructuredText
Raw Normal View History

.. _compiling:
2016-03-09 20:31:21 +00:00
Build systems
#############
For an overview of Python packaging including compiled packaging with a pybind11
example, along with a cookiecutter that includes several pybind11 options, see
the `Scientific Python Development Guide`_.
.. _Scientific Python Development Guide: https://learn.scientific-python.org/development/guides/packaging-compiled/
.. scikit-build-core:
Modules with CMake
==================
A Python extension module can be created with just a few lines of code:
.. code-block:: cmake
cmake_minimum_required(VERSION 3.15...3.30)
project(example LANGUAGES CXX)
set(PYBIND11_FINDPYTHON ON)
find_package(pybind11 CONFIG REQUIRED)
pybind11_add_module(example example.cpp)
2024-08-02 19:17:15 +00:00
install(TARGETS example DESTINATION .)
(You use the ``add_subdirectory`` instead, see the example in :ref:`cmake`.) In
this example, the code is located in a file named :file:`example.cpp`. Either
method will import the pybind11 project which provides the
``pybind11_add_module`` function. It will take care of all the details needed
to build a Python extension module on any platform.
To build with pip, build, cibuildwheel, uv, or other Python tools, you can
add a ``pyproject.toml`` file like this:
.. code-block:: toml
[build-system]
requires = ["scikit-build-core", "pybind11"]
build-backend = "scikit_build_core.build"
[project]
name = "example"
version = "0.1.0"
You don't need setuptools files like ``MANIFEST.in``, ``setup.py``, or
``setup.cfg``, as this is not setuptools. See `scikit-build-core`_ for details.
For projects you plan to upload to PyPI, be sure to fill out the ``[project]``
table with other important metadata as well (see `Writing pyproject.toml`_).
A working sample project can be found in the [scikit_build_example]_
repository. An older and harder-to-maintain method is in [cmake_example]_. More
details about our cmake support can be found below in :ref:`cmake`.
.. _scikit-build-core: https://scikit-build-core.readthedocs.io
.. [scikit_build_example] https://github.com/pybind/scikit_build_example
.. [cmake_example] https://github.com/pybind/cmake_example
.. _modules-meson-python:
Modules with meson-python
=========================
You can also build a package with `Meson`_ using `meson-python`_, if you prefer
that. Your ``meson.build`` file would look something like this:
.. _meson-example:
.. code-block:: meson
project(
'example',
'cpp',
version: '0.1.0',
default_options: [
'cpp_std=c++11',
],
)
py = import('python').find_installation(pure: false)
pybind11_dep = dependency('pybind11')
py.extension_module('example',
'example.cpp',
install: true,
dependencies : [pybind11_dep],
)
And you would need a ``pyproject.toml`` file like this:
.. code-block:: toml
[build-system]
requires = ["meson-python", "pybind11"]
build-backend = "mesonpy"
Meson-python *requires* your project to be in git (or mercurial) as it uses it
for the SDist creation. For projects you plan to upload to PyPI, be sure to fill out the
``[project]`` table as well (see `Writing pyproject.toml`_).
.. _Writing pyproject.toml: https://packaging.python.org/en/latest/guides/writing-pyproject-toml
.. _meson: https://mesonbuild.com
.. _meson-python: https://meson-python.readthedocs.io/en/latest
.. _build-setuptools:
Modules with setuptools
=======================
2016-03-09 20:31:21 +00:00
For projects on PyPI, a historically popular option is setuptools. Sylvain
Corlay has kindly provided an example project which shows how to set up
everything, including automatic generation of documentation using Sphinx.
Please refer to the [python_example]_ repository.
2016-03-09 20:31:21 +00:00
2016-06-03 12:24:17 +00:00
.. [python_example] https://github.com/pybind/python_example
2016-03-09 20:31:21 +00:00
A helper file is provided with pybind11 that can simplify usage with setuptools.
To use pybind11 inside your ``setup.py``, you have to have some system to
ensure that ``pybind11`` is installed when you build your package. There are
four possible ways to do this, and pybind11 supports all four: You can ask all
users to install pybind11 beforehand (bad), you can use
:ref:`setup_helpers-pep518` (good), ``setup_requires=`` (discouraged), or you
can :ref:`setup_helpers-copy-manually` (works but you have to manually sync
your copy to get updates). Third party packagers like conda-forge generally
strongly prefer the ``pyproject.toml`` method, as it gives them control over
the ``pybind11`` version, and they may apply patches, etc.
An example of a ``setup.py`` using pybind11's helpers:
.. code-block:: python
from glob import glob
from setuptools import setup
from pybind11.setup_helpers import Pybind11Extension
ext_modules = [
Pybind11Extension(
"python_example",
sorted(glob("src/*.cpp")), # Sort source files for reproducibility
),
]
setup(..., ext_modules=ext_modules)
If you want to do an automatic search for the highest supported C++ standard,
that is supported via a ``build_ext`` command override; it will only affect
``Pybind11Extensions``:
.. code-block:: python
from glob import glob
from setuptools import setup
from pybind11.setup_helpers import Pybind11Extension, build_ext
ext_modules = [
Pybind11Extension(
"python_example",
sorted(glob("src/*.cpp")),
),
]
setup(..., cmdclass={"build_ext": build_ext}, ext_modules=ext_modules)
If you have single-file extension modules that are directly stored in the
Python source tree (``foo.cpp`` in the same directory as where a ``foo.py``
would be located), you can also generate ``Pybind11Extensions`` using
``setup_helpers.intree_extensions``: ``intree_extensions(["path/to/foo.cpp",
...])`` returns a list of ``Pybind11Extensions`` which can be passed to
``ext_modules``, possibly after further customizing their attributes
(``libraries``, ``include_dirs``, etc.). By doing so, a ``foo.*.so`` extension
module will be generated and made available upon installation.
``intree_extension`` will automatically detect if you are using a ``src``-style
layout (as long as no namespace packages are involved), but you can also
explicitly pass ``package_dir`` to it (as in ``setuptools.setup``).
2020-10-02 14:03:35 +00:00
Since pybind11 does not require NumPy when building, a light-weight replacement
for NumPy's parallel compilation distutils tool is included. Use it like this:
.. code-block:: python
2020-10-02 14:03:35 +00:00
from pybind11.setup_helpers import ParallelCompile
# Optional multithreaded build
ParallelCompile("NPY_NUM_BUILD_JOBS").install()
setup(...)
2020-10-02 14:03:35 +00:00
The argument is the name of an environment variable to control the number of
threads, such as ``NPY_NUM_BUILD_JOBS`` (as used by NumPy), though you can set
something different if you want; ``CMAKE_BUILD_PARALLEL_LEVEL`` is another choice
a user might expect. You can also pass ``default=N`` to set the default number
of threads (0 will take the number of threads available) and ``max=N``, the
maximum number of threads; if you have a large extension you may want set this
to a memory dependent number.
2020-10-02 14:03:35 +00:00
If you are developing rapidly and have a lot of C++ files, you may want to
avoid rebuilding files that have not changed. For simple cases were you are
using ``pip install -e .`` and do not have local headers, you can skip the
rebuild if an object file is newer than its source (headers are not checked!)
with the following:
.. code-block:: python
from pybind11.setup_helpers import ParallelCompile, naive_recompile
ParallelCompile("NPY_NUM_BUILD_JOBS", needs_recompile=naive_recompile).install()
If you have a more complex build, you can implement a smarter function and pass
it to ``needs_recompile``, or you can use [Ccache]_ instead. ``CXX="cache g++"
pip install -e .`` would be the way to use it with GCC, for example. Unlike the
simple solution, this even works even when not compiling in editable mode, but
it does require Ccache to be installed.
Keep in mind that Pip will not even attempt to rebuild if it thinks it has
already built a copy of your code, which it deduces from the version number.
One way to avoid this is to use [setuptools_scm]_, which will generate a
version number that includes the number of commits since your last tag and a
hash for a dirty directory. Another way to force a rebuild is purge your cache
or use Pip's ``--no-cache-dir`` option.
You also need a ``MANIFEST.in`` file to include all relevant files so that you
can make an SDist. If you use `pypa-build`_, that will build an SDist then a
wheel from that SDist by default, so you can look inside those files (wheels
are just zip files with a ``.whl`` extension) to make sure you aren't missing
files. `check-manifest`_ (setuptools specific) or `check-sdist`_ (general) are
CLI tools that can compare the SDist contents with your source control.
.. [Ccache] https://ccache.dev
.. [setuptools_scm] https://github.com/pypa/setuptools_scm
.. _setup_helpers-pep518:
Build requirements
------------------
With a ``pyproject.toml`` file, you can ensure that ``pybind11`` is available
during the compilation of your project. When this file exists, Pip will make a
new virtual environment, download just the packages listed here in
``requires=``, and build a wheel (binary Python package). It will then throw
away the environment, and install your wheel.
Your ``pyproject.toml`` file will likely look something like this:
.. code-block:: toml
[build-system]
requires = ["setuptools", "pybind11"]
build-backend = "setuptools.build_meta"
.. _PEP 517: https://www.python.org/dev/peps/pep-0517/
.. _cibuildwheel: https://cibuildwheel.pypa.io
.. _pypa-build: https://build.pypa.io/en/latest/
.. _check-manifest: https://pypi.io/project/check-manifest
.. _check-sdist: https://pypi.io/project/check-sdist
.. _setup_helpers-copy-manually:
Copy manually
-------------
You can also copy ``setup_helpers.py`` directly to your project; it was
designed to be usable standalone, like the old example ``setup.py``. You can
set ``include_pybind11=False`` to skip including the pybind11 package headers,
so you can use it with git submodules and a specific git version. If you use
this, you will need to import from a local file in ``setup.py`` and ensure the
helper file is part of your MANIFEST.
Closely related, if you include pybind11 as a subproject, you can run the
``setup_helpers.py`` inplace. If loaded correctly, this should even pick up
the correct include for pybind11, though you can turn it off as shown above if
you want to input it manually.
Suggested usage if you have pybind11 as a submodule in ``extern/pybind11``:
.. code-block:: python
DIR = os.path.abspath(os.path.dirname(__file__))
sys.path.append(os.path.join(DIR, "extern", "pybind11"))
from pybind11.setup_helpers import Pybind11Extension # noqa: E402
del sys.path[-1]
.. versionchanged:: 2.6
Added ``setup_helpers`` file.
Building with cppimport
========================
[cppimport]_ is a small Python import hook that determines whether there is a C++
source file whose name matches the requested module. If there is, the file is
compiled as a Python extension using pybind11 and placed in the same folder as
the C++ source file. Python is then able to find the module and load it.
.. [cppimport] https://github.com/tbenthompson/cppimport
2015-10-13 00:57:16 +00:00
.. _cmake:
Building with CMake
===================
2016-09-06 04:02:29 +00:00
For C++ codebases that have an existing CMake-based build system, a Python
extension module can be created with just a few lines of code, as seen above in
the module section. Pybind11 currently defaults to the old mechanism, though be
aware that CMake 3.27 removed the old mechanism, so pybind11 will automatically
switch if the old mechanism is not available. Please opt into the new mechanism
if at all possible. Our default may change in future versions. This is the
minimum required:
2015-10-13 00:57:16 +00:00
2016-05-26 22:11:52 +00:00
.. versionchanged:: 2.6
CMake 3.4+ is required.
.. versionchanged:: 2.11
CMake 3.5+ is required.
.. versionchanged:: 2.14
CMake 3.15+ is required.
Further information can be found at :doc:`cmake/index`.
pybind11_add_module
-------------------
To ease the creation of Python extension modules, pybind11 provides a CMake
function with the following signature:
.. code-block:: cmake
pybind11_add_module(<name> [MODULE | SHARED] [EXCLUDE_FROM_ALL]
2020-09-04 21:31:05 +00:00
[NO_EXTRAS] [THIN_LTO] [OPT_SIZE] source1 [source2 ...])
This function behaves very much like CMake's builtin ``add_library`` (in fact,
it's a wrapper function around that command). It will add a library target
called ``<name>`` to be built from the listed source files. In addition, it
will take care of all the Python-specific compiler and linker flags as well
as the OS- and Python-version-specific file extension. The produced target
``<name>`` can be further manipulated with regular CMake commands.
``MODULE`` or ``SHARED`` may be given to specify the type of library. If no
type is given, ``MODULE`` is used by default which ensures the creation of a
Python-exclusive module. Specifying ``SHARED`` will create a more traditional
dynamic library which can also be linked from elsewhere. ``EXCLUDE_FROM_ALL``
removes this target from the default build (see CMake docs for details).
Since pybind11 is a template library, ``pybind11_add_module`` adds compiler
flags to ensure high quality code generation without bloat arising from long
symbol names and duplication of code in different translation units. It
sets default visibility to *hidden*, which is required for some pybind11
features and functionality when attempting to load multiple pybind11 modules
compiled under different pybind11 versions. It also adds additional flags
enabling LTO (Link Time Optimization) and strip unneeded symbols. See the
:ref:`FAQ entry <faq:symhidden>` for a more detailed explanation. These
latter optimizations are never applied in ``Debug`` mode. If ``NO_EXTRAS`` is
given, they will always be disabled, even in ``Release`` mode. However, this
will result in code bloat and is generally not recommended.
As stated above, LTO is enabled by default. Some newer compilers also support
different flavors of LTO such as `ThinLTO`_. Setting ``THIN_LTO`` will cause
the function to prefer this flavor if available. The function falls back to
regular LTO if ``-flto=thin`` is not available. If
``CMAKE_INTERPROCEDURAL_OPTIMIZATION`` is set (either ``ON`` or ``OFF``), then
that will be respected instead of the built-in flag search.
.. note::
If you want to set the property form on targets or the
``CMAKE_INTERPROCEDURAL_OPTIMIZATION_<CONFIG>`` versions of this, you should
still use ``set(CMAKE_INTERPROCEDURAL_OPTIMIZATION OFF)`` (otherwise a
no-op) to disable pybind11's ipo flags.
2020-09-04 21:31:05 +00:00
The ``OPT_SIZE`` flag enables size-based optimization equivalent to the
standard ``/Os`` or ``-Os`` compiler flags and the ``MinSizeRel`` build type,
which avoid optimizations that can substantially increase the size of the
2020-09-04 21:31:05 +00:00
resulting binary. This flag is particularly useful in projects that are split
into performance-critical parts and associated bindings. In this case, we can
compile the project in release mode (and hence, optimize performance globally),
and specify ``OPT_SIZE`` for the binding target, where size might be the main
concern as performance is often less critical here. A ~25% size reduction has
been observed in practice. This flag only changes the optimization behavior at
a per-target level and takes precedence over the global CMake build type
(``Release``, ``RelWithDebInfo``) except for ``Debug`` builds, where
optimizations remain disabled.
.. _ThinLTO: http://clang.llvm.org/docs/ThinLTO.html
Configuration variables
-----------------------
By default, pybind11 will compile modules with the compiler default or the
2020-07-30 20:04:26 +00:00
minimum standard required by pybind11, whichever is higher. You can set the
standard explicitly with
`CMAKE_CXX_STANDARD <https://cmake.org/cmake/help/latest/variable/CMAKE_CXX_STANDARD.html>`_:
.. code-block:: cmake
set(CMAKE_CXX_STANDARD 14 CACHE STRING "C++ version selection") # or 11, 14, 17, 20
set(CMAKE_CXX_STANDARD_REQUIRED ON) # optional, ensure standard is supported
2021-09-28 14:09:38 +00:00
set(CMAKE_CXX_EXTENSIONS OFF) # optional, keep compiler extensions off
The variables can also be set when calling CMake from the command line using
the ``-D<variable>=<value>`` flag. You can also manually set ``CXX_STANDARD``
on a target or use ``target_compile_features`` on your targets - anything that
CMake supports.
Classic Python support: The target Python version can be selected by setting
``PYBIND11_PYTHON_VERSION`` or an exact Python installation can be specified
with ``PYTHON_EXECUTABLE``. For example:
.. code-block:: bash
feat: remove Python 3.7 support (#5191) * First pass updating misc files, informed by https://github.com/pybind/pybind11/pull/5177/commits * Remove jobs using silkeh/clang and gcc docker containers that come with Python 3.7 * Add silkeh/clang:17-bookworm * Add job using GCC 7 * Revert "Add job using GCC 7" This reverts commit 518515a761ac37dc2cf5d0980da82d0de39edc28. * Try running in ubuntu-18.04 container under ubuntu-latest (to get GCC 7) * Fix `-` vs `:` mixup. * This reverts commit b1c4304475b8ad129c12330c7ed7eb85d15ba14a. Revert "Try running in ubuntu:18.04 container under ubuntu-latest (to get GCC 7)" This reverts commit b203a294bb444fc6ae57a0100fa91dc91b8d3264. * `git grep 0x03080000` cleanup. * `git grep -I -E '3\.7'` cleanup. Removes two changes made under pybind/pybind11#3702 * Revert "`git grep -I -E '3\.7'` cleanup." This reverts commit bb5b9d187bffbfb61e2977d7eee46b766fa1cce9. * Remove comments that are evidently incorrect: ``` ... -- The CXX compiler identification is Clang 15.0.7 ... - Found Python: /usr/bin/python3.9 (found suitable version "3.9.2", minimum required is "3.7") found components: Interpreter Development.Module Development.Embed ... /__w/pybind11/pybind11/include/pybind11/gil.h:150:13: error: 'auto key' can be declared as 'auto *key' [readability-qualified-auto,-warnings-as-errors] auto key = internals.tstate; ^~~~~ auto * /__w/pybind11/pybind11/include/pybind11/gil.h:174:13: error: 'auto key' can be declared as 'auto *key' [readability-qualified-auto,-warnings-as-errors] auto key = detail::get_internals().tstate; ^~~~~ auto * ``` * .github/workflows/configure.yml: Change from Python 3.7 to 3.8 * Misc cleanup pass * Miscellaneous changes based on manual review of the `git grep` matches below: ``` git_grep_37_38.sh |& sort | uniq -c ``` With git_grep_37_38.sh: ``` set -x git grep 0x0307 git grep 0x0308 git grep PY_MINOR_VERSION git grep PYPY_VERSION git grep -I -E '3\.7' git grep -I -E '3\.8' git grep -I -E '\(3, 7' git grep -I -E '\(3, 8' git grep -I -E '3[^A-Za-z0-9.]+7' git grep -I -E '3[^A-Za-z0-9.]+8' ``` Output: ``` 1 .appveyor.yml: $env:CMAKE_INCLUDE_PATH = "eigen-3.3.7;$env:CMAKE_INCLUDE_PATH" 1 .appveyor.yml: 7z x eigen-3.3.7.zip -y > $null 1 .appveyor.yml: Start-FileDownload 'https://gitlab.com/libeigen/eigen/-/archive/3.3.7/eigen-3.3.7.zip' 1 CMakeLists.txt: # Bug in macOS CMake < 3.7 is unable to download catch 1 CMakeLists.txt: elseif(WINDOWS AND CMAKE_VERSION VERSION_LESS 3.8) 1 CMakeLists.txt: if(OSX AND CMAKE_VERSION VERSION_LESS 3.7) 1 CMakeLists.txt: message(WARNING "CMAKE 3.7+ needed on macOS to download catch, and newer HIGHLY recommended") 1 CMakeLists.txt: message(WARNING "CMAKE 3.8+ tested on Windows, previous versions untested") 1 CMakeLists.txt: # Only tested with 3.8+ in CI. 1 docs/advanced/functions.rst:Python 3.8 introduced a new positional-only argument syntax, using ``/`` in the 1 docs/changelog.rst:* Adapt pybind11 to a C API convention change in Python 3.8. `#1950 1 docs/changelog.rst:* Allow thread termination to be avoided during shutdown for CPython 3.7+ via 1 docs/changelog.rst: considered as conversion, consistent with Python 3.8+. 1 docs/changelog.rst: CPython 3.8 and 3.9 debug builds. 1 docs/changelog.rst:* Enum now has an ``__index__`` method on Python <3.8 too. 1 docs/changelog.rst: on Python 3.8. `#1780 <https://github.com/pybind/pybind11/pull/1780>`_. 1 docs/changelog.rst:* PyPy 3.10 support was added, PyPy 3.7 support was dropped. 2 docs/changelog.rst:* Support PyPy 7.3.7 and the PyPy3.8 beta. Test python-3.11 on PRs with the 1 docs/changelog.rst:* Use ``macos-13`` (Intel) for CI jobs for now (will drop Python 3.7 soon). 1 docs/changelog.rst:* Use new Python 3.7 Thread Specific Storage (TSS) implementation if available. 1 docs/compiling.rst: cmake -DPYBIND11_PYTHON_VERSION=3.8 .. 1 docs/compiling.rst: find_package(Python 3.8 COMPONENTS Interpreter Development REQUIRED) 1 docs/limitations.rst:- PyPy3 7.3.1 and 7.3.2 have issues with several tests on 32-bit Windows. 1 docs/requirements.txt:idna==3.7 \ 1 + git grep 0x0307 1 + git grep 0x0308 1 + git grep -I -E '\(3, 7' 1 + git grep -I -E '3\.7' 1 + git grep -I -E '\(3, 8' 1 + git grep -I -E '3\.8' 1 + git grep -I -E '3[^A-Za-z0-9.]+7' 1 + git grep -I -E '3[^A-Za-z0-9.]+8' 1 + git grep PY_MINOR_VERSION 1 + git grep PYPY_VERSION 2 .github/workflows/ci.yml: - '3.8' 1 .github/workflows/ci.yml: - 3.8 1 .github/workflows/ci.yml: - name: Add Python 3.8 1 .github/workflows/ci.yml: - 'pypy-3.8' 2 .github/workflows/ci.yml: python: '3.8' 1 .github/workflows/ci.yml: - python: '3.8' 1 .github/workflows/ci.yml: - python: 3.8 1 .github/workflows/ci.yml: python: 'pypy-3.8' 1 .github/workflows/configure.yml: cmake: "3.8" 1 .github/workflows/configure.yml: name: 🐍 3.8 • CMake ${{ matrix.cmake }} • ${{ matrix.runs-on }} 1 .github/workflows/configure.yml: - name: Setup Python 3.8 1 .github/workflows/configure.yml: python-version: 3.8 1 .github/workflows/pip.yml: name: 🐍 3.8 • 📦 & 📦 tests • ubuntu-latest 1 .github/workflows/pip.yml: name: 🐍 3.8 • 📦 tests • windows-latest 2 .github/workflows/pip.yml: - name: Setup 🐍 3.8 2 .github/workflows/pip.yml: python-version: 3.8 2 include/pybind11/cast.h:#if !defined(PYPY_VERSION) 2 include/pybind11/cast.h:#if defined(PYPY_VERSION) 2 include/pybind11/cast.h: // PyPy: 7.3.7's 3.8 does not implement PyLong_*'s __index__ calls. 5 include/pybind11/detail/class.h:#if !defined(PYPY_VERSION) 1 include/pybind11/detail/class.h:#if defined(PYPY_VERSION) 1 include/pybind11/detail/class.h: // This was not needed before Python 3.8 (Python issue 35810) 1 include/pybind11/detail/common.h: && !defined(PYPY_VERSION) && !defined(PYBIND11_ASSERT_GIL_HELD_INCREF_DECREF) 2 include/pybind11/detail/common.h:# error "PYTHON < 3.8 IS UNSUPPORTED. pybind11 v2.13 was the last to support Python 3.7." 1 include/pybind11/detail/common.h:#if defined(PYPY_VERSION) && !defined(PYBIND11_SIMPLE_GIL_MANAGEMENT) 1 include/pybind11/detail/common.h:#if PY_VERSION_HEX < 0x03080000 1 include/pybind11/detail/common.h: = PYBIND11_TOSTRING(PY_MAJOR_VERSION) "." PYBIND11_TOSTRING(PY_MINOR_VERSION); \ 1 include/pybind11/detail/internals.h: // called. PYBIND11_TLS_FREE is PyThread_tss_free on python 3.7+. On older python, it does 1 include/pybind11/detail/internals.h:#if PYBIND11_INTERNALS_VERSION <= 4 || defined(PYPY_VERSION) 1 include/pybind11/detail/internals.h:// The old Python Thread Local Storage (TLS) API is deprecated in Python 3.7 in favor of the new 1 include/pybind11/detail/type_caster_base.h:#if defined(PYPY_VERSION) 1 include/pybind11/embed.h:# define PYBIND11_PYCONFIG_SUPPORT_PY_VERSION_HEX (0x03080000) 1 include/pybind11/embed.h:#if defined(PYPY_VERSION) 1 include/pybind11/eval.h: // globals if not yet present. Python 3.8 made PyRun_String behave 2 include/pybind11/eval.h:#if defined(PYPY_VERSION) 2 include/pybind11/eval.h: // was missing from PyPy3.8 7.3.7. 2 include/pybind11/gil.h: /// allowed during shutdown. Check _Py_IsFinalizing() on Python 3.7+, and 1 include/pybind11/pybind11.h:#if !defined(PYPY_VERSION) 4 include/pybind11/pybind11.h:#if !defined(PYPY_VERSION) && PY_MAJOR_VERSION == 3 && PY_MINOR_VERSION == 9 1 include/pybind11/pytypes.h:#endif //! defined(PYPY_VERSION) 2 include/pybind11/pytypes.h:#if !defined(PYPY_VERSION) 1 include/pybind11/pytypes.h:# if defined(PYPY_VERSION_NUM) && PYPY_VERSION_NUM < 0x07030a00 1 include/pybind11/pytypes.h:#ifdef PYPY_VERSION 1 include/pybind11/stl/filesystem.h:# if !defined(PYPY_VERSION) 2 pybind11/__init__.py:if sys.version_info < (3, 8): 2 pybind11/__init__.py: msg = "pybind11 does not support Python < 3.8. v2.13 was the last release supporting Python 3.7." 1 pyproject.toml:master.py-version = "3.8" 1 pyproject.toml:python_version = "3.8" 1 README.rst:lines of code and depend on Python (3.8+, or PyPy) and the C++ 2 README.rst:- Python 3.8+, and PyPy3 7.3 are supported with an implementation-agnostic 1 setup.cfg: Programming Language :: Python :: 3.8 1 setup.cfg:python_requires = >=3.8 1 setup.py:# TODO: use literals & overload (typing extensions or Python 3.8) 1 tests/CMakeLists.txt:if(NOT CMAKE_VERSION VERSION_LESS 3.8) 2 tests/constructor_stats.h:#if defined(PYPY_VERSION) 1 tests/env.py: doesn't work on CPython 3.8.0 with pytest==3.3.2 on Ubuntu 18.04 (#2922). 1 tests/requirements.txt:build~=1.0; python_version>="3.8" 1 tests/requirements.txt:numpy~=1.21.5; platform_python_implementation!="PyPy" and python_version>="3.8" and python_version<"3.10" 1 tests/requirements.txt:numpy~=1.23.0; python_version=="3.8" and platform_python_implementation=="PyPy" 1 tests/test_buffers.py: env.PYPY, reason="PyPy 7.3.7 doesn't clear this anymore", strict=False 1 tests/test_builtin_casters.py: # Before Python 3.8, `PyLong_AsLong` does not pick up on `obj.__index__`, 2 tests/test_builtin_casters.py: if (3, 8) <= sys.version_info < (3, 10) and env.CPYTHON: 4 tests/test_builtin_casters.py: # TODO: PyPy 3.8 does not behave like CPython 3.8 here yet (7.3.7) 1 tests/test_callbacks.py: assert m.test_callback3(z.double) == "func(43) = 86" 2 tests/test_call_policies.cpp:#if !defined(PYPY_VERSION) 1 tests/test_chrono.py: diff = m.test_chrono_float_diff(43.789012, 1.123456) 1 tests/test_constants_and_functions.py: assert m.f3(86) == 89 1 tests/test_eigen_matrix.py: a_copy3[8, 1] = 11 1 tests/test_eigen_matrix.py: assert np.all(cornersc == np.array([[1.0, 3], [7, 9]])) 1 tests/test_eigen_matrix.py: assert np.all(cornersr == np.array([[1.0, 3], [7, 9]])) 1 tests/test_eigen_matrix.py: mymat = chol(np.array([[1.0, 2, 4], [2, 13, 23], [4, 23, 77]])) 1 tests/test_exceptions.py: if hasattr(pytest, unraisable): # Python >= 3.8 and pytest >= 6 2 tests/test_exceptions.py:@pytest.mark.xfail(env.PYPY, reason="Failure on PyPy 3.8 (7.3.7)", strict=False) 1 tests/test_factory_constructors.py: assert [i.alive() for i in cstats] == [13, 7] 1 tests/test_kwargs_and_defaults.cpp:#ifdef PYPY_VERSION 1 tests/test_local_bindings.py: assert i1.get3() == 8 1 tests/test_methods_and_attributes.cpp:#if !defined(PYPY_VERSION) 1 tests/test_numpy_array.py: a = np.arange(3 * 7 * 2) + 1 1 tests/test_numpy_array.py: assert str(excinfo.value) == "cannot reshape array of size 42 into shape (3,7,1)" 2 tests/test_numpy_array.py: assert x.shape == (3, 7, 2) 2 tests/test_numpy_array.py: m.reshape_tuple(a, (3, 7, 1)) 2 tests/test_numpy_array.py: x = m.reshape_tuple(a, (3, 7, 2)) 1 tests/test_numpy_vectorize.py: assert np.isclose(m.vectorized_func3(np.array(3 + 7j)), [6 + 14j]) 1 tests/test_pickling.cpp:#if !defined(PYPY_VERSION) 1 tests/test_pytypes.cpp:#if (defined(__APPLE__) && defined(__clang__)) || defined(PYPY_VERSION) 1 tests/test_smart_ptr.cpp: m.def("make_myobject3_1", []() { return new MyObject3(8); }); 1 tests/test_smart_ptr.py: assert cstats.values() == ["MyObject3[9]", "MyObject3[8]", "MyObject3[9]"] 1 tests/test_stl_binders.py: assert v_int2 == m.VectorInt([0, 99, 2, 3, 4, 5, 6, 7, 0, 1, 2, 3, 88]) 1 tests/test_stl_binders.py: assert v_int2 == m.VectorInt([0, 99, 2, 3, 4, 5, 6, 7, 0, 1, 2, 3, 88, 4]) 1 tests/test_type_caster_pyobject_ptr.cpp:#if !defined(PYPY_VERSION) // It is not worth the trouble doing something special for PyPy. 1 tools/FindPythonLibsNew.cmake: set(PythonLibsNew_FIND_VERSION "3.8") 1 tools/JoinPaths.cmake:# https://docs.python.org/3.7/library/os.path.html#os.path.join 1 tools/pybind11NewTools.cmake: Python 3.8 REQUIRED COMPONENTS ${_pybind11_interp_component} ${_pybind11_dev_component} 1 tools/pybind11NewTools.cmake:# Python debug libraries expose slightly different objects before 3.8 1 tools/pybind11Tools.cmake: "3.12;3.11;3.10;3.9;3.8" 1 tools/pybind11Tools.cmake: if(NOT DEFINED PYPY_VERSION) 1 tools/pybind11Tools.cmake: message(STATUS "PYPY ${PYPY_VERSION} (Py ${PYTHON_VERSION})") 1 tools/pybind11Tools.cmake:# Python debug libraries expose slightly different objects before 3.8 1 tools/pybind11Tools.cmake: set(PYPY_VERSION ``` * Change `[tool.ruff]` `target-version` to `"py38"`, as suggested by @Skylion007
2024-07-30 16:18:35 +00:00
cmake -DPYBIND11_PYTHON_VERSION=3.8 ..
# Another method:
cmake -DPYTHON_EXECUTABLE=/path/to/python ..
2020-07-30 20:04:26 +00:00
# This often is a good way to get the current Python, works in environments:
cmake -DPYTHON_EXECUTABLE=$(python3 -c "import sys; print(sys.executable)") ..
find_package vs. add_subdirectory
---------------------------------
For CMake-based projects that don't include the pybind11 repository internally,
an external installation can be detected through ``find_package(pybind11)``.
See the `Config file`_ docstring for details of relevant CMake variables.
.. code-block:: cmake
cmake_minimum_required(VERSION 3.15...3.30)
project(example LANGUAGES CXX)
find_package(pybind11 REQUIRED)
pybind11_add_module(example example.cpp)
Note that ``find_package(pybind11)`` will only work correctly if pybind11
has been correctly installed on the system, e. g. after downloading or cloning
the pybind11 repository :
.. code-block:: bash
# Classic CMake
cd pybind11
mkdir build
cd build
cmake ..
make install
# CMake 3.15+
cd pybind11
cmake -S . -B build
cmake --build build -j 2 # Build on 2 cores
cmake --install build
Once detected, the aforementioned ``pybind11_add_module`` can be employed as
before. The function usage and configuration variables are identical no matter
if pybind11 is added as a subdirectory or found as an installed package. You
can refer to the same [cmake_example]_ repository for a full sample project
-- just swap out ``add_subdirectory`` for ``find_package``.
.. _Config file: https://github.com/pybind/pybind11/blob/master/tools/pybind11Config.cmake.in
.. _find-python-mode:
FindPython mode
---------------
Modern CMake (3.18.2+ ideal) added a new module called FindPython that had a
highly improved search algorithm and modern targets and tools. If you use
FindPython, pybind11 will detect this and use the existing targets instead:
.. code-block:: cmake
cmake_minimum_required(VERSION 3.15...3.30)
project(example LANGUAGES CXX)
feat: remove Python 3.7 support (#5191) * First pass updating misc files, informed by https://github.com/pybind/pybind11/pull/5177/commits * Remove jobs using silkeh/clang and gcc docker containers that come with Python 3.7 * Add silkeh/clang:17-bookworm * Add job using GCC 7 * Revert "Add job using GCC 7" This reverts commit 518515a761ac37dc2cf5d0980da82d0de39edc28. * Try running in ubuntu-18.04 container under ubuntu-latest (to get GCC 7) * Fix `-` vs `:` mixup. * This reverts commit b1c4304475b8ad129c12330c7ed7eb85d15ba14a. Revert "Try running in ubuntu:18.04 container under ubuntu-latest (to get GCC 7)" This reverts commit b203a294bb444fc6ae57a0100fa91dc91b8d3264. * `git grep 0x03080000` cleanup. * `git grep -I -E '3\.7'` cleanup. Removes two changes made under pybind/pybind11#3702 * Revert "`git grep -I -E '3\.7'` cleanup." This reverts commit bb5b9d187bffbfb61e2977d7eee46b766fa1cce9. * Remove comments that are evidently incorrect: ``` ... -- The CXX compiler identification is Clang 15.0.7 ... - Found Python: /usr/bin/python3.9 (found suitable version "3.9.2", minimum required is "3.7") found components: Interpreter Development.Module Development.Embed ... /__w/pybind11/pybind11/include/pybind11/gil.h:150:13: error: 'auto key' can be declared as 'auto *key' [readability-qualified-auto,-warnings-as-errors] auto key = internals.tstate; ^~~~~ auto * /__w/pybind11/pybind11/include/pybind11/gil.h:174:13: error: 'auto key' can be declared as 'auto *key' [readability-qualified-auto,-warnings-as-errors] auto key = detail::get_internals().tstate; ^~~~~ auto * ``` * .github/workflows/configure.yml: Change from Python 3.7 to 3.8 * Misc cleanup pass * Miscellaneous changes based on manual review of the `git grep` matches below: ``` git_grep_37_38.sh |& sort | uniq -c ``` With git_grep_37_38.sh: ``` set -x git grep 0x0307 git grep 0x0308 git grep PY_MINOR_VERSION git grep PYPY_VERSION git grep -I -E '3\.7' git grep -I -E '3\.8' git grep -I -E '\(3, 7' git grep -I -E '\(3, 8' git grep -I -E '3[^A-Za-z0-9.]+7' git grep -I -E '3[^A-Za-z0-9.]+8' ``` Output: ``` 1 .appveyor.yml: $env:CMAKE_INCLUDE_PATH = "eigen-3.3.7;$env:CMAKE_INCLUDE_PATH" 1 .appveyor.yml: 7z x eigen-3.3.7.zip -y > $null 1 .appveyor.yml: Start-FileDownload 'https://gitlab.com/libeigen/eigen/-/archive/3.3.7/eigen-3.3.7.zip' 1 CMakeLists.txt: # Bug in macOS CMake < 3.7 is unable to download catch 1 CMakeLists.txt: elseif(WINDOWS AND CMAKE_VERSION VERSION_LESS 3.8) 1 CMakeLists.txt: if(OSX AND CMAKE_VERSION VERSION_LESS 3.7) 1 CMakeLists.txt: message(WARNING "CMAKE 3.7+ needed on macOS to download catch, and newer HIGHLY recommended") 1 CMakeLists.txt: message(WARNING "CMAKE 3.8+ tested on Windows, previous versions untested") 1 CMakeLists.txt: # Only tested with 3.8+ in CI. 1 docs/advanced/functions.rst:Python 3.8 introduced a new positional-only argument syntax, using ``/`` in the 1 docs/changelog.rst:* Adapt pybind11 to a C API convention change in Python 3.8. `#1950 1 docs/changelog.rst:* Allow thread termination to be avoided during shutdown for CPython 3.7+ via 1 docs/changelog.rst: considered as conversion, consistent with Python 3.8+. 1 docs/changelog.rst: CPython 3.8 and 3.9 debug builds. 1 docs/changelog.rst:* Enum now has an ``__index__`` method on Python <3.8 too. 1 docs/changelog.rst: on Python 3.8. `#1780 <https://github.com/pybind/pybind11/pull/1780>`_. 1 docs/changelog.rst:* PyPy 3.10 support was added, PyPy 3.7 support was dropped. 2 docs/changelog.rst:* Support PyPy 7.3.7 and the PyPy3.8 beta. Test python-3.11 on PRs with the 1 docs/changelog.rst:* Use ``macos-13`` (Intel) for CI jobs for now (will drop Python 3.7 soon). 1 docs/changelog.rst:* Use new Python 3.7 Thread Specific Storage (TSS) implementation if available. 1 docs/compiling.rst: cmake -DPYBIND11_PYTHON_VERSION=3.8 .. 1 docs/compiling.rst: find_package(Python 3.8 COMPONENTS Interpreter Development REQUIRED) 1 docs/limitations.rst:- PyPy3 7.3.1 and 7.3.2 have issues with several tests on 32-bit Windows. 1 docs/requirements.txt:idna==3.7 \ 1 + git grep 0x0307 1 + git grep 0x0308 1 + git grep -I -E '\(3, 7' 1 + git grep -I -E '3\.7' 1 + git grep -I -E '\(3, 8' 1 + git grep -I -E '3\.8' 1 + git grep -I -E '3[^A-Za-z0-9.]+7' 1 + git grep -I -E '3[^A-Za-z0-9.]+8' 1 + git grep PY_MINOR_VERSION 1 + git grep PYPY_VERSION 2 .github/workflows/ci.yml: - '3.8' 1 .github/workflows/ci.yml: - 3.8 1 .github/workflows/ci.yml: - name: Add Python 3.8 1 .github/workflows/ci.yml: - 'pypy-3.8' 2 .github/workflows/ci.yml: python: '3.8' 1 .github/workflows/ci.yml: - python: '3.8' 1 .github/workflows/ci.yml: - python: 3.8 1 .github/workflows/ci.yml: python: 'pypy-3.8' 1 .github/workflows/configure.yml: cmake: "3.8" 1 .github/workflows/configure.yml: name: 🐍 3.8 • CMake ${{ matrix.cmake }} • ${{ matrix.runs-on }} 1 .github/workflows/configure.yml: - name: Setup Python 3.8 1 .github/workflows/configure.yml: python-version: 3.8 1 .github/workflows/pip.yml: name: 🐍 3.8 • 📦 & 📦 tests • ubuntu-latest 1 .github/workflows/pip.yml: name: 🐍 3.8 • 📦 tests • windows-latest 2 .github/workflows/pip.yml: - name: Setup 🐍 3.8 2 .github/workflows/pip.yml: python-version: 3.8 2 include/pybind11/cast.h:#if !defined(PYPY_VERSION) 2 include/pybind11/cast.h:#if defined(PYPY_VERSION) 2 include/pybind11/cast.h: // PyPy: 7.3.7's 3.8 does not implement PyLong_*'s __index__ calls. 5 include/pybind11/detail/class.h:#if !defined(PYPY_VERSION) 1 include/pybind11/detail/class.h:#if defined(PYPY_VERSION) 1 include/pybind11/detail/class.h: // This was not needed before Python 3.8 (Python issue 35810) 1 include/pybind11/detail/common.h: && !defined(PYPY_VERSION) && !defined(PYBIND11_ASSERT_GIL_HELD_INCREF_DECREF) 2 include/pybind11/detail/common.h:# error "PYTHON < 3.8 IS UNSUPPORTED. pybind11 v2.13 was the last to support Python 3.7." 1 include/pybind11/detail/common.h:#if defined(PYPY_VERSION) && !defined(PYBIND11_SIMPLE_GIL_MANAGEMENT) 1 include/pybind11/detail/common.h:#if PY_VERSION_HEX < 0x03080000 1 include/pybind11/detail/common.h: = PYBIND11_TOSTRING(PY_MAJOR_VERSION) "." PYBIND11_TOSTRING(PY_MINOR_VERSION); \ 1 include/pybind11/detail/internals.h: // called. PYBIND11_TLS_FREE is PyThread_tss_free on python 3.7+. On older python, it does 1 include/pybind11/detail/internals.h:#if PYBIND11_INTERNALS_VERSION <= 4 || defined(PYPY_VERSION) 1 include/pybind11/detail/internals.h:// The old Python Thread Local Storage (TLS) API is deprecated in Python 3.7 in favor of the new 1 include/pybind11/detail/type_caster_base.h:#if defined(PYPY_VERSION) 1 include/pybind11/embed.h:# define PYBIND11_PYCONFIG_SUPPORT_PY_VERSION_HEX (0x03080000) 1 include/pybind11/embed.h:#if defined(PYPY_VERSION) 1 include/pybind11/eval.h: // globals if not yet present. Python 3.8 made PyRun_String behave 2 include/pybind11/eval.h:#if defined(PYPY_VERSION) 2 include/pybind11/eval.h: // was missing from PyPy3.8 7.3.7. 2 include/pybind11/gil.h: /// allowed during shutdown. Check _Py_IsFinalizing() on Python 3.7+, and 1 include/pybind11/pybind11.h:#if !defined(PYPY_VERSION) 4 include/pybind11/pybind11.h:#if !defined(PYPY_VERSION) && PY_MAJOR_VERSION == 3 && PY_MINOR_VERSION == 9 1 include/pybind11/pytypes.h:#endif //! defined(PYPY_VERSION) 2 include/pybind11/pytypes.h:#if !defined(PYPY_VERSION) 1 include/pybind11/pytypes.h:# if defined(PYPY_VERSION_NUM) && PYPY_VERSION_NUM < 0x07030a00 1 include/pybind11/pytypes.h:#ifdef PYPY_VERSION 1 include/pybind11/stl/filesystem.h:# if !defined(PYPY_VERSION) 2 pybind11/__init__.py:if sys.version_info < (3, 8): 2 pybind11/__init__.py: msg = "pybind11 does not support Python < 3.8. v2.13 was the last release supporting Python 3.7." 1 pyproject.toml:master.py-version = "3.8" 1 pyproject.toml:python_version = "3.8" 1 README.rst:lines of code and depend on Python (3.8+, or PyPy) and the C++ 2 README.rst:- Python 3.8+, and PyPy3 7.3 are supported with an implementation-agnostic 1 setup.cfg: Programming Language :: Python :: 3.8 1 setup.cfg:python_requires = >=3.8 1 setup.py:# TODO: use literals & overload (typing extensions or Python 3.8) 1 tests/CMakeLists.txt:if(NOT CMAKE_VERSION VERSION_LESS 3.8) 2 tests/constructor_stats.h:#if defined(PYPY_VERSION) 1 tests/env.py: doesn't work on CPython 3.8.0 with pytest==3.3.2 on Ubuntu 18.04 (#2922). 1 tests/requirements.txt:build~=1.0; python_version>="3.8" 1 tests/requirements.txt:numpy~=1.21.5; platform_python_implementation!="PyPy" and python_version>="3.8" and python_version<"3.10" 1 tests/requirements.txt:numpy~=1.23.0; python_version=="3.8" and platform_python_implementation=="PyPy" 1 tests/test_buffers.py: env.PYPY, reason="PyPy 7.3.7 doesn't clear this anymore", strict=False 1 tests/test_builtin_casters.py: # Before Python 3.8, `PyLong_AsLong` does not pick up on `obj.__index__`, 2 tests/test_builtin_casters.py: if (3, 8) <= sys.version_info < (3, 10) and env.CPYTHON: 4 tests/test_builtin_casters.py: # TODO: PyPy 3.8 does not behave like CPython 3.8 here yet (7.3.7) 1 tests/test_callbacks.py: assert m.test_callback3(z.double) == "func(43) = 86" 2 tests/test_call_policies.cpp:#if !defined(PYPY_VERSION) 1 tests/test_chrono.py: diff = m.test_chrono_float_diff(43.789012, 1.123456) 1 tests/test_constants_and_functions.py: assert m.f3(86) == 89 1 tests/test_eigen_matrix.py: a_copy3[8, 1] = 11 1 tests/test_eigen_matrix.py: assert np.all(cornersc == np.array([[1.0, 3], [7, 9]])) 1 tests/test_eigen_matrix.py: assert np.all(cornersr == np.array([[1.0, 3], [7, 9]])) 1 tests/test_eigen_matrix.py: mymat = chol(np.array([[1.0, 2, 4], [2, 13, 23], [4, 23, 77]])) 1 tests/test_exceptions.py: if hasattr(pytest, unraisable): # Python >= 3.8 and pytest >= 6 2 tests/test_exceptions.py:@pytest.mark.xfail(env.PYPY, reason="Failure on PyPy 3.8 (7.3.7)", strict=False) 1 tests/test_factory_constructors.py: assert [i.alive() for i in cstats] == [13, 7] 1 tests/test_kwargs_and_defaults.cpp:#ifdef PYPY_VERSION 1 tests/test_local_bindings.py: assert i1.get3() == 8 1 tests/test_methods_and_attributes.cpp:#if !defined(PYPY_VERSION) 1 tests/test_numpy_array.py: a = np.arange(3 * 7 * 2) + 1 1 tests/test_numpy_array.py: assert str(excinfo.value) == "cannot reshape array of size 42 into shape (3,7,1)" 2 tests/test_numpy_array.py: assert x.shape == (3, 7, 2) 2 tests/test_numpy_array.py: m.reshape_tuple(a, (3, 7, 1)) 2 tests/test_numpy_array.py: x = m.reshape_tuple(a, (3, 7, 2)) 1 tests/test_numpy_vectorize.py: assert np.isclose(m.vectorized_func3(np.array(3 + 7j)), [6 + 14j]) 1 tests/test_pickling.cpp:#if !defined(PYPY_VERSION) 1 tests/test_pytypes.cpp:#if (defined(__APPLE__) && defined(__clang__)) || defined(PYPY_VERSION) 1 tests/test_smart_ptr.cpp: m.def("make_myobject3_1", []() { return new MyObject3(8); }); 1 tests/test_smart_ptr.py: assert cstats.values() == ["MyObject3[9]", "MyObject3[8]", "MyObject3[9]"] 1 tests/test_stl_binders.py: assert v_int2 == m.VectorInt([0, 99, 2, 3, 4, 5, 6, 7, 0, 1, 2, 3, 88]) 1 tests/test_stl_binders.py: assert v_int2 == m.VectorInt([0, 99, 2, 3, 4, 5, 6, 7, 0, 1, 2, 3, 88, 4]) 1 tests/test_type_caster_pyobject_ptr.cpp:#if !defined(PYPY_VERSION) // It is not worth the trouble doing something special for PyPy. 1 tools/FindPythonLibsNew.cmake: set(PythonLibsNew_FIND_VERSION "3.8") 1 tools/JoinPaths.cmake:# https://docs.python.org/3.7/library/os.path.html#os.path.join 1 tools/pybind11NewTools.cmake: Python 3.8 REQUIRED COMPONENTS ${_pybind11_interp_component} ${_pybind11_dev_component} 1 tools/pybind11NewTools.cmake:# Python debug libraries expose slightly different objects before 3.8 1 tools/pybind11Tools.cmake: "3.12;3.11;3.10;3.9;3.8" 1 tools/pybind11Tools.cmake: if(NOT DEFINED PYPY_VERSION) 1 tools/pybind11Tools.cmake: message(STATUS "PYPY ${PYPY_VERSION} (Py ${PYTHON_VERSION})") 1 tools/pybind11Tools.cmake:# Python debug libraries expose slightly different objects before 3.8 1 tools/pybind11Tools.cmake: set(PYPY_VERSION ``` * Change `[tool.ruff]` `target-version` to `"py38"`, as suggested by @Skylion007
2024-07-30 16:18:35 +00:00
find_package(Python 3.8 COMPONENTS Interpreter Development REQUIRED)
find_package(pybind11 CONFIG REQUIRED)
# or add_subdirectory(pybind11)
pybind11_add_module(example example.cpp)
You can also use the targets (as listed below) with FindPython. If you define
``PYBIND11_FINDPYTHON``, pybind11 will perform the FindPython step for you
(mostly useful when building pybind11's own tests, or as a way to change search
algorithms from the CMake invocation, with ``-DPYBIND11_FINDPYTHON=ON``.
.. warning::
If you use FindPython to multi-target Python versions, use the individual
targets listed below, and avoid targets that directly include Python parts.
There are `many ways to hint or force a discovery of a specific Python
installation <https://cmake.org/cmake/help/latest/module/FindPython.html>`_),
setting ``Python_ROOT_DIR`` may be the most common one (though with
virtualenv/venv support, and Conda support, this tends to find the correct
Python version more often than the old system did).
.. warning::
When the Python libraries (i.e. ``libpythonXX.a`` and ``libpythonXX.so``
on Unix) are not available, as is the case on a manylinux image, the
``Development`` component will not be resolved by ``FindPython``. When not
using the embedding functionality, CMake 3.18+ allows you to specify
``Development.Module`` instead of ``Development`` to resolve this issue.
.. versionadded:: 2.6
Advanced: interface library targets
-----------------------------------
Pybind11 supports modern CMake usage patterns with a set of interface targets,
available in all modes. The targets provided are:
``pybind11::headers``
Just the pybind11 headers and minimum compile requirements
``pybind11::pybind11``
Python headers + ``pybind11::headers``
``pybind11::python_link_helper``
Just the "linking" part of pybind11:module
``pybind11::module``
Everything for extension modules - ``pybind11::pybind11`` + ``Python::Module`` (FindPython) or ``pybind11::python_link_helper``
``pybind11::embed``
2022-02-06 00:11:59 +00:00
Everything for embedding the Python interpreter - ``pybind11::pybind11`` + ``Python::Python`` (FindPython) or Python libs
``pybind11::lto`` / ``pybind11::thin_lto``
An alternative to `INTERPROCEDURAL_OPTIMIZATION` for adding link-time optimization.
``pybind11::windows_extras``
``/bigobj`` and ``/mp`` for MSVC.
2020-09-04 21:31:05 +00:00
``pybind11::opt_size``
``/Os`` for MSVC, ``-Os`` for other compilers. Does nothing for debug builds.
Two helper functions are also provided:
``pybind11_strip(target)``
Strips a target (uses ``CMAKE_STRIP`` after the target is built)
``pybind11_extension(target)``
Sets the correct extension (with SOABI) for a target.
You can use these targets to build complex applications. For example, the
``add_python_module`` function is identical to:
.. code-block:: cmake
cmake_minimum_required(VERSION 3.15...3.30)
project(example LANGUAGES CXX)
find_package(pybind11 REQUIRED) # or add_subdirectory(pybind11)
add_library(example MODULE main.cpp)
target_link_libraries(example PRIVATE pybind11::module pybind11::lto pybind11::windows_extras)
pybind11_extension(example)
if(NOT MSVC AND NOT ${CMAKE_BUILD_TYPE} MATCHES Debug|RelWithDebInfo)
# Strip unnecessary sections of the binary on Linux/macOS
pybind11_strip(example)
endif()
set_target_properties(example PROPERTIES CXX_VISIBILITY_PRESET "hidden"
CUDA_VISIBILITY_PRESET "hidden")
Instead of setting properties, you can set ``CMAKE_*`` variables to initialize these correctly.
.. warning::
Since pybind11 is a metatemplate library, it is crucial that certain
compiler flags are provided to ensure high quality code generation. In
contrast to the ``pybind11_add_module()`` command, the CMake interface
provides a *composable* set of targets to ensure that you retain flexibility.
2021-06-17 20:39:59 +00:00
It can be especially important to provide or set these properties; the
:ref:`FAQ <faq:symhidden>` contains an explanation on why these are needed.
.. versionadded:: 2.6
.. _nopython-mode:
Advanced: NOPYTHON mode
-----------------------
If you want complete control, you can set ``PYBIND11_NOPYTHON`` to completely
disable Python integration (this also happens if you run ``FindPython2`` and
``FindPython3`` without running ``FindPython``). This gives you complete
freedom to integrate into an existing system (like `Scikit-Build's
<https://scikit-build.readthedocs.io>`_ ``PythonExtensions``).
``pybind11_add_module`` and ``pybind11_extension`` will be unavailable, and the
targets will be missing any Python specific behavior.
.. versionadded:: 2.6
Embedding the Python interpreter
--------------------------------
In addition to extension modules, pybind11 also supports embedding Python into
a C++ executable or library. In CMake, simply link with the ``pybind11::embed``
target. It provides everything needed to get the interpreter running. The Python
headers and libraries are attached to the target. Unlike ``pybind11::module``,
there is no need to manually set any additional properties here. For more
information about usage in C++, see :doc:`/advanced/embedding`.
.. code-block:: cmake
cmake_minimum_required(VERSION 3.15...3.30)
project(example LANGUAGES CXX)
find_package(pybind11 REQUIRED) # or add_subdirectory(pybind11)
add_executable(example main.cpp)
target_link_libraries(example PRIVATE pybind11::embed)
.. _building_manually:
Building manually
=================
pybind11 is a header-only library, hence it is not necessary to link against
any special libraries and there are no intermediate (magic) translation steps.
On Linux, you can compile an example such as the one given in
:ref:`simple_example` using the following command:
.. code-block:: bash
$ c++ -O3 -Wall -shared -std=c++11 -fPIC $(python3 -m pybind11 --includes) example.cpp -o example$(python3-config --extension-suffix)
The ``python3 -m pybind11 --includes`` command fetches the include paths for
both pybind11 and Python headers. This assumes that pybind11 has been installed
using ``pip`` or ``conda``. If it hasn't, you can also manually specify
``-I <path-to-pybind11>/include`` together with the Python includes path
``python3-config --includes``.
On macOS: the build command is almost the same but it also requires passing
the ``-undefined dynamic_lookup`` flag so as to ignore missing symbols when
building the module:
.. code-block:: bash
$ c++ -O3 -Wall -shared -std=c++11 -undefined dynamic_lookup $(python3 -m pybind11 --includes) example.cpp -o example$(python3-config --extension-suffix)
In general, it is advisable to include several additional build parameters
that can considerably reduce the size of the created binary. Refer to section
:ref:`cmake` for a detailed example of a suitable cross-platform CMake-based
build system that works on all platforms including Windows.
.. note::
On Linux and macOS, it's better to (intentionally) not link against
``libpython``. The symbols will be resolved when the extension library
is loaded into a Python binary. This is preferable because you might
have several different installations of a given Python version (e.g. the
system-provided Python, and one that ships with a piece of commercial
software). In this way, the plugin will work with both versions, instead
of possibly importing a second Python library into a process that already
contains one (which will lead to a segfault).
Building with Bazel
===================
You can build with the Bazel build system using the `pybind11_bazel
<https://github.com/pybind/pybind11_bazel>`_ repository.
Building with Meson
===================
You can use Meson, which has support for ``pybind11`` as a dependency (internally
relying on our ``pkg-config`` support). See the :ref:`module example above <meson-example>`.
Generating binding code automatically
=====================================
The ``Binder`` project is a tool for automatic generation of pybind11 binding
code by introspecting existing C++ codebases using LLVM/Clang. See the
[binder]_ documentation for details.
.. [binder] http://cppbinder.readthedocs.io/en/latest/about.html
[AutoWIG]_ is a Python library that wraps automatically compiled libraries into
high-level languages. It parses C++ code using LLVM/Clang technologies and
generates the wrappers using the Mako templating engine. The approach is automatic,
extensible, and applies to very complex C++ libraries, composed of thousands of
classes or incorporating modern meta-programming constructs.
.. [AutoWIG] https://github.com/StatisKit/AutoWIG
[robotpy-build]_ is a is a pure python, cross platform build tool that aims to
simplify creation of python wheels for pybind11 projects, and provide
cross-project dependency management. Additionally, it is able to autogenerate
customizable pybind11-based wrappers by parsing C++ header files.
.. [robotpy-build] https://robotpy-build.readthedocs.io
[litgen]_ is an automatic python bindings generator with a focus on generating
documented and discoverable bindings: bindings will nicely reproduce the documentation
found in headers. It is based on srcML (srcml.org), a highly scalable, multi-language
parsing tool with a developer centric approach. The API that you want to expose to python
must be C++14 compatible (but your implementation can use more modern constructs).
.. [litgen] https://pthom.github.io/litgen