pybind11/tests/test_cpp_conduit.py

167 lines
5.5 KiB
Python
Raw Normal View History

Enable type-safe interoperability between different independent Python/C++ bindings systems. (#5296) * `self.__cpp_transporter__()` proof of concept: Enable passing C++ pointers across extensions even if the `PYBIND11_INTERNALS_VERSION`s do not match. * Include cleanup (mainly to resolve PyPy build failures). * Fix clang-tidy errors. * Resolve `error: extra * factor out platform_abi_id.h from internals.h (no functional changes) * factor out internals_version.h from internals.h (no functional changes) * Update CMakeLists.txt, tests/extra_python_package/test_files.py * Revert "factor out internals_version.h from internals.h (no functional changes)" This reverts commit 3ccea8cd4302cfaf2b6842f77ee3a2eefd61182a. * Remove internals_version.h from CMakeLists.txt, tests/extra_python_package/test_files.py * `.__cpp_transporter__()` implementation: compare `pybind11_platform_abi_id`, `cpp_typeid_name` * Add PremiumTraveler * Rename test_cpp_transporter_traveler_type.h -> test_cpp_transporter_traveler_types.h * Expand tests: `PremiumTraveler`, `get_points()` * Shuffle order of tests (no real changes). * Move `__cpp_transporter__` lambda to `py::cpp_transporter()` regular function. * Use `type_caster_generic::load(self)` instead of `cast<T *>(self)` * Pass `const std::type_info *` via `py::capsule` (instead of `cpp_typeid_name`). * Make platform_abi_id.h completely stand-alone. * rename exo_planet.cpp -> exo_planet_pybind11.cpp * Add exo_planet_c_api.cpp (incomplete). * Fix silly oversight (wrong filename in `#include`). * Resolve clang-tidy errors: ``` /__w/pybind11/pybind11/tests/exo_planet_c_api.cpp:10:18: error: 'wrapGetLuggage' is a static definition in anonymous namespace; static is redundant here [readability-static-definition-in-anonymous-namespace,-warnings-as-errors] 10 | static PyObject *wrapGetLuggage(PyObject *, PyObject *) { return PyUnicode_FromString("TODO"); } | ~~~~~~ ^ /__w/pybind11/pybind11/tests/exo_planet_c_api.cpp:14:20: error: 'ThisMethodDef' is a static definition in anonymous namespace; static is redundant here [readability-static-definition-in-anonymous-namespace,-warnings-as-errors] 14 | static PyMethodDef ThisMethodDef[] | ~~~~~~ ^ /__w/pybind11/pybind11/tests/exo_planet_c_api.cpp:17:27: error: 'ThisModuleDef' is a static definition in anonymous namespace; static is redundant here [readability-static-definition-in-anonymous-namespace,-warnings-as-errors] 17 | static struct PyModuleDef ThisModuleDef = { | ~~~~~~ ^ ``` * Implement exo_planet_c_api GetLuggage(), GetPoints() * Move new code from test_cpp_transporter_traveler_bindings.h to pybind11/detail/type_caster_base.h, under the name `class_dunder_cpp_transporter()` * Fix oversight. * Unconditionally add `__cpp_transporter__` method to all `py::class_` objects, but do not include that magic method in docstring signatures. * Back out pybind11/detail/platform_abi_id.h for now. Maximizing reusability can be handled separately, later. * Small cleanup. * Restore and add to `test_call_cpp_transporter_*()` * Ensure https://github.com/pybind/pybind11/issues/3788 does not bite again. * `class_dunder_cpp_transporter()`: replace `obj.cast<std::string>()` with `std::string(obj)` * Add (simple) copyright notices in all newly added files. * Globally replace cpp_transporter with cpp_conduit * style: pre-commit fixes * IWYU fixes * Rename `class_dunder_cpp_conduit()` -> `cpp_conduit_method()` * Change `pybind11_platform_abi_id`, `pointer_kind` argument types from `str` to `bytes`. This avoids the unicode decode/encode roundtrips: * More robust (no decode/encode errors). * Minor runtime optimization. * Systematically rename `cap_cpp_type_info` -> `cpp_type_info_capsule` (no functional changes). * Systematically replace `cpp_type_info_capsule` `name`: `"const std::type_info *"` -> `typeid(std::type_info).name()` (this IS a functional change). This provides an extra layer of protection against C++ ABI mismatches: * The first and most important layer is that the `PYBIND11_PLATFORM_ABI_ID`s must match between extensions. * The second layer is that the `typeid(std::type_info).name()`s must match between extensions. * Fix sort order accident in tests/CMakeLists.txt * Apply suggestions from code review Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com> * style: pre-commit fixes * refactor: rename to _pybind_conduit_v1_ Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com> * Add test_home_planet_wrap_very_lonely_traveler(), test_exo_planet_pybind11_wrap_very_lonely_traveler() * Resolve clang-tidy errors: ``` /__w/pybind11/pybind11/tests/test_cpp_conduit_traveler_bindings.h:39:32: error: parameter 'm' is passed by value and only copied once; consider moving it to avoid unnecessary copies [performance-unnecessary-value-param,-warnings-as-errors] 10 | py::class_<LonelyTraveler>(m, "LonelyTraveler"); | ^ | std::move( ) /__w/pybind11/pybind11/tests/test_cpp_conduit_traveler_bindings.h:43:52: error: parameter 'm' is passed by value and only copied once; consider moving it to avoid unnecessary copies [performance-unnecessary-value-param,-warnings-as-errors] 43 | py::class_<VeryLonelyTraveler, LonelyTraveler>(m, "VeryLonelyTraveler"); | ^ | std::move( ) ``` --------- Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com> Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com> Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com>
2024-09-13 04:18:29 +00:00
# Copyright (c) 2024 The pybind Community.
from __future__ import annotations
import exo_planet_c_api
import exo_planet_pybind11
import home_planet_very_lonely_traveler
import pytest
import env
Enable type-safe interoperability between different independent Python/C++ bindings systems. (#5296) * `self.__cpp_transporter__()` proof of concept: Enable passing C++ pointers across extensions even if the `PYBIND11_INTERNALS_VERSION`s do not match. * Include cleanup (mainly to resolve PyPy build failures). * Fix clang-tidy errors. * Resolve `error: extra * factor out platform_abi_id.h from internals.h (no functional changes) * factor out internals_version.h from internals.h (no functional changes) * Update CMakeLists.txt, tests/extra_python_package/test_files.py * Revert "factor out internals_version.h from internals.h (no functional changes)" This reverts commit 3ccea8cd4302cfaf2b6842f77ee3a2eefd61182a. * Remove internals_version.h from CMakeLists.txt, tests/extra_python_package/test_files.py * `.__cpp_transporter__()` implementation: compare `pybind11_platform_abi_id`, `cpp_typeid_name` * Add PremiumTraveler * Rename test_cpp_transporter_traveler_type.h -> test_cpp_transporter_traveler_types.h * Expand tests: `PremiumTraveler`, `get_points()` * Shuffle order of tests (no real changes). * Move `__cpp_transporter__` lambda to `py::cpp_transporter()` regular function. * Use `type_caster_generic::load(self)` instead of `cast<T *>(self)` * Pass `const std::type_info *` via `py::capsule` (instead of `cpp_typeid_name`). * Make platform_abi_id.h completely stand-alone. * rename exo_planet.cpp -> exo_planet_pybind11.cpp * Add exo_planet_c_api.cpp (incomplete). * Fix silly oversight (wrong filename in `#include`). * Resolve clang-tidy errors: ``` /__w/pybind11/pybind11/tests/exo_planet_c_api.cpp:10:18: error: 'wrapGetLuggage' is a static definition in anonymous namespace; static is redundant here [readability-static-definition-in-anonymous-namespace,-warnings-as-errors] 10 | static PyObject *wrapGetLuggage(PyObject *, PyObject *) { return PyUnicode_FromString("TODO"); } | ~~~~~~ ^ /__w/pybind11/pybind11/tests/exo_planet_c_api.cpp:14:20: error: 'ThisMethodDef' is a static definition in anonymous namespace; static is redundant here [readability-static-definition-in-anonymous-namespace,-warnings-as-errors] 14 | static PyMethodDef ThisMethodDef[] | ~~~~~~ ^ /__w/pybind11/pybind11/tests/exo_planet_c_api.cpp:17:27: error: 'ThisModuleDef' is a static definition in anonymous namespace; static is redundant here [readability-static-definition-in-anonymous-namespace,-warnings-as-errors] 17 | static struct PyModuleDef ThisModuleDef = { | ~~~~~~ ^ ``` * Implement exo_planet_c_api GetLuggage(), GetPoints() * Move new code from test_cpp_transporter_traveler_bindings.h to pybind11/detail/type_caster_base.h, under the name `class_dunder_cpp_transporter()` * Fix oversight. * Unconditionally add `__cpp_transporter__` method to all `py::class_` objects, but do not include that magic method in docstring signatures. * Back out pybind11/detail/platform_abi_id.h for now. Maximizing reusability can be handled separately, later. * Small cleanup. * Restore and add to `test_call_cpp_transporter_*()` * Ensure https://github.com/pybind/pybind11/issues/3788 does not bite again. * `class_dunder_cpp_transporter()`: replace `obj.cast<std::string>()` with `std::string(obj)` * Add (simple) copyright notices in all newly added files. * Globally replace cpp_transporter with cpp_conduit * style: pre-commit fixes * IWYU fixes * Rename `class_dunder_cpp_conduit()` -> `cpp_conduit_method()` * Change `pybind11_platform_abi_id`, `pointer_kind` argument types from `str` to `bytes`. This avoids the unicode decode/encode roundtrips: * More robust (no decode/encode errors). * Minor runtime optimization. * Systematically rename `cap_cpp_type_info` -> `cpp_type_info_capsule` (no functional changes). * Systematically replace `cpp_type_info_capsule` `name`: `"const std::type_info *"` -> `typeid(std::type_info).name()` (this IS a functional change). This provides an extra layer of protection against C++ ABI mismatches: * The first and most important layer is that the `PYBIND11_PLATFORM_ABI_ID`s must match between extensions. * The second layer is that the `typeid(std::type_info).name()`s must match between extensions. * Fix sort order accident in tests/CMakeLists.txt * Apply suggestions from code review Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com> * style: pre-commit fixes * refactor: rename to _pybind_conduit_v1_ Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com> * Add test_home_planet_wrap_very_lonely_traveler(), test_exo_planet_pybind11_wrap_very_lonely_traveler() * Resolve clang-tidy errors: ``` /__w/pybind11/pybind11/tests/test_cpp_conduit_traveler_bindings.h:39:32: error: parameter 'm' is passed by value and only copied once; consider moving it to avoid unnecessary copies [performance-unnecessary-value-param,-warnings-as-errors] 10 | py::class_<LonelyTraveler>(m, "LonelyTraveler"); | ^ | std::move( ) /__w/pybind11/pybind11/tests/test_cpp_conduit_traveler_bindings.h:43:52: error: parameter 'm' is passed by value and only copied once; consider moving it to avoid unnecessary copies [performance-unnecessary-value-param,-warnings-as-errors] 43 | py::class_<VeryLonelyTraveler, LonelyTraveler>(m, "VeryLonelyTraveler"); | ^ | std::move( ) ``` --------- Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com> Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com> Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com>
2024-09-13 04:18:29 +00:00
from pybind11_tests import cpp_conduit as home_planet
def test_traveler_getattr_actually_exists():
t_h = home_planet.Traveler("home")
assert t_h.any_name == "Traveler GetAttr: any_name luggage: home"
def test_premium_traveler_getattr_actually_exists():
t_h = home_planet.PremiumTraveler("home", 7)
assert t_h.secret_name == "PremiumTraveler GetAttr: secret_name points: 7"
def test_call_cpp_conduit_success():
t_h = home_planet.Traveler("home")
cap = t_h._pybind11_conduit_v1_(
home_planet.PYBIND11_PLATFORM_ABI_ID,
home_planet.cpp_type_info_capsule_Traveler,
b"raw_pointer_ephemeral",
)
assert cap.__class__.__name__ == "PyCapsule" or (
# Note: this will become unnecessary in the next GraalPy release
env.GRAALPY and cap.__class__.__name__ == "capsule"
)
Enable type-safe interoperability between different independent Python/C++ bindings systems. (#5296) * `self.__cpp_transporter__()` proof of concept: Enable passing C++ pointers across extensions even if the `PYBIND11_INTERNALS_VERSION`s do not match. * Include cleanup (mainly to resolve PyPy build failures). * Fix clang-tidy errors. * Resolve `error: extra * factor out platform_abi_id.h from internals.h (no functional changes) * factor out internals_version.h from internals.h (no functional changes) * Update CMakeLists.txt, tests/extra_python_package/test_files.py * Revert "factor out internals_version.h from internals.h (no functional changes)" This reverts commit 3ccea8cd4302cfaf2b6842f77ee3a2eefd61182a. * Remove internals_version.h from CMakeLists.txt, tests/extra_python_package/test_files.py * `.__cpp_transporter__()` implementation: compare `pybind11_platform_abi_id`, `cpp_typeid_name` * Add PremiumTraveler * Rename test_cpp_transporter_traveler_type.h -> test_cpp_transporter_traveler_types.h * Expand tests: `PremiumTraveler`, `get_points()` * Shuffle order of tests (no real changes). * Move `__cpp_transporter__` lambda to `py::cpp_transporter()` regular function. * Use `type_caster_generic::load(self)` instead of `cast<T *>(self)` * Pass `const std::type_info *` via `py::capsule` (instead of `cpp_typeid_name`). * Make platform_abi_id.h completely stand-alone. * rename exo_planet.cpp -> exo_planet_pybind11.cpp * Add exo_planet_c_api.cpp (incomplete). * Fix silly oversight (wrong filename in `#include`). * Resolve clang-tidy errors: ``` /__w/pybind11/pybind11/tests/exo_planet_c_api.cpp:10:18: error: 'wrapGetLuggage' is a static definition in anonymous namespace; static is redundant here [readability-static-definition-in-anonymous-namespace,-warnings-as-errors] 10 | static PyObject *wrapGetLuggage(PyObject *, PyObject *) { return PyUnicode_FromString("TODO"); } | ~~~~~~ ^ /__w/pybind11/pybind11/tests/exo_planet_c_api.cpp:14:20: error: 'ThisMethodDef' is a static definition in anonymous namespace; static is redundant here [readability-static-definition-in-anonymous-namespace,-warnings-as-errors] 14 | static PyMethodDef ThisMethodDef[] | ~~~~~~ ^ /__w/pybind11/pybind11/tests/exo_planet_c_api.cpp:17:27: error: 'ThisModuleDef' is a static definition in anonymous namespace; static is redundant here [readability-static-definition-in-anonymous-namespace,-warnings-as-errors] 17 | static struct PyModuleDef ThisModuleDef = { | ~~~~~~ ^ ``` * Implement exo_planet_c_api GetLuggage(), GetPoints() * Move new code from test_cpp_transporter_traveler_bindings.h to pybind11/detail/type_caster_base.h, under the name `class_dunder_cpp_transporter()` * Fix oversight. * Unconditionally add `__cpp_transporter__` method to all `py::class_` objects, but do not include that magic method in docstring signatures. * Back out pybind11/detail/platform_abi_id.h for now. Maximizing reusability can be handled separately, later. * Small cleanup. * Restore and add to `test_call_cpp_transporter_*()` * Ensure https://github.com/pybind/pybind11/issues/3788 does not bite again. * `class_dunder_cpp_transporter()`: replace `obj.cast<std::string>()` with `std::string(obj)` * Add (simple) copyright notices in all newly added files. * Globally replace cpp_transporter with cpp_conduit * style: pre-commit fixes * IWYU fixes * Rename `class_dunder_cpp_conduit()` -> `cpp_conduit_method()` * Change `pybind11_platform_abi_id`, `pointer_kind` argument types from `str` to `bytes`. This avoids the unicode decode/encode roundtrips: * More robust (no decode/encode errors). * Minor runtime optimization. * Systematically rename `cap_cpp_type_info` -> `cpp_type_info_capsule` (no functional changes). * Systematically replace `cpp_type_info_capsule` `name`: `"const std::type_info *"` -> `typeid(std::type_info).name()` (this IS a functional change). This provides an extra layer of protection against C++ ABI mismatches: * The first and most important layer is that the `PYBIND11_PLATFORM_ABI_ID`s must match between extensions. * The second layer is that the `typeid(std::type_info).name()`s must match between extensions. * Fix sort order accident in tests/CMakeLists.txt * Apply suggestions from code review Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com> * style: pre-commit fixes * refactor: rename to _pybind_conduit_v1_ Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com> * Add test_home_planet_wrap_very_lonely_traveler(), test_exo_planet_pybind11_wrap_very_lonely_traveler() * Resolve clang-tidy errors: ``` /__w/pybind11/pybind11/tests/test_cpp_conduit_traveler_bindings.h:39:32: error: parameter 'm' is passed by value and only copied once; consider moving it to avoid unnecessary copies [performance-unnecessary-value-param,-warnings-as-errors] 10 | py::class_<LonelyTraveler>(m, "LonelyTraveler"); | ^ | std::move( ) /__w/pybind11/pybind11/tests/test_cpp_conduit_traveler_bindings.h:43:52: error: parameter 'm' is passed by value and only copied once; consider moving it to avoid unnecessary copies [performance-unnecessary-value-param,-warnings-as-errors] 43 | py::class_<VeryLonelyTraveler, LonelyTraveler>(m, "VeryLonelyTraveler"); | ^ | std::move( ) ``` --------- Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com> Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com> Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com>
2024-09-13 04:18:29 +00:00
def test_call_cpp_conduit_platform_abi_id_mismatch():
t_h = home_planet.Traveler("home")
cap = t_h._pybind11_conduit_v1_(
home_planet.PYBIND11_PLATFORM_ABI_ID + b"MISMATCH",
home_planet.cpp_type_info_capsule_Traveler,
b"raw_pointer_ephemeral",
)
assert cap is None
def test_call_cpp_conduit_cpp_type_info_capsule_mismatch():
t_h = home_planet.Traveler("home")
cap = t_h._pybind11_conduit_v1_(
home_planet.PYBIND11_PLATFORM_ABI_ID,
home_planet.cpp_type_info_capsule_int,
b"raw_pointer_ephemeral",
)
assert cap is None
def test_call_cpp_conduit_pointer_kind_invalid():
t_h = home_planet.Traveler("home")
with pytest.raises(
RuntimeError, match='^Invalid pointer_kind: "raw_pointer_ephemreal"$'
):
t_h._pybind11_conduit_v1_(
home_planet.PYBIND11_PLATFORM_ABI_ID,
home_planet.cpp_type_info_capsule_Traveler,
b"raw_pointer_ephemreal",
)
def test_home_only_basic():
t_h = home_planet.Traveler("home")
assert t_h.luggage == "home"
assert home_planet.get_luggage(t_h) == "home"
def test_home_only_premium():
p_h = home_planet.PremiumTraveler("home", 2)
assert p_h.luggage == "home"
assert home_planet.get_luggage(p_h) == "home"
assert home_planet.get_points(p_h) == 2
def test_exo_only_basic():
t_e = exo_planet_pybind11.Traveler("exo")
assert t_e.luggage == "exo"
assert exo_planet_pybind11.get_luggage(t_e) == "exo"
def test_exo_only_premium():
p_e = exo_planet_pybind11.PremiumTraveler("exo", 3)
assert p_e.luggage == "exo"
assert exo_planet_pybind11.get_luggage(p_e) == "exo"
assert exo_planet_pybind11.get_points(p_e) == 3
def test_home_passed_to_exo_basic():
t_h = home_planet.Traveler("home")
assert exo_planet_pybind11.get_luggage(t_h) == "home"
def test_exo_passed_to_home_basic():
t_e = exo_planet_pybind11.Traveler("exo")
assert home_planet.get_luggage(t_e) == "exo"
def test_home_passed_to_exo_premium():
p_h = home_planet.PremiumTraveler("home", 2)
assert exo_planet_pybind11.get_luggage(p_h) == "home"
assert exo_planet_pybind11.get_points(p_h) == 2
def test_exo_passed_to_home_premium():
p_e = exo_planet_pybind11.PremiumTraveler("exo", 3)
assert home_planet.get_luggage(p_e) == "exo"
assert home_planet.get_points(p_e) == 3
@pytest.mark.parametrize(
"traveler_type", [home_planet.Traveler, exo_planet_pybind11.Traveler]
)
def test_exo_planet_c_api_traveler(traveler_type):
t = traveler_type("socks")
assert exo_planet_c_api.GetLuggage(t) == "socks"
@pytest.mark.parametrize(
"premium_traveler_type",
[home_planet.PremiumTraveler, exo_planet_pybind11.PremiumTraveler],
)
def test_exo_planet_c_api_premium_traveler(premium_traveler_type):
pt = premium_traveler_type("gucci", 5)
assert exo_planet_c_api.GetLuggage(pt) == "gucci"
assert exo_planet_c_api.GetPoints(pt) == 5
def test_home_planet_wrap_very_lonely_traveler():
# This does not exercise the cpp_conduit feature, but is here to
# demonstrate that the cpp_conduit feature does not solve all
# cross-extension interoperability issues.
# Here is the proof that the following works for extensions with
# matching `PYBIND11_INTERNALS_ID`s:
# test_cpp_conduit.cpp:
# py::class_<LonelyTraveler>
# home_planet_very_lonely_traveler.cpp:
# py::class_<VeryLonelyTraveler, LonelyTraveler>
# See test_exo_planet_pybind11_wrap_very_lonely_traveler() for the negative
# test.
assert home_planet.LonelyTraveler is not None # Verify that the base class exists.
home_planet_very_lonely_traveler.wrap_very_lonely_traveler()
# Ensure that the derived class exists.
assert home_planet_very_lonely_traveler.VeryLonelyTraveler is not None
def test_exo_planet_pybind11_wrap_very_lonely_traveler():
# See comment under test_home_planet_wrap_very_lonely_traveler() first.
# Here the `PYBIND11_INTERNALS_ID`s don't match between:
# test_cpp_conduit.cpp:
# py::class_<LonelyTraveler>
# exo_planet_pybind11.cpp:
# py::class_<VeryLonelyTraveler, LonelyTraveler>
assert home_planet.LonelyTraveler is not None # Verify that the base class exists.
with pytest.raises(
RuntimeError,
match='^generic_type: type "VeryLonelyTraveler" referenced unknown base type '
'"pybind11_tests::test_cpp_conduit::LonelyTraveler"$',
):
exo_planet_pybind11.wrap_very_lonely_traveler()