Merge branch 'master' into sh_merge_master

This commit is contained in:
Ralf W. Grosse-Kunstleve 2024-12-20 01:44:48 -08:00
commit 4629fbc4d5
No known key found for this signature in database
25 changed files with 693 additions and 140 deletions

View File

@ -365,10 +365,6 @@ jobs:
strategy:
fail-fast: false
matrix:
clang:
- dev
std:
- 11
container_suffix:
- ""
include:

View File

@ -23,7 +23,7 @@ jobs:
submodules: true
fetch-depth: 0
- uses: pypa/cibuildwheel@v2.21
- uses: pypa/cibuildwheel@v2.22
env:
PYODIDE_BUILD_EXPORTS: whole_archive
with:

View File

@ -104,7 +104,7 @@ jobs:
- uses: actions/download-artifact@v4
- name: Generate artifact attestation for sdist and wheel
uses: actions/attest-build-provenance@ef244123eb79f2f7a7e75d99086184180e6d0018 # v1.4.4
uses: actions/attest-build-provenance@7668571508540a607bdfd90a87a560489fe372eb # v2.1.0
with:
subject-path: "*/pybind11*"

View File

@ -25,14 +25,14 @@ repos:
# Clang format the codebase automatically
- repo: https://github.com/pre-commit/mirrors-clang-format
rev: "v19.1.3"
rev: "v19.1.4"
hooks:
- id: clang-format
types_or: [c++, c, cuda]
# Ruff, the Python auto-correcting linter/formatter written in Rust
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.7.2
rev: v0.8.1
hooks:
- id: ruff
args: ["--fix", "--show-fixes"]
@ -95,7 +95,7 @@ repos:
# Avoid directional quotes
- repo: https://github.com/sirosen/texthooks
rev: "0.6.7"
rev: "0.6.8"
hooks:
- id: fix-ligatures
- id: fix-smartquotes
@ -144,14 +144,14 @@ repos:
# PyLint has native support - not always usable, but works for us
- repo: https://github.com/PyCQA/pylint
rev: "v3.3.1"
rev: "v3.3.2"
hooks:
- id: pylint
files: ^pybind11
# Check schemas on some of our YAML files
- repo: https://github.com/python-jsonschema/check-jsonschema
rev: 0.29.4
rev: 0.30.0
hooks:
- id: check-readthedocs
- id: check-github-workflows

View File

@ -1,35 +1,53 @@
Custom type casters
===================
In very rare cases, applications may require custom type casters that cannot be
expressed using the abstractions provided by pybind11, thus requiring raw
Python C API calls. This is fairly advanced usage and should only be pursued by
experts who are familiar with the intricacies of Python reference counting.
Some applications may prefer custom type casters that convert between existing
Python types and C++ types, similar to the ``list````std::vector``
and ``dict````std::map`` conversions which are built into pybind11.
Implementing custom type casters is fairly advanced usage.
While it is recommended to use the pybind11 API as much as possible, more complex examples may
require familiarity with the intricacies of the Python C API.
You can refer to the `Python/C API Reference Manual <https://docs.python.org/3/c-api/index.html>`_
for more information.
The following snippets demonstrate how this works for a very simple ``inty``
type that that should be convertible from Python types that provide a
``__int__(self)`` method.
The following snippets demonstrate how this works for a very simple ``Point2D`` type.
We want this type to be convertible to C++ from Python types implementing the
``Sequence`` protocol and having two elements of type ``float``.
When returned from C++ to Python, it should be converted to a Python ``tuple[float, float]``.
For this type we could provide Python bindings for different arithmetic functions implemented
in C++ (here demonstrated by a simple ``negate`` function).
..
PLEASE KEEP THE CODE BLOCKS IN SYNC WITH
tests/test_docs_advanced_cast_custom.cpp
tests/test_docs_advanced_cast_custom.py
Ideally, change the test, run pre-commit (incl. clang-format),
then copy the changed code back here.
Also use TEST_SUBMODULE in tests, but PYBIND11_MODULE in docs.
.. code-block:: cpp
struct inty { long long_value; };
namespace user_space {
void print(inty s) {
std::cout << s.long_value << std::endl;
}
struct Point2D {
double x;
double y;
};
The following Python snippet demonstrates the intended usage from the Python side:
Point2D negate(const Point2D &point) { return Point2D{-point.x, -point.y}; }
} // namespace user_space
The following Python snippet demonstrates the intended usage of ``negate`` from the Python side:
.. code-block:: python
class A:
def __int__(self):
return 123
from my_math_module import docs_advanced_cast_custom as m
from example import print
print(A())
point1 = [1.0, -1.0]
point2 = m.negate(point1)
assert point2 == (-1.0, 1.0)
To register the necessary conversion routines, it is necessary to add an
instantiation of the ``pybind11::detail::type_caster<T>`` template.
@ -38,47 +56,59 @@ type is explicitly allowed.
.. code-block:: cpp
namespace PYBIND11_NAMESPACE { namespace detail {
template <> struct type_caster<inty> {
public:
/**
* This macro establishes the name 'inty' in
* function signatures and declares a local variable
* 'value' of type inty
*/
PYBIND11_TYPE_CASTER(inty, const_name("inty"));
namespace pybind11 {
namespace detail {
/**
* Conversion part 1 (Python->C++): convert a PyObject into a inty
* instance or return false upon failure. The second argument
* indicates whether implicit conversions should be applied.
*/
bool load(handle src, bool) {
/* Extract PyObject from handle */
PyObject *source = src.ptr();
/* Try converting into a Python integer value */
PyObject *tmp = PyNumber_Long(source);
if (!tmp)
template <>
struct type_caster<user_space::Point2D> {
// This macro inserts a lot of boilerplate code and sets the default type hint to `tuple`
PYBIND11_TYPE_CASTER(user_space::Point2D, const_name("tuple"));
// `arg_name` and `return_name` may optionally be used to specify type hints separately for
// arguments and return values.
// The signature of our negate function would then look like:
// `negate(Sequence[float]) -> tuple[float, float]`
static constexpr auto arg_name = const_name("Sequence[float]");
static constexpr auto return_name = const_name("tuple[float, float]");
// C++ -> Python: convert `Point2D` to `tuple[float, float]`. The second and third arguments
// are used to indicate the return value policy and parent object (for
// return_value_policy::reference_internal) and are often ignored by custom casters.
// The return value should reflect the type hint specified by `return_name`.
static handle
cast(const user_space::Point2D &number, return_value_policy /*policy*/, handle /*parent*/) {
return py::make_tuple(number.x, number.y).release();
}
// Python -> C++: convert a `PyObject` into a `Point2D` and return false upon failure. The
// second argument indicates whether implicit conversions should be allowed.
// The accepted types should reflect the type hint specified by `arg_name`.
bool load(handle src, bool /*convert*/) {
// Check if handle is a Sequence
if (!py::isinstance<py::sequence>(src)) {
return false;
}
auto seq = py::reinterpret_borrow<py::sequence>(src);
// Check if exactly two values are in the Sequence
if (seq.size() != 2) {
return false;
}
// Check if each element is either a float or an int
for (auto item : seq) {
if (!py::isinstance<py::float_>(item) && !py::isinstance<py::int_>(item)) {
return false;
/* Now try to convert into a C++ int */
value.long_value = PyLong_AsLong(tmp);
Py_DECREF(tmp);
/* Ensure return code was OK (to avoid out-of-range errors etc) */
return !(value.long_value == -1 && !PyErr_Occurred());
}
}
value.x = seq[0].cast<double>();
value.y = seq[1].cast<double>();
return true;
}
};
/**
* Conversion part 2 (C++ -> Python): convert an inty instance into
* a Python object. The second and third arguments are used to
* indicate the return value policy and parent object (for
* ``return_value_policy::reference_internal``) and are generally
* ignored by implicit casters.
*/
static handle cast(inty src, return_value_policy /* policy */, handle /* parent */) {
return PyLong_FromLong(src.long_value);
}
};
}} // namespace PYBIND11_NAMESPACE::detail
} // namespace detail
} // namespace pybind11
// Bind the negate function
PYBIND11_MODULE(docs_advanced_cast_custom, m) { m.def("negate", user_space::negate); }
.. note::
@ -86,8 +116,22 @@ type is explicitly allowed.
that ``T`` is default-constructible (``value`` is first default constructed
and then ``load()`` assigns to it).
.. note::
For further information on the ``return_value_policy`` argument of ``cast`` refer to :ref:`return_value_policies`.
To learn about the ``convert`` argument of ``load`` see :ref:`nonconverting_arguments`.
.. warning::
When using custom type casters, it's important to declare them consistently
in every compilation unit of the Python extension module. Otherwise,
in every compilation unit of the Python extension module to satisfy the C++ One Definition Rule
(`ODR <https://en.cppreference.com/w/cpp/language/definition>`_). Otherwise,
undefined behavior can ensue.
.. note::
Using the type hint ``Sequence[float]`` signals to static type checkers, that not only tuples may be
passed, but any type implementing the Sequence protocol, e.g., ``list[float]``.
Unfortunately, that loses the length information ``tuple[float, float]`` provides.
One way of still providing some length information in type hints is using ``typing.Annotated``, e.g.,
``Annotated[Sequence[float], 2]``, or further add libraries like
`annotated-types <https://github.com/annotated-types/annotated-types>`_.

View File

@ -151,7 +151,7 @@ as arguments and return values, refer to the section on binding :ref:`classes`.
+------------------------------------+---------------------------+-----------------------------------+
| ``std::variant<...>`` | Type-safe union (C++17) | :file:`pybind11/stl.h` |
+------------------------------------+---------------------------+-----------------------------------+
| ``std::filesystem::path<T>`` | STL path (C++17) [#]_ | :file:`pybind11/stl/filesystem.h` |
| ``std::filesystem::path`` | STL path (C++17) [#]_ | :file:`pybind11/stl/filesystem.h` |
+------------------------------------+---------------------------+-----------------------------------+
| ``std::function<...>`` | STL polymorphic function | :file:`pybind11/functional.h` |
+------------------------------------+---------------------------+-----------------------------------+
@ -167,4 +167,4 @@ as arguments and return values, refer to the section on binding :ref:`classes`.
+------------------------------------+---------------------------+-----------------------------------+
.. [#] ``std::filesystem::path`` is converted to ``pathlib.Path`` and
``os.PathLike`` is converted to ``std::filesystem::path``.
can be loaded from ``os.PathLike``, ``str``, and ``bytes``.

View File

@ -48,7 +48,7 @@ def generate_dummy_code_boost(nclasses=10):
decl += "\n"
for cl in range(nclasses):
decl += "class cl%03i {\n" % cl
decl += f"class cl{cl:03} {{\n"
decl += "public:\n"
bindings += f' py::class_<cl{cl:03}>("cl{cl:03}")\n'
for fn in range(nfns):
@ -85,5 +85,5 @@ for codegen in [generate_dummy_code_pybind11, generate_dummy_code_boost]:
n2 = dt.datetime.now()
elapsed = (n2 - n1).total_seconds()
size = os.stat("test.so").st_size
print(" {%i, %f, %i}," % (nclasses * nfns, elapsed, size))
print(f" {{{nclasses * nfns}, {elapsed:.6f}, {size}}},")
print("}")

View File

@ -34,6 +34,39 @@ PYBIND11_WARNING_DISABLE_MSVC(4127)
PYBIND11_NAMESPACE_BEGIN(detail)
// Type trait checker for `descr`
template <typename>
struct is_descr : std::false_type {};
template <size_t N, typename... Ts>
struct is_descr<descr<N, Ts...>> : std::true_type {};
template <size_t N, typename... Ts>
struct is_descr<const descr<N, Ts...>> : std::true_type {};
// Use arg_name instead of name when available
template <typename T, typename SFINAE = void>
struct as_arg_type {
static constexpr auto name = T::name;
};
template <typename T>
struct as_arg_type<T, typename std::enable_if<is_descr<decltype(T::arg_name)>::value>::type> {
static constexpr auto name = T::arg_name;
};
// Use return_name instead of name when available
template <typename T, typename SFINAE = void>
struct as_return_type {
static constexpr auto name = T::name;
};
template <typename T>
struct as_return_type<T,
typename std::enable_if<is_descr<decltype(T::return_name)>::value>::type> {
static constexpr auto name = T::return_name;
};
template <typename type, typename SFINAE = void>
class type_caster : public type_caster_base<type> {};
template <typename type>
@ -1140,18 +1173,20 @@ using type_caster_holder = conditional_t<is_copy_constructible<holder_type>::val
copyable_holder_caster<type, holder_type>,
move_only_holder_caster<type, holder_type>>;
template <typename T, bool Value = false>
struct always_construct_holder {
template <bool Value = false>
struct always_construct_holder_value {
static constexpr bool value = Value;
};
template <typename T, bool Value = false>
struct always_construct_holder : always_construct_holder_value<Value> {};
/// Create a specialization for custom holder types (silently ignores std::shared_ptr)
#define PYBIND11_DECLARE_HOLDER_TYPE(type, holder_type, ...) \
PYBIND11_NAMESPACE_BEGIN(PYBIND11_NAMESPACE) \
namespace detail { \
template <typename type> \
struct always_construct_holder<holder_type> : always_construct_holder<void, ##__VA_ARGS__> { \
}; \
struct always_construct_holder<holder_type> : always_construct_holder_value<__VA_ARGS__> {}; \
template <typename type> \
class type_caster<holder_type, enable_if_t<!is_shared_ptr<holder_type>::value>> \
: public type_caster_holder<type, holder_type> {}; \
@ -1361,6 +1396,8 @@ struct pyobject_caster {
return src.inc_ref();
}
PYBIND11_TYPE_CASTER(type, handle_type_name<type>::name);
static constexpr auto arg_name = as_arg_type<handle_type_name<type>>::name;
static constexpr auto return_name = as_return_type<handle_type_name<type>>::name;
};
template <typename T>
@ -1889,7 +1926,7 @@ public:
"py::args cannot be specified more than once");
static constexpr auto arg_names
= ::pybind11::detail::concat(type_descr(make_caster<Args>::name)...);
= ::pybind11::detail::concat(type_descr(as_arg_type<make_caster<Args>>::name)...);
bool load_args(function_call &call) { return load_impl_sequence(call, indices{}); }

View File

@ -12,51 +12,32 @@
#define PYBIND11_PLATFORM_ABI_ID_STRINGIFY(x) #x
#define PYBIND11_PLATFORM_ABI_ID_TOSTRING(x) PYBIND11_PLATFORM_ABI_ID_STRINGIFY(x)
// On MSVC, debug and release builds are not ABI-compatible!
#if defined(_MSC_VER) && defined(_DEBUG)
# define PYBIND11_BUILD_TYPE "_debug"
#ifdef PYBIND11_COMPILER_TYPE
// // To maintain backward compatibility (see PR #5439).
# define PYBIND11_COMPILER_TYPE_LEADING_UNDERSCORE ""
#else
# define PYBIND11_BUILD_TYPE ""
#endif
// Let's assume that different compilers are ABI-incompatible.
// A user can manually set this string if they know their
// compiler is compatible.
#ifndef PYBIND11_COMPILER_TYPE
# if defined(_MSC_VER)
# define PYBIND11_COMPILER_TYPE "_msvc"
# elif defined(__INTEL_COMPILER)
# define PYBIND11_COMPILER_TYPE "_icc"
# elif defined(__clang__)
# define PYBIND11_COMPILER_TYPE "_clang"
# elif defined(__PGI)
# define PYBIND11_COMPILER_TYPE "_pgi"
# elif defined(__MINGW32__)
# define PYBIND11_COMPILER_TYPE "_mingw"
# define PYBIND11_COMPILER_TYPE_LEADING_UNDERSCORE "_"
# if defined(__MINGW32__)
# define PYBIND11_COMPILER_TYPE "mingw"
# elif defined(__CYGWIN__)
# define PYBIND11_COMPILER_TYPE "_gcc_cygwin"
# elif defined(__GNUC__)
# define PYBIND11_COMPILER_TYPE "_gcc"
# define PYBIND11_COMPILER_TYPE "gcc_cygwin"
# elif defined(_MSC_VER)
# define PYBIND11_COMPILER_TYPE "msvc"
# elif defined(__clang__) || defined(__GNUC__)
# define PYBIND11_COMPILER_TYPE "system" // Assumed compatible with system compiler.
# else
# define PYBIND11_COMPILER_TYPE "_unknown"
# error "Unknown PYBIND11_COMPILER_TYPE: PLEASE REVISE THIS CODE."
# endif
#endif
// Also standard libs
// PR #5439 made this macro obsolete. However, there are many manipulations of this macro in the
// wild. Therefore, to maintain backward compatibility, it is kept around.
#ifndef PYBIND11_STDLIB
# if defined(_LIBCPP_VERSION)
# define PYBIND11_STDLIB "_libcpp"
# elif defined(__GLIBCXX__) || defined(__GLIBCPP__)
# define PYBIND11_STDLIB "_libstdcpp"
# else
# define PYBIND11_STDLIB ""
# endif
# define PYBIND11_STDLIB ""
#endif
#ifndef PYBIND11_BUILD_ABI
# if defined(__GXX_ABI_VERSION) // Linux/OSX.
# define PYBIND11_BUILD_ABI "_cxxabi" PYBIND11_PLATFORM_ABI_ID_TOSTRING(__GXX_ABI_VERSION)
# elif defined(_MSC_VER) // See PR #4953.
# if defined(_MSC_VER) // See PR #4953.
# if defined(_MT) && defined(_DLL) // Corresponding to CL command line options /MD or /MDd.
# if (_MSC_VER) / 100 == 19
# define PYBIND11_BUILD_ABI "_md_mscver19"
@ -72,17 +53,35 @@
# error "Unknown major version for MSC_VER: PLEASE REVISE THIS CODE."
# endif
# endif
# elif defined(__NVCOMPILER) // NVHPC (PGI-based).
# define PYBIND11_BUILD_ABI "" // TODO: What should be here, to prevent UB?
# elif defined(_LIBCPP_ABI_VERSION) // https://libcxx.llvm.org/DesignDocs/ABIVersioning.html
# define PYBIND11_BUILD_ABI \
"_libcpp_abi" PYBIND11_PLATFORM_ABI_ID_TOSTRING(_LIBCPP_ABI_VERSION)
# elif defined(_GLIBCXX_USE_CXX11_ABI) // See PR #5439.
# if defined(__NVCOMPILER)
// // Assume that NVHPC is in the 1xxx ABI family.
// // THIS ASSUMPTION IS NOT FUTURE PROOF but apparently the best we can do.
// // Please let us know if there is a way to validate the assumption here.
# elif !defined(__GXX_ABI_VERSION)
# error \
"Unknown platform or compiler (_GLIBCXX_USE_CXX11_ABI): PLEASE REVISE THIS CODE."
# endif
# if defined(__GXX_ABI_VERSION) && __GXX_ABI_VERSION < 1002 || __GXX_ABI_VERSION >= 2000
# error "Unknown platform or compiler (__GXX_ABI_VERSION): PLEASE REVISE THIS CODE."
# endif
# define PYBIND11_BUILD_ABI \
"_libstdcpp_gxx_abi_1xxx_use_cxx11_abi_" PYBIND11_PLATFORM_ABI_ID_TOSTRING( \
_GLIBCXX_USE_CXX11_ABI)
# else
# error "Unknown platform or compiler: PLEASE REVISE THIS CODE."
# endif
#endif
#ifndef PYBIND11_INTERNALS_KIND
# define PYBIND11_INTERNALS_KIND ""
// On MSVC, debug and release builds are not ABI-compatible!
#if defined(_MSC_VER) && defined(_DEBUG)
# define PYBIND11_BUILD_TYPE "_debug"
#else
# define PYBIND11_BUILD_TYPE ""
#endif
#define PYBIND11_PLATFORM_ABI_ID \
PYBIND11_INTERNALS_KIND PYBIND11_COMPILER_TYPE PYBIND11_STDLIB PYBIND11_BUILD_ABI \
PYBIND11_BUILD_TYPE
PYBIND11_COMPILER_TYPE PYBIND11_STDLIB PYBIND11_BUILD_ABI PYBIND11_BUILD_TYPE

View File

@ -46,7 +46,7 @@
# define PYBIND11_COMPILER_CLANG
# define PYBIND11_PRAGMA(...) _Pragma(#__VA_ARGS__)
# define PYBIND11_WARNING_PUSH PYBIND11_PRAGMA(clang diagnostic push)
# define PYBIND11_WARNING_POP PYBIND11_PRAGMA(clang diagnostic push)
# define PYBIND11_WARNING_POP PYBIND11_PRAGMA(clang diagnostic pop)
#elif defined(__GNUC__)
# define PYBIND11_COMPILER_GCC
# define PYBIND11_PRAGMA(...) _Pragma(#__VA_ARGS__)

View File

@ -298,11 +298,11 @@ struct type_info {
#define PYBIND11_INTERNALS_ID \
"__pybind11_internals_v" PYBIND11_TOSTRING(PYBIND11_INTERNALS_VERSION) \
PYBIND11_PLATFORM_ABI_ID "__"
PYBIND11_COMPILER_TYPE_LEADING_UNDERSCORE PYBIND11_PLATFORM_ABI_ID "__"
#define PYBIND11_MODULE_LOCAL_ID \
"__pybind11_module_local_v" PYBIND11_TOSTRING(PYBIND11_INTERNALS_VERSION) \
PYBIND11_PLATFORM_ABI_ID "__"
PYBIND11_COMPILER_TYPE_LEADING_UNDERSCORE PYBIND11_PLATFORM_ABI_ID "__"
/// Each module locally stores a pointer to the `internals` data. The data
/// itself is shared among modules with the same `PYBIND11_INTERNALS_ID`.

View File

@ -28,6 +28,12 @@
#include <utility>
#include <vector>
// See PR #5448. This warning suppression is needed for the PYBIND11_OVERRIDE macro family.
// NOTE that this is NOT embedded in a push/pop pair because that is very difficult to achieve.
#if defined(__clang_major__) && __clang_major__ < 14
PYBIND11_WARNING_DISABLE_CLANG("-Wgnu-zero-variadic-macro-arguments")
#endif
#if defined(__cpp_lib_launder) && !(defined(_MSC_VER) && (_MSC_VER < 1914))
# define PYBIND11_STD_LAUNDER std::launder
# define PYBIND11_HAS_STD_LAUNDER 1
@ -332,8 +338,8 @@ protected:
/* Generate a readable signature describing the function's arguments and return
value types */
static constexpr auto signature
= const_name("(") + cast_in::arg_names + const_name(") -> ") + cast_out::name;
static constexpr auto signature = const_name("(") + cast_in::arg_names
+ const_name(") -> ") + as_return_type<cast_out>::name;
PYBIND11_DESCR_CONSTEXPR auto types = decltype(signature)::types();
/* Register the function with Python from generic (non-templated) code */

View File

@ -107,6 +107,8 @@ public:
}
PYBIND11_TYPE_CASTER(T, const_name("os.PathLike"));
static constexpr auto arg_name = const_name("Union[os.PathLike, str, bytes]");
static constexpr auto return_name = const_name("Path");
};
#endif // PYBIND11_HAS_FILESYSTEM || defined(PYBIND11_HAS_EXPERIMENTAL_FILESYSTEM)

View File

@ -131,6 +131,13 @@ struct handle_type_name<typing::Tuple<Types...>> {
static constexpr auto name = const_name("tuple[")
+ ::pybind11::detail::concat(make_caster<Types>::name...)
+ const_name("]");
static constexpr auto arg_name
= const_name("tuple[")
+ ::pybind11::detail::concat(as_arg_type<make_caster<Types>>::name...) + const_name("]");
static constexpr auto return_name
= const_name("tuple[")
+ ::pybind11::detail::concat(as_return_type<make_caster<Types>>::name...)
+ const_name("]");
};
template <>
@ -144,48 +151,76 @@ struct handle_type_name<typing::Tuple<T, ellipsis>> {
// PEP 484 specifies this syntax for a variable-length tuple
static constexpr auto name
= const_name("tuple[") + make_caster<T>::name + const_name(", ...]");
static constexpr auto arg_name
= const_name("tuple[") + as_arg_type<make_caster<T>>::name + const_name(", ...]");
static constexpr auto return_name
= const_name("tuple[") + as_return_type<make_caster<T>>::name + const_name(", ...]");
};
template <typename K, typename V>
struct handle_type_name<typing::Dict<K, V>> {
static constexpr auto name = const_name("dict[") + make_caster<K>::name + const_name(", ")
+ make_caster<V>::name + const_name("]");
static constexpr auto arg_name = const_name("dict[") + as_arg_type<make_caster<K>>::name
+ const_name(", ") + as_arg_type<make_caster<V>>::name
+ const_name("]");
static constexpr auto return_name = const_name("dict[") + as_return_type<make_caster<K>>::name
+ const_name(", ") + as_return_type<make_caster<V>>::name
+ const_name("]");
};
template <typename T>
struct handle_type_name<typing::List<T>> {
static constexpr auto name = const_name("list[") + make_caster<T>::name + const_name("]");
static constexpr auto arg_name
= const_name("list[") + as_arg_type<make_caster<T>>::name + const_name("]");
static constexpr auto return_name
= const_name("list[") + as_return_type<make_caster<T>>::name + const_name("]");
};
template <typename T>
struct handle_type_name<typing::Set<T>> {
static constexpr auto name = const_name("set[") + make_caster<T>::name + const_name("]");
static constexpr auto arg_name
= const_name("set[") + as_arg_type<make_caster<T>>::name + const_name("]");
static constexpr auto return_name
= const_name("set[") + as_return_type<make_caster<T>>::name + const_name("]");
};
template <typename T>
struct handle_type_name<typing::Iterable<T>> {
static constexpr auto name = const_name("Iterable[") + make_caster<T>::name + const_name("]");
static constexpr auto arg_name
= const_name("Iterable[") + as_arg_type<make_caster<T>>::name + const_name("]");
static constexpr auto return_name
= const_name("Iterable[") + as_return_type<make_caster<T>>::name + const_name("]");
};
template <typename T>
struct handle_type_name<typing::Iterator<T>> {
static constexpr auto name = const_name("Iterator[") + make_caster<T>::name + const_name("]");
static constexpr auto arg_name
= const_name("Iterator[") + as_arg_type<make_caster<T>>::name + const_name("]");
static constexpr auto return_name
= const_name("Iterator[") + as_return_type<make_caster<T>>::name + const_name("]");
};
template <typename Return, typename... Args>
struct handle_type_name<typing::Callable<Return(Args...)>> {
using retval_type = conditional_t<std::is_same<Return, void>::value, void_type, Return>;
static constexpr auto name
= const_name("Callable[[") + ::pybind11::detail::concat(make_caster<Args>::name...)
+ const_name("], ") + make_caster<retval_type>::name + const_name("]");
= const_name("Callable[[")
+ ::pybind11::detail::concat(as_arg_type<make_caster<Args>>::name...) + const_name("], ")
+ as_return_type<make_caster<retval_type>>::name + const_name("]");
};
template <typename Return>
struct handle_type_name<typing::Callable<Return(ellipsis)>> {
// PEP 484 specifies this syntax for defining only return types of callables
using retval_type = conditional_t<std::is_same<Return, void>::value, void_type, Return>;
static constexpr auto name
= const_name("Callable[..., ") + make_caster<retval_type>::name + const_name("]");
static constexpr auto name = const_name("Callable[..., ")
+ as_return_type<make_caster<retval_type>>::name
+ const_name("]");
};
template <typename T>
@ -198,21 +233,37 @@ struct handle_type_name<typing::Union<Types...>> {
static constexpr auto name = const_name("Union[")
+ ::pybind11::detail::concat(make_caster<Types>::name...)
+ const_name("]");
static constexpr auto arg_name
= const_name("Union[")
+ ::pybind11::detail::concat(as_arg_type<make_caster<Types>>::name...) + const_name("]");
static constexpr auto return_name
= const_name("Union[")
+ ::pybind11::detail::concat(as_return_type<make_caster<Types>>::name...)
+ const_name("]");
};
template <typename T>
struct handle_type_name<typing::Optional<T>> {
static constexpr auto name = const_name("Optional[") + make_caster<T>::name + const_name("]");
static constexpr auto arg_name
= const_name("Optional[") + as_arg_type<make_caster<T>>::name + const_name("]");
static constexpr auto return_name
= const_name("Optional[") + as_return_type<make_caster<T>>::name + const_name("]");
};
// TypeGuard and TypeIs use as_return_type to use the return type if available, which is usually
// the narrower type.
template <typename T>
struct handle_type_name<typing::TypeGuard<T>> {
static constexpr auto name = const_name("TypeGuard[") + make_caster<T>::name + const_name("]");
static constexpr auto name
= const_name("TypeGuard[") + as_return_type<make_caster<T>>::name + const_name("]");
};
template <typename T>
struct handle_type_name<typing::TypeIs<T>> {
static constexpr auto name = const_name("TypeIs[") + make_caster<T>::name + const_name("]");
static constexpr auto name
= const_name("TypeIs[") + as_return_type<make_caster<T>>::name + const_name("]");
};
template <>

View File

@ -71,7 +71,6 @@ ignore = [
"PLR", # Design related pylint
"E501", # Line too long (Black is enough)
"PT011", # Too broad with raises in pytest
"PT004", # Fixture that doesn't return needs underscore (no, it is fine)
"SIM118", # iter(x) is not always the same as iter(x.keys())
]
unfixable = ["T20"]

View File

@ -139,6 +139,7 @@ set(PYBIND11_TEST_FILES
test_custom_type_casters
test_custom_type_setup
test_docstring_options
test_docs_advanced_cast_custom
test_eigen_matrix
test_eigen_tensor
test_enum

View File

@ -312,8 +312,16 @@ void print_created(T *inst, Values &&...values) {
}
template <class T, typename... Values>
void print_destroyed(T *inst, Values &&...values) { // Prints but doesn't store given values
/*
* On GraalPy, destructors can trigger anywhere and this can cause random
* failures in unrelated tests.
*/
#if !defined(GRAALVM_PYTHON)
print_constr_details(inst, "destroyed", values...);
track_destroyed(inst);
#else
py::detail::silence_unused_warnings(inst, values...);
#endif
}
template <class T, typename... Values>
void print_values(T *inst, Values &&...values) {

View File

@ -0,0 +1,70 @@
// #########################################################################
// PLEASE UPDATE docs/advanced/cast/custom.rst IF ANY CHANGES ARE MADE HERE.
// #########################################################################
#include "pybind11_tests.h"
namespace user_space {
struct Point2D {
double x;
double y;
};
Point2D negate(const Point2D &point) { return Point2D{-point.x, -point.y}; }
} // namespace user_space
namespace pybind11 {
namespace detail {
template <>
struct type_caster<user_space::Point2D> {
// This macro inserts a lot of boilerplate code and sets the default type hint to `tuple`
PYBIND11_TYPE_CASTER(user_space::Point2D, const_name("tuple"));
// `arg_name` and `return_name` may optionally be used to specify type hints separately for
// arguments and return values.
// The signature of our negate function would then look like:
// `negate(Sequence[float]) -> tuple[float, float]`
static constexpr auto arg_name = const_name("Sequence[float]");
static constexpr auto return_name = const_name("tuple[float, float]");
// C++ -> Python: convert `Point2D` to `tuple[float, float]`. The second and third arguments
// are used to indicate the return value policy and parent object (for
// return_value_policy::reference_internal) and are often ignored by custom casters.
// The return value should reflect the type hint specified by `return_name`.
static handle
cast(const user_space::Point2D &number, return_value_policy /*policy*/, handle /*parent*/) {
return py::make_tuple(number.x, number.y).release();
}
// Python -> C++: convert a `PyObject` into a `Point2D` and return false upon failure. The
// second argument indicates whether implicit conversions should be allowed.
// The accepted types should reflect the type hint specified by `arg_name`.
bool load(handle src, bool /*convert*/) {
// Check if handle is a Sequence
if (!py::isinstance<py::sequence>(src)) {
return false;
}
auto seq = py::reinterpret_borrow<py::sequence>(src);
// Check if exactly two values are in the Sequence
if (seq.size() != 2) {
return false;
}
// Check if each element is either a float or an int
for (auto item : seq) {
if (!py::isinstance<py::float_>(item) && !py::isinstance<py::int_>(item)) {
return false;
}
}
value.x = seq[0].cast<double>();
value.y = seq[1].cast<double>();
return true;
}
};
} // namespace detail
} // namespace pybind11
// Bind the negate function
TEST_SUBMODULE(docs_advanced_cast_custom, m) { m.def("negate", user_space::negate); }

View File

@ -0,0 +1,37 @@
from __future__ import annotations
from typing import TYPE_CHECKING, Sequence
if TYPE_CHECKING:
from conftest import SanitizedString
from pybind11_tests import docs_advanced_cast_custom as m
def assert_negate_function(
input_sequence: Sequence[float],
target: tuple[float, float],
) -> None:
output = m.negate(input_sequence)
assert isinstance(output, tuple)
assert len(output) == 2
assert isinstance(output[0], float)
assert isinstance(output[1], float)
assert output == target
def test_negate(doc: SanitizedString) -> None:
assert doc(m.negate) == "negate(arg0: Sequence[float]) -> tuple[float, float]"
assert_negate_function([1.0, -1.0], (-1.0, 1.0))
assert_negate_function((1.0, -1.0), (-1.0, 1.0))
assert_negate_function([1, -1], (-1.0, 1.0))
assert_negate_function((1, -1), (-1.0, 1.0))
def test_docs() -> None:
###########################################################################
# PLEASE UPDATE docs/advanced/cast/custom.rst IF ANY CHANGES ARE MADE HERE.
###########################################################################
point1 = [1.0, -1.0]
point2 = m.negate(point1)
assert point2 == (-1.0, 1.0)

View File

@ -6,14 +6,8 @@ from io import StringIO
import pytest
import env # noqa: F401
from pybind11_tests import iostream as m
pytestmark = pytest.mark.skipif(
"env.GRAALPY",
reason="Delayed prints from finalizers from other tests can end up in the output",
)
def test_captured(capsys):
msg = "I've been redirected to Python, I hope!"

View File

@ -7,6 +7,7 @@
BSD-style license that can be found in the LICENSE file.
*/
#include <pybind11/stl.h>
#include <pybind11/typing.h>
#include "pybind11_tests.h"
@ -137,6 +138,44 @@ typedef py::typing::TypeVar<"V"> TypeVarV;
} // namespace typevar
#endif
// Custom type for testing arg_name/return_name type hints
// RealNumber:
// * in arguments -> float | int
// * in return -> float
// * fallback -> complex
// The choice of types is not really useful, but just made different for testing purposes.
// According to `PEP 484 Type Hints` annotating with `float` also allows `int`,
// so using `float | int` could be replaced by just `float`.
struct RealNumber {
double value;
};
namespace pybind11 {
namespace detail {
template <>
struct type_caster<RealNumber> {
PYBIND11_TYPE_CASTER(RealNumber, const_name("complex"));
static constexpr auto arg_name = const_name("Union[float, int]");
static constexpr auto return_name = const_name("float");
static handle cast(const RealNumber &number, return_value_policy, handle) {
return py::float_(number.value).release();
}
bool load(handle src, bool) {
if (!py::isinstance<py::float_>(src) && !py::isinstance<py::int_>(src)) {
return false;
}
value.value = src.cast<double>();
return true;
}
};
} // namespace detail
} // namespace pybind11
TEST_SUBMODULE(pytypes, m) {
m.def("obj_class_name", [](py::handle obj) { return py::detail::obj_class_name(obj.ptr()); });
@ -998,4 +1037,94 @@ TEST_SUBMODULE(pytypes, m) {
#else
m.attr("defined_PYBIND11_TEST_PYTYPES_HAS_RANGES") = false;
#endif
m.def("half_of_number", [](const RealNumber &x) { return RealNumber{x.value / 2}; });
// std::vector<T>
m.def("half_of_number_vector", [](const std::vector<RealNumber> &x) {
std::vector<RealNumber> result;
result.reserve(x.size());
for (auto num : x) {
result.push_back(RealNumber{num.value / 2});
}
return result;
});
// Tuple<T, T>
m.def("half_of_number_tuple", [](const py::typing::Tuple<RealNumber, RealNumber> &x) {
py::typing::Tuple<RealNumber, RealNumber> result
= py::make_tuple(RealNumber{x[0].cast<RealNumber>().value / 2},
RealNumber{x[1].cast<RealNumber>().value / 2});
return result;
});
// Tuple<T, ...>
m.def("half_of_number_tuple_ellipsis",
[](const py::typing::Tuple<RealNumber, py::ellipsis> &x) {
py::typing::Tuple<RealNumber, py::ellipsis> result(x.size());
for (size_t i = 0; i < x.size(); ++i) {
result[i] = x[i].cast<RealNumber>().value / 2;
}
return result;
});
// Dict<K, V>
m.def("half_of_number_dict", [](const py::typing::Dict<std::string, RealNumber> &x) {
py::typing::Dict<std::string, RealNumber> result;
for (auto it : x) {
result[it.first] = RealNumber{it.second.cast<RealNumber>().value / 2};
}
return result;
});
// List<T>
m.def("half_of_number_list", [](const py::typing::List<RealNumber> &x) {
py::typing::List<RealNumber> result;
for (auto num : x) {
result.append(RealNumber{num.cast<RealNumber>().value / 2});
}
return result;
});
// List<List<T>>
m.def("half_of_number_nested_list",
[](const py::typing::List<py::typing::List<RealNumber>> &x) {
py::typing::List<py::typing::List<RealNumber>> result_lists;
for (auto nums : x) {
py::typing::List<RealNumber> result;
for (auto num : nums) {
result.append(RealNumber{num.cast<RealNumber>().value / 2});
}
result_lists.append(result);
}
return result_lists;
});
// Set<T>
m.def("identity_set", [](const py::typing::Set<RealNumber> &x) { return x; });
// Iterable<T>
m.def("identity_iterable", [](const py::typing::Iterable<RealNumber> &x) { return x; });
// Iterator<T>
m.def("identity_iterator", [](const py::typing::Iterator<RealNumber> &x) { return x; });
// Callable<R(A)>
m.def("apply_callable",
[](const RealNumber &x, const py::typing::Callable<RealNumber(const RealNumber &)> &f) {
return f(x).cast<RealNumber>();
});
// Callable<R(...)>
m.def("apply_callable_ellipsis",
[](const RealNumber &x, const py::typing::Callable<RealNumber(py::ellipsis)> &f) {
return f(x).cast<RealNumber>();
});
// Union<T1, T2>
m.def("identity_union", [](const py::typing::Union<RealNumber, std::string> &x) { return x; });
// Optional<T>
m.def("identity_optional", [](const py::typing::Optional<RealNumber> &x) { return x; });
// TypeGuard<T>
m.def("check_type_guard",
[](const py::typing::List<py::object> &x)
-> py::typing::TypeGuard<py::typing::List<RealNumber>> {
for (const auto &item : x) {
if (!py::isinstance<RealNumber>(item)) {
return false;
}
}
return true;
});
// TypeIs<T>
m.def("check_type_is", [](const py::object &x) -> py::typing::TypeIs<RealNumber> {
return py::isinstance<RealNumber>(x);
});
}

View File

@ -1101,3 +1101,84 @@ def test_list_ranges(tested_list, expected):
def test_dict_ranges(tested_dict, expected):
assert m.dict_iterator_default_initialization()
assert m.transform_dict_plus_one(tested_dict) == expected
def test_arg_return_type_hints(doc):
assert doc(m.half_of_number) == "half_of_number(arg0: Union[float, int]) -> float"
assert m.half_of_number(2.0) == 1.0
assert m.half_of_number(2) == 1.0
assert m.half_of_number(0) == 0
assert isinstance(m.half_of_number(0), float)
assert not isinstance(m.half_of_number(0), int)
# std::vector<T> should use fallback type (complex is not really useful but just used for testing)
assert (
doc(m.half_of_number_vector)
== "half_of_number_vector(arg0: list[complex]) -> list[complex]"
)
# Tuple<T, T>
assert (
doc(m.half_of_number_tuple)
== "half_of_number_tuple(arg0: tuple[Union[float, int], Union[float, int]]) -> tuple[float, float]"
)
# Tuple<T, ...>
assert (
doc(m.half_of_number_tuple_ellipsis)
== "half_of_number_tuple_ellipsis(arg0: tuple[Union[float, int], ...]) -> tuple[float, ...]"
)
# Dict<K, V>
assert (
doc(m.half_of_number_dict)
== "half_of_number_dict(arg0: dict[str, Union[float, int]]) -> dict[str, float]"
)
# List<T>
assert (
doc(m.half_of_number_list)
== "half_of_number_list(arg0: list[Union[float, int]]) -> list[float]"
)
# List<List<T>>
assert (
doc(m.half_of_number_nested_list)
== "half_of_number_nested_list(arg0: list[list[Union[float, int]]]) -> list[list[float]]"
)
# Set<T>
assert (
doc(m.identity_set)
== "identity_set(arg0: set[Union[float, int]]) -> set[float]"
)
# Iterable<T>
assert (
doc(m.identity_iterable)
== "identity_iterable(arg0: Iterable[Union[float, int]]) -> Iterable[float]"
)
# Iterator<T>
assert (
doc(m.identity_iterator)
== "identity_iterator(arg0: Iterator[Union[float, int]]) -> Iterator[float]"
)
# Callable<R(A)>
assert (
doc(m.apply_callable)
== "apply_callable(arg0: Union[float, int], arg1: Callable[[Union[float, int]], float]) -> float"
)
# Callable<R(...)>
assert (
doc(m.apply_callable_ellipsis)
== "apply_callable_ellipsis(arg0: Union[float, int], arg1: Callable[..., float]) -> float"
)
# Union<T1, T2>
assert (
doc(m.identity_union)
== "identity_union(arg0: Union[Union[float, int], str]) -> Union[float, str]"
)
# Optional<T>
assert (
doc(m.identity_optional)
== "identity_optional(arg0: Optional[Union[float, int]]) -> Optional[float]"
)
# TypeGuard<T>
assert (
doc(m.check_type_guard)
== "check_type_guard(arg0: list[object]) -> TypeGuard[list[float]]"
)
# TypeIs<T>
assert doc(m.check_type_is) == "check_type_is(arg0: object) -> TypeIs[float]"

View File

@ -16,6 +16,7 @@
# define PYBIND11_HAS_FILESYSTEM_IS_OPTIONAL
#endif
#include <pybind11/stl/filesystem.h>
#include <pybind11/typing.h>
#include <string>
#include <vector>
@ -453,7 +454,57 @@ TEST_SUBMODULE(stl, m) {
#ifdef PYBIND11_HAS_FILESYSTEM
// test_fs_path
m.attr("has_filesystem") = true;
m.def("parent_path", [](const std::filesystem::path &p) { return p.parent_path(); });
m.def("parent_path", [](const std::filesystem::path &path) { return path.parent_path(); });
m.def("parent_paths", [](const std::vector<std::filesystem::path> &paths) {
std::vector<std::filesystem::path> result;
result.reserve(paths.size());
for (const auto &path : paths) {
result.push_back(path.parent_path());
}
return result;
});
m.def("parent_paths_list", [](const py::typing::List<std::filesystem::path> &paths) {
py::typing::List<std::filesystem::path> result;
for (auto path : paths) {
result.append(path.cast<std::filesystem::path>().parent_path());
}
return result;
});
m.def("parent_paths_nested_list",
[](const py::typing::List<py::typing::List<std::filesystem::path>> &paths_lists) {
py::typing::List<py::typing::List<std::filesystem::path>> result_lists;
for (auto paths : paths_lists) {
py::typing::List<std::filesystem::path> result;
for (auto path : paths) {
result.append(path.cast<std::filesystem::path>().parent_path());
}
result_lists.append(result);
}
return result_lists;
});
m.def("parent_paths_tuple",
[](const py::typing::Tuple<std::filesystem::path, std::filesystem::path> &paths) {
py::typing::Tuple<std::filesystem::path, std::filesystem::path> result
= py::make_tuple(paths[0].cast<std::filesystem::path>().parent_path(),
paths[1].cast<std::filesystem::path>().parent_path());
return result;
});
m.def("parent_paths_tuple_ellipsis",
[](const py::typing::Tuple<std::filesystem::path, py::ellipsis> &paths) {
py::typing::Tuple<std::filesystem::path, py::ellipsis> result(paths.size());
for (size_t i = 0; i < paths.size(); ++i) {
result[i] = paths[i].cast<std::filesystem::path>().parent_path();
}
return result;
});
m.def("parent_paths_dict",
[](const py::typing::Dict<std::string, std::filesystem::path> &paths) {
py::typing::Dict<std::string, std::filesystem::path> result;
for (auto it : paths) {
result[it.first] = it.second.cast<std::filesystem::path>().parent_path();
}
return result;
});
#endif
#ifdef PYBIND11_TEST_VARIANT

View File

@ -246,7 +246,7 @@ def test_reference_sensitive_optional():
@pytest.mark.skipif(not hasattr(m, "has_filesystem"), reason="no <filesystem>")
def test_fs_path():
def test_fs_path(doc):
from pathlib import Path
class PseudoStrPath:
@ -257,11 +257,59 @@ def test_fs_path():
def __fspath__(self):
return b"foo/bar"
# Single argument
assert m.parent_path(Path("foo/bar")) == Path("foo")
assert m.parent_path("foo/bar") == Path("foo")
assert m.parent_path(b"foo/bar") == Path("foo")
assert m.parent_path(PseudoStrPath()) == Path("foo")
assert m.parent_path(PseudoBytesPath()) == Path("foo")
assert (
doc(m.parent_path)
== "parent_path(arg0: Union[os.PathLike, str, bytes]) -> Path"
)
# std::vector should use name (for arg_name/return_name typing classes must be used)
assert m.parent_paths(["foo/bar", "foo/baz"]) == [Path("foo"), Path("foo")]
assert (
doc(m.parent_paths)
== "parent_paths(arg0: list[os.PathLike]) -> list[os.PathLike]"
)
# py::typing::List
assert m.parent_paths_list(["foo/bar", "foo/baz"]) == [Path("foo"), Path("foo")]
assert (
doc(m.parent_paths_list)
== "parent_paths_list(arg0: list[Union[os.PathLike, str, bytes]]) -> list[Path]"
)
# Nested py::typing::List
assert m.parent_paths_nested_list([["foo/bar"], ["foo/baz", "foo/buzz"]]) == [
[Path("foo")],
[Path("foo"), Path("foo")],
]
assert (
doc(m.parent_paths_nested_list)
== "parent_paths_nested_list(arg0: list[list[Union[os.PathLike, str, bytes]]]) -> list[list[Path]]"
)
# py::typing::Tuple
assert m.parent_paths_tuple(("foo/bar", "foo/baz")) == (Path("foo"), Path("foo"))
assert (
doc(m.parent_paths_tuple)
== "parent_paths_tuple(arg0: tuple[Union[os.PathLike, str, bytes], Union[os.PathLike, str, bytes]]) -> tuple[Path, Path]"
)
# py::typing::Dict
assert m.parent_paths_dict(
{
"key1": Path("foo/bar"),
"key2": "foo/baz",
"key3": b"foo/buzz",
}
) == {
"key1": Path("foo"),
"key2": Path("foo"),
"key3": Path("foo"),
}
assert (
doc(m.parent_paths_dict)
== "parent_paths_dict(arg0: dict[str, Union[os.PathLike, str, bytes]]) -> dict[str, Path]"
)
@pytest.mark.skipif(not hasattr(m, "load_variant"), reason="no <variant>")

View File

@ -59,9 +59,9 @@ for issue in issues:
msg += "."
msg += f"\n `#{issue.number} <{issue.html_url}>`_"
for cat in cats:
for cat, cat_list in cats.items():
if issue.title.lower().startswith(f"{cat}:"):
cats[cat].append(msg)
cat_list.append(msg)
break
else:
cats["unknown"].append(msg)