Option for arg/return type hints and correct typing for std::filesystem::path (#5450)

* Added arg/return type handling.

* Added support for nested arg/return type in py::typing::List

* Added support for arg/return type in stl/filesystem

* Added tests for arg/return type in stl/filesystem and py::typing::List

* Added arg/return name to more py::typing classes

* Added arg/return type to Callable[...]

* Added tests for typing container classes (also nested)

* Changed typing classes to avoid using C++14 auto return type deduction.

* Fixed clang-tidy errors.

* Changed Enable to SFINAE

* Added test for Tuple[T, ...]

* Added RealNumber with custom caster for testing typing classes.

* Added tests for Set, Iterable, Iterator, Union, and Optional

* Added tests for Callable

* Fixed Callable with ellipsis test

* Changed TypeGuard/TypeIs to use return type (being the narrower type) + Tests

* Added test for use of fallback type name with stl vector

* Updated documentation.

* Fixed unnecessary constructor call in test.

* Fixed reference counting in example type caster.

* Fixed clang-tidy issues.

* Fix for clang-tidy

* Updated cast method to use pybind11 API rather than Python C API in custom caster example

* Updated load to use pybind11 API rather than Python C API in custom caster example

* Changed test of arg/return name to use pybind11 API instead of Python C API

* Updated code in adcanced/cast example and improved documentation text

* Fixed references in custom type caster docs

* Fixed wrong logical and operator in test

* Fixed wrong logical operator in doc example

* Added comment to test about `float` vs `float | int`

* Updated std::filesystem::path docs in cast/overview section

* Remove one stray dot.

---------

Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>

docs/advanced/cast/custom.rst

@@ -1,35 +1,53 @@
 Custom type casters
 ===================
 
-In very rare cases, applications may require custom type casters that cannot be
+Some applications may prefer custom type casters that convert between existing
-expressed using the abstractions provided by pybind11, thus requiring raw
+Python types and C++ types, similar to the ``list`` ↔ ``std::vector``
-Python C API calls. This is fairly advanced usage and should only be pursued by
+and ``dict`` ↔ ``std::map`` conversions which are built into pybind11.
-experts who are familiar with the intricacies of Python reference counting.
+Implementing custom type casters is fairly advanced usage.
+While it is recommended to use the pybind11 API as much as possible, more complex examples may
+require familiarity with the intricacies of the Python C API.
+You can refer to the `Python/C API Reference Manual <https://docs.python.org/3/c-api/index.html>`_
+for more information.
 
-The following snippets demonstrate how this works for a very simple ``inty``
+The following snippets demonstrate how this works for a very simple ``Point2D`` type.
-type that that should be convertible from Python types that provide a
+We want this type to be convertible to C++ from Python types implementing the
-``__int__(self)`` method.
+``Sequence`` protocol and having two elements of type ``float``.
+When returned from C++ to Python, it should be converted to a Python ``tuple[float, float]``.
+For this type we could provide Python bindings for different arithmetic functions implemented
+in C++ (here demonstrated by a simple ``negate`` function).
+
+..
+    PLEASE KEEP THE CODE BLOCKS IN SYNC WITH
+        tests/test_docs_advanced_cast_custom.cpp
+        tests/test_docs_advanced_cast_custom.py
+    Ideally, change the test, run pre-commit (incl. clang-format),
+    then copy the changed code back here.
+    Also use TEST_SUBMODULE in tests, but PYBIND11_MODULE in docs.
 
 .. code-block:: cpp
 
-    struct inty { long long_value; };
+    namespace user_space {
 
-    void print(inty s) {
+    struct Point2D {
-        std::cout << s.long_value << std::endl;
+        double x;
-    }
+        double y;
+    };
 
-The following Python snippet demonstrates the intended usage from the Python side:
+    Point2D negate(const Point2D &point) { return Point2D{-point.x, -point.y}; }
+
+    } // namespace user_space
+
+
+The following Python snippet demonstrates the intended usage of ``negate`` from the Python side:
 
 .. code-block:: python
 
-    class A:
+    from my_math_module import docs_advanced_cast_custom as m
-        def __int__(self):
-            return 123
 
-
+    point1 = [1.0, -1.0]
-    from example import print
+    point2 = m.negate(point1)
-
+    assert point2 == (-1.0, 1.0)
-    print(A())
 
 To register the necessary conversion routines, it is necessary to add an
 instantiation of the ``pybind11::detail::type_caster<T>`` template.
@@ -38,47 +56,59 @@ type is explicitly allowed.
 
 .. code-block:: cpp
 
-    namespace PYBIND11_NAMESPACE { namespace detail {
+    namespace pybind11 {
-        template <> struct type_caster<inty> {
+    namespace detail {
-        public:
-            /**
-             * This macro establishes the name 'inty' in
-             * function signatures and declares a local variable
-             * 'value' of type inty
-             */
-            PYBIND11_TYPE_CASTER(inty, const_name("inty"));
 
-            /**
+    template <>
-             * Conversion part 1 (Python->C++): convert a PyObject into a inty
+    struct type_caster<user_space::Point2D> {
-             * instance or return false upon failure. The second argument
+        // This macro inserts a lot of boilerplate code and sets the default type hint to `tuple`
-             * indicates whether implicit conversions should be applied.
+        PYBIND11_TYPE_CASTER(user_space::Point2D, const_name("tuple"));
-             */
+        // `arg_name` and `return_name` may optionally be used to specify type hints separately for
-            bool load(handle src, bool) {
+        // arguments and return values.
-                /* Extract PyObject from handle */
+        // The signature of our negate function would then look like:
-                PyObject *source = src.ptr();
+        // `negate(Sequence[float]) -> tuple[float, float]`
-                /* Try converting into a Python integer value */
+        static constexpr auto arg_name = const_name("Sequence[float]");
-                PyObject *tmp = PyNumber_Long(source);
+        static constexpr auto return_name = const_name("tuple[float, float]");
-                if (!tmp)
+
+        // C++ -> Python: convert `Point2D` to `tuple[float, float]`. The second and third arguments
+        // are used to indicate the return value policy and parent object (for
+        // return_value_policy::reference_internal) and are often ignored by custom casters.
+        // The return value should reflect the type hint specified by `return_name`.
+        static handle
+        cast(const user_space::Point2D &number, return_value_policy /*policy*/, handle /*parent*/) {
+            return py::make_tuple(number.x, number.y).release();
+        }
+
+        // Python -> C++: convert a `PyObject` into a `Point2D` and return false upon failure. The
+        // second argument indicates whether implicit conversions should be allowed.
+        // The accepted types should reflect the type hint specified by `arg_name`.
+        bool load(handle src, bool /*convert*/) {
+            // Check if handle is a Sequence
+            if (!py::isinstance<py::sequence>(src)) {
+                return false;
+            }
+            auto seq = py::reinterpret_borrow<py::sequence>(src);
+            // Check if exactly two values are in the Sequence
+            if (seq.size() != 2) {
+                return false;
+            }
+            // Check if each element is either a float or an int
+            for (auto item : seq) {
+                if (!py::isinstance<py::float_>(item) && !py::isinstance<py::int_>(item)) {
                     return false;
-                /* Now try to convert into a C++ int */
+                }
-                value.long_value = PyLong_AsLong(tmp);
-                Py_DECREF(tmp);
-                /* Ensure return code was OK (to avoid out-of-range errors etc) */
-                return !(value.long_value == -1 && !PyErr_Occurred());
             }
+            value.x = seq[0].cast<double>();
+            value.y = seq[1].cast<double>();
+            return true;
+        }
+    };
 
-            /**
+    } // namespace detail
-             * Conversion part 2 (C++ -> Python): convert an inty instance into
+    } // namespace pybind11
-             * a Python object. The second and third arguments are used to
+
-             * indicate the return value policy and parent object (for
+    // Bind the negate function
-             * ``return_value_policy::reference_internal``) and are generally
+    PYBIND11_MODULE(docs_advanced_cast_custom, m) { m.def("negate", user_space::negate); }
-             * ignored by implicit casters.
-             */
-            static handle cast(inty src, return_value_policy /* policy */, handle /* parent */) {
-                return PyLong_FromLong(src.long_value);
-            }
-        };
-    }} // namespace PYBIND11_NAMESPACE::detail
 
 .. note::
 
@@ -86,8 +116,22 @@ type is explicitly allowed.
     that ``T`` is default-constructible (``value`` is first default constructed
     and then ``load()`` assigns to it).
 
+.. note::
+    For further information on the ``return_value_policy`` argument of ``cast`` refer to :ref:`return_value_policies`.
+    To learn about the ``convert`` argument of ``load`` see :ref:`nonconverting_arguments`.
+
 .. warning::
 
     When using custom type casters, it's important to declare them consistently
-    in every compilation unit of the Python extension module. Otherwise,
+    in every compilation unit of the Python extension module to satisfy the C++ One Definition Rule
+    (`ODR <https://en.cppreference.com/w/cpp/language/definition>`_). Otherwise,
     undefined behavior can ensue.
+
+.. note::
+
+    Using the type hint ``Sequence[float]`` signals to static type checkers, that not only tuples may be
+    passed, but any type implementing the Sequence protocol, e.g., ``list[float]``.
+    Unfortunately, that loses the length information ``tuple[float, float]`` provides.
+    One way of still providing some length information in type hints is using ``typing.Annotated``, e.g.,
+    ``Annotated[Sequence[float], 2]``, or further add libraries like
+    `annotated-types <https://github.com/annotated-types/annotated-types>`_.

docs/advanced/cast/overview.rst

@@ -151,7 +151,7 @@ as arguments and return values, refer to the section on binding :ref:`classes`.
 +------------------------------------+---------------------------+-----------------------------------+
 | ``std::variant<...>``              | Type-safe union (C++17)   | :file:`pybind11/stl.h`            |
 +------------------------------------+---------------------------+-----------------------------------+
-| ``std::filesystem::path<T>``       | STL path (C++17) [#]_     | :file:`pybind11/stl/filesystem.h` |
+| ``std::filesystem::path``          | STL path (C++17) [#]_     | :file:`pybind11/stl/filesystem.h` |
 +------------------------------------+---------------------------+-----------------------------------+
 | ``std::function<...>``             | STL polymorphic function  | :file:`pybind11/functional.h`     |
 +------------------------------------+---------------------------+-----------------------------------+
@@ -167,4 +167,4 @@ as arguments and return values, refer to the section on binding :ref:`classes`.
 +------------------------------------+---------------------------+-----------------------------------+
 
 .. [#] ``std::filesystem::path`` is converted to ``pathlib.Path`` and
-   ``os.PathLike`` is converted to ``std::filesystem::path``.
+   can be loaded from ``os.PathLike``, ``str``, and ``bytes``.

include/pybind11/cast.h

@@ -34,6 +34,39 @@ PYBIND11_WARNING_DISABLE_MSVC(4127)
 
 PYBIND11_NAMESPACE_BEGIN(detail)
 
+// Type trait checker for `descr`
+template <typename>
+struct is_descr : std::false_type {};
+
+template <size_t N, typename... Ts>
+struct is_descr<descr<N, Ts...>> : std::true_type {};
+
+template <size_t N, typename... Ts>
+struct is_descr<const descr<N, Ts...>> : std::true_type {};
+
+// Use arg_name instead of name when available
+template <typename T, typename SFINAE = void>
+struct as_arg_type {
+    static constexpr auto name = T::name;
+};
+
+template <typename T>
+struct as_arg_type<T, typename std::enable_if<is_descr<decltype(T::arg_name)>::value>::type> {
+    static constexpr auto name = T::arg_name;
+};
+
+// Use return_name instead of name when available
+template <typename T, typename SFINAE = void>
+struct as_return_type {
+    static constexpr auto name = T::name;
+};
+
+template <typename T>
+struct as_return_type<T,
+                      typename std::enable_if<is_descr<decltype(T::return_name)>::value>::type> {
+    static constexpr auto name = T::return_name;
+};
+
 template <typename type, typename SFINAE = void>
 class type_caster : public type_caster_base<type> {};
 template <typename type>
@@ -1080,6 +1113,8 @@ struct pyobject_caster {
         return src.inc_ref();
     }
     PYBIND11_TYPE_CASTER(type, handle_type_name<type>::name);
+    static constexpr auto arg_name = as_arg_type<handle_type_name<type>>::name;
+    static constexpr auto return_name = as_return_type<handle_type_name<type>>::name;
 };
 
 template <typename T>
@@ -1608,7 +1643,7 @@ public:
                   "py::args cannot be specified more than once");
 
     static constexpr auto arg_names
-        = ::pybind11::detail::concat(type_descr(make_caster<Args>::name)...);
+        = ::pybind11::detail::concat(type_descr(as_arg_type<make_caster<Args>>::name)...);
 
     bool load_args(function_call &call) { return load_impl_sequence(call, indices{}); }
 

include/pybind11/pybind11.h

@@ -336,8 +336,8 @@ protected:
 
         /* Generate a readable signature describing the function's arguments and return
            value types */
-        static constexpr auto signature
+        static constexpr auto signature = const_name("(") + cast_in::arg_names
-            = const_name("(") + cast_in::arg_names + const_name(") -> ") + cast_out::name;
+                                          + const_name(") -> ") + as_return_type<cast_out>::name;
         PYBIND11_DESCR_CONSTEXPR auto types = decltype(signature)::types();
 
         /* Register the function with Python from generic (non-templated) code */

include/pybind11/stl/filesystem.h

@@ -107,6 +107,8 @@ public:
     }
 
     PYBIND11_TYPE_CASTER(T, const_name("os.PathLike"));
+    static constexpr auto arg_name = const_name("Union[os.PathLike, str, bytes]");
+    static constexpr auto return_name = const_name("Path");
 };
 
 #endif // PYBIND11_HAS_FILESYSTEM || defined(PYBIND11_HAS_EXPERIMENTAL_FILESYSTEM)

include/pybind11/typing.h

@@ -131,6 +131,13 @@ struct handle_type_name<typing::Tuple<Types...>> {
     static constexpr auto name = const_name("tuple[")
                                  + ::pybind11::detail::concat(make_caster<Types>::name...)
                                  + const_name("]");
+    static constexpr auto arg_name
+        = const_name("tuple[")
+          + ::pybind11::detail::concat(as_arg_type<make_caster<Types>>::name...) + const_name("]");
+    static constexpr auto return_name
+        = const_name("tuple[")
+          + ::pybind11::detail::concat(as_return_type<make_caster<Types>>::name...)
+          + const_name("]");
 };
 
 template <>
@@ -144,48 +151,76 @@ struct handle_type_name<typing::Tuple<T, ellipsis>> {
     // PEP 484 specifies this syntax for a variable-length tuple
     static constexpr auto name
         = const_name("tuple[") + make_caster<T>::name + const_name(", ...]");
+    static constexpr auto arg_name
+        = const_name("tuple[") + as_arg_type<make_caster<T>>::name + const_name(", ...]");
+    static constexpr auto return_name
+        = const_name("tuple[") + as_return_type<make_caster<T>>::name + const_name(", ...]");
 };
 
 template <typename K, typename V>
 struct handle_type_name<typing::Dict<K, V>> {
     static constexpr auto name = const_name("dict[") + make_caster<K>::name + const_name(", ")
                                  + make_caster<V>::name + const_name("]");
+    static constexpr auto arg_name = const_name("dict[") + as_arg_type<make_caster<K>>::name
+                                     + const_name(", ") + as_arg_type<make_caster<V>>::name
+                                     + const_name("]");
+    static constexpr auto return_name = const_name("dict[") + as_return_type<make_caster<K>>::name
+                                        + const_name(", ") + as_return_type<make_caster<V>>::name
+                                        + const_name("]");
 };
 
 template <typename T>
 struct handle_type_name<typing::List<T>> {
     static constexpr auto name = const_name("list[") + make_caster<T>::name + const_name("]");
+    static constexpr auto arg_name
+        = const_name("list[") + as_arg_type<make_caster<T>>::name + const_name("]");
+    static constexpr auto return_name
+        = const_name("list[") + as_return_type<make_caster<T>>::name + const_name("]");
 };
 
 template <typename T>
 struct handle_type_name<typing::Set<T>> {
     static constexpr auto name = const_name("set[") + make_caster<T>::name + const_name("]");
+    static constexpr auto arg_name
+        = const_name("set[") + as_arg_type<make_caster<T>>::name + const_name("]");
+    static constexpr auto return_name
+        = const_name("set[") + as_return_type<make_caster<T>>::name + const_name("]");
 };
 
 template <typename T>
 struct handle_type_name<typing::Iterable<T>> {
     static constexpr auto name = const_name("Iterable[") + make_caster<T>::name + const_name("]");
+    static constexpr auto arg_name
+        = const_name("Iterable[") + as_arg_type<make_caster<T>>::name + const_name("]");
+    static constexpr auto return_name
+        = const_name("Iterable[") + as_return_type<make_caster<T>>::name + const_name("]");
 };
 
 template <typename T>
 struct handle_type_name<typing::Iterator<T>> {
     static constexpr auto name = const_name("Iterator[") + make_caster<T>::name + const_name("]");
+    static constexpr auto arg_name
+        = const_name("Iterator[") + as_arg_type<make_caster<T>>::name + const_name("]");
+    static constexpr auto return_name
+        = const_name("Iterator[") + as_return_type<make_caster<T>>::name + const_name("]");
 };
 
 template <typename Return, typename... Args>
 struct handle_type_name<typing::Callable<Return(Args...)>> {
     using retval_type = conditional_t<std::is_same<Return, void>::value, void_type, Return>;
     static constexpr auto name
-        = const_name("Callable[[") + ::pybind11::detail::concat(make_caster<Args>::name...)
+        = const_name("Callable[[")
-          + const_name("], ") + make_caster<retval_type>::name + const_name("]");
+          + ::pybind11::detail::concat(as_arg_type<make_caster<Args>>::name...) + const_name("], ")
+          + as_return_type<make_caster<retval_type>>::name + const_name("]");
 };
 
 template <typename Return>
 struct handle_type_name<typing::Callable<Return(ellipsis)>> {
     // PEP 484 specifies this syntax for defining only return types of callables
     using retval_type = conditional_t<std::is_same<Return, void>::value, void_type, Return>;
-    static constexpr auto name
+    static constexpr auto name = const_name("Callable[..., ")
-        = const_name("Callable[..., ") + make_caster<retval_type>::name + const_name("]");
+                                 + as_return_type<make_caster<retval_type>>::name
+                                 + const_name("]");
 };
 
 template <typename T>
@@ -198,21 +233,37 @@ struct handle_type_name<typing::Union<Types...>> {
     static constexpr auto name = const_name("Union[")
                                  + ::pybind11::detail::concat(make_caster<Types>::name...)
                                  + const_name("]");
+    static constexpr auto arg_name
+        = const_name("Union[")
+          + ::pybind11::detail::concat(as_arg_type<make_caster<Types>>::name...) + const_name("]");
+    static constexpr auto return_name
+        = const_name("Union[")
+          + ::pybind11::detail::concat(as_return_type<make_caster<Types>>::name...)
+          + const_name("]");
 };
 
 template <typename T>
 struct handle_type_name<typing::Optional<T>> {
     static constexpr auto name = const_name("Optional[") + make_caster<T>::name + const_name("]");
+    static constexpr auto arg_name
+        = const_name("Optional[") + as_arg_type<make_caster<T>>::name + const_name("]");
+    static constexpr auto return_name
+        = const_name("Optional[") + as_return_type<make_caster<T>>::name + const_name("]");
 };
 
+// TypeGuard and TypeIs use as_return_type to use the return type if available, which is usually
+// the narrower type.
+
 template <typename T>
 struct handle_type_name<typing::TypeGuard<T>> {
-    static constexpr auto name = const_name("TypeGuard[") + make_caster<T>::name + const_name("]");
+    static constexpr auto name
+        = const_name("TypeGuard[") + as_return_type<make_caster<T>>::name + const_name("]");
 };
 
 template <typename T>
 struct handle_type_name<typing::TypeIs<T>> {
-    static constexpr auto name = const_name("TypeIs[") + make_caster<T>::name + const_name("]");
+    static constexpr auto name
+        = const_name("TypeIs[") + as_return_type<make_caster<T>>::name + const_name("]");
 };
 
 template <>

tests/CMakeLists.txt

@@ -122,6 +122,7 @@ set(PYBIND11_TEST_FILES
     test_custom_type_casters
     test_custom_type_setup
     test_docstring_options
+    test_docs_advanced_cast_custom
     test_eigen_matrix
     test_eigen_tensor
     test_enum

tests/test_docs_advanced_cast_custom.cpp

@@ -0,0 +1,70 @@
+// #########################################################################
+// PLEASE UPDATE docs/advanced/cast/custom.rst IF ANY CHANGES ARE MADE HERE.
+// #########################################################################
+
+#include "pybind11_tests.h"
+
+namespace user_space {
+
+struct Point2D {
+    double x;
+    double y;
+};
+
+Point2D negate(const Point2D &point) { return Point2D{-point.x, -point.y}; }
+
+} // namespace user_space
+
+namespace pybind11 {
+namespace detail {
+
+template <>
+struct type_caster<user_space::Point2D> {
+    // This macro inserts a lot of boilerplate code and sets the default type hint to `tuple`
+    PYBIND11_TYPE_CASTER(user_space::Point2D, const_name("tuple"));
+    // `arg_name` and `return_name` may optionally be used to specify type hints separately for
+    // arguments and return values.
+    // The signature of our negate function would then look like:
+    // `negate(Sequence[float]) -> tuple[float, float]`
+    static constexpr auto arg_name = const_name("Sequence[float]");
+    static constexpr auto return_name = const_name("tuple[float, float]");
+
+    // C++ -> Python: convert `Point2D` to `tuple[float, float]`. The second and third arguments
+    // are used to indicate the return value policy and parent object (for
+    // return_value_policy::reference_internal) and are often ignored by custom casters.
+    // The return value should reflect the type hint specified by `return_name`.
+    static handle
+    cast(const user_space::Point2D &number, return_value_policy /*policy*/, handle /*parent*/) {
+        return py::make_tuple(number.x, number.y).release();
+    }
+
+    // Python -> C++: convert a `PyObject` into a `Point2D` and return false upon failure. The
+    // second argument indicates whether implicit conversions should be allowed.
+    // The accepted types should reflect the type hint specified by `arg_name`.
+    bool load(handle src, bool /*convert*/) {
+        // Check if handle is a Sequence
+        if (!py::isinstance<py::sequence>(src)) {
+            return false;
+        }
+        auto seq = py::reinterpret_borrow<py::sequence>(src);
+        // Check if exactly two values are in the Sequence
+        if (seq.size() != 2) {
+            return false;
+        }
+        // Check if each element is either a float or an int
+        for (auto item : seq) {
+            if (!py::isinstance<py::float_>(item) && !py::isinstance<py::int_>(item)) {
+                return false;
+            }
+        }
+        value.x = seq[0].cast<double>();
+        value.y = seq[1].cast<double>();
+        return true;
+    }
+};
+
+} // namespace detail
+} // namespace pybind11
+
+// Bind the negate function
+TEST_SUBMODULE(docs_advanced_cast_custom, m) { m.def("negate", user_space::negate); }

tests/test_docs_advanced_cast_custom.py

@@ -0,0 +1,37 @@
+from __future__ import annotations
+
+from typing import TYPE_CHECKING, Sequence
+
+if TYPE_CHECKING:
+    from conftest import SanitizedString
+
+from pybind11_tests import docs_advanced_cast_custom as m
+
+
+def assert_negate_function(
+    input_sequence: Sequence[float],
+    target: tuple[float, float],
+) -> None:
+    output = m.negate(input_sequence)
+    assert isinstance(output, tuple)
+    assert len(output) == 2
+    assert isinstance(output[0], float)
+    assert isinstance(output[1], float)
+    assert output == target
+
+
+def test_negate(doc: SanitizedString) -> None:
+    assert doc(m.negate) == "negate(arg0: Sequence[float]) -> tuple[float, float]"
+    assert_negate_function([1.0, -1.0], (-1.0, 1.0))
+    assert_negate_function((1.0, -1.0), (-1.0, 1.0))
+    assert_negate_function([1, -1], (-1.0, 1.0))
+    assert_negate_function((1, -1), (-1.0, 1.0))
+
+
+def test_docs() -> None:
+    ###########################################################################
+    # PLEASE UPDATE docs/advanced/cast/custom.rst IF ANY CHANGES ARE MADE HERE.
+    ###########################################################################
+    point1 = [1.0, -1.0]
+    point2 = m.negate(point1)
+    assert point2 == (-1.0, 1.0)

tests/test_pytypes.cpp

@@ -7,6 +7,7 @@
     BSD-style license that can be found in the LICENSE file.
 */
 
+#include <pybind11/stl.h>
 #include <pybind11/typing.h>
 
 #include "pybind11_tests.h"
@@ -137,6 +138,44 @@ typedef py::typing::TypeVar<"V"> TypeVarV;
 } // namespace typevar
 #endif
 
+// Custom type for testing arg_name/return_name type hints
+// RealNumber:
+// * in arguments -> float | int
+// * in return -> float
+// * fallback -> complex
+// The choice of types is not really useful, but just made different for testing purposes.
+// According to `PEP 484 – Type Hints` annotating with `float` also allows `int`,
+// so using `float | int` could be replaced by just `float`.
+
+struct RealNumber {
+    double value;
+};
+
+namespace pybind11 {
+namespace detail {
+
+template <>
+struct type_caster<RealNumber> {
+    PYBIND11_TYPE_CASTER(RealNumber, const_name("complex"));
+    static constexpr auto arg_name = const_name("Union[float, int]");
+    static constexpr auto return_name = const_name("float");
+
+    static handle cast(const RealNumber &number, return_value_policy, handle) {
+        return py::float_(number.value).release();
+    }
+
+    bool load(handle src, bool) {
+        if (!py::isinstance<py::float_>(src) && !py::isinstance<py::int_>(src)) {
+            return false;
+        }
+        value.value = src.cast<double>();
+        return true;
+    }
+};
+
+} // namespace detail
+} // namespace pybind11
+
 TEST_SUBMODULE(pytypes, m) {
     m.def("obj_class_name", [](py::handle obj) { return py::detail::obj_class_name(obj.ptr()); });
 
@@ -998,4 +1037,94 @@ TEST_SUBMODULE(pytypes, m) {
 #else
     m.attr("defined_PYBIND11_TEST_PYTYPES_HAS_RANGES") = false;
 #endif
+    m.def("half_of_number", [](const RealNumber &x) { return RealNumber{x.value / 2}; });
+    // std::vector<T>
+    m.def("half_of_number_vector", [](const std::vector<RealNumber> &x) {
+        std::vector<RealNumber> result;
+        result.reserve(x.size());
+        for (auto num : x) {
+            result.push_back(RealNumber{num.value / 2});
+        }
+        return result;
+    });
+    // Tuple<T, T>
+    m.def("half_of_number_tuple", [](const py::typing::Tuple<RealNumber, RealNumber> &x) {
+        py::typing::Tuple<RealNumber, RealNumber> result
+            = py::make_tuple(RealNumber{x[0].cast<RealNumber>().value / 2},
+                             RealNumber{x[1].cast<RealNumber>().value / 2});
+        return result;
+    });
+    // Tuple<T, ...>
+    m.def("half_of_number_tuple_ellipsis",
+          [](const py::typing::Tuple<RealNumber, py::ellipsis> &x) {
+              py::typing::Tuple<RealNumber, py::ellipsis> result(x.size());
+              for (size_t i = 0; i < x.size(); ++i) {
+                  result[i] = x[i].cast<RealNumber>().value / 2;
+              }
+              return result;
+          });
+    // Dict<K, V>
+    m.def("half_of_number_dict", [](const py::typing::Dict<std::string, RealNumber> &x) {
+        py::typing::Dict<std::string, RealNumber> result;
+        for (auto it : x) {
+            result[it.first] = RealNumber{it.second.cast<RealNumber>().value / 2};
+        }
+        return result;
+    });
+    // List<T>
+    m.def("half_of_number_list", [](const py::typing::List<RealNumber> &x) {
+        py::typing::List<RealNumber> result;
+        for (auto num : x) {
+            result.append(RealNumber{num.cast<RealNumber>().value / 2});
+        }
+        return result;
+    });
+    // List<List<T>>
+    m.def("half_of_number_nested_list",
+          [](const py::typing::List<py::typing::List<RealNumber>> &x) {
+              py::typing::List<py::typing::List<RealNumber>> result_lists;
+              for (auto nums : x) {
+                  py::typing::List<RealNumber> result;
+                  for (auto num : nums) {
+                      result.append(RealNumber{num.cast<RealNumber>().value / 2});
+                  }
+                  result_lists.append(result);
+              }
+              return result_lists;
+          });
+    // Set<T>
+    m.def("identity_set", [](const py::typing::Set<RealNumber> &x) { return x; });
+    // Iterable<T>
+    m.def("identity_iterable", [](const py::typing::Iterable<RealNumber> &x) { return x; });
+    // Iterator<T>
+    m.def("identity_iterator", [](const py::typing::Iterator<RealNumber> &x) { return x; });
+    // Callable<R(A)>
+    m.def("apply_callable",
+          [](const RealNumber &x, const py::typing::Callable<RealNumber(const RealNumber &)> &f) {
+              return f(x).cast<RealNumber>();
+          });
+    // Callable<R(...)>
+    m.def("apply_callable_ellipsis",
+          [](const RealNumber &x, const py::typing::Callable<RealNumber(py::ellipsis)> &f) {
+              return f(x).cast<RealNumber>();
+          });
+    // Union<T1, T2>
+    m.def("identity_union", [](const py::typing::Union<RealNumber, std::string> &x) { return x; });
+    // Optional<T>
+    m.def("identity_optional", [](const py::typing::Optional<RealNumber> &x) { return x; });
+    // TypeGuard<T>
+    m.def("check_type_guard",
+          [](const py::typing::List<py::object> &x)
+              -> py::typing::TypeGuard<py::typing::List<RealNumber>> {
+              for (const auto &item : x) {
+                  if (!py::isinstance<RealNumber>(item)) {
+                      return false;
+                  }
+              }
+              return true;
+          });
+    // TypeIs<T>
+    m.def("check_type_is", [](const py::object &x) -> py::typing::TypeIs<RealNumber> {
+        return py::isinstance<RealNumber>(x);
+    });
 }

tests/test_pytypes.py

@@ -1101,3 +1101,84 @@ def test_list_ranges(tested_list, expected):
 def test_dict_ranges(tested_dict, expected):
     assert m.dict_iterator_default_initialization()
     assert m.transform_dict_plus_one(tested_dict) == expected
+
+
+def test_arg_return_type_hints(doc):
+    assert doc(m.half_of_number) == "half_of_number(arg0: Union[float, int]) -> float"
+    assert m.half_of_number(2.0) == 1.0
+    assert m.half_of_number(2) == 1.0
+    assert m.half_of_number(0) == 0
+    assert isinstance(m.half_of_number(0), float)
+    assert not isinstance(m.half_of_number(0), int)
+    # std::vector<T> should use fallback type (complex is not really useful but just used for testing)
+    assert (
+        doc(m.half_of_number_vector)
+        == "half_of_number_vector(arg0: list[complex]) -> list[complex]"
+    )
+    # Tuple<T, T>
+    assert (
+        doc(m.half_of_number_tuple)
+        == "half_of_number_tuple(arg0: tuple[Union[float, int], Union[float, int]]) -> tuple[float, float]"
+    )
+    # Tuple<T, ...>
+    assert (
+        doc(m.half_of_number_tuple_ellipsis)
+        == "half_of_number_tuple_ellipsis(arg0: tuple[Union[float, int], ...]) -> tuple[float, ...]"
+    )
+    # Dict<K, V>
+    assert (
+        doc(m.half_of_number_dict)
+        == "half_of_number_dict(arg0: dict[str, Union[float, int]]) -> dict[str, float]"
+    )
+    # List<T>
+    assert (
+        doc(m.half_of_number_list)
+        == "half_of_number_list(arg0: list[Union[float, int]]) -> list[float]"
+    )
+    # List<List<T>>
+    assert (
+        doc(m.half_of_number_nested_list)
+        == "half_of_number_nested_list(arg0: list[list[Union[float, int]]]) -> list[list[float]]"
+    )
+    # Set<T>
+    assert (
+        doc(m.identity_set)
+        == "identity_set(arg0: set[Union[float, int]]) -> set[float]"
+    )
+    # Iterable<T>
+    assert (
+        doc(m.identity_iterable)
+        == "identity_iterable(arg0: Iterable[Union[float, int]]) -> Iterable[float]"
+    )
+    # Iterator<T>
+    assert (
+        doc(m.identity_iterator)
+        == "identity_iterator(arg0: Iterator[Union[float, int]]) -> Iterator[float]"
+    )
+    # Callable<R(A)>
+    assert (
+        doc(m.apply_callable)
+        == "apply_callable(arg0: Union[float, int], arg1: Callable[[Union[float, int]], float]) -> float"
+    )
+    # Callable<R(...)>
+    assert (
+        doc(m.apply_callable_ellipsis)
+        == "apply_callable_ellipsis(arg0: Union[float, int], arg1: Callable[..., float]) -> float"
+    )
+    # Union<T1, T2>
+    assert (
+        doc(m.identity_union)
+        == "identity_union(arg0: Union[Union[float, int], str]) -> Union[float, str]"
+    )
+    # Optional<T>
+    assert (
+        doc(m.identity_optional)
+        == "identity_optional(arg0: Optional[Union[float, int]]) -> Optional[float]"
+    )
+    # TypeGuard<T>
+    assert (
+        doc(m.check_type_guard)
+        == "check_type_guard(arg0: list[object]) -> TypeGuard[list[float]]"
+    )
+    # TypeIs<T>
+    assert doc(m.check_type_is) == "check_type_is(arg0: object) -> TypeIs[float]"

tests/test_stl.cpp

@@ -16,6 +16,7 @@
 #    define PYBIND11_HAS_FILESYSTEM_IS_OPTIONAL
 #endif
 #include <pybind11/stl/filesystem.h>
+#include <pybind11/typing.h>
 
 #include <string>
 #include <vector>
@@ -453,7 +454,57 @@ TEST_SUBMODULE(stl, m) {
 #ifdef PYBIND11_HAS_FILESYSTEM
     // test_fs_path
     m.attr("has_filesystem") = true;
-    m.def("parent_path", [](const std::filesystem::path &p) { return p.parent_path(); });
+    m.def("parent_path", [](const std::filesystem::path &path) { return path.parent_path(); });
+    m.def("parent_paths", [](const std::vector<std::filesystem::path> &paths) {
+        std::vector<std::filesystem::path> result;
+        result.reserve(paths.size());
+        for (const auto &path : paths) {
+            result.push_back(path.parent_path());
+        }
+        return result;
+    });
+    m.def("parent_paths_list", [](const py::typing::List<std::filesystem::path> &paths) {
+        py::typing::List<std::filesystem::path> result;
+        for (auto path : paths) {
+            result.append(path.cast<std::filesystem::path>().parent_path());
+        }
+        return result;
+    });
+    m.def("parent_paths_nested_list",
+          [](const py::typing::List<py::typing::List<std::filesystem::path>> &paths_lists) {
+              py::typing::List<py::typing::List<std::filesystem::path>> result_lists;
+              for (auto paths : paths_lists) {
+                  py::typing::List<std::filesystem::path> result;
+                  for (auto path : paths) {
+                      result.append(path.cast<std::filesystem::path>().parent_path());
+                  }
+                  result_lists.append(result);
+              }
+              return result_lists;
+          });
+    m.def("parent_paths_tuple",
+          [](const py::typing::Tuple<std::filesystem::path, std::filesystem::path> &paths) {
+              py::typing::Tuple<std::filesystem::path, std::filesystem::path> result
+                  = py::make_tuple(paths[0].cast<std::filesystem::path>().parent_path(),
+                                   paths[1].cast<std::filesystem::path>().parent_path());
+              return result;
+          });
+    m.def("parent_paths_tuple_ellipsis",
+          [](const py::typing::Tuple<std::filesystem::path, py::ellipsis> &paths) {
+              py::typing::Tuple<std::filesystem::path, py::ellipsis> result(paths.size());
+              for (size_t i = 0; i < paths.size(); ++i) {
+                  result[i] = paths[i].cast<std::filesystem::path>().parent_path();
+              }
+              return result;
+          });
+    m.def("parent_paths_dict",
+          [](const py::typing::Dict<std::string, std::filesystem::path> &paths) {
+              py::typing::Dict<std::string, std::filesystem::path> result;
+              for (auto it : paths) {
+                  result[it.first] = it.second.cast<std::filesystem::path>().parent_path();
+              }
+              return result;
+          });
 #endif
 
 #ifdef PYBIND11_TEST_VARIANT

tests/test_stl.py

@@ -246,7 +246,7 @@ def test_reference_sensitive_optional():
 
 
 @pytest.mark.skipif(not hasattr(m, "has_filesystem"), reason="no <filesystem>")
-def test_fs_path():
+def test_fs_path(doc):
     from pathlib import Path
 
     class PseudoStrPath:
@@ -257,11 +257,59 @@ def test_fs_path():
         def __fspath__(self):
             return b"foo/bar"
 
+    # Single argument
     assert m.parent_path(Path("foo/bar")) == Path("foo")
     assert m.parent_path("foo/bar") == Path("foo")
     assert m.parent_path(b"foo/bar") == Path("foo")
     assert m.parent_path(PseudoStrPath()) == Path("foo")
     assert m.parent_path(PseudoBytesPath()) == Path("foo")
+    assert (
+        doc(m.parent_path)
+        == "parent_path(arg0: Union[os.PathLike, str, bytes]) -> Path"
+    )
+    # std::vector should use name (for arg_name/return_name typing classes must be used)
+    assert m.parent_paths(["foo/bar", "foo/baz"]) == [Path("foo"), Path("foo")]
+    assert (
+        doc(m.parent_paths)
+        == "parent_paths(arg0: list[os.PathLike]) -> list[os.PathLike]"
+    )
+    # py::typing::List
+    assert m.parent_paths_list(["foo/bar", "foo/baz"]) == [Path("foo"), Path("foo")]
+    assert (
+        doc(m.parent_paths_list)
+        == "parent_paths_list(arg0: list[Union[os.PathLike, str, bytes]]) -> list[Path]"
+    )
+    # Nested py::typing::List
+    assert m.parent_paths_nested_list([["foo/bar"], ["foo/baz", "foo/buzz"]]) == [
+        [Path("foo")],
+        [Path("foo"), Path("foo")],
+    ]
+    assert (
+        doc(m.parent_paths_nested_list)
+        == "parent_paths_nested_list(arg0: list[list[Union[os.PathLike, str, bytes]]]) -> list[list[Path]]"
+    )
+    # py::typing::Tuple
+    assert m.parent_paths_tuple(("foo/bar", "foo/baz")) == (Path("foo"), Path("foo"))
+    assert (
+        doc(m.parent_paths_tuple)
+        == "parent_paths_tuple(arg0: tuple[Union[os.PathLike, str, bytes], Union[os.PathLike, str, bytes]]) -> tuple[Path, Path]"
+    )
+    # py::typing::Dict
+    assert m.parent_paths_dict(
+        {
+            "key1": Path("foo/bar"),
+            "key2": "foo/baz",
+            "key3": b"foo/buzz",
+        }
+    ) == {
+        "key1": Path("foo"),
+        "key2": Path("foo"),
+        "key3": Path("foo"),
+    }
+    assert (
+        doc(m.parent_paths_dict)
+        == "parent_paths_dict(arg0: dict[str, Union[os.PathLike, str, bytes]]) -> dict[str, Path]"
+    )
 
 
 @pytest.mark.skipif(not hasattr(m, "load_variant"), reason="no <variant>")