pybind11/docs/advanced/exceptions.rst

423 lines
18 KiB
ReStructuredText
Raw Normal View History

2016-10-16 17:12:43 +00:00
Exceptions
##########
Built-in C++ to Python exception translation
============================================
When Python calls C++ code through pybind11, pybind11 provides a C++ exception handler
that will trap C++ exceptions, translate them to the corresponding Python exception,
and raise them so that Python code can handle them.
2016-10-16 17:12:43 +00:00
pybind11 defines translations for ``std::exception`` and its standard
subclasses, and several special exception classes that translate to specific
Python exceptions. Note that these are not actually Python exceptions, so they
cannot be examined using the Python C API. Instead, they are pure C++ objects
that pybind11 will translate the corresponding Python exception when they arrive
at its exception handler.
2016-10-16 17:12:43 +00:00
.. tabularcolumns:: |p{0.5\textwidth}|p{0.45\textwidth}|
+--------------------------------------+--------------------------------------+
| Exception thrown by C++ | Translated to Python exception type |
+======================================+======================================+
| :class:`std::exception` | ``RuntimeError`` |
+--------------------------------------+--------------------------------------+
| :class:`std::bad_alloc` | ``MemoryError`` |
+--------------------------------------+--------------------------------------+
| :class:`std::domain_error` | ``ValueError`` |
+--------------------------------------+--------------------------------------+
| :class:`std::invalid_argument` | ``ValueError`` |
+--------------------------------------+--------------------------------------+
| :class:`std::length_error` | ``ValueError`` |
+--------------------------------------+--------------------------------------+
| :class:`std::out_of_range` | ``IndexError`` |
+--------------------------------------+--------------------------------------+
| :class:`std::range_error` | ``ValueError`` |
+--------------------------------------+--------------------------------------+
| :class:`std::overflow_error` | ``OverflowError`` |
+--------------------------------------+--------------------------------------+
| :class:`pybind11::stop_iteration` | ``StopIteration`` (used to implement |
| | custom iterators) |
+--------------------------------------+--------------------------------------+
| :class:`pybind11::index_error` | ``IndexError`` (used to indicate out |
| | of bounds access in ``__getitem__``, |
| | ``__setitem__``, etc.) |
+--------------------------------------+--------------------------------------+
| :class:`pybind11::key_error` | ``KeyError`` (used to indicate out |
| | of bounds access in ``__getitem__``, |
| | ``__setitem__`` in dict-like |
| | objects, etc.) |
+--------------------------------------+--------------------------------------+
| :class:`pybind11::value_error` | ``ValueError`` (used to indicate |
| | wrong value passed in |
| | ``container.remove(...)``) |
+--------------------------------------+--------------------------------------+
| :class:`pybind11::type_error` | ``TypeError`` |
+--------------------------------------+--------------------------------------+
| :class:`pybind11::buffer_error` | ``BufferError`` |
+--------------------------------------+--------------------------------------+
| :class:`pybind11::import_error` | ``ImportError`` |
+--------------------------------------+--------------------------------------+
| :class:`pybind11::attribute_error` | ``AttributeError`` |
+--------------------------------------+--------------------------------------+
| Any other exception | ``RuntimeError`` |
+--------------------------------------+--------------------------------------+
2016-10-16 17:12:43 +00:00
Exception translation is not bidirectional. That is, *catching* the C++
exceptions defined above will not trap exceptions that originate from
Python. For that, catch :class:`pybind11::error_already_set`. See :ref:`below
<handling_python_exceptions_cpp>` for further details.
2016-10-16 17:12:43 +00:00
There is also a special exception :class:`cast_error` that is thrown by
:func:`handle::call` when the input arguments cannot be converted to Python
objects.
Registering custom translators
==============================
If the default exception conversion policy described above is insufficient,
pybind11 also provides support for registering custom exception translators.
Feature/local exception translator (#2650) * Create a module_internals struct Since we now have two things that are going to be module local, it felt correct to add a struct to manage them. * Add local exception translators These are added via the register_local_exception_translator function and are then applied before the global translators * Add unit tests to show the local exception translator works * Fix a bug in the unit test with the string value of KeyError * Fix a formatting issue * Rename registered_local_types_cpp() Rename it to get_registered_local_types_cpp() to disambiguate from the new member of module_internals * Add additional comments to new local exception code path * Add a register_local_exception function * Add additional unit tests for register_local_exception * Use get_local_internals like get_internals * Update documentation for new local exception feature * Add back a missing space * Clean-up some issues in the docs * Remove the code duplication when translating exceptions Separated out the exception processing into a standalone function in the details namespace. Clean-up some comments as per PR notes as well * Remove the code duplication in register_exception * Cleanup some formatting things caught by clang-format * Remove the templates from exception translators But I added a using declaration to alias the type. * Remove the extra local from local_internals variable names * Add an extra explanatory comment to local_internals * Fix a typo in the code
2021-07-21 12:22:18 +00:00
Similar to pybind11 classes, exception translators can be local to the module
they are defined in or global to the entire python session. To register a simple
exception conversion that translates a C++ exception into a new Python exception
using the C++ exception's ``what()`` method, a helper function is available:
2016-10-16 17:12:43 +00:00
.. code-block:: cpp
py::register_exception<CppExp>(module, "PyExp");
This call creates a Python exception class with the name ``PyExp`` in the given
module and automatically converts any encountered exceptions of type ``CppExp``
into Python exceptions of type ``PyExp``.
Feature/local exception translator (#2650) * Create a module_internals struct Since we now have two things that are going to be module local, it felt correct to add a struct to manage them. * Add local exception translators These are added via the register_local_exception_translator function and are then applied before the global translators * Add unit tests to show the local exception translator works * Fix a bug in the unit test with the string value of KeyError * Fix a formatting issue * Rename registered_local_types_cpp() Rename it to get_registered_local_types_cpp() to disambiguate from the new member of module_internals * Add additional comments to new local exception code path * Add a register_local_exception function * Add additional unit tests for register_local_exception * Use get_local_internals like get_internals * Update documentation for new local exception feature * Add back a missing space * Clean-up some issues in the docs * Remove the code duplication when translating exceptions Separated out the exception processing into a standalone function in the details namespace. Clean-up some comments as per PR notes as well * Remove the code duplication in register_exception * Cleanup some formatting things caught by clang-format * Remove the templates from exception translators But I added a using declaration to alias the type. * Remove the extra local from local_internals variable names * Add an extra explanatory comment to local_internals * Fix a typo in the code
2021-07-21 12:22:18 +00:00
A matching function is available for registering a local exception translator:
.. code-block:: cpp
py::register_local_exception<CppExp>(module, "PyExp");
It is possible to specify base class for the exception using the third
parameter, a ``handle``:
.. code-block:: cpp
py::register_exception<CppExp>(module, "PyExp", PyExc_RuntimeError);
Feature/local exception translator (#2650) * Create a module_internals struct Since we now have two things that are going to be module local, it felt correct to add a struct to manage them. * Add local exception translators These are added via the register_local_exception_translator function and are then applied before the global translators * Add unit tests to show the local exception translator works * Fix a bug in the unit test with the string value of KeyError * Fix a formatting issue * Rename registered_local_types_cpp() Rename it to get_registered_local_types_cpp() to disambiguate from the new member of module_internals * Add additional comments to new local exception code path * Add a register_local_exception function * Add additional unit tests for register_local_exception * Use get_local_internals like get_internals * Update documentation for new local exception feature * Add back a missing space * Clean-up some issues in the docs * Remove the code duplication when translating exceptions Separated out the exception processing into a standalone function in the details namespace. Clean-up some comments as per PR notes as well * Remove the code duplication in register_exception * Cleanup some formatting things caught by clang-format * Remove the templates from exception translators But I added a using declaration to alias the type. * Remove the extra local from local_internals variable names * Add an extra explanatory comment to local_internals * Fix a typo in the code
2021-07-21 12:22:18 +00:00
py::register_local_exception<CppExp>(module, "PyExp", PyExc_RuntimeError);
Then ``PyExp`` can be caught both as ``PyExp`` and ``RuntimeError``.
The class objects of the built-in Python exceptions are listed in the Python
documentation on `Standard Exceptions <https://docs.python.org/3/c-api/exceptions.html#standard-exceptions>`_.
The default base class is ``PyExc_Exception``.
Feature/local exception translator (#2650) * Create a module_internals struct Since we now have two things that are going to be module local, it felt correct to add a struct to manage them. * Add local exception translators These are added via the register_local_exception_translator function and are then applied before the global translators * Add unit tests to show the local exception translator works * Fix a bug in the unit test with the string value of KeyError * Fix a formatting issue * Rename registered_local_types_cpp() Rename it to get_registered_local_types_cpp() to disambiguate from the new member of module_internals * Add additional comments to new local exception code path * Add a register_local_exception function * Add additional unit tests for register_local_exception * Use get_local_internals like get_internals * Update documentation for new local exception feature * Add back a missing space * Clean-up some issues in the docs * Remove the code duplication when translating exceptions Separated out the exception processing into a standalone function in the details namespace. Clean-up some comments as per PR notes as well * Remove the code duplication in register_exception * Cleanup some formatting things caught by clang-format * Remove the templates from exception translators But I added a using declaration to alias the type. * Remove the extra local from local_internals variable names * Add an extra explanatory comment to local_internals * Fix a typo in the code
2021-07-21 12:22:18 +00:00
When more advanced exception translation is needed, the functions
``py::register_exception_translator(translator)`` and
``py::register_local_exception_translator(translator)`` can be used to register
2016-10-16 17:12:43 +00:00
functions that can translate arbitrary exception types (and which may include
Feature/local exception translator (#2650) * Create a module_internals struct Since we now have two things that are going to be module local, it felt correct to add a struct to manage them. * Add local exception translators These are added via the register_local_exception_translator function and are then applied before the global translators * Add unit tests to show the local exception translator works * Fix a bug in the unit test with the string value of KeyError * Fix a formatting issue * Rename registered_local_types_cpp() Rename it to get_registered_local_types_cpp() to disambiguate from the new member of module_internals * Add additional comments to new local exception code path * Add a register_local_exception function * Add additional unit tests for register_local_exception * Use get_local_internals like get_internals * Update documentation for new local exception feature * Add back a missing space * Clean-up some issues in the docs * Remove the code duplication when translating exceptions Separated out the exception processing into a standalone function in the details namespace. Clean-up some comments as per PR notes as well * Remove the code duplication in register_exception * Cleanup some formatting things caught by clang-format * Remove the templates from exception translators But I added a using declaration to alias the type. * Remove the extra local from local_internals variable names * Add an extra explanatory comment to local_internals * Fix a typo in the code
2021-07-21 12:22:18 +00:00
additional logic to do so). The functions takes a stateless callable (e.g. a
2016-10-16 17:12:43 +00:00
function pointer or a lambda function without captured variables) with the call
signature ``void(std::exception_ptr)``.
When a C++ exception is thrown, the registered exception translators are tried
in reverse order of registration (i.e. the last registered translator gets the
Feature/local exception translator (#2650) * Create a module_internals struct Since we now have two things that are going to be module local, it felt correct to add a struct to manage them. * Add local exception translators These are added via the register_local_exception_translator function and are then applied before the global translators * Add unit tests to show the local exception translator works * Fix a bug in the unit test with the string value of KeyError * Fix a formatting issue * Rename registered_local_types_cpp() Rename it to get_registered_local_types_cpp() to disambiguate from the new member of module_internals * Add additional comments to new local exception code path * Add a register_local_exception function * Add additional unit tests for register_local_exception * Use get_local_internals like get_internals * Update documentation for new local exception feature * Add back a missing space * Clean-up some issues in the docs * Remove the code duplication when translating exceptions Separated out the exception processing into a standalone function in the details namespace. Clean-up some comments as per PR notes as well * Remove the code duplication in register_exception * Cleanup some formatting things caught by clang-format * Remove the templates from exception translators But I added a using declaration to alias the type. * Remove the extra local from local_internals variable names * Add an extra explanatory comment to local_internals * Fix a typo in the code
2021-07-21 12:22:18 +00:00
first shot at handling the exception). All local translators will be tried
before a global translator is tried.
2016-10-16 17:12:43 +00:00
Inside the translator, ``std::rethrow_exception`` should be used within
a try block to re-throw the exception. One or more catch clauses to catch
the appropriate exceptions should then be used with each clause using
Add `py::set_error()`, use in updated `py::exception<>` documentation (#4772) * Copy clang 17 compatibility fixes from PR #4762 to a separate PR. * static py::exception<> -> static py::handle * Add `py::set_error()` but also try the suggestion of @malfet (https://github.com/pytorch/pytorch/pull/106401#pullrequestreview-1559961407). * clang 17 compatibility fixes (#4767) * Copy clang 17 compatibility fixes from PR #4762 to a separate PR. * Add gcc:13 C++20 * Add silkeh/clang:16-bullseye C++20 * chore(deps): update pre-commit hooks (#4770) updates: - [github.com/psf/black: 23.3.0 → 23.7.0](https://github.com/psf/black/compare/23.3.0...23.7.0) - [github.com/astral-sh/ruff-pre-commit: v0.0.276 → v0.0.281](https://github.com/astral-sh/ruff-pre-commit/compare/v0.0.276...v0.0.281) - [github.com/asottile/blacken-docs: 1.14.0 → 1.15.0](https://github.com/asottile/blacken-docs/compare/1.14.0...1.15.0) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * docs: Remove upper bound on pybind11 in example pyproject.toml for setuptools (#4774) * docs: Remove upper bound on pybind11 in example pyproject.toml for setuptools * Update docs/compiling.rst --------- Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com> * Provide better type hints for a variety of generic types (#4259) * Provide better type hints for a variety of generic types * Makes better documentation * tuple, dict, list, set, function * Move to py::typing * style: pre-commit fixes * Update copyright line with correct year and actual author. The author information was copy-pasted from the git log output. --------- Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Use `py::set_error()` everywhere possible (only one special case, in common.h). Overload `py::set_error(py::handle, py::handle)`. Change back to `static py::handle exc = ... .release();` Deprecate `py::exception<>::operator()` * Add `PYBIND11_WARNING_DISABLE` for INTEL and MSVC (and sort alphabetically). * `PYBIND11_WARNING_DISABLE_INTEL(10441)` does not work. For ICC only, falling back to the recommended `py::set_error()` to keep the testing simple. It is troublesome to add `--diag-disable=10441` specifically for test_exceptions.cpp, even that is non-ideal because it covers the entire file, not just the one line we need it for, and the value of exercising the trivial deprecated `operator()` on this one extra platform is practically zero. * Fix silly oversight. * NVHPC 23.5.0 generates deprecation warnings. They are currently not treated as errors, but falling back to using `py::set_error()` to not have to deal with that distraction. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Keto D. Zhang <keto.zhang@gmail.com> Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com> Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com>
2023-08-08 03:48:20 +00:00
``py::set_error()`` (see below).
2016-10-16 17:12:43 +00:00
To declare a custom Python exception type, declare a ``py::exception`` variable
and use this in the associated exception translator (note: it is often useful
to make this a static declaration when using it inside a lambda expression
without requiring capturing).
The following example demonstrates this for a hypothetical exception classes
``MyCustomException`` and ``OtherException``: the first is translated to a
custom python exception ``MyCustomError``, while the second is translated to a
standard python RuntimeError:
.. code-block:: cpp
Add pybind11/gil_safe_call_once.h (to fix deadlocks in pybind11/numpy.h) (#4877) * LazyInitializeAtLeastOnceDestroyNever v1 * Go back to using `union` as originally suggested by jbms@. The trick (also suggested by jbms@) is to add empty ctor + dtor. * Revert "Go back to using `union` as originally suggested by jbms@. The trick (also suggested by jbms@) is to add empty ctor + dtor." This reverts commit e7b8c4f0fcd72191e88d1c17abf5da08fe3a9c6f. * Remove `#include <stdalign.h>` * `include\pybind11/numpy.h(24,10): fatal error C1083: Cannot open include file: 'stdalign.h': No such file or directory` * @tkoeppe wrote: this is a C interop header (and we're not writing C) * Suppress gcc 4.8.5 (CentOS 7) warning. ``` include/pybind11/eigen/../numpy.h:63:53: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing] return *reinterpret_cast<T *>(value_storage_); ^ ``` * Replace comments: Document PRECONDITION. Adopt comment suggested by @tkoeppe: https://github.com/pybind/pybind11/pull/4877#discussion_r1350356093 * Adopt suggestion by @tkoeppe: * https://github.com/pybind/pybind11/pull/4877#issuecomment-1752969127 * https://godbolt.org/z/Wa79nKz6e * Add `PYBIND11_CONSTINIT`, but it does not work for the current use cases: ``` g++ -o pybind11/tests/test_numpy_array.os -c -std=c++20 -fPIC -fvisibility=hidden -O0 -g -Wall -Wextra -Wconversion -Wcast-qual -Wdeprecated -Wundef -Wnon-virtual-dtor -Wunused-result -Werror -isystem /usr/include/python3.11 -isystem /usr/include/eigen3 -DPYBIND11_STRICT_ASSERTS_CLASS_HOLDER_VS_TYPE_CASTER_MIX -DPYBIND11_ENABLE_TYPE_CASTER_ODR_GUARD_IF_AVAILABLE -DPYBIND11_TEST_BOOST -Ipybind11/include -I/usr/local/google/home/rwgk/forked/pybind11/include -I/usr/local/google/home/rwgk/clone/pybind11/include /usr/local/google/home/rwgk/forked/pybind11/tests/test_numpy_array.cpp ``` ``` In file included from /usr/local/google/home/rwgk/forked/pybind11/tests/test_numpy_array.cpp:10: /usr/local/google/home/rwgk/forked/pybind11/include/pybind11/numpy.h: In static member function ‘static pybind11::detail::npy_api& pybind11::detail::npy_api::get()’: /usr/local/google/home/rwgk/forked/pybind11/include/pybind11/numpy.h:258:82: error: ‘constinit’ variable ‘api_init’ does not have a constant initializer 258 | PYBIND11_CONSTINIT static LazyInitializeAtLeastOnceDestroyNever<npy_api> api_init; | ^~~~~~~~ ``` ``` In file included from /usr/local/google/home/rwgk/forked/pybind11/tests/test_numpy_array.cpp:10: /usr/local/google/home/rwgk/forked/pybind11/include/pybind11/numpy.h: In static member function ‘static pybind11::object& pybind11::dtype::_dtype_from_pep3118()’: /usr/local/google/home/rwgk/forked/pybind11/include/pybind11/numpy.h:697:13: error: ‘constinit’ variable ‘imported_obj’ does not have a constant initializer 697 | imported_obj; | ^~~~~~~~~~~~ ``` * Revert "Add `PYBIND11_CONSTINIT`, but it does not work for the current use cases:" This reverts commit f07b28bda9f91fb723aa898a21c81b6dd6857072. * Reapply "Add `PYBIND11_CONSTINIT`, but it does not work for the current use cases:" This reverts commit 36be645758aa82b576d24003808386bec6e55bf9. * Add Default Member Initializer on `value_storage_` as suggested by @tkoeppe: https://github.com/pybind/pybind11/pull/4877#issuecomment-1753201342 This fixes the errors reported under commit f07b28bda9f91fb723aa898a21c81b6dd6857072. * Fix copy-paste-missed-a-change mishap in commit 88cec1152ab5576db19bab95c484672f06f5989a. * Semi-paranoid placement new (based on https://github.com/pybind/pybind11/pull/4877#discussion_r1350573114). * Move PYBIND11_CONSTINIT to detail/common.h * Move code to the right places, rename new class and some variables. * Fix oversight: update tests/extra_python_package/test_files.py * Get the name right first. * Use `std::call_once`, `std::atomic`, following a pattern developed by @tkoeppe * Make the API more self-documenting (and possibly more easily reusable). * google-clang-tidy IWYU fixes * Rewrite comment as suggested by @tkoeppe * Update test_exceptions.cpp and exceptions.rst * Fix oversight in previous commit: add `PYBIND11_CONSTINIT` * Make `get_stored()` non-const for simplicity. As suggested by @tkoeppe: not seeing any reasonable use in which `get_stored` has to be const. * Add comment regarding `KeyboardInterrupt` behavior, based heavily on information provided by @jbms. * Add `assert(PyGILState_Check())` in `gil_scoped_release` ctor (simple & non-simple implementation) as suggested by @EthanSteinberg. * Fix oversight in previous commit (missing include cassert). * Remove use of std::atomic, leaving comments with rationale, why it is not needed. * Rewrite comment re `std:optional` based on deeper reflection (aka 2nd thoughts). * Additional comment with the conclusion of a discussion under PR #4877. * https://github.com/pybind/pybind11/pull/4877#issuecomment-1757363179 * Small comment changes suggested by @tkoeppe.
2023-10-12 04:05:31 +00:00
PYBIND11_CONSTINIT static py::gil_safe_call_once_and_store<py::object> exc_storage;
exc_storage.call_once_and_store_result(
[&]() { return py::exception<MyCustomException>(m, "MyCustomError"); });
2016-10-16 17:12:43 +00:00
py::register_exception_translator([](std::exception_ptr p) {
try {
if (p) std::rethrow_exception(p);
} catch (const MyCustomException &e) {
Add pybind11/gil_safe_call_once.h (to fix deadlocks in pybind11/numpy.h) (#4877) * LazyInitializeAtLeastOnceDestroyNever v1 * Go back to using `union` as originally suggested by jbms@. The trick (also suggested by jbms@) is to add empty ctor + dtor. * Revert "Go back to using `union` as originally suggested by jbms@. The trick (also suggested by jbms@) is to add empty ctor + dtor." This reverts commit e7b8c4f0fcd72191e88d1c17abf5da08fe3a9c6f. * Remove `#include <stdalign.h>` * `include\pybind11/numpy.h(24,10): fatal error C1083: Cannot open include file: 'stdalign.h': No such file or directory` * @tkoeppe wrote: this is a C interop header (and we're not writing C) * Suppress gcc 4.8.5 (CentOS 7) warning. ``` include/pybind11/eigen/../numpy.h:63:53: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing] return *reinterpret_cast<T *>(value_storage_); ^ ``` * Replace comments: Document PRECONDITION. Adopt comment suggested by @tkoeppe: https://github.com/pybind/pybind11/pull/4877#discussion_r1350356093 * Adopt suggestion by @tkoeppe: * https://github.com/pybind/pybind11/pull/4877#issuecomment-1752969127 * https://godbolt.org/z/Wa79nKz6e * Add `PYBIND11_CONSTINIT`, but it does not work for the current use cases: ``` g++ -o pybind11/tests/test_numpy_array.os -c -std=c++20 -fPIC -fvisibility=hidden -O0 -g -Wall -Wextra -Wconversion -Wcast-qual -Wdeprecated -Wundef -Wnon-virtual-dtor -Wunused-result -Werror -isystem /usr/include/python3.11 -isystem /usr/include/eigen3 -DPYBIND11_STRICT_ASSERTS_CLASS_HOLDER_VS_TYPE_CASTER_MIX -DPYBIND11_ENABLE_TYPE_CASTER_ODR_GUARD_IF_AVAILABLE -DPYBIND11_TEST_BOOST -Ipybind11/include -I/usr/local/google/home/rwgk/forked/pybind11/include -I/usr/local/google/home/rwgk/clone/pybind11/include /usr/local/google/home/rwgk/forked/pybind11/tests/test_numpy_array.cpp ``` ``` In file included from /usr/local/google/home/rwgk/forked/pybind11/tests/test_numpy_array.cpp:10: /usr/local/google/home/rwgk/forked/pybind11/include/pybind11/numpy.h: In static member function ‘static pybind11::detail::npy_api& pybind11::detail::npy_api::get()’: /usr/local/google/home/rwgk/forked/pybind11/include/pybind11/numpy.h:258:82: error: ‘constinit’ variable ‘api_init’ does not have a constant initializer 258 | PYBIND11_CONSTINIT static LazyInitializeAtLeastOnceDestroyNever<npy_api> api_init; | ^~~~~~~~ ``` ``` In file included from /usr/local/google/home/rwgk/forked/pybind11/tests/test_numpy_array.cpp:10: /usr/local/google/home/rwgk/forked/pybind11/include/pybind11/numpy.h: In static member function ‘static pybind11::object& pybind11::dtype::_dtype_from_pep3118()’: /usr/local/google/home/rwgk/forked/pybind11/include/pybind11/numpy.h:697:13: error: ‘constinit’ variable ‘imported_obj’ does not have a constant initializer 697 | imported_obj; | ^~~~~~~~~~~~ ``` * Revert "Add `PYBIND11_CONSTINIT`, but it does not work for the current use cases:" This reverts commit f07b28bda9f91fb723aa898a21c81b6dd6857072. * Reapply "Add `PYBIND11_CONSTINIT`, but it does not work for the current use cases:" This reverts commit 36be645758aa82b576d24003808386bec6e55bf9. * Add Default Member Initializer on `value_storage_` as suggested by @tkoeppe: https://github.com/pybind/pybind11/pull/4877#issuecomment-1753201342 This fixes the errors reported under commit f07b28bda9f91fb723aa898a21c81b6dd6857072. * Fix copy-paste-missed-a-change mishap in commit 88cec1152ab5576db19bab95c484672f06f5989a. * Semi-paranoid placement new (based on https://github.com/pybind/pybind11/pull/4877#discussion_r1350573114). * Move PYBIND11_CONSTINIT to detail/common.h * Move code to the right places, rename new class and some variables. * Fix oversight: update tests/extra_python_package/test_files.py * Get the name right first. * Use `std::call_once`, `std::atomic`, following a pattern developed by @tkoeppe * Make the API more self-documenting (and possibly more easily reusable). * google-clang-tidy IWYU fixes * Rewrite comment as suggested by @tkoeppe * Update test_exceptions.cpp and exceptions.rst * Fix oversight in previous commit: add `PYBIND11_CONSTINIT` * Make `get_stored()` non-const for simplicity. As suggested by @tkoeppe: not seeing any reasonable use in which `get_stored` has to be const. * Add comment regarding `KeyboardInterrupt` behavior, based heavily on information provided by @jbms. * Add `assert(PyGILState_Check())` in `gil_scoped_release` ctor (simple & non-simple implementation) as suggested by @EthanSteinberg. * Fix oversight in previous commit (missing include cassert). * Remove use of std::atomic, leaving comments with rationale, why it is not needed. * Rewrite comment re `std:optional` based on deeper reflection (aka 2nd thoughts). * Additional comment with the conclusion of a discussion under PR #4877. * https://github.com/pybind/pybind11/pull/4877#issuecomment-1757363179 * Small comment changes suggested by @tkoeppe.
2023-10-12 04:05:31 +00:00
py::set_error(exc_storage.get_stored(), e.what());
2016-10-16 17:12:43 +00:00
} catch (const OtherException &e) {
Add `py::set_error()`, use in updated `py::exception<>` documentation (#4772) * Copy clang 17 compatibility fixes from PR #4762 to a separate PR. * static py::exception<> -> static py::handle * Add `py::set_error()` but also try the suggestion of @malfet (https://github.com/pytorch/pytorch/pull/106401#pullrequestreview-1559961407). * clang 17 compatibility fixes (#4767) * Copy clang 17 compatibility fixes from PR #4762 to a separate PR. * Add gcc:13 C++20 * Add silkeh/clang:16-bullseye C++20 * chore(deps): update pre-commit hooks (#4770) updates: - [github.com/psf/black: 23.3.0 → 23.7.0](https://github.com/psf/black/compare/23.3.0...23.7.0) - [github.com/astral-sh/ruff-pre-commit: v0.0.276 → v0.0.281](https://github.com/astral-sh/ruff-pre-commit/compare/v0.0.276...v0.0.281) - [github.com/asottile/blacken-docs: 1.14.0 → 1.15.0](https://github.com/asottile/blacken-docs/compare/1.14.0...1.15.0) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * docs: Remove upper bound on pybind11 in example pyproject.toml for setuptools (#4774) * docs: Remove upper bound on pybind11 in example pyproject.toml for setuptools * Update docs/compiling.rst --------- Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com> * Provide better type hints for a variety of generic types (#4259) * Provide better type hints for a variety of generic types * Makes better documentation * tuple, dict, list, set, function * Move to py::typing * style: pre-commit fixes * Update copyright line with correct year and actual author. The author information was copy-pasted from the git log output. --------- Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Use `py::set_error()` everywhere possible (only one special case, in common.h). Overload `py::set_error(py::handle, py::handle)`. Change back to `static py::handle exc = ... .release();` Deprecate `py::exception<>::operator()` * Add `PYBIND11_WARNING_DISABLE` for INTEL and MSVC (and sort alphabetically). * `PYBIND11_WARNING_DISABLE_INTEL(10441)` does not work. For ICC only, falling back to the recommended `py::set_error()` to keep the testing simple. It is troublesome to add `--diag-disable=10441` specifically for test_exceptions.cpp, even that is non-ideal because it covers the entire file, not just the one line we need it for, and the value of exercising the trivial deprecated `operator()` on this one extra platform is practically zero. * Fix silly oversight. * NVHPC 23.5.0 generates deprecation warnings. They are currently not treated as errors, but falling back to using `py::set_error()` to not have to deal with that distraction. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Keto D. Zhang <keto.zhang@gmail.com> Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com> Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com>
2023-08-08 03:48:20 +00:00
py::set_error(PyExc_RuntimeError, e.what());
2016-10-16 17:12:43 +00:00
}
});
Multiple exceptions can be handled by a single translator, as shown in the
example above. If the exception is not caught by the current translator, the
previously registered one gets a chance.
If none of the registered exception translators is able to handle the
exception, it is handled by the default converter as described in the previous
section.
.. seealso::
The file :file:`tests/test_exceptions.cpp` contains examples
of various custom exception translators and custom exception types.
.. note::
Add `py::set_error()`, use in updated `py::exception<>` documentation (#4772) * Copy clang 17 compatibility fixes from PR #4762 to a separate PR. * static py::exception<> -> static py::handle * Add `py::set_error()` but also try the suggestion of @malfet (https://github.com/pytorch/pytorch/pull/106401#pullrequestreview-1559961407). * clang 17 compatibility fixes (#4767) * Copy clang 17 compatibility fixes from PR #4762 to a separate PR. * Add gcc:13 C++20 * Add silkeh/clang:16-bullseye C++20 * chore(deps): update pre-commit hooks (#4770) updates: - [github.com/psf/black: 23.3.0 → 23.7.0](https://github.com/psf/black/compare/23.3.0...23.7.0) - [github.com/astral-sh/ruff-pre-commit: v0.0.276 → v0.0.281](https://github.com/astral-sh/ruff-pre-commit/compare/v0.0.276...v0.0.281) - [github.com/asottile/blacken-docs: 1.14.0 → 1.15.0](https://github.com/asottile/blacken-docs/compare/1.14.0...1.15.0) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * docs: Remove upper bound on pybind11 in example pyproject.toml for setuptools (#4774) * docs: Remove upper bound on pybind11 in example pyproject.toml for setuptools * Update docs/compiling.rst --------- Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com> * Provide better type hints for a variety of generic types (#4259) * Provide better type hints for a variety of generic types * Makes better documentation * tuple, dict, list, set, function * Move to py::typing * style: pre-commit fixes * Update copyright line with correct year and actual author. The author information was copy-pasted from the git log output. --------- Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Use `py::set_error()` everywhere possible (only one special case, in common.h). Overload `py::set_error(py::handle, py::handle)`. Change back to `static py::handle exc = ... .release();` Deprecate `py::exception<>::operator()` * Add `PYBIND11_WARNING_DISABLE` for INTEL and MSVC (and sort alphabetically). * `PYBIND11_WARNING_DISABLE_INTEL(10441)` does not work. For ICC only, falling back to the recommended `py::set_error()` to keep the testing simple. It is troublesome to add `--diag-disable=10441` specifically for test_exceptions.cpp, even that is non-ideal because it covers the entire file, not just the one line we need it for, and the value of exercising the trivial deprecated `operator()` on this one extra platform is practically zero. * Fix silly oversight. * NVHPC 23.5.0 generates deprecation warnings. They are currently not treated as errors, but falling back to using `py::set_error()` to not have to deal with that distraction. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Keto D. Zhang <keto.zhang@gmail.com> Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com> Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com>
2023-08-08 03:48:20 +00:00
Call ``py::set_error()`` for every exception caught in a custom exception
2016-10-16 17:12:43 +00:00
translator. Failure to do so will cause Python to crash with ``SystemError:
error return without exception set``.
Exceptions that you do not plan to handle should simply not be caught, or
2018-01-09 17:30:19 +00:00
may be explicitly (re-)thrown to delegate it to the other,
2016-10-16 17:12:43 +00:00
previously-declared existing exception translators.
Note that ``libc++`` and ``libstdc++`` `behave differently under macOS
<https://stackoverflow.com/questions/19496643/using-clang-fvisibility-hidden-and-typeinfo-and-type-erasure/28827430>`_
with ``-fvisibility=hidden``. Therefore exceptions that are used across ABI
boundaries need to be explicitly exported, as exercised in
``tests/test_exceptions.h``. See also:
"Problems with C++ exceptions" under `GCC Wiki <https://gcc.gnu.org/wiki/Visibility>`_.
Feature/local exception translator (#2650) * Create a module_internals struct Since we now have two things that are going to be module local, it felt correct to add a struct to manage them. * Add local exception translators These are added via the register_local_exception_translator function and are then applied before the global translators * Add unit tests to show the local exception translator works * Fix a bug in the unit test with the string value of KeyError * Fix a formatting issue * Rename registered_local_types_cpp() Rename it to get_registered_local_types_cpp() to disambiguate from the new member of module_internals * Add additional comments to new local exception code path * Add a register_local_exception function * Add additional unit tests for register_local_exception * Use get_local_internals like get_internals * Update documentation for new local exception feature * Add back a missing space * Clean-up some issues in the docs * Remove the code duplication when translating exceptions Separated out the exception processing into a standalone function in the details namespace. Clean-up some comments as per PR notes as well * Remove the code duplication in register_exception * Cleanup some formatting things caught by clang-format * Remove the templates from exception translators But I added a using declaration to alias the type. * Remove the extra local from local_internals variable names * Add an extra explanatory comment to local_internals * Fix a typo in the code
2021-07-21 12:22:18 +00:00
Local vs Global Exception Translators
=====================================
When a global exception translator is registered, it will be applied across all
modules in the reverse order of registration. This can create behavior where the
order of module import influences how exceptions are translated.
If module1 has the following translator:
.. code-block:: cpp
py::register_exception_translator([](std::exception_ptr p) {
try {
if (p) std::rethrow_exception(p);
} catch (const std::invalid_argument &e) {
Add `py::set_error()`, use in updated `py::exception<>` documentation (#4772) * Copy clang 17 compatibility fixes from PR #4762 to a separate PR. * static py::exception<> -> static py::handle * Add `py::set_error()` but also try the suggestion of @malfet (https://github.com/pytorch/pytorch/pull/106401#pullrequestreview-1559961407). * clang 17 compatibility fixes (#4767) * Copy clang 17 compatibility fixes from PR #4762 to a separate PR. * Add gcc:13 C++20 * Add silkeh/clang:16-bullseye C++20 * chore(deps): update pre-commit hooks (#4770) updates: - [github.com/psf/black: 23.3.0 → 23.7.0](https://github.com/psf/black/compare/23.3.0...23.7.0) - [github.com/astral-sh/ruff-pre-commit: v0.0.276 → v0.0.281](https://github.com/astral-sh/ruff-pre-commit/compare/v0.0.276...v0.0.281) - [github.com/asottile/blacken-docs: 1.14.0 → 1.15.0](https://github.com/asottile/blacken-docs/compare/1.14.0...1.15.0) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * docs: Remove upper bound on pybind11 in example pyproject.toml for setuptools (#4774) * docs: Remove upper bound on pybind11 in example pyproject.toml for setuptools * Update docs/compiling.rst --------- Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com> * Provide better type hints for a variety of generic types (#4259) * Provide better type hints for a variety of generic types * Makes better documentation * tuple, dict, list, set, function * Move to py::typing * style: pre-commit fixes * Update copyright line with correct year and actual author. The author information was copy-pasted from the git log output. --------- Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Use `py::set_error()` everywhere possible (only one special case, in common.h). Overload `py::set_error(py::handle, py::handle)`. Change back to `static py::handle exc = ... .release();` Deprecate `py::exception<>::operator()` * Add `PYBIND11_WARNING_DISABLE` for INTEL and MSVC (and sort alphabetically). * `PYBIND11_WARNING_DISABLE_INTEL(10441)` does not work. For ICC only, falling back to the recommended `py::set_error()` to keep the testing simple. It is troublesome to add `--diag-disable=10441` specifically for test_exceptions.cpp, even that is non-ideal because it covers the entire file, not just the one line we need it for, and the value of exercising the trivial deprecated `operator()` on this one extra platform is practically zero. * Fix silly oversight. * NVHPC 23.5.0 generates deprecation warnings. They are currently not treated as errors, but falling back to using `py::set_error()` to not have to deal with that distraction. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Keto D. Zhang <keto.zhang@gmail.com> Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com> Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com>
2023-08-08 03:48:20 +00:00
py::set_error(PyExc_ArgumentError, "module1 handled this");
Feature/local exception translator (#2650) * Create a module_internals struct Since we now have two things that are going to be module local, it felt correct to add a struct to manage them. * Add local exception translators These are added via the register_local_exception_translator function and are then applied before the global translators * Add unit tests to show the local exception translator works * Fix a bug in the unit test with the string value of KeyError * Fix a formatting issue * Rename registered_local_types_cpp() Rename it to get_registered_local_types_cpp() to disambiguate from the new member of module_internals * Add additional comments to new local exception code path * Add a register_local_exception function * Add additional unit tests for register_local_exception * Use get_local_internals like get_internals * Update documentation for new local exception feature * Add back a missing space * Clean-up some issues in the docs * Remove the code duplication when translating exceptions Separated out the exception processing into a standalone function in the details namespace. Clean-up some comments as per PR notes as well * Remove the code duplication in register_exception * Cleanup some formatting things caught by clang-format * Remove the templates from exception translators But I added a using declaration to alias the type. * Remove the extra local from local_internals variable names * Add an extra explanatory comment to local_internals * Fix a typo in the code
2021-07-21 12:22:18 +00:00
}
}
and module2 has the following similar translator:
.. code-block:: cpp
py::register_exception_translator([](std::exception_ptr p) {
try {
if (p) std::rethrow_exception(p);
} catch (const std::invalid_argument &e) {
Add `py::set_error()`, use in updated `py::exception<>` documentation (#4772) * Copy clang 17 compatibility fixes from PR #4762 to a separate PR. * static py::exception<> -> static py::handle * Add `py::set_error()` but also try the suggestion of @malfet (https://github.com/pytorch/pytorch/pull/106401#pullrequestreview-1559961407). * clang 17 compatibility fixes (#4767) * Copy clang 17 compatibility fixes from PR #4762 to a separate PR. * Add gcc:13 C++20 * Add silkeh/clang:16-bullseye C++20 * chore(deps): update pre-commit hooks (#4770) updates: - [github.com/psf/black: 23.3.0 → 23.7.0](https://github.com/psf/black/compare/23.3.0...23.7.0) - [github.com/astral-sh/ruff-pre-commit: v0.0.276 → v0.0.281](https://github.com/astral-sh/ruff-pre-commit/compare/v0.0.276...v0.0.281) - [github.com/asottile/blacken-docs: 1.14.0 → 1.15.0](https://github.com/asottile/blacken-docs/compare/1.14.0...1.15.0) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * docs: Remove upper bound on pybind11 in example pyproject.toml for setuptools (#4774) * docs: Remove upper bound on pybind11 in example pyproject.toml for setuptools * Update docs/compiling.rst --------- Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com> * Provide better type hints for a variety of generic types (#4259) * Provide better type hints for a variety of generic types * Makes better documentation * tuple, dict, list, set, function * Move to py::typing * style: pre-commit fixes * Update copyright line with correct year and actual author. The author information was copy-pasted from the git log output. --------- Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Use `py::set_error()` everywhere possible (only one special case, in common.h). Overload `py::set_error(py::handle, py::handle)`. Change back to `static py::handle exc = ... .release();` Deprecate `py::exception<>::operator()` * Add `PYBIND11_WARNING_DISABLE` for INTEL and MSVC (and sort alphabetically). * `PYBIND11_WARNING_DISABLE_INTEL(10441)` does not work. For ICC only, falling back to the recommended `py::set_error()` to keep the testing simple. It is troublesome to add `--diag-disable=10441` specifically for test_exceptions.cpp, even that is non-ideal because it covers the entire file, not just the one line we need it for, and the value of exercising the trivial deprecated `operator()` on this one extra platform is practically zero. * Fix silly oversight. * NVHPC 23.5.0 generates deprecation warnings. They are currently not treated as errors, but falling back to using `py::set_error()` to not have to deal with that distraction. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Keto D. Zhang <keto.zhang@gmail.com> Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com> Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com>
2023-08-08 03:48:20 +00:00
py::set_error(PyExc_ArgumentError, "module2 handled this");
Feature/local exception translator (#2650) * Create a module_internals struct Since we now have two things that are going to be module local, it felt correct to add a struct to manage them. * Add local exception translators These are added via the register_local_exception_translator function and are then applied before the global translators * Add unit tests to show the local exception translator works * Fix a bug in the unit test with the string value of KeyError * Fix a formatting issue * Rename registered_local_types_cpp() Rename it to get_registered_local_types_cpp() to disambiguate from the new member of module_internals * Add additional comments to new local exception code path * Add a register_local_exception function * Add additional unit tests for register_local_exception * Use get_local_internals like get_internals * Update documentation for new local exception feature * Add back a missing space * Clean-up some issues in the docs * Remove the code duplication when translating exceptions Separated out the exception processing into a standalone function in the details namespace. Clean-up some comments as per PR notes as well * Remove the code duplication in register_exception * Cleanup some formatting things caught by clang-format * Remove the templates from exception translators But I added a using declaration to alias the type. * Remove the extra local from local_internals variable names * Add an extra explanatory comment to local_internals * Fix a typo in the code
2021-07-21 12:22:18 +00:00
}
}
then which translator handles the invalid_argument will be determined by the
order that module1 and module2 are imported. Since exception translators are
applied in the reverse order of registration, which ever module was imported
last will "win" and that translator will be applied.
If there are multiple pybind11 modules that share exception types (either
standard built-in or custom) loaded into a single python instance and
consistent error handling behavior is needed, then local translators should be
used.
Changing the previous example to use ``register_local_exception_translator``
would mean that when invalid_argument is thrown in the module2 code, the
module2 translator will always handle it, while in module1, the module1
translator will do the same.
.. _handling_python_exceptions_cpp:
Handling exceptions from Python in C++
======================================
When C++ calls Python functions, such as in a callback function or when
manipulating Python objects, and Python raises an ``Exception``, pybind11
converts the Python exception into a C++ exception of type
:class:`pybind11::error_already_set` whose payload contains a C++ string textual
summary and the actual Python exception. ``error_already_set`` is used to
propagate Python exception back to Python (or possibly, handle them in C++).
.. tabularcolumns:: |p{0.5\textwidth}|p{0.45\textwidth}|
+--------------------------------------+--------------------------------------+
| Exception raised in Python | Thrown as C++ exception type |
+======================================+======================================+
| Any Python ``Exception`` | :class:`pybind11::error_already_set` |
+--------------------------------------+--------------------------------------+
For example:
.. code-block:: cpp
try {
// open("missing.txt", "r")
auto file = py::module_::import("io").attr("open")("missing.txt", "r");
auto text = file.attr("read")();
file.attr("close")();
} catch (py::error_already_set &e) {
if (e.matches(PyExc_FileNotFoundError)) {
py::print("missing.txt not found");
} else if (e.matches(PyExc_PermissionError)) {
py::print("missing.txt found but not accessible");
} else {
throw;
}
}
Note that C++ to Python exception translation does not apply here, since that is
a method for translating C++ exceptions to Python, not vice versa. The error raised
from Python is always ``error_already_set``.
This example illustrates this behavior:
.. code-block:: cpp
try {
py::eval("raise ValueError('The Ring')");
} catch (py::value_error &boromir) {
// Boromir never gets the ring
assert(false);
} catch (py::error_already_set &frodo) {
// Frodo gets the ring
py::print("I will take the ring");
}
try {
// py::value_error is a request for pybind11 to raise a Python exception
throw py::value_error("The ball");
} catch (py::error_already_set &cat) {
// cat won't catch the ball since
// py::value_error is not a Python exception
assert(false);
} catch (py::value_error &dog) {
// dog will catch the ball
py::print("Run Spot run");
throw; // Throw it again (pybind11 will raise ValueError)
}
Handling errors from the Python C API
=====================================
Where possible, use :ref:`pybind11 wrappers <wrappers>` instead of calling
the Python C API directly. When calling the Python C API directly, in
addition to manually managing reference counts, one must follow the pybind11
error protocol, which is outlined here.
After calling the Python C API, if Python returns an error,
``throw py::error_already_set();``, which allows pybind11 to deal with the
exception and pass it back to the Python interpreter. This includes calls to
Add `py::set_error()`, use in updated `py::exception<>` documentation (#4772) * Copy clang 17 compatibility fixes from PR #4762 to a separate PR. * static py::exception<> -> static py::handle * Add `py::set_error()` but also try the suggestion of @malfet (https://github.com/pytorch/pytorch/pull/106401#pullrequestreview-1559961407). * clang 17 compatibility fixes (#4767) * Copy clang 17 compatibility fixes from PR #4762 to a separate PR. * Add gcc:13 C++20 * Add silkeh/clang:16-bullseye C++20 * chore(deps): update pre-commit hooks (#4770) updates: - [github.com/psf/black: 23.3.0 → 23.7.0](https://github.com/psf/black/compare/23.3.0...23.7.0) - [github.com/astral-sh/ruff-pre-commit: v0.0.276 → v0.0.281](https://github.com/astral-sh/ruff-pre-commit/compare/v0.0.276...v0.0.281) - [github.com/asottile/blacken-docs: 1.14.0 → 1.15.0](https://github.com/asottile/blacken-docs/compare/1.14.0...1.15.0) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * docs: Remove upper bound on pybind11 in example pyproject.toml for setuptools (#4774) * docs: Remove upper bound on pybind11 in example pyproject.toml for setuptools * Update docs/compiling.rst --------- Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com> * Provide better type hints for a variety of generic types (#4259) * Provide better type hints for a variety of generic types * Makes better documentation * tuple, dict, list, set, function * Move to py::typing * style: pre-commit fixes * Update copyright line with correct year and actual author. The author information was copy-pasted from the git log output. --------- Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Use `py::set_error()` everywhere possible (only one special case, in common.h). Overload `py::set_error(py::handle, py::handle)`. Change back to `static py::handle exc = ... .release();` Deprecate `py::exception<>::operator()` * Add `PYBIND11_WARNING_DISABLE` for INTEL and MSVC (and sort alphabetically). * `PYBIND11_WARNING_DISABLE_INTEL(10441)` does not work. For ICC only, falling back to the recommended `py::set_error()` to keep the testing simple. It is troublesome to add `--diag-disable=10441` specifically for test_exceptions.cpp, even that is non-ideal because it covers the entire file, not just the one line we need it for, and the value of exercising the trivial deprecated `operator()` on this one extra platform is practically zero. * Fix silly oversight. * NVHPC 23.5.0 generates deprecation warnings. They are currently not treated as errors, but falling back to using `py::set_error()` to not have to deal with that distraction. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Keto D. Zhang <keto.zhang@gmail.com> Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com> Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com>
2023-08-08 03:48:20 +00:00
the error setting functions such as ``py::set_error()``.
.. code-block:: cpp
Add `py::set_error()`, use in updated `py::exception<>` documentation (#4772) * Copy clang 17 compatibility fixes from PR #4762 to a separate PR. * static py::exception<> -> static py::handle * Add `py::set_error()` but also try the suggestion of @malfet (https://github.com/pytorch/pytorch/pull/106401#pullrequestreview-1559961407). * clang 17 compatibility fixes (#4767) * Copy clang 17 compatibility fixes from PR #4762 to a separate PR. * Add gcc:13 C++20 * Add silkeh/clang:16-bullseye C++20 * chore(deps): update pre-commit hooks (#4770) updates: - [github.com/psf/black: 23.3.0 → 23.7.0](https://github.com/psf/black/compare/23.3.0...23.7.0) - [github.com/astral-sh/ruff-pre-commit: v0.0.276 → v0.0.281](https://github.com/astral-sh/ruff-pre-commit/compare/v0.0.276...v0.0.281) - [github.com/asottile/blacken-docs: 1.14.0 → 1.15.0](https://github.com/asottile/blacken-docs/compare/1.14.0...1.15.0) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * docs: Remove upper bound on pybind11 in example pyproject.toml for setuptools (#4774) * docs: Remove upper bound on pybind11 in example pyproject.toml for setuptools * Update docs/compiling.rst --------- Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com> * Provide better type hints for a variety of generic types (#4259) * Provide better type hints for a variety of generic types * Makes better documentation * tuple, dict, list, set, function * Move to py::typing * style: pre-commit fixes * Update copyright line with correct year and actual author. The author information was copy-pasted from the git log output. --------- Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Use `py::set_error()` everywhere possible (only one special case, in common.h). Overload `py::set_error(py::handle, py::handle)`. Change back to `static py::handle exc = ... .release();` Deprecate `py::exception<>::operator()` * Add `PYBIND11_WARNING_DISABLE` for INTEL and MSVC (and sort alphabetically). * `PYBIND11_WARNING_DISABLE_INTEL(10441)` does not work. For ICC only, falling back to the recommended `py::set_error()` to keep the testing simple. It is troublesome to add `--diag-disable=10441` specifically for test_exceptions.cpp, even that is non-ideal because it covers the entire file, not just the one line we need it for, and the value of exercising the trivial deprecated `operator()` on this one extra platform is practically zero. * Fix silly oversight. * NVHPC 23.5.0 generates deprecation warnings. They are currently not treated as errors, but falling back to using `py::set_error()` to not have to deal with that distraction. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Keto D. Zhang <keto.zhang@gmail.com> Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com> Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com>
2023-08-08 03:48:20 +00:00
py::set_error(PyExc_TypeError, "C API type error demo");
throw py::error_already_set();
// But it would be easier to simply...
throw py::type_error("pybind11 wrapper type error");
Alternately, to ignore the error, call `PyErr_Clear
<https://docs.python.org/3/c-api/exceptions.html#c.PyErr_Clear>`_.
Any Python error must be thrown or cleared, or Python/pybind11 will be left in
an invalid state.
2024-09-05 21:28:53 +00:00
Handling warnings from the Python C API
=====================================
2024-09-30 22:30:52 +00:00
Wrappers for handling Python warnings are implemented in ``pybind11/warnings.h``,
2024-09-30 22:35:41 +00:00
which means that ``#include`` must be added explicitly (in other words, it is not
transitively included with ``pybind11/pybind11.h``).
2024-09-05 21:28:53 +00:00
2024-09-30 22:30:52 +00:00
Warnings can be raised with the ``warn`` function:
2024-09-05 21:28:53 +00:00
.. code-block:: cpp
py::warnings::warn("This is warning!", PyExc_Warning);
2024-09-30 22:30:52 +00:00
// Optionally, `stack_level` can be specified.
2024-09-05 21:28:53 +00:00
py::warnings::warn("Another one!", PyExc_DeprecationWarning, 3);
2024-09-05 21:59:11 +00:00
New warning types can be registered on the module level with ``new_warning_type``:
2024-09-05 21:28:53 +00:00
.. code-block:: cpp
py::warnings::new_warning_type(m, "CustomWarning", PyExc_RuntimeWarning);
Chaining exceptions ('raise from')
==================================
Python has a mechanism for indicating that exceptions were caused by other
exceptions:
.. code-block:: py
try:
print(1 / 0)
except Exception as exc:
raise RuntimeError("could not divide by zero") from exc
To do a similar thing in pybind11, you can use the ``py::raise_from`` function. It
sets the current python error indicator, so to continue propagating the exception
you should ``throw py::error_already_set()``.
.. code-block:: cpp
try {
py::eval("print(1 / 0"));
} catch (py::error_already_set &e) {
py::raise_from(e, PyExc_RuntimeError, "could not divide by zero");
throw py::error_already_set();
}
.. versionadded:: 2.8
.. _unraisable_exceptions:
Handling unraisable exceptions
==============================
If a Python function invoked from a C++ destructor or any function marked
``noexcept(true)`` (collectively, "noexcept functions") throws an exception, there
is no way to propagate the exception, as such functions may not throw.
Should they throw or fail to catch any exceptions in their call graph,
the C++ runtime calls ``std::terminate()`` to abort immediately.
Similarly, Python exceptions raised in a class's ``__del__`` method do not
feat: remove Python 3.7 support (#5191) * First pass updating misc files, informed by https://github.com/pybind/pybind11/pull/5177/commits * Remove jobs using silkeh/clang and gcc docker containers that come with Python 3.7 * Add silkeh/clang:17-bookworm * Add job using GCC 7 * Revert "Add job using GCC 7" This reverts commit 518515a761ac37dc2cf5d0980da82d0de39edc28. * Try running in ubuntu-18.04 container under ubuntu-latest (to get GCC 7) * Fix `-` vs `:` mixup. * This reverts commit b1c4304475b8ad129c12330c7ed7eb85d15ba14a. Revert "Try running in ubuntu:18.04 container under ubuntu-latest (to get GCC 7)" This reverts commit b203a294bb444fc6ae57a0100fa91dc91b8d3264. * `git grep 0x03080000` cleanup. * `git grep -I -E '3\.7'` cleanup. Removes two changes made under pybind/pybind11#3702 * Revert "`git grep -I -E '3\.7'` cleanup." This reverts commit bb5b9d187bffbfb61e2977d7eee46b766fa1cce9. * Remove comments that are evidently incorrect: ``` ... -- The CXX compiler identification is Clang 15.0.7 ... - Found Python: /usr/bin/python3.9 (found suitable version "3.9.2", minimum required is "3.7") found components: Interpreter Development.Module Development.Embed ... /__w/pybind11/pybind11/include/pybind11/gil.h:150:13: error: 'auto key' can be declared as 'auto *key' [readability-qualified-auto,-warnings-as-errors] auto key = internals.tstate; ^~~~~ auto * /__w/pybind11/pybind11/include/pybind11/gil.h:174:13: error: 'auto key' can be declared as 'auto *key' [readability-qualified-auto,-warnings-as-errors] auto key = detail::get_internals().tstate; ^~~~~ auto * ``` * .github/workflows/configure.yml: Change from Python 3.7 to 3.8 * Misc cleanup pass * Miscellaneous changes based on manual review of the `git grep` matches below: ``` git_grep_37_38.sh |& sort | uniq -c ``` With git_grep_37_38.sh: ``` set -x git grep 0x0307 git grep 0x0308 git grep PY_MINOR_VERSION git grep PYPY_VERSION git grep -I -E '3\.7' git grep -I -E '3\.8' git grep -I -E '\(3, 7' git grep -I -E '\(3, 8' git grep -I -E '3[^A-Za-z0-9.]+7' git grep -I -E '3[^A-Za-z0-9.]+8' ``` Output: ``` 1 .appveyor.yml: $env:CMAKE_INCLUDE_PATH = "eigen-3.3.7;$env:CMAKE_INCLUDE_PATH" 1 .appveyor.yml: 7z x eigen-3.3.7.zip -y > $null 1 .appveyor.yml: Start-FileDownload 'https://gitlab.com/libeigen/eigen/-/archive/3.3.7/eigen-3.3.7.zip' 1 CMakeLists.txt: # Bug in macOS CMake < 3.7 is unable to download catch 1 CMakeLists.txt: elseif(WINDOWS AND CMAKE_VERSION VERSION_LESS 3.8) 1 CMakeLists.txt: if(OSX AND CMAKE_VERSION VERSION_LESS 3.7) 1 CMakeLists.txt: message(WARNING "CMAKE 3.7+ needed on macOS to download catch, and newer HIGHLY recommended") 1 CMakeLists.txt: message(WARNING "CMAKE 3.8+ tested on Windows, previous versions untested") 1 CMakeLists.txt: # Only tested with 3.8+ in CI. 1 docs/advanced/functions.rst:Python 3.8 introduced a new positional-only argument syntax, using ``/`` in the 1 docs/changelog.rst:* Adapt pybind11 to a C API convention change in Python 3.8. `#1950 1 docs/changelog.rst:* Allow thread termination to be avoided during shutdown for CPython 3.7+ via 1 docs/changelog.rst: considered as conversion, consistent with Python 3.8+. 1 docs/changelog.rst: CPython 3.8 and 3.9 debug builds. 1 docs/changelog.rst:* Enum now has an ``__index__`` method on Python <3.8 too. 1 docs/changelog.rst: on Python 3.8. `#1780 <https://github.com/pybind/pybind11/pull/1780>`_. 1 docs/changelog.rst:* PyPy 3.10 support was added, PyPy 3.7 support was dropped. 2 docs/changelog.rst:* Support PyPy 7.3.7 and the PyPy3.8 beta. Test python-3.11 on PRs with the 1 docs/changelog.rst:* Use ``macos-13`` (Intel) for CI jobs for now (will drop Python 3.7 soon). 1 docs/changelog.rst:* Use new Python 3.7 Thread Specific Storage (TSS) implementation if available. 1 docs/compiling.rst: cmake -DPYBIND11_PYTHON_VERSION=3.8 .. 1 docs/compiling.rst: find_package(Python 3.8 COMPONENTS Interpreter Development REQUIRED) 1 docs/limitations.rst:- PyPy3 7.3.1 and 7.3.2 have issues with several tests on 32-bit Windows. 1 docs/requirements.txt:idna==3.7 \ 1 + git grep 0x0307 1 + git grep 0x0308 1 + git grep -I -E '\(3, 7' 1 + git grep -I -E '3\.7' 1 + git grep -I -E '\(3, 8' 1 + git grep -I -E '3\.8' 1 + git grep -I -E '3[^A-Za-z0-9.]+7' 1 + git grep -I -E '3[^A-Za-z0-9.]+8' 1 + git grep PY_MINOR_VERSION 1 + git grep PYPY_VERSION 2 .github/workflows/ci.yml: - '3.8' 1 .github/workflows/ci.yml: - 3.8 1 .github/workflows/ci.yml: - name: Add Python 3.8 1 .github/workflows/ci.yml: - 'pypy-3.8' 2 .github/workflows/ci.yml: python: '3.8' 1 .github/workflows/ci.yml: - python: '3.8' 1 .github/workflows/ci.yml: - python: 3.8 1 .github/workflows/ci.yml: python: 'pypy-3.8' 1 .github/workflows/configure.yml: cmake: "3.8" 1 .github/workflows/configure.yml: name: 🐍 3.8 • CMake ${{ matrix.cmake }} • ${{ matrix.runs-on }} 1 .github/workflows/configure.yml: - name: Setup Python 3.8 1 .github/workflows/configure.yml: python-version: 3.8 1 .github/workflows/pip.yml: name: 🐍 3.8 • 📦 & 📦 tests • ubuntu-latest 1 .github/workflows/pip.yml: name: 🐍 3.8 • 📦 tests • windows-latest 2 .github/workflows/pip.yml: - name: Setup 🐍 3.8 2 .github/workflows/pip.yml: python-version: 3.8 2 include/pybind11/cast.h:#if !defined(PYPY_VERSION) 2 include/pybind11/cast.h:#if defined(PYPY_VERSION) 2 include/pybind11/cast.h: // PyPy: 7.3.7's 3.8 does not implement PyLong_*'s __index__ calls. 5 include/pybind11/detail/class.h:#if !defined(PYPY_VERSION) 1 include/pybind11/detail/class.h:#if defined(PYPY_VERSION) 1 include/pybind11/detail/class.h: // This was not needed before Python 3.8 (Python issue 35810) 1 include/pybind11/detail/common.h: && !defined(PYPY_VERSION) && !defined(PYBIND11_ASSERT_GIL_HELD_INCREF_DECREF) 2 include/pybind11/detail/common.h:# error "PYTHON < 3.8 IS UNSUPPORTED. pybind11 v2.13 was the last to support Python 3.7." 1 include/pybind11/detail/common.h:#if defined(PYPY_VERSION) && !defined(PYBIND11_SIMPLE_GIL_MANAGEMENT) 1 include/pybind11/detail/common.h:#if PY_VERSION_HEX < 0x03080000 1 include/pybind11/detail/common.h: = PYBIND11_TOSTRING(PY_MAJOR_VERSION) "." PYBIND11_TOSTRING(PY_MINOR_VERSION); \ 1 include/pybind11/detail/internals.h: // called. PYBIND11_TLS_FREE is PyThread_tss_free on python 3.7+. On older python, it does 1 include/pybind11/detail/internals.h:#if PYBIND11_INTERNALS_VERSION <= 4 || defined(PYPY_VERSION) 1 include/pybind11/detail/internals.h:// The old Python Thread Local Storage (TLS) API is deprecated in Python 3.7 in favor of the new 1 include/pybind11/detail/type_caster_base.h:#if defined(PYPY_VERSION) 1 include/pybind11/embed.h:# define PYBIND11_PYCONFIG_SUPPORT_PY_VERSION_HEX (0x03080000) 1 include/pybind11/embed.h:#if defined(PYPY_VERSION) 1 include/pybind11/eval.h: // globals if not yet present. Python 3.8 made PyRun_String behave 2 include/pybind11/eval.h:#if defined(PYPY_VERSION) 2 include/pybind11/eval.h: // was missing from PyPy3.8 7.3.7. 2 include/pybind11/gil.h: /// allowed during shutdown. Check _Py_IsFinalizing() on Python 3.7+, and 1 include/pybind11/pybind11.h:#if !defined(PYPY_VERSION) 4 include/pybind11/pybind11.h:#if !defined(PYPY_VERSION) && PY_MAJOR_VERSION == 3 && PY_MINOR_VERSION == 9 1 include/pybind11/pytypes.h:#endif //! defined(PYPY_VERSION) 2 include/pybind11/pytypes.h:#if !defined(PYPY_VERSION) 1 include/pybind11/pytypes.h:# if defined(PYPY_VERSION_NUM) && PYPY_VERSION_NUM < 0x07030a00 1 include/pybind11/pytypes.h:#ifdef PYPY_VERSION 1 include/pybind11/stl/filesystem.h:# if !defined(PYPY_VERSION) 2 pybind11/__init__.py:if sys.version_info < (3, 8): 2 pybind11/__init__.py: msg = "pybind11 does not support Python < 3.8. v2.13 was the last release supporting Python 3.7." 1 pyproject.toml:master.py-version = "3.8" 1 pyproject.toml:python_version = "3.8" 1 README.rst:lines of code and depend on Python (3.8+, or PyPy) and the C++ 2 README.rst:- Python 3.8+, and PyPy3 7.3 are supported with an implementation-agnostic 1 setup.cfg: Programming Language :: Python :: 3.8 1 setup.cfg:python_requires = >=3.8 1 setup.py:# TODO: use literals & overload (typing extensions or Python 3.8) 1 tests/CMakeLists.txt:if(NOT CMAKE_VERSION VERSION_LESS 3.8) 2 tests/constructor_stats.h:#if defined(PYPY_VERSION) 1 tests/env.py: doesn't work on CPython 3.8.0 with pytest==3.3.2 on Ubuntu 18.04 (#2922). 1 tests/requirements.txt:build~=1.0; python_version>="3.8" 1 tests/requirements.txt:numpy~=1.21.5; platform_python_implementation!="PyPy" and python_version>="3.8" and python_version<"3.10" 1 tests/requirements.txt:numpy~=1.23.0; python_version=="3.8" and platform_python_implementation=="PyPy" 1 tests/test_buffers.py: env.PYPY, reason="PyPy 7.3.7 doesn't clear this anymore", strict=False 1 tests/test_builtin_casters.py: # Before Python 3.8, `PyLong_AsLong` does not pick up on `obj.__index__`, 2 tests/test_builtin_casters.py: if (3, 8) <= sys.version_info < (3, 10) and env.CPYTHON: 4 tests/test_builtin_casters.py: # TODO: PyPy 3.8 does not behave like CPython 3.8 here yet (7.3.7) 1 tests/test_callbacks.py: assert m.test_callback3(z.double) == "func(43) = 86" 2 tests/test_call_policies.cpp:#if !defined(PYPY_VERSION) 1 tests/test_chrono.py: diff = m.test_chrono_float_diff(43.789012, 1.123456) 1 tests/test_constants_and_functions.py: assert m.f3(86) == 89 1 tests/test_eigen_matrix.py: a_copy3[8, 1] = 11 1 tests/test_eigen_matrix.py: assert np.all(cornersc == np.array([[1.0, 3], [7, 9]])) 1 tests/test_eigen_matrix.py: assert np.all(cornersr == np.array([[1.0, 3], [7, 9]])) 1 tests/test_eigen_matrix.py: mymat = chol(np.array([[1.0, 2, 4], [2, 13, 23], [4, 23, 77]])) 1 tests/test_exceptions.py: if hasattr(pytest, unraisable): # Python >= 3.8 and pytest >= 6 2 tests/test_exceptions.py:@pytest.mark.xfail(env.PYPY, reason="Failure on PyPy 3.8 (7.3.7)", strict=False) 1 tests/test_factory_constructors.py: assert [i.alive() for i in cstats] == [13, 7] 1 tests/test_kwargs_and_defaults.cpp:#ifdef PYPY_VERSION 1 tests/test_local_bindings.py: assert i1.get3() == 8 1 tests/test_methods_and_attributes.cpp:#if !defined(PYPY_VERSION) 1 tests/test_numpy_array.py: a = np.arange(3 * 7 * 2) + 1 1 tests/test_numpy_array.py: assert str(excinfo.value) == "cannot reshape array of size 42 into shape (3,7,1)" 2 tests/test_numpy_array.py: assert x.shape == (3, 7, 2) 2 tests/test_numpy_array.py: m.reshape_tuple(a, (3, 7, 1)) 2 tests/test_numpy_array.py: x = m.reshape_tuple(a, (3, 7, 2)) 1 tests/test_numpy_vectorize.py: assert np.isclose(m.vectorized_func3(np.array(3 + 7j)), [6 + 14j]) 1 tests/test_pickling.cpp:#if !defined(PYPY_VERSION) 1 tests/test_pytypes.cpp:#if (defined(__APPLE__) && defined(__clang__)) || defined(PYPY_VERSION) 1 tests/test_smart_ptr.cpp: m.def("make_myobject3_1", []() { return new MyObject3(8); }); 1 tests/test_smart_ptr.py: assert cstats.values() == ["MyObject3[9]", "MyObject3[8]", "MyObject3[9]"] 1 tests/test_stl_binders.py: assert v_int2 == m.VectorInt([0, 99, 2, 3, 4, 5, 6, 7, 0, 1, 2, 3, 88]) 1 tests/test_stl_binders.py: assert v_int2 == m.VectorInt([0, 99, 2, 3, 4, 5, 6, 7, 0, 1, 2, 3, 88, 4]) 1 tests/test_type_caster_pyobject_ptr.cpp:#if !defined(PYPY_VERSION) // It is not worth the trouble doing something special for PyPy. 1 tools/FindPythonLibsNew.cmake: set(PythonLibsNew_FIND_VERSION "3.8") 1 tools/JoinPaths.cmake:# https://docs.python.org/3.7/library/os.path.html#os.path.join 1 tools/pybind11NewTools.cmake: Python 3.8 REQUIRED COMPONENTS ${_pybind11_interp_component} ${_pybind11_dev_component} 1 tools/pybind11NewTools.cmake:# Python debug libraries expose slightly different objects before 3.8 1 tools/pybind11Tools.cmake: "3.12;3.11;3.10;3.9;3.8" 1 tools/pybind11Tools.cmake: if(NOT DEFINED PYPY_VERSION) 1 tools/pybind11Tools.cmake: message(STATUS "PYPY ${PYPY_VERSION} (Py ${PYTHON_VERSION})") 1 tools/pybind11Tools.cmake:# Python debug libraries expose slightly different objects before 3.8 1 tools/pybind11Tools.cmake: set(PYPY_VERSION ``` * Change `[tool.ruff]` `target-version` to `"py38"`, as suggested by @Skylion007
2024-07-30 16:18:35 +00:00
propagate, but ``sys.unraisablehook()`` `is triggered
<https://docs.python.org/3/library/sys.html#sys.unraisablehook>`_
and an auditing event is logged.
Any noexcept function should have a try-catch block that traps
class:`error_already_set` (or any other exception that can occur). Note that
pybind11 wrappers around Python exceptions such as
:class:`pybind11::value_error` are *not* Python exceptions; they are C++
exceptions that pybind11 catches and converts to Python exceptions. Noexcept
functions cannot propagate these exceptions either. A useful approach is to
convert them to Python exceptions and then ``discard_as_unraisable`` as shown
below.
.. code-block:: cpp
void nonthrowing_func() noexcept(true) {
try {
// ...
} catch (py::error_already_set &eas) {
// Discard the Python error using Python APIs, using the C++ magic
// variable __func__. Python already knows the type and value and of the
// exception object.
eas.discard_as_unraisable(__func__);
} catch (const std::exception &e) {
// Log and discard C++ exceptions.
third_party::log(e);
}
}
.. versionadded:: 2.6