Zengtudor
/
pybind11
mirror of https://github.com/pybind/pybind11.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Improve documentation of Python and C++ exceptions (#2408) 

The main change is to treat error_already_set as a separate category
of exception that arises in different circumstances and needs to be
handled differently. The asymmetry between Python and C++ exceptions
is further emphasized.
This commit is contained in:
jbarlow83 2020-08-22 15:11:09 -07:00 committed by GitHub
parent c58f7b745b
commit b8863698d6
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
3 changed files with 141 additions and 37 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.gitignore vendored
View File

@ -38,3 +38,4 @@ sosize-*.txt
pybind11Config*.cmake
pybind11Targets.cmake
/*env*
/.vscode

167
docs/advanced/exceptions.rst
View File

@ -1,18 +1,24 @@
Exceptions
##########
Built-in exception translation
==============================
Built-in C++ to Python exception translation
============================================
When C++ code invoked from Python throws an ``std::exception``, it is
automatically converted into a Python ``Exception``. pybind11 defines multiple
special exception classes that will map to different types of Python
exceptions:
When Python calls C++ code through pybind11, pybind11 provides a C++ exception handler
that will trap C++ exceptions, translate them to the corresponding Python exception,
and raise them so that Python code can handle them.
pybind11 defines translations for ``std::exception`` and its standard
subclasses, and several special exception classes that translate to specific
Python exceptions. Note that these are not actually Python exceptions, so they
cannot be examined using the Python C API. Instead, they are pure C++ objects
that pybind11 will translate the corresponding Python exception when they arrive
at its exception handler.
.. tabularcolumns:: |p{0.5\textwidth}|p{0.45\textwidth}|
+--------------------------------------+--------------------------------------+
| C++ exception type | Python exception type |
| Exception thrown by C++ | Translated to Python exception type |
+======================================+======================================+
| :class:`std::exception` | ``RuntimeError`` |
+--------------------------------------+--------------------------------------+
@ -46,22 +52,11 @@ exceptions:
| | ``__setitem__`` in dict-like |
| | objects, etc.) |
+--------------------------------------+--------------------------------------+
| :class:`pybind11::error_already_set` | Indicates that the Python exception |
| | flag has already been set via Python |
| | API calls from C++ code; this C++ |
| | exception is used to propagate such |
| | a Python exception back to Python. |
+--------------------------------------+--------------------------------------+
When a Python function invoked from C++ throws an exception, pybind11 will convert
it into a C++ exception of type :class:`error_already_set` whose string payload
contains a textual summary. If you call the Python C-API directly, and it
returns an error, you should ``throw py::error_already_set();``, which allows
pybind11 to deal with the exception and pass it back to the Python interpreter.
(Another option is to call ``PyErr_Clear`` in the
`Python C-API <https://docs.python.org/3/c-api/exceptions.html#c.PyErr_Clear>`_
to clear the error. The Python error must be thrown or cleared, or Python/pybind11
will be left in an invalid state.)
Exception translation is not bidirectional. That is, *catching* the C++
exceptions defined above above will not trap exceptions that originate from
Python. For that, catch :class:`pybind11::error_already_set`. See :ref:`below
<handling_python_exceptions_cpp>` for further details.
There is also a special exception :class:`cast_error` that is thrown by
:func:`handle::call` when the input arguments cannot be converted to Python
@ -106,7 +101,6 @@ and use this in the associated exception translator (note: it is often useful
to make this a static declaration when using it inside a lambda expression
without requiring capturing).
The following example demonstrates this for a hypothetical exception classes
``MyCustomException`` and ``OtherException``: the first is translated to a
custom python exception ``MyCustomError``, while the second is translated to a
@ -140,7 +134,7 @@ section.
.. note::
You must call either ``PyErr_SetString`` or a custom exception's call
Call either ``PyErr_SetString`` or a custom exception's call
operator (``exc(string)``) for every exception caught in a custom exception
translator. Failure to do so will cause Python to crash with ``SystemError:
error return without exception set``.
@ -149,6 +143,103 @@ section.
may be explicitly (re-)thrown to delegate it to the other,
previously-declared existing exception translators.
.. _handling_python_exceptions_cpp:
Handling exceptions from Python in C++
======================================
When C++ calls Python functions, such as in a callback function or when
manipulating Python objects, and Python raises an ``Exception``, pybind11
converts the Python exception into a C++ exception of type
:class:`pybind11::error_already_set` whose payload contains a C++ string textual
summary and the actual Python exception. ``error_already_set`` is used to
propagate Python exception back to Python (or possibly, handle them in C++).
.. tabularcolumns:: |p{0.5\textwidth}|p{0.45\textwidth}|
+--------------------------------------+--------------------------------------+
| Exception raised in Python | Thrown as C++ exception type |
+======================================+======================================+
| Any Python ``Exception`` | :class:`pybind11::error_already_set` |
+--------------------------------------+--------------------------------------+
For example:
.. code-block:: cpp
try {
// open("missing.txt", "r")
auto file = py::module::import("io").attr("open")("missing.txt", "r");
auto text = file.attr("read")();
file.attr("close")();
} catch (py::error_already_set &e) {
if (e.matches(PyExc_FileNotFoundError)) {
py::print("missing.txt not found");
} else if (e.match(PyExc_PermissionError)) {
py::print("missing.txt found but not accessible");
} else {
throw;
}
}
Note that C++ to Python exception translation does not apply here, since that is
a method for translating C++ exceptions to Python, not vice versa. The error raised
from Python is always ``error_already_set``.
This example illustrates this behavior:
.. code-block:: cpp
try {
py::eval("raise ValueError('The Ring')");
} catch (py::value_error &boromir) {
// Boromir never gets the ring
assert(false);
} catch (py::error_already_set &frodo) {
// Frodo gets the ring
py::print("I will take the ring");
}
try {
// py::value_error is a request for pybind11 to raise a Python exception
throw py::value_error("The ball");
} catch (py::error_already_set &cat) {
// cat won't catch the ball since
// py::value_error is not a Python exception
assert(false);
} catch (py::value_error &dog) {
// dog will catch the ball
py::print("Run Spot run");
throw; // Throw it again (pybind11 will raise ValueError)
}
Handling errors from the Python C API
=====================================
Where possible, use :ref:`pybind11 wrappers <wrappers>` instead of calling
the Python C API directly. When calling the Python C API directly, in
addition to manually managing reference counts, one must follow the pybind11
error protocol, which is outlined here.
After calling the Python C API, if Python returns an error,
``throw py::error_already_set();``, which allows pybind11 to deal with the
exception and pass it back to the Python interpreter. This includes calls to
the error setting functions such as ``PyErr_SetString``.
.. code-block:: cpp
PyErr_SetString(PyExc_TypeError, "C API type error demo");
throw py::error_already_set();
// But it would be easier to simply...
throw py::type_error("pybind11 wrapper type error");
Alternately, to ignore the error, call `PyErr_Clear
<https://docs.python.org/3/c-api/exceptions.html#c.PyErr_Clear>`_.
Any Python error must be thrown or cleared, or Python/pybind11 will be left in
an invalid state.
.. _unraisable_exceptions:
Handling unraisable exceptions
@ -156,20 +247,24 @@ Handling unraisable exceptions
If a Python function invoked from a C++ destructor or any function marked
``noexcept(true)`` (collectively, "noexcept functions") throws an exception, there
is no way to propagate the exception, as such functions may not throw at
run-time.
is no way to propagate the exception, as such functions may not throw.
Should they throw or fail to catch any exceptions in their call graph,
the C++ runtime calls ``std::terminate()`` to abort immediately.
Neither Python nor C++ allow exceptions raised in a noexcept function to propagate. In
Python, an exception raised in a class's ``__del__`` method is logged as an
unraisable error. In Python 3.8+, a system hook is triggered and an auditing
event is logged. In C++, ``std::terminate()`` is called to abort immediately.
Similarly, Python exceptions raised in a class's ``__del__`` method do not
propagate, but are logged by Python as an unraisable error. In Python 3.8+, a
`system hook is triggered
<https://docs.python.org/3/library/sys.html#sys.unraisablehook>`_
and an auditing event is logged.
Any noexcept function should have a try-catch block that traps
class:`error_already_set` (or any other exception that can occur). Note that pybind11
wrappers around Python exceptions such as :class:`pybind11::value_error` are *not*
Python exceptions; they are C++ exceptions that pybind11 catches and converts to
Python exceptions. Noexcept functions cannot propagate these exceptions either.
You can convert them to Python exceptions and then discard as unraisable.
class:`error_already_set` (or any other exception that can occur). Note that
pybind11 wrappers around Python exceptions such as
:class:`pybind11::value_error` are *not* Python exceptions; they are C++
exceptions that pybind11 catches and converts to Python exceptions. Noexcept
functions cannot propagate these exceptions either. A useful approach is to
convert them to Python exceptions and then ``discard_as_unraisable`` as shown
below.
.. code-block:: cpp
@ -183,8 +278,6 @@ You can convert them to Python exceptions and then discard as unraisable.
eas.discard_as_unraisable(__func__);
} catch (const std::exception &e) {
// Log and discard C++ exceptions.
// (We cannot use discard_as_unraisable, since we have a generic C++
// exception, not an exception that originated from Python.)
third_party::log(e);
}
}

10
docs/advanced/pycpp/object.rst
View File

@ -1,6 +1,8 @@
Python types
############
.. _wrappers:
Available wrappers
==================
@ -168,3 +170,11 @@ Generalized unpacking according to PEP448_ is also supported:
Python functions from C++, including keywords arguments and unpacking.
.. _PEP448: https://www.python.org/dev/peps/pep-0448/
Handling exceptions
===================
Python exceptions from wrapper classes will be thrown as a ``py::error_already_set``.
See :ref:`Handling exceptions from Python in C++
<handling_python_exceptions_cpp>` for more information on handling exceptions
raised when calling C++ wrapper classes.