diff --git a/.gitignore b/.gitignore index 47e010ce2..5613b367d 100644 --- a/.gitignore +++ b/.gitignore @@ -38,3 +38,4 @@ sosize-*.txt pybind11Config*.cmake pybind11Targets.cmake /*env* +/.vscode diff --git a/docs/advanced/exceptions.rst b/docs/advanced/exceptions.rst index 72063108c..b7d36014a 100644 --- a/docs/advanced/exceptions.rst +++ b/docs/advanced/exceptions.rst @@ -1,18 +1,24 @@ Exceptions ########## -Built-in exception translation -============================== +Built-in C++ to Python exception translation +============================================ -When C++ code invoked from Python throws an ``std::exception``, it is -automatically converted into a Python ``Exception``. pybind11 defines multiple -special exception classes that will map to different types of Python -exceptions: +When Python calls C++ code through pybind11, pybind11 provides a C++ exception handler +that will trap C++ exceptions, translate them to the corresponding Python exception, +and raise them so that Python code can handle them. + +pybind11 defines translations for ``std::exception`` and its standard +subclasses, and several special exception classes that translate to specific +Python exceptions. Note that these are not actually Python exceptions, so they +cannot be examined using the Python C API. Instead, they are pure C++ objects +that pybind11 will translate the corresponding Python exception when they arrive +at its exception handler. .. tabularcolumns:: |p{0.5\textwidth}|p{0.45\textwidth}| +--------------------------------------+--------------------------------------+ -| C++ exception type | Python exception type | +| Exception thrown by C++ | Translated to Python exception type | +======================================+======================================+ | :class:`std::exception` | ``RuntimeError`` | +--------------------------------------+--------------------------------------+ @@ -46,22 +52,11 @@ exceptions: | | ``__setitem__`` in dict-like | | | objects, etc.) | +--------------------------------------+--------------------------------------+ -| :class:`pybind11::error_already_set` | Indicates that the Python exception | -| | flag has already been set via Python | -| | API calls from C++ code; this C++ | -| | exception is used to propagate such | -| | a Python exception back to Python. | -+--------------------------------------+--------------------------------------+ -When a Python function invoked from C++ throws an exception, pybind11 will convert -it into a C++ exception of type :class:`error_already_set` whose string payload -contains a textual summary. If you call the Python C-API directly, and it -returns an error, you should ``throw py::error_already_set();``, which allows -pybind11 to deal with the exception and pass it back to the Python interpreter. -(Another option is to call ``PyErr_Clear`` in the -`Python C-API `_ -to clear the error. The Python error must be thrown or cleared, or Python/pybind11 -will be left in an invalid state.) +Exception translation is not bidirectional. That is, *catching* the C++ +exceptions defined above above will not trap exceptions that originate from +Python. For that, catch :class:`pybind11::error_already_set`. See :ref:`below +` for further details. There is also a special exception :class:`cast_error` that is thrown by :func:`handle::call` when the input arguments cannot be converted to Python @@ -106,7 +101,6 @@ and use this in the associated exception translator (note: it is often useful to make this a static declaration when using it inside a lambda expression without requiring capturing). - The following example demonstrates this for a hypothetical exception classes ``MyCustomException`` and ``OtherException``: the first is translated to a custom python exception ``MyCustomError``, while the second is translated to a @@ -140,7 +134,7 @@ section. .. note:: - You must call either ``PyErr_SetString`` or a custom exception's call + Call either ``PyErr_SetString`` or a custom exception's call operator (``exc(string)``) for every exception caught in a custom exception translator. Failure to do so will cause Python to crash with ``SystemError: error return without exception set``. @@ -149,6 +143,103 @@ section. may be explicitly (re-)thrown to delegate it to the other, previously-declared existing exception translators. +.. _handling_python_exceptions_cpp: + +Handling exceptions from Python in C++ +====================================== + +When C++ calls Python functions, such as in a callback function or when +manipulating Python objects, and Python raises an ``Exception``, pybind11 +converts the Python exception into a C++ exception of type +:class:`pybind11::error_already_set` whose payload contains a C++ string textual +summary and the actual Python exception. ``error_already_set`` is used to +propagate Python exception back to Python (or possibly, handle them in C++). + +.. tabularcolumns:: |p{0.5\textwidth}|p{0.45\textwidth}| + ++--------------------------------------+--------------------------------------+ +| Exception raised in Python | Thrown as C++ exception type | ++======================================+======================================+ +| Any Python ``Exception`` | :class:`pybind11::error_already_set` | ++--------------------------------------+--------------------------------------+ + +For example: + +.. code-block:: cpp + + try { + // open("missing.txt", "r") + auto file = py::module::import("io").attr("open")("missing.txt", "r"); + auto text = file.attr("read")(); + file.attr("close")(); + } catch (py::error_already_set &e) { + if (e.matches(PyExc_FileNotFoundError)) { + py::print("missing.txt not found"); + } else if (e.match(PyExc_PermissionError)) { + py::print("missing.txt found but not accessible"); + } else { + throw; + } + } + +Note that C++ to Python exception translation does not apply here, since that is +a method for translating C++ exceptions to Python, not vice versa. The error raised +from Python is always ``error_already_set``. + +This example illustrates this behavior: + +.. code-block:: cpp + + try { + py::eval("raise ValueError('The Ring')"); + } catch (py::value_error &boromir) { + // Boromir never gets the ring + assert(false); + } catch (py::error_already_set &frodo) { + // Frodo gets the ring + py::print("I will take the ring"); + } + + try { + // py::value_error is a request for pybind11 to raise a Python exception + throw py::value_error("The ball"); + } catch (py::error_already_set &cat) { + // cat won't catch the ball since + // py::value_error is not a Python exception + assert(false); + } catch (py::value_error &dog) { + // dog will catch the ball + py::print("Run Spot run"); + throw; // Throw it again (pybind11 will raise ValueError) + } + +Handling errors from the Python C API +===================================== + +Where possible, use :ref:`pybind11 wrappers ` instead of calling +the Python C API directly. When calling the Python C API directly, in +addition to manually managing reference counts, one must follow the pybind11 +error protocol, which is outlined here. + +After calling the Python C API, if Python returns an error, +``throw py::error_already_set();``, which allows pybind11 to deal with the +exception and pass it back to the Python interpreter. This includes calls to +the error setting functions such as ``PyErr_SetString``. + +.. code-block:: cpp + + PyErr_SetString(PyExc_TypeError, "C API type error demo"); + throw py::error_already_set(); + + // But it would be easier to simply... + throw py::type_error("pybind11 wrapper type error"); + +Alternately, to ignore the error, call `PyErr_Clear +`_. + +Any Python error must be thrown or cleared, or Python/pybind11 will be left in +an invalid state. + .. _unraisable_exceptions: Handling unraisable exceptions @@ -156,20 +247,24 @@ Handling unraisable exceptions If a Python function invoked from a C++ destructor or any function marked ``noexcept(true)`` (collectively, "noexcept functions") throws an exception, there -is no way to propagate the exception, as such functions may not throw at -run-time. +is no way to propagate the exception, as such functions may not throw. +Should they throw or fail to catch any exceptions in their call graph, +the C++ runtime calls ``std::terminate()`` to abort immediately. -Neither Python nor C++ allow exceptions raised in a noexcept function to propagate. In -Python, an exception raised in a class's ``__del__`` method is logged as an -unraisable error. In Python 3.8+, a system hook is triggered and an auditing -event is logged. In C++, ``std::terminate()`` is called to abort immediately. +Similarly, Python exceptions raised in a class's ``__del__`` method do not +propagate, but are logged by Python as an unraisable error. In Python 3.8+, a +`system hook is triggered +`_ +and an auditing event is logged. Any noexcept function should have a try-catch block that traps -class:`error_already_set` (or any other exception that can occur). Note that pybind11 -wrappers around Python exceptions such as :class:`pybind11::value_error` are *not* -Python exceptions; they are C++ exceptions that pybind11 catches and converts to -Python exceptions. Noexcept functions cannot propagate these exceptions either. -You can convert them to Python exceptions and then discard as unraisable. +class:`error_already_set` (or any other exception that can occur). Note that +pybind11 wrappers around Python exceptions such as +:class:`pybind11::value_error` are *not* Python exceptions; they are C++ +exceptions that pybind11 catches and converts to Python exceptions. Noexcept +functions cannot propagate these exceptions either. A useful approach is to +convert them to Python exceptions and then ``discard_as_unraisable`` as shown +below. .. code-block:: cpp @@ -183,8 +278,6 @@ You can convert them to Python exceptions and then discard as unraisable. eas.discard_as_unraisable(__func__); } catch (const std::exception &e) { // Log and discard C++ exceptions. - // (We cannot use discard_as_unraisable, since we have a generic C++ - // exception, not an exception that originated from Python.) third_party::log(e); } } diff --git a/docs/advanced/pycpp/object.rst b/docs/advanced/pycpp/object.rst index 19a226a87..07525d0dc 100644 --- a/docs/advanced/pycpp/object.rst +++ b/docs/advanced/pycpp/object.rst @@ -1,6 +1,8 @@ Python types ############ +.. _wrappers: + Available wrappers ================== @@ -168,3 +170,11 @@ Generalized unpacking according to PEP448_ is also supported: Python functions from C++, including keywords arguments and unpacking. .. _PEP448: https://www.python.org/dev/peps/pep-0448/ + +Handling exceptions +=================== + +Python exceptions from wrapper classes will be thrown as a ``py::error_already_set``. +See :ref:`Handling exceptions from Python in C++ +` for more information on handling exceptions +raised when calling C++ wrapper classes.