pybind11/tests/test_exceptions.py

273 lines
8.7 KiB
Python
Raw Normal View History

import sys
import pytest
import env
Fix builtin exception handlers to work across modules The builtin exception handler currently doesn't work across modules under clang/libc++ for builtin pybind exceptions like `pybind11::error_already_set` or `pybind11::stop_iteration`: under RTLD_LOCAL module loading clang considers each module's exception classes distinct types. This then means that the base exception translator fails to catch the exceptions and the fall through to the generic `std::exception` handler, which completely breaks things like `stop_iteration`: only the `stop_iteration` of the first module loaded actually works properly; later modules raise a RuntimeError with no message when trying to invoke their iterators. For example, two modules defined like this exhibit the behaviour under clang++/libc++: z1.cpp: #include <pybind11/pybind11.h> #include <pybind11/stl_bind.h> namespace py = pybind11; PYBIND11_MODULE(z1, m) { py::bind_vector<std::vector<long>>(m, "IntVector"); } z2.cpp: #include <pybind11/pybind11.h> #include <pybind11/stl_bind.h> namespace py = pybind11; PYBIND11_MODULE(z2, m) { py::bind_vector<std::vector<double>>(m, "FloatVector"); } Python: import z1, z2 for i in z2.FloatVector(): pass results in: Traceback (most recent call last): File "zs.py", line 2, in <module> for i in z2.FloatVector(): RuntimeError This commit fixes the issue by adding a new exception translator each time the internals pointer is initialized from python builtins: this generally means the internals data was initialized by some other module. (The extra translator(s) are skipped under libstdc++).
2017-07-29 01:38:23 +00:00
import pybind11_cross_module_tests as cm
from pybind11_tests import exceptions as m
def test_std_exception(msg):
with pytest.raises(RuntimeError) as excinfo:
m.throw_std_exception()
assert msg(excinfo.value) == "This exception was intentionally thrown."
def test_error_already_set(msg):
with pytest.raises(RuntimeError) as excinfo:
m.throw_already_set(False)
assert msg(excinfo.value) == "Unknown internal error occurred"
with pytest.raises(ValueError) as excinfo:
m.throw_already_set(True)
assert msg(excinfo.value) == "foo"
def test_raise_from(msg):
with pytest.raises(ValueError) as excinfo:
m.raise_from()
assert msg(excinfo.value) == "outer"
assert msg(excinfo.value.__cause__) == "inner"
def test_raise_from_already_set(msg):
with pytest.raises(ValueError) as excinfo:
m.raise_from_already_set()
assert msg(excinfo.value) == "outer"
assert msg(excinfo.value.__cause__) == "inner"
Feature/local exception translator (#2650) * Create a module_internals struct Since we now have two things that are going to be module local, it felt correct to add a struct to manage them. * Add local exception translators These are added via the register_local_exception_translator function and are then applied before the global translators * Add unit tests to show the local exception translator works * Fix a bug in the unit test with the string value of KeyError * Fix a formatting issue * Rename registered_local_types_cpp() Rename it to get_registered_local_types_cpp() to disambiguate from the new member of module_internals * Add additional comments to new local exception code path * Add a register_local_exception function * Add additional unit tests for register_local_exception * Use get_local_internals like get_internals * Update documentation for new local exception feature * Add back a missing space * Clean-up some issues in the docs * Remove the code duplication when translating exceptions Separated out the exception processing into a standalone function in the details namespace. Clean-up some comments as per PR notes as well * Remove the code duplication in register_exception * Cleanup some formatting things caught by clang-format * Remove the templates from exception translators But I added a using declaration to alias the type. * Remove the extra local from local_internals variable names * Add an extra explanatory comment to local_internals * Fix a typo in the code
2021-07-21 12:22:18 +00:00
def test_cross_module_exceptions(msg):
Fix builtin exception handlers to work across modules The builtin exception handler currently doesn't work across modules under clang/libc++ for builtin pybind exceptions like `pybind11::error_already_set` or `pybind11::stop_iteration`: under RTLD_LOCAL module loading clang considers each module's exception classes distinct types. This then means that the base exception translator fails to catch the exceptions and the fall through to the generic `std::exception` handler, which completely breaks things like `stop_iteration`: only the `stop_iteration` of the first module loaded actually works properly; later modules raise a RuntimeError with no message when trying to invoke their iterators. For example, two modules defined like this exhibit the behaviour under clang++/libc++: z1.cpp: #include <pybind11/pybind11.h> #include <pybind11/stl_bind.h> namespace py = pybind11; PYBIND11_MODULE(z1, m) { py::bind_vector<std::vector<long>>(m, "IntVector"); } z2.cpp: #include <pybind11/pybind11.h> #include <pybind11/stl_bind.h> namespace py = pybind11; PYBIND11_MODULE(z2, m) { py::bind_vector<std::vector<double>>(m, "FloatVector"); } Python: import z1, z2 for i in z2.FloatVector(): pass results in: Traceback (most recent call last): File "zs.py", line 2, in <module> for i in z2.FloatVector(): RuntimeError This commit fixes the issue by adding a new exception translator each time the internals pointer is initialized from python builtins: this generally means the internals data was initialized by some other module. (The extra translator(s) are skipped under libstdc++).
2017-07-29 01:38:23 +00:00
with pytest.raises(RuntimeError) as excinfo:
cm.raise_runtime_error()
assert str(excinfo.value) == "My runtime error"
with pytest.raises(ValueError) as excinfo:
cm.raise_value_error()
assert str(excinfo.value) == "My value error"
with pytest.raises(ValueError) as excinfo:
cm.throw_pybind_value_error()
assert str(excinfo.value) == "pybind11 value error"
with pytest.raises(TypeError) as excinfo:
cm.throw_pybind_type_error()
assert str(excinfo.value) == "pybind11 type error"
with pytest.raises(StopIteration) as excinfo:
cm.throw_stop_iteration()
Feature/local exception translator (#2650) * Create a module_internals struct Since we now have two things that are going to be module local, it felt correct to add a struct to manage them. * Add local exception translators These are added via the register_local_exception_translator function and are then applied before the global translators * Add unit tests to show the local exception translator works * Fix a bug in the unit test with the string value of KeyError * Fix a formatting issue * Rename registered_local_types_cpp() Rename it to get_registered_local_types_cpp() to disambiguate from the new member of module_internals * Add additional comments to new local exception code path * Add a register_local_exception function * Add additional unit tests for register_local_exception * Use get_local_internals like get_internals * Update documentation for new local exception feature * Add back a missing space * Clean-up some issues in the docs * Remove the code duplication when translating exceptions Separated out the exception processing into a standalone function in the details namespace. Clean-up some comments as per PR notes as well * Remove the code duplication in register_exception * Cleanup some formatting things caught by clang-format * Remove the templates from exception translators But I added a using declaration to alias the type. * Remove the extra local from local_internals variable names * Add an extra explanatory comment to local_internals * Fix a typo in the code
2021-07-21 12:22:18 +00:00
with pytest.raises(cm.LocalSimpleException) as excinfo:
cm.throw_local_simple_error()
assert msg(excinfo.value) == "external mod"
with pytest.raises(KeyError) as excinfo:
cm.throw_local_error()
# KeyError is a repr of the key, so it has an extra set of quotes
assert str(excinfo.value) == "'just local'"
Fix builtin exception handlers to work across modules The builtin exception handler currently doesn't work across modules under clang/libc++ for builtin pybind exceptions like `pybind11::error_already_set` or `pybind11::stop_iteration`: under RTLD_LOCAL module loading clang considers each module's exception classes distinct types. This then means that the base exception translator fails to catch the exceptions and the fall through to the generic `std::exception` handler, which completely breaks things like `stop_iteration`: only the `stop_iteration` of the first module loaded actually works properly; later modules raise a RuntimeError with no message when trying to invoke their iterators. For example, two modules defined like this exhibit the behaviour under clang++/libc++: z1.cpp: #include <pybind11/pybind11.h> #include <pybind11/stl_bind.h> namespace py = pybind11; PYBIND11_MODULE(z1, m) { py::bind_vector<std::vector<long>>(m, "IntVector"); } z2.cpp: #include <pybind11/pybind11.h> #include <pybind11/stl_bind.h> namespace py = pybind11; PYBIND11_MODULE(z2, m) { py::bind_vector<std::vector<double>>(m, "FloatVector"); } Python: import z1, z2 for i in z2.FloatVector(): pass results in: Traceback (most recent call last): File "zs.py", line 2, in <module> for i in z2.FloatVector(): RuntimeError This commit fixes the issue by adding a new exception translator each time the internals pointer is initialized from python builtins: this generally means the internals data was initialized by some other module. (The extra translator(s) are skipped under libstdc++).
2017-07-29 01:38:23 +00:00
# TODO: FIXME
@pytest.mark.xfail(
"env.PYPY and env.MACOS",
raises=RuntimeError,
reason="Expected failure with PyPy and libc++ (Issue #2847 & PR #2999)",
)
def test_cross_module_exception_translator():
with pytest.raises(KeyError):
# translator registered in cross_module_tests
m.throw_should_be_translated_to_key_error()
def test_python_call_in_catch():
d = {}
assert m.python_call_in_destructor(d) is True
assert d["good"] is True
def ignore_pytest_unraisable_warning(f):
unraisable = "PytestUnraisableExceptionWarning"
if hasattr(pytest, unraisable): # Python >= 3.8 and pytest >= 6
dec = pytest.mark.filterwarnings("ignore::pytest.{}".format(unraisable))
return dec(f)
else:
return f
# TODO: find out why this fails on PyPy, https://foss.heptapod.net/pypy/pypy/-/issues/3583
@pytest.mark.xfail(env.PYPY, reason="Failure on PyPy 3.8 (7.3.7)", strict=False)
@ignore_pytest_unraisable_warning
def test_python_alreadyset_in_destructor(monkeypatch, capsys):
hooked = False
triggered = False
if hasattr(sys, "unraisablehook"): # Python 3.8+
hooked = True
# Don't take `sys.unraisablehook`, as that's overwritten by pytest
default_hook = sys.__unraisablehook__
def hook(unraisable_hook_args):
exc_type, exc_value, exc_tb, err_msg, obj = unraisable_hook_args
if obj == "already_set demo":
nonlocal triggered
triggered = True
default_hook(unraisable_hook_args)
return
# Use monkeypatch so pytest can apply and remove the patch as appropriate
monkeypatch.setattr(sys, "unraisablehook", hook)
assert m.python_alreadyset_in_destructor("already_set demo") is True
if hooked:
assert triggered is True
_, captured_stderr = capsys.readouterr()
assert captured_stderr.startswith("Exception ignored in: 'already_set demo'")
assert captured_stderr.rstrip().endswith("KeyError: 'bar'")
def test_exception_matches():
assert m.exception_matches()
assert m.exception_matches_base()
assert m.modulenotfound_exception_matches_base()
def test_custom(msg):
# Can we catch a MyException?
with pytest.raises(m.MyException) as excinfo:
m.throws1()
assert msg(excinfo.value) == "this error should go to a custom type"
# Can we translate to standard Python exceptions?
with pytest.raises(RuntimeError) as excinfo:
m.throws2()
assert msg(excinfo.value) == "this error should go to a standard Python exception"
# Can we handle unknown exceptions?
with pytest.raises(RuntimeError) as excinfo:
m.throws3()
assert msg(excinfo.value) == "Caught an unknown exception!"
# Can we delegate to another handler by rethrowing?
with pytest.raises(m.MyException) as excinfo:
m.throws4()
assert msg(excinfo.value) == "this error is rethrown"
# Can we fall-through to the default handler?
with pytest.raises(RuntimeError) as excinfo:
m.throws_logic_error()
assert (
msg(excinfo.value) == "this error should fall through to the standard handler"
)
# OverFlow error translation.
with pytest.raises(OverflowError) as excinfo:
m.throws_overflow_error()
# Can we handle a helper-declared exception?
with pytest.raises(m.MyException5) as excinfo:
m.throws5()
assert msg(excinfo.value) == "this is a helper-defined translated exception"
# Exception subclassing:
with pytest.raises(m.MyException5) as excinfo:
m.throws5_1()
assert msg(excinfo.value) == "MyException5 subclass"
assert isinstance(excinfo.value, m.MyException5_1)
with pytest.raises(m.MyException5_1) as excinfo:
m.throws5_1()
assert msg(excinfo.value) == "MyException5 subclass"
with pytest.raises(m.MyException5) as excinfo:
try:
m.throws5()
except m.MyException5_1:
raise RuntimeError("Exception error: caught child from parent")
assert msg(excinfo.value) == "this is a helper-defined translated exception"
Simplify error_already_set `error_already_set` is more complicated than it needs to be, partly because it manages reference counts itself rather than using `py::object`, and partly because it tries to do more exception clearing than is needed. This commit greatly simplifies it, and fixes #927. Using `py::object` instead of `PyObject *` means we can rely on implicit copy/move constructors. The current logic did both a `PyErr_Clear` on deletion *and* a `PyErr_Fetch` on creation. I can't see how the `PyErr_Clear` on deletion is ever useful: the `Fetch` on creation itself clears the error, so the only way doing a `PyErr_Clear` on deletion could do anything if is some *other* exception was raised while the `error_already_set` object was alive--but in that case, clearing some other exception seems wrong. (Code that is worried about an exception handler raising another exception would already catch a second `error_already_set` from exception code). The destructor itself called `clear()`, but `clear()` was a little bit more paranoid that needed: it called `restore()` to restore the currently captured error, but then immediately cleared it, using the `PyErr_Restore` to release the references. That's unnecessary: it's valid for us to release the references manually. This updates the code to simply release the references on the three objects (preserving the gil acquire). `clear()`, however, also had the side effect of clearing the current error, even if the current `error_already_set` didn't have a current error (e.g. because of a previous `restore()` or `clear()` call). I don't really see how clearing the error here can ever actually be useful: the only way the current error could be set is if you called `restore()` (in which case the current stored error-related members have already been released), or if some *other* code raised the error, in which case `clear()` on *this* object is clearing an error for which it shouldn't be responsible. Neither of those seem like intentional or desirable features, and manually requesting deletion of the stored references similarly seems pointless, so I've just made `clear()` an empty method and marked it deprecated. This also fixes a minor potential issue with the destruction: it is technically possible for `value` to be null (though this seems likely to be rare in practice); this updates the check to look at `type` which will always be non-null for a `Fetch`ed exception. This also adds error_already_set round-trip throw tests to the test suite.
2017-07-21 03:14:33 +00:00
def test_nested_throws(capture):
"""Tests nested (e.g. C++ -> Python -> C++) exception handling"""
def throw_myex():
raise m.MyException("nested error")
def throw_myex5():
raise m.MyException5("nested error 5")
# In the comments below, the exception is caught in the first step, thrown in the last step
# C++ -> Python
with capture:
m.try_catch(m.MyException5, throw_myex5)
assert str(capture).startswith("MyException5: nested error 5")
# Python -> C++ -> Python
with pytest.raises(m.MyException) as excinfo:
m.try_catch(m.MyException5, throw_myex)
assert str(excinfo.value) == "nested error"
def pycatch(exctype, f, *args):
try:
f(*args)
except m.MyException as e:
print(e)
# C++ -> Python -> C++ -> Python
with capture:
m.try_catch(
m.MyException5,
pycatch,
m.MyException,
m.try_catch,
m.MyException,
throw_myex5,
)
Simplify error_already_set `error_already_set` is more complicated than it needs to be, partly because it manages reference counts itself rather than using `py::object`, and partly because it tries to do more exception clearing than is needed. This commit greatly simplifies it, and fixes #927. Using `py::object` instead of `PyObject *` means we can rely on implicit copy/move constructors. The current logic did both a `PyErr_Clear` on deletion *and* a `PyErr_Fetch` on creation. I can't see how the `PyErr_Clear` on deletion is ever useful: the `Fetch` on creation itself clears the error, so the only way doing a `PyErr_Clear` on deletion could do anything if is some *other* exception was raised while the `error_already_set` object was alive--but in that case, clearing some other exception seems wrong. (Code that is worried about an exception handler raising another exception would already catch a second `error_already_set` from exception code). The destructor itself called `clear()`, but `clear()` was a little bit more paranoid that needed: it called `restore()` to restore the currently captured error, but then immediately cleared it, using the `PyErr_Restore` to release the references. That's unnecessary: it's valid for us to release the references manually. This updates the code to simply release the references on the three objects (preserving the gil acquire). `clear()`, however, also had the side effect of clearing the current error, even if the current `error_already_set` didn't have a current error (e.g. because of a previous `restore()` or `clear()` call). I don't really see how clearing the error here can ever actually be useful: the only way the current error could be set is if you called `restore()` (in which case the current stored error-related members have already been released), or if some *other* code raised the error, in which case `clear()` on *this* object is clearing an error for which it shouldn't be responsible. Neither of those seem like intentional or desirable features, and manually requesting deletion of the stored references similarly seems pointless, so I've just made `clear()` an empty method and marked it deprecated. This also fixes a minor potential issue with the destruction: it is technically possible for `value` to be null (though this seems likely to be rare in practice); this updates the check to look at `type` which will always be non-null for a `Fetch`ed exception. This also adds error_already_set round-trip throw tests to the test suite.
2017-07-21 03:14:33 +00:00
assert str(capture).startswith("MyException5: nested error 5")
# C++ -> Python -> C++
with capture:
m.try_catch(m.MyException, pycatch, m.MyException5, m.throws4)
assert capture == "this error is rethrown"
# Python -> C++ -> Python -> C++
with pytest.raises(m.MyException5) as excinfo:
m.try_catch(m.MyException, pycatch, m.MyException, m.throws5)
assert str(excinfo.value) == "this is a helper-defined translated exception"
def test_throw_nested_exception():
with pytest.raises(RuntimeError) as excinfo:
m.throw_nested_exception()
assert str(excinfo.value) == "Outer Exception"
assert str(excinfo.value.__cause__) == "Inner Exception"
# This can often happen if you wrap a pybind11 class in a Python wrapper
def test_invalid_repr():
class MyRepr:
def __repr__(self):
raise AttributeError("Example error")
with pytest.raises(TypeError):
m.simple_bool_passthrough(MyRepr())
Feature/local exception translator (#2650) * Create a module_internals struct Since we now have two things that are going to be module local, it felt correct to add a struct to manage them. * Add local exception translators These are added via the register_local_exception_translator function and are then applied before the global translators * Add unit tests to show the local exception translator works * Fix a bug in the unit test with the string value of KeyError * Fix a formatting issue * Rename registered_local_types_cpp() Rename it to get_registered_local_types_cpp() to disambiguate from the new member of module_internals * Add additional comments to new local exception code path * Add a register_local_exception function * Add additional unit tests for register_local_exception * Use get_local_internals like get_internals * Update documentation for new local exception feature * Add back a missing space * Clean-up some issues in the docs * Remove the code duplication when translating exceptions Separated out the exception processing into a standalone function in the details namespace. Clean-up some comments as per PR notes as well * Remove the code duplication in register_exception * Cleanup some formatting things caught by clang-format * Remove the templates from exception translators But I added a using declaration to alias the type. * Remove the extra local from local_internals variable names * Add an extra explanatory comment to local_internals * Fix a typo in the code
2021-07-21 12:22:18 +00:00
def test_local_translator(msg):
"""Tests that a local translator works and that the local translator from
the cross module is not applied"""
with pytest.raises(RuntimeError) as excinfo:
m.throws6()
assert msg(excinfo.value) == "MyException6 only handled in this module"
with pytest.raises(RuntimeError) as excinfo:
m.throws_local_error()
assert not isinstance(excinfo.value, KeyError)
assert msg(excinfo.value) == "never caught"
with pytest.raises(Exception) as excinfo:
m.throws_local_simple_error()
assert not isinstance(excinfo.value, cm.LocalSimpleException)
assert msg(excinfo.value) == "this mod"