mirror of
https://github.com/pybind/pybind11.git
synced 2024-11-25 06:35:12 +00:00
error_already_set::what() is now constructed lazily (#1895)
* error_already_set::what() is now constructed lazily Prior to this commit throwing error_already_set was expensive due to the eager construction of the error string (which required traversing the Python stack). See #1853 for more context and an alternative take on the issue. Note that error_already_set no longer inherits from std::runtime_error because the latter has no default constructor. * Do not attempt to normalize if no exception occurred This is not supported on PyPy-2.7 5.8.0. * Extract exception name via tp_name This is faster than dynamically looking up __name__ via GetAttrString. Note though that the runtime of the code throwing an error_already_set will be dominated by stack unwinding so the improvement will not be noticeable. Before: 396 ns ± 0.913 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each) After: 277 ns ± 0.549 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each) Benchmark: const std::string foo() { PyErr_SetString(PyExc_KeyError, ""); const std::string &s = py::detail::error_string(); PyErr_Clear(); return s; } PYBIND11_MODULE(foo, m) { m.def("foo", &::foo); } * Reverted error_already_set to subclass std::runtime_error * Revert "Extract exception name via tp_name" The implementation of __name__ is slightly more complex than that. It handles the module name prefix, and heap-allocated types. We could port it to pybind11 later on but for now it seems like an overkill. This reverts commitf1435c7e6b. * Cosmit following @YannickJadoul's comments Note that detail::error_string() no longer calls PyException_SetTraceback as it is unncessary for pretty-printing the exception. * Fixed PyPy build * Moved normalization to error_already_set ctor * Fix merge bugs * Fix more merge errors * Improve formatting * Improve error message in rare case * Revert back if statements * Fix clang-tidy * Try removing mutable * Does build_mode release fix it * Set to Debug to expose segfault * Fix remove set error string * Do not run error_string() more than once * Trying setting the tracebackk to the value * guard if m_type is null * Try to debug PGI * One last try for PGI * Does reverting this fix PyPy * Reviewer suggestions * Remove unnecessary initialization * Add noexcept move and explicit fail throw * Optimize error_string creation * Fix typo * Revert noexcept * Fix merge conflict error * Abuse assignment operator * Revert operator abuse * See if we still need debug * Remove unnecessary mutable * Report "FATAL failure building pybind11::error_already_set error_string" and terminate process. * Try specifying noexcept again * Try explicit ctor * default ctor is noexcept too * Apply reviewer suggestions, simplify code, and make helper method private * Remove unnecessary include * Clang-Tidy fix * detail::obj_class_name(), fprintf with [STDERR], [STDOUT] tags, polish comments * consistently check m_lazy_what.empty() also in production builds * Make a comment slightly less ambiguous. * Bug fix: Remove `what();` from `restore()`. It sure would need to be guarded by `if (m_type)`, otherwise `what()` fails and masks that no error was set (see update unit test). But since `error_already_set` is copyable, there is no point in releasing m_type, m_value, m_trace, therefore we can just as well avoid the runtime overhead of force-building `m_lazy_what`, it may never be used. * Replace extremely opaque (unhelpful) error message with a truthful reflection of what we know. * Fix clang-tidy error [performance-move-constructor-init]. * Make expected error message less specific. * Various changes. * bug fix: error_string(PyObject **, ...) * Putting back the two eager PyErr_NormalizeException() calls. * Change error_already_set() to call pybind11_fail() if the Python error indicator not set. The net result is that a std::runtime_error is thrown instead of error_already_set, but all tests pass as is. * Remove mutable (fixes oversight in the previous commit). * Normalize the exception only locally in error_string(). Python 3.6 & 3.7 test failures expected. This is meant for benchmarking, to determine if it is worth the trouble looking into the failures. * clang-tidy: use auto * Use `gil_scoped_acquire_local` in `error_already_set` destructor. See long comment. * For Python < 3.8: `PyErr_NormalizeException` before `PyErr_WriteUnraisable` * Go back to replacing the held Python exception with then normalized exception, if & when needed. Consistently document the side-effect. * Slightly rewording comment. (There were also other failures.) * Add 1-line comment for obj_class_name() * Benchmark code, with results in this commit message. function #calls test time [s] μs / call master pure_unwind 729540 1.061 14.539876 err_set_unwind_err_clear 681476 1.040 15.260282 err_set_error_already_set 508038 1.049 20.640525 error_already_set_restore 555578 1.052 18.933288 pr1895_original_foo 244113 1.050 43.018168 PR / master PR #1895 pure_unwind 736981 1.054 14.295685 98.32% err_set_unwind_err_clear 685820 1.045 15.237399 99.85% err_set_error_already_set 661374 1.046 15.811879 76.61% error_already_set_restore 669881 1.048 15.645176 82.63% pr1895_original_foo 318243 1.059 33.290806 77.39% master @ commitad146b2a18Running tests in directory "/usr/local/google/home/rwgk/forked/pybind11/tests": ============================= test session starts ============================== platform linux -- Python 3.9.10, pytest-6.2.3, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3 cachedir: .pytest_cache rootdir: /usr/local/google/home/rwgk/forked/pybind11/tests, configfile: pytest.ini collecting ... collected 5 items test_perf_error_already_set.py::test_perf[pure_unwind] PERF pure_unwind,729540,1.061,14.539876 PASSED test_perf_error_already_set.py::test_perf[err_set_unwind_err_clear] PERF err_set_unwind_err_clear,681476,1.040,15.260282 PASSED test_perf_error_already_set.py::test_perf[err_set_error_already_set] PERF err_set_error_already_set,508038,1.049,20.640525 PASSED test_perf_error_already_set.py::test_perf[error_already_set_restore] PERF error_already_set_restore,555578,1.052,18.933288 PASSED test_perf_error_already_set.py::test_perf[pr1895_original_foo] PERF pr1895_original_foo,244113,1.050,43.018168 PASSED ============================== 5 passed in 12.38s ============================== pr1895 @ commit8dff51d12eRunning tests in directory "/usr/local/google/home/rwgk/forked/pybind11/tests": ============================= test session starts ============================== platform linux -- Python 3.9.10, pytest-6.2.3, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3 cachedir: .pytest_cache rootdir: /usr/local/google/home/rwgk/forked/pybind11/tests, configfile: pytest.ini collecting ... collected 5 items test_perf_error_already_set.py::test_perf[pure_unwind] PERF pure_unwind,736981,1.054,14.295685 PASSED test_perf_error_already_set.py::test_perf[err_set_unwind_err_clear] PERF err_set_unwind_err_clear,685820,1.045,15.237399 PASSED test_perf_error_already_set.py::test_perf[err_set_error_already_set] PERF err_set_error_already_set,661374,1.046,15.811879 PASSED test_perf_error_already_set.py::test_perf[error_already_set_restore] PERF error_already_set_restore,669881,1.048,15.645176 PASSED test_perf_error_already_set.py::test_perf[pr1895_original_foo] PERF pr1895_original_foo,318243,1.059,33.290806 PASSED ============================== 5 passed in 12.40s ============================== clang++ -o pybind11/tests/test_perf_error_already_set.os -c -std=c++17 -fPIC -fvisibility=hidden -Os -flto -Wall -Wextra -Wconversion -Wcast-qual -Wdeprecated -Wnon-virtual-dtor -Wunused-result -isystem /usr/include/python3.9 -isystem /usr/include/eigen3 -DPYBIND11_STRICT_ASSERTS_CLASS_HOLDER_VS_TYPE_CASTER_MIX -DPYBIND11_TEST_BOOST -Ipybind11/include -I/usr/local/google/home/rwgk/forked/pybind11/include -I/usr/local/google/home/rwgk/clone/pybind11/include /usr/local/google/home/rwgk/forked/pybind11/tests/test_perf_error_already_set.cpp clang++ -o lib/pybind11_tests.so -shared -fPIC -Os -flto -shared ... Debian clang version 13.0.1-3+build2 Target: x86_64-pc-linux-gnu Thread model: posix * Changing call_repetitions_target_elapsed_secs to 0.1 for regular unit testing. * Adding in `recursion_depth` * Optimized ctor * Fix silly bug in recurse_first_then_call() * Add tests that have equivalent PyErr_Fetch(), PyErr_Restore() but no try-catch. * Add call_error_string to tests. Sample only recursion_depth 0, 100. * Show lazy-what speed-up in percent. * Include real_work in benchmarks. * Replace all PyErr_SetString() with generate_python_exception_with_traceback() * Better organization of test loops. * Add test_error_already_set_copy_move * Fix bug in newly added test (discovered by clang-tidy): actually use move ctor * MSVC detects the unreachable return * change test_perf_error_already_set.py back to quick mode * Inherit from std::exception (instead of std::runtime_error, which does not make sense anymore with the lazy what) * Special handling under Windows. * print with leading newline * Removing test_perf_error_already_set (copies are under7765113fbb). * Avoid gil and scope overhead if there is nothing to release. * Restore default move ctor. "member function" instead of "function" (note that "method" is Python terminology). * Delete error_already_set copy ctor. * Make restore() non-const again to resolve clang-tidy failure (still experimenting). * Bring back error_already_set copy ctor, to see if that resolves the 4 MSVC test failures. * Add noexcept to error_already_set copy & move ctors (as suggested by @skylion007 IIUC). * Trying one-by-one noexcept copy ctor for old compilers. * Add back test covering copy ctor. Add another simple test that exercises the copy ctor. * Exclude more older compilers from using the noexcept = default ctors. (The tests in the previous commit exposed that those are broken.) * Factor out & reuse gil_scoped_acquire_local as gil_scoped_acquire_simple * Guard gil_scoped_acquire_simple by _Py_IsFinalizing() check. * what() GIL safety * clang-tidy & Python 3.6 fixes * Use `gil_scoped_acquire` in dtor, copy ctor, `what()`. Remove `_Py_IsFinalizing()` checks (they are racy: https://github.com/python/cpython/pull/28525). * Remove error_scope from copy ctor. * Add `error_scope` to `get_internals()`, to cover the situation that `get_internals()` is called from the `error_already_set` dtor while a new Python error is in flight already. Also backing out `gil_scoped_acquire_simple` change. * Add `FlakyException` tests with failure triggers in `__init__` and `__str__` THIS IS STILL A WORK IN PROGRESS. This commit is only an important resting point. This commit is a first attempt at addressing the observation that `PyErr_NormalizeException()` completely replaces the original exception if `__init__` fails. This can be very confusing even in small applications, and extremely confusing in large ones. * Tweaks to resolve Py 3.6 and PyPy CI failures. * Normalize Python exception immediately in error_already_set ctor. For background see: https://github.com/pybind/pybind11/pull/1895#issuecomment-1135304081 * Fix oversights based on CI failures (copy & move ctor initialization). * Move @pytest.mark.xfail("env.PYPY") after @pytest.mark.parametrize(...) * Use @pytest.mark.skipif (xfail does not work for segfaults, of course). * Remove unused obj_class_name_or() function (it was added only under this PR). * Remove already obsolete C++ comments and code that were added only under this PR. * Slightly better (newly added) comments. * Factor out detail::error_fetch_and_normalize. Preparation for producing identical results from error_already_set::what() and detail::error_string(). Note that this is a very conservative refactoring. It would be much better to first move detail::error_string into detail/error_string.h * Copy most of error_string() code to new error_fetch_and_normalize::complete_lazy_error_string() * Remove all error_string() code from detail/type_caster_base.h. Note that this commit includes a subtle bug fix: previously error_string() restored the Python error, which will upset pybind11_fail(). This never was a problem in practice because the two PyType_Ready() calls in detail/class.h do not usually fail. * Return const std::string& instead of const char * and move error_string() to pytypes.h * Remove gil_scope_acquire from error_fetch_and_normalize, add back to error_already_set * Better handling of FlakyException __str__ failure. * Move error_fetch_and_normalize::complete_lazy_error_string() implementation from pybind11.h to pytypes.h * Add error_fetch_and_normalize::release_py_object_references() and use from error_already_set dtor. * Use shared_ptr for m_fetched_error => 1. non-racy, copy ctor that does not need the GIL; 2. enables guard against duplicate restore() calls. * Add comments. * Trivial renaming of a newly introduced member function. * Workaround for PyPy * Bug fix (oversight). Only valgrind got this one. * Use shared_ptr custom deleter for m_fetched_error in error_already_set. This enables removing the dtor, copy ctor, move ctor completely. * Further small simplification. With the GIL held, simply deleting the raw_ptr takes care of everything. * IWYU cleanup ``` iwyu version: include-what-you-use 0.17 based on Debian clang version 13.0.1-3+build2 ``` Command used: ``` iwyu -c -std=c++17 -DPYBIND11_TEST_BOOST -Iinclude/pybind11 -I/usr/include/python3.9 -I/usr/include/eigen3 include/pybind11/pytypes.cpp ``` pytypes.cpp is a temporary file: `#include "pytypes.h"` The raw output is very long and noisy. I decided to use `#include <cstddef>` instead of `#include <cstdio>` for `std::size_t` (iwyu sticks to the manual choice). I ignored all iwyu suggestions that are indirectly covered by `#include <Python.h>`. I manually verified that all added includes are actually needed. Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com> Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
This commit is contained in:
Sergei Lebedev
GitHub
parent
58802de41b
commit
a05bc3d235
@ -455,6 +455,8 @@ extern "C" inline void pybind11_object_dealloc(PyObject *self) {
|
||||
#endif
|
||||
}
|
||||
|
||||
std::string error_string();
|
||||
|
||||
/** Create the type which can be used as a common base for all classes. This is
|
||||
needed in order to satisfy Python's requirements for multiple inheritance.
|
||||
Return value: New reference. */
|
||||
@ -490,7 +492,7 @@ inline PyObject *make_object_base_type(PyTypeObject *metaclass) {
|
||||
type->tp_weaklistoffset = offsetof(instance, weakrefs);
|
||||
|
||||
if (PyType_Ready(type) < 0) {
|
||||
pybind11_fail("PyType_Ready failed in make_object_base_type():" + error_string());
|
||||
pybind11_fail("PyType_Ready failed in make_object_base_type(): " + error_string());
|
||||
}
|
||||
|
||||
setattr((PyObject *) type, "__module__", str("pybind11_builtins"));
|
||||
@ -707,7 +709,7 @@ inline PyObject *make_new_python_type(const type_record &rec) {
|
||||
}
|
||||
|
||||
if (PyType_Ready(type) < 0) {
|
||||
pybind11_fail(std::string(rec.name) + ": PyType_Ready failed (" + error_string() + ")!");
|
||||
pybind11_fail(std::string(rec.name) + ": PyType_Ready failed: " + error_string());
|
||||
}
|
||||
|
||||
assert(!rec.dynamic_attr || PyType_HasFeature(type, Py_TPFLAGS_HAVE_GC));
|
||||
|
||||
@ -470,73 +470,6 @@ PYBIND11_NOINLINE bool isinstance_generic(handle obj, const std::type_info &tp)
|
||||
return isinstance(obj, type);
|
||||
}
|
||||
|
||||
PYBIND11_NOINLINE std::string error_string(const char *called) {
|
||||
error_scope scope; // Fetch error state (will be restored when this function returns).
|
||||
if (scope.type == nullptr) {
|
||||
if (called == nullptr) {
|
||||
called = "pybind11::detail::error_string()";
|
||||
}
|
||||
pybind11_fail("Internal error: " + std::string(called)
|
||||
+ " called while Python error indicator not set.");
|
||||
}
|
||||
|
||||
PyErr_NormalizeException(&scope.type, &scope.value, &scope.trace);
|
||||
if (scope.trace != nullptr) {
|
||||
PyException_SetTraceback(scope.value, scope.trace);
|
||||
}
|
||||
|
||||
std::string errorString;
|
||||
if (scope.type) {
|
||||
errorString += handle(scope.type).attr("__name__").cast<std::string>();
|
||||
errorString += ": ";
|
||||
}
|
||||
if (scope.value) {
|
||||
errorString += (std::string) str(scope.value);
|
||||
}
|
||||
|
||||
#if !defined(PYPY_VERSION)
|
||||
if (scope.trace) {
|
||||
auto *trace = (PyTracebackObject *) scope.trace;
|
||||
|
||||
/* Get the deepest trace possible */
|
||||
while (trace->tb_next) {
|
||||
trace = trace->tb_next;
|
||||
}
|
||||
|
||||
PyFrameObject *frame = trace->tb_frame;
|
||||
Py_XINCREF(frame);
|
||||
errorString += "\n\nAt:\n";
|
||||
while (frame) {
|
||||
# if PY_VERSION_HEX >= 0x030900B1
|
||||
PyCodeObject *f_code = PyFrame_GetCode(frame);
|
||||
# else
|
||||
PyCodeObject *f_code = frame->f_code;
|
||||
Py_INCREF(f_code);
|
||||
# endif
|
||||
int lineno = PyFrame_GetLineNumber(frame);
|
||||
errorString += " ";
|
||||
errorString += handle(f_code->co_filename).cast<std::string>();
|
||||
errorString += '(';
|
||||
errorString += std::to_string(lineno);
|
||||
errorString += "): ";
|
||||
errorString += handle(f_code->co_name).cast<std::string>();
|
||||
errorString += '\n';
|
||||
Py_DECREF(f_code);
|
||||
# if PY_VERSION_HEX >= 0x030900B1
|
||||
auto *b_frame = PyFrame_GetBack(frame);
|
||||
# else
|
||||
auto *b_frame = frame->f_back;
|
||||
Py_XINCREF(b_frame);
|
||||
# endif
|
||||
Py_DECREF(frame);
|
||||
frame = b_frame;
|
||||
}
|
||||
}
|
||||
#endif
|
||||
|
||||
return errorString;
|
||||
}
|
||||
|
||||
PYBIND11_NOINLINE handle get_object_handle(const void *ptr, const detail::type_info *type) {
|
||||
auto &instances = get_internals().registered_instances;
|
||||
auto range = instances.equal_range(ptr);
|
||||
|
||||
@ -2625,17 +2625,21 @@ void print(Args &&...args) {
|
||||
detail::print(c.args(), c.kwargs());
|
||||
}
|
||||
|
||||
error_already_set::~error_already_set() {
|
||||
if (m_type) {
|
||||
gil_scoped_acquire gil;
|
||||
error_scope scope;
|
||||
m_type.release().dec_ref();
|
||||
m_value.release().dec_ref();
|
||||
m_trace.release().dec_ref();
|
||||
}
|
||||
inline void
|
||||
error_already_set::m_fetched_error_deleter(detail::error_fetch_and_normalize *raw_ptr) {
|
||||
gil_scoped_acquire gil;
|
||||
error_scope scope;
|
||||
delete raw_ptr;
|
||||
}
|
||||
|
||||
inline const char *error_already_set::what() const noexcept {
|
||||
gil_scoped_acquire gil;
|
||||
error_scope scope;
|
||||
return m_fetched_error->error_string().c_str();
|
||||
}
|
||||
|
||||
PYBIND11_NAMESPACE_BEGIN(detail)
|
||||
|
||||
inline function
|
||||
get_type_override(const void *this_ptr, const type_info *this_type, const char *name) {
|
||||
handle self = get_object_handle(this_ptr, this_type);
|
||||
|
||||
@ -12,7 +12,15 @@
|
||||
#include "detail/common.h"
|
||||
#include "buffer_info.h"
|
||||
|
||||
#include <assert.h>
|
||||
#include <cstddef>
|
||||
#include <exception>
|
||||
#include <frameobject.h>
|
||||
#include <iterator>
|
||||
#include <memory>
|
||||
#include <string>
|
||||
#include <type_traits>
|
||||
#include <typeinfo>
|
||||
#include <utility>
|
||||
|
||||
#if defined(PYBIND11_HAS_OPTIONAL)
|
||||
@ -383,7 +391,175 @@ T reinterpret_steal(handle h) {
|
||||
}
|
||||
|
||||
PYBIND11_NAMESPACE_BEGIN(detail)
|
||||
std::string error_string(const char *called = nullptr);
|
||||
|
||||
// Equivalent to obj.__class__.__name__ (or obj.__name__ if obj is a class).
|
||||
inline const char *obj_class_name(PyObject *obj) {
|
||||
if (Py_TYPE(obj) == &PyType_Type) {
|
||||
return reinterpret_cast<PyTypeObject *>(obj)->tp_name;
|
||||
}
|
||||
return Py_TYPE(obj)->tp_name;
|
||||
}
|
||||
|
||||
std::string error_string();
|
||||
|
||||
struct error_fetch_and_normalize {
|
||||
// Immediate normalization is long-established behavior (starting with
|
||||
// https://github.com/pybind/pybind11/commit/135ba8deafb8bf64a15b24d1513899eb600e2011
|
||||
// from Sep 2016) and safest. Normalization could be deferred, but this could mask
|
||||
// errors elsewhere, the performance gain is very minor in typical situations
|
||||
// (usually the dominant bottleneck is EH unwinding), and the implementation here
|
||||
// would be more complex.
|
||||
explicit error_fetch_and_normalize(const char *called) {
|
||||
PyErr_Fetch(&m_type.ptr(), &m_value.ptr(), &m_trace.ptr());
|
||||
if (!m_type) {
|
||||
pybind11_fail("Internal error: " + std::string(called)
|
||||
+ " called while "
|
||||
"Python error indicator not set.");
|
||||
}
|
||||
const char *exc_type_name_orig = detail::obj_class_name(m_type.ptr());
|
||||
if (exc_type_name_orig == nullptr) {
|
||||
pybind11_fail("Internal error: " + std::string(called)
|
||||
+ " failed to obtain the name "
|
||||
"of the original active exception type.");
|
||||
}
|
||||
m_lazy_error_string = exc_type_name_orig;
|
||||
// PyErr_NormalizeException() may change the exception type if there are cascading
|
||||
// failures. This can potentially be extremely confusing.
|
||||
PyErr_NormalizeException(&m_type.ptr(), &m_value.ptr(), &m_trace.ptr());
|
||||
if (m_type.ptr() == nullptr) {
|
||||
pybind11_fail("Internal error: " + std::string(called)
|
||||
+ " failed to normalize the "
|
||||
"active exception.");
|
||||
}
|
||||
const char *exc_type_name_norm = detail::obj_class_name(m_type.ptr());
|
||||
if (exc_type_name_orig == nullptr) {
|
||||
pybind11_fail("Internal error: " + std::string(called)
|
||||
+ " failed to obtain the name "
|
||||
"of the normalized active exception type.");
|
||||
}
|
||||
if (exc_type_name_norm != m_lazy_error_string) {
|
||||
std::string msg = std::string(called)
|
||||
+ ": MISMATCH of original and normalized "
|
||||
"active exception types: ";
|
||||
msg += "ORIGINAL ";
|
||||
msg += m_lazy_error_string;
|
||||
msg += " REPLACED BY ";
|
||||
msg += exc_type_name_norm;
|
||||
msg += ": " + format_value_and_trace();
|
||||
pybind11_fail(msg);
|
||||
}
|
||||
}
|
||||
|
||||
error_fetch_and_normalize(const error_fetch_and_normalize &) = delete;
|
||||
error_fetch_and_normalize(error_fetch_and_normalize &&) = delete;
|
||||
|
||||
std::string format_value_and_trace() const {
|
||||
std::string result;
|
||||
std::string message_error_string;
|
||||
if (m_value) {
|
||||
auto value_str = reinterpret_steal<object>(PyObject_Str(m_value.ptr()));
|
||||
if (!value_str) {
|
||||
message_error_string = detail::error_string();
|
||||
result = "<MESSAGE UNAVAILABLE DUE TO ANOTHER EXCEPTION>";
|
||||
} else {
|
||||
result = value_str.cast<std::string>();
|
||||
}
|
||||
} else {
|
||||
result = "<MESSAGE UNAVAILABLE>";
|
||||
}
|
||||
if (result.empty()) {
|
||||
result = "<EMPTY MESSAGE>";
|
||||
}
|
||||
|
||||
bool have_trace = false;
|
||||
if (m_trace) {
|
||||
#if !defined(PYPY_VERSION)
|
||||
auto *tb = reinterpret_cast<PyTracebackObject *>(m_trace.ptr());
|
||||
|
||||
// Get the deepest trace possible.
|
||||
while (tb->tb_next) {
|
||||
tb = tb->tb_next;
|
||||
}
|
||||
|
||||
PyFrameObject *frame = tb->tb_frame;
|
||||
Py_XINCREF(frame);
|
||||
result += "\n\nAt:\n";
|
||||
while (frame) {
|
||||
# if PY_VERSION_HEX >= 0x030900B1
|
||||
PyCodeObject *f_code = PyFrame_GetCode(frame);
|
||||
# else
|
||||
PyCodeObject *f_code = frame->f_code;
|
||||
Py_INCREF(f_code);
|
||||
# endif
|
||||
int lineno = PyFrame_GetLineNumber(frame);
|
||||
result += " ";
|
||||
result += handle(f_code->co_filename).cast<std::string>();
|
||||
result += '(';
|
||||
result += std::to_string(lineno);
|
||||
result += "): ";
|
||||
result += handle(f_code->co_name).cast<std::string>();
|
||||
result += '\n';
|
||||
Py_DECREF(f_code);
|
||||
# if PY_VERSION_HEX >= 0x030900B1
|
||||
auto *b_frame = PyFrame_GetBack(frame);
|
||||
# else
|
||||
auto *b_frame = frame->f_back;
|
||||
Py_XINCREF(b_frame);
|
||||
# endif
|
||||
Py_DECREF(frame);
|
||||
frame = b_frame;
|
||||
}
|
||||
|
||||
have_trace = true;
|
||||
#endif //! defined(PYPY_VERSION)
|
||||
}
|
||||
|
||||
if (!message_error_string.empty()) {
|
||||
if (!have_trace) {
|
||||
result += '\n';
|
||||
}
|
||||
result += "\nMESSAGE UNAVAILABLE DUE TO EXCEPTION: " + message_error_string;
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
std::string const &error_string() const {
|
||||
if (!m_lazy_error_string_completed) {
|
||||
m_lazy_error_string += ": " + format_value_and_trace();
|
||||
m_lazy_error_string_completed = true;
|
||||
}
|
||||
return m_lazy_error_string;
|
||||
}
|
||||
|
||||
void restore() {
|
||||
if (m_restore_called) {
|
||||
pybind11_fail("Internal error: pybind11::detail::error_fetch_and_normalize::restore() "
|
||||
"called a second time. ORIGINAL ERROR: "
|
||||
+ error_string());
|
||||
}
|
||||
PyErr_Restore(m_type.inc_ref().ptr(), m_value.inc_ref().ptr(), m_trace.inc_ref().ptr());
|
||||
m_restore_called = true;
|
||||
}
|
||||
|
||||
bool matches(handle exc) const {
|
||||
return (PyErr_GivenExceptionMatches(m_type.ptr(), exc.ptr()) != 0);
|
||||
}
|
||||
|
||||
// Not protecting these for simplicity.
|
||||
object m_type, m_value, m_trace;
|
||||
|
||||
private:
|
||||
// Only protecting invariants.
|
||||
mutable std::string m_lazy_error_string;
|
||||
mutable bool m_lazy_error_string_completed = false;
|
||||
mutable bool m_restore_called = false;
|
||||
};
|
||||
|
||||
inline std::string error_string() {
|
||||
return error_fetch_and_normalize("pybind11::detail::error_string").error_string();
|
||||
}
|
||||
|
||||
PYBIND11_NAMESPACE_END(detail)
|
||||
|
||||
#if defined(_MSC_VER)
|
||||
@ -396,39 +572,30 @@ PYBIND11_NAMESPACE_END(detail)
|
||||
/// thrown to propagate python-side errors back through C++ which can either be caught manually or
|
||||
/// else falls back to the function dispatcher (which then raises the captured error back to
|
||||
/// python).
|
||||
class PYBIND11_EXPORT_EXCEPTION error_already_set : public std::runtime_error {
|
||||
class PYBIND11_EXPORT_EXCEPTION error_already_set : public std::exception {
|
||||
public:
|
||||
/// Constructs a new exception from the current Python error indicator. The current
|
||||
/// Python error indicator will be cleared.
|
||||
error_already_set() : std::runtime_error(detail::error_string("pybind11::error_already_set")) {
|
||||
PyErr_Fetch(&m_type.ptr(), &m_value.ptr(), &m_trace.ptr());
|
||||
}
|
||||
/// Fetches the current Python exception (using PyErr_Fetch()), which will clear the
|
||||
/// current Python error indicator.
|
||||
error_already_set()
|
||||
: m_fetched_error{new detail::error_fetch_and_normalize("pybind11::error_already_set"),
|
||||
m_fetched_error_deleter} {}
|
||||
|
||||
/// WARNING: The GIL must be held when this copy constructor is invoked!
|
||||
error_already_set(const error_already_set &) = default;
|
||||
error_already_set(error_already_set &&) = default;
|
||||
|
||||
/// WARNING: This destructor needs to acquire the Python GIL. This can lead to
|
||||
/// The what() result is built lazily on demand.
|
||||
/// WARNING: This member function needs to acquire the Python GIL. This can lead to
|
||||
/// crashes (undefined behavior) if the Python interpreter is finalizing.
|
||||
inline ~error_already_set() override;
|
||||
const char *what() const noexcept override;
|
||||
|
||||
/// Restores the currently-held Python error (which will clear the Python error indicator first
|
||||
/// if already set). After this call, the current object no longer stores the error variables.
|
||||
/// NOTE: Any copies of this object may still store the error variables. Currently there is no
|
||||
// protection against calling restore() from multiple copies.
|
||||
/// if already set).
|
||||
/// NOTE: This member function will always restore the normalized exception, which may or may
|
||||
/// not be the original Python exception.
|
||||
/// WARNING: The GIL must be held when this member function is called!
|
||||
void restore() {
|
||||
PyErr_Restore(m_type.release().ptr(), m_value.release().ptr(), m_trace.release().ptr());
|
||||
}
|
||||
void restore() { m_fetched_error->restore(); }
|
||||
|
||||
/// If it is impossible to raise the currently-held error, such as in a destructor, we can
|
||||
/// write it out using Python's unraisable hook (`sys.unraisablehook`). The error context
|
||||
/// should be some object whose `repr()` helps identify the location of the error. Python
|
||||
/// already knows the type and value of the error, so there is no need to repeat that. After
|
||||
/// this call, the current object no longer stores the error variables, and neither does
|
||||
/// Python.
|
||||
/// already knows the type and value of the error, so there is no need to repeat that.
|
||||
void discard_as_unraisable(object err_context) {
|
||||
restore();
|
||||
PyErr_WriteUnraisable(err_context.ptr());
|
||||
@ -447,16 +614,18 @@ public:
|
||||
/// Check if the currently trapped error type matches the given Python exception class (or a
|
||||
/// subclass thereof). May also be passed a tuple to search for any exception class matches in
|
||||
/// the given tuple.
|
||||
bool matches(handle exc) const {
|
||||
return (PyErr_GivenExceptionMatches(m_type.ptr(), exc.ptr()) != 0);
|
||||
}
|
||||
bool matches(handle exc) const { return m_fetched_error->matches(exc); }
|
||||
|
||||
const object &type() const { return m_type; }
|
||||
const object &value() const { return m_value; }
|
||||
const object &trace() const { return m_trace; }
|
||||
const object &type() const { return m_fetched_error->m_type; }
|
||||
const object &value() const { return m_fetched_error->m_value; }
|
||||
const object &trace() const { return m_fetched_error->m_trace; }
|
||||
|
||||
private:
|
||||
object m_type, m_value, m_trace;
|
||||
std::shared_ptr<detail::error_fetch_and_normalize> m_fetched_error;
|
||||
|
||||
/// WARNING: This custom deleter needs to acquire the Python GIL. This can lead to
|
||||
/// crashes (undefined behavior) if the Python interpreter is finalizing.
|
||||
static void m_fetched_error_deleter(detail::error_fetch_and_normalize *raw_ptr);
|
||||
};
|
||||
#if defined(_MSC_VER)
|
||||
# pragma warning(pop)
|
||||
@ -492,8 +661,7 @@ inline void raise_from(PyObject *type, const char *message) {
|
||||
|
||||
/// Sets the current Python error indicator with the chosen error, performing a 'raise from'
|
||||
/// from the error contained in error_already_set to indicate that the chosen error was
|
||||
/// caused by the original error. After this function is called error_already_set will
|
||||
/// no longer contain an error.
|
||||
/// caused by the original error.
|
||||
inline void raise_from(error_already_set &err, PyObject *type, const char *message) {
|
||||
err.restore();
|
||||
raise_from(type, message);
|
||||
|
||||
@ -105,6 +105,11 @@ struct PythonAlreadySetInDestructor {
|
||||
py::str s;
|
||||
};
|
||||
|
||||
std::string error_already_set_what(const py::object &exc_type, const py::object &exc_value) {
|
||||
PyErr_SetObject(exc_type.ptr(), exc_value.ptr());
|
||||
return py::error_already_set().what();
|
||||
}
|
||||
|
||||
TEST_SUBMODULE(exceptions, m) {
|
||||
m.def("throw_std_exception",
|
||||
[]() { throw std::runtime_error("This exception was intentionally thrown."); });
|
||||
@ -269,7 +274,9 @@ TEST_SUBMODULE(exceptions, m) {
|
||||
if (ex.matches(exc_type)) {
|
||||
py::print(ex.what());
|
||||
} else {
|
||||
throw;
|
||||
// Simply `throw;` also works and is better, but using `throw ex;`
|
||||
// here to cover that situation (as observed in the wild).
|
||||
throw ex; // Invokes the copy ctor.
|
||||
}
|
||||
}
|
||||
});
|
||||
@ -317,4 +324,14 @@ TEST_SUBMODULE(exceptions, m) {
|
||||
= reinterpret_cast<void (*)()>(PyLong_AsVoidPtr(cm.attr("funcaddr").ptr()));
|
||||
interleaved_error_already_set();
|
||||
});
|
||||
|
||||
m.def("test_error_already_set_double_restore", [](bool dry_run) {
|
||||
PyErr_SetString(PyExc_ValueError, "Random error.");
|
||||
py::error_already_set e;
|
||||
e.restore();
|
||||
PyErr_Clear();
|
||||
if (!dry_run) {
|
||||
e.restore();
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
@ -305,13 +305,16 @@ def test_error_already_set_what_with_happy_exceptions(
|
||||
|
||||
@pytest.mark.skipif("env.PYPY", reason="PyErr_NormalizeException Segmentation fault")
|
||||
def test_flaky_exception_failure_point_init():
|
||||
what, py_err_set_after_what = m.error_already_set_what(
|
||||
FlakyException, ("failure_point_init",)
|
||||
)
|
||||
assert not py_err_set_after_what
|
||||
lines = what.splitlines()
|
||||
with pytest.raises(RuntimeError) as excinfo:
|
||||
m.error_already_set_what(FlakyException, ("failure_point_init",))
|
||||
lines = str(excinfo.value).splitlines()
|
||||
# PyErr_NormalizeException replaces the original FlakyException with ValueError:
|
||||
assert lines[:3] == ["ValueError: triggered_failure_point_init", "", "At:"]
|
||||
assert lines[:3] == [
|
||||
"pybind11::error_already_set: MISMATCH of original and normalized active exception types:"
|
||||
" ORIGINAL FlakyException REPLACED BY ValueError: triggered_failure_point_init",
|
||||
"",
|
||||
"At:",
|
||||
]
|
||||
# Checking the first two lines of the traceback as formatted in error_string():
|
||||
assert "test_exceptions.py(" in lines[3]
|
||||
assert lines[3].endswith("): __init__")
|
||||
@ -319,10 +322,25 @@ def test_flaky_exception_failure_point_init():
|
||||
|
||||
|
||||
def test_flaky_exception_failure_point_str():
|
||||
# The error_already_set ctor fails due to a ValueError in error_string():
|
||||
with pytest.raises(ValueError) as excinfo:
|
||||
m.error_already_set_what(FlakyException, ("failure_point_str",))
|
||||
assert str(excinfo.value) == "triggered_failure_point_str"
|
||||
what, py_err_set_after_what = m.error_already_set_what(
|
||||
FlakyException, ("failure_point_str",)
|
||||
)
|
||||
assert not py_err_set_after_what
|
||||
lines = what.splitlines()
|
||||
if env.PYPY and len(lines) == 3:
|
||||
n = 3 # Traceback is missing.
|
||||
else:
|
||||
n = 5
|
||||
assert (
|
||||
lines[:n]
|
||||
== [
|
||||
"FlakyException: <MESSAGE UNAVAILABLE DUE TO ANOTHER EXCEPTION>",
|
||||
"",
|
||||
"MESSAGE UNAVAILABLE DUE TO EXCEPTION: ValueError: triggered_failure_point_str",
|
||||
"",
|
||||
"At:",
|
||||
][:n]
|
||||
)
|
||||
|
||||
|
||||
def test_cross_module_interleaved_error_already_set():
|
||||
@ -332,3 +350,13 @@ def test_cross_module_interleaved_error_already_set():
|
||||
"2nd error.", # Almost all platforms.
|
||||
"RuntimeError: 2nd error.", # Some PyPy builds (seen under macOS).
|
||||
)
|
||||
|
||||
|
||||
def test_error_already_set_double_restore():
|
||||
m.test_error_already_set_double_restore(True) # dry_run
|
||||
with pytest.raises(RuntimeError) as excinfo:
|
||||
m.test_error_already_set_double_restore(False)
|
||||
assert str(excinfo.value) == (
|
||||
"Internal error: pybind11::detail::error_fetch_and_normalize::restore()"
|
||||
" called a second time. ORIGINAL ERROR: ValueError: Random error."
|
||||
)
|
||||
|
||||
Loading…
Reference in New Issue
Block a user