pybind11/tests/pybind11_tests.cpp
Chekov2k b07d08f600
Add PYBIND11_SIMPLE_GIL_MANAGEMENT option (cmake, C++ define) (#4216)
* Add option to force the use of the PYPY GIL scoped acquire/release logic to support nested gil access, see https://github.com/pybind/pybind11/issues/1276 and https://github.com/pytorch/pytorch/issues/83101

* Apply suggestions from code review

* Update CMakeLists.txt

* docs: update upgrade guide

* Update docs/upgrade.rst

* All bells & whistles.

* Add Reminder to common.h, so that we will not forget to purge `!WITH_THREAD` branches when dropping Python 3.6

* New sentence instead of semicolon.

* Temporarily pull in snapshot of PR #4246

* Add `test_release_acquire`

* Add more unit tests for nested gil locking

* Add test_report_builtins_internals_keys

* Very minor enhancement: sort list only after filtering.

* Revert change in docs/upgrade.rst

* Add test_multi_acquire_release_cross_module, while also forcing unique PYBIND11_INTERNALS_VERSION for cross_module_gil_utils.cpp

* Hopefully fix apparently new ICC error.

```
2022-10-28T07:57:54.5187728Z -- The CXX compiler identification is Intel 2021.7.0.20220726
...
2022-10-28T07:58:53.6758994Z icpc: remark #10441: The Intel(R) C++ Compiler Classic (ICC) is deprecated and will be removed from product release in the second half of 2023. The Intel(R) oneAPI DPC++/C++ Compiler (ICX) is the recommended compiler moving forward. Please transition to use this compiler. Use '-diag-disable=10441' to disable this message.
2022-10-28T07:58:54.5801597Z In file included from /home/runner/work/pybind11/pybind11/include/pybind11/detail/../detail/type_caster_base.h(15),
2022-10-28T07:58:54.5803794Z                  from /home/runner/work/pybind11/pybind11/include/pybind11/detail/../cast.h(15),
2022-10-28T07:58:54.5805740Z                  from /home/runner/work/pybind11/pybind11/include/pybind11/detail/../attr.h(14),
2022-10-28T07:58:54.5809556Z                  from /home/runner/work/pybind11/pybind11/include/pybind11/detail/class.h(12),
2022-10-28T07:58:54.5812154Z                  from /home/runner/work/pybind11/pybind11/include/pybind11/pybind11.h(13),
2022-10-28T07:58:54.5948523Z                  from /home/runner/work/pybind11/pybind11/tests/cross_module_gil_utils.cpp(13):
2022-10-28T07:58:54.5949009Z /home/runner/work/pybind11/pybind11/include/pybind11/detail/../detail/internals.h(177): error #2282: unrecognized GCC pragma
2022-10-28T07:58:54.5949374Z       PYBIND11_TLS_KEY_INIT(tstate)
2022-10-28T07:58:54.5949579Z       ^
2022-10-28T07:58:54.5949695Z
```

* clang-tidy fixes

* Workaround for PYPY WIN exitcode None

* Revert "Temporarily pull in snapshot of PR #4246"

This reverts commit 23ac16e859150f27fda25ca865cabcb4444e0770.

* Another workaround for PYPY WIN exitcode None

* Clean up how the tests are run "run in process" Part 1: uniformity

* Clean up how the tests are run "run in process" Part 2: use `@pytest.mark.parametrize` and clean up the naming.

* Skip some tests `#if defined(THREAD_SANITIZER)` (tested with TSAN using the Google-internal toolchain).

* Run all tests again but ignore ThreadSanitizer exitcode 66 (this is less likely to mask unrelated ThreadSanitizer issues in the future).

* bug fix: missing common.h include before using `PYBIND11_SIMPLE_GIL_MANAGEMENT`

For the tests in the github CI this does not matter, because
`PYBIND11_SIMPLE_GIL_MANAGEMENT` is always defined from the command line,
but when monkey-patching common.h locally, it matters.

* if process.exitcode is None: assert t_delta > 9.9

* More sophisiticated `_run_in_process()` implementation, clearly reporting `DEADLOCK`, additionally exercised via added `intentional_deadlock()`

* Wrap m.intentional_deadlock in a Python function, for `ForkingPickler` compatibility.

```
>       ForkingPickler(file, protocol).dump(obj)
E       TypeError: cannot pickle 'PyCapsule' object
```

Observed with all Windows builds including mingw but not PyPy, and macos-latest with Python 3.9, 3.10, 3.11 but not 3.6.

* Add link to potential solution for WOULD-BE-NICE-TO-HAVE feature.

* Add `SKIP_IF_DEADLOCK = True` option, to not pollute the CI results with expected `DEADLOCK` failures while we figure out what to do about them.

* Add COPY-PASTE-THIS: gdb ... command (to be used for debugging the detected deadlock)

* style: pre-commit fixes

* Do better than automatic pre-commit fixes.

* Add `PYBIND11_SIMPLE_GIL_MANAGEMENT` to `pytest_report_header()` (so that we can easily know when harvesting deadlock information from the CI logs).

Co-authored-by: Arnim Balzer <arnim@seechange.ai>
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2022-10-30 08:57:23 -07:00

124 lines
4.4 KiB
C++

/*
tests/pybind11_tests.cpp -- pybind example plugin
Copyright (c) 2016 Wenzel Jakob <wenzel.jakob@epfl.ch>
All rights reserved. Use of this source code is governed by a
BSD-style license that can be found in the LICENSE file.
*/
#include "pybind11_tests.h"
#include "constructor_stats.h"
#include <functional>
#include <list>
/*
For testing purposes, we define a static global variable here in a function that each individual
test .cpp calls with its initialization lambda. It's convenient here because we can just not
compile some test files to disable/ignore some of the test code.
It is NOT recommended as a way to use pybind11 in practice, however: the initialization order will
be essentially random, which is okay for our test scripts (there are no dependencies between the
individual pybind11 test .cpp files), but most likely not what you want when using pybind11
productively.
Instead, see the "How can I reduce the build time?" question in the "Frequently asked questions"
section of the documentation for good practice on splitting binding code over multiple files.
*/
std::list<std::function<void(py::module_ &)>> &initializers() {
static std::list<std::function<void(py::module_ &)>> inits;
return inits;
}
test_initializer::test_initializer(Initializer init) { initializers().emplace_back(init); }
test_initializer::test_initializer(const char *submodule_name, Initializer init) {
initializers().emplace_back([=](py::module_ &parent) {
auto m = parent.def_submodule(submodule_name);
init(m);
});
}
void bind_ConstructorStats(py::module_ &m) {
py::class_<ConstructorStats>(m, "ConstructorStats")
.def("alive", &ConstructorStats::alive)
.def("values", &ConstructorStats::values)
.def_readwrite("default_constructions", &ConstructorStats::default_constructions)
.def_readwrite("copy_assignments", &ConstructorStats::copy_assignments)
.def_readwrite("move_assignments", &ConstructorStats::move_assignments)
.def_readwrite("copy_constructions", &ConstructorStats::copy_constructions)
.def_readwrite("move_constructions", &ConstructorStats::move_constructions)
.def_static("get",
(ConstructorStats & (*) (py::object)) & ConstructorStats::get,
py::return_value_policy::reference_internal)
// Not exactly ConstructorStats, but related: expose the internal pybind number of
// registered instances to allow instance cleanup checks (invokes a GC first)
.def_static("detail_reg_inst", []() {
ConstructorStats::gc();
return py::detail::get_internals().registered_instances.size();
});
}
const char *cpp_std() {
return
#if defined(PYBIND11_CPP20)
"C++20";
#elif defined(PYBIND11_CPP17)
"C++17";
#elif defined(PYBIND11_CPP14)
"C++14";
#else
"C++11";
#endif
}
PYBIND11_MODULE(pybind11_tests, m) {
m.doc() = "pybind11 test module";
// Intentionally kept minimal to not create a maintenance chore
// ("just enough" to be conclusive).
#if defined(_MSC_FULL_VER)
m.attr("compiler_info") = "MSVC " PYBIND11_TOSTRING(_MSC_FULL_VER);
#elif defined(__VERSION__)
m.attr("compiler_info") = __VERSION__;
#else
m.attr("compiler_info") = py::none();
#endif
m.attr("cpp_std") = cpp_std();
m.attr("PYBIND11_INTERNALS_ID") = PYBIND11_INTERNALS_ID;
m.attr("PYBIND11_SIMPLE_GIL_MANAGEMENT") =
#if defined(PYBIND11_SIMPLE_GIL_MANAGEMENT)
true;
#else
false;
#endif
bind_ConstructorStats(m);
#if defined(PYBIND11_DETAILED_ERROR_MESSAGES)
m.attr("detailed_error_messages_enabled") = true;
#else
m.attr("detailed_error_messages_enabled") = false;
#endif
py::class_<UserType>(m, "UserType", "A `py::class_` type for testing")
.def(py::init<>())
.def(py::init<int>())
.def("get_value", &UserType::value, "Get value using a method")
.def("set_value", &UserType::set, "Set value using a method")
.def_property("value", &UserType::value, &UserType::set, "Get/set value using a property")
.def("__repr__", [](const UserType &u) { return "UserType({})"_s.format(u.value()); });
py::class_<IncType, UserType>(m, "IncType")
.def(py::init<>())
.def(py::init<int>())
.def("__repr__", [](const IncType &u) { return "IncType({})"_s.format(u.value()); });
for (const auto &initializer : initializers()) {
initializer(m);
}
}