pybind11/tests/pybind11_tests.cpp

124 lines
4.4 KiB
C++
Raw Normal View History

2015-07-05 18:05:44 +00:00
/*
tests/pybind11_tests.cpp -- pybind example plugin
2015-07-05 18:05:44 +00:00
Copyright (c) 2016 Wenzel Jakob <wenzel.jakob@epfl.ch>
2015-07-05 18:05:44 +00:00
All rights reserved. Use of this source code is governed by a
BSD-style license that can be found in the LICENSE file.
*/
#include "pybind11_tests.h"
#include "constructor_stats.h"
2015-07-05 18:05:44 +00:00
#include <functional>
#include <list>
/*
For testing purposes, we define a static global variable here in a function that each individual
test .cpp calls with its initialization lambda. It's convenient here because we can just not
compile some test files to disable/ignore some of the test code.
It is NOT recommended as a way to use pybind11 in practice, however: the initialization order will
be essentially random, which is okay for our test scripts (there are no dependencies between the
individual pybind11 test .cpp files), but most likely not what you want when using pybind11
productively.
Instead, see the "How can I reduce the build time?" question in the "Frequently asked questions"
section of the documentation for good practice on splitting binding code over multiple files.
*/
std::list<std::function<void(py::module_ &)>> &initializers() {
static std::list<std::function<void(py::module_ &)>> inits;
return inits;
}
2015-07-05 18:05:44 +00:00
test_initializer::test_initializer(Initializer init) { initializers().emplace_back(init); }
test_initializer::test_initializer(const char *submodule_name, Initializer init) {
initializers().emplace_back([=](py::module_ &parent) {
auto m = parent.def_submodule(submodule_name);
init(m);
});
}
void bind_ConstructorStats(py::module_ &m) {
Improve constructor/destructor tracking This commit rewrites the examples that look for constructor/destructor calls to do so via static variable tracking rather than output parsing. The added ConstructorStats class provides methods to keep track of constructors and destructors, number of default/copy/move constructors, and number of copy/move assignments. It also provides a mechanism for storing values (e.g. for value construction), and then allows all of this to be checked at the end of a test by getting the statistics for a C++ (or python mapping) class. By not relying on the precise pattern of constructions/destructions, but rather simply ensuring that every construction is matched with a destruction on the same object, we ensure that everything that gets created also gets destroyed as expected. This replaces all of the various "std::cout << whatever" code in constructors/destructors with `print_created(this)`/`print_destroyed(this)`/etc. functions which provide similar output, but now has a unified format across the different examples, including a new ### prefix that makes mixed example output and lifecycle events easier to distinguish. With this change, relaxed mode is no longer needed, which enables testing for proper destruction under MSVC, and under any other compiler that generates code calling extra constructors, or optimizes away any constructors. GCC/clang are used as the baseline for move constructors; the tests are adapted to allow more move constructors to be evoked (but other types are constructors much have matching counts). This commit also disables output buffering of tests, as the buffering sometimes results in C++ output ending up in the middle of python output (or vice versa), depending on the OS/python version.
2016-08-07 17:05:26 +00:00
py::class_<ConstructorStats>(m, "ConstructorStats")
.def("alive", &ConstructorStats::alive)
.def("values", &ConstructorStats::values)
.def_readwrite("default_constructions", &ConstructorStats::default_constructions)
.def_readwrite("copy_assignments", &ConstructorStats::copy_assignments)
.def_readwrite("move_assignments", &ConstructorStats::move_assignments)
.def_readwrite("copy_constructions", &ConstructorStats::copy_constructions)
.def_readwrite("move_constructions", &ConstructorStats::move_constructions)
.def_static("get",
(ConstructorStats & (*) (py::object)) & ConstructorStats::get,
py::return_value_policy::reference_internal)
// Not exactly ConstructorStats, but related: expose the internal pybind number of
// registered instances to allow instance cleanup checks (invokes a GC first)
.def_static("detail_reg_inst", []() {
ConstructorStats::gc();
return py::detail::get_internals().registered_instances.size();
});
Improve constructor/destructor tracking This commit rewrites the examples that look for constructor/destructor calls to do so via static variable tracking rather than output parsing. The added ConstructorStats class provides methods to keep track of constructors and destructors, number of default/copy/move constructors, and number of copy/move assignments. It also provides a mechanism for storing values (e.g. for value construction), and then allows all of this to be checked at the end of a test by getting the statistics for a C++ (or python mapping) class. By not relying on the precise pattern of constructions/destructions, but rather simply ensuring that every construction is matched with a destruction on the same object, we ensure that everything that gets created also gets destroyed as expected. This replaces all of the various "std::cout << whatever" code in constructors/destructors with `print_created(this)`/`print_destroyed(this)`/etc. functions which provide similar output, but now has a unified format across the different examples, including a new ### prefix that makes mixed example output and lifecycle events easier to distinguish. With this change, relaxed mode is no longer needed, which enables testing for proper destruction under MSVC, and under any other compiler that generates code calling extra constructors, or optimizes away any constructors. GCC/clang are used as the baseline for move constructors; the tests are adapted to allow more move constructors to be evoked (but other types are constructors much have matching counts). This commit also disables output buffering of tests, as the buffering sometimes results in C++ output ending up in the middle of python output (or vice versa), depending on the OS/python version.
2016-08-07 17:05:26 +00:00
}
const char *cpp_std() {
return
#if defined(PYBIND11_CPP20)
"C++20";
#elif defined(PYBIND11_CPP17)
"C++17";
#elif defined(PYBIND11_CPP14)
"C++14";
#else
"C++11";
#endif
}
PYBIND11_MODULE(pybind11_tests, m) {
m.doc() = "pybind11 test module";
2015-07-05 18:05:44 +00:00
// Intentionally kept minimal to not create a maintenance chore
// ("just enough" to be conclusive).
#if defined(__VERSION__)
m.attr("compiler_info") = __VERSION__;
#elif defined(_MSC_FULL_VER)
m.attr("compiler_info") = "MSVC " PYBIND11_TOSTRING(_MSC_FULL_VER);
#else
m.attr("compiler_info") = py::none();
#endif
m.attr("cpp_std") = cpp_std();
m.attr("PYBIND11_INTERNALS_ID") = PYBIND11_INTERNALS_ID;
Add `PYBIND11_SIMPLE_GIL_MANAGEMENT` option (cmake, C++ define) (#4216) * Add option to force the use of the PYPY GIL scoped acquire/release logic to support nested gil access, see https://github.com/pybind/pybind11/issues/1276 and https://github.com/pytorch/pytorch/issues/83101 * Apply suggestions from code review * Update CMakeLists.txt * docs: update upgrade guide * Update docs/upgrade.rst * All bells & whistles. * Add Reminder to common.h, so that we will not forget to purge `!WITH_THREAD` branches when dropping Python 3.6 * New sentence instead of semicolon. * Temporarily pull in snapshot of PR #4246 * Add `test_release_acquire` * Add more unit tests for nested gil locking * Add test_report_builtins_internals_keys * Very minor enhancement: sort list only after filtering. * Revert change in docs/upgrade.rst * Add test_multi_acquire_release_cross_module, while also forcing unique PYBIND11_INTERNALS_VERSION for cross_module_gil_utils.cpp * Hopefully fix apparently new ICC error. ``` 2022-10-28T07:57:54.5187728Z -- The CXX compiler identification is Intel 2021.7.0.20220726 ... 2022-10-28T07:58:53.6758994Z icpc: remark #10441: The Intel(R) C++ Compiler Classic (ICC) is deprecated and will be removed from product release in the second half of 2023. The Intel(R) oneAPI DPC++/C++ Compiler (ICX) is the recommended compiler moving forward. Please transition to use this compiler. Use '-diag-disable=10441' to disable this message. 2022-10-28T07:58:54.5801597Z In file included from /home/runner/work/pybind11/pybind11/include/pybind11/detail/../detail/type_caster_base.h(15), 2022-10-28T07:58:54.5803794Z from /home/runner/work/pybind11/pybind11/include/pybind11/detail/../cast.h(15), 2022-10-28T07:58:54.5805740Z from /home/runner/work/pybind11/pybind11/include/pybind11/detail/../attr.h(14), 2022-10-28T07:58:54.5809556Z from /home/runner/work/pybind11/pybind11/include/pybind11/detail/class.h(12), 2022-10-28T07:58:54.5812154Z from /home/runner/work/pybind11/pybind11/include/pybind11/pybind11.h(13), 2022-10-28T07:58:54.5948523Z from /home/runner/work/pybind11/pybind11/tests/cross_module_gil_utils.cpp(13): 2022-10-28T07:58:54.5949009Z /home/runner/work/pybind11/pybind11/include/pybind11/detail/../detail/internals.h(177): error #2282: unrecognized GCC pragma 2022-10-28T07:58:54.5949374Z PYBIND11_TLS_KEY_INIT(tstate) 2022-10-28T07:58:54.5949579Z ^ 2022-10-28T07:58:54.5949695Z ``` * clang-tidy fixes * Workaround for PYPY WIN exitcode None * Revert "Temporarily pull in snapshot of PR #4246" This reverts commit 23ac16e859150f27fda25ca865cabcb4444e0770. * Another workaround for PYPY WIN exitcode None * Clean up how the tests are run "run in process" Part 1: uniformity * Clean up how the tests are run "run in process" Part 2: use `@pytest.mark.parametrize` and clean up the naming. * Skip some tests `#if defined(THREAD_SANITIZER)` (tested with TSAN using the Google-internal toolchain). * Run all tests again but ignore ThreadSanitizer exitcode 66 (this is less likely to mask unrelated ThreadSanitizer issues in the future). * bug fix: missing common.h include before using `PYBIND11_SIMPLE_GIL_MANAGEMENT` For the tests in the github CI this does not matter, because `PYBIND11_SIMPLE_GIL_MANAGEMENT` is always defined from the command line, but when monkey-patching common.h locally, it matters. * if process.exitcode is None: assert t_delta > 9.9 * More sophisiticated `_run_in_process()` implementation, clearly reporting `DEADLOCK`, additionally exercised via added `intentional_deadlock()` * Wrap m.intentional_deadlock in a Python function, for `ForkingPickler` compatibility. ``` > ForkingPickler(file, protocol).dump(obj) E TypeError: cannot pickle 'PyCapsule' object ``` Observed with all Windows builds including mingw but not PyPy, and macos-latest with Python 3.9, 3.10, 3.11 but not 3.6. * Add link to potential solution for WOULD-BE-NICE-TO-HAVE feature. * Add `SKIP_IF_DEADLOCK = True` option, to not pollute the CI results with expected `DEADLOCK` failures while we figure out what to do about them. * Add COPY-PASTE-THIS: gdb ... command (to be used for debugging the detected deadlock) * style: pre-commit fixes * Do better than automatic pre-commit fixes. * Add `PYBIND11_SIMPLE_GIL_MANAGEMENT` to `pytest_report_header()` (so that we can easily know when harvesting deadlock information from the CI logs). Co-authored-by: Arnim Balzer <arnim@seechange.ai> Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com> Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2022-10-30 15:57:23 +00:00
m.attr("PYBIND11_SIMPLE_GIL_MANAGEMENT") =
#if defined(PYBIND11_SIMPLE_GIL_MANAGEMENT)
true;
#else
false;
#endif
Improve constructor/destructor tracking This commit rewrites the examples that look for constructor/destructor calls to do so via static variable tracking rather than output parsing. The added ConstructorStats class provides methods to keep track of constructors and destructors, number of default/copy/move constructors, and number of copy/move assignments. It also provides a mechanism for storing values (e.g. for value construction), and then allows all of this to be checked at the end of a test by getting the statistics for a C++ (or python mapping) class. By not relying on the precise pattern of constructions/destructions, but rather simply ensuring that every construction is matched with a destruction on the same object, we ensure that everything that gets created also gets destroyed as expected. This replaces all of the various "std::cout << whatever" code in constructors/destructors with `print_created(this)`/`print_destroyed(this)`/etc. functions which provide similar output, but now has a unified format across the different examples, including a new ### prefix that makes mixed example output and lifecycle events easier to distinguish. With this change, relaxed mode is no longer needed, which enables testing for proper destruction under MSVC, and under any other compiler that generates code calling extra constructors, or optimizes away any constructors. GCC/clang are used as the baseline for move constructors; the tests are adapted to allow more move constructors to be evoked (but other types are constructors much have matching counts). This commit also disables output buffering of tests, as the buffering sometimes results in C++ output ending up in the middle of python output (or vice versa), depending on the OS/python version.
2016-08-07 17:05:26 +00:00
bind_ConstructorStats(m);
#if defined(PYBIND11_DETAILED_ERROR_MESSAGES)
m.attr("detailed_error_messages_enabled") = true;
#else
m.attr("detailed_error_messages_enabled") = false;
#endif
py::class_<UserType>(m, "UserType", "A `py::class_` type for testing")
.def(py::init<>())
.def(py::init<int>())
.def("get_value", &UserType::value, "Get value using a method")
Update all remaining tests to new test styles This udpates all the remaining tests to the new test suite code and comment styles started in #898. For the most part, the test coverage here is unchanged, with a few minor exceptions as noted below. - test_constants_and_functions: this adds more overload tests with overloads with different number of arguments for more comprehensive overload_cast testing. The test style conversion broke the overload tests under MSVC 2015, prompting the additional tests while looking for a workaround. - test_eigen: this dropped the unused functions `get_cm_corners` and `get_cm_corners_const`--these same tests were duplicates of the same things provided (and used) via ReturnTester methods. - test_opaque_types: this test had a hidden dependence on ExampleMandA which is now fixed by using the global UserType which suffices for the relevant test. - test_methods_and_attributes: this required some additions to UserType to make it usable as a replacement for the test's previous SimpleType: UserType gained a value mutator, and the `value` property is not mutable (it was previously readonly). Some overload tests were also added to better test overload_cast (as described above). - test_numpy_array: removed the untemplated mutate_data/mutate_data_t: the templated versions with an empty parameter pack expand to the same thing. - test_stl: this was already mostly in the new style; this just tweaks things a bit, localizing a class, and adding some missing `// test_whatever` comments. - test_virtual_functions: like `test_stl`, this was mostly in the new test style already, but needed some `// test_whatever` comments. This commit also moves the inherited virtual example code to the end of the file, after the main set of tests (since it is less important than the other tests, and rather length); it also got renamed to `test_inherited_virtuals` (from `test_inheriting_repeat`) because it tests both inherited virtual approaches, not just the repeat approach.
2017-07-25 20:47:36 +00:00
.def("set_value", &UserType::set, "Set value using a method")
.def_property("value", &UserType::value, &UserType::set, "Get/set value using a property")
.def("__repr__", [](const UserType &u) { return "UserType({})"_s.format(u.value()); });
py::class_<IncType, UserType>(m, "IncType")
.def(py::init<>())
.def(py::init<int>())
.def("__repr__", [](const IncType &u) { return "IncType({})"_s.format(u.value()); });
for (const auto &initializer : initializers()) {
initializer(m);
}
2015-07-05 18:05:44 +00:00
}