pybind11/pybind11
Laramie Leavitt 0e599589fe
Fix thread safety for pybind11 loader_life_support (#3237)
* Fix thread safety for pybind11 loader_life_support

Fixes issue: https://github.com/pybind/pybind11/issues/2765

This converts the vector of PyObjects to either a single void* or
a per-thread void* depending on the WITH_THREAD define.

The new field is used by each thread to construct a stack
of loader_life_support frames that can extend the life of python
objects.

The pointer is updated when the loader_life_support object is allocated
(which happens before a call) as well as on release.

Each loader_life_support maintains a set of PyObject references
that need to be lifetime extended; this is done by storing them
in a c++ std::unordered_set and clearing the references when the
method completes.

* Also update the internals version as the internal struct is no longer compatible

* Add test demonstrating threading works correctly.

It may be appropriate to run this under msan/tsan/etc.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update test to use lifetime-extended references rather than
std::string_view, as that's a C++ 17 feature.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Make loader_life_support members private

* Update version to dev2

* Update test to use python threading rather than concurrent.futures

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Remove unnecessary env in test

* Remove unnecessary pytest in test

* Use native C++ thread_local in place of python per-thread data structures to retain compatability

* clang-format test_thread.cpp

* Add a note about debugging the py::cast() error

* thread_test.py now propagates exceptions on join() calls.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* remove unused sys / merge

* Update include order in test_thread.cpp

* Remove spurious whitespace

* Update comment / whitespace.

* Address review comments

* lint cleanup

* Fix test IntStruct constructor.

* Add explicit to constructor

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
2021-09-10 12:29:21 -04:00
..
__init__.py maint(precommit): Apply isort (#3195) 2021-08-13 12:37:05 -04:00
__main__.py maint(precommit): Apply isort (#3195) 2021-08-13 12:37:05 -04:00
_version.py Fix thread safety for pybind11 loader_life_support (#3237) 2021-09-10 12:29:21 -04:00
_version.pyi maint(precommit): Apply isort (#3195) 2021-08-13 12:37:05 -04:00
commands.py maint(precommit): Apply isort (#3195) 2021-08-13 12:37:05 -04:00
py.typed feat: typing support for helpers (#2588) 2020-10-14 14:08:41 -04:00
setup_helpers.py maint(precommit): Apply isort (#3195) 2021-08-13 12:37:05 -04:00
setup_helpers.pyi maint(precommit): Apply isort (#3195) 2021-08-13 12:37:05 -04:00