Background: #2999, #4105, #4283, #4284
In a nutshell:
* Only macOS actually needs `PYBIND11_EXPORT_EXCEPTION` (#4284).
* Evidently (#4283), under macOS `PYBIND11_EXPORT_EXCEPTION` does not run the risk of introducing ODR violations,
* but evidently (#4283) under Linux it does, in the presumably rare/unusual situation that `RTLD_GLOBAL` is used.
* Windows does no have the equivalent of `RTLD_GLOBAL`, therefore `PYBIND11_EXPORT_EXCEPTION` has no practical benefit, on the contrary, noisy warning suppression pragmas are needed, therefore it is best left empty.
* Add option to force the use of the PYPY GIL scoped acquire/release logic to support nested gil access, see https://github.com/pybind/pybind11/issues/1276 and https://github.com/pytorch/pytorch/issues/83101
* Apply suggestions from code review
* Update CMakeLists.txt
* docs: update upgrade guide
* Update docs/upgrade.rst
* All bells & whistles.
* Add Reminder to common.h, so that we will not forget to purge `!WITH_THREAD` branches when dropping Python 3.6
* New sentence instead of semicolon.
* Temporarily pull in snapshot of PR #4246
* Add `test_release_acquire`
* Add more unit tests for nested gil locking
* Add test_report_builtins_internals_keys
* Very minor enhancement: sort list only after filtering.
* Revert change in docs/upgrade.rst
* Add test_multi_acquire_release_cross_module, while also forcing unique PYBIND11_INTERNALS_VERSION for cross_module_gil_utils.cpp
* Hopefully fix apparently new ICC error.
```
2022-10-28T07:57:54.5187728Z -- The CXX compiler identification is Intel 2021.7.0.20220726
...
2022-10-28T07:58:53.6758994Z icpc: remark #10441: The Intel(R) C++ Compiler Classic (ICC) is deprecated and will be removed from product release in the second half of 2023. The Intel(R) oneAPI DPC++/C++ Compiler (ICX) is the recommended compiler moving forward. Please transition to use this compiler. Use '-diag-disable=10441' to disable this message.
2022-10-28T07:58:54.5801597Z In file included from /home/runner/work/pybind11/pybind11/include/pybind11/detail/../detail/type_caster_base.h(15),
2022-10-28T07:58:54.5803794Z from /home/runner/work/pybind11/pybind11/include/pybind11/detail/../cast.h(15),
2022-10-28T07:58:54.5805740Z from /home/runner/work/pybind11/pybind11/include/pybind11/detail/../attr.h(14),
2022-10-28T07:58:54.5809556Z from /home/runner/work/pybind11/pybind11/include/pybind11/detail/class.h(12),
2022-10-28T07:58:54.5812154Z from /home/runner/work/pybind11/pybind11/include/pybind11/pybind11.h(13),
2022-10-28T07:58:54.5948523Z from /home/runner/work/pybind11/pybind11/tests/cross_module_gil_utils.cpp(13):
2022-10-28T07:58:54.5949009Z /home/runner/work/pybind11/pybind11/include/pybind11/detail/../detail/internals.h(177): error #2282: unrecognized GCC pragma
2022-10-28T07:58:54.5949374Z PYBIND11_TLS_KEY_INIT(tstate)
2022-10-28T07:58:54.5949579Z ^
2022-10-28T07:58:54.5949695Z
```
* clang-tidy fixes
* Workaround for PYPY WIN exitcode None
* Revert "Temporarily pull in snapshot of PR #4246"
This reverts commit 23ac16e859150f27fda25ca865cabcb4444e0770.
* Another workaround for PYPY WIN exitcode None
* Clean up how the tests are run "run in process" Part 1: uniformity
* Clean up how the tests are run "run in process" Part 2: use `@pytest.mark.parametrize` and clean up the naming.
* Skip some tests `#if defined(THREAD_SANITIZER)` (tested with TSAN using the Google-internal toolchain).
* Run all tests again but ignore ThreadSanitizer exitcode 66 (this is less likely to mask unrelated ThreadSanitizer issues in the future).
* bug fix: missing common.h include before using `PYBIND11_SIMPLE_GIL_MANAGEMENT`
For the tests in the github CI this does not matter, because
`PYBIND11_SIMPLE_GIL_MANAGEMENT` is always defined from the command line,
but when monkey-patching common.h locally, it matters.
* if process.exitcode is None: assert t_delta > 9.9
* More sophisiticated `_run_in_process()` implementation, clearly reporting `DEADLOCK`, additionally exercised via added `intentional_deadlock()`
* Wrap m.intentional_deadlock in a Python function, for `ForkingPickler` compatibility.
```
> ForkingPickler(file, protocol).dump(obj)
E TypeError: cannot pickle 'PyCapsule' object
```
Observed with all Windows builds including mingw but not PyPy, and macos-latest with Python 3.9, 3.10, 3.11 but not 3.6.
* Add link to potential solution for WOULD-BE-NICE-TO-HAVE feature.
* Add `SKIP_IF_DEADLOCK = True` option, to not pollute the CI results with expected `DEADLOCK` failures while we figure out what to do about them.
* Add COPY-PASTE-THIS: gdb ... command (to be used for debugging the detected deadlock)
* style: pre-commit fixes
* Do better than automatic pre-commit fixes.
* Add `PYBIND11_SIMPLE_GIL_MANAGEMENT` to `pytest_report_header()` (so that we can easily know when harvesting deadlock information from the CI logs).
Co-authored-by: Arnim Balzer <arnim@seechange.ai>
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* (bugfix): Improve bytes to str decoding error handling
* regroup test
* Further broaden tests
* Add another decode error test
* Fix bug in tests
* Reviewer suggestions
* fix: PyPy needs to overrite broken FindPythonInterp values
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* fix: add flag to opt-in to new (cross-compile) behavior
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* Apply suggestions from code review
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* Simply replace "22.3" with "22.9" to see what happens.
* Remove PYBIND11_TEST_FILTER to see what happens.
* Revert "Remove PYBIND11_TEST_FILTER to see what happens."
This reverts commit 0cba2cef0c.
* Remove only test_smart_ptr.cpp to see what happens.
* Revert "Remove only test_smart_ptr.cpp to see what happens."
This reverts commit 8e9df22c85.
* Remove only test_virtual_functions.cpp to see what happens.
* First draft of Eigen::Tensor support
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix build errors
* Weird allocator stuff?
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove unused + additional allocator junk
* Disable warning
* Use constexpr
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* clang tidy fixes
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Resolve comments
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove auto constexpr function
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Try again for older C++
* Oops forgot constexpr
* Move to new files as suggested
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix weird tests
* Fix nits
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Oops, forgot import
* Fix clang 3.6 bug
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* More comprehensive test suite
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Refactor allocators to make things more clear
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Switch to std::copy
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Switch to DSizes instead of array
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address feedback
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix python + dummy c++ change to trigger build
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Alignment
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add include guard
* Forgot inline
* Fix compiler warning
* Remove bad test
* Better type signatures
* Add guards to make compiler requirements more explicit
* style: pre-commit fixes
* Force rerun of tests due to flake
* style: pre-commit fixes
* Keep pragmas & all related comments together, add PLEASE KEEP IN SYNC
* Move headers out of detail
* style: pre-commit fixes
* Fix cmake
* Improve casting
* style: pre-commit fixes
* Add a ton more tests + refactor
* Improve names
* style: pre-commit fixes
* Update include/pybind11/eigen/tensor.h
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
* Fix tests
* style: pre-commit fixes
* Update
* Add a test to verify that strange numpy arrays work
* Fix dumb compiler warning
* Better tests
* Better tests
* Fix tests
* style: pre-commit fixes
* More test fixes
* style: pre-commit fixes
* A ton more test coverage
* Fix tests
* style: pre-commit fixes
* style: pre-commit fixes
* Add back constexpr
* Another test
* style: pre-commit fixes
* Improve tests
* Whoops
* Less magic numbers
* Update tests/test_eigen_tensor.py
Co-authored-by: Sergiu Deitsch <sergiud@users.noreply.github.com>
* Update tests/test_eigen_tensor.py
Co-authored-by: Sergiu Deitsch <sergiud@users.noreply.github.com>
* style: pre-commit fixes
* Fix tests
* style: pre-commit fixes
* Fix memory leak
* style: pre-commit fixes
* Fix order
* style: pre-commit fixes
* Add test to make sure unsafe casts fail
* Minor bug fix to work on 32 bit machines
* Implement convert flag
* style: pre-commit fixes
* Switch to correct TensorMap const use
* style: pre-commit fixes
* Support older versions of eigen
* Weird c++ compilers
* Fix Eigen bug
* Fix another eigen bug
* Yet another eigen bug
* Potential flakes?
* style: pre-commit fixes
* Rerun tests with dummy exception to find out what is going on
* Another dummy test run
* Ablate more
* Found the broken test?
* One step closer
* one step further
* Double check
* one thing at a time
* Give up and disable the test
* Clang lies about being gcc
* Oops, fix matrix test
* style: pre-commit fixes
* Add tests to verify scalar conversions
* style: pre-commit fixes
* Fix nits
* Support no_array
* Fix tests
* style: pre-commit fixes
* Silence compiler warning
* Improve build system for ancient compilers
* Make clang happy
* Make gcc happy
* Implement Skylion's suggestions
* Fix warning
* Inline const pointer check
* Implement suggestions
* style: pre-commit fixes
* Improve tests
* Typo
* style: pre-commit fixes
* Support Google's build environment
* style: pre-commit fixes
* Update include/pybind11/eigen/tensor.h
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
* style: pre-commit fixes
* Test cleanup per Skylion
* Switch to remvove_cv_t
* Cleaner test
* style: pre-commit fixes
* Remove tensor from eigen.h, update tests
* style: pre-commit fixes
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com>
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
Co-authored-by: Sergiu Deitsch <sergiud@users.noreply.github.com>
* bugfix: contains now throws an exception if the key is not hashable
* Fix tests and improve robustness
* Remove todo
* Workaround PyPy corner case
* PyPy xfail
* Fix typo
* fix xfail
* Make clang-tidy happy
* Remove redundant exc checking
Use `PyCapsule_Destructor` (part of the stable Python ABI) instead of spelling out the C `typedef`.
The deprecation message is misleading. Replace with a message pointing to another existing ctor.
Background: According to @wjakob the original motivation for deprecating the ctor (in PR #752) was to hide Python C API details, but PR #902 brought those back with a new ctor, it cannot be avoided. Having a `PyCapsule_Destructor` or a `void (*destructor)(void *)` are two separate and valid use cases.
* Reproducer for issue encountered in smart_holder update.
* clang-tidy compatibility (untested).
* Add `enable_if_t` to workaround.
* Bug fix: Move `PYBIND11_USING_WORKAROUND_FOR_CUDA_11_4_THROUGH_8` determination to detail/common.h
So that it actually is defined in pybind11.h
* Try using the workaround (which is nicer than the original code) universally.
* Reduce reproducer for CUDA 11.7 issue encountered in smart_holder update.
This commit tested in isolation on top of current master + first version of reproducer (62311eb431).
Succeeds with Debian Clang 14.0.6 C++17 (and probably all other compilers).
Fails for CUDA 11.7:
```
cd /build/tests && /usr/local/cuda/bin/nvcc -forward-unknown-to-host-compiler -Dpybind11_tests_EXPORTS -I/mounted_pybind11/include -isystem=/usr/include/python3.10 -g --generate-code=arch=compute_52,code=[compute_52,sm_52] -Xcompiler=-fPIC -Xcompiler=-fvisibility=hidden -Werror all-warnings -std=c++17 -MD -MT tests/CMakeFiles/pybind11_tests.dir/test_class.cpp.o -MF CMakeFiles/pybind11_tests.dir/test_class.cpp.o.d -x cu -c /mounted_pybind11/tests/test_class.cpp -o CMakeFiles/pybind11_tests.dir/test_class.cpp.o
/mounted_pybind11/tests/test_class.cpp(53): error: more than one instance of overloaded function "pybind11::class_<type_, options...>::def [with type_=test_class::pr4220_tripped_over_this::Empty0, options=<>]" matches the argument list:
function template "pybind11::class_<test_class::pr4220_tripped_over_this::Empty0> &pybind11::class_<type_, options...>::def(const char *, Func &&, const Extra &...) [with type_=test_class::pr4220_tripped_over_this::Empty0, options=<>]"
/mounted_pybind11/include/pybind11/pybind11.h(1557): here
function template "pybind11::class_<test_class::pr4220_tripped_over_this::Empty0> &pybind11::class_<type_, options...>::def(const T &, const Extra &...) [with type_=test_class::pr4220_tripped_over_this::Empty0, options=<>]"
/mounted_pybind11/include/pybind11/pybind11.h(1586): here
argument types are: (const char [8], <unknown-type>)
object type is: pybind11::class_<test_class::pr4220_tripped_over_this::Empty0>
1 error detected in the compilation of "/mounted_pybind11/tests/test_class.cpp".
```
* fix: PyCapsule_GetDestructor is allowed to return a nullptr destructor
Previously, this code would error out if the destructor happened to be
a nullptr. This is incorrect. nullptrs are allowed for capsule
destructors.
"It is legal for a capsule to have a NULL destructor. This makes a
NULL return code somewhat ambiguous; use PyCapsule_IsValid() or
PyErr_Occurred() to disambiguate."
See:
https://docs.python.org/3/c-api/capsule.html#c.PyCapsule_GetDestructor
I noticed this while working on a type caster related to #3858 DLPack
happens to allow the destructor not to be defined on a capsule, and I
encountered such a case. See:
e2bdd3bee8/include/dlpack/dlpack.h (L219)
* Add test for the fix.
* Update tests/test_pytypes.cpp
I tried this locally and it works!
I never knew that there are cases where `reinterpret_cast` does not work but `static_cast` does. Let's see if all compilers are happy with this.
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
* style: pre-commit fixes
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgkio@gmail.com>
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Disable test triggering ASAN failure (to pin-point where the problem is).
* Fix unsafe "block" implementation in test_eigen.cpp
* Undo changes (i.e. revert back to master).
* Detect "type_caster for Eigen::Ref made a copy."
This is achieved without
* reaching into internals,
* making test_eigen.cpp depend on pybind11/numpy.h.
* Add comment pointing to PR, for easy reference.
* Work-Around: NVCC 11.4.0 - 11.8.0
Adds a targeted NVCC work around for limited number of CUDA
releases. Fixed in NVCC development.
* style: pre-commit fixes
* CI: Bump CTK Version 11.2 -> 11.7
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Call reserve method in set and map casters too
* Refactor template logic into has_reserve_method
* Adjust comment for reviews
* Rearrange reserve_maybe to not be underneath macro
* Add debug fprintf to test_interpreter.cpp
* Update `sys.path` from `PYTHONPATH` in Python >= 3.11 branch of `initialize_interpreter()`
* Use `config.isolated = 0; config.use_environment = 1;`
As suggsted by @vstinner here: https://github.com/pybind/pybind11/pull/4119#issuecomment-1219442853
* Add `TEST_CASE("PYTHONPATH is used to update sys.path")`
* Fix clang-tidy error.
* Use `_putenv_s()` under Windows.
* Fix clang-tidy error: argument name ... in comment does not match parameter name
* Remove slash from PYTHONPATH addition, to work around Windows slash-vs-backslash issue.
* Use `py::str(...)` instead of `.attr("__str__")` as suggested by @skylion007
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
