* Auto select return value policy for clif_automatic
* Try fixing test failures
* Add more tests.
* remove comments
* Fix test failures
* Fix test failures
* Fix test failure for windows platform
* Fix clangtidy
* scoped_interpreter overloaded ctor: PyConfig param
* style: pre-commit fixes
* refact: some logics extracted into funcs (precheck_interpreter, _initialize_interpreter); config_guard
* style: pre-commit fixes
* refact: scoped_config, some funcs hidden in detail ns
* refact: macro PYBIND11_PYCONFIG_SUPPORT_PY_VERSION + undef
* feat: PYBIND11_PYCONFIG_SUPPORT_PY_VERSION set to 3.8
* tests: Custom PyConfig
* ci: python 3.6 -> 3.8
* ci: reverted py 38 back to 36; refact: initialize_interpreter overloads
* style: pre-commit fixes
* fix: readability-implicit-bool-conversion
* refact: each initialize_interpreter overloads in pybind11 ns
* Move `initialize_interpreter_pre_pyconfig()` into the `detail` namespace.
Move the `PYBIND11_PYCONFIG_SUPPORT_PY_VERSION_HEX` define down to where it is used for the first time, and check if it is defined already, so that it is possible to customize from the compilation command line, just in case there is some unforeseen issue for Python 3.8, 3.9, 3.10.
* tests: Add program dir to path, Custom PyConfig with argv
* refact: clang-formatted
* tests: Add-program-dir-to-path covers both scoped_interpreter overloads
* tests: Add-program-dir-to-path fixed
* tests: Add-program-dir-to-path py_version dependant validation
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
* Use `multiprocessing` `start_method` `"forkserver"`
Alternative to PR #4305
* Add link to comment under PR #4105
* Unconditionally `pytest.skip("DEADLOCK")` for PyPy Windows
* Remove `SKIP_IF_DEADLOCK` entirely, for simplicity. Hopefully this PR will resolve the deadlocks for good.
* Add "In a nutshell" comment, in response to request by @EricCousineau-TRI
* Illustrate bug in functional.h
* style: pre-commit fixes
* Make functional casting more robust / add workaround
* Make function_record* casting even more robust
* See if this fixes PyPy issue
* It still fails on PyPy sadly
* Do not make new CTOR just yet
* Fix test
* Add name to ensure correctness
* style: pre-commit fixes
* Clean up tests + remove ifdef guards
* Add comments
* Improve comments, error handling, and safety
* Fix compile error
* Fix magic logic
* Extract helper function
* Fix func signature
* move to local internals
* style: pre-commit fixes
* Switch to simpler design
* style: pre-commit fixes
* Move to function_record
* style: pre-commit fixes
* Switch to internals, update tests and docs
* Fix lint
* Oops, forgot to resolve last comment
* Fix typo
* Update in response to comments
* Implement suggestion to improve test
* Update comment
* Simple fixes
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com>
* Illustrate bug in functional.h
* style: pre-commit fixes
* Make functional casting more robust / add workaround
* Make function_record* casting even more robust
* See if this fixes PyPy issue
* It still fails on PyPy sadly
* Do not make new CTOR just yet
* Fix test
* Add name to ensure correctness
* style: pre-commit fixes
* Clean up tests + remove ifdef guards
* Add comments
* Improve comments, error handling, and safety
* Fix compile error
* Fix magic logic
* Extract helper function
* Fix func signature
* move to local internals
* style: pre-commit fixes
* Switch to simpler design
* style: pre-commit fixes
* Move to function_record
* style: pre-commit fixes
* Switch to internals, update tests and docs
* Fix lint
* Oops, forgot to resolve last comment
* Fix typo
* Update in response to comments
* Implement suggestion to improve test
* Update comment
* Simple fixes
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com>
* Add option to force the use of the PYPY GIL scoped acquire/release logic to support nested gil access, see https://github.com/pybind/pybind11/issues/1276 and https://github.com/pytorch/pytorch/issues/83101
* Apply suggestions from code review
* Update CMakeLists.txt
* docs: update upgrade guide
* Update docs/upgrade.rst
* All bells & whistles.
* Add Reminder to common.h, so that we will not forget to purge `!WITH_THREAD` branches when dropping Python 3.6
* New sentence instead of semicolon.
* Temporarily pull in snapshot of PR #4246
* Add `test_release_acquire`
* Add more unit tests for nested gil locking
* Add test_report_builtins_internals_keys
* Very minor enhancement: sort list only after filtering.
* Revert change in docs/upgrade.rst
* Add test_multi_acquire_release_cross_module, while also forcing unique PYBIND11_INTERNALS_VERSION for cross_module_gil_utils.cpp
* Hopefully fix apparently new ICC error.
```
2022-10-28T07:57:54.5187728Z -- The CXX compiler identification is Intel 2021.7.0.20220726
...
2022-10-28T07:58:53.6758994Z icpc: remark #10441: The Intel(R) C++ Compiler Classic (ICC) is deprecated and will be removed from product release in the second half of 2023. The Intel(R) oneAPI DPC++/C++ Compiler (ICX) is the recommended compiler moving forward. Please transition to use this compiler. Use '-diag-disable=10441' to disable this message.
2022-10-28T07:58:54.5801597Z In file included from /home/runner/work/pybind11/pybind11/include/pybind11/detail/../detail/type_caster_base.h(15),
2022-10-28T07:58:54.5803794Z from /home/runner/work/pybind11/pybind11/include/pybind11/detail/../cast.h(15),
2022-10-28T07:58:54.5805740Z from /home/runner/work/pybind11/pybind11/include/pybind11/detail/../attr.h(14),
2022-10-28T07:58:54.5809556Z from /home/runner/work/pybind11/pybind11/include/pybind11/detail/class.h(12),
2022-10-28T07:58:54.5812154Z from /home/runner/work/pybind11/pybind11/include/pybind11/pybind11.h(13),
2022-10-28T07:58:54.5948523Z from /home/runner/work/pybind11/pybind11/tests/cross_module_gil_utils.cpp(13):
2022-10-28T07:58:54.5949009Z /home/runner/work/pybind11/pybind11/include/pybind11/detail/../detail/internals.h(177): error #2282: unrecognized GCC pragma
2022-10-28T07:58:54.5949374Z PYBIND11_TLS_KEY_INIT(tstate)
2022-10-28T07:58:54.5949579Z ^
2022-10-28T07:58:54.5949695Z
```
* clang-tidy fixes
* Workaround for PYPY WIN exitcode None
* Revert "Temporarily pull in snapshot of PR #4246"
This reverts commit 23ac16e859150f27fda25ca865cabcb4444e0770.
* Another workaround for PYPY WIN exitcode None
* Clean up how the tests are run "run in process" Part 1: uniformity
* Clean up how the tests are run "run in process" Part 2: use `@pytest.mark.parametrize` and clean up the naming.
* Skip some tests `#if defined(THREAD_SANITIZER)` (tested with TSAN using the Google-internal toolchain).
* Run all tests again but ignore ThreadSanitizer exitcode 66 (this is less likely to mask unrelated ThreadSanitizer issues in the future).
* bug fix: missing common.h include before using `PYBIND11_SIMPLE_GIL_MANAGEMENT`
For the tests in the github CI this does not matter, because
`PYBIND11_SIMPLE_GIL_MANAGEMENT` is always defined from the command line,
but when monkey-patching common.h locally, it matters.
* if process.exitcode is None: assert t_delta > 9.9
* More sophisiticated `_run_in_process()` implementation, clearly reporting `DEADLOCK`, additionally exercised via added `intentional_deadlock()`
* Wrap m.intentional_deadlock in a Python function, for `ForkingPickler` compatibility.
```
> ForkingPickler(file, protocol).dump(obj)
E TypeError: cannot pickle 'PyCapsule' object
```
Observed with all Windows builds including mingw but not PyPy, and macos-latest with Python 3.9, 3.10, 3.11 but not 3.6.
* Add link to potential solution for WOULD-BE-NICE-TO-HAVE feature.
* Add `SKIP_IF_DEADLOCK = True` option, to not pollute the CI results with expected `DEADLOCK` failures while we figure out what to do about them.
* Add COPY-PASTE-THIS: gdb ... command (to be used for debugging the detected deadlock)
* style: pre-commit fixes
* Do better than automatic pre-commit fixes.
* Add `PYBIND11_SIMPLE_GIL_MANAGEMENT` to `pytest_report_header()` (so that we can easily know when harvesting deadlock information from the CI logs).
Co-authored-by: Arnim Balzer <arnim@seechange.ai>
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* (bugfix): Improve bytes to str decoding error handling
* regroup test
* Further broaden tests
* Add another decode error test
* Fix bug in tests
* Reviewer suggestions
* First draft of Eigen::Tensor support
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix build errors
* Weird allocator stuff?
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove unused + additional allocator junk
* Disable warning
* Use constexpr
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* clang tidy fixes
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Resolve comments
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove auto constexpr function
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Try again for older C++
* Oops forgot constexpr
* Move to new files as suggested
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix weird tests
* Fix nits
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Oops, forgot import
* Fix clang 3.6 bug
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* More comprehensive test suite
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Refactor allocators to make things more clear
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Switch to std::copy
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Switch to DSizes instead of array
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address feedback
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix python + dummy c++ change to trigger build
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Alignment
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add include guard
* Forgot inline
* Fix compiler warning
* Remove bad test
* Better type signatures
* Add guards to make compiler requirements more explicit
* style: pre-commit fixes
* Force rerun of tests due to flake
* style: pre-commit fixes
* Keep pragmas & all related comments together, add PLEASE KEEP IN SYNC
* Move headers out of detail
* style: pre-commit fixes
* Fix cmake
* Improve casting
* style: pre-commit fixes
* Add a ton more tests + refactor
* Improve names
* style: pre-commit fixes
* Update include/pybind11/eigen/tensor.h
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
* Fix tests
* style: pre-commit fixes
* Update
* Add a test to verify that strange numpy arrays work
* Fix dumb compiler warning
* Better tests
* Better tests
* Fix tests
* style: pre-commit fixes
* More test fixes
* style: pre-commit fixes
* A ton more test coverage
* Fix tests
* style: pre-commit fixes
* style: pre-commit fixes
* Add back constexpr
* Another test
* style: pre-commit fixes
* Improve tests
* Whoops
* Less magic numbers
* Update tests/test_eigen_tensor.py
Co-authored-by: Sergiu Deitsch <sergiud@users.noreply.github.com>
* Update tests/test_eigen_tensor.py
Co-authored-by: Sergiu Deitsch <sergiud@users.noreply.github.com>
* style: pre-commit fixes
* Fix tests
* style: pre-commit fixes
* Fix memory leak
* style: pre-commit fixes
* Fix order
* style: pre-commit fixes
* Add test to make sure unsafe casts fail
* Minor bug fix to work on 32 bit machines
* Implement convert flag
* style: pre-commit fixes
* Switch to correct TensorMap const use
* style: pre-commit fixes
* Support older versions of eigen
* Weird c++ compilers
* Fix Eigen bug
* Fix another eigen bug
* Yet another eigen bug
* Potential flakes?
* style: pre-commit fixes
* Rerun tests with dummy exception to find out what is going on
* Another dummy test run
* Ablate more
* Found the broken test?
* One step closer
* one step further
* Double check
* one thing at a time
* Give up and disable the test
* Clang lies about being gcc
* Oops, fix matrix test
* style: pre-commit fixes
* Add tests to verify scalar conversions
* style: pre-commit fixes
* Fix nits
* Support no_array
* Fix tests
* style: pre-commit fixes
* Silence compiler warning
* Improve build system for ancient compilers
* Make clang happy
* Make gcc happy
* Implement Skylion's suggestions
* Fix warning
* Inline const pointer check
* Implement suggestions
* style: pre-commit fixes
* Improve tests
* Typo
* style: pre-commit fixes
* Support Google's build environment
* style: pre-commit fixes
* Update include/pybind11/eigen/tensor.h
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
* style: pre-commit fixes
* Test cleanup per Skylion
* Switch to remvove_cv_t
* Cleaner test
* style: pre-commit fixes
* Remove tensor from eigen.h, update tests
* style: pre-commit fixes
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com>
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
Co-authored-by: Sergiu Deitsch <sergiud@users.noreply.github.com>
* bugfix: contains now throws an exception if the key is not hashable
* Fix tests and improve robustness
* Remove todo
* Workaround PyPy corner case
* PyPy xfail
* Fix typo
* fix xfail
* Make clang-tidy happy
* Remove redundant exc checking
* Reproducer for issue encountered in smart_holder update.
* clang-tidy compatibility (untested).
* Add `enable_if_t` to workaround.
* Bug fix: Move `PYBIND11_USING_WORKAROUND_FOR_CUDA_11_4_THROUGH_8` determination to detail/common.h
So that it actually is defined in pybind11.h
* Try using the workaround (which is nicer than the original code) universally.
* Reduce reproducer for CUDA 11.7 issue encountered in smart_holder update.
This commit tested in isolation on top of current master + first version of reproducer (62311eb431).
Succeeds with Debian Clang 14.0.6 C++17 (and probably all other compilers).
Fails for CUDA 11.7:
```
cd /build/tests && /usr/local/cuda/bin/nvcc -forward-unknown-to-host-compiler -Dpybind11_tests_EXPORTS -I/mounted_pybind11/include -isystem=/usr/include/python3.10 -g --generate-code=arch=compute_52,code=[compute_52,sm_52] -Xcompiler=-fPIC -Xcompiler=-fvisibility=hidden -Werror all-warnings -std=c++17 -MD -MT tests/CMakeFiles/pybind11_tests.dir/test_class.cpp.o -MF CMakeFiles/pybind11_tests.dir/test_class.cpp.o.d -x cu -c /mounted_pybind11/tests/test_class.cpp -o CMakeFiles/pybind11_tests.dir/test_class.cpp.o
/mounted_pybind11/tests/test_class.cpp(53): error: more than one instance of overloaded function "pybind11::class_<type_, options...>::def [with type_=test_class::pr4220_tripped_over_this::Empty0, options=<>]" matches the argument list:
function template "pybind11::class_<test_class::pr4220_tripped_over_this::Empty0> &pybind11::class_<type_, options...>::def(const char *, Func &&, const Extra &...) [with type_=test_class::pr4220_tripped_over_this::Empty0, options=<>]"
/mounted_pybind11/include/pybind11/pybind11.h(1557): here
function template "pybind11::class_<test_class::pr4220_tripped_over_this::Empty0> &pybind11::class_<type_, options...>::def(const T &, const Extra &...) [with type_=test_class::pr4220_tripped_over_this::Empty0, options=<>]"
/mounted_pybind11/include/pybind11/pybind11.h(1586): here
argument types are: (const char [8], <unknown-type>)
object type is: pybind11::class_<test_class::pr4220_tripped_over_this::Empty0>
1 error detected in the compilation of "/mounted_pybind11/tests/test_class.cpp".
```
* fix: PyCapsule_GetDestructor is allowed to return a nullptr destructor
Previously, this code would error out if the destructor happened to be
a nullptr. This is incorrect. nullptrs are allowed for capsule
destructors.
"It is legal for a capsule to have a NULL destructor. This makes a
NULL return code somewhat ambiguous; use PyCapsule_IsValid() or
PyErr_Occurred() to disambiguate."
See:
https://docs.python.org/3/c-api/capsule.html#c.PyCapsule_GetDestructor
I noticed this while working on a type caster related to #3858 DLPack
happens to allow the destructor not to be defined on a capsule, and I
encountered such a case. See:
e2bdd3bee8/include/dlpack/dlpack.h (L219)
* Add test for the fix.
* Update tests/test_pytypes.cpp
I tried this locally and it works!
I never knew that there are cases where `reinterpret_cast` does not work but `static_cast` does. Let's see if all compilers are happy with this.
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
* style: pre-commit fixes
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgkio@gmail.com>
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Disable test triggering ASAN failure (to pin-point where the problem is).
* Fix unsafe "block" implementation in test_eigen.cpp
* Undo changes (i.e. revert back to master).
* Detect "type_caster for Eigen::Ref made a copy."
This is achieved without
* reaching into internals,
* making test_eigen.cpp depend on pybind11/numpy.h.
* Add comment pointing to PR, for easy reference.
* Add debug fprintf to test_interpreter.cpp
* Update `sys.path` from `PYTHONPATH` in Python >= 3.11 branch of `initialize_interpreter()`
* Use `config.isolated = 0; config.use_environment = 1;`
As suggsted by @vstinner here: https://github.com/pybind/pybind11/pull/4119#issuecomment-1219442853
* Add `TEST_CASE("PYTHONPATH is used to update sys.path")`
* Fix clang-tidy error.
* Use `_putenv_s()` under Windows.
* Fix clang-tidy error: argument name ... in comment does not match parameter name
* Remove slash from PYTHONPATH addition, to work around Windows slash-vs-backslash issue.
* Use `py::str(...)` instead of `.attr("__str__")` as suggested by @skylion007
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
* add installation support for pkg-config dependency detection
pkg-config is a buildsystem-agnostic alternative to
`pybind11Config.cmake` that can be used from build systems other than
cmake.
Fixes#230
* tests: add test for pkg config
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
* Add `-DPYBIND11_WERROR=ON` to mingw cmake commands (and `-DCMAKE_VERBOSE_MAKEFILE:BOOL=ON`).
* Using no-destructor idiom to side-step overzealous MINGW warning.
* Add __MINGW32__ pragma GCC diagnostic ignored in eigen.h
* Add another no-destructor workaround.
* Temporarily add -k (keep-going) flags to hopefully speed up finding all warnings.
* Revert "Temporarily add -k (keep-going) flags to hopefully speed up finding all warnings."
This reverts commit f36b0af8f9.
* Very minor shuffle to avoid MSVC warnings.
* Remove all `:BOOL` as suggested by @henryiii
* Insert type_caster_odr_guard<> (an empty struct to start with).
* Add odr_guard_registry() used in type_caster_odr_guard() default constructor.
* Add minimal_real_caster (from PR #3862) to test_async, test_buffers
* VERY MESSY SNAPSHOT of WIP, this was the starting point for cl/454658864, which has more changes on top.
* Restore original test_async, test_buffers from current smart_holder HEAD
* Copy from cl/454991845 snapshot Jun 14, 5:08 PM
* Cleanup of tests. Systematically insert `if (make_caster<T>::translation_unit_local) {`
* Small simplification of odr_guard_impl()
* WIP
* Add PYBIND11_SOURCE_FILE_LINE macro.
* Replace PYBIND11_TYPE_CASTER_UNIQUE_IDENTIFIER with PYBIND11_TYPE_CASTER_SOURCE_FILE_LINE, baked into PYBIND11_TYPE_CASTER macro.
* Add more PYBIND11_DETAIL_TYPE_CASTER_ACCESS_TRANSLATION_UNIT_LOCAL; resolves "unused" warning when compiling test_custom_type_casters.cpp
* load_type fixes & follow-on cleanup
* Strip ./ from source_file_line
* Add new tests to CMakeLists.txt, disable PYBIND11_WERROR
* Replace C++17 syntax. Compiles with Debian clang 13 C++11 mode, but fails to link. Trying GitHub Actions anyway to see if there are any platforms that support https://en.cppreference.com/w/cpp/language/tu_local before C++20. Note that Debian clang 13 C++17 works locally.
* Show C++ version along with ODR VIOLATION DETECTED message.
* Add source_file_line_basename()
* Introduce PYBIND11_TYPE_CASTER_ODR_GUARD_ON (but not set automatically).
* Minor cleanup.
* Set PYBIND11_TYPE_CASTER_ODR_GUARD_ON automatically.
* Resolve clang-tidy error.
* Compatibility with old compilers.
* Fix off-by-one in source_file_line_basename()
* Report PYBIND11_INTERNALS_ID & C++ Version from pytest_configure()
* Restore use of PYBIND11_WERROR
* Move cpp_version_in_use() from cast.h to pybind11_tests.cpp
* define PYBIND11_DETAIL_ODR_GUARD_IMPL_THROW_DISABLED true in test_odr_guard_1,2.cpp
* IWYU cleanup of detail/type_caster_odr_guard.h
* Replace `throw err;` to resolve clang-tidy error.
* Add new header filename to CMakeLists.txt, test_files.py
* Experiment: Try any C++17 compiler.
* Fix ifdef for pragma GCC diagnostic.
* type_caster_odr_guard_impl() cleanup
* Move type_caster_odr_guard to type_caster_odr_guard.h
* Rename test_odr_guard* to test_type_caster_odr_guard*
* Remove comments that are (now) more distracting than helpful.
* Mark tu_local_no_data_always_false operator bool as explicit (clang-tidy). See also: https://stackoverflow.com/questions/39995573/when-can-i-use-explicit-operator-bool-without-a-cast
* New PYBIND11_TYPE_CASTER_ODR_GUARD_STRICT option (current on by default).
* Add test_type_caster_odr_registry_values(), test_type_caster_odr_violation_detected_counter()
* Report UNEXPECTED: test_type_caster_odr_guard_2.cpp prevailed (but do not fail).
* Apply clang-tidy suggestion.
* Attempt to handle valgrind behavior.
* Another attempt to handle valgrind behavior.
* Yet another attempt to handle valgrind behavior.
* Trying a new direction: show compiler info & std for UNEXPECTED: type_caster_odr_violation_detected_count() == 0
* compiler_info MSVC fix. num_violations == 0 condition.
* assert pybind11_tests.compiler_info is not None
* Introduce `make_caster_intrinsic<T>`, to be able to undo the 2 changes from `load_type` to `load_type<T>`. This is to avoid breaking 2 `pybind11::detail::load_type()` calls found in the wild (Google global testing).
One of the breakages in the wild was: 0f0f600767/python/tensorstore/subscript_method.h (L61)
* Add test for stl.h / stl_bind.h mix.
Manually verified that the ODR guard detects the ODR violation:
```
C++ Info: Debian Clang 13.0.1 C++17 __pybind11_internals_v4_clang_libstdcpp_cxxabi1002_sh_def__
=========================================================== test session starts ============================================================
platform linux -- Python 3.9.12, pytest-6.2.3, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3
...
================================================================= FAILURES =================================================================
_____________________________________________ test_type_caster_odr_violation_detected_counter ______________________________________________
def test_type_caster_odr_violation_detected_counter():
...
else:
> assert num_violations == 1
E assert 2 == 1
E +2
E -1
num_violations = 2
test_type_caster_odr_guard_1.py:51: AssertionError
========================================================= short test summary info ==========================================================
FAILED test_type_caster_odr_guard_1.py::test_type_caster_odr_violation_detected_counter - assert 2 == 1
======================================================= 1 failed, 5 passed in 0.08s ========================================================
```
* Eliminate need for `PYBIND11_DETAIL_TYPE_CASTER_ACCESS_TRANSLATION_UNIT_LOCAL` macro.
Copying code first developed by @amauryfa. I tried this at an earlier stage, but by itself this was insufficient. In the meantime I added in the TU-local mechanisms: trying again.
Passes local testing:
```
DISABLED std::system_error: ODR VIOLATION DETECTED: pybind11::detail::type_caster<mrc_ns::type_mrc>: SourceLocation1="/usr/local/google/home/rwgk/forked/pybind11/tests/test_type_caster_odr_guard_1.cpp:18", SourceLocation2="/usr/local/google/home/rwgk/forked/pybind11/tests/test_type_caster_odr_guard_2.cpp:19"
C++ Info: Debian Clang 13.0.1 C++17 __pybind11_internals_v4_clang_libstdcpp_cxxabi1002_sh_def__
=========================================================== test session starts ============================================================
platform linux -- Python 3.9.12, pytest-6.2.3, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /usr/local/google/home/rwgk/forked/pybind11/tests, configfile: pytest.ini
collected 6 items
test_type_caster_odr_guard_1.py::test_type_mrc_to_python PASSED
test_type_caster_odr_guard_1.py::test_type_mrc_from_python PASSED
test_type_caster_odr_guard_1.py::test_type_caster_odr_registry_values PASSED
test_type_caster_odr_guard_1.py::test_type_caster_odr_violation_detected_counter PASSED
test_type_caster_odr_guard_2.py::test_type_mrc_to_python PASSED
test_type_caster_odr_guard_2.py::test_type_mrc_from_python PASSED
============================================================ 6 passed in 0.01s =============================================================
```
* tu_local_descr with src_loc experiment
* clang-tidy suggested fixes
* Use source_file_line_from_sloc in type_caster_odr_guard_registry
* Disable type_caster ODR guard for __INTEL_COMPILER (see comment). Also turn off printf.
* Add missing include (discovered via google-internal testing).
* Work `scr_loc` into `descr`
* Use `TypeCasterType::name.sloc` instead of `source_file_line.sloc`
Manual re-verification:
```
+++ b/tests/test_type_caster_odr_guard_2.cpp
- // m.def("pass_vector_type_mrc", mrc_ns::pass_vector_type_mrc);
+ m.def("pass_vector_type_mrc", mrc_ns::pass_vector_type_mrc);
```
```
> assert num_violations == 1
E assert 2 == 1
num_violations = 2
test_type_caster_odr_guard_1.py:51: AssertionError
```
* Fix small oversight (src_loc::here() -> src_loc{nullptr, 0}).
* Remove PYBIND11_DETAIL_TYPE_CASTER_ACCESS_TRANSLATION_UNIT_LOCAL macro completely.
* Remove PYBIND11_TYPE_CASTER_SOURCE_FILE_LINE macro completely. Some small extra cleanup.
* Minor tweaks looking at the PR with a fresh eye.
* src_loc comments
* Add new test_descr_src_loc & and fix descr.h `concat()` `src_loc` bug discovered while working on the test.
* Some more work on source code comments.
* Fully document the ODR violations in the ODR guard itself and introduce `PYBIND11_TYPE_CASTER_ODR_GUARD_ON_IF_AVAILABLE`
* Update comment (incl. mention of deadsnakes known to not work as intended).
* Use no-destructor idiom for type_caster_odr_guard_registry, as suggested by @laramiel
* Fix clang-tidy error: 'auto reg' can be declared as 'auto *reg' [readability-qualified-auto,-warnings-as-errors]
* WIP
* Revert "WIP" (tu_local_no_data_always_false_base experiment).
This reverts commit 31e8ac562f.
* Change `PYBIND11_TYPE_CASTER_ODR_GUARD_ON` to `PYBIND11_ENABLE_TYPE_CASTER_ODR_GUARD`, based on a suggestion by @rainwoodman
* Improved `#if` determining `PYBIND11_ENABLE_TYPE_CASTER_ODR_GUARD`, based on suggestion by @laramiel
* Make `descr::sloc` `const`, as suggested by @rainwoodman
* Rename macro to `PYBIND11_DETAIL_TYPE_CASTER_ODR_GUARD_IMPL_DEBUG`, as suggested by @laramiel
* Tweak comments some more (add "white hat hacker" analogy).
* Bring back `PYBIND11_CPP17` in determining `PYBIND11_ENABLE_TYPE_CASTER_ODR_GUARD`, to hopefully resolve most if not all of the many CI failures (89 failing, 32 successful: https://github.com/pybind/pybind11/runs/7430295771).
* Try another workaround for `__has_builtin`-related breakages (https://github.com/pybind/pybind11/runs/7430720321).
* Remove `defined(__has_builtin)` and subconditions.
* Update "known to not work" expectation in test and comment.
* `pytest.skip` `num_violations == 0` only `#ifdef __NO_INLINE__` (irrespective of the compiler)
* Systematically change all new `#ifdef` to `#if defined` (review suggestion).
* Bring back MSVC comment that got lost while experimenting.
* For PyPy only, re-enable old behavior (likely to mask bugs), to avoid segfault with unknown root cause.
Change prompted by https://github.com/pybind/pybind11/issues/4075
* Undo the change in tests/test_exceptions.py
I turns out (I forgot) that PyPy segfaults in `test_flaky_exception_failure_point_init` already before the `MISMATCH` code path is reached:
https://github.com/pybind/pybind11/runs/7383663596
```
RPython traceback:
test_exceptions.py .......X.........Error in cpyext, CPython compatibility layer:
File "pypy_module_cpyext.c", line 14052, in wrapper_second_level__star_3_1
File "pypy_module_cpyext_1.c", line 35750, in not_supposed_to_fail
Fatal Python error: Segmentation fault
Stack (most recent call first, approximate line numbers):
File "/home/runner/work/pybind11/pybind11/tests/test_exceptions.py", line 306 in test_flaky_exception_failure_point_init
The function PyErr_NormalizeException was not supposed to fail
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/python.py", line 185 in pytest_pyfunc_call
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_callers.py", line 9 in _multicall
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_manager.py", line 77 in _hookexec
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_hooks.py", line 244 in __call__
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/python.py", line 1716 in runtest
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/runner.py", line 159 in pytest_runtest_call
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_callers.py", line 9 in _multicall
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_manager.py", line 77 in _hookexec
Fatal error in cpyext, CPython compatibility layer, calling PyErr_NormalizeException
Either report a bug or consider not using this particular extension
<SystemError object at 0x7fcc8cea6868>
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_hooks.py", line 244 in __call__
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/runner.py", line 261 in <lambda>
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/runner.py", line 317 in from_call
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/runner.py", line 246 in call_runtest_hook
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/runner.py", line 218 in call_and_report
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/runner.py", line 118 in runtestprotocol
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/runner.py", line 110 in pytest_runtest_protocol
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_callers.py", line 9 in _multicall
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_manager.py", line 77 in _hookexec
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_hooks.py", line 244 in __call__
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/main.py", line 335 in pytest_runtestloop
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_callers.py", line 9 in _multicall
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_manager.py", line 77 in _hookexec
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_hooks.py", line 244 in __call__
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/main.py", line 318 in _main
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/main.py", line 255 in wrap_session
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/main.py", line 314 in pytest_cmdline_main
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_callers.py", line 9 in _multicall
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_manager.py", line 77 in _hookexec
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_hooks.py", line 244 in __call__
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/config/__init__.py", line 133 in main
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/config/__init__.py", line 181 in console_main
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pytest/__main__.py", line 1 in <module>
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/lib-python/3/runpy.py", line 62 in _run_code
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/lib-python/3/runpy.py", line 170 in _run_module_as_main
File "<builtin>/app_main.py", line 109 in run_toplevel
File "<builtin>/app_main.py", line 652 in run_command_line
File "<builtin>/app_main.py", line 996 in entry_point
Segmentation fault (core dumped)
```
* Add test_pypy_oserror_normalization
* Disable new `PYPY_VERSION` `#if`, to verify that the new test actually fails.
* Restore PYPY_VERSION workaround and update comment to reflect what was learned.
* [ci skip] Fix trivial oversight in comment.
* Disable implicit conversion from `0` to `pybind11::handle`.
* Reverse or-ed condition in an attempt to resolve GCC 8.3.0 errors (i386/debian:buster).
* Trying the simpler `std::is_same<T, PyObject *>`
* Add implicit_conversion_from_pytorch_THPObjectPtr_to_handle test.
* Accommodate types with implicit conversions to `PyObject *`, other than `handle` & `handle` subclasses, or integral types.
* Fix copy-paste mishap (picked wrong name).
* Revamp SFINAE construct to actually fix the pytorch issue (already validated against pytorch proper).
The first version of the reduced pytorch code was critically missing the move ctor. The first version of the accompanying test was meaningless.
Note: It turns out the `!std::is_arithmetic<T>` condition is not needed: `int` is not in general implicitly convertible to `PyObject *`, only the literal `0` is.
* Use `NOLINT(performance-noexcept-move-constructor)` for reduced code from the wild (rather than changing the code).
* Use any_of, all_of, negation. It turns out to clang-format nicer.
* Clean up comments for changed code.
* Reduce pytorch situation further, add test for operator ... const.
* Use `none_of` as suggested by @skylion007
* Add `pure_compile_tests_for_handle_from_PyObject_pointers()`
* Fix inconsequential oversight (retested).
* Factor our `is_pyobj_ptr_or_nullptr_t` to make the SFINAE conditions more readable.
* Remove stray line (oversight).
* Make the `pure_compile_tests_for_handle_from_PyObject_pointers()` "rhs-const-complete", too.
* Remove the temporary PYBIND11_UNDO_PR4008 `#ifdef`.
* Add test_namespace_visibility
To probe environment/toolchain/platform-specific behavior under the exact same conditions as normal tests.
(An earlier version of this code was used to inform PR #4043.)
* Disable flake8 in ubench/holder_comparison_*.py, to suppress new & useless diagnostics.
* Disable namespace_visibility_1s.cpp (tosee if that resolves the MSVC and CUDA `test_cross_module_exception_translator` failures).
* Turn off flake8 completely for ubench (the Strip unnecessary `# noqa`s action un-helpfully removed the added noqa).
* Disable test_namespace_visibility completely. Just keep the two .cpp files, only setting the module docstring and doing nothing else.
* Rename test_namespace_visibility.py to test_exc_namespace_visibility.py, so that it is imported by pytest before test_exceptions.py
* Add `set_property(SOURCE namespace_visibility_1s.cpp PROPERTY LANGUAGE CUDA)`
* Add reference to PR #4054
* Complete the documentation (comments in test_exc_namespace_visibility.py).
* Rename namespace_visibility.h to namespace_visibility.inl, as suggested by @charlesbeattie
* Placeholder commit for 3.11 testing
* Does this fix it?
* Try suggestion
* Placeholder commit for 3.11 testing
* Does this fix it?
* Try suggestion
* fix: try using modern init for embedded interp
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* fix: error message changed in 3.11
* fix: apply logic in Python manually
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* fix autodetect dynamic attrs in 3.11
* fix: include error message if possible in error
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* ci: enable standard Python 3.11 testing
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* Make dynamic attrs condtiion exclusive to ver.
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
When converting an array to an Eigen matrix, ignore the strides if any
dimension size is 0. If the array is empty, the strides aren't relevant,
and especially numpy ≥ 1.23 explicitly sets the strides to 0 in this
case. (See numpy commit dd5ab7b11520.)
Update tests to verify that this works, and continues to work.
* error_already_set::what() is now constructed lazily
Prior to this commit throwing error_already_set was expensive due to the
eager construction of the error string (which required traversing the
Python stack). See #1853 for more context and an alternative take on the
issue.
Note that error_already_set no longer inherits from std::runtime_error
because the latter has no default constructor.
* Do not attempt to normalize if no exception occurred
This is not supported on PyPy-2.7 5.8.0.
* Extract exception name via tp_name
This is faster than dynamically looking up __name__ via GetAttrString.
Note though that the runtime of the code throwing an error_already_set
will be dominated by stack unwinding so the improvement will not be
noticeable.
Before:
396 ns ± 0.913 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
After:
277 ns ± 0.549 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
Benchmark:
const std::string foo() {
PyErr_SetString(PyExc_KeyError, "");
const std::string &s = py::detail::error_string();
PyErr_Clear();
return s;
}
PYBIND11_MODULE(foo, m) {
m.def("foo", &::foo);
}
* Reverted error_already_set to subclass std::runtime_error
* Revert "Extract exception name via tp_name"
The implementation of __name__ is slightly more complex than that.
It handles the module name prefix, and heap-allocated types. We could
port it to pybind11 later on but for now it seems like an overkill.
This reverts commit f1435c7e6b.
* Cosmit following @YannickJadoul's comments
Note that detail::error_string() no longer calls PyException_SetTraceback
as it is unncessary for pretty-printing the exception.
* Fixed PyPy build
* Moved normalization to error_already_set ctor
* Fix merge bugs
* Fix more merge errors
* Improve formatting
* Improve error message in rare case
* Revert back if statements
* Fix clang-tidy
* Try removing mutable
* Does build_mode release fix it
* Set to Debug to expose segfault
* Fix remove set error string
* Do not run error_string() more than once
* Trying setting the tracebackk to the value
* guard if m_type is null
* Try to debug PGI
* One last try for PGI
* Does reverting this fix PyPy
* Reviewer suggestions
* Remove unnecessary initialization
* Add noexcept move and explicit fail throw
* Optimize error_string creation
* Fix typo
* Revert noexcept
* Fix merge conflict error
* Abuse assignment operator
* Revert operator abuse
* See if we still need debug
* Remove unnecessary mutable
* Report "FATAL failure building pybind11::error_already_set error_string" and terminate process.
* Try specifying noexcept again
* Try explicit ctor
* default ctor is noexcept too
* Apply reviewer suggestions, simplify code, and make helper method private
* Remove unnecessary include
* Clang-Tidy fix
* detail::obj_class_name(), fprintf with [STDERR], [STDOUT] tags, polish comments
* consistently check m_lazy_what.empty() also in production builds
* Make a comment slightly less ambiguous.
* Bug fix: Remove `what();` from `restore()`.
It sure would need to be guarded by `if (m_type)`, otherwise `what()` fails and masks that no error was set (see update unit test). But since `error_already_set` is copyable, there is no point in releasing m_type, m_value, m_trace, therefore we can just as well avoid the runtime overhead of force-building `m_lazy_what`, it may never be used.
* Replace extremely opaque (unhelpful) error message with a truthful reflection of what we know.
* Fix clang-tidy error [performance-move-constructor-init].
* Make expected error message less specific.
* Various changes.
* bug fix: error_string(PyObject **, ...)
* Putting back the two eager PyErr_NormalizeException() calls.
* Change error_already_set() to call pybind11_fail() if the Python error indicator not set. The net result is that a std::runtime_error is thrown instead of error_already_set, but all tests pass as is.
* Remove mutable (fixes oversight in the previous commit).
* Normalize the exception only locally in error_string(). Python 3.6 & 3.7 test failures expected. This is meant for benchmarking, to determine if it is worth the trouble looking into the failures.
* clang-tidy: use auto
* Use `gil_scoped_acquire_local` in `error_already_set` destructor. See long comment.
* For Python < 3.8: `PyErr_NormalizeException` before `PyErr_WriteUnraisable`
* Go back to replacing the held Python exception with then normalized exception, if & when needed. Consistently document the side-effect.
* Slightly rewording comment. (There were also other failures.)
* Add 1-line comment for obj_class_name()
* Benchmark code, with results in this commit message.
function #calls test time [s] μs / call
master pure_unwind 729540 1.061 14.539876
err_set_unwind_err_clear 681476 1.040 15.260282
err_set_error_already_set 508038 1.049 20.640525
error_already_set_restore 555578 1.052 18.933288
pr1895_original_foo 244113 1.050 43.018168
PR / master
PR #1895 pure_unwind 736981 1.054 14.295685 98.32%
err_set_unwind_err_clear 685820 1.045 15.237399 99.85%
err_set_error_already_set 661374 1.046 15.811879 76.61%
error_already_set_restore 669881 1.048 15.645176 82.63%
pr1895_original_foo 318243 1.059 33.290806 77.39%
master @ commit ad146b2a18
Running tests in directory "/usr/local/google/home/rwgk/forked/pybind11/tests":
============================= test session starts ==============================
platform linux -- Python 3.9.10, pytest-6.2.3, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /usr/local/google/home/rwgk/forked/pybind11/tests, configfile: pytest.ini
collecting ... collected 5 items
test_perf_error_already_set.py::test_perf[pure_unwind]
PERF pure_unwind,729540,1.061,14.539876
PASSED
test_perf_error_already_set.py::test_perf[err_set_unwind_err_clear]
PERF err_set_unwind_err_clear,681476,1.040,15.260282
PASSED
test_perf_error_already_set.py::test_perf[err_set_error_already_set]
PERF err_set_error_already_set,508038,1.049,20.640525
PASSED
test_perf_error_already_set.py::test_perf[error_already_set_restore]
PERF error_already_set_restore,555578,1.052,18.933288
PASSED
test_perf_error_already_set.py::test_perf[pr1895_original_foo]
PERF pr1895_original_foo,244113,1.050,43.018168
PASSED
============================== 5 passed in 12.38s ==============================
pr1895 @ commit 8dff51d12e
Running tests in directory "/usr/local/google/home/rwgk/forked/pybind11/tests":
============================= test session starts ==============================
platform linux -- Python 3.9.10, pytest-6.2.3, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /usr/local/google/home/rwgk/forked/pybind11/tests, configfile: pytest.ini
collecting ... collected 5 items
test_perf_error_already_set.py::test_perf[pure_unwind]
PERF pure_unwind,736981,1.054,14.295685
PASSED
test_perf_error_already_set.py::test_perf[err_set_unwind_err_clear]
PERF err_set_unwind_err_clear,685820,1.045,15.237399
PASSED
test_perf_error_already_set.py::test_perf[err_set_error_already_set]
PERF err_set_error_already_set,661374,1.046,15.811879
PASSED
test_perf_error_already_set.py::test_perf[error_already_set_restore]
PERF error_already_set_restore,669881,1.048,15.645176
PASSED
test_perf_error_already_set.py::test_perf[pr1895_original_foo]
PERF pr1895_original_foo,318243,1.059,33.290806
PASSED
============================== 5 passed in 12.40s ==============================
clang++ -o pybind11/tests/test_perf_error_already_set.os -c -std=c++17 -fPIC -fvisibility=hidden -Os -flto -Wall -Wextra -Wconversion -Wcast-qual -Wdeprecated -Wnon-virtual-dtor -Wunused-result -isystem /usr/include/python3.9 -isystem /usr/include/eigen3 -DPYBIND11_STRICT_ASSERTS_CLASS_HOLDER_VS_TYPE_CASTER_MIX -DPYBIND11_TEST_BOOST -Ipybind11/include -I/usr/local/google/home/rwgk/forked/pybind11/include -I/usr/local/google/home/rwgk/clone/pybind11/include /usr/local/google/home/rwgk/forked/pybind11/tests/test_perf_error_already_set.cpp
clang++ -o lib/pybind11_tests.so -shared -fPIC -Os -flto -shared ...
Debian clang version 13.0.1-3+build2
Target: x86_64-pc-linux-gnu
Thread model: posix
* Changing call_repetitions_target_elapsed_secs to 0.1 for regular unit testing.
* Adding in `recursion_depth`
* Optimized ctor
* Fix silly bug in recurse_first_then_call()
* Add tests that have equivalent PyErr_Fetch(), PyErr_Restore() but no try-catch.
* Add call_error_string to tests. Sample only recursion_depth 0, 100.
* Show lazy-what speed-up in percent.
* Include real_work in benchmarks.
* Replace all PyErr_SetString() with generate_python_exception_with_traceback()
* Better organization of test loops.
* Add test_error_already_set_copy_move
* Fix bug in newly added test (discovered by clang-tidy): actually use move ctor
* MSVC detects the unreachable return
* change test_perf_error_already_set.py back to quick mode
* Inherit from std::exception (instead of std::runtime_error, which does not make sense anymore with the lazy what)
* Special handling under Windows.
* print with leading newline
* Removing test_perf_error_already_set (copies are under 7765113fbb).
* Avoid gil and scope overhead if there is nothing to release.
* Restore default move ctor. "member function" instead of "function" (note that "method" is Python terminology).
* Delete error_already_set copy ctor.
* Make restore() non-const again to resolve clang-tidy failure (still experimenting).
* Bring back error_already_set copy ctor, to see if that resolves the 4 MSVC test failures.
* Add noexcept to error_already_set copy & move ctors (as suggested by @skylion007 IIUC).
* Trying one-by-one noexcept copy ctor for old compilers.
* Add back test covering copy ctor. Add another simple test that exercises the copy ctor.
* Exclude more older compilers from using the noexcept = default ctors. (The tests in the previous commit exposed that those are broken.)
* Factor out & reuse gil_scoped_acquire_local as gil_scoped_acquire_simple
* Guard gil_scoped_acquire_simple by _Py_IsFinalizing() check.
* what() GIL safety
* clang-tidy & Python 3.6 fixes
* Use `gil_scoped_acquire` in dtor, copy ctor, `what()`. Remove `_Py_IsFinalizing()` checks (they are racy: https://github.com/python/cpython/pull/28525).
* Remove error_scope from copy ctor.
* Add `error_scope` to `get_internals()`, to cover the situation that `get_internals()` is called from the `error_already_set` dtor while a new Python error is in flight already. Also backing out `gil_scoped_acquire_simple` change.
* Add `FlakyException` tests with failure triggers in `__init__` and `__str__`
THIS IS STILL A WORK IN PROGRESS. This commit is only an important resting point.
This commit is a first attempt at addressing the observation that `PyErr_NormalizeException()` completely replaces the original exception if `__init__` fails. This can be very confusing even in small applications, and extremely confusing in large ones.
* Tweaks to resolve Py 3.6 and PyPy CI failures.
* Normalize Python exception immediately in error_already_set ctor.
For background see: https://github.com/pybind/pybind11/pull/1895#issuecomment-1135304081
* Fix oversights based on CI failures (copy & move ctor initialization).
* Move @pytest.mark.xfail("env.PYPY") after @pytest.mark.parametrize(...)
* Use @pytest.mark.skipif (xfail does not work for segfaults, of course).
* Remove unused obj_class_name_or() function (it was added only under this PR).
* Remove already obsolete C++ comments and code that were added only under this PR.
* Slightly better (newly added) comments.
* Factor out detail::error_fetch_and_normalize. Preparation for producing identical results from error_already_set::what() and detail::error_string(). Note that this is a very conservative refactoring. It would be much better to first move detail::error_string into detail/error_string.h
* Copy most of error_string() code to new error_fetch_and_normalize::complete_lazy_error_string()
* Remove all error_string() code from detail/type_caster_base.h. Note that this commit includes a subtle bug fix: previously error_string() restored the Python error, which will upset pybind11_fail(). This never was a problem in practice because the two PyType_Ready() calls in detail/class.h do not usually fail.
* Return const std::string& instead of const char * and move error_string() to pytypes.h
* Remove gil_scope_acquire from error_fetch_and_normalize, add back to error_already_set
* Better handling of FlakyException __str__ failure.
* Move error_fetch_and_normalize::complete_lazy_error_string() implementation from pybind11.h to pytypes.h
* Add error_fetch_and_normalize::release_py_object_references() and use from error_already_set dtor.
* Use shared_ptr for m_fetched_error => 1. non-racy, copy ctor that does not need the GIL; 2. enables guard against duplicate restore() calls.
* Add comments.
* Trivial renaming of a newly introduced member function.
* Workaround for PyPy
* Bug fix (oversight). Only valgrind got this one.
* Use shared_ptr custom deleter for m_fetched_error in error_already_set. This enables removing the dtor, copy ctor, move ctor completely.
* Further small simplification. With the GIL held, simply deleting the raw_ptr takes care of everything.
* IWYU cleanup
```
iwyu version: include-what-you-use 0.17 based on Debian clang version 13.0.1-3+build2
```
Command used:
```
iwyu -c -std=c++17 -DPYBIND11_TEST_BOOST -Iinclude/pybind11 -I/usr/include/python3.9 -I/usr/include/eigen3 include/pybind11/pytypes.cpp
```
pytypes.cpp is a temporary file: `#include "pytypes.h"`
The raw output is very long and noisy.
I decided to use `#include <cstddef>` instead of `#include <cstdio>` for `std::size_t` (iwyu sticks to the manual choice).
I ignored all iwyu suggestions that are indirectly covered by `#include <Python.h>`.
I manually verified that all added includes are actually needed.
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
* Add test_perf_accessors (to be merged into test_pytypes).
* Python < 3.8 f-string compatibility
* Use thread_local in inc_ref_counter()
* Intentional breakage, brute-force way to quickly find out how many platforms reach the PYBIND11_HANDLE_REF_DEBUG code, with and without threads.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove Intentional breakage
* Drop perf test, move inc_refs tests to test_pytypes
* Fold in PR #3970 with `#ifdef`s
* Complete test coverage for all newly added code.
* Condense new unit tests via a simple local helper macro.
* Remove PYBIND11_PR3970 define. See https://github.com/pybind/pybind11/pull/3977#issuecomment-1142526417
* Move static keyword first (fixes silly oversight).
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add missing error handling to module_::def_submodule
* Add test_def_submodule_failures
* PyPy only: Skip test with trigger for PyModule_GetName() failure.
* Reapply minor fix that accidentally got lost in transfer from PR #3964
* Add error_already_set_what what tests, asserting the status quo.
* Move PyErr_NormalizeException() up a few lines.
* @pytest.mark.skipif("env.PYPY") from PR #1895 is required even for this much simpler PR
* Move PyException_SetTraceback() with PyErr_NormalizeException() as suggested by @skylion007
* Insert a std::move() as suggested by @skylion007
* Don't return pointers to static objects with return_value_policy::take_ownership.
This fixes -Wfree-nonheap-object warnings produced by GCC.
* Use return value policy fix instead
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
* Transferred net diff from PR #3581, as-is.
* Automatic `pre-commit run --all-files` fixes. NO manual changes.
* Removing trailing `//` (originally added to manipulate clang-format), as suggested by @charlesbeattie back in Jan/Feb under PR #3581.
* Renaming `xetter_cpp_function` to `property_cpp_function` as suggested by @rainwoodman
* Fully explain the terse variable naming scheme in test_class_sh_property (as suggested by @rainwoodman)
* Also use parametrize for readonly, readwrite (as suggested by @rainwoodman)
* Apply change suggested by @skylion007 (with clang-format).
* Fix py::cast from pytype rvalue to pytype
Previously, py::cast blindly assumed that the destination type was a C++
type rather than a python type when the source type was an rvalue.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add frozenset, and allow it cast to std::set
For the reverse direction, std::set still casts to set. This is in concordance with the behavior for sequence containers, where e.g. tuple casts to std::vector but std::vector casts to list.
Extracted from #3886.
* Rename set_base to any_set to match Python C API
since this will be part of pybind11 public API
* PR: static_cast, anyset
* Add tests for frozenset
and rename anyset methods
* Remove frozenset default ctor, add tests
Making frozenset non-default constructible means that we need to adjust pyobject_caster to not require that its value is default constructible, by initializing value to a nil handle. This also allows writing C++ functions taking anyset, and is arguably a performance improvement, since there is no need to allocate an object that will just be replaced by load.
Add some more tests, including anyset::empty, anyset::size, set::add and set::clear.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add rationale to `pyobject_caster` default ctor
* Remove ineffectual protected: access control
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Test pickling a simple callable (does not work).
Currently only documents that it does not work. Starting point for future fix.
* Use re.search to accommodate variations of the TypeError message.
* PyPy: exercise full dumps/loads cycle.
* Adding explicit "broken" comment.
* Update cast.h
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Move definition to detail/common, change name, apply everywhere
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Rename debug_enabled in tests to detailed_error_messages_enabled
* Add return_as_bytes policy
* Fix format
* Fix test failures
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix std::variant
* Resolve comments
* Note this policy experimental
* Add tests for `return_as_bytes` with `def_property`.
* Change comment for the new return_as_bytes enum to note that the policy is not available on master.
* Applying pr3838_sh.patch (exactly as used Google-internally since 2022-03-31).
* Add `case return_as_bytes` to `switch`es in detail/type_caster_base.h and eigen.h
Based on systematic review under https://github.com/pybind/pybind11/pull/3838#issuecomment-1094390333
* Add missing break (clang-tidy).
* More clang-tidy fixes (this time around clang-tidy was run interactively to pre-empt repeat trips through the CI).
* Underscore prefix: _return_as_bytes
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
* Added constructor based on typenum, based on PyArray_DescrFromType
Added accessors for typenum, alignment, byteorder and flags fields of
PyArray_Descr struct.
* Added tests for new py::dtype constructor, and for accessors
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fixed the comment for alignment method
* Update include/pybind11/numpy.h
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
* Adds set_name method of pybind11::capsule class
This calls PyCapsule_SetName on the underlying capsule object.
modified destructors to query capsules's Name
[pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Handle possible exception thrown by PyCapsule_GetName
Also removed accidentally reintroduced use of `const char *&`.
[pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Fixed function name
* Introduced private static function to reuse get_name_or_throw
* added tests for capsule renaming
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* handle python error in flight
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Initialized PyObject * variables to nullptr
* use write-unraisable if PyCapsule_GetName raises
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* get_name_or_throw->get_name_no_throw
If PyCapsule_GetName raises an error we should write as unraisable
to consume it and notify user, and then restore the error in flight if any.
This way this method called from destructor would not modify interpreter
error state.
* used error_scope struct
* Renamed get_name_no_throw->get_name_in_error_scope
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add type_caster<std::monostate> for std::variant
Add type_caster<std::monostate>, allowing std::variant<std::monostate, ...>
* Add variant<std::monostate, ...> test methods
* Add std::monostate tests
* Update test_stl.py
Remove erroneous extra tests
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update test fn name
* And update the doc() test
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* PYBIND11_OBJECT_CVT should use namespace for error_already_set()
This change makes the macro usable outside of pybind11 namespace.
* added test for use of PYBIND11_OBJECT_CVT for classes in external to pybind11 namespaces
* Extended test_pytypes.cpp and test_pytest.py
The added test defines a dummy function that takes a custom-defined class external::float_
that uses PYBIND11_OBJECT_CVT
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fixed issues pointed out by CI
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fixed memory leak in default constructor
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fixes issue
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix lint error
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix flake8
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix test
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix clang tidy
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix again
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix test
* Add comments
* Try fix Valgrind
* Resolve comments
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* PYBIND11_OBJECT_CVT should use namespace for error_already_set()
This change makes the macro usable outside of pybind11 namespace.
* added test for use of PYBIND11_OBJECT_CVT for classes in external to pybind11 namespaces
* Extended test_pytypes.cpp and test_pytest.py
The added test defines a dummy function that takes a custom-defined class external::float_
that uses PYBIND11_OBJECT_CVT
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fixed issues pointed out by CI
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fixed memory leak in default constructor
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Reproducer for https://github.com/pybind/pybind11/issues/3788
Expected to build & run as-is. Uncommenting reproduces the infinite recursion.
* Moving try_as_void_ptr_capsule() to the end of load_impl()
* Moving new test into the existing test_class_sh_void_ptr_capsule
* Experiment
* Remove comments and simplify the test cases.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add bytearray to string cast, testcase and rename load_bytes to load_raw
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* New bytearray test case and convert failure to pybind11_fail
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix merge comments
* Actually fix merge comments
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Assert early if AsString fails
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Porras Huang <pohuang@jackx-vm-1.nvidia.com>
* Clear local internals after finalizing interpreter
* Add descriptive comments
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
