* Allow specializations based on callback function return values.
* clang-tidy auto fix
* Add a test case for function specialization.
* Add test for callback function that raises Python exception.
* Fix test failures.
* style: pre-commit fixes
* Add `#define PYBIND11_HAS_TYPE_CASTER_STD_FUNCTION_SPECIALIZATIONS`
---------
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Put bakein branch @ 18b72c0ffa on top of smart_holder branch:
Commands used:
```
git checkout bakein
git diff smart_holder > ~/zd
git checkout smart_holder
git checkout -b bakein_sh
patch -p 1 < ~/zd
git checkout smart_holder \
MANIFEST.in \
README.rst \
README_smart_holder.rst \
docs/advanced/smart_ptrs.rst \
ubench/holder_comparison.cpp \
ubench/holder_comparison.py \
ubench/holder_comparison_extract_sheet_data.py \
ubench/number_bucket.h \
ubench/python/number_bucket.clif
git add -A
```
* Add back README_smart_holder.rst in tests/extra_python_package/test_files.py
* Restore smart_holder_poc.h as-is on smart_holder branch (i.e. undo `PYBIND11_SMART_HOLDER_PADDING`, which was meant for stress-testing only).
* Insert `std::move()` as suggested by @laramiel
* `property_cpp_function_sh_*` named specializations, as suggested by @laramiel (https://github.com/pybind/pybind11/pull/5257#discussion_r1688346807)
* Call `property_cpp_function_classic` member functions, rather than inlining the implementations.
* Use `PYBIND11_HAVE_INTERNALS_WITH_SMART_HOLDER_SUPPORT` in holder_comparison.cpp (holder_comparison.py is NOT changed accordingly in this commit, i.e. can still only be run if the smart_holder functionality is available).
* Systematically rename `loaded_as` to `load_as` (`shared_ptr`, `unique_ptr`) as suggested by @laramiel
* Make change as suggested by @laramiel. This makes it much more obvious that the latest implementation of `smart_holder_from_unique_ptr()` accepts all existing `return_value_policy` enum values except `copy`.
* Resolve `BAKEIN_WIP: Rewrite comment.` for `property_cpp_function_*` specializations.
* Resolve `BAKEIN_WIP: Add comment to explain: This is meant for stress-testing only.`
* Resolve all remaining BAKEIN_WIP (in pybind11/cast.h).
Leave only two pairs of SMART_HOLDER_BAKEIN_FOLLOW_ON comments: refactoring of copyable_holder_caster, move_only_holder_caster. This is best left until after the smart_holder branch is merged into the master branch.
* Remove obsolete `using holder_type = smart_holder;` in `load_helper`
* Add SMART_HOLDER_BAKEIN_FOLLOW_ON comment for `internals::default_holder`
* README_smart_holder.rst update (line count reduced from 356 to 123).
Follow-on to https://github.com/pybind/pybind11/pull/5272 — clang-tidy upgrade (to version 18)
Fixes this error:
```
/__w/pybind11/pybind11/tests/pure_cpp/smart_holder_poc_test.cpp:167:12: error: the value returned by this function should not be disregarded; neglecting it may lead to errors [bugprone-unused-return-value,-warnings-as-errors]
167 | (void) new_owner.release(); // Manually verified: without this, clang++ -fsanitize=address
| ^~~~~~~~~~~~~~~~~~~
```
See also: https://clang.llvm.org/extra/clang-tidy/checks/bugprone/unused-return-value.html
Specifically there:
> std::unique_ptr::release(). Not using the return value can lead to resource leaks if the same pointer isn’t stored anywhere else. Often, ignoring the release() return value indicates that the programmer confused the function with reset().
I.e. the only way to tell clang-tidy that the smart_holder_poc_test.cpp is correct is to add the `NOLINT`.
* `container: silkeh/clang:18-bookworm` in .github/workflows/format.yml
* clang-tidy auto-fix (trivial, in test only)
* Disable `performance-enum-size` (noisy, low value)
* Temporarily turn off 3 diagnostics (to be tackled one-by-one).
* Add explicit `switch` `default` to resolve clang-tidy `bugprone-switch-missing-default-case`
Debian clang version 18.1.8 (++20240718080534+3b5b5c1ec4a3-1~exp1~20240718200641.143)
Target: x86_64-pc-linux-gnu
tests/test_numpy_dtypes.cpp:212:5: warning: switching on non-enum value without default case may not cover all cases [bugprone-switch-missing-default-case]
* Add clang-17 and clang-18 testing.
* Add `NOLINTNEXTLINE(clang-analyzer-optin.core.EnumCastOutOfRange)` in test_tagbased_polymorphic.cpp
Debian clang version 18.1.8 (++20240718080534+3b5b5c1ec4a3-1~exp1~20240718200641.143)
Target: x86_64-pc-linux-gnu
tests/test_tagbased_polymorphic.cpp:77:40: warning: The value '150' provided to the cast expression is not in the valid range of values for 'Kind' [clang-analyzer-optin.core.EnumCastOutOfRange]
* Fix inconsistent pybind11/eigen/tensor.h behavior:
This existing comment in pybind11/eigen/tensor.h
```
// move, take_ownership don't make any sense for a ref/map:
```
is at odds with the `delete src;` three lines up.
In real-world client code `take_ownership` will not exist (unless the client code is untested and unused). I.e. the `delete` is essentially only useful to avoid leaks in the pybind11 unit tests.
While upgrading to clang-tidy 18, the warning below appeared. Apparently it is produced during LTO, and it appears difficult to suppress. Regardless, the best way to resolve this is to remove the `delete` and to simply make the test objects `static` in the unit test code.
________
Debian clang version 18.1.8 (++20240718080534+3b5b5c1ec4a3-1~exp1~20240718200641.143)
Target: x86_64-pc-linux-gnu
________
```
lto-wrapper: warning: using serial compilation of 3 LTRANS jobs
lto-wrapper: note: see the ‘-flto’ option documentation for more information
In function ‘cast_impl’,
inlined from ‘cast’ at /mounted_pybind11/include/pybind11/eigen/tensor.h:414:25,
inlined from ‘operator()’ at /mounted_pybind11/include/pybind11/eigen/../pybind11.h:296:40,
inlined from ‘_FUN’ at /mounted_pybind11/include/pybind11/eigen/../pybind11.h:267:21:
/mounted_pybind11/include/pybind11/eigen/tensor.h:475:17: warning: ‘operator delete’ called on unallocated object ‘<anonymous>’ [-Wfree-nonheap-object]
475 | delete src;
| ^
/mounted_pybind11/include/pybind11/eigen/../pybind11.h: In function ‘_FUN’:
/mounted_pybind11/include/pybind11/eigen/../pybind11.h:297:75: note: declared here
297 | std::move(args_converter).template call<Return, Guard>(cap->f),
| ^
```
* Disable `bugprone-chained-comparison`: this clang-tidy check is incompatible with the Catch2 `REQUIRE` macro (26 warnings like the one below).
________
Debian clang version 18.1.8 (++20240718080534+3b5b5c1ec4a3-1~exp1~20240718200641.143)
Target: x86_64-pc-linux-gnu
________
```
/mounted_pybind11/tests/test_embed/test_interpreter.cpp:127:9: warning: chained comparison 'v0 <= v1 == v2' may generate unintended results, use parentheses to specify order of evaluation or a logical operator to separate comparison expressions [bugprone-chained-comparison]
127 | REQUIRE(ret == 42);
| ^
/build/tests/catch/catch.hpp:17670:24: note: expanded from macro 'REQUIRE'
17670 | #define REQUIRE( ... ) INTERNAL_CATCH_TEST( "REQUIRE", Catch::ResultDisposition::Normal, __VA_ARGS__ )
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/build/tests/catch/catch.hpp:2710:47: note: expanded from macro 'INTERNAL_CATCH_TEST'
2710 | catchAssertionHandler.handleExpr( Catch::Decomposer() <= __VA_ARGS__ ); \
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/mounted_pybind11/tests/test_embed/test_interpreter.cpp:127:9: note: operand 'v0' is here
127 | REQUIRE(ret == 42);
| ^
/build/tests/catch/catch.hpp:17670:24: note: expanded from macro 'REQUIRE'
17670 | #define REQUIRE( ... ) INTERNAL_CATCH_TEST( "REQUIRE", Catch::ResultDisposition::Normal, __VA_ARGS__ )
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/build/tests/catch/catch.hpp:2710:47: note: expanded from macro 'INTERNAL_CATCH_TEST'
2710 | catchAssertionHandler.handleExpr( Catch::Decomposer() <= __VA_ARGS__ ); \
| ^~~~~~~~~~~~~~~~~~~
/mounted_pybind11/tests/test_embed/test_interpreter.cpp:127:17: note: operand 'v1' is here
127 | REQUIRE(ret == 42);
| ^
/build/tests/catch/catch.hpp:17670:90: note: expanded from macro 'REQUIRE'
17670 | #define REQUIRE( ... ) INTERNAL_CATCH_TEST( "REQUIRE", Catch::ResultDisposition::Normal, __VA_ARGS__ )
| ^~~~~~~~~~~
/build/tests/catch/catch.hpp:2710:70: note: expanded from macro 'INTERNAL_CATCH_TEST'
2710 | catchAssertionHandler.handleExpr( Catch::Decomposer() <= __VA_ARGS__ ); \
| ^~~~~~~~~~~
/mounted_pybind11/tests/test_embed/test_interpreter.cpp:127:24: note: operand 'v2' is here
127 | REQUIRE(ret == 42);
| ^
/build/tests/catch/catch.hpp:17670:90: note: expanded from macro 'REQUIRE'
17670 | #define REQUIRE( ... ) INTERNAL_CATCH_TEST( "REQUIRE", Catch::ResultDisposition::Normal, __VA_ARGS__ )
| ^~~~~~~~~~~
/build/tests/catch/catch.hpp:2710:70: note: expanded from macro 'INTERNAL_CATCH_TEST'
2710 | catchAssertionHandler.handleExpr( Catch::Decomposer() <= __VA_ARGS__ ); \
| ^~~~~~~~~~~
```
* Add 8 `// NOLINT(bugprone-empty-catch)`
* Resolve clang-tidy `bugprone-multi-level-implicit-pointer-conversion` warnings.
________
Debian clang version 18.1.8 (++20240718080534+3b5b5c1ec4a3-1~exp1~20240718200641.143)
Target: x86_64-pc-linux-gnu
________
```
pybind11/detail/internals.h:556:53: warning: multilevel pointer conversion from 'internals **' to 'const void *', please use explicit cast [bugprone-multi-level-implicit-pointer-conversion]
pybind11/detail/type_caster_base.h:431:20: warning: multilevel pointer conversion from 'void **' to 'void *', please use explicit cast [bugprone-multi-level-implicit-pointer-conversion]
pybind11/numpy.h:904:81: warning: multilevel pointer conversion from '_object *const *' to 'const void *', please use explicit cast [bugprone-multi-level-implicit-pointer-conversion]
pybind11/numpy.h:1989:39: warning: multilevel pointer conversion from 'typename vectorize_arg<const double *>::type *' (aka 'const double **') to 'void *', please use explicit cast [bugprone-multi-level-implicit-pointer-conversion]
pybind11/numpy.h:1989:39: warning: multilevel pointer conversion from 'typename vectorize_arg<const VectorizeTestClass *>::type *' (aka 'const VectorizeTestClass **') to 'void *', please use explicit cast [bugprone-multi-level-implicit-pointer-conversion]
pybind11/stl/filesystem.h:75:44: warning: multilevel pointer conversion from 'PyObject **' (aka '_object **') to 'void *', please use explicit cast [bugprone-multi-level-implicit-pointer-conversion]
pybind11/stl/filesystem.h:83:42: warning: multilevel pointer conversion from 'PyObject **' (aka '_object **') to 'void *', please use explicit cast [bugprone-multi-level-implicit-pointer-conversion]
```
* Introduce `PYBIND11_REINTERPRET_CAST_VOID_PTR_IF_NOT_PYPY` to resolve PyPy build errors:
```
In file included from /Users/runner/work/pybind11/pybind11/tests/test_stl.cpp:18:
/Users/runner/work/pybind11/pybind11/include/pybind11/stl/filesystem.h:75:17: error: no matching function for call to 'PyPyUnicode_FSConverter'
if (PyUnicode_FSConverter(buf, reinterpret_cast<void *>(&native)) != 0) {
^~~~~~~~~~~~~~~~~~~~~
/Users/runner/hostedtoolcache/PyPy/3.10.14/x64/include/pypy3.10/pypy_decl.h:969:31: note: expanded from macro 'PyUnicode_FSConverter'
^~~~~~~~~~~~~~~~~~~~~~~
/Users/runner/hostedtoolcache/PyPy/3.10.14/x64/include/pypy3.10/pypy_decl.h:970:17: note: candidate function not viable: cannot convert argument of incomplete type 'void *' to 'struct _object **' for 2nd argument
PyAPI_FUNC(int) PyUnicode_FSConverter(struct _object *arg0, struct _object **arg1);
^
/Users/runner/hostedtoolcache/PyPy/3.10.14/x64/include/pypy3.10/pypy_decl.h:969:31: note: expanded from macro 'PyUnicode_FSConverter'
^
In file included from /Users/runner/work/pybind11/pybind11/tests/test_stl.cpp:18:
/Users/runner/work/pybind11/pybind11/include/pybind11/stl/filesystem.h:83:17: error: no matching function for call to 'PyPyUnicode_FSDecoder'
if (PyUnicode_FSDecoder(buf, reinterpret_cast<void *>(&native)) != 0) {
^~~~~~~~~~~~~~~~~~~
/Users/runner/hostedtoolcache/PyPy/3.10.14/x64/include/pypy3.10/pypy_decl.h:971:29: note: expanded from macro 'PyUnicode_FSDecoder'
^~~~~~~~~~~~~~~~~~~~~
/Users/runner/hostedtoolcache/PyPy/3.10.14/x64/include/pypy3.10/pypy_decl.h:972:17: note: candidate function not viable: cannot convert argument of incomplete type 'void *' to 'struct _object **' for 2nd argument
PyAPI_FUNC(int) PyUnicode_FSDecoder(struct _object *arg0, struct _object **arg1);
^
/Users/runner/hostedtoolcache/PyPy/3.10.14/x64/include/pypy3.10/pypy_decl.h:971:29: note: expanded from macro 'PyUnicode_FSDecoder'
^
```
* clang-tidy auto-fix
* Fix silly oversight.
* Factor out detail/value_and_holder.h (from detail/type_caster_base.h)
This is in support of PR #5213:
* trampoline_self_life_support.h depends on value_and_holder.h
* type_caster_base.h depends on trampoline_self_life_support.h
* Fix a minor and inconsequential inconsistency in `copyable_holder_caster`: the correct `load_value()` return type is `void` (as defined in `type_caster_generic`)
For easy future reference, this is the long-standing inconsistency:
* dbf848aff7/include/pybind11/detail/type_caster_base.h (L634)
* dbf848aff7/include/pybind11/cast.h (L797)
Noticed in passing while working on PR #5213.
* Add `DANGER ZONE` comment in detail/init.h, similar to a comment added on the smart_holder branch (all the way back in 2021).
* tests: run on pyodide
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* ci: use cibuildwheel for pyodide test
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* tests: revert changes to test_embed
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
---------
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* fix: Make gil_safe_call_once thread-safe in free-threaded CPython
The "is_initialized_" flags is not protected by the GIL in free-threaded
Python, so it needs to be an atomic field.
Fixes#5245
* style: pre-commit fixes
* Apply changes from code review
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Use PyMutex instead of std::mutex in free-threaded build.
PyMutex is now part of the public C API as of 3.13.0b3 and generally has
slightly less overhead than std::mutex.
* style: pre-commit fixes
* Fix instance_map_shard padding
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Use manual padding of instance_map_shard.
The alignas(64) specifier requires aligned allocation, which is not
available on macOS when targeting versions before 10.14.
* Add 'see #5200'
* Update include/pybind11/detail/internals.h
* Update include/pybind11/detail/internals.h
---------
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
* Adds support for TypeGuard and TypeIs
* style: pre-commit fixes
---------
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* ci: release with trusted publisher and attestations
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* Update pip.yml
---------
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
