* Fix data race all_type_info_populate in free-threading mode
Description:
- fixed data race all_type_info_populate in free-threading mode
- added test
For example, we have 2 threads entering `all_type_info`.
Both enter `all_type_info_get_cache`` function and
there is a first one which inserts a tuple (type, empty_vector) to the map
and second is waiting. Inserting thread gets the (iter_to_key, True) and non-inserting thread
after waiting gets (iter_to_key, False).
Inserting thread than will add a weakref and will then call into `all_type_info_populate`.
However, non-inserting thread is not entering `if (ins.second) {` clause and
returns `ins.first->second;`` which is just empty_vector.
Finally, non-inserting thread is failing the check in `allocate_layout`:
```c++
if (n_types == 0) {
pybind11_fail(
"instance allocation failed: new instance has no pybind11-registered base types");
}
```
* style: pre-commit fixes
* Addressed PR comments
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix buffer protocol implementation
According to the buffer protocol, `ndim` is a _required_ field [1], and
should always be set correctly. Additionally, `shape` should be set if
flags includes `PyBUF_ND` or higher [2]. The current implementation only
set those fields if flags was `PyBUF_STRIDES`.
[1] https://docs.python.org/3/c-api/buffer.html#request-independent-fields
[2] https://docs.python.org/3/c-api/buffer.html#shape-strides-suboffsets
* Apply suggestions from review
* Obey contiguity requests for buffer protocol
If a contiguous buffer is requested, and the underlying buffer isn't,
then that should raise. This matches NumPy behaviour if you do something
like:
```
struct.unpack_from('5d', np.arange(20.0)[::4]) # Raises for contiguity
```
Also, if a buffer is contiguous, then it can masquerade as a
less-complex buffer, either by dropping strides, or even pretending to
be 1D. This matches NumPy behaviour if you do something like:
```
a = np.full((3, 5), 30.0)
struct.unpack_from('15d', a) # --> Produces 1D tuple from 2D buffer.
```
* Handle review comments
* Test buffer protocol against NumPy
* Also check PyBUF_FORMAT results
* Incomplete attempt to address regression introduced in #5381
* style: pre-commit fixes
* Revert "style: pre-commit fixes"
This reverts commit 9d107d2f75.
* Revert "Incomplete attempt to address regression introduced in #5381"
This reverts commit 8cf1cdbc96.
* Simpler fix for the regression introduced in #5381
* style: pre-commit fixes
* Added if constexpr workaround
This can probably be done better but at least this is a start.
* style: pre-commit fixes
* Replace if constexpr with template struct
if constexpr was not added until C++ 17.
I think this should do the same thing as before.
* style: pre-commit fixes
* Made comment clearer
* Added test cases
* style: pre-commit fixes
* Fixed is_same_or_base_of reference
* style: pre-commit fixes
* Added static assert messages
* style: pre-commit fixes
* Replaced typedef with using
* style: pre-commit fixes
* Back out `ForwardClassPtr` (to be discussed separately). Tested locally with clang-tidy.
* Shuffle new `static_assert()` and leave error messages blank (they are more distracting than helpful here).
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: gentlegiantJGC <gentlegiantJGC@users.noreply.github.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
* [docs/advanced] A document about deadlock potential with C++ statics
* [docs/advanced] Refer to deadlock.md from misc.rst
* [docs/advanced] Fix tables in deadlock.md
* Use :ref:`deadlock-reference-label`
* Revert "Use :ref:`deadlock-reference-label`"
This reverts commit e5734d275f.
* Add simple references to docs/advanced/deadlock.md filename. (Maybe someone can work on clickable links later.)
---------
Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
* Print key in map.getitem/delitem KeyError
* Add tests
* Fix tests
* Make robust
* Make clang-tidy happy
* Return a Python str
* Show beginning and end of the message
* Avoid implicit conversion
* Split out `format_message_key_error_key_object()` to reduce amount of templated code.
* Use `"✄✄✄"` instead of `"..."`
Also rename variable to `cut_length`, to not get into even/odd issues with the meaning of "half".
---------
Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
* Initial support for GraalPy
* Mark tests that currently fail on GraalPy with xfail
* Add graalpy to CI
* Limit test deps on graalpy to available binary wheels
* Skip cmake test installed_function on GraalPy
CMake won't find libpython on GraalPy, it either fails or silently picks
CPython's libpython.
* Factor out setting function docstrings into a macro
* Try to narrow down skipped tests
* Allow subclasses of py::args and py::kwargs
The current implementation does not allow subclasses of args or kwargs.
This change allows subclasses to be used.
* Added test case
* style: pre-commit fixes
* Added missing semi-colons
* style: pre-commit fixes
* Added handle_type_name
* Moved classes outside of function
* Added namespaces
* style: pre-commit fixes
* Refactored tests
Added more tests and moved tests to more appropriate locations.
* style: pre-commit fixes
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Issue: #5224
The `internals.registered_types_py...` line in pybind11.h triggers a
false-positive bounds checking warning in GCC 12.
This is discussed in
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115824.
The workaround implemented is taken from suggestions then refactored to
use the `PYBIND11_WARNING_PUSH` and `PYBIND11_WARNING_POP` MACROS.
* `self.__cpp_transporter__()` proof of concept: Enable passing C++ pointers across extensions even if the `PYBIND11_INTERNALS_VERSION`s do not match.
* Include cleanup (mainly to resolve PyPy build failures).
* Fix clang-tidy errors.
* Resolve `error: extra
* factor out platform_abi_id.h from internals.h (no functional changes)
* factor out internals_version.h from internals.h (no functional changes)
* Update CMakeLists.txt, tests/extra_python_package/test_files.py
* Revert "factor out internals_version.h from internals.h (no functional changes)"
This reverts commit 3ccea8cd43.
* Remove internals_version.h from CMakeLists.txt, tests/extra_python_package/test_files.py
* `.__cpp_transporter__()` implementation: compare `pybind11_platform_abi_id`, `cpp_typeid_name`
* Add PremiumTraveler
* Rename test_cpp_transporter_traveler_type.h -> test_cpp_transporter_traveler_types.h
* Expand tests: `PremiumTraveler`, `get_points()`
* Shuffle order of tests (no real changes).
* Move `__cpp_transporter__` lambda to `py::cpp_transporter()` regular function.
* Use `type_caster_generic::load(self)` instead of `cast<T *>(self)`
* Pass `const std::type_info *` via `py::capsule` (instead of `cpp_typeid_name`).
* Make platform_abi_id.h completely stand-alone.
* rename exo_planet.cpp -> exo_planet_pybind11.cpp
* Add exo_planet_c_api.cpp (incomplete).
* Fix silly oversight (wrong filename in `#include`).
* Resolve clang-tidy errors:
```
/__w/pybind11/pybind11/tests/exo_planet_c_api.cpp:10:18: error: 'wrapGetLuggage' is a static definition in anonymous namespace; static is redundant here [readability-static-definition-in-anonymous-namespace,-warnings-as-errors]
10 | static PyObject *wrapGetLuggage(PyObject *, PyObject *) { return PyUnicode_FromString("TODO"); }
| ~~~~~~ ^
/__w/pybind11/pybind11/tests/exo_planet_c_api.cpp:14:20: error: 'ThisMethodDef' is a static definition in anonymous namespace; static is redundant here [readability-static-definition-in-anonymous-namespace,-warnings-as-errors]
14 | static PyMethodDef ThisMethodDef[]
| ~~~~~~ ^
/__w/pybind11/pybind11/tests/exo_planet_c_api.cpp:17:27: error: 'ThisModuleDef' is a static definition in anonymous namespace; static is redundant here [readability-static-definition-in-anonymous-namespace,-warnings-as-errors]
17 | static struct PyModuleDef ThisModuleDef = {
| ~~~~~~ ^
```
* Implement exo_planet_c_api GetLuggage(), GetPoints()
* Move new code from test_cpp_transporter_traveler_bindings.h to pybind11/detail/type_caster_base.h, under the name `class_dunder_cpp_transporter()`
* Fix oversight.
* Unconditionally add `__cpp_transporter__` method to all `py::class_` objects, but do not include that magic method in docstring signatures.
* Back out pybind11/detail/platform_abi_id.h for now. Maximizing reusability can be handled separately, later.
* Small cleanup.
* Restore and add to `test_call_cpp_transporter_*()`
* Ensure https://github.com/pybind/pybind11/issues/3788 does not bite again.
* `class_dunder_cpp_transporter()`: replace `obj.cast<std::string>()` with `std::string(obj)`
* Add (simple) copyright notices in all newly added files.
* Globally replace cpp_transporter with cpp_conduit
* style: pre-commit fixes
* IWYU fixes
* Rename `class_dunder_cpp_conduit()` -> `cpp_conduit_method()`
* Change `pybind11_platform_abi_id`, `pointer_kind` argument types from `str` to `bytes`.
This avoids the unicode decode/encode roundtrips:
* More robust (no decode/encode errors).
* Minor runtime optimization.
* Systematically rename `cap_cpp_type_info` -> `cpp_type_info_capsule` (no functional changes).
* Systematically replace `cpp_type_info_capsule` `name`: `"const std::type_info *"` -> `typeid(std::type_info).name()` (this IS a functional change).
This provides an extra layer of protection against C++ ABI mismatches:
* The first and most important layer is that the `PYBIND11_PLATFORM_ABI_ID`s must match between extensions.
* The second layer is that the `typeid(std::type_info).name()`s must match between extensions.
* Fix sort order accident in tests/CMakeLists.txt
* Apply suggestions from code review
Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com>
* style: pre-commit fixes
* refactor: rename to _pybind_conduit_v1_
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* Add test_home_planet_wrap_very_lonely_traveler(), test_exo_planet_pybind11_wrap_very_lonely_traveler()
* Resolve clang-tidy errors:
```
/__w/pybind11/pybind11/tests/test_cpp_conduit_traveler_bindings.h:39:32: error: parameter 'm' is passed by value and only copied once; consider moving it to avoid unnecessary copies [performance-unnecessary-value-param,-warnings-as-errors]
10 | py::class_<LonelyTraveler>(m, "LonelyTraveler");
| ^
| std::move( )
/__w/pybind11/pybind11/tests/test_cpp_conduit_traveler_bindings.h:43:52: error: parameter 'm' is passed by value and only copied once; consider moving it to avoid unnecessary copies [performance-unnecessary-value-param,-warnings-as-errors]
43 | py::class_<VeryLonelyTraveler, LonelyTraveler>(m, "VeryLonelyTraveler");
| ^
| std::move( )
```
---------
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com>
* Add test for throwing def_buffer case
* Translate C++ -> Python exceptions in buffer creation
This required a little refactoring to extract exception translation to a separate method
* Fix code formatting
* Fix "unused parameter" warning
* Refactor per review
* style: pre-commit fixes
* Address review comments
* Address review comments
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix: use `__cpp_nontype_template_args` instead of gnu extensions
* fix: add feature test value
* fix: change `PYBIND11_TYPING_H_HAS_STRING_LITERAL` skip reason
* DOC: Clarify requirements for including pybind11
Inherited from requirements for including Python.h
Closes#4999
* style: pre-commit fixes
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix: allow -Wpedantic again
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* tests: ignore pedantic warning for PYBIND11_DECLARE_HOLDER_TYPE
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* tests: try just turning off pedantic for one file
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* tests: only run pedantic in C++20 mode
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* Update tests/local_bindings.h
---------
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* feat: add `<ranges>` support for `py::tuple` and `py::list`
* fix: format the code
* fix: disable `ranges` in clang < 16
* refactor: move `<ranges>` test macro to `test_pytypes.h`
* refactor: seperate `ranges` test into 3 funcs
* style: compress the if statement
* style: pre-commit fixes
* style: better formatting
---------
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix: never use `..` in a header include
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* fix: one more parent include
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
---------
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* fix: Escape paths with spaces in include list from --includes
* fix: --includes should not use shlex on Windows platforms
* Apply suggestions from code review
* fix: use custom impl
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* Support trailing backslashes
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
---------
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Markus Bauer <markus.bauer@cispa.saarland>
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
* Add `test_pass_std_vector_int()`, `test_pass_std_set_int()` in test_stl
* Change `list_caster` to also accept generator objects (`PyGen_Check(src.ptr()`).
Note for completeness: This is a more conservative change than https://github.com/google/pywrapcc/pull/30042
* Drop in (currently unpublished) PyCLIF code, use in `list_caster`, adjust tests.
* Use `PyObjectTypeIsConvertibleToStdSet()` in `set_caster`, adjust tests.
* Use `PyObjectTypeIsConvertibleToStdMap()` in `map_caster`, add tests.
* Simplify `list_caster` `load()` implementation, push str/bytes check into `PyObjectTypeIsConvertibleToStdVector()`.
* clang-tidy cleanup with a few extra `(... != 0)` to be more consistent.
* Also use `PyObjectTypeIsConvertibleToStdVector()` in `array_caster`.
* Update comment pointing to clif/python/runtime.cc (code is unchanged).
* Comprehensive test coverage, enhanced set_caster load implementation.
* Resolve clang-tidy eror.
* Add a long C++ comment explaining what led to the `PyObjectTypeIsConvertibleTo*()` implementations.
* Minor function name change in test.
* strcmp -> std::strcmp (thanks @Skylion007 for catching this)
* Add `PyCallable_Check(items)` in `PyObjectTypeIsConvertibleToStdMap()`
* Resolve clang-tidy error
* Use `PyMapping_Items()` instead of `src.attr("items")()`, to be internally consistent with `PyMapping_Check()`
* Update link to PyCLIF sources.
* Fix typo (thanks @wangxf123456 for catching this)
* Add `test_pass_std_vector_int()`, `test_pass_std_set_int()` in test_stl
* Change `list_caster` to also accept generator objects (`PyGen_Check(src.ptr()`).
Note for completeness: This is a more conservative change than https://github.com/google/pywrapcc/pull/30042
* Drop in (currently unpublished) PyCLIF code, use in `list_caster`, adjust tests.
* Use `PyObjectTypeIsConvertibleToStdSet()` in `set_caster`, adjust tests.
* Use `PyObjectTypeIsConvertibleToStdMap()` in `map_caster`, add tests.
* Simplify `list_caster` `load()` implementation, push str/bytes check into `PyObjectTypeIsConvertibleToStdVector()`.
* clang-tidy cleanup with a few extra `(... != 0)` to be more consistent.
* Also use `PyObjectTypeIsConvertibleToStdVector()` in `array_caster`.
* Update comment pointing to clif/python/runtime.cc (code is unchanged).
* Comprehensive test coverage, enhanced set_caster load implementation.
* Resolve clang-tidy eror.
* Add a long C++ comment explaining what led to the `PyObjectTypeIsConvertibleTo*()` implementations.
* Minor function name change in test.
* strcmp -> std::strcmp (thanks @Skylion007 for catching this)
* Add `PyCallable_Check(items)` in `PyObjectTypeIsConvertibleToStdMap()`
* Resolve clang-tidy error
* Use `PyMapping_Items()` instead of `src.attr("items")()`, to be internally consistent with `PyMapping_Check()`
* Update link to PyCLIF sources.
* Fix typo (thanks @wangxf123456 for catching this)
* Fix typo discovered by new version of codespell.
* fix(cmake): add required emscripten flags
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* Update emscripten.yaml
* fix(cmake): add required emscripten flags to headers target
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* fix(cmake): incorrect detection of Emscripten
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* fix(cmake): allow pybind11::headers to be modified
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* fix(cmake): hide a warning when building the tests standalone
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* fix(cmake): use explicit variable for is config
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* fix(cmake): go back to ALIAS target
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* chore: reduce overall diff
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* chore: reduce overall diff
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* chore: shorten code a bit
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
---------
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* Allow specializations based on callback function return values.
* clang-tidy auto fix
* Add a test case for function specialization.
* Add test for callback function that raises Python exception.
* Fix test failures.
* style: pre-commit fixes
* Add `#define PYBIND11_HAS_TYPE_CASTER_STD_FUNCTION_SPECIALIZATIONS`
---------
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
