Commit Graph

3001 Commits

Author SHA1 Message Date
Ralf W. Grosse-Kunstleve
ecd01da7ac smart_holder_from_unique_ptr: also accept return_value_policy::take_ownership
There are no strong reasons for accepting or rejecting `return_value_policy::take_ownership`.

Accepting to accommodate existing use cases in the wild.
2024-07-15 01:33:15 -07:00
Ralf W. Grosse-Kunstleve
12f384bcbb smart_holder_from_unique_ptr: also accept return_value_policy::reference
There are no strong reasons for accepting or rejecting `return_value_policy::reference`.

Accepting to accommodate existing use cases in the wild.
2024-07-14 23:52:54 -07:00
Ralf W. Grosse-Kunstleve
884305e953 Add "_sh_baked_in" to PYBIND11_INTERNALS_ID, PYBIND11_MODULE_LOCAL_ID 2024-07-14 21:56:36 -07:00
Ralf W. Grosse-Kunstleve
0f874deee2 Move must_be_member_function_pointer to the detail namespace. 2024-07-14 21:39:33 -07:00
Ralf W. Grosse-Kunstleve
ff3693f1bb Introduce detail::both_t_and_d_use_type_caster_base<T, D> helper and use systematically in all property_cpp_function<> specializations.
This fixes a PyTorch build error (using the Google-internal toolchain):

```
In file included from third_party/py/torch/torch/csrc/distributed/c10d/init.cpp:15:
In file included from third_party/py/torch/torch/csrc/distributed/c10d/PyProcessGroup.hpp:4:
In file included from third_party/py/torch/torch/csrc/jit/python/pybind_utils.h:7:
In file included from third_party/pybind11/include/pybind11/complex.h:14:
In file included from third_party/pybind11/include/pybind11/pybind11.h:14:
In file included from third_party/pybind11/include/pybind11/detail/class.h:14:
In file included from third_party/pybind11/include/pybind11/detail/../attr.h:16:
third_party/pybind11/include/pybind11/detail/../cast.h:856:19: error: static assertion failed due to requirement 'std::is_base_of<pybind11::detail::type_caster_base<c10::intrusive_ptr<c10d::Store, c10::detail::intrusive_target_default_null_type<c10d::Store>>>, pybind11::detail::type_caster<c10::intrusive_ptr<c10d::Store, c10::detail::intrusive_target_default_null_type<c10d::Store>>, void>>::value': Holder classes are only supported for custom types
  856 |     static_assert(std::is_base_of<base, type_caster<type>>::value,
      |                   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
third_party/pybind11/include/pybind11/detail/../cast.h:982:48: note: in instantiation of template class 'pybind11::detail::copyable_holder_caster<c10::intrusive_ptr<c10d::Store>, std::shared_ptr<c10::intrusive_ptr<c10d::Store>>>' requested here
  982 | class type_caster<std::shared_ptr<T>> : public copyable_holder_caster<T, std::shared_ptr<T>> {};
      |                                                ^
third_party/crosstool/v18/stable/src/libcxx/include/__type_traits/is_base_of.h:22:91: note: in instantiation of template class 'pybind11::detail::type_caster<std::shared_ptr<c10::intrusive_ptr<c10d::Store>>>' requested here
   22 | struct _LIBCPP_TEMPLATE_VIS is_base_of : public integral_constant<bool, __is_base_of(_Bp, _Dp)> {};
      |                                                                                           ^
third_party/pybind11/include/pybind11/detail/../cast.h:1382:30: note: in instantiation of template class 'std::is_base_of<pybind11::detail::type_caster_generic, pybind11::detail::type_caster<std::shared_ptr<c10::intrusive_ptr<c10d::Store>>>>' requested here
 1382 |     detail::enable_if_t<std::is_base_of<type_caster_generic, make_caster<Return>>::value, void>> {
      |                              ^
third_party/pybind11/include/pybind11/pybind11.h:287:19: note: during template argument deduction for class template partial specialization 'return_value_policy_override<Return, detail::enable_if_t<std::is_base_of<type_caster_generic, make_caster<Return>>::value, void>>' [with Return = std::shared_ptr<c10::intrusive_ptr<c10d::Store>>]
  287 |                 = return_value_policy_override<Return>::policy(call.func.policy);
      |                   ^
third_party/pybind11/include/pybind11/pybind11.h:287:19: note: in instantiation of template class 'pybind11::detail::return_value_policy_override<std::shared_ptr<c10::intrusive_ptr<c10d::Store>>>' requested here
third_party/pybind11/include/pybind11/pybind11.h:269:55: note: while substituting into a lambda expression here
  269 |         rec->impl = [](function_call &call) -> handle {
      |                                                       ^
third_party/pybind11/include/pybind11/pybind11.h:147:9: note: in instantiation of function template specialization 'pybind11::cpp_function::initialize<(lambda at third_party/pybind11/include/pybind11/pybind11.h:1714:17), std::shared_ptr<c10::intrusive_ptr<c10d::Store>>, pybind11::handle, pybind11::is_method>' requested here
  147 |         initialize(
      |         ^
third_party/pybind11/include/pybind11/pybind11.h:1713:20: note: in instantiation of function template specialization 'pybind11::cpp_function::cpp_function<(lambda at third_party/pybind11/include/pybind11/pybind11.h:1714:17), pybind11::is_method, void>' requested here
 1713 |             return cpp_function(
      |                    ^
third_party/pybind11/include/pybind11/pybind11.h:1964:54: note: in instantiation of function template specialization 'pybind11::property_cpp_function<c10d::DistributedBackendOptions, c10::intrusive_ptr<c10d::Store>>::read<c10::intrusive_ptr<c10d::Store> c10d::DistributedBackendOptions::*, 0>' requested here
 1964 |                      property_cpp_function<type, D>::read(pm, *this),
      |                                                      ^
third_party/py/torch/torch/csrc/distributed/c10d/init.cpp:945:8: note: in instantiation of function template specialization 'pybind11::class_<c10d::DistributedBackendOptions>::def_readwrite<c10d::DistributedBackendOptions, c10::intrusive_ptr<c10d::Store>>' requested here
  945 |       .def_readwrite("store", &::c10d::DistributedBackendOptions::store)
      |        ^
1 error generated.
```
2024-07-14 21:19:50 -07:00
Ralf W. Grosse-Kunstleve
89da1e2ef7 Use memcpy instead of strcpy to resolve MSVC errors. 2024-07-14 01:15:37 -07:00
Ralf W. Grosse-Kunstleve
c65d5dfaa8 Systematically use is_base_of<type_caster_generic, make_caster<D>> in property_cpp_function<> specializations. 2024-07-14 00:54:41 -07:00
Ralf W. Grosse-Kunstleve
3406be6877 New test_class_sh_property_bakein.py
Use cases in the wild:

* `test_readonly_char6_member()`: 4410c44ae6/torch/csrc/cuda/Module.cpp (L961)

* `test_readonly_const_char_ptr_member()`: 862a439a84/include/permonst.h (L43)
2024-07-13 23:46:36 -07:00
Ralf W. Grosse-Kunstleve
2600bb67c5 Remove test_ccccccccccccccccccccc.cpp,py (not needed anymore). 2024-07-13 01:15:09 -07:00
Ralf W. Grosse-Kunstleve
6fa0a2e439 PYBIND11_HAVE_HOLDER_CASTER_WITH_SMART_HOLDER_SUPPORT
Introduce two helper types:

* copyable_holder_caster_shared_ptr_with_smart_holder_support_enabled

* move_only_holder_caster_unique_ptr_with_smart_holder_support_enabled
2024-07-13 00:07:57 -07:00
Ralf W. Grosse-Kunstleve
092d236b4e Remove test_wip.cpp,py (not needed anymore). 2024-07-12 08:33:55 -07:00
Ralf W. Grosse-Kunstleve
d021bb10aa Remove redundant check_holder_compat() no-op implementation. 2024-07-12 08:31:35 -07:00
Ralf W. Grosse-Kunstleve
63044cc199 Bring in default_holder_type from smart_holder branch including using it in pybind11/stl_bind.h 2024-07-12 08:26:34 -07:00
Ralf W. Grosse-Kunstleve
d373a082ad Remove remaining BAKEIN_BREAK in test_class_sh_property.cpp,py and adjust (simplify) the property_cpp_function<...unique_ptr...>::write implementation (all tests pass).
This PR (#5213) at this commit also passes ASAN, MSAN, UBSAN testing with the Google-internal toolchain.
2024-07-12 02:38:14 -07:00
Ralf W. Grosse-Kunstleve
0b14d5bce9 test_class_sh_property.py test_valu_getter() works without any further changes. 2024-07-12 02:24:27 -07:00
Ralf W. Grosse-Kunstleve
048e36df40 Bring in shared_ptr_with_responsible_parent() from smart_holder branch:
smart_holder branch:

```
    static std::shared_ptr<type> shared_ptr_from_python(handle responsible_parent)
```

Renamed in this commit:

```
    static std::shared_ptr<type> shared_ptr_with_responsible_parent(handle responsible_parent)
```

Use in `property_cpp_function<>` specializations.

Fixes all 8 test failures introduced with the previous commit (1bcf572780):

```
FAILED test_class_sh_property_non_owning.py::test_core_fld_common[core_fld_value_ro-expected0-True] - RuntimeError: Non-owning holder (loaded_as_shared_ptr).
FAILED test_class_sh_property_non_owning.py::test_core_fld_common[core_fld_value_rw-expected1-True] - RuntimeError: Non-owning holder (loaded_as_shared_ptr).
FAILED test_class_sh_property_non_owning.py::test_core_fld_common[core_fld_value_rw-expected1-False] - RuntimeError: Non-owning holder (loaded_as_shared_ptr).
FAILED test_class_sh_property_non_owning.py::test_core_fld_common[core_fld_raw_ptr_ro-expected4-True] - RuntimeError: Non-owning holder (loaded_as_shared_ptr).
FAILED test_class_sh_property_non_owning.py::test_core_fld_common[core_fld_raw_ptr_ro-expected4-False] - RuntimeError: Non-owning holder (loaded_as_shared_ptr).
FAILED test_class_sh_property_non_owning.py::test_core_fld_common[core_fld_raw_ptr_rw-expected5-True] - RuntimeError: Non-owning holder (loaded_as_shared_ptr).
FAILED test_class_sh_property_non_owning.py::test_core_fld_common[core_fld_value_ro-expected0-False] - RuntimeError: Non-owning holder (loaded_as_shared_ptr).
FAILED test_class_sh_property_non_owning.py::test_core_fld_common[core_fld_raw_ptr_rw-expected5-False] - RuntimeError: Non-owning holder (loaded_as_shared_ptr).
```
2024-07-12 02:11:09 -07:00
Ralf W. Grosse-Kunstleve
1bcf572780 First pass adapting all property_cpp_function<> implementations. Still needs debugging and more testing:
```
========================================================= short test summary info ==========================================================
SKIPPED [2] test_class_sh_property.py:19: BAKEIN_BREAK: Failed: DID NOT RAISE <class 'ValueError'>
SKIPPED [1] test_class_sh_property.py:87: BAKEIN_BREAK: m_uqcp_readwrite does not build
SKIPPED [1] test_class_sh_property.py:140: BAKEIN_BREAK: m_uqmp_readwrite does not build
SKIPPED [1] test_class_sh_property.py:140: BAKEIN_BREAK: m_uqcp_readwrite does not build
SKIPPED [1] test_class_sh_basic.py:156: unconditional skip
SKIPPED [1] test_class_sh_property.py:87: BAKEIN_BREAK: m_uqmp_readwrite does not build
SKIPPED [1] test_stl.py:149: no <experimental/optional>
SKIPPED [1] test_smart_ptr.py:301: BAKEIN_EXPECTED: Failed: DID NOT RAISE <class 'RuntimeError'>
FAILED test_class_sh_property_non_owning.py::test_core_fld_common[core_fld_value_rw-expected1-True] - RuntimeError: Non-owning holder (loaded_as_shared_ptr).
FAILED test_class_sh_property_non_owning.py::test_core_fld_common[core_fld_value_rw-expected1-False] - RuntimeError: Non-owning holder (loaded_as_shared_ptr).
FAILED test_class_sh_property_non_owning.py::test_core_fld_common[core_fld_value_ro-expected0-True] - RuntimeError: Non-owning holder (loaded_as_shared_ptr).
FAILED test_class_sh_property_non_owning.py::test_core_fld_common[core_fld_raw_ptr_ro-expected4-True] - RuntimeError: Non-owning holder (loaded_as_shared_ptr).
FAILED test_class_sh_property_non_owning.py::test_core_fld_common[core_fld_raw_ptr_ro-expected4-False] - RuntimeError: Non-owning holder (loaded_as_shared_ptr).
FAILED test_class_sh_property_non_owning.py::test_core_fld_common[core_fld_raw_ptr_rw-expected5-True] - RuntimeError: Non-owning holder (loaded_as_shared_ptr).
FAILED test_class_sh_property_non_owning.py::test_core_fld_common[core_fld_value_ro-expected0-False] - RuntimeError: Non-owning holder (loaded_as_shared_ptr).
FAILED test_class_sh_property_non_owning.py::test_core_fld_common[core_fld_raw_ptr_rw-expected5-False] - RuntimeError: Non-owning holder (loaded_as_shared_ptr).
================================================ 8 failed, 1056 passed, 9 skipped in 3.96s =================================================
```
2024-07-10 15:21:09 -07:00
Ralf W. Grosse-Kunstleve
f852cf5714 Copy code property_cpp_function<> code from smart_holder branch as-is. Ifdef out all but the "classic" implementation. 2024-07-10 15:16:59 -07:00
Ralf W. Grosse-Kunstleve
127058cb16 Replace loaded_v_h.type = get_type_info(typeid(type)) with loaded_v_h.type = typeinfo in unique_ptr type_caster and remove pytest.skip() in test_class_sh_disowning_mi.py 2024-07-10 08:18:24 -07:00
Ralf W. Grosse-Kunstleve
f855283be4 Restore test_class_sh_disowning_mi.cpp,py as it last worked with GHA (commit 575919098a). 2024-07-09 20:52:03 -07:00
Ralf W. Grosse-Kunstleve
7e1eced440 test_ccccccccccccccccccccc 2024-07-09 20:51:06 -07:00
Ralf W. Grosse-Kunstleve
d3537e4236 Reducing 2: rename C++ namespace from class_sh_disowning_mi to wip. 2024-07-09 20:37:47 -07:00
Ralf W. Grosse-Kunstleve
1ea3855b22 Reducing 1. 2024-07-09 20:29:58 -07:00
Ralf W. Grosse-Kunstleve
575919098a Remove all LOOOK in type_caster_base.h 2024-07-09 15:14:05 -07:00
Ralf W. Grosse-Kunstleve
e254264cc5 Restore original test_class_sh_disowning.py from smart_holder branch. 2024-07-09 15:09:39 -07:00
Ralf W. Grosse-Kunstleve
c6a87e8897 Call new load_helper.get_void_ptr_or_nullptr() instead of calling type_caster_generic::load_value() in shared_ptr and unique_ptr casters. 2024-07-09 15:07:10 -07:00
Ralf W. Grosse-Kunstleve
36bbac1b47 LOOOK to trace disowning 2024-07-09 14:09:38 -07:00
Ralf W. Grosse-Kunstleve
c5278145a0 Transfer test_mixed from test_class_sh_disowning to test_wip (preparation for debugging behavior difference). 2024-07-09 13:43:57 -07:00
Ralf W. Grosse-Kunstleve
c0f5078263 Pull in from smart_holder branch: Additional assert is_disowned() in test_class_sh_disowning.py (#5234) 2024-07-09 13:31:59 -07:00
Ralf W. Grosse-Kunstleve
5a4442f583 pytest.skip() right before m.disown_b(...) in test_class_sh_property_non_owning.py. Remove pytest.skip() in test_class_sh_shared_ptr_copy_move.py and test_class_sh_property_non_owning.py 2024-07-09 11:31:57 -07:00
Ralf W. Grosse-Kunstleve
752626d607 SegFault 20.04 C++11 MinSizeRel all tests, but only first pass after git clean -fdx
NO SegFault 20.04 C++11 Debug all tests, even in first pass after git clean -fdx
BUT then suddenly SegFaults not reproducible anymore even with MinSizeRel!?

Current thread 0x00007f6c7165e740 (most recent call first):
  File "/mounted_pybind11/tests/test_class_sh_shared_ptr_copy_move.py", line 9 in test_shptr_copy

line 9:
    lst = m.test_ShPtr_copy()

This line was added after last observed SegFault
    del mth
2024-07-09 08:39:48 -07:00
Ralf W. Grosse-Kunstleve
0296d33b1d Stress test: use PYBIND11_SMART_HOLDER_PADDING unconditionally. 2024-07-08 17:43:31 -07:00
Ralf W. Grosse-Kunstleve
625c88b845 Add two more pytest.skip("BAKEIN_BREAK: Segmentation fault") in test_class_sh_shared_ptr_copy_move.py 2024-07-08 16:12:44 -07:00
Ralf W. Grosse-Kunstleve
3ddfc58a2f Add two pytest.skip("BAKEIN_BREAK: Segmentation fault") in test_class_sh_shared_ptr_copy_move.py (suspected (!) holder mismatch). 2024-07-08 15:58:59 -07:00
Ralf W. Grosse-Kunstleve
cc86fb3256 test_class_sh_property_non_owning.py appears to cause memory corruption leading to failures in test_class_sh_shared_ptr_copy_move.py:
* Disable all tests in test_class_sh_property_non_owning.py

* Disable only `test_properties()` in test_class_sh_shared_ptr_copy_move.py

* Go back to original .github/workflows/ci.yml
2024-07-08 15:18:17 -07:00
Ralf W. Grosse-Kunstleve
91d40035a2 Disable for now (to be debugged later): CI / 🐍 3 • GCC 7 • C++17• x64 2024-07-07 13:27:16 -07:00
Ralf W. Grosse-Kunstleve
68c9d1311f Bring in tests/*class_sh_module_local* from smart_holder branch as-is (tests pass without any further changes). 2024-07-07 13:25:50 -07:00
Ralf W. Grosse-Kunstleve
9e3bee066b Bring in tests/test_classh_mock.cpp,py from smart_holder branch as-is (tests pass without any further changes). 2024-07-07 13:17:27 -07:00
Ralf W. Grosse-Kunstleve
ac055a41fc Add BAKEIN_BREAK in test_class_sh_property_non_owning.cpp,py 2024-07-07 13:04:41 -07:00
Ralf W. Grosse-Kunstleve
f52456e07a Bring in tests/test_class_sh_property_non_owning.cpp,py from smart_holder branch as-is (does not build). 2024-07-07 13:01:30 -07:00
Ralf W. Grosse-Kunstleve
b89ec00202 Add BAKEIN_BREAK in test_class_sh_property.cpp,py 2024-07-07 12:59:17 -07:00
Ralf W. Grosse-Kunstleve
c7bd78ccd6 Bring in tests/test_class_sh_property.cpp,py from smart_holder branch as-is (does not build). 2024-07-07 12:58:00 -07:00
Ralf W. Grosse-Kunstleve
c453013583 Move pytest.skip("BAKEIN_BREAK: ...) in test_class_sh_shared_ptr_copy_move.py to top level (still appears to fail on ~17 platforms). 2024-07-07 10:25:24 -07:00
Ralf W. Grosse-Kunstleve
f1f0eef3cb Remove cruft from smart_holder_type_caster_support::load_helper<> 2024-07-07 10:14:06 -07:00
Ralf W. Grosse-Kunstleve
7fc2ef4be5 Merge branch 'master' into bakein 2024-07-07 09:54:59 -07:00
Ralf W. Grosse-Kunstleve
89c58c4f7f Add pytest.skip("BAKEIN_BREAK: ...) in test_class_sh_shared_ptr_copy_move.py test_properties() (appears to fail on ~17 platforms). 2024-07-07 09:54:41 -07:00
Ralf W. Grosse-Kunstleve
b82892ae6f Resolve clang-tidy errors and warnings. 2024-07-07 09:43:53 -07:00
Ralf W. Grosse-Kunstleve
56019a2d68 Clean out debug code in test_class_sh_mi_thunks.cpp,py 2024-07-07 09:06:00 -07:00
Ralf W. Grosse-Kunstleve
011c795b5f test_class_sh_disowning.py: BAKEIN_WIP: Why is the behavior different? 2024-07-07 09:03:00 -07:00
Ralf W. Grosse-Kunstleve
bd37d69b67 Bug fix: typeid(cpptype) -> cpptype->name() 2024-07-07 08:54:55 -07:00