pybind11/tests/test_class_sh_basic.cpp

268 lines
11 KiB
C++
Raw Normal View History

Adding py::smart_holder (for smart-pointer interoperability). (#2672) * Adding test_unique_ptr_member (for desired PyCLIF behavior). See also: https://github.com/pybind/pybind11/issues/2583 Does not build with upstream master or https://github.com/pybind/pybind11/pull/2047, but builds with https://github.com/RobotLocomotion/pybind11 and almost runs: ``` Running tests in directory "/usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests": ================================================================================= test session starts ================================================================================= platform linux -- Python 3.8.5, pytest-5.4.3, py-1.9.0, pluggy-0.13.1 rootdir: /usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests, inifile: pytest.ini collected 2 items test_unique_ptr_member.py .F [100%] ====================================================================================== FAILURES ======================================================================================= _____________________________________________________________________________ test_pointee_and_ptr_owner ______________________________________________________________________________ def test_pointee_and_ptr_owner(): obj = m.pointee() assert obj.get_int() == 213 m.ptr_owner(obj) with pytest.raises(ValueError) as exc_info: > obj.get_int() E Failed: DID NOT RAISE <class 'ValueError'> test_unique_ptr_member.py:17: Failed ============================================================================= 1 failed, 1 passed in 0.06s ============================================================================= ``` * unique_ptr or shared_ptr return * new test_variant_unique_shared with vptr_holder prototype * moving prototype code to pybind11/vptr_holder.h, adding type_caster specialization to make the bindings involving unique_ptr passing compile, but load and cast implementations are missing * disabling GitHub Actions on pull_request (for this PR) * disabling AppVeyor (for this PR) * TRIGGER_SEGSEV macro, annotations for GET_STACK (vptr::get), GET_INT_STACK (pointee) * adding test_promotion_of_disowned_to_shared * Copying tests as-is from xxx_value_ptr_xxx_holder branch. https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder Systematically exercising returning and passing unique_ptr<T>, shared_ptr<T> with unique_ptr, shared_ptr holder. Observations: test_holder_unique_ptr: make_unique_pointee OK pass_unique_pointee BUILD_FAIL (as documented) make_shared_pointee Abort free(): double free detected pass_shared_pointee RuntimeError: Unable to load a custom holder type from a default-holder instance test_holder_shared_ptr: make_unique_pointee Segmentation fault (#1138) pass_unique_pointee BUILD_FAIL (as documented) make_shared_pointee OK pass_shared_pointee OK * Copying tests as-is from xxx_value_ptr_xxx_holder branch. https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder Systematically exercising casting between shared_ptr<base>, shared_ptr<derived>. * Demonstration of Undefined Behavior in handling of shared_ptr holder. Based on https://godbolt.org/z/4fdjaW by jorgbrown@ (thanks Jorg!). * Additional demonstration of Undefined Behavior in handling of shared_ptr holder. * fixing up-down mixup in comment * Demonstration of Undefined Behavior in handling of polymorphic pointers. (This demo does NOT involve smart pointers at all, unlike the otherwise similar test_smart_ptr_private_first_base.) * minor test_private_first_base.cpp simplification (after discovering that this can be wrapped with Boost.Python, using boost::noncopyable) * pybind11 equivalent of Boost.Python test similar to reproducer under #1333 * Snapshot of WIP, TODO: shared_ptr deleter with on/off switch * Adding vptr_deleter. * Adding from/as unique_ptr<T> and unique_ptr<T, D>. * Adding from_shared_ptr. Some polishing. * New tests/core/smart_holder_poc_test.cpp, using Catch2. * Adding in vptr_deleter_guard_flag. * Improved labeling of TEST_CASEs. * Shuffling existing TEST_CASEs into systematic matrix. * Implementing all [S]uccess tests. * Implementing all [E]xception tests. * Testing of exceptions not covered by the from-as matrix. * Adding top-level comment. * Converting from methods to factory functions (no functional change). * Removing obsolete and very incomplete test (replaced by Catch2-based test). * Removing stray file. * Adding type_caster_bare_interface_demo. * Adding shared_ptr<mpty>, shared_ptr<mpty const> casters. * Adding unique_ptr<mpty>, unique_ptr<mpty const> casters. * Pure copy of `class class_` implementation in pybind11.h (master commit 98f1bbb8004f654ba9e26717bdf5912fb899b05a). * classh.h: renaming of class_ to classh + namespace; forking test_classh_wip from test_type_caster_bare_interface_demo. * Hard-coding smart_holder into classh. * Adding mpty::mtxt string member. * Adding isinstance<mpty> in type_caster::load functions. * Adding rvalue_ref, renaming const_value_ref to lvalue_ref & removing const. * Retrieving smart_holder pointer in type_caster<mpty>::load, and using it cast_op operators. * Factoring out smart_holder_type_caster_load. * Retrieving smart_holder pointer in type_caster<std::shared_ptr<mpty[ const]>>::load, and using it cast_op operators. * Improved error messaging: Cannot disown nullptr (as_unique_ptr). * Retrieving smart_holder pointer in type_caster<std::unique_ptr<mpty[ const]>>::load, and using it cast_op operators. * Pure `clang-format --style=file -i` change. * Pure `clang-format --style=file -i` change, with two `clang-format off` directives. * Fixing oversight (discovered by flake8). * flake8 cleanup * Systematically setting mtxt for all rtrn_mpty_* functions (preparation, the values are not actually used yet). * static cast handle for rtrn_cptr works by simply dropping in code from type_caster_base (marked with comments). * static cast handle for rtrn_cref works by simply dropping in code from type_caster_base (marked with comments). rtrn_mref and rtrn_mptr work via const_cast (to add const). * static cast handle for rtrn_valu works by simply dropping in code from type_caster_base (marked with comments). rtrn_rref raises a RuntimeError, to be investigated. * Copying type_caster_generic::cast into type_caster<mpty> as-is (preparation for handling smart pointers). * Pure clang-format change (applied to original type_caster_generic::cast). * Adding comment re potential use_count data race. * static handle cast implementations for rtrn_shmp, rtrn_shcp. * Adding MISSING comments in operator std::unique_ptr<mpty[ const]>. * static handle cast implementations for rtrn_uqmp, rtrn_uqcp. * Bug fix: vptr_deleter_armed_flag_ptr has to live on the heap. See new bullet point in comment section near the top. The variable was also renamed to reflect its function more accurately. * Fixing bugs discovered by ASAN. The code is now ASAN, MSAN, UBSAN clean. * Making test_type_caster_bare_interface_demo.cpp slightly more realistic, ASAN, MSAN, UBSAN clean. * Calling deregister_instance after disowning via unique_ptr. * Removing enable_shared_from_this stub, simplifying existing code, clang-format. Open question, with respect to the original code: https://github.com/pybind/pybind11/blob/76a160070b369f8d82b945c97924227e8b835c94/include/pybind11/pybind11.h#L1510 To me it looks like the exact situation marked as `std::shared_ptr<Good> gp1 = not_so_good.getptr();` here: https://en.cppreference.com/w/cpp/memory/enable_shared_from_this The comment there is: `// undefined behavior (until C++17) and std::bad_weak_ptr thrown (since C++17)` Does the existing code have UB pre C++17? I'll leave handling of enable_shared_from_this for later, as the need arises. * Cosmetical change around helper functions. * Using type_caster_base<mpty>::src_and_type directly, removing copy. Also renaming one cast to cast_const_raw_ptr, for clarity. * Fixing clang-format oversight. * Using factored-out make_constructor (PR #2798), removing duplicate code. * Inserting additional assert to ensure a returned unique_ptr is always a new Python instance. * Adding minor comment (change to internals needed to distinguish uninitialized/disowned in error message). * Factoring out find_existing_python_instance(). * Moving factored-out make_constructor to test_classh_wip.cpp, restoring previous version of cast.h. This is currently the most practical approach. See PR #2798 for background. * Copying classh type_casters from test_classh_wip.cpp UNMODIFIED, as a baseline for generalizing the code. * Using pybind11/detail/classh_type_casters.h from test_classh_wip.cpp. * Adding & using PYBIND11_CLASSH_TYPE_CASTERS define. * Adding test_classh_inheritance, currently failing (passes with class_). * Removing .clang-format before git rebase master (where the file was added). * Bringing back .clang-format, the previous rm was a bad idea. * Folding in modified_type_caster_generic_load_impl, just enough to pass test_class_wip. test_classh_inheritance is still failing, but with a different error: [RuntimeError: Incompatible type (as_raw_ptr_unowned).] * Minimal changes needed to pass test_classh_inheritance. * First pass adjusting try_implicit_casts and try_load_foreign_module_local to capture loaded_v_h, but untested and guarded with pybind11_failure("Untested"). This was done mainly to determine general feasibility. Note the TODO in pybind11.h, where type_caster_generic::local_load is currently hard-coded. test_classh_wip and test_classh_inheritance still pass, as before. * Decoupling generic_type from type_caster_generic. * Changes and tests covering classh_type_casters try_implicit_casts. * Minimal test covering classh_type_casters load_impl Case 2b. * Removing stray isinstance<T>(src): it interferes with the py::module_local feature. Adding missing #includes. * Tests for classh py::module_local() feature. * Pure renaming of function names in test_classh_inheritance, similar to the systematic approach used in test_class_wip. NO functional changes. * Pure renaming of function and variable names, for better generalization when convoluting with inheritance. NO functional changes. * Adopting systematic naming scheme from test_classh_wip. NO functional changes. * Moving const after type name, for functions that cover a systematic scheme. NO functional changes. * Adding smart_holder_type_caster_load::loaded_as_shared_ptr, currently bypassing smart_holder shared_ptr tracking completely, but the tests pass and are sanitizer clean. * Removing rtti_held from smart_holder. See updated comment. * Cleaning up loaded_as_raw_ptr_unowned, loaded_as_shared_ptr. * Factoring out convert_type and folding into loaded_as_unique_ptr. * Folding convert_type into lvalue_ref and rvalue_ref paths. Some smart_holder_type_caster_load cleanup. * Using unique_ptr in local_load to replace static variable. Also adding local_load_safety_guard. * Converting test_unique_ptr_member to using classh: fully working, ASAN, MSAN, UBSAN clean. * Removing debugging comments (GET_STACK, GET_INT_STACK). cast.h is identical to current master again, pybind11.h only has the generic_type::initialize(..., &type_caster_generic::local_load) change. * Purging obsolete pybind11/vptr_holder.h and associated test. * Moving several tests to github.com/rwgk/rwgk_tbx/tree/main/pybind11_tests https://github.com/rwgk/rwgk_tbx/commit/a2c2f88174a30f5de80d7d26e0f77c7b60f5fb85 These tests are from experimenting, and for demonstrating UB in pybind11 multiple inheritance handling ("first_base"), to be fixed later. * Adding py::smart_holder support to py::class_, purging py::classh completely. * Renaming files in include directory, creating pybind11/smart_holder.h. * Renaming all "classh" to "smart_holder" in pybind11/detail/smart_holder_type_casters.h. The user-facing macro is now PYBIND11_SMART_HOLDER_TYPE_CASTERS. * Systematically renaming tests to use "class_sh" in the name. * Renaming test_type_caster_bare_interface_demo to test_type_caster_bare_interface. * Renaming new tests/core subdirectory to tests/pure_cpp. * Adding new tests to CMake config, resetting CI config. * Changing CMake file so that test_class_sh_module_local.py actually runs. * clang-tidy fixes. * 32-bit compatibility. * Reusing type_caster_base make_copy_constructor, make_move_constructor with a trick. * CMake COMPARE NATURAL is not available with older versions. * Adding copyright notices to new header files. * Explicitly define copy/move constructors/assignments. * Adding new header files to tests/extra_python_package/test_files.py. * Adding tests/pure_cpp/CMakeLists.txt. * Making use of the new find_existing_python_instance() function factored out with PR #2822. * Moving define PYBIND11_SMART_HOLDER_TYPE_CASTERS(T) down in the file. NO functional changes. Preparation for follow-up work (to keep that diff smaller). * Reintroducing py::classh, this time as a simple alias for py::class_<U, py::smart_holder>. * Replacing detail::is_smart_holder<H> in cast.h with detail::is_smart_holder_type_caster<T>. Moving get_local_load_function_ptr, init_instance_for_type to smart_holder_type_caster_class_hooks. Expanding static_assert in py::type::handle_of<> to accommodate smart_holder_type_casters. * Fixing oversight. * Adding classu alias for class_<U, std::unique_ptr<U>>. * Giving up on idea to use legacy init_instance only if is_base_of<type_caster_generic, type_caster<T>. There are use cases in the wild that define both a custom type_caster and class_. * Removing test_type_caster_bare_interface, which was moved to the separate PR #2834. * Moving up is_smart_holder_type_caster, to also use in cast_is_temporary_value_reference. * Adding smart_holder_type_casters for unique_ptr with custom deleter. SEVERE CODE DUPLICATION. This commit is to establish a baseline for consolidating the unique_ptr code. * Unification of unique_ptr, unique_ptr_with_deleter code in smart_holder_poc.h. Leads to more fitting error messages. Enables use of unique_ptr<T, D> smart_holder_type_casters also for unique_ptr<T>. * Copying files as-is from branch test_unique_ptr_member (PR #2672). * Adding comment, simplifying naming, cmake addition. * Introducing PYBIND11_USE_SMART_HOLDER_AS_DEFAULT macro (tested only undefined; there are many errors with the macro defined). * Removing test_type_caster_bare_interface, which was moved to the separate PR #2834. * Fixing oversight introduced with commit 95425f13d6c14fcb6ee479b62b602dc8a605ec49. * Setting record.default_holder correctly for PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. With this test_class.cpp builds and even mostly runs, except `test_multiple_instances_with_same_pointer`, which segfaults because it is using a `unique_ptr` holder but `smart_holder` `type_caster`. Also adding `static_assert`s to generate build errors for such situations, but guarding with `#if 0` to first pivot to test_factory_constructors.cpp. * Fixing up cast.h and smart_holder.h after rebase. * Removing detail/smart_holder_type_casters.h in separate commit. * Commenting out const in def_buffer(... const). With this, test_buffers builds and runs with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. Explanation why the const needs to be removed, or fix elsewhere, is still needed, but left for later. * Adding test_class_sh_factory_constructors, reproducing test_factory_constructors failure. Using py::class_ in this commit, to be changed to py::classh for debugging. * Removing include/pybind11/detail/smart_holder_type_casters.h from CMakeLists.txt, test_files.py (since it does not exist in this branch). * Adding // DANGER ZONE reminders. * Converting as many py::class_ to py::classh as possible, not breaking tests. * Adding initimpl::construct() overloads, resulting in test_class_sh_factory_constructors feature parity for py::class_ and py::classh. * Adding enable_if !is_smart_holder_type_caster to existing initimpl::construct(). With this test_factory_constructors.cpp builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. * Disabling shared_ptr&, shared_ptr* tests when building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT for now, pending work on smart_holder_type_caster<shared_ptr>. * Factoring out struct and class definitions into anonymous namespace. Preparation for building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. * Simplifying from_unique_ptr(): typename D = std::default_delete<T> is not needed. Factoring out is_std_default_delete<T>() for consistentcy between ensure_compatible_rtti_uqp_del() and from_unique_ptr(). * Introducing PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. Using it in test_smart_ptr.cpp. With this test_smart_ptr builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT and all but one test run successfully. * Introducing 1. type_caster_for_class_, used in PYBIND11_MAKE_OPAQUE, and 2. default_holder_type, used in stl_bind.h. * Using __VA_ARGS__ in PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. * Replacing condense_for_macro with much simpler approach. * Softening static_assert, to only check specifically that smart_holder is not mixed with type_caster_base, and unique_ptr/shared_ptr holders are not mixed with smart_holder_type_casters. * Adding PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS in test_class.cpp (with this all but one test succeed with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT). * Adding remaining PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. static_assert for "necessary conditions" for both types of default holder, static_assert for "strict conditions" guarded by new PYBIND11_STRICT_ASSERTS_CLASS_HOLDER_VS_TYPE_CASTER_MIX. All tests build & run as before with unique_ptr as the default holder, all tests build for smart_holder as the default holder, even with the strict static_assert. * Introducing check_is_smart_holder_type_caster() function for runtime check, and reinterpreting record.default_holder as "uses_unique_ptr_holder". With this test_smart_ptr succeeds. (All 42 tests build, 35 tests succeed, 5 run but have some failures, 2 segfault.) * Bug fix: Adding have_value() to smart_holder_type_caster_load. With this test_builtin_casters succeeds. (All 42 tests build, 36 tests succeed, 5 run but have some failures, 1 segfault.) * Adding unowned_void_ptr_from_direct_conversion to modified_type_caster_generic_load_impl. This fixes the last remaining segfault (test_numpy_dtypes). New stats for all tests combined: 12 failed, 458 passed. * Adding "Lazy allocation for unallocated values" (for old-style __init__) into load_value_and_holder. Deferring destruction of disowned holder until clear_instance, to remain inspectable for "uninitialized" or "disowned" detection. New stats for all tests combined: 5 failed, 465 passed. * Changing std::shared_ptr pointer/reference to const pointer/reference. New stats for all tests combined: 4 failed, 466 passed. * Adding return_value_policy::move to permissible policies for unique_ptr returns. New stats for all tests combined: 3 failed, 467 passed. * Overlooked flake8 fixes. * Manipulating failing ConstructorStats test to pass, to be able to run all tests with ASAN. This version of the code is ASAN clean with unique_ptr or smart_holder as the default. This change needs to be reverted after adopting the existing move-only-if-refcount-is-1 logic used by type_caster_base. * Adding copy constructor and move constructor tracking to atyp. Preparation for a follow-up change in smart_holder_type_caster, to make this test sensitive to the changing behavior. [skip ci] * Removing `operator T&&() &&` from smart_holder_type_caster, for compatibility with the behavior of type_caster_base. Enables reverting 2 of 3 test manipulations applied under commit 249df7cbdb09817fed0ddf80f01ba5af12466820. The manipulation in test_factory_constructors.py is NOT reverted in this commit. [skip ci] * Fixing unfortunate editing mishap. This reverts the last remaining test manipulation in commit 249df7cbdb09817fed0ddf80f01ba5af12466820 and makes all existing unit tests pass with smart_holder as default holder. * GitHub CI clang-tidy fixes. * Adding messages to terse `static_assert`s, for pre-C++17 compatibility. * Using @pytest.mark.parametrize to run each assert separately (to see all errors, not just the first). * Systematically removing _atyp from function names, to make the test code simpler. * Using re.match to accommodate variable number of intermediate MvCtor. * Also removing `operator T()` from smart_holder_type_caster, to fix gcc compilation errors. The only loss is pass_rref in test_class_sh_basic. * Systematically replacing `detail::enable_if_t<...smart_holder...>` with `typename std::enable_if<...smart_holder...>::type`. Attempt to work around MSVC 2015 issues, to be tested via GitHub CI. The idea for this change originates from this comment: https://github.com/pybind/pybind11/issues/1616#issuecomment-444536813 * Importing re before pytest after observing a PyPy CI flake when importing pytest first. * Copying MSVC 2015 compatibility change from branch pr2672_use_smart_holder_as_default. * Introducing is_smart_holder_type_caster_base_tag, to keep smart_holder code more disconnected. * Working around MSVC 2015 bug. * Expanding comment for MSVC 2015 workaround. * Systematically changing std::enable_if back to detail::enable_if_t, effectively reverting commit 5d4b6890a337ae1bbaec4091f4195606f89a3b06. * Removing unused smart_holder_type_caster_load::loaded_as_rvalue_ref (it was an oversight that it was not removed with commit 23036a45eb4731a06b488ec1fdf83bca677b7f67). * Removing py::classu, because it does not seem useful enough. * Reverting commit 63495313066119dcf7510c2ae8b468b46c12ef8f by un-commenting `const` in `def_buffer(...)`. To make this possible, `operator T const&` and `operator T const*` in `smart_holder_type_caster` need to be marked as `const` member functions. * Adding construct() overloads for constructing smart_holder from alias unique_ptr, shared_ptr returns. * Adding test_class_sh_factory_constructors.cpp to tests/CMakeLists.txt (fixes oversight, this should have been added long before). * Compatibility with old clang versions (clang 3.6, 3.7 C++11). * Cleaning up changes to existing unit tests. * Systematically adding SMART_HOLDER_WIP tag. Removing minor UNTESTED tags (only the throw are not actually exercised, investing time there has a high cost but very little benefit). * Splitting out smart_holder_type_casters again, into new detail/smart_holder_type_casters_inline_include.h. * Splitting out smart_holder_init_inline_include.h. * Adding additional new include files to CMakeLists.txt, tests/extra_python_package/test_files.py. * clang-format cleanup of most smart_holder code. * Adding source code comments in response to review. * Simple micro-benchmark ("ubench") comparing runtime performance for several holders. Tested using github.com/rwgk/pybind11_scons and Google-internal build system. Sorry, no cmake support at the moment. First results: https://docs.google.com/spreadsheets/d/1InapCYws2Gt-stmFf_Bwl33eOMo3aLE_gc9adveY7RU/edit#gid=0 * Breaking out number_bucket.h, adding hook for also collecting performance data for PyCLIF. * Accounting for ubench in MANIFEST.in (simply prune, for now). * Smarter determination of call_repetitions. [skip ci] * Also scaling performance data to PyCLIF. [skip ci] * Adding ubench/python/number_bucket.clif here for general visibility. * Fix after rebase * Merging detail/smart_holder_init_inline_include.h into detail/init.h. * Renaming detail/is_smart_holder_type_caster.h -> detail/smart_holder_sfinae_hooks_only.h. * Renaming is_smart_holder_type_caster -> type_uses_smart_holder_type_caster for clarity. * Renaming type_caster_type_is_smart_holder_type_caster -> wrapped_type_uses_smart_holder_type_caster for clarity. * Renaming is_smart_holder_type_caster_base_tag -> smart_holder_type_caster_base_tag for simplicity. * Adding copyright notices and minor colateral cleanup. * iwyu cleanup (comprehensive only for cast.h and smart_holder*.h files). * Fixing `git rebase master` accident. * Moving large `pragma warning` block from pybind11.h to detail/common.h. * Fixing another `git rebase master` accident.
2021-02-24 05:50:42 +00:00
#include <pybind11/smart_holder.h>
#include "pybind11_tests.h"
Adding py::smart_holder (for smart-pointer interoperability). (#2672) * Adding test_unique_ptr_member (for desired PyCLIF behavior). See also: https://github.com/pybind/pybind11/issues/2583 Does not build with upstream master or https://github.com/pybind/pybind11/pull/2047, but builds with https://github.com/RobotLocomotion/pybind11 and almost runs: ``` Running tests in directory "/usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests": ================================================================================= test session starts ================================================================================= platform linux -- Python 3.8.5, pytest-5.4.3, py-1.9.0, pluggy-0.13.1 rootdir: /usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests, inifile: pytest.ini collected 2 items test_unique_ptr_member.py .F [100%] ====================================================================================== FAILURES ======================================================================================= _____________________________________________________________________________ test_pointee_and_ptr_owner ______________________________________________________________________________ def test_pointee_and_ptr_owner(): obj = m.pointee() assert obj.get_int() == 213 m.ptr_owner(obj) with pytest.raises(ValueError) as exc_info: > obj.get_int() E Failed: DID NOT RAISE <class 'ValueError'> test_unique_ptr_member.py:17: Failed ============================================================================= 1 failed, 1 passed in 0.06s ============================================================================= ``` * unique_ptr or shared_ptr return * new test_variant_unique_shared with vptr_holder prototype * moving prototype code to pybind11/vptr_holder.h, adding type_caster specialization to make the bindings involving unique_ptr passing compile, but load and cast implementations are missing * disabling GitHub Actions on pull_request (for this PR) * disabling AppVeyor (for this PR) * TRIGGER_SEGSEV macro, annotations for GET_STACK (vptr::get), GET_INT_STACK (pointee) * adding test_promotion_of_disowned_to_shared * Copying tests as-is from xxx_value_ptr_xxx_holder branch. https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder Systematically exercising returning and passing unique_ptr<T>, shared_ptr<T> with unique_ptr, shared_ptr holder. Observations: test_holder_unique_ptr: make_unique_pointee OK pass_unique_pointee BUILD_FAIL (as documented) make_shared_pointee Abort free(): double free detected pass_shared_pointee RuntimeError: Unable to load a custom holder type from a default-holder instance test_holder_shared_ptr: make_unique_pointee Segmentation fault (#1138) pass_unique_pointee BUILD_FAIL (as documented) make_shared_pointee OK pass_shared_pointee OK * Copying tests as-is from xxx_value_ptr_xxx_holder branch. https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder Systematically exercising casting between shared_ptr<base>, shared_ptr<derived>. * Demonstration of Undefined Behavior in handling of shared_ptr holder. Based on https://godbolt.org/z/4fdjaW by jorgbrown@ (thanks Jorg!). * Additional demonstration of Undefined Behavior in handling of shared_ptr holder. * fixing up-down mixup in comment * Demonstration of Undefined Behavior in handling of polymorphic pointers. (This demo does NOT involve smart pointers at all, unlike the otherwise similar test_smart_ptr_private_first_base.) * minor test_private_first_base.cpp simplification (after discovering that this can be wrapped with Boost.Python, using boost::noncopyable) * pybind11 equivalent of Boost.Python test similar to reproducer under #1333 * Snapshot of WIP, TODO: shared_ptr deleter with on/off switch * Adding vptr_deleter. * Adding from/as unique_ptr<T> and unique_ptr<T, D>. * Adding from_shared_ptr. Some polishing. * New tests/core/smart_holder_poc_test.cpp, using Catch2. * Adding in vptr_deleter_guard_flag. * Improved labeling of TEST_CASEs. * Shuffling existing TEST_CASEs into systematic matrix. * Implementing all [S]uccess tests. * Implementing all [E]xception tests. * Testing of exceptions not covered by the from-as matrix. * Adding top-level comment. * Converting from methods to factory functions (no functional change). * Removing obsolete and very incomplete test (replaced by Catch2-based test). * Removing stray file. * Adding type_caster_bare_interface_demo. * Adding shared_ptr<mpty>, shared_ptr<mpty const> casters. * Adding unique_ptr<mpty>, unique_ptr<mpty const> casters. * Pure copy of `class class_` implementation in pybind11.h (master commit 98f1bbb8004f654ba9e26717bdf5912fb899b05a). * classh.h: renaming of class_ to classh + namespace; forking test_classh_wip from test_type_caster_bare_interface_demo. * Hard-coding smart_holder into classh. * Adding mpty::mtxt string member. * Adding isinstance<mpty> in type_caster::load functions. * Adding rvalue_ref, renaming const_value_ref to lvalue_ref & removing const. * Retrieving smart_holder pointer in type_caster<mpty>::load, and using it cast_op operators. * Factoring out smart_holder_type_caster_load. * Retrieving smart_holder pointer in type_caster<std::shared_ptr<mpty[ const]>>::load, and using it cast_op operators. * Improved error messaging: Cannot disown nullptr (as_unique_ptr). * Retrieving smart_holder pointer in type_caster<std::unique_ptr<mpty[ const]>>::load, and using it cast_op operators. * Pure `clang-format --style=file -i` change. * Pure `clang-format --style=file -i` change, with two `clang-format off` directives. * Fixing oversight (discovered by flake8). * flake8 cleanup * Systematically setting mtxt for all rtrn_mpty_* functions (preparation, the values are not actually used yet). * static cast handle for rtrn_cptr works by simply dropping in code from type_caster_base (marked with comments). * static cast handle for rtrn_cref works by simply dropping in code from type_caster_base (marked with comments). rtrn_mref and rtrn_mptr work via const_cast (to add const). * static cast handle for rtrn_valu works by simply dropping in code from type_caster_base (marked with comments). rtrn_rref raises a RuntimeError, to be investigated. * Copying type_caster_generic::cast into type_caster<mpty> as-is (preparation for handling smart pointers). * Pure clang-format change (applied to original type_caster_generic::cast). * Adding comment re potential use_count data race. * static handle cast implementations for rtrn_shmp, rtrn_shcp. * Adding MISSING comments in operator std::unique_ptr<mpty[ const]>. * static handle cast implementations for rtrn_uqmp, rtrn_uqcp. * Bug fix: vptr_deleter_armed_flag_ptr has to live on the heap. See new bullet point in comment section near the top. The variable was also renamed to reflect its function more accurately. * Fixing bugs discovered by ASAN. The code is now ASAN, MSAN, UBSAN clean. * Making test_type_caster_bare_interface_demo.cpp slightly more realistic, ASAN, MSAN, UBSAN clean. * Calling deregister_instance after disowning via unique_ptr. * Removing enable_shared_from_this stub, simplifying existing code, clang-format. Open question, with respect to the original code: https://github.com/pybind/pybind11/blob/76a160070b369f8d82b945c97924227e8b835c94/include/pybind11/pybind11.h#L1510 To me it looks like the exact situation marked as `std::shared_ptr<Good> gp1 = not_so_good.getptr();` here: https://en.cppreference.com/w/cpp/memory/enable_shared_from_this The comment there is: `// undefined behavior (until C++17) and std::bad_weak_ptr thrown (since C++17)` Does the existing code have UB pre C++17? I'll leave handling of enable_shared_from_this for later, as the need arises. * Cosmetical change around helper functions. * Using type_caster_base<mpty>::src_and_type directly, removing copy. Also renaming one cast to cast_const_raw_ptr, for clarity. * Fixing clang-format oversight. * Using factored-out make_constructor (PR #2798), removing duplicate code. * Inserting additional assert to ensure a returned unique_ptr is always a new Python instance. * Adding minor comment (change to internals needed to distinguish uninitialized/disowned in error message). * Factoring out find_existing_python_instance(). * Moving factored-out make_constructor to test_classh_wip.cpp, restoring previous version of cast.h. This is currently the most practical approach. See PR #2798 for background. * Copying classh type_casters from test_classh_wip.cpp UNMODIFIED, as a baseline for generalizing the code. * Using pybind11/detail/classh_type_casters.h from test_classh_wip.cpp. * Adding & using PYBIND11_CLASSH_TYPE_CASTERS define. * Adding test_classh_inheritance, currently failing (passes with class_). * Removing .clang-format before git rebase master (where the file was added). * Bringing back .clang-format, the previous rm was a bad idea. * Folding in modified_type_caster_generic_load_impl, just enough to pass test_class_wip. test_classh_inheritance is still failing, but with a different error: [RuntimeError: Incompatible type (as_raw_ptr_unowned).] * Minimal changes needed to pass test_classh_inheritance. * First pass adjusting try_implicit_casts and try_load_foreign_module_local to capture loaded_v_h, but untested and guarded with pybind11_failure("Untested"). This was done mainly to determine general feasibility. Note the TODO in pybind11.h, where type_caster_generic::local_load is currently hard-coded. test_classh_wip and test_classh_inheritance still pass, as before. * Decoupling generic_type from type_caster_generic. * Changes and tests covering classh_type_casters try_implicit_casts. * Minimal test covering classh_type_casters load_impl Case 2b. * Removing stray isinstance<T>(src): it interferes with the py::module_local feature. Adding missing #includes. * Tests for classh py::module_local() feature. * Pure renaming of function names in test_classh_inheritance, similar to the systematic approach used in test_class_wip. NO functional changes. * Pure renaming of function and variable names, for better generalization when convoluting with inheritance. NO functional changes. * Adopting systematic naming scheme from test_classh_wip. NO functional changes. * Moving const after type name, for functions that cover a systematic scheme. NO functional changes. * Adding smart_holder_type_caster_load::loaded_as_shared_ptr, currently bypassing smart_holder shared_ptr tracking completely, but the tests pass and are sanitizer clean. * Removing rtti_held from smart_holder. See updated comment. * Cleaning up loaded_as_raw_ptr_unowned, loaded_as_shared_ptr. * Factoring out convert_type and folding into loaded_as_unique_ptr. * Folding convert_type into lvalue_ref and rvalue_ref paths. Some smart_holder_type_caster_load cleanup. * Using unique_ptr in local_load to replace static variable. Also adding local_load_safety_guard. * Converting test_unique_ptr_member to using classh: fully working, ASAN, MSAN, UBSAN clean. * Removing debugging comments (GET_STACK, GET_INT_STACK). cast.h is identical to current master again, pybind11.h only has the generic_type::initialize(..., &type_caster_generic::local_load) change. * Purging obsolete pybind11/vptr_holder.h and associated test. * Moving several tests to github.com/rwgk/rwgk_tbx/tree/main/pybind11_tests https://github.com/rwgk/rwgk_tbx/commit/a2c2f88174a30f5de80d7d26e0f77c7b60f5fb85 These tests are from experimenting, and for demonstrating UB in pybind11 multiple inheritance handling ("first_base"), to be fixed later. * Adding py::smart_holder support to py::class_, purging py::classh completely. * Renaming files in include directory, creating pybind11/smart_holder.h. * Renaming all "classh" to "smart_holder" in pybind11/detail/smart_holder_type_casters.h. The user-facing macro is now PYBIND11_SMART_HOLDER_TYPE_CASTERS. * Systematically renaming tests to use "class_sh" in the name. * Renaming test_type_caster_bare_interface_demo to test_type_caster_bare_interface. * Renaming new tests/core subdirectory to tests/pure_cpp. * Adding new tests to CMake config, resetting CI config. * Changing CMake file so that test_class_sh_module_local.py actually runs. * clang-tidy fixes. * 32-bit compatibility. * Reusing type_caster_base make_copy_constructor, make_move_constructor with a trick. * CMake COMPARE NATURAL is not available with older versions. * Adding copyright notices to new header files. * Explicitly define copy/move constructors/assignments. * Adding new header files to tests/extra_python_package/test_files.py. * Adding tests/pure_cpp/CMakeLists.txt. * Making use of the new find_existing_python_instance() function factored out with PR #2822. * Moving define PYBIND11_SMART_HOLDER_TYPE_CASTERS(T) down in the file. NO functional changes. Preparation for follow-up work (to keep that diff smaller). * Reintroducing py::classh, this time as a simple alias for py::class_<U, py::smart_holder>. * Replacing detail::is_smart_holder<H> in cast.h with detail::is_smart_holder_type_caster<T>. Moving get_local_load_function_ptr, init_instance_for_type to smart_holder_type_caster_class_hooks. Expanding static_assert in py::type::handle_of<> to accommodate smart_holder_type_casters. * Fixing oversight. * Adding classu alias for class_<U, std::unique_ptr<U>>. * Giving up on idea to use legacy init_instance only if is_base_of<type_caster_generic, type_caster<T>. There are use cases in the wild that define both a custom type_caster and class_. * Removing test_type_caster_bare_interface, which was moved to the separate PR #2834. * Moving up is_smart_holder_type_caster, to also use in cast_is_temporary_value_reference. * Adding smart_holder_type_casters for unique_ptr with custom deleter. SEVERE CODE DUPLICATION. This commit is to establish a baseline for consolidating the unique_ptr code. * Unification of unique_ptr, unique_ptr_with_deleter code in smart_holder_poc.h. Leads to more fitting error messages. Enables use of unique_ptr<T, D> smart_holder_type_casters also for unique_ptr<T>. * Copying files as-is from branch test_unique_ptr_member (PR #2672). * Adding comment, simplifying naming, cmake addition. * Introducing PYBIND11_USE_SMART_HOLDER_AS_DEFAULT macro (tested only undefined; there are many errors with the macro defined). * Removing test_type_caster_bare_interface, which was moved to the separate PR #2834. * Fixing oversight introduced with commit 95425f13d6c14fcb6ee479b62b602dc8a605ec49. * Setting record.default_holder correctly for PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. With this test_class.cpp builds and even mostly runs, except `test_multiple_instances_with_same_pointer`, which segfaults because it is using a `unique_ptr` holder but `smart_holder` `type_caster`. Also adding `static_assert`s to generate build errors for such situations, but guarding with `#if 0` to first pivot to test_factory_constructors.cpp. * Fixing up cast.h and smart_holder.h after rebase. * Removing detail/smart_holder_type_casters.h in separate commit. * Commenting out const in def_buffer(... const). With this, test_buffers builds and runs with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. Explanation why the const needs to be removed, or fix elsewhere, is still needed, but left for later. * Adding test_class_sh_factory_constructors, reproducing test_factory_constructors failure. Using py::class_ in this commit, to be changed to py::classh for debugging. * Removing include/pybind11/detail/smart_holder_type_casters.h from CMakeLists.txt, test_files.py (since it does not exist in this branch). * Adding // DANGER ZONE reminders. * Converting as many py::class_ to py::classh as possible, not breaking tests. * Adding initimpl::construct() overloads, resulting in test_class_sh_factory_constructors feature parity for py::class_ and py::classh. * Adding enable_if !is_smart_holder_type_caster to existing initimpl::construct(). With this test_factory_constructors.cpp builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. * Disabling shared_ptr&, shared_ptr* tests when building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT for now, pending work on smart_holder_type_caster<shared_ptr>. * Factoring out struct and class definitions into anonymous namespace. Preparation for building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. * Simplifying from_unique_ptr(): typename D = std::default_delete<T> is not needed. Factoring out is_std_default_delete<T>() for consistentcy between ensure_compatible_rtti_uqp_del() and from_unique_ptr(). * Introducing PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. Using it in test_smart_ptr.cpp. With this test_smart_ptr builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT and all but one test run successfully. * Introducing 1. type_caster_for_class_, used in PYBIND11_MAKE_OPAQUE, and 2. default_holder_type, used in stl_bind.h. * Using __VA_ARGS__ in PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. * Replacing condense_for_macro with much simpler approach. * Softening static_assert, to only check specifically that smart_holder is not mixed with type_caster_base, and unique_ptr/shared_ptr holders are not mixed with smart_holder_type_casters. * Adding PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS in test_class.cpp (with this all but one test succeed with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT). * Adding remaining PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. static_assert for "necessary conditions" for both types of default holder, static_assert for "strict conditions" guarded by new PYBIND11_STRICT_ASSERTS_CLASS_HOLDER_VS_TYPE_CASTER_MIX. All tests build & run as before with unique_ptr as the default holder, all tests build for smart_holder as the default holder, even with the strict static_assert. * Introducing check_is_smart_holder_type_caster() function for runtime check, and reinterpreting record.default_holder as "uses_unique_ptr_holder". With this test_smart_ptr succeeds. (All 42 tests build, 35 tests succeed, 5 run but have some failures, 2 segfault.) * Bug fix: Adding have_value() to smart_holder_type_caster_load. With this test_builtin_casters succeeds. (All 42 tests build, 36 tests succeed, 5 run but have some failures, 1 segfault.) * Adding unowned_void_ptr_from_direct_conversion to modified_type_caster_generic_load_impl. This fixes the last remaining segfault (test_numpy_dtypes). New stats for all tests combined: 12 failed, 458 passed. * Adding "Lazy allocation for unallocated values" (for old-style __init__) into load_value_and_holder. Deferring destruction of disowned holder until clear_instance, to remain inspectable for "uninitialized" or "disowned" detection. New stats for all tests combined: 5 failed, 465 passed. * Changing std::shared_ptr pointer/reference to const pointer/reference. New stats for all tests combined: 4 failed, 466 passed. * Adding return_value_policy::move to permissible policies for unique_ptr returns. New stats for all tests combined: 3 failed, 467 passed. * Overlooked flake8 fixes. * Manipulating failing ConstructorStats test to pass, to be able to run all tests with ASAN. This version of the code is ASAN clean with unique_ptr or smart_holder as the default. This change needs to be reverted after adopting the existing move-only-if-refcount-is-1 logic used by type_caster_base. * Adding copy constructor and move constructor tracking to atyp. Preparation for a follow-up change in smart_holder_type_caster, to make this test sensitive to the changing behavior. [skip ci] * Removing `operator T&&() &&` from smart_holder_type_caster, for compatibility with the behavior of type_caster_base. Enables reverting 2 of 3 test manipulations applied under commit 249df7cbdb09817fed0ddf80f01ba5af12466820. The manipulation in test_factory_constructors.py is NOT reverted in this commit. [skip ci] * Fixing unfortunate editing mishap. This reverts the last remaining test manipulation in commit 249df7cbdb09817fed0ddf80f01ba5af12466820 and makes all existing unit tests pass with smart_holder as default holder. * GitHub CI clang-tidy fixes. * Adding messages to terse `static_assert`s, for pre-C++17 compatibility. * Using @pytest.mark.parametrize to run each assert separately (to see all errors, not just the first). * Systematically removing _atyp from function names, to make the test code simpler. * Using re.match to accommodate variable number of intermediate MvCtor. * Also removing `operator T()` from smart_holder_type_caster, to fix gcc compilation errors. The only loss is pass_rref in test_class_sh_basic. * Systematically replacing `detail::enable_if_t<...smart_holder...>` with `typename std::enable_if<...smart_holder...>::type`. Attempt to work around MSVC 2015 issues, to be tested via GitHub CI. The idea for this change originates from this comment: https://github.com/pybind/pybind11/issues/1616#issuecomment-444536813 * Importing re before pytest after observing a PyPy CI flake when importing pytest first. * Copying MSVC 2015 compatibility change from branch pr2672_use_smart_holder_as_default. * Introducing is_smart_holder_type_caster_base_tag, to keep smart_holder code more disconnected. * Working around MSVC 2015 bug. * Expanding comment for MSVC 2015 workaround. * Systematically changing std::enable_if back to detail::enable_if_t, effectively reverting commit 5d4b6890a337ae1bbaec4091f4195606f89a3b06. * Removing unused smart_holder_type_caster_load::loaded_as_rvalue_ref (it was an oversight that it was not removed with commit 23036a45eb4731a06b488ec1fdf83bca677b7f67). * Removing py::classu, because it does not seem useful enough. * Reverting commit 63495313066119dcf7510c2ae8b468b46c12ef8f by un-commenting `const` in `def_buffer(...)`. To make this possible, `operator T const&` and `operator T const*` in `smart_holder_type_caster` need to be marked as `const` member functions. * Adding construct() overloads for constructing smart_holder from alias unique_ptr, shared_ptr returns. * Adding test_class_sh_factory_constructors.cpp to tests/CMakeLists.txt (fixes oversight, this should have been added long before). * Compatibility with old clang versions (clang 3.6, 3.7 C++11). * Cleaning up changes to existing unit tests. * Systematically adding SMART_HOLDER_WIP tag. Removing minor UNTESTED tags (only the throw are not actually exercised, investing time there has a high cost but very little benefit). * Splitting out smart_holder_type_casters again, into new detail/smart_holder_type_casters_inline_include.h. * Splitting out smart_holder_init_inline_include.h. * Adding additional new include files to CMakeLists.txt, tests/extra_python_package/test_files.py. * clang-format cleanup of most smart_holder code. * Adding source code comments in response to review. * Simple micro-benchmark ("ubench") comparing runtime performance for several holders. Tested using github.com/rwgk/pybind11_scons and Google-internal build system. Sorry, no cmake support at the moment. First results: https://docs.google.com/spreadsheets/d/1InapCYws2Gt-stmFf_Bwl33eOMo3aLE_gc9adveY7RU/edit#gid=0 * Breaking out number_bucket.h, adding hook for also collecting performance data for PyCLIF. * Accounting for ubench in MANIFEST.in (simply prune, for now). * Smarter determination of call_repetitions. [skip ci] * Also scaling performance data to PyCLIF. [skip ci] * Adding ubench/python/number_bucket.clif here for general visibility. * Fix after rebase * Merging detail/smart_holder_init_inline_include.h into detail/init.h. * Renaming detail/is_smart_holder_type_caster.h -> detail/smart_holder_sfinae_hooks_only.h. * Renaming is_smart_holder_type_caster -> type_uses_smart_holder_type_caster for clarity. * Renaming type_caster_type_is_smart_holder_type_caster -> wrapped_type_uses_smart_holder_type_caster for clarity. * Renaming is_smart_holder_type_caster_base_tag -> smart_holder_type_caster_base_tag for simplicity. * Adding copyright notices and minor colateral cleanup. * iwyu cleanup (comprehensive only for cast.h and smart_holder*.h files). * Fixing `git rebase master` accident. * Moving large `pragma warning` block from pybind11.h to detail/common.h. * Fixing another `git rebase master` accident.
2021-02-24 05:50:42 +00:00
#include <memory>
#include <string>
#include <vector>
Adding py::smart_holder (for smart-pointer interoperability). (#2672) * Adding test_unique_ptr_member (for desired PyCLIF behavior). See also: https://github.com/pybind/pybind11/issues/2583 Does not build with upstream master or https://github.com/pybind/pybind11/pull/2047, but builds with https://github.com/RobotLocomotion/pybind11 and almost runs: ``` Running tests in directory "/usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests": ================================================================================= test session starts ================================================================================= platform linux -- Python 3.8.5, pytest-5.4.3, py-1.9.0, pluggy-0.13.1 rootdir: /usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests, inifile: pytest.ini collected 2 items test_unique_ptr_member.py .F [100%] ====================================================================================== FAILURES ======================================================================================= _____________________________________________________________________________ test_pointee_and_ptr_owner ______________________________________________________________________________ def test_pointee_and_ptr_owner(): obj = m.pointee() assert obj.get_int() == 213 m.ptr_owner(obj) with pytest.raises(ValueError) as exc_info: > obj.get_int() E Failed: DID NOT RAISE <class 'ValueError'> test_unique_ptr_member.py:17: Failed ============================================================================= 1 failed, 1 passed in 0.06s ============================================================================= ``` * unique_ptr or shared_ptr return * new test_variant_unique_shared with vptr_holder prototype * moving prototype code to pybind11/vptr_holder.h, adding type_caster specialization to make the bindings involving unique_ptr passing compile, but load and cast implementations are missing * disabling GitHub Actions on pull_request (for this PR) * disabling AppVeyor (for this PR) * TRIGGER_SEGSEV macro, annotations for GET_STACK (vptr::get), GET_INT_STACK (pointee) * adding test_promotion_of_disowned_to_shared * Copying tests as-is from xxx_value_ptr_xxx_holder branch. https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder Systematically exercising returning and passing unique_ptr<T>, shared_ptr<T> with unique_ptr, shared_ptr holder. Observations: test_holder_unique_ptr: make_unique_pointee OK pass_unique_pointee BUILD_FAIL (as documented) make_shared_pointee Abort free(): double free detected pass_shared_pointee RuntimeError: Unable to load a custom holder type from a default-holder instance test_holder_shared_ptr: make_unique_pointee Segmentation fault (#1138) pass_unique_pointee BUILD_FAIL (as documented) make_shared_pointee OK pass_shared_pointee OK * Copying tests as-is from xxx_value_ptr_xxx_holder branch. https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder Systematically exercising casting between shared_ptr<base>, shared_ptr<derived>. * Demonstration of Undefined Behavior in handling of shared_ptr holder. Based on https://godbolt.org/z/4fdjaW by jorgbrown@ (thanks Jorg!). * Additional demonstration of Undefined Behavior in handling of shared_ptr holder. * fixing up-down mixup in comment * Demonstration of Undefined Behavior in handling of polymorphic pointers. (This demo does NOT involve smart pointers at all, unlike the otherwise similar test_smart_ptr_private_first_base.) * minor test_private_first_base.cpp simplification (after discovering that this can be wrapped with Boost.Python, using boost::noncopyable) * pybind11 equivalent of Boost.Python test similar to reproducer under #1333 * Snapshot of WIP, TODO: shared_ptr deleter with on/off switch * Adding vptr_deleter. * Adding from/as unique_ptr<T> and unique_ptr<T, D>. * Adding from_shared_ptr. Some polishing. * New tests/core/smart_holder_poc_test.cpp, using Catch2. * Adding in vptr_deleter_guard_flag. * Improved labeling of TEST_CASEs. * Shuffling existing TEST_CASEs into systematic matrix. * Implementing all [S]uccess tests. * Implementing all [E]xception tests. * Testing of exceptions not covered by the from-as matrix. * Adding top-level comment. * Converting from methods to factory functions (no functional change). * Removing obsolete and very incomplete test (replaced by Catch2-based test). * Removing stray file. * Adding type_caster_bare_interface_demo. * Adding shared_ptr<mpty>, shared_ptr<mpty const> casters. * Adding unique_ptr<mpty>, unique_ptr<mpty const> casters. * Pure copy of `class class_` implementation in pybind11.h (master commit 98f1bbb8004f654ba9e26717bdf5912fb899b05a). * classh.h: renaming of class_ to classh + namespace; forking test_classh_wip from test_type_caster_bare_interface_demo. * Hard-coding smart_holder into classh. * Adding mpty::mtxt string member. * Adding isinstance<mpty> in type_caster::load functions. * Adding rvalue_ref, renaming const_value_ref to lvalue_ref & removing const. * Retrieving smart_holder pointer in type_caster<mpty>::load, and using it cast_op operators. * Factoring out smart_holder_type_caster_load. * Retrieving smart_holder pointer in type_caster<std::shared_ptr<mpty[ const]>>::load, and using it cast_op operators. * Improved error messaging: Cannot disown nullptr (as_unique_ptr). * Retrieving smart_holder pointer in type_caster<std::unique_ptr<mpty[ const]>>::load, and using it cast_op operators. * Pure `clang-format --style=file -i` change. * Pure `clang-format --style=file -i` change, with two `clang-format off` directives. * Fixing oversight (discovered by flake8). * flake8 cleanup * Systematically setting mtxt for all rtrn_mpty_* functions (preparation, the values are not actually used yet). * static cast handle for rtrn_cptr works by simply dropping in code from type_caster_base (marked with comments). * static cast handle for rtrn_cref works by simply dropping in code from type_caster_base (marked with comments). rtrn_mref and rtrn_mptr work via const_cast (to add const). * static cast handle for rtrn_valu works by simply dropping in code from type_caster_base (marked with comments). rtrn_rref raises a RuntimeError, to be investigated. * Copying type_caster_generic::cast into type_caster<mpty> as-is (preparation for handling smart pointers). * Pure clang-format change (applied to original type_caster_generic::cast). * Adding comment re potential use_count data race. * static handle cast implementations for rtrn_shmp, rtrn_shcp. * Adding MISSING comments in operator std::unique_ptr<mpty[ const]>. * static handle cast implementations for rtrn_uqmp, rtrn_uqcp. * Bug fix: vptr_deleter_armed_flag_ptr has to live on the heap. See new bullet point in comment section near the top. The variable was also renamed to reflect its function more accurately. * Fixing bugs discovered by ASAN. The code is now ASAN, MSAN, UBSAN clean. * Making test_type_caster_bare_interface_demo.cpp slightly more realistic, ASAN, MSAN, UBSAN clean. * Calling deregister_instance after disowning via unique_ptr. * Removing enable_shared_from_this stub, simplifying existing code, clang-format. Open question, with respect to the original code: https://github.com/pybind/pybind11/blob/76a160070b369f8d82b945c97924227e8b835c94/include/pybind11/pybind11.h#L1510 To me it looks like the exact situation marked as `std::shared_ptr<Good> gp1 = not_so_good.getptr();` here: https://en.cppreference.com/w/cpp/memory/enable_shared_from_this The comment there is: `// undefined behavior (until C++17) and std::bad_weak_ptr thrown (since C++17)` Does the existing code have UB pre C++17? I'll leave handling of enable_shared_from_this for later, as the need arises. * Cosmetical change around helper functions. * Using type_caster_base<mpty>::src_and_type directly, removing copy. Also renaming one cast to cast_const_raw_ptr, for clarity. * Fixing clang-format oversight. * Using factored-out make_constructor (PR #2798), removing duplicate code. * Inserting additional assert to ensure a returned unique_ptr is always a new Python instance. * Adding minor comment (change to internals needed to distinguish uninitialized/disowned in error message). * Factoring out find_existing_python_instance(). * Moving factored-out make_constructor to test_classh_wip.cpp, restoring previous version of cast.h. This is currently the most practical approach. See PR #2798 for background. * Copying classh type_casters from test_classh_wip.cpp UNMODIFIED, as a baseline for generalizing the code. * Using pybind11/detail/classh_type_casters.h from test_classh_wip.cpp. * Adding & using PYBIND11_CLASSH_TYPE_CASTERS define. * Adding test_classh_inheritance, currently failing (passes with class_). * Removing .clang-format before git rebase master (where the file was added). * Bringing back .clang-format, the previous rm was a bad idea. * Folding in modified_type_caster_generic_load_impl, just enough to pass test_class_wip. test_classh_inheritance is still failing, but with a different error: [RuntimeError: Incompatible type (as_raw_ptr_unowned).] * Minimal changes needed to pass test_classh_inheritance. * First pass adjusting try_implicit_casts and try_load_foreign_module_local to capture loaded_v_h, but untested and guarded with pybind11_failure("Untested"). This was done mainly to determine general feasibility. Note the TODO in pybind11.h, where type_caster_generic::local_load is currently hard-coded. test_classh_wip and test_classh_inheritance still pass, as before. * Decoupling generic_type from type_caster_generic. * Changes and tests covering classh_type_casters try_implicit_casts. * Minimal test covering classh_type_casters load_impl Case 2b. * Removing stray isinstance<T>(src): it interferes with the py::module_local feature. Adding missing #includes. * Tests for classh py::module_local() feature. * Pure renaming of function names in test_classh_inheritance, similar to the systematic approach used in test_class_wip. NO functional changes. * Pure renaming of function and variable names, for better generalization when convoluting with inheritance. NO functional changes. * Adopting systematic naming scheme from test_classh_wip. NO functional changes. * Moving const after type name, for functions that cover a systematic scheme. NO functional changes. * Adding smart_holder_type_caster_load::loaded_as_shared_ptr, currently bypassing smart_holder shared_ptr tracking completely, but the tests pass and are sanitizer clean. * Removing rtti_held from smart_holder. See updated comment. * Cleaning up loaded_as_raw_ptr_unowned, loaded_as_shared_ptr. * Factoring out convert_type and folding into loaded_as_unique_ptr. * Folding convert_type into lvalue_ref and rvalue_ref paths. Some smart_holder_type_caster_load cleanup. * Using unique_ptr in local_load to replace static variable. Also adding local_load_safety_guard. * Converting test_unique_ptr_member to using classh: fully working, ASAN, MSAN, UBSAN clean. * Removing debugging comments (GET_STACK, GET_INT_STACK). cast.h is identical to current master again, pybind11.h only has the generic_type::initialize(..., &type_caster_generic::local_load) change. * Purging obsolete pybind11/vptr_holder.h and associated test. * Moving several tests to github.com/rwgk/rwgk_tbx/tree/main/pybind11_tests https://github.com/rwgk/rwgk_tbx/commit/a2c2f88174a30f5de80d7d26e0f77c7b60f5fb85 These tests are from experimenting, and for demonstrating UB in pybind11 multiple inheritance handling ("first_base"), to be fixed later. * Adding py::smart_holder support to py::class_, purging py::classh completely. * Renaming files in include directory, creating pybind11/smart_holder.h. * Renaming all "classh" to "smart_holder" in pybind11/detail/smart_holder_type_casters.h. The user-facing macro is now PYBIND11_SMART_HOLDER_TYPE_CASTERS. * Systematically renaming tests to use "class_sh" in the name. * Renaming test_type_caster_bare_interface_demo to test_type_caster_bare_interface. * Renaming new tests/core subdirectory to tests/pure_cpp. * Adding new tests to CMake config, resetting CI config. * Changing CMake file so that test_class_sh_module_local.py actually runs. * clang-tidy fixes. * 32-bit compatibility. * Reusing type_caster_base make_copy_constructor, make_move_constructor with a trick. * CMake COMPARE NATURAL is not available with older versions. * Adding copyright notices to new header files. * Explicitly define copy/move constructors/assignments. * Adding new header files to tests/extra_python_package/test_files.py. * Adding tests/pure_cpp/CMakeLists.txt. * Making use of the new find_existing_python_instance() function factored out with PR #2822. * Moving define PYBIND11_SMART_HOLDER_TYPE_CASTERS(T) down in the file. NO functional changes. Preparation for follow-up work (to keep that diff smaller). * Reintroducing py::classh, this time as a simple alias for py::class_<U, py::smart_holder>. * Replacing detail::is_smart_holder<H> in cast.h with detail::is_smart_holder_type_caster<T>. Moving get_local_load_function_ptr, init_instance_for_type to smart_holder_type_caster_class_hooks. Expanding static_assert in py::type::handle_of<> to accommodate smart_holder_type_casters. * Fixing oversight. * Adding classu alias for class_<U, std::unique_ptr<U>>. * Giving up on idea to use legacy init_instance only if is_base_of<type_caster_generic, type_caster<T>. There are use cases in the wild that define both a custom type_caster and class_. * Removing test_type_caster_bare_interface, which was moved to the separate PR #2834. * Moving up is_smart_holder_type_caster, to also use in cast_is_temporary_value_reference. * Adding smart_holder_type_casters for unique_ptr with custom deleter. SEVERE CODE DUPLICATION. This commit is to establish a baseline for consolidating the unique_ptr code. * Unification of unique_ptr, unique_ptr_with_deleter code in smart_holder_poc.h. Leads to more fitting error messages. Enables use of unique_ptr<T, D> smart_holder_type_casters also for unique_ptr<T>. * Copying files as-is from branch test_unique_ptr_member (PR #2672). * Adding comment, simplifying naming, cmake addition. * Introducing PYBIND11_USE_SMART_HOLDER_AS_DEFAULT macro (tested only undefined; there are many errors with the macro defined). * Removing test_type_caster_bare_interface, which was moved to the separate PR #2834. * Fixing oversight introduced with commit 95425f13d6c14fcb6ee479b62b602dc8a605ec49. * Setting record.default_holder correctly for PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. With this test_class.cpp builds and even mostly runs, except `test_multiple_instances_with_same_pointer`, which segfaults because it is using a `unique_ptr` holder but `smart_holder` `type_caster`. Also adding `static_assert`s to generate build errors for such situations, but guarding with `#if 0` to first pivot to test_factory_constructors.cpp. * Fixing up cast.h and smart_holder.h after rebase. * Removing detail/smart_holder_type_casters.h in separate commit. * Commenting out const in def_buffer(... const). With this, test_buffers builds and runs with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. Explanation why the const needs to be removed, or fix elsewhere, is still needed, but left for later. * Adding test_class_sh_factory_constructors, reproducing test_factory_constructors failure. Using py::class_ in this commit, to be changed to py::classh for debugging. * Removing include/pybind11/detail/smart_holder_type_casters.h from CMakeLists.txt, test_files.py (since it does not exist in this branch). * Adding // DANGER ZONE reminders. * Converting as many py::class_ to py::classh as possible, not breaking tests. * Adding initimpl::construct() overloads, resulting in test_class_sh_factory_constructors feature parity for py::class_ and py::classh. * Adding enable_if !is_smart_holder_type_caster to existing initimpl::construct(). With this test_factory_constructors.cpp builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. * Disabling shared_ptr&, shared_ptr* tests when building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT for now, pending work on smart_holder_type_caster<shared_ptr>. * Factoring out struct and class definitions into anonymous namespace. Preparation for building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. * Simplifying from_unique_ptr(): typename D = std::default_delete<T> is not needed. Factoring out is_std_default_delete<T>() for consistentcy between ensure_compatible_rtti_uqp_del() and from_unique_ptr(). * Introducing PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. Using it in test_smart_ptr.cpp. With this test_smart_ptr builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT and all but one test run successfully. * Introducing 1. type_caster_for_class_, used in PYBIND11_MAKE_OPAQUE, and 2. default_holder_type, used in stl_bind.h. * Using __VA_ARGS__ in PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. * Replacing condense_for_macro with much simpler approach. * Softening static_assert, to only check specifically that smart_holder is not mixed with type_caster_base, and unique_ptr/shared_ptr holders are not mixed with smart_holder_type_casters. * Adding PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS in test_class.cpp (with this all but one test succeed with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT). * Adding remaining PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. static_assert for "necessary conditions" for both types of default holder, static_assert for "strict conditions" guarded by new PYBIND11_STRICT_ASSERTS_CLASS_HOLDER_VS_TYPE_CASTER_MIX. All tests build & run as before with unique_ptr as the default holder, all tests build for smart_holder as the default holder, even with the strict static_assert. * Introducing check_is_smart_holder_type_caster() function for runtime check, and reinterpreting record.default_holder as "uses_unique_ptr_holder". With this test_smart_ptr succeeds. (All 42 tests build, 35 tests succeed, 5 run but have some failures, 2 segfault.) * Bug fix: Adding have_value() to smart_holder_type_caster_load. With this test_builtin_casters succeeds. (All 42 tests build, 36 tests succeed, 5 run but have some failures, 1 segfault.) * Adding unowned_void_ptr_from_direct_conversion to modified_type_caster_generic_load_impl. This fixes the last remaining segfault (test_numpy_dtypes). New stats for all tests combined: 12 failed, 458 passed. * Adding "Lazy allocation for unallocated values" (for old-style __init__) into load_value_and_holder. Deferring destruction of disowned holder until clear_instance, to remain inspectable for "uninitialized" or "disowned" detection. New stats for all tests combined: 5 failed, 465 passed. * Changing std::shared_ptr pointer/reference to const pointer/reference. New stats for all tests combined: 4 failed, 466 passed. * Adding return_value_policy::move to permissible policies for unique_ptr returns. New stats for all tests combined: 3 failed, 467 passed. * Overlooked flake8 fixes. * Manipulating failing ConstructorStats test to pass, to be able to run all tests with ASAN. This version of the code is ASAN clean with unique_ptr or smart_holder as the default. This change needs to be reverted after adopting the existing move-only-if-refcount-is-1 logic used by type_caster_base. * Adding copy constructor and move constructor tracking to atyp. Preparation for a follow-up change in smart_holder_type_caster, to make this test sensitive to the changing behavior. [skip ci] * Removing `operator T&&() &&` from smart_holder_type_caster, for compatibility with the behavior of type_caster_base. Enables reverting 2 of 3 test manipulations applied under commit 249df7cbdb09817fed0ddf80f01ba5af12466820. The manipulation in test_factory_constructors.py is NOT reverted in this commit. [skip ci] * Fixing unfortunate editing mishap. This reverts the last remaining test manipulation in commit 249df7cbdb09817fed0ddf80f01ba5af12466820 and makes all existing unit tests pass with smart_holder as default holder. * GitHub CI clang-tidy fixes. * Adding messages to terse `static_assert`s, for pre-C++17 compatibility. * Using @pytest.mark.parametrize to run each assert separately (to see all errors, not just the first). * Systematically removing _atyp from function names, to make the test code simpler. * Using re.match to accommodate variable number of intermediate MvCtor. * Also removing `operator T()` from smart_holder_type_caster, to fix gcc compilation errors. The only loss is pass_rref in test_class_sh_basic. * Systematically replacing `detail::enable_if_t<...smart_holder...>` with `typename std::enable_if<...smart_holder...>::type`. Attempt to work around MSVC 2015 issues, to be tested via GitHub CI. The idea for this change originates from this comment: https://github.com/pybind/pybind11/issues/1616#issuecomment-444536813 * Importing re before pytest after observing a PyPy CI flake when importing pytest first. * Copying MSVC 2015 compatibility change from branch pr2672_use_smart_holder_as_default. * Introducing is_smart_holder_type_caster_base_tag, to keep smart_holder code more disconnected. * Working around MSVC 2015 bug. * Expanding comment for MSVC 2015 workaround. * Systematically changing std::enable_if back to detail::enable_if_t, effectively reverting commit 5d4b6890a337ae1bbaec4091f4195606f89a3b06. * Removing unused smart_holder_type_caster_load::loaded_as_rvalue_ref (it was an oversight that it was not removed with commit 23036a45eb4731a06b488ec1fdf83bca677b7f67). * Removing py::classu, because it does not seem useful enough. * Reverting commit 63495313066119dcf7510c2ae8b468b46c12ef8f by un-commenting `const` in `def_buffer(...)`. To make this possible, `operator T const&` and `operator T const*` in `smart_holder_type_caster` need to be marked as `const` member functions. * Adding construct() overloads for constructing smart_holder from alias unique_ptr, shared_ptr returns. * Adding test_class_sh_factory_constructors.cpp to tests/CMakeLists.txt (fixes oversight, this should have been added long before). * Compatibility with old clang versions (clang 3.6, 3.7 C++11). * Cleaning up changes to existing unit tests. * Systematically adding SMART_HOLDER_WIP tag. Removing minor UNTESTED tags (only the throw are not actually exercised, investing time there has a high cost but very little benefit). * Splitting out smart_holder_type_casters again, into new detail/smart_holder_type_casters_inline_include.h. * Splitting out smart_holder_init_inline_include.h. * Adding additional new include files to CMakeLists.txt, tests/extra_python_package/test_files.py. * clang-format cleanup of most smart_holder code. * Adding source code comments in response to review. * Simple micro-benchmark ("ubench") comparing runtime performance for several holders. Tested using github.com/rwgk/pybind11_scons and Google-internal build system. Sorry, no cmake support at the moment. First results: https://docs.google.com/spreadsheets/d/1InapCYws2Gt-stmFf_Bwl33eOMo3aLE_gc9adveY7RU/edit#gid=0 * Breaking out number_bucket.h, adding hook for also collecting performance data for PyCLIF. * Accounting for ubench in MANIFEST.in (simply prune, for now). * Smarter determination of call_repetitions. [skip ci] * Also scaling performance data to PyCLIF. [skip ci] * Adding ubench/python/number_bucket.clif here for general visibility. * Fix after rebase * Merging detail/smart_holder_init_inline_include.h into detail/init.h. * Renaming detail/is_smart_holder_type_caster.h -> detail/smart_holder_sfinae_hooks_only.h. * Renaming is_smart_holder_type_caster -> type_uses_smart_holder_type_caster for clarity. * Renaming type_caster_type_is_smart_holder_type_caster -> wrapped_type_uses_smart_holder_type_caster for clarity. * Renaming is_smart_holder_type_caster_base_tag -> smart_holder_type_caster_base_tag for simplicity. * Adding copyright notices and minor colateral cleanup. * iwyu cleanup (comprehensive only for cast.h and smart_holder*.h files). * Fixing `git rebase master` accident. * Moving large `pragma warning` block from pybind11.h to detail/common.h. * Fixing another `git rebase master` accident.
2021-02-24 05:50:42 +00:00
namespace pybind11_tests {
namespace class_sh_basic {
struct atyp { // Short for "any type".
std::string mtxt;
atyp() : mtxt("DefaultConstructor") {}
2021-09-09 02:42:56 +00:00
explicit atyp(const std::string &mtxt_) : mtxt(mtxt_) {}
Adding py::smart_holder (for smart-pointer interoperability). (#2672) * Adding test_unique_ptr_member (for desired PyCLIF behavior). See also: https://github.com/pybind/pybind11/issues/2583 Does not build with upstream master or https://github.com/pybind/pybind11/pull/2047, but builds with https://github.com/RobotLocomotion/pybind11 and almost runs: ``` Running tests in directory "/usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests": ================================================================================= test session starts ================================================================================= platform linux -- Python 3.8.5, pytest-5.4.3, py-1.9.0, pluggy-0.13.1 rootdir: /usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests, inifile: pytest.ini collected 2 items test_unique_ptr_member.py .F [100%] ====================================================================================== FAILURES ======================================================================================= _____________________________________________________________________________ test_pointee_and_ptr_owner ______________________________________________________________________________ def test_pointee_and_ptr_owner(): obj = m.pointee() assert obj.get_int() == 213 m.ptr_owner(obj) with pytest.raises(ValueError) as exc_info: > obj.get_int() E Failed: DID NOT RAISE <class 'ValueError'> test_unique_ptr_member.py:17: Failed ============================================================================= 1 failed, 1 passed in 0.06s ============================================================================= ``` * unique_ptr or shared_ptr return * new test_variant_unique_shared with vptr_holder prototype * moving prototype code to pybind11/vptr_holder.h, adding type_caster specialization to make the bindings involving unique_ptr passing compile, but load and cast implementations are missing * disabling GitHub Actions on pull_request (for this PR) * disabling AppVeyor (for this PR) * TRIGGER_SEGSEV macro, annotations for GET_STACK (vptr::get), GET_INT_STACK (pointee) * adding test_promotion_of_disowned_to_shared * Copying tests as-is from xxx_value_ptr_xxx_holder branch. https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder Systematically exercising returning and passing unique_ptr<T>, shared_ptr<T> with unique_ptr, shared_ptr holder. Observations: test_holder_unique_ptr: make_unique_pointee OK pass_unique_pointee BUILD_FAIL (as documented) make_shared_pointee Abort free(): double free detected pass_shared_pointee RuntimeError: Unable to load a custom holder type from a default-holder instance test_holder_shared_ptr: make_unique_pointee Segmentation fault (#1138) pass_unique_pointee BUILD_FAIL (as documented) make_shared_pointee OK pass_shared_pointee OK * Copying tests as-is from xxx_value_ptr_xxx_holder branch. https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder Systematically exercising casting between shared_ptr<base>, shared_ptr<derived>. * Demonstration of Undefined Behavior in handling of shared_ptr holder. Based on https://godbolt.org/z/4fdjaW by jorgbrown@ (thanks Jorg!). * Additional demonstration of Undefined Behavior in handling of shared_ptr holder. * fixing up-down mixup in comment * Demonstration of Undefined Behavior in handling of polymorphic pointers. (This demo does NOT involve smart pointers at all, unlike the otherwise similar test_smart_ptr_private_first_base.) * minor test_private_first_base.cpp simplification (after discovering that this can be wrapped with Boost.Python, using boost::noncopyable) * pybind11 equivalent of Boost.Python test similar to reproducer under #1333 * Snapshot of WIP, TODO: shared_ptr deleter with on/off switch * Adding vptr_deleter. * Adding from/as unique_ptr<T> and unique_ptr<T, D>. * Adding from_shared_ptr. Some polishing. * New tests/core/smart_holder_poc_test.cpp, using Catch2. * Adding in vptr_deleter_guard_flag. * Improved labeling of TEST_CASEs. * Shuffling existing TEST_CASEs into systematic matrix. * Implementing all [S]uccess tests. * Implementing all [E]xception tests. * Testing of exceptions not covered by the from-as matrix. * Adding top-level comment. * Converting from methods to factory functions (no functional change). * Removing obsolete and very incomplete test (replaced by Catch2-based test). * Removing stray file. * Adding type_caster_bare_interface_demo. * Adding shared_ptr<mpty>, shared_ptr<mpty const> casters. * Adding unique_ptr<mpty>, unique_ptr<mpty const> casters. * Pure copy of `class class_` implementation in pybind11.h (master commit 98f1bbb8004f654ba9e26717bdf5912fb899b05a). * classh.h: renaming of class_ to classh + namespace; forking test_classh_wip from test_type_caster_bare_interface_demo. * Hard-coding smart_holder into classh. * Adding mpty::mtxt string member. * Adding isinstance<mpty> in type_caster::load functions. * Adding rvalue_ref, renaming const_value_ref to lvalue_ref & removing const. * Retrieving smart_holder pointer in type_caster<mpty>::load, and using it cast_op operators. * Factoring out smart_holder_type_caster_load. * Retrieving smart_holder pointer in type_caster<std::shared_ptr<mpty[ const]>>::load, and using it cast_op operators. * Improved error messaging: Cannot disown nullptr (as_unique_ptr). * Retrieving smart_holder pointer in type_caster<std::unique_ptr<mpty[ const]>>::load, and using it cast_op operators. * Pure `clang-format --style=file -i` change. * Pure `clang-format --style=file -i` change, with two `clang-format off` directives. * Fixing oversight (discovered by flake8). * flake8 cleanup * Systematically setting mtxt for all rtrn_mpty_* functions (preparation, the values are not actually used yet). * static cast handle for rtrn_cptr works by simply dropping in code from type_caster_base (marked with comments). * static cast handle for rtrn_cref works by simply dropping in code from type_caster_base (marked with comments). rtrn_mref and rtrn_mptr work via const_cast (to add const). * static cast handle for rtrn_valu works by simply dropping in code from type_caster_base (marked with comments). rtrn_rref raises a RuntimeError, to be investigated. * Copying type_caster_generic::cast into type_caster<mpty> as-is (preparation for handling smart pointers). * Pure clang-format change (applied to original type_caster_generic::cast). * Adding comment re potential use_count data race. * static handle cast implementations for rtrn_shmp, rtrn_shcp. * Adding MISSING comments in operator std::unique_ptr<mpty[ const]>. * static handle cast implementations for rtrn_uqmp, rtrn_uqcp. * Bug fix: vptr_deleter_armed_flag_ptr has to live on the heap. See new bullet point in comment section near the top. The variable was also renamed to reflect its function more accurately. * Fixing bugs discovered by ASAN. The code is now ASAN, MSAN, UBSAN clean. * Making test_type_caster_bare_interface_demo.cpp slightly more realistic, ASAN, MSAN, UBSAN clean. * Calling deregister_instance after disowning via unique_ptr. * Removing enable_shared_from_this stub, simplifying existing code, clang-format. Open question, with respect to the original code: https://github.com/pybind/pybind11/blob/76a160070b369f8d82b945c97924227e8b835c94/include/pybind11/pybind11.h#L1510 To me it looks like the exact situation marked as `std::shared_ptr<Good> gp1 = not_so_good.getptr();` here: https://en.cppreference.com/w/cpp/memory/enable_shared_from_this The comment there is: `// undefined behavior (until C++17) and std::bad_weak_ptr thrown (since C++17)` Does the existing code have UB pre C++17? I'll leave handling of enable_shared_from_this for later, as the need arises. * Cosmetical change around helper functions. * Using type_caster_base<mpty>::src_and_type directly, removing copy. Also renaming one cast to cast_const_raw_ptr, for clarity. * Fixing clang-format oversight. * Using factored-out make_constructor (PR #2798), removing duplicate code. * Inserting additional assert to ensure a returned unique_ptr is always a new Python instance. * Adding minor comment (change to internals needed to distinguish uninitialized/disowned in error message). * Factoring out find_existing_python_instance(). * Moving factored-out make_constructor to test_classh_wip.cpp, restoring previous version of cast.h. This is currently the most practical approach. See PR #2798 for background. * Copying classh type_casters from test_classh_wip.cpp UNMODIFIED, as a baseline for generalizing the code. * Using pybind11/detail/classh_type_casters.h from test_classh_wip.cpp. * Adding & using PYBIND11_CLASSH_TYPE_CASTERS define. * Adding test_classh_inheritance, currently failing (passes with class_). * Removing .clang-format before git rebase master (where the file was added). * Bringing back .clang-format, the previous rm was a bad idea. * Folding in modified_type_caster_generic_load_impl, just enough to pass test_class_wip. test_classh_inheritance is still failing, but with a different error: [RuntimeError: Incompatible type (as_raw_ptr_unowned).] * Minimal changes needed to pass test_classh_inheritance. * First pass adjusting try_implicit_casts and try_load_foreign_module_local to capture loaded_v_h, but untested and guarded with pybind11_failure("Untested"). This was done mainly to determine general feasibility. Note the TODO in pybind11.h, where type_caster_generic::local_load is currently hard-coded. test_classh_wip and test_classh_inheritance still pass, as before. * Decoupling generic_type from type_caster_generic. * Changes and tests covering classh_type_casters try_implicit_casts. * Minimal test covering classh_type_casters load_impl Case 2b. * Removing stray isinstance<T>(src): it interferes with the py::module_local feature. Adding missing #includes. * Tests for classh py::module_local() feature. * Pure renaming of function names in test_classh_inheritance, similar to the systematic approach used in test_class_wip. NO functional changes. * Pure renaming of function and variable names, for better generalization when convoluting with inheritance. NO functional changes. * Adopting systematic naming scheme from test_classh_wip. NO functional changes. * Moving const after type name, for functions that cover a systematic scheme. NO functional changes. * Adding smart_holder_type_caster_load::loaded_as_shared_ptr, currently bypassing smart_holder shared_ptr tracking completely, but the tests pass and are sanitizer clean. * Removing rtti_held from smart_holder. See updated comment. * Cleaning up loaded_as_raw_ptr_unowned, loaded_as_shared_ptr. * Factoring out convert_type and folding into loaded_as_unique_ptr. * Folding convert_type into lvalue_ref and rvalue_ref paths. Some smart_holder_type_caster_load cleanup. * Using unique_ptr in local_load to replace static variable. Also adding local_load_safety_guard. * Converting test_unique_ptr_member to using classh: fully working, ASAN, MSAN, UBSAN clean. * Removing debugging comments (GET_STACK, GET_INT_STACK). cast.h is identical to current master again, pybind11.h only has the generic_type::initialize(..., &type_caster_generic::local_load) change. * Purging obsolete pybind11/vptr_holder.h and associated test. * Moving several tests to github.com/rwgk/rwgk_tbx/tree/main/pybind11_tests https://github.com/rwgk/rwgk_tbx/commit/a2c2f88174a30f5de80d7d26e0f77c7b60f5fb85 These tests are from experimenting, and for demonstrating UB in pybind11 multiple inheritance handling ("first_base"), to be fixed later. * Adding py::smart_holder support to py::class_, purging py::classh completely. * Renaming files in include directory, creating pybind11/smart_holder.h. * Renaming all "classh" to "smart_holder" in pybind11/detail/smart_holder_type_casters.h. The user-facing macro is now PYBIND11_SMART_HOLDER_TYPE_CASTERS. * Systematically renaming tests to use "class_sh" in the name. * Renaming test_type_caster_bare_interface_demo to test_type_caster_bare_interface. * Renaming new tests/core subdirectory to tests/pure_cpp. * Adding new tests to CMake config, resetting CI config. * Changing CMake file so that test_class_sh_module_local.py actually runs. * clang-tidy fixes. * 32-bit compatibility. * Reusing type_caster_base make_copy_constructor, make_move_constructor with a trick. * CMake COMPARE NATURAL is not available with older versions. * Adding copyright notices to new header files. * Explicitly define copy/move constructors/assignments. * Adding new header files to tests/extra_python_package/test_files.py. * Adding tests/pure_cpp/CMakeLists.txt. * Making use of the new find_existing_python_instance() function factored out with PR #2822. * Moving define PYBIND11_SMART_HOLDER_TYPE_CASTERS(T) down in the file. NO functional changes. Preparation for follow-up work (to keep that diff smaller). * Reintroducing py::classh, this time as a simple alias for py::class_<U, py::smart_holder>. * Replacing detail::is_smart_holder<H> in cast.h with detail::is_smart_holder_type_caster<T>. Moving get_local_load_function_ptr, init_instance_for_type to smart_holder_type_caster_class_hooks. Expanding static_assert in py::type::handle_of<> to accommodate smart_holder_type_casters. * Fixing oversight. * Adding classu alias for class_<U, std::unique_ptr<U>>. * Giving up on idea to use legacy init_instance only if is_base_of<type_caster_generic, type_caster<T>. There are use cases in the wild that define both a custom type_caster and class_. * Removing test_type_caster_bare_interface, which was moved to the separate PR #2834. * Moving up is_smart_holder_type_caster, to also use in cast_is_temporary_value_reference. * Adding smart_holder_type_casters for unique_ptr with custom deleter. SEVERE CODE DUPLICATION. This commit is to establish a baseline for consolidating the unique_ptr code. * Unification of unique_ptr, unique_ptr_with_deleter code in smart_holder_poc.h. Leads to more fitting error messages. Enables use of unique_ptr<T, D> smart_holder_type_casters also for unique_ptr<T>. * Copying files as-is from branch test_unique_ptr_member (PR #2672). * Adding comment, simplifying naming, cmake addition. * Introducing PYBIND11_USE_SMART_HOLDER_AS_DEFAULT macro (tested only undefined; there are many errors with the macro defined). * Removing test_type_caster_bare_interface, which was moved to the separate PR #2834. * Fixing oversight introduced with commit 95425f13d6c14fcb6ee479b62b602dc8a605ec49. * Setting record.default_holder correctly for PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. With this test_class.cpp builds and even mostly runs, except `test_multiple_instances_with_same_pointer`, which segfaults because it is using a `unique_ptr` holder but `smart_holder` `type_caster`. Also adding `static_assert`s to generate build errors for such situations, but guarding with `#if 0` to first pivot to test_factory_constructors.cpp. * Fixing up cast.h and smart_holder.h after rebase. * Removing detail/smart_holder_type_casters.h in separate commit. * Commenting out const in def_buffer(... const). With this, test_buffers builds and runs with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. Explanation why the const needs to be removed, or fix elsewhere, is still needed, but left for later. * Adding test_class_sh_factory_constructors, reproducing test_factory_constructors failure. Using py::class_ in this commit, to be changed to py::classh for debugging. * Removing include/pybind11/detail/smart_holder_type_casters.h from CMakeLists.txt, test_files.py (since it does not exist in this branch). * Adding // DANGER ZONE reminders. * Converting as many py::class_ to py::classh as possible, not breaking tests. * Adding initimpl::construct() overloads, resulting in test_class_sh_factory_constructors feature parity for py::class_ and py::classh. * Adding enable_if !is_smart_holder_type_caster to existing initimpl::construct(). With this test_factory_constructors.cpp builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. * Disabling shared_ptr&, shared_ptr* tests when building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT for now, pending work on smart_holder_type_caster<shared_ptr>. * Factoring out struct and class definitions into anonymous namespace. Preparation for building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. * Simplifying from_unique_ptr(): typename D = std::default_delete<T> is not needed. Factoring out is_std_default_delete<T>() for consistentcy between ensure_compatible_rtti_uqp_del() and from_unique_ptr(). * Introducing PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. Using it in test_smart_ptr.cpp. With this test_smart_ptr builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT and all but one test run successfully. * Introducing 1. type_caster_for_class_, used in PYBIND11_MAKE_OPAQUE, and 2. default_holder_type, used in stl_bind.h. * Using __VA_ARGS__ in PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. * Replacing condense_for_macro with much simpler approach. * Softening static_assert, to only check specifically that smart_holder is not mixed with type_caster_base, and unique_ptr/shared_ptr holders are not mixed with smart_holder_type_casters. * Adding PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS in test_class.cpp (with this all but one test succeed with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT). * Adding remaining PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. static_assert for "necessary conditions" for both types of default holder, static_assert for "strict conditions" guarded by new PYBIND11_STRICT_ASSERTS_CLASS_HOLDER_VS_TYPE_CASTER_MIX. All tests build & run as before with unique_ptr as the default holder, all tests build for smart_holder as the default holder, even with the strict static_assert. * Introducing check_is_smart_holder_type_caster() function for runtime check, and reinterpreting record.default_holder as "uses_unique_ptr_holder". With this test_smart_ptr succeeds. (All 42 tests build, 35 tests succeed, 5 run but have some failures, 2 segfault.) * Bug fix: Adding have_value() to smart_holder_type_caster_load. With this test_builtin_casters succeeds. (All 42 tests build, 36 tests succeed, 5 run but have some failures, 1 segfault.) * Adding unowned_void_ptr_from_direct_conversion to modified_type_caster_generic_load_impl. This fixes the last remaining segfault (test_numpy_dtypes). New stats for all tests combined: 12 failed, 458 passed. * Adding "Lazy allocation for unallocated values" (for old-style __init__) into load_value_and_holder. Deferring destruction of disowned holder until clear_instance, to remain inspectable for "uninitialized" or "disowned" detection. New stats for all tests combined: 5 failed, 465 passed. * Changing std::shared_ptr pointer/reference to const pointer/reference. New stats for all tests combined: 4 failed, 466 passed. * Adding return_value_policy::move to permissible policies for unique_ptr returns. New stats for all tests combined: 3 failed, 467 passed. * Overlooked flake8 fixes. * Manipulating failing ConstructorStats test to pass, to be able to run all tests with ASAN. This version of the code is ASAN clean with unique_ptr or smart_holder as the default. This change needs to be reverted after adopting the existing move-only-if-refcount-is-1 logic used by type_caster_base. * Adding copy constructor and move constructor tracking to atyp. Preparation for a follow-up change in smart_holder_type_caster, to make this test sensitive to the changing behavior. [skip ci] * Removing `operator T&&() &&` from smart_holder_type_caster, for compatibility with the behavior of type_caster_base. Enables reverting 2 of 3 test manipulations applied under commit 249df7cbdb09817fed0ddf80f01ba5af12466820. The manipulation in test_factory_constructors.py is NOT reverted in this commit. [skip ci] * Fixing unfortunate editing mishap. This reverts the last remaining test manipulation in commit 249df7cbdb09817fed0ddf80f01ba5af12466820 and makes all existing unit tests pass with smart_holder as default holder. * GitHub CI clang-tidy fixes. * Adding messages to terse `static_assert`s, for pre-C++17 compatibility. * Using @pytest.mark.parametrize to run each assert separately (to see all errors, not just the first). * Systematically removing _atyp from function names, to make the test code simpler. * Using re.match to accommodate variable number of intermediate MvCtor. * Also removing `operator T()` from smart_holder_type_caster, to fix gcc compilation errors. The only loss is pass_rref in test_class_sh_basic. * Systematically replacing `detail::enable_if_t<...smart_holder...>` with `typename std::enable_if<...smart_holder...>::type`. Attempt to work around MSVC 2015 issues, to be tested via GitHub CI. The idea for this change originates from this comment: https://github.com/pybind/pybind11/issues/1616#issuecomment-444536813 * Importing re before pytest after observing a PyPy CI flake when importing pytest first. * Copying MSVC 2015 compatibility change from branch pr2672_use_smart_holder_as_default. * Introducing is_smart_holder_type_caster_base_tag, to keep smart_holder code more disconnected. * Working around MSVC 2015 bug. * Expanding comment for MSVC 2015 workaround. * Systematically changing std::enable_if back to detail::enable_if_t, effectively reverting commit 5d4b6890a337ae1bbaec4091f4195606f89a3b06. * Removing unused smart_holder_type_caster_load::loaded_as_rvalue_ref (it was an oversight that it was not removed with commit 23036a45eb4731a06b488ec1fdf83bca677b7f67). * Removing py::classu, because it does not seem useful enough. * Reverting commit 63495313066119dcf7510c2ae8b468b46c12ef8f by un-commenting `const` in `def_buffer(...)`. To make this possible, `operator T const&` and `operator T const*` in `smart_holder_type_caster` need to be marked as `const` member functions. * Adding construct() overloads for constructing smart_holder from alias unique_ptr, shared_ptr returns. * Adding test_class_sh_factory_constructors.cpp to tests/CMakeLists.txt (fixes oversight, this should have been added long before). * Compatibility with old clang versions (clang 3.6, 3.7 C++11). * Cleaning up changes to existing unit tests. * Systematically adding SMART_HOLDER_WIP tag. Removing minor UNTESTED tags (only the throw are not actually exercised, investing time there has a high cost but very little benefit). * Splitting out smart_holder_type_casters again, into new detail/smart_holder_type_casters_inline_include.h. * Splitting out smart_holder_init_inline_include.h. * Adding additional new include files to CMakeLists.txt, tests/extra_python_package/test_files.py. * clang-format cleanup of most smart_holder code. * Adding source code comments in response to review. * Simple micro-benchmark ("ubench") comparing runtime performance for several holders. Tested using github.com/rwgk/pybind11_scons and Google-internal build system. Sorry, no cmake support at the moment. First results: https://docs.google.com/spreadsheets/d/1InapCYws2Gt-stmFf_Bwl33eOMo3aLE_gc9adveY7RU/edit#gid=0 * Breaking out number_bucket.h, adding hook for also collecting performance data for PyCLIF. * Accounting for ubench in MANIFEST.in (simply prune, for now). * Smarter determination of call_repetitions. [skip ci] * Also scaling performance data to PyCLIF. [skip ci] * Adding ubench/python/number_bucket.clif here for general visibility. * Fix after rebase * Merging detail/smart_holder_init_inline_include.h into detail/init.h. * Renaming detail/is_smart_holder_type_caster.h -> detail/smart_holder_sfinae_hooks_only.h. * Renaming is_smart_holder_type_caster -> type_uses_smart_holder_type_caster for clarity. * Renaming type_caster_type_is_smart_holder_type_caster -> wrapped_type_uses_smart_holder_type_caster for clarity. * Renaming is_smart_holder_type_caster_base_tag -> smart_holder_type_caster_base_tag for simplicity. * Adding copyright notices and minor colateral cleanup. * iwyu cleanup (comprehensive only for cast.h and smart_holder*.h files). * Fixing `git rebase master` accident. * Moving large `pragma warning` block from pybind11.h to detail/common.h. * Fixing another `git rebase master` accident.
2021-02-24 05:50:42 +00:00
atyp(const atyp &other) { mtxt = other.mtxt + "_CpCtor"; }
atyp(atyp &&other) noexcept { mtxt = other.mtxt + "_MvCtor"; }
Adding py::smart_holder (for smart-pointer interoperability). (#2672) * Adding test_unique_ptr_member (for desired PyCLIF behavior). See also: https://github.com/pybind/pybind11/issues/2583 Does not build with upstream master or https://github.com/pybind/pybind11/pull/2047, but builds with https://github.com/RobotLocomotion/pybind11 and almost runs: ``` Running tests in directory "/usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests": ================================================================================= test session starts ================================================================================= platform linux -- Python 3.8.5, pytest-5.4.3, py-1.9.0, pluggy-0.13.1 rootdir: /usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests, inifile: pytest.ini collected 2 items test_unique_ptr_member.py .F [100%] ====================================================================================== FAILURES ======================================================================================= _____________________________________________________________________________ test_pointee_and_ptr_owner ______________________________________________________________________________ def test_pointee_and_ptr_owner(): obj = m.pointee() assert obj.get_int() == 213 m.ptr_owner(obj) with pytest.raises(ValueError) as exc_info: > obj.get_int() E Failed: DID NOT RAISE <class 'ValueError'> test_unique_ptr_member.py:17: Failed ============================================================================= 1 failed, 1 passed in 0.06s ============================================================================= ``` * unique_ptr or shared_ptr return * new test_variant_unique_shared with vptr_holder prototype * moving prototype code to pybind11/vptr_holder.h, adding type_caster specialization to make the bindings involving unique_ptr passing compile, but load and cast implementations are missing * disabling GitHub Actions on pull_request (for this PR) * disabling AppVeyor (for this PR) * TRIGGER_SEGSEV macro, annotations for GET_STACK (vptr::get), GET_INT_STACK (pointee) * adding test_promotion_of_disowned_to_shared * Copying tests as-is from xxx_value_ptr_xxx_holder branch. https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder Systematically exercising returning and passing unique_ptr<T>, shared_ptr<T> with unique_ptr, shared_ptr holder. Observations: test_holder_unique_ptr: make_unique_pointee OK pass_unique_pointee BUILD_FAIL (as documented) make_shared_pointee Abort free(): double free detected pass_shared_pointee RuntimeError: Unable to load a custom holder type from a default-holder instance test_holder_shared_ptr: make_unique_pointee Segmentation fault (#1138) pass_unique_pointee BUILD_FAIL (as documented) make_shared_pointee OK pass_shared_pointee OK * Copying tests as-is from xxx_value_ptr_xxx_holder branch. https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder Systematically exercising casting between shared_ptr<base>, shared_ptr<derived>. * Demonstration of Undefined Behavior in handling of shared_ptr holder. Based on https://godbolt.org/z/4fdjaW by jorgbrown@ (thanks Jorg!). * Additional demonstration of Undefined Behavior in handling of shared_ptr holder. * fixing up-down mixup in comment * Demonstration of Undefined Behavior in handling of polymorphic pointers. (This demo does NOT involve smart pointers at all, unlike the otherwise similar test_smart_ptr_private_first_base.) * minor test_private_first_base.cpp simplification (after discovering that this can be wrapped with Boost.Python, using boost::noncopyable) * pybind11 equivalent of Boost.Python test similar to reproducer under #1333 * Snapshot of WIP, TODO: shared_ptr deleter with on/off switch * Adding vptr_deleter. * Adding from/as unique_ptr<T> and unique_ptr<T, D>. * Adding from_shared_ptr. Some polishing. * New tests/core/smart_holder_poc_test.cpp, using Catch2. * Adding in vptr_deleter_guard_flag. * Improved labeling of TEST_CASEs. * Shuffling existing TEST_CASEs into systematic matrix. * Implementing all [S]uccess tests. * Implementing all [E]xception tests. * Testing of exceptions not covered by the from-as matrix. * Adding top-level comment. * Converting from methods to factory functions (no functional change). * Removing obsolete and very incomplete test (replaced by Catch2-based test). * Removing stray file. * Adding type_caster_bare_interface_demo. * Adding shared_ptr<mpty>, shared_ptr<mpty const> casters. * Adding unique_ptr<mpty>, unique_ptr<mpty const> casters. * Pure copy of `class class_` implementation in pybind11.h (master commit 98f1bbb8004f654ba9e26717bdf5912fb899b05a). * classh.h: renaming of class_ to classh + namespace; forking test_classh_wip from test_type_caster_bare_interface_demo. * Hard-coding smart_holder into classh. * Adding mpty::mtxt string member. * Adding isinstance<mpty> in type_caster::load functions. * Adding rvalue_ref, renaming const_value_ref to lvalue_ref & removing const. * Retrieving smart_holder pointer in type_caster<mpty>::load, and using it cast_op operators. * Factoring out smart_holder_type_caster_load. * Retrieving smart_holder pointer in type_caster<std::shared_ptr<mpty[ const]>>::load, and using it cast_op operators. * Improved error messaging: Cannot disown nullptr (as_unique_ptr). * Retrieving smart_holder pointer in type_caster<std::unique_ptr<mpty[ const]>>::load, and using it cast_op operators. * Pure `clang-format --style=file -i` change. * Pure `clang-format --style=file -i` change, with two `clang-format off` directives. * Fixing oversight (discovered by flake8). * flake8 cleanup * Systematically setting mtxt for all rtrn_mpty_* functions (preparation, the values are not actually used yet). * static cast handle for rtrn_cptr works by simply dropping in code from type_caster_base (marked with comments). * static cast handle for rtrn_cref works by simply dropping in code from type_caster_base (marked with comments). rtrn_mref and rtrn_mptr work via const_cast (to add const). * static cast handle for rtrn_valu works by simply dropping in code from type_caster_base (marked with comments). rtrn_rref raises a RuntimeError, to be investigated. * Copying type_caster_generic::cast into type_caster<mpty> as-is (preparation for handling smart pointers). * Pure clang-format change (applied to original type_caster_generic::cast). * Adding comment re potential use_count data race. * static handle cast implementations for rtrn_shmp, rtrn_shcp. * Adding MISSING comments in operator std::unique_ptr<mpty[ const]>. * static handle cast implementations for rtrn_uqmp, rtrn_uqcp. * Bug fix: vptr_deleter_armed_flag_ptr has to live on the heap. See new bullet point in comment section near the top. The variable was also renamed to reflect its function more accurately. * Fixing bugs discovered by ASAN. The code is now ASAN, MSAN, UBSAN clean. * Making test_type_caster_bare_interface_demo.cpp slightly more realistic, ASAN, MSAN, UBSAN clean. * Calling deregister_instance after disowning via unique_ptr. * Removing enable_shared_from_this stub, simplifying existing code, clang-format. Open question, with respect to the original code: https://github.com/pybind/pybind11/blob/76a160070b369f8d82b945c97924227e8b835c94/include/pybind11/pybind11.h#L1510 To me it looks like the exact situation marked as `std::shared_ptr<Good> gp1 = not_so_good.getptr();` here: https://en.cppreference.com/w/cpp/memory/enable_shared_from_this The comment there is: `// undefined behavior (until C++17) and std::bad_weak_ptr thrown (since C++17)` Does the existing code have UB pre C++17? I'll leave handling of enable_shared_from_this for later, as the need arises. * Cosmetical change around helper functions. * Using type_caster_base<mpty>::src_and_type directly, removing copy. Also renaming one cast to cast_const_raw_ptr, for clarity. * Fixing clang-format oversight. * Using factored-out make_constructor (PR #2798), removing duplicate code. * Inserting additional assert to ensure a returned unique_ptr is always a new Python instance. * Adding minor comment (change to internals needed to distinguish uninitialized/disowned in error message). * Factoring out find_existing_python_instance(). * Moving factored-out make_constructor to test_classh_wip.cpp, restoring previous version of cast.h. This is currently the most practical approach. See PR #2798 for background. * Copying classh type_casters from test_classh_wip.cpp UNMODIFIED, as a baseline for generalizing the code. * Using pybind11/detail/classh_type_casters.h from test_classh_wip.cpp. * Adding & using PYBIND11_CLASSH_TYPE_CASTERS define. * Adding test_classh_inheritance, currently failing (passes with class_). * Removing .clang-format before git rebase master (where the file was added). * Bringing back .clang-format, the previous rm was a bad idea. * Folding in modified_type_caster_generic_load_impl, just enough to pass test_class_wip. test_classh_inheritance is still failing, but with a different error: [RuntimeError: Incompatible type (as_raw_ptr_unowned).] * Minimal changes needed to pass test_classh_inheritance. * First pass adjusting try_implicit_casts and try_load_foreign_module_local to capture loaded_v_h, but untested and guarded with pybind11_failure("Untested"). This was done mainly to determine general feasibility. Note the TODO in pybind11.h, where type_caster_generic::local_load is currently hard-coded. test_classh_wip and test_classh_inheritance still pass, as before. * Decoupling generic_type from type_caster_generic. * Changes and tests covering classh_type_casters try_implicit_casts. * Minimal test covering classh_type_casters load_impl Case 2b. * Removing stray isinstance<T>(src): it interferes with the py::module_local feature. Adding missing #includes. * Tests for classh py::module_local() feature. * Pure renaming of function names in test_classh_inheritance, similar to the systematic approach used in test_class_wip. NO functional changes. * Pure renaming of function and variable names, for better generalization when convoluting with inheritance. NO functional changes. * Adopting systematic naming scheme from test_classh_wip. NO functional changes. * Moving const after type name, for functions that cover a systematic scheme. NO functional changes. * Adding smart_holder_type_caster_load::loaded_as_shared_ptr, currently bypassing smart_holder shared_ptr tracking completely, but the tests pass and are sanitizer clean. * Removing rtti_held from smart_holder. See updated comment. * Cleaning up loaded_as_raw_ptr_unowned, loaded_as_shared_ptr. * Factoring out convert_type and folding into loaded_as_unique_ptr. * Folding convert_type into lvalue_ref and rvalue_ref paths. Some smart_holder_type_caster_load cleanup. * Using unique_ptr in local_load to replace static variable. Also adding local_load_safety_guard. * Converting test_unique_ptr_member to using classh: fully working, ASAN, MSAN, UBSAN clean. * Removing debugging comments (GET_STACK, GET_INT_STACK). cast.h is identical to current master again, pybind11.h only has the generic_type::initialize(..., &type_caster_generic::local_load) change. * Purging obsolete pybind11/vptr_holder.h and associated test. * Moving several tests to github.com/rwgk/rwgk_tbx/tree/main/pybind11_tests https://github.com/rwgk/rwgk_tbx/commit/a2c2f88174a30f5de80d7d26e0f77c7b60f5fb85 These tests are from experimenting, and for demonstrating UB in pybind11 multiple inheritance handling ("first_base"), to be fixed later. * Adding py::smart_holder support to py::class_, purging py::classh completely. * Renaming files in include directory, creating pybind11/smart_holder.h. * Renaming all "classh" to "smart_holder" in pybind11/detail/smart_holder_type_casters.h. The user-facing macro is now PYBIND11_SMART_HOLDER_TYPE_CASTERS. * Systematically renaming tests to use "class_sh" in the name. * Renaming test_type_caster_bare_interface_demo to test_type_caster_bare_interface. * Renaming new tests/core subdirectory to tests/pure_cpp. * Adding new tests to CMake config, resetting CI config. * Changing CMake file so that test_class_sh_module_local.py actually runs. * clang-tidy fixes. * 32-bit compatibility. * Reusing type_caster_base make_copy_constructor, make_move_constructor with a trick. * CMake COMPARE NATURAL is not available with older versions. * Adding copyright notices to new header files. * Explicitly define copy/move constructors/assignments. * Adding new header files to tests/extra_python_package/test_files.py. * Adding tests/pure_cpp/CMakeLists.txt. * Making use of the new find_existing_python_instance() function factored out with PR #2822. * Moving define PYBIND11_SMART_HOLDER_TYPE_CASTERS(T) down in the file. NO functional changes. Preparation for follow-up work (to keep that diff smaller). * Reintroducing py::classh, this time as a simple alias for py::class_<U, py::smart_holder>. * Replacing detail::is_smart_holder<H> in cast.h with detail::is_smart_holder_type_caster<T>. Moving get_local_load_function_ptr, init_instance_for_type to smart_holder_type_caster_class_hooks. Expanding static_assert in py::type::handle_of<> to accommodate smart_holder_type_casters. * Fixing oversight. * Adding classu alias for class_<U, std::unique_ptr<U>>. * Giving up on idea to use legacy init_instance only if is_base_of<type_caster_generic, type_caster<T>. There are use cases in the wild that define both a custom type_caster and class_. * Removing test_type_caster_bare_interface, which was moved to the separate PR #2834. * Moving up is_smart_holder_type_caster, to also use in cast_is_temporary_value_reference. * Adding smart_holder_type_casters for unique_ptr with custom deleter. SEVERE CODE DUPLICATION. This commit is to establish a baseline for consolidating the unique_ptr code. * Unification of unique_ptr, unique_ptr_with_deleter code in smart_holder_poc.h. Leads to more fitting error messages. Enables use of unique_ptr<T, D> smart_holder_type_casters also for unique_ptr<T>. * Copying files as-is from branch test_unique_ptr_member (PR #2672). * Adding comment, simplifying naming, cmake addition. * Introducing PYBIND11_USE_SMART_HOLDER_AS_DEFAULT macro (tested only undefined; there are many errors with the macro defined). * Removing test_type_caster_bare_interface, which was moved to the separate PR #2834. * Fixing oversight introduced with commit 95425f13d6c14fcb6ee479b62b602dc8a605ec49. * Setting record.default_holder correctly for PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. With this test_class.cpp builds and even mostly runs, except `test_multiple_instances_with_same_pointer`, which segfaults because it is using a `unique_ptr` holder but `smart_holder` `type_caster`. Also adding `static_assert`s to generate build errors for such situations, but guarding with `#if 0` to first pivot to test_factory_constructors.cpp. * Fixing up cast.h and smart_holder.h after rebase. * Removing detail/smart_holder_type_casters.h in separate commit. * Commenting out const in def_buffer(... const). With this, test_buffers builds and runs with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. Explanation why the const needs to be removed, or fix elsewhere, is still needed, but left for later. * Adding test_class_sh_factory_constructors, reproducing test_factory_constructors failure. Using py::class_ in this commit, to be changed to py::classh for debugging. * Removing include/pybind11/detail/smart_holder_type_casters.h from CMakeLists.txt, test_files.py (since it does not exist in this branch). * Adding // DANGER ZONE reminders. * Converting as many py::class_ to py::classh as possible, not breaking tests. * Adding initimpl::construct() overloads, resulting in test_class_sh_factory_constructors feature parity for py::class_ and py::classh. * Adding enable_if !is_smart_holder_type_caster to existing initimpl::construct(). With this test_factory_constructors.cpp builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. * Disabling shared_ptr&, shared_ptr* tests when building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT for now, pending work on smart_holder_type_caster<shared_ptr>. * Factoring out struct and class definitions into anonymous namespace. Preparation for building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. * Simplifying from_unique_ptr(): typename D = std::default_delete<T> is not needed. Factoring out is_std_default_delete<T>() for consistentcy between ensure_compatible_rtti_uqp_del() and from_unique_ptr(). * Introducing PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. Using it in test_smart_ptr.cpp. With this test_smart_ptr builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT and all but one test run successfully. * Introducing 1. type_caster_for_class_, used in PYBIND11_MAKE_OPAQUE, and 2. default_holder_type, used in stl_bind.h. * Using __VA_ARGS__ in PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. * Replacing condense_for_macro with much simpler approach. * Softening static_assert, to only check specifically that smart_holder is not mixed with type_caster_base, and unique_ptr/shared_ptr holders are not mixed with smart_holder_type_casters. * Adding PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS in test_class.cpp (with this all but one test succeed with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT). * Adding remaining PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. static_assert for "necessary conditions" for both types of default holder, static_assert for "strict conditions" guarded by new PYBIND11_STRICT_ASSERTS_CLASS_HOLDER_VS_TYPE_CASTER_MIX. All tests build & run as before with unique_ptr as the default holder, all tests build for smart_holder as the default holder, even with the strict static_assert. * Introducing check_is_smart_holder_type_caster() function for runtime check, and reinterpreting record.default_holder as "uses_unique_ptr_holder". With this test_smart_ptr succeeds. (All 42 tests build, 35 tests succeed, 5 run but have some failures, 2 segfault.) * Bug fix: Adding have_value() to smart_holder_type_caster_load. With this test_builtin_casters succeeds. (All 42 tests build, 36 tests succeed, 5 run but have some failures, 1 segfault.) * Adding unowned_void_ptr_from_direct_conversion to modified_type_caster_generic_load_impl. This fixes the last remaining segfault (test_numpy_dtypes). New stats for all tests combined: 12 failed, 458 passed. * Adding "Lazy allocation for unallocated values" (for old-style __init__) into load_value_and_holder. Deferring destruction of disowned holder until clear_instance, to remain inspectable for "uninitialized" or "disowned" detection. New stats for all tests combined: 5 failed, 465 passed. * Changing std::shared_ptr pointer/reference to const pointer/reference. New stats for all tests combined: 4 failed, 466 passed. * Adding return_value_policy::move to permissible policies for unique_ptr returns. New stats for all tests combined: 3 failed, 467 passed. * Overlooked flake8 fixes. * Manipulating failing ConstructorStats test to pass, to be able to run all tests with ASAN. This version of the code is ASAN clean with unique_ptr or smart_holder as the default. This change needs to be reverted after adopting the existing move-only-if-refcount-is-1 logic used by type_caster_base. * Adding copy constructor and move constructor tracking to atyp. Preparation for a follow-up change in smart_holder_type_caster, to make this test sensitive to the changing behavior. [skip ci] * Removing `operator T&&() &&` from smart_holder_type_caster, for compatibility with the behavior of type_caster_base. Enables reverting 2 of 3 test manipulations applied under commit 249df7cbdb09817fed0ddf80f01ba5af12466820. The manipulation in test_factory_constructors.py is NOT reverted in this commit. [skip ci] * Fixing unfortunate editing mishap. This reverts the last remaining test manipulation in commit 249df7cbdb09817fed0ddf80f01ba5af12466820 and makes all existing unit tests pass with smart_holder as default holder. * GitHub CI clang-tidy fixes. * Adding messages to terse `static_assert`s, for pre-C++17 compatibility. * Using @pytest.mark.parametrize to run each assert separately (to see all errors, not just the first). * Systematically removing _atyp from function names, to make the test code simpler. * Using re.match to accommodate variable number of intermediate MvCtor. * Also removing `operator T()` from smart_holder_type_caster, to fix gcc compilation errors. The only loss is pass_rref in test_class_sh_basic. * Systematically replacing `detail::enable_if_t<...smart_holder...>` with `typename std::enable_if<...smart_holder...>::type`. Attempt to work around MSVC 2015 issues, to be tested via GitHub CI. The idea for this change originates from this comment: https://github.com/pybind/pybind11/issues/1616#issuecomment-444536813 * Importing re before pytest after observing a PyPy CI flake when importing pytest first. * Copying MSVC 2015 compatibility change from branch pr2672_use_smart_holder_as_default. * Introducing is_smart_holder_type_caster_base_tag, to keep smart_holder code more disconnected. * Working around MSVC 2015 bug. * Expanding comment for MSVC 2015 workaround. * Systematically changing std::enable_if back to detail::enable_if_t, effectively reverting commit 5d4b6890a337ae1bbaec4091f4195606f89a3b06. * Removing unused smart_holder_type_caster_load::loaded_as_rvalue_ref (it was an oversight that it was not removed with commit 23036a45eb4731a06b488ec1fdf83bca677b7f67). * Removing py::classu, because it does not seem useful enough. * Reverting commit 63495313066119dcf7510c2ae8b468b46c12ef8f by un-commenting `const` in `def_buffer(...)`. To make this possible, `operator T const&` and `operator T const*` in `smart_holder_type_caster` need to be marked as `const` member functions. * Adding construct() overloads for constructing smart_holder from alias unique_ptr, shared_ptr returns. * Adding test_class_sh_factory_constructors.cpp to tests/CMakeLists.txt (fixes oversight, this should have been added long before). * Compatibility with old clang versions (clang 3.6, 3.7 C++11). * Cleaning up changes to existing unit tests. * Systematically adding SMART_HOLDER_WIP tag. Removing minor UNTESTED tags (only the throw are not actually exercised, investing time there has a high cost but very little benefit). * Splitting out smart_holder_type_casters again, into new detail/smart_holder_type_casters_inline_include.h. * Splitting out smart_holder_init_inline_include.h. * Adding additional new include files to CMakeLists.txt, tests/extra_python_package/test_files.py. * clang-format cleanup of most smart_holder code. * Adding source code comments in response to review. * Simple micro-benchmark ("ubench") comparing runtime performance for several holders. Tested using github.com/rwgk/pybind11_scons and Google-internal build system. Sorry, no cmake support at the moment. First results: https://docs.google.com/spreadsheets/d/1InapCYws2Gt-stmFf_Bwl33eOMo3aLE_gc9adveY7RU/edit#gid=0 * Breaking out number_bucket.h, adding hook for also collecting performance data for PyCLIF. * Accounting for ubench in MANIFEST.in (simply prune, for now). * Smarter determination of call_repetitions. [skip ci] * Also scaling performance data to PyCLIF. [skip ci] * Adding ubench/python/number_bucket.clif here for general visibility. * Fix after rebase * Merging detail/smart_holder_init_inline_include.h into detail/init.h. * Renaming detail/is_smart_holder_type_caster.h -> detail/smart_holder_sfinae_hooks_only.h. * Renaming is_smart_holder_type_caster -> type_uses_smart_holder_type_caster for clarity. * Renaming type_caster_type_is_smart_holder_type_caster -> wrapped_type_uses_smart_holder_type_caster for clarity. * Renaming is_smart_holder_type_caster_base_tag -> smart_holder_type_caster_base_tag for simplicity. * Adding copyright notices and minor colateral cleanup. * iwyu cleanup (comprehensive only for cast.h and smart_holder*.h files). * Fixing `git rebase master` accident. * Moving large `pragma warning` block from pybind11.h to detail/common.h. * Fixing another `git rebase master` accident.
2021-02-24 05:50:42 +00:00
};
struct uconsumer { // unique_ptr consumer
std::unique_ptr<atyp> held;
bool valid() const { return static_cast<bool>(held); }
void pass_valu(std::unique_ptr<atyp> obj) { held = std::move(obj); }
void pass_rref(std::unique_ptr<atyp> &&obj) { held = std::move(obj); }
std::unique_ptr<atyp> rtrn_valu() { return std::move(held); }
std::unique_ptr<atyp> &rtrn_lref() { return held; }
const std::unique_ptr<atyp> &rtrn_cref() const { return held; }
};
/// Custom deleter that is default constructible.
struct custom_deleter {
std::string trace_txt;
custom_deleter() = default;
explicit custom_deleter(const std::string &trace_txt_) : trace_txt(trace_txt_) {}
custom_deleter(const custom_deleter &other) { trace_txt = other.trace_txt + "_CpCtor"; }
custom_deleter &operator=(const custom_deleter &rhs) {
trace_txt = rhs.trace_txt + "_CpLhs";
return *this;
}
custom_deleter(custom_deleter &&other) noexcept {
trace_txt = other.trace_txt + "_MvCtorTo";
other.trace_txt += "_MvCtorFrom";
}
custom_deleter &operator=(custom_deleter &&rhs) noexcept {
trace_txt = rhs.trace_txt + "_MvLhs";
rhs.trace_txt += "_MvRhs";
return *this;
}
void operator()(atyp *p) const { std::default_delete<atyp>()(p); }
void operator()(const atyp *p) const { std::default_delete<const atyp>()(p); }
};
static_assert(std::is_default_constructible<custom_deleter>::value, "");
/// Custom deleter that is not default constructible.
struct custom_deleter_nd : custom_deleter {
custom_deleter_nd() = delete;
explicit custom_deleter_nd(const std::string &trace_txt_) : custom_deleter(trace_txt_) {}
};
static_assert(!std::is_default_constructible<custom_deleter_nd>::value, "");
Adding py::smart_holder (for smart-pointer interoperability). (#2672) * Adding test_unique_ptr_member (for desired PyCLIF behavior). See also: https://github.com/pybind/pybind11/issues/2583 Does not build with upstream master or https://github.com/pybind/pybind11/pull/2047, but builds with https://github.com/RobotLocomotion/pybind11 and almost runs: ``` Running tests in directory "/usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests": ================================================================================= test session starts ================================================================================= platform linux -- Python 3.8.5, pytest-5.4.3, py-1.9.0, pluggy-0.13.1 rootdir: /usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests, inifile: pytest.ini collected 2 items test_unique_ptr_member.py .F [100%] ====================================================================================== FAILURES ======================================================================================= _____________________________________________________________________________ test_pointee_and_ptr_owner ______________________________________________________________________________ def test_pointee_and_ptr_owner(): obj = m.pointee() assert obj.get_int() == 213 m.ptr_owner(obj) with pytest.raises(ValueError) as exc_info: > obj.get_int() E Failed: DID NOT RAISE <class 'ValueError'> test_unique_ptr_member.py:17: Failed ============================================================================= 1 failed, 1 passed in 0.06s ============================================================================= ``` * unique_ptr or shared_ptr return * new test_variant_unique_shared with vptr_holder prototype * moving prototype code to pybind11/vptr_holder.h, adding type_caster specialization to make the bindings involving unique_ptr passing compile, but load and cast implementations are missing * disabling GitHub Actions on pull_request (for this PR) * disabling AppVeyor (for this PR) * TRIGGER_SEGSEV macro, annotations for GET_STACK (vptr::get), GET_INT_STACK (pointee) * adding test_promotion_of_disowned_to_shared * Copying tests as-is from xxx_value_ptr_xxx_holder branch. https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder Systematically exercising returning and passing unique_ptr<T>, shared_ptr<T> with unique_ptr, shared_ptr holder. Observations: test_holder_unique_ptr: make_unique_pointee OK pass_unique_pointee BUILD_FAIL (as documented) make_shared_pointee Abort free(): double free detected pass_shared_pointee RuntimeError: Unable to load a custom holder type from a default-holder instance test_holder_shared_ptr: make_unique_pointee Segmentation fault (#1138) pass_unique_pointee BUILD_FAIL (as documented) make_shared_pointee OK pass_shared_pointee OK * Copying tests as-is from xxx_value_ptr_xxx_holder branch. https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder Systematically exercising casting between shared_ptr<base>, shared_ptr<derived>. * Demonstration of Undefined Behavior in handling of shared_ptr holder. Based on https://godbolt.org/z/4fdjaW by jorgbrown@ (thanks Jorg!). * Additional demonstration of Undefined Behavior in handling of shared_ptr holder. * fixing up-down mixup in comment * Demonstration of Undefined Behavior in handling of polymorphic pointers. (This demo does NOT involve smart pointers at all, unlike the otherwise similar test_smart_ptr_private_first_base.) * minor test_private_first_base.cpp simplification (after discovering that this can be wrapped with Boost.Python, using boost::noncopyable) * pybind11 equivalent of Boost.Python test similar to reproducer under #1333 * Snapshot of WIP, TODO: shared_ptr deleter with on/off switch * Adding vptr_deleter. * Adding from/as unique_ptr<T> and unique_ptr<T, D>. * Adding from_shared_ptr. Some polishing. * New tests/core/smart_holder_poc_test.cpp, using Catch2. * Adding in vptr_deleter_guard_flag. * Improved labeling of TEST_CASEs. * Shuffling existing TEST_CASEs into systematic matrix. * Implementing all [S]uccess tests. * Implementing all [E]xception tests. * Testing of exceptions not covered by the from-as matrix. * Adding top-level comment. * Converting from methods to factory functions (no functional change). * Removing obsolete and very incomplete test (replaced by Catch2-based test). * Removing stray file. * Adding type_caster_bare_interface_demo. * Adding shared_ptr<mpty>, shared_ptr<mpty const> casters. * Adding unique_ptr<mpty>, unique_ptr<mpty const> casters. * Pure copy of `class class_` implementation in pybind11.h (master commit 98f1bbb8004f654ba9e26717bdf5912fb899b05a). * classh.h: renaming of class_ to classh + namespace; forking test_classh_wip from test_type_caster_bare_interface_demo. * Hard-coding smart_holder into classh. * Adding mpty::mtxt string member. * Adding isinstance<mpty> in type_caster::load functions. * Adding rvalue_ref, renaming const_value_ref to lvalue_ref & removing const. * Retrieving smart_holder pointer in type_caster<mpty>::load, and using it cast_op operators. * Factoring out smart_holder_type_caster_load. * Retrieving smart_holder pointer in type_caster<std::shared_ptr<mpty[ const]>>::load, and using it cast_op operators. * Improved error messaging: Cannot disown nullptr (as_unique_ptr). * Retrieving smart_holder pointer in type_caster<std::unique_ptr<mpty[ const]>>::load, and using it cast_op operators. * Pure `clang-format --style=file -i` change. * Pure `clang-format --style=file -i` change, with two `clang-format off` directives. * Fixing oversight (discovered by flake8). * flake8 cleanup * Systematically setting mtxt for all rtrn_mpty_* functions (preparation, the values are not actually used yet). * static cast handle for rtrn_cptr works by simply dropping in code from type_caster_base (marked with comments). * static cast handle for rtrn_cref works by simply dropping in code from type_caster_base (marked with comments). rtrn_mref and rtrn_mptr work via const_cast (to add const). * static cast handle for rtrn_valu works by simply dropping in code from type_caster_base (marked with comments). rtrn_rref raises a RuntimeError, to be investigated. * Copying type_caster_generic::cast into type_caster<mpty> as-is (preparation for handling smart pointers). * Pure clang-format change (applied to original type_caster_generic::cast). * Adding comment re potential use_count data race. * static handle cast implementations for rtrn_shmp, rtrn_shcp. * Adding MISSING comments in operator std::unique_ptr<mpty[ const]>. * static handle cast implementations for rtrn_uqmp, rtrn_uqcp. * Bug fix: vptr_deleter_armed_flag_ptr has to live on the heap. See new bullet point in comment section near the top. The variable was also renamed to reflect its function more accurately. * Fixing bugs discovered by ASAN. The code is now ASAN, MSAN, UBSAN clean. * Making test_type_caster_bare_interface_demo.cpp slightly more realistic, ASAN, MSAN, UBSAN clean. * Calling deregister_instance after disowning via unique_ptr. * Removing enable_shared_from_this stub, simplifying existing code, clang-format. Open question, with respect to the original code: https://github.com/pybind/pybind11/blob/76a160070b369f8d82b945c97924227e8b835c94/include/pybind11/pybind11.h#L1510 To me it looks like the exact situation marked as `std::shared_ptr<Good> gp1 = not_so_good.getptr();` here: https://en.cppreference.com/w/cpp/memory/enable_shared_from_this The comment there is: `// undefined behavior (until C++17) and std::bad_weak_ptr thrown (since C++17)` Does the existing code have UB pre C++17? I'll leave handling of enable_shared_from_this for later, as the need arises. * Cosmetical change around helper functions. * Using type_caster_base<mpty>::src_and_type directly, removing copy. Also renaming one cast to cast_const_raw_ptr, for clarity. * Fixing clang-format oversight. * Using factored-out make_constructor (PR #2798), removing duplicate code. * Inserting additional assert to ensure a returned unique_ptr is always a new Python instance. * Adding minor comment (change to internals needed to distinguish uninitialized/disowned in error message). * Factoring out find_existing_python_instance(). * Moving factored-out make_constructor to test_classh_wip.cpp, restoring previous version of cast.h. This is currently the most practical approach. See PR #2798 for background. * Copying classh type_casters from test_classh_wip.cpp UNMODIFIED, as a baseline for generalizing the code. * Using pybind11/detail/classh_type_casters.h from test_classh_wip.cpp. * Adding & using PYBIND11_CLASSH_TYPE_CASTERS define. * Adding test_classh_inheritance, currently failing (passes with class_). * Removing .clang-format before git rebase master (where the file was added). * Bringing back .clang-format, the previous rm was a bad idea. * Folding in modified_type_caster_generic_load_impl, just enough to pass test_class_wip. test_classh_inheritance is still failing, but with a different error: [RuntimeError: Incompatible type (as_raw_ptr_unowned).] * Minimal changes needed to pass test_classh_inheritance. * First pass adjusting try_implicit_casts and try_load_foreign_module_local to capture loaded_v_h, but untested and guarded with pybind11_failure("Untested"). This was done mainly to determine general feasibility. Note the TODO in pybind11.h, where type_caster_generic::local_load is currently hard-coded. test_classh_wip and test_classh_inheritance still pass, as before. * Decoupling generic_type from type_caster_generic. * Changes and tests covering classh_type_casters try_implicit_casts. * Minimal test covering classh_type_casters load_impl Case 2b. * Removing stray isinstance<T>(src): it interferes with the py::module_local feature. Adding missing #includes. * Tests for classh py::module_local() feature. * Pure renaming of function names in test_classh_inheritance, similar to the systematic approach used in test_class_wip. NO functional changes. * Pure renaming of function and variable names, for better generalization when convoluting with inheritance. NO functional changes. * Adopting systematic naming scheme from test_classh_wip. NO functional changes. * Moving const after type name, for functions that cover a systematic scheme. NO functional changes. * Adding smart_holder_type_caster_load::loaded_as_shared_ptr, currently bypassing smart_holder shared_ptr tracking completely, but the tests pass and are sanitizer clean. * Removing rtti_held from smart_holder. See updated comment. * Cleaning up loaded_as_raw_ptr_unowned, loaded_as_shared_ptr. * Factoring out convert_type and folding into loaded_as_unique_ptr. * Folding convert_type into lvalue_ref and rvalue_ref paths. Some smart_holder_type_caster_load cleanup. * Using unique_ptr in local_load to replace static variable. Also adding local_load_safety_guard. * Converting test_unique_ptr_member to using classh: fully working, ASAN, MSAN, UBSAN clean. * Removing debugging comments (GET_STACK, GET_INT_STACK). cast.h is identical to current master again, pybind11.h only has the generic_type::initialize(..., &type_caster_generic::local_load) change. * Purging obsolete pybind11/vptr_holder.h and associated test. * Moving several tests to github.com/rwgk/rwgk_tbx/tree/main/pybind11_tests https://github.com/rwgk/rwgk_tbx/commit/a2c2f88174a30f5de80d7d26e0f77c7b60f5fb85 These tests are from experimenting, and for demonstrating UB in pybind11 multiple inheritance handling ("first_base"), to be fixed later. * Adding py::smart_holder support to py::class_, purging py::classh completely. * Renaming files in include directory, creating pybind11/smart_holder.h. * Renaming all "classh" to "smart_holder" in pybind11/detail/smart_holder_type_casters.h. The user-facing macro is now PYBIND11_SMART_HOLDER_TYPE_CASTERS. * Systematically renaming tests to use "class_sh" in the name. * Renaming test_type_caster_bare_interface_demo to test_type_caster_bare_interface. * Renaming new tests/core subdirectory to tests/pure_cpp. * Adding new tests to CMake config, resetting CI config. * Changing CMake file so that test_class_sh_module_local.py actually runs. * clang-tidy fixes. * 32-bit compatibility. * Reusing type_caster_base make_copy_constructor, make_move_constructor with a trick. * CMake COMPARE NATURAL is not available with older versions. * Adding copyright notices to new header files. * Explicitly define copy/move constructors/assignments. * Adding new header files to tests/extra_python_package/test_files.py. * Adding tests/pure_cpp/CMakeLists.txt. * Making use of the new find_existing_python_instance() function factored out with PR #2822. * Moving define PYBIND11_SMART_HOLDER_TYPE_CASTERS(T) down in the file. NO functional changes. Preparation for follow-up work (to keep that diff smaller). * Reintroducing py::classh, this time as a simple alias for py::class_<U, py::smart_holder>. * Replacing detail::is_smart_holder<H> in cast.h with detail::is_smart_holder_type_caster<T>. Moving get_local_load_function_ptr, init_instance_for_type to smart_holder_type_caster_class_hooks. Expanding static_assert in py::type::handle_of<> to accommodate smart_holder_type_casters. * Fixing oversight. * Adding classu alias for class_<U, std::unique_ptr<U>>. * Giving up on idea to use legacy init_instance only if is_base_of<type_caster_generic, type_caster<T>. There are use cases in the wild that define both a custom type_caster and class_. * Removing test_type_caster_bare_interface, which was moved to the separate PR #2834. * Moving up is_smart_holder_type_caster, to also use in cast_is_temporary_value_reference. * Adding smart_holder_type_casters for unique_ptr with custom deleter. SEVERE CODE DUPLICATION. This commit is to establish a baseline for consolidating the unique_ptr code. * Unification of unique_ptr, unique_ptr_with_deleter code in smart_holder_poc.h. Leads to more fitting error messages. Enables use of unique_ptr<T, D> smart_holder_type_casters also for unique_ptr<T>. * Copying files as-is from branch test_unique_ptr_member (PR #2672). * Adding comment, simplifying naming, cmake addition. * Introducing PYBIND11_USE_SMART_HOLDER_AS_DEFAULT macro (tested only undefined; there are many errors with the macro defined). * Removing test_type_caster_bare_interface, which was moved to the separate PR #2834. * Fixing oversight introduced with commit 95425f13d6c14fcb6ee479b62b602dc8a605ec49. * Setting record.default_holder correctly for PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. With this test_class.cpp builds and even mostly runs, except `test_multiple_instances_with_same_pointer`, which segfaults because it is using a `unique_ptr` holder but `smart_holder` `type_caster`. Also adding `static_assert`s to generate build errors for such situations, but guarding with `#if 0` to first pivot to test_factory_constructors.cpp. * Fixing up cast.h and smart_holder.h after rebase. * Removing detail/smart_holder_type_casters.h in separate commit. * Commenting out const in def_buffer(... const). With this, test_buffers builds and runs with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. Explanation why the const needs to be removed, or fix elsewhere, is still needed, but left for later. * Adding test_class_sh_factory_constructors, reproducing test_factory_constructors failure. Using py::class_ in this commit, to be changed to py::classh for debugging. * Removing include/pybind11/detail/smart_holder_type_casters.h from CMakeLists.txt, test_files.py (since it does not exist in this branch). * Adding // DANGER ZONE reminders. * Converting as many py::class_ to py::classh as possible, not breaking tests. * Adding initimpl::construct() overloads, resulting in test_class_sh_factory_constructors feature parity for py::class_ and py::classh. * Adding enable_if !is_smart_holder_type_caster to existing initimpl::construct(). With this test_factory_constructors.cpp builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. * Disabling shared_ptr&, shared_ptr* tests when building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT for now, pending work on smart_holder_type_caster<shared_ptr>. * Factoring out struct and class definitions into anonymous namespace. Preparation for building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. * Simplifying from_unique_ptr(): typename D = std::default_delete<T> is not needed. Factoring out is_std_default_delete<T>() for consistentcy between ensure_compatible_rtti_uqp_del() and from_unique_ptr(). * Introducing PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. Using it in test_smart_ptr.cpp. With this test_smart_ptr builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT and all but one test run successfully. * Introducing 1. type_caster_for_class_, used in PYBIND11_MAKE_OPAQUE, and 2. default_holder_type, used in stl_bind.h. * Using __VA_ARGS__ in PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. * Replacing condense_for_macro with much simpler approach. * Softening static_assert, to only check specifically that smart_holder is not mixed with type_caster_base, and unique_ptr/shared_ptr holders are not mixed with smart_holder_type_casters. * Adding PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS in test_class.cpp (with this all but one test succeed with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT). * Adding remaining PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. static_assert for "necessary conditions" for both types of default holder, static_assert for "strict conditions" guarded by new PYBIND11_STRICT_ASSERTS_CLASS_HOLDER_VS_TYPE_CASTER_MIX. All tests build & run as before with unique_ptr as the default holder, all tests build for smart_holder as the default holder, even with the strict static_assert. * Introducing check_is_smart_holder_type_caster() function for runtime check, and reinterpreting record.default_holder as "uses_unique_ptr_holder". With this test_smart_ptr succeeds. (All 42 tests build, 35 tests succeed, 5 run but have some failures, 2 segfault.) * Bug fix: Adding have_value() to smart_holder_type_caster_load. With this test_builtin_casters succeeds. (All 42 tests build, 36 tests succeed, 5 run but have some failures, 1 segfault.) * Adding unowned_void_ptr_from_direct_conversion to modified_type_caster_generic_load_impl. This fixes the last remaining segfault (test_numpy_dtypes). New stats for all tests combined: 12 failed, 458 passed. * Adding "Lazy allocation for unallocated values" (for old-style __init__) into load_value_and_holder. Deferring destruction of disowned holder until clear_instance, to remain inspectable for "uninitialized" or "disowned" detection. New stats for all tests combined: 5 failed, 465 passed. * Changing std::shared_ptr pointer/reference to const pointer/reference. New stats for all tests combined: 4 failed, 466 passed. * Adding return_value_policy::move to permissible policies for unique_ptr returns. New stats for all tests combined: 3 failed, 467 passed. * Overlooked flake8 fixes. * Manipulating failing ConstructorStats test to pass, to be able to run all tests with ASAN. This version of the code is ASAN clean with unique_ptr or smart_holder as the default. This change needs to be reverted after adopting the existing move-only-if-refcount-is-1 logic used by type_caster_base. * Adding copy constructor and move constructor tracking to atyp. Preparation for a follow-up change in smart_holder_type_caster, to make this test sensitive to the changing behavior. [skip ci] * Removing `operator T&&() &&` from smart_holder_type_caster, for compatibility with the behavior of type_caster_base. Enables reverting 2 of 3 test manipulations applied under commit 249df7cbdb09817fed0ddf80f01ba5af12466820. The manipulation in test_factory_constructors.py is NOT reverted in this commit. [skip ci] * Fixing unfortunate editing mishap. This reverts the last remaining test manipulation in commit 249df7cbdb09817fed0ddf80f01ba5af12466820 and makes all existing unit tests pass with smart_holder as default holder. * GitHub CI clang-tidy fixes. * Adding messages to terse `static_assert`s, for pre-C++17 compatibility. * Using @pytest.mark.parametrize to run each assert separately (to see all errors, not just the first). * Systematically removing _atyp from function names, to make the test code simpler. * Using re.match to accommodate variable number of intermediate MvCtor. * Also removing `operator T()` from smart_holder_type_caster, to fix gcc compilation errors. The only loss is pass_rref in test_class_sh_basic. * Systematically replacing `detail::enable_if_t<...smart_holder...>` with `typename std::enable_if<...smart_holder...>::type`. Attempt to work around MSVC 2015 issues, to be tested via GitHub CI. The idea for this change originates from this comment: https://github.com/pybind/pybind11/issues/1616#issuecomment-444536813 * Importing re before pytest after observing a PyPy CI flake when importing pytest first. * Copying MSVC 2015 compatibility change from branch pr2672_use_smart_holder_as_default. * Introducing is_smart_holder_type_caster_base_tag, to keep smart_holder code more disconnected. * Working around MSVC 2015 bug. * Expanding comment for MSVC 2015 workaround. * Systematically changing std::enable_if back to detail::enable_if_t, effectively reverting commit 5d4b6890a337ae1bbaec4091f4195606f89a3b06. * Removing unused smart_holder_type_caster_load::loaded_as_rvalue_ref (it was an oversight that it was not removed with commit 23036a45eb4731a06b488ec1fdf83bca677b7f67). * Removing py::classu, because it does not seem useful enough. * Reverting commit 63495313066119dcf7510c2ae8b468b46c12ef8f by un-commenting `const` in `def_buffer(...)`. To make this possible, `operator T const&` and `operator T const*` in `smart_holder_type_caster` need to be marked as `const` member functions. * Adding construct() overloads for constructing smart_holder from alias unique_ptr, shared_ptr returns. * Adding test_class_sh_factory_constructors.cpp to tests/CMakeLists.txt (fixes oversight, this should have been added long before). * Compatibility with old clang versions (clang 3.6, 3.7 C++11). * Cleaning up changes to existing unit tests. * Systematically adding SMART_HOLDER_WIP tag. Removing minor UNTESTED tags (only the throw are not actually exercised, investing time there has a high cost but very little benefit). * Splitting out smart_holder_type_casters again, into new detail/smart_holder_type_casters_inline_include.h. * Splitting out smart_holder_init_inline_include.h. * Adding additional new include files to CMakeLists.txt, tests/extra_python_package/test_files.py. * clang-format cleanup of most smart_holder code. * Adding source code comments in response to review. * Simple micro-benchmark ("ubench") comparing runtime performance for several holders. Tested using github.com/rwgk/pybind11_scons and Google-internal build system. Sorry, no cmake support at the moment. First results: https://docs.google.com/spreadsheets/d/1InapCYws2Gt-stmFf_Bwl33eOMo3aLE_gc9adveY7RU/edit#gid=0 * Breaking out number_bucket.h, adding hook for also collecting performance data for PyCLIF. * Accounting for ubench in MANIFEST.in (simply prune, for now). * Smarter determination of call_repetitions. [skip ci] * Also scaling performance data to PyCLIF. [skip ci] * Adding ubench/python/number_bucket.clif here for general visibility. * Fix after rebase * Merging detail/smart_holder_init_inline_include.h into detail/init.h. * Renaming detail/is_smart_holder_type_caster.h -> detail/smart_holder_sfinae_hooks_only.h. * Renaming is_smart_holder_type_caster -> type_uses_smart_holder_type_caster for clarity. * Renaming type_caster_type_is_smart_holder_type_caster -> wrapped_type_uses_smart_holder_type_caster for clarity. * Renaming is_smart_holder_type_caster_base_tag -> smart_holder_type_caster_base_tag for simplicity. * Adding copyright notices and minor colateral cleanup. * iwyu cleanup (comprehensive only for cast.h and smart_holder*.h files). * Fixing `git rebase master` accident. * Moving large `pragma warning` block from pybind11.h to detail/common.h. * Fixing another `git rebase master` accident.
2021-02-24 05:50:42 +00:00
// clang-format off
atyp rtrn_valu() { atyp obj{"rtrn_valu"}; return obj; }
atyp&& rtrn_rref() { static atyp obj; obj.mtxt = "rtrn_rref"; return std::move(obj); }
atyp const& rtrn_cref() { static atyp obj; obj.mtxt = "rtrn_cref"; return obj; }
atyp& rtrn_mref() { static atyp obj; obj.mtxt = "rtrn_mref"; return obj; }
atyp const* rtrn_cptr() { return new atyp{"rtrn_cptr"}; }
atyp* rtrn_mptr() { return new atyp{"rtrn_mptr"}; }
std::string pass_valu(atyp obj) { return "pass_valu:" + obj.mtxt; } // NOLINT
Adding py::smart_holder (for smart-pointer interoperability). (#2672) * Adding test_unique_ptr_member (for desired PyCLIF behavior). See also: https://github.com/pybind/pybind11/issues/2583 Does not build with upstream master or https://github.com/pybind/pybind11/pull/2047, but builds with https://github.com/RobotLocomotion/pybind11 and almost runs: ``` Running tests in directory "/usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests": ================================================================================= test session starts ================================================================================= platform linux -- Python 3.8.5, pytest-5.4.3, py-1.9.0, pluggy-0.13.1 rootdir: /usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests, inifile: pytest.ini collected 2 items test_unique_ptr_member.py .F [100%] ====================================================================================== FAILURES ======================================================================================= _____________________________________________________________________________ test_pointee_and_ptr_owner ______________________________________________________________________________ def test_pointee_and_ptr_owner(): obj = m.pointee() assert obj.get_int() == 213 m.ptr_owner(obj) with pytest.raises(ValueError) as exc_info: > obj.get_int() E Failed: DID NOT RAISE <class 'ValueError'> test_unique_ptr_member.py:17: Failed ============================================================================= 1 failed, 1 passed in 0.06s ============================================================================= ``` * unique_ptr or shared_ptr return * new test_variant_unique_shared with vptr_holder prototype * moving prototype code to pybind11/vptr_holder.h, adding type_caster specialization to make the bindings involving unique_ptr passing compile, but load and cast implementations are missing * disabling GitHub Actions on pull_request (for this PR) * disabling AppVeyor (for this PR) * TRIGGER_SEGSEV macro, annotations for GET_STACK (vptr::get), GET_INT_STACK (pointee) * adding test_promotion_of_disowned_to_shared * Copying tests as-is from xxx_value_ptr_xxx_holder branch. https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder Systematically exercising returning and passing unique_ptr<T>, shared_ptr<T> with unique_ptr, shared_ptr holder. Observations: test_holder_unique_ptr: make_unique_pointee OK pass_unique_pointee BUILD_FAIL (as documented) make_shared_pointee Abort free(): double free detected pass_shared_pointee RuntimeError: Unable to load a custom holder type from a default-holder instance test_holder_shared_ptr: make_unique_pointee Segmentation fault (#1138) pass_unique_pointee BUILD_FAIL (as documented) make_shared_pointee OK pass_shared_pointee OK * Copying tests as-is from xxx_value_ptr_xxx_holder branch. https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder Systematically exercising casting between shared_ptr<base>, shared_ptr<derived>. * Demonstration of Undefined Behavior in handling of shared_ptr holder. Based on https://godbolt.org/z/4fdjaW by jorgbrown@ (thanks Jorg!). * Additional demonstration of Undefined Behavior in handling of shared_ptr holder. * fixing up-down mixup in comment * Demonstration of Undefined Behavior in handling of polymorphic pointers. (This demo does NOT involve smart pointers at all, unlike the otherwise similar test_smart_ptr_private_first_base.) * minor test_private_first_base.cpp simplification (after discovering that this can be wrapped with Boost.Python, using boost::noncopyable) * pybind11 equivalent of Boost.Python test similar to reproducer under #1333 * Snapshot of WIP, TODO: shared_ptr deleter with on/off switch * Adding vptr_deleter. * Adding from/as unique_ptr<T> and unique_ptr<T, D>. * Adding from_shared_ptr. Some polishing. * New tests/core/smart_holder_poc_test.cpp, using Catch2. * Adding in vptr_deleter_guard_flag. * Improved labeling of TEST_CASEs. * Shuffling existing TEST_CASEs into systematic matrix. * Implementing all [S]uccess tests. * Implementing all [E]xception tests. * Testing of exceptions not covered by the from-as matrix. * Adding top-level comment. * Converting from methods to factory functions (no functional change). * Removing obsolete and very incomplete test (replaced by Catch2-based test). * Removing stray file. * Adding type_caster_bare_interface_demo. * Adding shared_ptr<mpty>, shared_ptr<mpty const> casters. * Adding unique_ptr<mpty>, unique_ptr<mpty const> casters. * Pure copy of `class class_` implementation in pybind11.h (master commit 98f1bbb8004f654ba9e26717bdf5912fb899b05a). * classh.h: renaming of class_ to classh + namespace; forking test_classh_wip from test_type_caster_bare_interface_demo. * Hard-coding smart_holder into classh. * Adding mpty::mtxt string member. * Adding isinstance<mpty> in type_caster::load functions. * Adding rvalue_ref, renaming const_value_ref to lvalue_ref & removing const. * Retrieving smart_holder pointer in type_caster<mpty>::load, and using it cast_op operators. * Factoring out smart_holder_type_caster_load. * Retrieving smart_holder pointer in type_caster<std::shared_ptr<mpty[ const]>>::load, and using it cast_op operators. * Improved error messaging: Cannot disown nullptr (as_unique_ptr). * Retrieving smart_holder pointer in type_caster<std::unique_ptr<mpty[ const]>>::load, and using it cast_op operators. * Pure `clang-format --style=file -i` change. * Pure `clang-format --style=file -i` change, with two `clang-format off` directives. * Fixing oversight (discovered by flake8). * flake8 cleanup * Systematically setting mtxt for all rtrn_mpty_* functions (preparation, the values are not actually used yet). * static cast handle for rtrn_cptr works by simply dropping in code from type_caster_base (marked with comments). * static cast handle for rtrn_cref works by simply dropping in code from type_caster_base (marked with comments). rtrn_mref and rtrn_mptr work via const_cast (to add const). * static cast handle for rtrn_valu works by simply dropping in code from type_caster_base (marked with comments). rtrn_rref raises a RuntimeError, to be investigated. * Copying type_caster_generic::cast into type_caster<mpty> as-is (preparation for handling smart pointers). * Pure clang-format change (applied to original type_caster_generic::cast). * Adding comment re potential use_count data race. * static handle cast implementations for rtrn_shmp, rtrn_shcp. * Adding MISSING comments in operator std::unique_ptr<mpty[ const]>. * static handle cast implementations for rtrn_uqmp, rtrn_uqcp. * Bug fix: vptr_deleter_armed_flag_ptr has to live on the heap. See new bullet point in comment section near the top. The variable was also renamed to reflect its function more accurately. * Fixing bugs discovered by ASAN. The code is now ASAN, MSAN, UBSAN clean. * Making test_type_caster_bare_interface_demo.cpp slightly more realistic, ASAN, MSAN, UBSAN clean. * Calling deregister_instance after disowning via unique_ptr. * Removing enable_shared_from_this stub, simplifying existing code, clang-format. Open question, with respect to the original code: https://github.com/pybind/pybind11/blob/76a160070b369f8d82b945c97924227e8b835c94/include/pybind11/pybind11.h#L1510 To me it looks like the exact situation marked as `std::shared_ptr<Good> gp1 = not_so_good.getptr();` here: https://en.cppreference.com/w/cpp/memory/enable_shared_from_this The comment there is: `// undefined behavior (until C++17) and std::bad_weak_ptr thrown (since C++17)` Does the existing code have UB pre C++17? I'll leave handling of enable_shared_from_this for later, as the need arises. * Cosmetical change around helper functions. * Using type_caster_base<mpty>::src_and_type directly, removing copy. Also renaming one cast to cast_const_raw_ptr, for clarity. * Fixing clang-format oversight. * Using factored-out make_constructor (PR #2798), removing duplicate code. * Inserting additional assert to ensure a returned unique_ptr is always a new Python instance. * Adding minor comment (change to internals needed to distinguish uninitialized/disowned in error message). * Factoring out find_existing_python_instance(). * Moving factored-out make_constructor to test_classh_wip.cpp, restoring previous version of cast.h. This is currently the most practical approach. See PR #2798 for background. * Copying classh type_casters from test_classh_wip.cpp UNMODIFIED, as a baseline for generalizing the code. * Using pybind11/detail/classh_type_casters.h from test_classh_wip.cpp. * Adding & using PYBIND11_CLASSH_TYPE_CASTERS define. * Adding test_classh_inheritance, currently failing (passes with class_). * Removing .clang-format before git rebase master (where the file was added). * Bringing back .clang-format, the previous rm was a bad idea. * Folding in modified_type_caster_generic_load_impl, just enough to pass test_class_wip. test_classh_inheritance is still failing, but with a different error: [RuntimeError: Incompatible type (as_raw_ptr_unowned).] * Minimal changes needed to pass test_classh_inheritance. * First pass adjusting try_implicit_casts and try_load_foreign_module_local to capture loaded_v_h, but untested and guarded with pybind11_failure("Untested"). This was done mainly to determine general feasibility. Note the TODO in pybind11.h, where type_caster_generic::local_load is currently hard-coded. test_classh_wip and test_classh_inheritance still pass, as before. * Decoupling generic_type from type_caster_generic. * Changes and tests covering classh_type_casters try_implicit_casts. * Minimal test covering classh_type_casters load_impl Case 2b. * Removing stray isinstance<T>(src): it interferes with the py::module_local feature. Adding missing #includes. * Tests for classh py::module_local() feature. * Pure renaming of function names in test_classh_inheritance, similar to the systematic approach used in test_class_wip. NO functional changes. * Pure renaming of function and variable names, for better generalization when convoluting with inheritance. NO functional changes. * Adopting systematic naming scheme from test_classh_wip. NO functional changes. * Moving const after type name, for functions that cover a systematic scheme. NO functional changes. * Adding smart_holder_type_caster_load::loaded_as_shared_ptr, currently bypassing smart_holder shared_ptr tracking completely, but the tests pass and are sanitizer clean. * Removing rtti_held from smart_holder. See updated comment. * Cleaning up loaded_as_raw_ptr_unowned, loaded_as_shared_ptr. * Factoring out convert_type and folding into loaded_as_unique_ptr. * Folding convert_type into lvalue_ref and rvalue_ref paths. Some smart_holder_type_caster_load cleanup. * Using unique_ptr in local_load to replace static variable. Also adding local_load_safety_guard. * Converting test_unique_ptr_member to using classh: fully working, ASAN, MSAN, UBSAN clean. * Removing debugging comments (GET_STACK, GET_INT_STACK). cast.h is identical to current master again, pybind11.h only has the generic_type::initialize(..., &type_caster_generic::local_load) change. * Purging obsolete pybind11/vptr_holder.h and associated test. * Moving several tests to github.com/rwgk/rwgk_tbx/tree/main/pybind11_tests https://github.com/rwgk/rwgk_tbx/commit/a2c2f88174a30f5de80d7d26e0f77c7b60f5fb85 These tests are from experimenting, and for demonstrating UB in pybind11 multiple inheritance handling ("first_base"), to be fixed later. * Adding py::smart_holder support to py::class_, purging py::classh completely. * Renaming files in include directory, creating pybind11/smart_holder.h. * Renaming all "classh" to "smart_holder" in pybind11/detail/smart_holder_type_casters.h. The user-facing macro is now PYBIND11_SMART_HOLDER_TYPE_CASTERS. * Systematically renaming tests to use "class_sh" in the name. * Renaming test_type_caster_bare_interface_demo to test_type_caster_bare_interface. * Renaming new tests/core subdirectory to tests/pure_cpp. * Adding new tests to CMake config, resetting CI config. * Changing CMake file so that test_class_sh_module_local.py actually runs. * clang-tidy fixes. * 32-bit compatibility. * Reusing type_caster_base make_copy_constructor, make_move_constructor with a trick. * CMake COMPARE NATURAL is not available with older versions. * Adding copyright notices to new header files. * Explicitly define copy/move constructors/assignments. * Adding new header files to tests/extra_python_package/test_files.py. * Adding tests/pure_cpp/CMakeLists.txt. * Making use of the new find_existing_python_instance() function factored out with PR #2822. * Moving define PYBIND11_SMART_HOLDER_TYPE_CASTERS(T) down in the file. NO functional changes. Preparation for follow-up work (to keep that diff smaller). * Reintroducing py::classh, this time as a simple alias for py::class_<U, py::smart_holder>. * Replacing detail::is_smart_holder<H> in cast.h with detail::is_smart_holder_type_caster<T>. Moving get_local_load_function_ptr, init_instance_for_type to smart_holder_type_caster_class_hooks. Expanding static_assert in py::type::handle_of<> to accommodate smart_holder_type_casters. * Fixing oversight. * Adding classu alias for class_<U, std::unique_ptr<U>>. * Giving up on idea to use legacy init_instance only if is_base_of<type_caster_generic, type_caster<T>. There are use cases in the wild that define both a custom type_caster and class_. * Removing test_type_caster_bare_interface, which was moved to the separate PR #2834. * Moving up is_smart_holder_type_caster, to also use in cast_is_temporary_value_reference. * Adding smart_holder_type_casters for unique_ptr with custom deleter. SEVERE CODE DUPLICATION. This commit is to establish a baseline for consolidating the unique_ptr code. * Unification of unique_ptr, unique_ptr_with_deleter code in smart_holder_poc.h. Leads to more fitting error messages. Enables use of unique_ptr<T, D> smart_holder_type_casters also for unique_ptr<T>. * Copying files as-is from branch test_unique_ptr_member (PR #2672). * Adding comment, simplifying naming, cmake addition. * Introducing PYBIND11_USE_SMART_HOLDER_AS_DEFAULT macro (tested only undefined; there are many errors with the macro defined). * Removing test_type_caster_bare_interface, which was moved to the separate PR #2834. * Fixing oversight introduced with commit 95425f13d6c14fcb6ee479b62b602dc8a605ec49. * Setting record.default_holder correctly for PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. With this test_class.cpp builds and even mostly runs, except `test_multiple_instances_with_same_pointer`, which segfaults because it is using a `unique_ptr` holder but `smart_holder` `type_caster`. Also adding `static_assert`s to generate build errors for such situations, but guarding with `#if 0` to first pivot to test_factory_constructors.cpp. * Fixing up cast.h and smart_holder.h after rebase. * Removing detail/smart_holder_type_casters.h in separate commit. * Commenting out const in def_buffer(... const). With this, test_buffers builds and runs with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. Explanation why the const needs to be removed, or fix elsewhere, is still needed, but left for later. * Adding test_class_sh_factory_constructors, reproducing test_factory_constructors failure. Using py::class_ in this commit, to be changed to py::classh for debugging. * Removing include/pybind11/detail/smart_holder_type_casters.h from CMakeLists.txt, test_files.py (since it does not exist in this branch). * Adding // DANGER ZONE reminders. * Converting as many py::class_ to py::classh as possible, not breaking tests. * Adding initimpl::construct() overloads, resulting in test_class_sh_factory_constructors feature parity for py::class_ and py::classh. * Adding enable_if !is_smart_holder_type_caster to existing initimpl::construct(). With this test_factory_constructors.cpp builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. * Disabling shared_ptr&, shared_ptr* tests when building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT for now, pending work on smart_holder_type_caster<shared_ptr>. * Factoring out struct and class definitions into anonymous namespace. Preparation for building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. * Simplifying from_unique_ptr(): typename D = std::default_delete<T> is not needed. Factoring out is_std_default_delete<T>() for consistentcy between ensure_compatible_rtti_uqp_del() and from_unique_ptr(). * Introducing PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. Using it in test_smart_ptr.cpp. With this test_smart_ptr builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT and all but one test run successfully. * Introducing 1. type_caster_for_class_, used in PYBIND11_MAKE_OPAQUE, and 2. default_holder_type, used in stl_bind.h. * Using __VA_ARGS__ in PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. * Replacing condense_for_macro with much simpler approach. * Softening static_assert, to only check specifically that smart_holder is not mixed with type_caster_base, and unique_ptr/shared_ptr holders are not mixed with smart_holder_type_casters. * Adding PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS in test_class.cpp (with this all but one test succeed with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT). * Adding remaining PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. static_assert for "necessary conditions" for both types of default holder, static_assert for "strict conditions" guarded by new PYBIND11_STRICT_ASSERTS_CLASS_HOLDER_VS_TYPE_CASTER_MIX. All tests build & run as before with unique_ptr as the default holder, all tests build for smart_holder as the default holder, even with the strict static_assert. * Introducing check_is_smart_holder_type_caster() function for runtime check, and reinterpreting record.default_holder as "uses_unique_ptr_holder". With this test_smart_ptr succeeds. (All 42 tests build, 35 tests succeed, 5 run but have some failures, 2 segfault.) * Bug fix: Adding have_value() to smart_holder_type_caster_load. With this test_builtin_casters succeeds. (All 42 tests build, 36 tests succeed, 5 run but have some failures, 1 segfault.) * Adding unowned_void_ptr_from_direct_conversion to modified_type_caster_generic_load_impl. This fixes the last remaining segfault (test_numpy_dtypes). New stats for all tests combined: 12 failed, 458 passed. * Adding "Lazy allocation for unallocated values" (for old-style __init__) into load_value_and_holder. Deferring destruction of disowned holder until clear_instance, to remain inspectable for "uninitialized" or "disowned" detection. New stats for all tests combined: 5 failed, 465 passed. * Changing std::shared_ptr pointer/reference to const pointer/reference. New stats for all tests combined: 4 failed, 466 passed. * Adding return_value_policy::move to permissible policies for unique_ptr returns. New stats for all tests combined: 3 failed, 467 passed. * Overlooked flake8 fixes. * Manipulating failing ConstructorStats test to pass, to be able to run all tests with ASAN. This version of the code is ASAN clean with unique_ptr or smart_holder as the default. This change needs to be reverted after adopting the existing move-only-if-refcount-is-1 logic used by type_caster_base. * Adding copy constructor and move constructor tracking to atyp. Preparation for a follow-up change in smart_holder_type_caster, to make this test sensitive to the changing behavior. [skip ci] * Removing `operator T&&() &&` from smart_holder_type_caster, for compatibility with the behavior of type_caster_base. Enables reverting 2 of 3 test manipulations applied under commit 249df7cbdb09817fed0ddf80f01ba5af12466820. The manipulation in test_factory_constructors.py is NOT reverted in this commit. [skip ci] * Fixing unfortunate editing mishap. This reverts the last remaining test manipulation in commit 249df7cbdb09817fed0ddf80f01ba5af12466820 and makes all existing unit tests pass with smart_holder as default holder. * GitHub CI clang-tidy fixes. * Adding messages to terse `static_assert`s, for pre-C++17 compatibility. * Using @pytest.mark.parametrize to run each assert separately (to see all errors, not just the first). * Systematically removing _atyp from function names, to make the test code simpler. * Using re.match to accommodate variable number of intermediate MvCtor. * Also removing `operator T()` from smart_holder_type_caster, to fix gcc compilation errors. The only loss is pass_rref in test_class_sh_basic. * Systematically replacing `detail::enable_if_t<...smart_holder...>` with `typename std::enable_if<...smart_holder...>::type`. Attempt to work around MSVC 2015 issues, to be tested via GitHub CI. The idea for this change originates from this comment: https://github.com/pybind/pybind11/issues/1616#issuecomment-444536813 * Importing re before pytest after observing a PyPy CI flake when importing pytest first. * Copying MSVC 2015 compatibility change from branch pr2672_use_smart_holder_as_default. * Introducing is_smart_holder_type_caster_base_tag, to keep smart_holder code more disconnected. * Working around MSVC 2015 bug. * Expanding comment for MSVC 2015 workaround. * Systematically changing std::enable_if back to detail::enable_if_t, effectively reverting commit 5d4b6890a337ae1bbaec4091f4195606f89a3b06. * Removing unused smart_holder_type_caster_load::loaded_as_rvalue_ref (it was an oversight that it was not removed with commit 23036a45eb4731a06b488ec1fdf83bca677b7f67). * Removing py::classu, because it does not seem useful enough. * Reverting commit 63495313066119dcf7510c2ae8b468b46c12ef8f by un-commenting `const` in `def_buffer(...)`. To make this possible, `operator T const&` and `operator T const*` in `smart_holder_type_caster` need to be marked as `const` member functions. * Adding construct() overloads for constructing smart_holder from alias unique_ptr, shared_ptr returns. * Adding test_class_sh_factory_constructors.cpp to tests/CMakeLists.txt (fixes oversight, this should have been added long before). * Compatibility with old clang versions (clang 3.6, 3.7 C++11). * Cleaning up changes to existing unit tests. * Systematically adding SMART_HOLDER_WIP tag. Removing minor UNTESTED tags (only the throw are not actually exercised, investing time there has a high cost but very little benefit). * Splitting out smart_holder_type_casters again, into new detail/smart_holder_type_casters_inline_include.h. * Splitting out smart_holder_init_inline_include.h. * Adding additional new include files to CMakeLists.txt, tests/extra_python_package/test_files.py. * clang-format cleanup of most smart_holder code. * Adding source code comments in response to review. * Simple micro-benchmark ("ubench") comparing runtime performance for several holders. Tested using github.com/rwgk/pybind11_scons and Google-internal build system. Sorry, no cmake support at the moment. First results: https://docs.google.com/spreadsheets/d/1InapCYws2Gt-stmFf_Bwl33eOMo3aLE_gc9adveY7RU/edit#gid=0 * Breaking out number_bucket.h, adding hook for also collecting performance data for PyCLIF. * Accounting for ubench in MANIFEST.in (simply prune, for now). * Smarter determination of call_repetitions. [skip ci] * Also scaling performance data to PyCLIF. [skip ci] * Adding ubench/python/number_bucket.clif here for general visibility. * Fix after rebase * Merging detail/smart_holder_init_inline_include.h into detail/init.h. * Renaming detail/is_smart_holder_type_caster.h -> detail/smart_holder_sfinae_hooks_only.h. * Renaming is_smart_holder_type_caster -> type_uses_smart_holder_type_caster for clarity. * Renaming type_caster_type_is_smart_holder_type_caster -> wrapped_type_uses_smart_holder_type_caster for clarity. * Renaming is_smart_holder_type_caster_base_tag -> smart_holder_type_caster_base_tag for simplicity. * Adding copyright notices and minor colateral cleanup. * iwyu cleanup (comprehensive only for cast.h and smart_holder*.h files). * Fixing `git rebase master` accident. * Moving large `pragma warning` block from pybind11.h to detail/common.h. * Fixing another `git rebase master` accident.
2021-02-24 05:50:42 +00:00
std::string pass_cref(atyp const& obj) { return "pass_cref:" + obj.mtxt; }
std::string pass_mref(atyp& obj) { return "pass_mref:" + obj.mtxt; }
std::string pass_cptr(atyp const* obj) { return "pass_cptr:" + obj->mtxt; }
std::string pass_mptr(atyp* obj) { return "pass_mptr:" + obj->mtxt; }
2021-07-27 23:18:30 +00:00
std::shared_ptr<atyp> rtrn_shmp() { return std::make_shared<atyp>("rtrn_shmp"); }
Adding py::smart_holder (for smart-pointer interoperability). (#2672) * Adding test_unique_ptr_member (for desired PyCLIF behavior). See also: https://github.com/pybind/pybind11/issues/2583 Does not build with upstream master or https://github.com/pybind/pybind11/pull/2047, but builds with https://github.com/RobotLocomotion/pybind11 and almost runs: ``` Running tests in directory "/usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests": ================================================================================= test session starts ================================================================================= platform linux -- Python 3.8.5, pytest-5.4.3, py-1.9.0, pluggy-0.13.1 rootdir: /usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests, inifile: pytest.ini collected 2 items test_unique_ptr_member.py .F [100%] ====================================================================================== FAILURES ======================================================================================= _____________________________________________________________________________ test_pointee_and_ptr_owner ______________________________________________________________________________ def test_pointee_and_ptr_owner(): obj = m.pointee() assert obj.get_int() == 213 m.ptr_owner(obj) with pytest.raises(ValueError) as exc_info: > obj.get_int() E Failed: DID NOT RAISE <class 'ValueError'> test_unique_ptr_member.py:17: Failed ============================================================================= 1 failed, 1 passed in 0.06s ============================================================================= ``` * unique_ptr or shared_ptr return * new test_variant_unique_shared with vptr_holder prototype * moving prototype code to pybind11/vptr_holder.h, adding type_caster specialization to make the bindings involving unique_ptr passing compile, but load and cast implementations are missing * disabling GitHub Actions on pull_request (for this PR) * disabling AppVeyor (for this PR) * TRIGGER_SEGSEV macro, annotations for GET_STACK (vptr::get), GET_INT_STACK (pointee) * adding test_promotion_of_disowned_to_shared * Copying tests as-is from xxx_value_ptr_xxx_holder branch. https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder Systematically exercising returning and passing unique_ptr<T>, shared_ptr<T> with unique_ptr, shared_ptr holder. Observations: test_holder_unique_ptr: make_unique_pointee OK pass_unique_pointee BUILD_FAIL (as documented) make_shared_pointee Abort free(): double free detected pass_shared_pointee RuntimeError: Unable to load a custom holder type from a default-holder instance test_holder_shared_ptr: make_unique_pointee Segmentation fault (#1138) pass_unique_pointee BUILD_FAIL (as documented) make_shared_pointee OK pass_shared_pointee OK * Copying tests as-is from xxx_value_ptr_xxx_holder branch. https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder Systematically exercising casting between shared_ptr<base>, shared_ptr<derived>. * Demonstration of Undefined Behavior in handling of shared_ptr holder. Based on https://godbolt.org/z/4fdjaW by jorgbrown@ (thanks Jorg!). * Additional demonstration of Undefined Behavior in handling of shared_ptr holder. * fixing up-down mixup in comment * Demonstration of Undefined Behavior in handling of polymorphic pointers. (This demo does NOT involve smart pointers at all, unlike the otherwise similar test_smart_ptr_private_first_base.) * minor test_private_first_base.cpp simplification (after discovering that this can be wrapped with Boost.Python, using boost::noncopyable) * pybind11 equivalent of Boost.Python test similar to reproducer under #1333 * Snapshot of WIP, TODO: shared_ptr deleter with on/off switch * Adding vptr_deleter. * Adding from/as unique_ptr<T> and unique_ptr<T, D>. * Adding from_shared_ptr. Some polishing. * New tests/core/smart_holder_poc_test.cpp, using Catch2. * Adding in vptr_deleter_guard_flag. * Improved labeling of TEST_CASEs. * Shuffling existing TEST_CASEs into systematic matrix. * Implementing all [S]uccess tests. * Implementing all [E]xception tests. * Testing of exceptions not covered by the from-as matrix. * Adding top-level comment. * Converting from methods to factory functions (no functional change). * Removing obsolete and very incomplete test (replaced by Catch2-based test). * Removing stray file. * Adding type_caster_bare_interface_demo. * Adding shared_ptr<mpty>, shared_ptr<mpty const> casters. * Adding unique_ptr<mpty>, unique_ptr<mpty const> casters. * Pure copy of `class class_` implementation in pybind11.h (master commit 98f1bbb8004f654ba9e26717bdf5912fb899b05a). * classh.h: renaming of class_ to classh + namespace; forking test_classh_wip from test_type_caster_bare_interface_demo. * Hard-coding smart_holder into classh. * Adding mpty::mtxt string member. * Adding isinstance<mpty> in type_caster::load functions. * Adding rvalue_ref, renaming const_value_ref to lvalue_ref & removing const. * Retrieving smart_holder pointer in type_caster<mpty>::load, and using it cast_op operators. * Factoring out smart_holder_type_caster_load. * Retrieving smart_holder pointer in type_caster<std::shared_ptr<mpty[ const]>>::load, and using it cast_op operators. * Improved error messaging: Cannot disown nullptr (as_unique_ptr). * Retrieving smart_holder pointer in type_caster<std::unique_ptr<mpty[ const]>>::load, and using it cast_op operators. * Pure `clang-format --style=file -i` change. * Pure `clang-format --style=file -i` change, with two `clang-format off` directives. * Fixing oversight (discovered by flake8). * flake8 cleanup * Systematically setting mtxt for all rtrn_mpty_* functions (preparation, the values are not actually used yet). * static cast handle for rtrn_cptr works by simply dropping in code from type_caster_base (marked with comments). * static cast handle for rtrn_cref works by simply dropping in code from type_caster_base (marked with comments). rtrn_mref and rtrn_mptr work via const_cast (to add const). * static cast handle for rtrn_valu works by simply dropping in code from type_caster_base (marked with comments). rtrn_rref raises a RuntimeError, to be investigated. * Copying type_caster_generic::cast into type_caster<mpty> as-is (preparation for handling smart pointers). * Pure clang-format change (applied to original type_caster_generic::cast). * Adding comment re potential use_count data race. * static handle cast implementations for rtrn_shmp, rtrn_shcp. * Adding MISSING comments in operator std::unique_ptr<mpty[ const]>. * static handle cast implementations for rtrn_uqmp, rtrn_uqcp. * Bug fix: vptr_deleter_armed_flag_ptr has to live on the heap. See new bullet point in comment section near the top. The variable was also renamed to reflect its function more accurately. * Fixing bugs discovered by ASAN. The code is now ASAN, MSAN, UBSAN clean. * Making test_type_caster_bare_interface_demo.cpp slightly more realistic, ASAN, MSAN, UBSAN clean. * Calling deregister_instance after disowning via unique_ptr. * Removing enable_shared_from_this stub, simplifying existing code, clang-format. Open question, with respect to the original code: https://github.com/pybind/pybind11/blob/76a160070b369f8d82b945c97924227e8b835c94/include/pybind11/pybind11.h#L1510 To me it looks like the exact situation marked as `std::shared_ptr<Good> gp1 = not_so_good.getptr();` here: https://en.cppreference.com/w/cpp/memory/enable_shared_from_this The comment there is: `// undefined behavior (until C++17) and std::bad_weak_ptr thrown (since C++17)` Does the existing code have UB pre C++17? I'll leave handling of enable_shared_from_this for later, as the need arises. * Cosmetical change around helper functions. * Using type_caster_base<mpty>::src_and_type directly, removing copy. Also renaming one cast to cast_const_raw_ptr, for clarity. * Fixing clang-format oversight. * Using factored-out make_constructor (PR #2798), removing duplicate code. * Inserting additional assert to ensure a returned unique_ptr is always a new Python instance. * Adding minor comment (change to internals needed to distinguish uninitialized/disowned in error message). * Factoring out find_existing_python_instance(). * Moving factored-out make_constructor to test_classh_wip.cpp, restoring previous version of cast.h. This is currently the most practical approach. See PR #2798 for background. * Copying classh type_casters from test_classh_wip.cpp UNMODIFIED, as a baseline for generalizing the code. * Using pybind11/detail/classh_type_casters.h from test_classh_wip.cpp. * Adding & using PYBIND11_CLASSH_TYPE_CASTERS define. * Adding test_classh_inheritance, currently failing (passes with class_). * Removing .clang-format before git rebase master (where the file was added). * Bringing back .clang-format, the previous rm was a bad idea. * Folding in modified_type_caster_generic_load_impl, just enough to pass test_class_wip. test_classh_inheritance is still failing, but with a different error: [RuntimeError: Incompatible type (as_raw_ptr_unowned).] * Minimal changes needed to pass test_classh_inheritance. * First pass adjusting try_implicit_casts and try_load_foreign_module_local to capture loaded_v_h, but untested and guarded with pybind11_failure("Untested"). This was done mainly to determine general feasibility. Note the TODO in pybind11.h, where type_caster_generic::local_load is currently hard-coded. test_classh_wip and test_classh_inheritance still pass, as before. * Decoupling generic_type from type_caster_generic. * Changes and tests covering classh_type_casters try_implicit_casts. * Minimal test covering classh_type_casters load_impl Case 2b. * Removing stray isinstance<T>(src): it interferes with the py::module_local feature. Adding missing #includes. * Tests for classh py::module_local() feature. * Pure renaming of function names in test_classh_inheritance, similar to the systematic approach used in test_class_wip. NO functional changes. * Pure renaming of function and variable names, for better generalization when convoluting with inheritance. NO functional changes. * Adopting systematic naming scheme from test_classh_wip. NO functional changes. * Moving const after type name, for functions that cover a systematic scheme. NO functional changes. * Adding smart_holder_type_caster_load::loaded_as_shared_ptr, currently bypassing smart_holder shared_ptr tracking completely, but the tests pass and are sanitizer clean. * Removing rtti_held from smart_holder. See updated comment. * Cleaning up loaded_as_raw_ptr_unowned, loaded_as_shared_ptr. * Factoring out convert_type and folding into loaded_as_unique_ptr. * Folding convert_type into lvalue_ref and rvalue_ref paths. Some smart_holder_type_caster_load cleanup. * Using unique_ptr in local_load to replace static variable. Also adding local_load_safety_guard. * Converting test_unique_ptr_member to using classh: fully working, ASAN, MSAN, UBSAN clean. * Removing debugging comments (GET_STACK, GET_INT_STACK). cast.h is identical to current master again, pybind11.h only has the generic_type::initialize(..., &type_caster_generic::local_load) change. * Purging obsolete pybind11/vptr_holder.h and associated test. * Moving several tests to github.com/rwgk/rwgk_tbx/tree/main/pybind11_tests https://github.com/rwgk/rwgk_tbx/commit/a2c2f88174a30f5de80d7d26e0f77c7b60f5fb85 These tests are from experimenting, and for demonstrating UB in pybind11 multiple inheritance handling ("first_base"), to be fixed later. * Adding py::smart_holder support to py::class_, purging py::classh completely. * Renaming files in include directory, creating pybind11/smart_holder.h. * Renaming all "classh" to "smart_holder" in pybind11/detail/smart_holder_type_casters.h. The user-facing macro is now PYBIND11_SMART_HOLDER_TYPE_CASTERS. * Systematically renaming tests to use "class_sh" in the name. * Renaming test_type_caster_bare_interface_demo to test_type_caster_bare_interface. * Renaming new tests/core subdirectory to tests/pure_cpp. * Adding new tests to CMake config, resetting CI config. * Changing CMake file so that test_class_sh_module_local.py actually runs. * clang-tidy fixes. * 32-bit compatibility. * Reusing type_caster_base make_copy_constructor, make_move_constructor with a trick. * CMake COMPARE NATURAL is not available with older versions. * Adding copyright notices to new header files. * Explicitly define copy/move constructors/assignments. * Adding new header files to tests/extra_python_package/test_files.py. * Adding tests/pure_cpp/CMakeLists.txt. * Making use of the new find_existing_python_instance() function factored out with PR #2822. * Moving define PYBIND11_SMART_HOLDER_TYPE_CASTERS(T) down in the file. NO functional changes. Preparation for follow-up work (to keep that diff smaller). * Reintroducing py::classh, this time as a simple alias for py::class_<U, py::smart_holder>. * Replacing detail::is_smart_holder<H> in cast.h with detail::is_smart_holder_type_caster<T>. Moving get_local_load_function_ptr, init_instance_for_type to smart_holder_type_caster_class_hooks. Expanding static_assert in py::type::handle_of<> to accommodate smart_holder_type_casters. * Fixing oversight. * Adding classu alias for class_<U, std::unique_ptr<U>>. * Giving up on idea to use legacy init_instance only if is_base_of<type_caster_generic, type_caster<T>. There are use cases in the wild that define both a custom type_caster and class_. * Removing test_type_caster_bare_interface, which was moved to the separate PR #2834. * Moving up is_smart_holder_type_caster, to also use in cast_is_temporary_value_reference. * Adding smart_holder_type_casters for unique_ptr with custom deleter. SEVERE CODE DUPLICATION. This commit is to establish a baseline for consolidating the unique_ptr code. * Unification of unique_ptr, unique_ptr_with_deleter code in smart_holder_poc.h. Leads to more fitting error messages. Enables use of unique_ptr<T, D> smart_holder_type_casters also for unique_ptr<T>. * Copying files as-is from branch test_unique_ptr_member (PR #2672). * Adding comment, simplifying naming, cmake addition. * Introducing PYBIND11_USE_SMART_HOLDER_AS_DEFAULT macro (tested only undefined; there are many errors with the macro defined). * Removing test_type_caster_bare_interface, which was moved to the separate PR #2834. * Fixing oversight introduced with commit 95425f13d6c14fcb6ee479b62b602dc8a605ec49. * Setting record.default_holder correctly for PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. With this test_class.cpp builds and even mostly runs, except `test_multiple_instances_with_same_pointer`, which segfaults because it is using a `unique_ptr` holder but `smart_holder` `type_caster`. Also adding `static_assert`s to generate build errors for such situations, but guarding with `#if 0` to first pivot to test_factory_constructors.cpp. * Fixing up cast.h and smart_holder.h after rebase. * Removing detail/smart_holder_type_casters.h in separate commit. * Commenting out const in def_buffer(... const). With this, test_buffers builds and runs with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. Explanation why the const needs to be removed, or fix elsewhere, is still needed, but left for later. * Adding test_class_sh_factory_constructors, reproducing test_factory_constructors failure. Using py::class_ in this commit, to be changed to py::classh for debugging. * Removing include/pybind11/detail/smart_holder_type_casters.h from CMakeLists.txt, test_files.py (since it does not exist in this branch). * Adding // DANGER ZONE reminders. * Converting as many py::class_ to py::classh as possible, not breaking tests. * Adding initimpl::construct() overloads, resulting in test_class_sh_factory_constructors feature parity for py::class_ and py::classh. * Adding enable_if !is_smart_holder_type_caster to existing initimpl::construct(). With this test_factory_constructors.cpp builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. * Disabling shared_ptr&, shared_ptr* tests when building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT for now, pending work on smart_holder_type_caster<shared_ptr>. * Factoring out struct and class definitions into anonymous namespace. Preparation for building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. * Simplifying from_unique_ptr(): typename D = std::default_delete<T> is not needed. Factoring out is_std_default_delete<T>() for consistentcy between ensure_compatible_rtti_uqp_del() and from_unique_ptr(). * Introducing PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. Using it in test_smart_ptr.cpp. With this test_smart_ptr builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT and all but one test run successfully. * Introducing 1. type_caster_for_class_, used in PYBIND11_MAKE_OPAQUE, and 2. default_holder_type, used in stl_bind.h. * Using __VA_ARGS__ in PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. * Replacing condense_for_macro with much simpler approach. * Softening static_assert, to only check specifically that smart_holder is not mixed with type_caster_base, and unique_ptr/shared_ptr holders are not mixed with smart_holder_type_casters. * Adding PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS in test_class.cpp (with this all but one test succeed with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT). * Adding remaining PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. static_assert for "necessary conditions" for both types of default holder, static_assert for "strict conditions" guarded by new PYBIND11_STRICT_ASSERTS_CLASS_HOLDER_VS_TYPE_CASTER_MIX. All tests build & run as before with unique_ptr as the default holder, all tests build for smart_holder as the default holder, even with the strict static_assert. * Introducing check_is_smart_holder_type_caster() function for runtime check, and reinterpreting record.default_holder as "uses_unique_ptr_holder". With this test_smart_ptr succeeds. (All 42 tests build, 35 tests succeed, 5 run but have some failures, 2 segfault.) * Bug fix: Adding have_value() to smart_holder_type_caster_load. With this test_builtin_casters succeeds. (All 42 tests build, 36 tests succeed, 5 run but have some failures, 1 segfault.) * Adding unowned_void_ptr_from_direct_conversion to modified_type_caster_generic_load_impl. This fixes the last remaining segfault (test_numpy_dtypes). New stats for all tests combined: 12 failed, 458 passed. * Adding "Lazy allocation for unallocated values" (for old-style __init__) into load_value_and_holder. Deferring destruction of disowned holder until clear_instance, to remain inspectable for "uninitialized" or "disowned" detection. New stats for all tests combined: 5 failed, 465 passed. * Changing std::shared_ptr pointer/reference to const pointer/reference. New stats for all tests combined: 4 failed, 466 passed. * Adding return_value_policy::move to permissible policies for unique_ptr returns. New stats for all tests combined: 3 failed, 467 passed. * Overlooked flake8 fixes. * Manipulating failing ConstructorStats test to pass, to be able to run all tests with ASAN. This version of the code is ASAN clean with unique_ptr or smart_holder as the default. This change needs to be reverted after adopting the existing move-only-if-refcount-is-1 logic used by type_caster_base. * Adding copy constructor and move constructor tracking to atyp. Preparation for a follow-up change in smart_holder_type_caster, to make this test sensitive to the changing behavior. [skip ci] * Removing `operator T&&() &&` from smart_holder_type_caster, for compatibility with the behavior of type_caster_base. Enables reverting 2 of 3 test manipulations applied under commit 249df7cbdb09817fed0ddf80f01ba5af12466820. The manipulation in test_factory_constructors.py is NOT reverted in this commit. [skip ci] * Fixing unfortunate editing mishap. This reverts the last remaining test manipulation in commit 249df7cbdb09817fed0ddf80f01ba5af12466820 and makes all existing unit tests pass with smart_holder as default holder. * GitHub CI clang-tidy fixes. * Adding messages to terse `static_assert`s, for pre-C++17 compatibility. * Using @pytest.mark.parametrize to run each assert separately (to see all errors, not just the first). * Systematically removing _atyp from function names, to make the test code simpler. * Using re.match to accommodate variable number of intermediate MvCtor. * Also removing `operator T()` from smart_holder_type_caster, to fix gcc compilation errors. The only loss is pass_rref in test_class_sh_basic. * Systematically replacing `detail::enable_if_t<...smart_holder...>` with `typename std::enable_if<...smart_holder...>::type`. Attempt to work around MSVC 2015 issues, to be tested via GitHub CI. The idea for this change originates from this comment: https://github.com/pybind/pybind11/issues/1616#issuecomment-444536813 * Importing re before pytest after observing a PyPy CI flake when importing pytest first. * Copying MSVC 2015 compatibility change from branch pr2672_use_smart_holder_as_default. * Introducing is_smart_holder_type_caster_base_tag, to keep smart_holder code more disconnected. * Working around MSVC 2015 bug. * Expanding comment for MSVC 2015 workaround. * Systematically changing std::enable_if back to detail::enable_if_t, effectively reverting commit 5d4b6890a337ae1bbaec4091f4195606f89a3b06. * Removing unused smart_holder_type_caster_load::loaded_as_rvalue_ref (it was an oversight that it was not removed with commit 23036a45eb4731a06b488ec1fdf83bca677b7f67). * Removing py::classu, because it does not seem useful enough. * Reverting commit 63495313066119dcf7510c2ae8b468b46c12ef8f by un-commenting `const` in `def_buffer(...)`. To make this possible, `operator T const&` and `operator T const*` in `smart_holder_type_caster` need to be marked as `const` member functions. * Adding construct() overloads for constructing smart_holder from alias unique_ptr, shared_ptr returns. * Adding test_class_sh_factory_constructors.cpp to tests/CMakeLists.txt (fixes oversight, this should have been added long before). * Compatibility with old clang versions (clang 3.6, 3.7 C++11). * Cleaning up changes to existing unit tests. * Systematically adding SMART_HOLDER_WIP tag. Removing minor UNTESTED tags (only the throw are not actually exercised, investing time there has a high cost but very little benefit). * Splitting out smart_holder_type_casters again, into new detail/smart_holder_type_casters_inline_include.h. * Splitting out smart_holder_init_inline_include.h. * Adding additional new include files to CMakeLists.txt, tests/extra_python_package/test_files.py. * clang-format cleanup of most smart_holder code. * Adding source code comments in response to review. * Simple micro-benchmark ("ubench") comparing runtime performance for several holders. Tested using github.com/rwgk/pybind11_scons and Google-internal build system. Sorry, no cmake support at the moment. First results: https://docs.google.com/spreadsheets/d/1InapCYws2Gt-stmFf_Bwl33eOMo3aLE_gc9adveY7RU/edit#gid=0 * Breaking out number_bucket.h, adding hook for also collecting performance data for PyCLIF. * Accounting for ubench in MANIFEST.in (simply prune, for now). * Smarter determination of call_repetitions. [skip ci] * Also scaling performance data to PyCLIF. [skip ci] * Adding ubench/python/number_bucket.clif here for general visibility. * Fix after rebase * Merging detail/smart_holder_init_inline_include.h into detail/init.h. * Renaming detail/is_smart_holder_type_caster.h -> detail/smart_holder_sfinae_hooks_only.h. * Renaming is_smart_holder_type_caster -> type_uses_smart_holder_type_caster for clarity. * Renaming type_caster_type_is_smart_holder_type_caster -> wrapped_type_uses_smart_holder_type_caster for clarity. * Renaming is_smart_holder_type_caster_base_tag -> smart_holder_type_caster_base_tag for simplicity. * Adding copyright notices and minor colateral cleanup. * iwyu cleanup (comprehensive only for cast.h and smart_holder*.h files). * Fixing `git rebase master` accident. * Moving large `pragma warning` block from pybind11.h to detail/common.h. * Fixing another `git rebase master` accident.
2021-02-24 05:50:42 +00:00
std::shared_ptr<atyp const> rtrn_shcp() { return std::shared_ptr<atyp const>(new atyp{"rtrn_shcp"}); }
std::string pass_shmp(std::shared_ptr<atyp> obj) { return "pass_shmp:" + obj->mtxt; } // NOLINT
std::string pass_shcp(std::shared_ptr<atyp const> obj) { return "pass_shcp:" + obj->mtxt; } // NOLINT
Adding py::smart_holder (for smart-pointer interoperability). (#2672) * Adding test_unique_ptr_member (for desired PyCLIF behavior). See also: https://github.com/pybind/pybind11/issues/2583 Does not build with upstream master or https://github.com/pybind/pybind11/pull/2047, but builds with https://github.com/RobotLocomotion/pybind11 and almost runs: ``` Running tests in directory "/usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests": ================================================================================= test session starts ================================================================================= platform linux -- Python 3.8.5, pytest-5.4.3, py-1.9.0, pluggy-0.13.1 rootdir: /usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests, inifile: pytest.ini collected 2 items test_unique_ptr_member.py .F [100%] ====================================================================================== FAILURES ======================================================================================= _____________________________________________________________________________ test_pointee_and_ptr_owner ______________________________________________________________________________ def test_pointee_and_ptr_owner(): obj = m.pointee() assert obj.get_int() == 213 m.ptr_owner(obj) with pytest.raises(ValueError) as exc_info: > obj.get_int() E Failed: DID NOT RAISE <class 'ValueError'> test_unique_ptr_member.py:17: Failed ============================================================================= 1 failed, 1 passed in 0.06s ============================================================================= ``` * unique_ptr or shared_ptr return * new test_variant_unique_shared with vptr_holder prototype * moving prototype code to pybind11/vptr_holder.h, adding type_caster specialization to make the bindings involving unique_ptr passing compile, but load and cast implementations are missing * disabling GitHub Actions on pull_request (for this PR) * disabling AppVeyor (for this PR) * TRIGGER_SEGSEV macro, annotations for GET_STACK (vptr::get), GET_INT_STACK (pointee) * adding test_promotion_of_disowned_to_shared * Copying tests as-is from xxx_value_ptr_xxx_holder branch. https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder Systematically exercising returning and passing unique_ptr<T>, shared_ptr<T> with unique_ptr, shared_ptr holder. Observations: test_holder_unique_ptr: make_unique_pointee OK pass_unique_pointee BUILD_FAIL (as documented) make_shared_pointee Abort free(): double free detected pass_shared_pointee RuntimeError: Unable to load a custom holder type from a default-holder instance test_holder_shared_ptr: make_unique_pointee Segmentation fault (#1138) pass_unique_pointee BUILD_FAIL (as documented) make_shared_pointee OK pass_shared_pointee OK * Copying tests as-is from xxx_value_ptr_xxx_holder branch. https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder Systematically exercising casting between shared_ptr<base>, shared_ptr<derived>. * Demonstration of Undefined Behavior in handling of shared_ptr holder. Based on https://godbolt.org/z/4fdjaW by jorgbrown@ (thanks Jorg!). * Additional demonstration of Undefined Behavior in handling of shared_ptr holder. * fixing up-down mixup in comment * Demonstration of Undefined Behavior in handling of polymorphic pointers. (This demo does NOT involve smart pointers at all, unlike the otherwise similar test_smart_ptr_private_first_base.) * minor test_private_first_base.cpp simplification (after discovering that this can be wrapped with Boost.Python, using boost::noncopyable) * pybind11 equivalent of Boost.Python test similar to reproducer under #1333 * Snapshot of WIP, TODO: shared_ptr deleter with on/off switch * Adding vptr_deleter. * Adding from/as unique_ptr<T> and unique_ptr<T, D>. * Adding from_shared_ptr. Some polishing. * New tests/core/smart_holder_poc_test.cpp, using Catch2. * Adding in vptr_deleter_guard_flag. * Improved labeling of TEST_CASEs. * Shuffling existing TEST_CASEs into systematic matrix. * Implementing all [S]uccess tests. * Implementing all [E]xception tests. * Testing of exceptions not covered by the from-as matrix. * Adding top-level comment. * Converting from methods to factory functions (no functional change). * Removing obsolete and very incomplete test (replaced by Catch2-based test). * Removing stray file. * Adding type_caster_bare_interface_demo. * Adding shared_ptr<mpty>, shared_ptr<mpty const> casters. * Adding unique_ptr<mpty>, unique_ptr<mpty const> casters. * Pure copy of `class class_` implementation in pybind11.h (master commit 98f1bbb8004f654ba9e26717bdf5912fb899b05a). * classh.h: renaming of class_ to classh + namespace; forking test_classh_wip from test_type_caster_bare_interface_demo. * Hard-coding smart_holder into classh. * Adding mpty::mtxt string member. * Adding isinstance<mpty> in type_caster::load functions. * Adding rvalue_ref, renaming const_value_ref to lvalue_ref & removing const. * Retrieving smart_holder pointer in type_caster<mpty>::load, and using it cast_op operators. * Factoring out smart_holder_type_caster_load. * Retrieving smart_holder pointer in type_caster<std::shared_ptr<mpty[ const]>>::load, and using it cast_op operators. * Improved error messaging: Cannot disown nullptr (as_unique_ptr). * Retrieving smart_holder pointer in type_caster<std::unique_ptr<mpty[ const]>>::load, and using it cast_op operators. * Pure `clang-format --style=file -i` change. * Pure `clang-format --style=file -i` change, with two `clang-format off` directives. * Fixing oversight (discovered by flake8). * flake8 cleanup * Systematically setting mtxt for all rtrn_mpty_* functions (preparation, the values are not actually used yet). * static cast handle for rtrn_cptr works by simply dropping in code from type_caster_base (marked with comments). * static cast handle for rtrn_cref works by simply dropping in code from type_caster_base (marked with comments). rtrn_mref and rtrn_mptr work via const_cast (to add const). * static cast handle for rtrn_valu works by simply dropping in code from type_caster_base (marked with comments). rtrn_rref raises a RuntimeError, to be investigated. * Copying type_caster_generic::cast into type_caster<mpty> as-is (preparation for handling smart pointers). * Pure clang-format change (applied to original type_caster_generic::cast). * Adding comment re potential use_count data race. * static handle cast implementations for rtrn_shmp, rtrn_shcp. * Adding MISSING comments in operator std::unique_ptr<mpty[ const]>. * static handle cast implementations for rtrn_uqmp, rtrn_uqcp. * Bug fix: vptr_deleter_armed_flag_ptr has to live on the heap. See new bullet point in comment section near the top. The variable was also renamed to reflect its function more accurately. * Fixing bugs discovered by ASAN. The code is now ASAN, MSAN, UBSAN clean. * Making test_type_caster_bare_interface_demo.cpp slightly more realistic, ASAN, MSAN, UBSAN clean. * Calling deregister_instance after disowning via unique_ptr. * Removing enable_shared_from_this stub, simplifying existing code, clang-format. Open question, with respect to the original code: https://github.com/pybind/pybind11/blob/76a160070b369f8d82b945c97924227e8b835c94/include/pybind11/pybind11.h#L1510 To me it looks like the exact situation marked as `std::shared_ptr<Good> gp1 = not_so_good.getptr();` here: https://en.cppreference.com/w/cpp/memory/enable_shared_from_this The comment there is: `// undefined behavior (until C++17) and std::bad_weak_ptr thrown (since C++17)` Does the existing code have UB pre C++17? I'll leave handling of enable_shared_from_this for later, as the need arises. * Cosmetical change around helper functions. * Using type_caster_base<mpty>::src_and_type directly, removing copy. Also renaming one cast to cast_const_raw_ptr, for clarity. * Fixing clang-format oversight. * Using factored-out make_constructor (PR #2798), removing duplicate code. * Inserting additional assert to ensure a returned unique_ptr is always a new Python instance. * Adding minor comment (change to internals needed to distinguish uninitialized/disowned in error message). * Factoring out find_existing_python_instance(). * Moving factored-out make_constructor to test_classh_wip.cpp, restoring previous version of cast.h. This is currently the most practical approach. See PR #2798 for background. * Copying classh type_casters from test_classh_wip.cpp UNMODIFIED, as a baseline for generalizing the code. * Using pybind11/detail/classh_type_casters.h from test_classh_wip.cpp. * Adding & using PYBIND11_CLASSH_TYPE_CASTERS define. * Adding test_classh_inheritance, currently failing (passes with class_). * Removing .clang-format before git rebase master (where the file was added). * Bringing back .clang-format, the previous rm was a bad idea. * Folding in modified_type_caster_generic_load_impl, just enough to pass test_class_wip. test_classh_inheritance is still failing, but with a different error: [RuntimeError: Incompatible type (as_raw_ptr_unowned).] * Minimal changes needed to pass test_classh_inheritance. * First pass adjusting try_implicit_casts and try_load_foreign_module_local to capture loaded_v_h, but untested and guarded with pybind11_failure("Untested"). This was done mainly to determine general feasibility. Note the TODO in pybind11.h, where type_caster_generic::local_load is currently hard-coded. test_classh_wip and test_classh_inheritance still pass, as before. * Decoupling generic_type from type_caster_generic. * Changes and tests covering classh_type_casters try_implicit_casts. * Minimal test covering classh_type_casters load_impl Case 2b. * Removing stray isinstance<T>(src): it interferes with the py::module_local feature. Adding missing #includes. * Tests for classh py::module_local() feature. * Pure renaming of function names in test_classh_inheritance, similar to the systematic approach used in test_class_wip. NO functional changes. * Pure renaming of function and variable names, for better generalization when convoluting with inheritance. NO functional changes. * Adopting systematic naming scheme from test_classh_wip. NO functional changes. * Moving const after type name, for functions that cover a systematic scheme. NO functional changes. * Adding smart_holder_type_caster_load::loaded_as_shared_ptr, currently bypassing smart_holder shared_ptr tracking completely, but the tests pass and are sanitizer clean. * Removing rtti_held from smart_holder. See updated comment. * Cleaning up loaded_as_raw_ptr_unowned, loaded_as_shared_ptr. * Factoring out convert_type and folding into loaded_as_unique_ptr. * Folding convert_type into lvalue_ref and rvalue_ref paths. Some smart_holder_type_caster_load cleanup. * Using unique_ptr in local_load to replace static variable. Also adding local_load_safety_guard. * Converting test_unique_ptr_member to using classh: fully working, ASAN, MSAN, UBSAN clean. * Removing debugging comments (GET_STACK, GET_INT_STACK). cast.h is identical to current master again, pybind11.h only has the generic_type::initialize(..., &type_caster_generic::local_load) change. * Purging obsolete pybind11/vptr_holder.h and associated test. * Moving several tests to github.com/rwgk/rwgk_tbx/tree/main/pybind11_tests https://github.com/rwgk/rwgk_tbx/commit/a2c2f88174a30f5de80d7d26e0f77c7b60f5fb85 These tests are from experimenting, and for demonstrating UB in pybind11 multiple inheritance handling ("first_base"), to be fixed later. * Adding py::smart_holder support to py::class_, purging py::classh completely. * Renaming files in include directory, creating pybind11/smart_holder.h. * Renaming all "classh" to "smart_holder" in pybind11/detail/smart_holder_type_casters.h. The user-facing macro is now PYBIND11_SMART_HOLDER_TYPE_CASTERS. * Systematically renaming tests to use "class_sh" in the name. * Renaming test_type_caster_bare_interface_demo to test_type_caster_bare_interface. * Renaming new tests/core subdirectory to tests/pure_cpp. * Adding new tests to CMake config, resetting CI config. * Changing CMake file so that test_class_sh_module_local.py actually runs. * clang-tidy fixes. * 32-bit compatibility. * Reusing type_caster_base make_copy_constructor, make_move_constructor with a trick. * CMake COMPARE NATURAL is not available with older versions. * Adding copyright notices to new header files. * Explicitly define copy/move constructors/assignments. * Adding new header files to tests/extra_python_package/test_files.py. * Adding tests/pure_cpp/CMakeLists.txt. * Making use of the new find_existing_python_instance() function factored out with PR #2822. * Moving define PYBIND11_SMART_HOLDER_TYPE_CASTERS(T) down in the file. NO functional changes. Preparation for follow-up work (to keep that diff smaller). * Reintroducing py::classh, this time as a simple alias for py::class_<U, py::smart_holder>. * Replacing detail::is_smart_holder<H> in cast.h with detail::is_smart_holder_type_caster<T>. Moving get_local_load_function_ptr, init_instance_for_type to smart_holder_type_caster_class_hooks. Expanding static_assert in py::type::handle_of<> to accommodate smart_holder_type_casters. * Fixing oversight. * Adding classu alias for class_<U, std::unique_ptr<U>>. * Giving up on idea to use legacy init_instance only if is_base_of<type_caster_generic, type_caster<T>. There are use cases in the wild that define both a custom type_caster and class_. * Removing test_type_caster_bare_interface, which was moved to the separate PR #2834. * Moving up is_smart_holder_type_caster, to also use in cast_is_temporary_value_reference. * Adding smart_holder_type_casters for unique_ptr with custom deleter. SEVERE CODE DUPLICATION. This commit is to establish a baseline for consolidating the unique_ptr code. * Unification of unique_ptr, unique_ptr_with_deleter code in smart_holder_poc.h. Leads to more fitting error messages. Enables use of unique_ptr<T, D> smart_holder_type_casters also for unique_ptr<T>. * Copying files as-is from branch test_unique_ptr_member (PR #2672). * Adding comment, simplifying naming, cmake addition. * Introducing PYBIND11_USE_SMART_HOLDER_AS_DEFAULT macro (tested only undefined; there are many errors with the macro defined). * Removing test_type_caster_bare_interface, which was moved to the separate PR #2834. * Fixing oversight introduced with commit 95425f13d6c14fcb6ee479b62b602dc8a605ec49. * Setting record.default_holder correctly for PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. With this test_class.cpp builds and even mostly runs, except `test_multiple_instances_with_same_pointer`, which segfaults because it is using a `unique_ptr` holder but `smart_holder` `type_caster`. Also adding `static_assert`s to generate build errors for such situations, but guarding with `#if 0` to first pivot to test_factory_constructors.cpp. * Fixing up cast.h and smart_holder.h after rebase. * Removing detail/smart_holder_type_casters.h in separate commit. * Commenting out const in def_buffer(... const). With this, test_buffers builds and runs with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. Explanation why the const needs to be removed, or fix elsewhere, is still needed, but left for later. * Adding test_class_sh_factory_constructors, reproducing test_factory_constructors failure. Using py::class_ in this commit, to be changed to py::classh for debugging. * Removing include/pybind11/detail/smart_holder_type_casters.h from CMakeLists.txt, test_files.py (since it does not exist in this branch). * Adding // DANGER ZONE reminders. * Converting as many py::class_ to py::classh as possible, not breaking tests. * Adding initimpl::construct() overloads, resulting in test_class_sh_factory_constructors feature parity for py::class_ and py::classh. * Adding enable_if !is_smart_holder_type_caster to existing initimpl::construct(). With this test_factory_constructors.cpp builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. * Disabling shared_ptr&, shared_ptr* tests when building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT for now, pending work on smart_holder_type_caster<shared_ptr>. * Factoring out struct and class definitions into anonymous namespace. Preparation for building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. * Simplifying from_unique_ptr(): typename D = std::default_delete<T> is not needed. Factoring out is_std_default_delete<T>() for consistentcy between ensure_compatible_rtti_uqp_del() and from_unique_ptr(). * Introducing PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. Using it in test_smart_ptr.cpp. With this test_smart_ptr builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT and all but one test run successfully. * Introducing 1. type_caster_for_class_, used in PYBIND11_MAKE_OPAQUE, and 2. default_holder_type, used in stl_bind.h. * Using __VA_ARGS__ in PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. * Replacing condense_for_macro with much simpler approach. * Softening static_assert, to only check specifically that smart_holder is not mixed with type_caster_base, and unique_ptr/shared_ptr holders are not mixed with smart_holder_type_casters. * Adding PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS in test_class.cpp (with this all but one test succeed with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT). * Adding remaining PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. static_assert for "necessary conditions" for both types of default holder, static_assert for "strict conditions" guarded by new PYBIND11_STRICT_ASSERTS_CLASS_HOLDER_VS_TYPE_CASTER_MIX. All tests build & run as before with unique_ptr as the default holder, all tests build for smart_holder as the default holder, even with the strict static_assert. * Introducing check_is_smart_holder_type_caster() function for runtime check, and reinterpreting record.default_holder as "uses_unique_ptr_holder". With this test_smart_ptr succeeds. (All 42 tests build, 35 tests succeed, 5 run but have some failures, 2 segfault.) * Bug fix: Adding have_value() to smart_holder_type_caster_load. With this test_builtin_casters succeeds. (All 42 tests build, 36 tests succeed, 5 run but have some failures, 1 segfault.) * Adding unowned_void_ptr_from_direct_conversion to modified_type_caster_generic_load_impl. This fixes the last remaining segfault (test_numpy_dtypes). New stats for all tests combined: 12 failed, 458 passed. * Adding "Lazy allocation for unallocated values" (for old-style __init__) into load_value_and_holder. Deferring destruction of disowned holder until clear_instance, to remain inspectable for "uninitialized" or "disowned" detection. New stats for all tests combined: 5 failed, 465 passed. * Changing std::shared_ptr pointer/reference to const pointer/reference. New stats for all tests combined: 4 failed, 466 passed. * Adding return_value_policy::move to permissible policies for unique_ptr returns. New stats for all tests combined: 3 failed, 467 passed. * Overlooked flake8 fixes. * Manipulating failing ConstructorStats test to pass, to be able to run all tests with ASAN. This version of the code is ASAN clean with unique_ptr or smart_holder as the default. This change needs to be reverted after adopting the existing move-only-if-refcount-is-1 logic used by type_caster_base. * Adding copy constructor and move constructor tracking to atyp. Preparation for a follow-up change in smart_holder_type_caster, to make this test sensitive to the changing behavior. [skip ci] * Removing `operator T&&() &&` from smart_holder_type_caster, for compatibility with the behavior of type_caster_base. Enables reverting 2 of 3 test manipulations applied under commit 249df7cbdb09817fed0ddf80f01ba5af12466820. The manipulation in test_factory_constructors.py is NOT reverted in this commit. [skip ci] * Fixing unfortunate editing mishap. This reverts the last remaining test manipulation in commit 249df7cbdb09817fed0ddf80f01ba5af12466820 and makes all existing unit tests pass with smart_holder as default holder. * GitHub CI clang-tidy fixes. * Adding messages to terse `static_assert`s, for pre-C++17 compatibility. * Using @pytest.mark.parametrize to run each assert separately (to see all errors, not just the first). * Systematically removing _atyp from function names, to make the test code simpler. * Using re.match to accommodate variable number of intermediate MvCtor. * Also removing `operator T()` from smart_holder_type_caster, to fix gcc compilation errors. The only loss is pass_rref in test_class_sh_basic. * Systematically replacing `detail::enable_if_t<...smart_holder...>` with `typename std::enable_if<...smart_holder...>::type`. Attempt to work around MSVC 2015 issues, to be tested via GitHub CI. The idea for this change originates from this comment: https://github.com/pybind/pybind11/issues/1616#issuecomment-444536813 * Importing re before pytest after observing a PyPy CI flake when importing pytest first. * Copying MSVC 2015 compatibility change from branch pr2672_use_smart_holder_as_default. * Introducing is_smart_holder_type_caster_base_tag, to keep smart_holder code more disconnected. * Working around MSVC 2015 bug. * Expanding comment for MSVC 2015 workaround. * Systematically changing std::enable_if back to detail::enable_if_t, effectively reverting commit 5d4b6890a337ae1bbaec4091f4195606f89a3b06. * Removing unused smart_holder_type_caster_load::loaded_as_rvalue_ref (it was an oversight that it was not removed with commit 23036a45eb4731a06b488ec1fdf83bca677b7f67). * Removing py::classu, because it does not seem useful enough. * Reverting commit 63495313066119dcf7510c2ae8b468b46c12ef8f by un-commenting `const` in `def_buffer(...)`. To make this possible, `operator T const&` and `operator T const*` in `smart_holder_type_caster` need to be marked as `const` member functions. * Adding construct() overloads for constructing smart_holder from alias unique_ptr, shared_ptr returns. * Adding test_class_sh_factory_constructors.cpp to tests/CMakeLists.txt (fixes oversight, this should have been added long before). * Compatibility with old clang versions (clang 3.6, 3.7 C++11). * Cleaning up changes to existing unit tests. * Systematically adding SMART_HOLDER_WIP tag. Removing minor UNTESTED tags (only the throw are not actually exercised, investing time there has a high cost but very little benefit). * Splitting out smart_holder_type_casters again, into new detail/smart_holder_type_casters_inline_include.h. * Splitting out smart_holder_init_inline_include.h. * Adding additional new include files to CMakeLists.txt, tests/extra_python_package/test_files.py. * clang-format cleanup of most smart_holder code. * Adding source code comments in response to review. * Simple micro-benchmark ("ubench") comparing runtime performance for several holders. Tested using github.com/rwgk/pybind11_scons and Google-internal build system. Sorry, no cmake support at the moment. First results: https://docs.google.com/spreadsheets/d/1InapCYws2Gt-stmFf_Bwl33eOMo3aLE_gc9adveY7RU/edit#gid=0 * Breaking out number_bucket.h, adding hook for also collecting performance data for PyCLIF. * Accounting for ubench in MANIFEST.in (simply prune, for now). * Smarter determination of call_repetitions. [skip ci] * Also scaling performance data to PyCLIF. [skip ci] * Adding ubench/python/number_bucket.clif here for general visibility. * Fix after rebase * Merging detail/smart_holder_init_inline_include.h into detail/init.h. * Renaming detail/is_smart_holder_type_caster.h -> detail/smart_holder_sfinae_hooks_only.h. * Renaming is_smart_holder_type_caster -> type_uses_smart_holder_type_caster for clarity. * Renaming type_caster_type_is_smart_holder_type_caster -> wrapped_type_uses_smart_holder_type_caster for clarity. * Renaming is_smart_holder_type_caster_base_tag -> smart_holder_type_caster_base_tag for simplicity. * Adding copyright notices and minor colateral cleanup. * iwyu cleanup (comprehensive only for cast.h and smart_holder*.h files). * Fixing `git rebase master` accident. * Moving large `pragma warning` block from pybind11.h to detail/common.h. * Fixing another `git rebase master` accident.
2021-02-24 05:50:42 +00:00
std::unique_ptr<atyp> rtrn_uqmp() { return std::unique_ptr<atyp >(new atyp{"rtrn_uqmp"}); }
std::unique_ptr<atyp const> rtrn_uqcp() { return std::unique_ptr<atyp const>(new atyp{"rtrn_uqcp"}); }
std::string pass_uqmp(std::unique_ptr<atyp > obj) { return "pass_uqmp:" + obj->mtxt; }
std::string pass_uqcp(std::unique_ptr<atyp const> obj) { return "pass_uqcp:" + obj->mtxt; }
struct sddm : std::default_delete<atyp > {};
struct sddc : std::default_delete<atyp const> {};
std::unique_ptr<atyp, sddm> rtrn_udmp() { return std::unique_ptr<atyp, sddm>(new atyp{"rtrn_udmp"}); }
std::unique_ptr<atyp const, sddc> rtrn_udcp() { return std::unique_ptr<atyp const, sddc>(new atyp{"rtrn_udcp"}); }
std::string pass_udmp(std::unique_ptr<atyp, sddm> obj) { return "pass_udmp:" + obj->mtxt; }
std::string pass_udcp(std::unique_ptr<atyp const, sddc> obj) { return "pass_udcp:" + obj->mtxt; }
std::unique_ptr<atyp, custom_deleter> rtrn_udmp_del() { return std::unique_ptr<atyp, custom_deleter>(new atyp{"rtrn_udmp_del"}, custom_deleter{"udmp_deleter"}); }
std::unique_ptr<atyp const, custom_deleter> rtrn_udcp_del() { return std::unique_ptr<atyp const, custom_deleter>(new atyp{"rtrn_udcp_del"}, custom_deleter{"udcp_deleter"}); }
std::string pass_udmp_del(std::unique_ptr<atyp, custom_deleter> obj) { return "pass_udmp_del:" + obj->mtxt + "," + obj.get_deleter().trace_txt; }
std::string pass_udcp_del(std::unique_ptr<atyp const, custom_deleter> obj) { return "pass_udcp_del:" + obj->mtxt + "," + obj.get_deleter().trace_txt; }
std::unique_ptr<atyp, custom_deleter_nd> rtrn_udmp_del_nd() { return std::unique_ptr<atyp, custom_deleter_nd>(new atyp{"rtrn_udmp_del_nd"}, custom_deleter_nd{"udmp_deleter_nd"}); }
std::unique_ptr<atyp const, custom_deleter_nd> rtrn_udcp_del_nd() { return std::unique_ptr<atyp const, custom_deleter_nd>(new atyp{"rtrn_udcp_del_nd"}, custom_deleter_nd{"udcp_deleter_nd"}); }
std::string pass_udmp_del_nd(std::unique_ptr<atyp, custom_deleter_nd> obj) { return "pass_udmp_del_nd:" + obj->mtxt + "," + obj.get_deleter().trace_txt; }
std::string pass_udcp_del_nd(std::unique_ptr<atyp const, custom_deleter_nd> obj) { return "pass_udcp_del_nd:" + obj->mtxt + "," + obj.get_deleter().trace_txt; }
Adding py::smart_holder (for smart-pointer interoperability). (#2672) * Adding test_unique_ptr_member (for desired PyCLIF behavior). See also: https://github.com/pybind/pybind11/issues/2583 Does not build with upstream master or https://github.com/pybind/pybind11/pull/2047, but builds with https://github.com/RobotLocomotion/pybind11 and almost runs: ``` Running tests in directory "/usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests": ================================================================================= test session starts ================================================================================= platform linux -- Python 3.8.5, pytest-5.4.3, py-1.9.0, pluggy-0.13.1 rootdir: /usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests, inifile: pytest.ini collected 2 items test_unique_ptr_member.py .F [100%] ====================================================================================== FAILURES ======================================================================================= _____________________________________________________________________________ test_pointee_and_ptr_owner ______________________________________________________________________________ def test_pointee_and_ptr_owner(): obj = m.pointee() assert obj.get_int() == 213 m.ptr_owner(obj) with pytest.raises(ValueError) as exc_info: > obj.get_int() E Failed: DID NOT RAISE <class 'ValueError'> test_unique_ptr_member.py:17: Failed ============================================================================= 1 failed, 1 passed in 0.06s ============================================================================= ``` * unique_ptr or shared_ptr return * new test_variant_unique_shared with vptr_holder prototype * moving prototype code to pybind11/vptr_holder.h, adding type_caster specialization to make the bindings involving unique_ptr passing compile, but load and cast implementations are missing * disabling GitHub Actions on pull_request (for this PR) * disabling AppVeyor (for this PR) * TRIGGER_SEGSEV macro, annotations for GET_STACK (vptr::get), GET_INT_STACK (pointee) * adding test_promotion_of_disowned_to_shared * Copying tests as-is from xxx_value_ptr_xxx_holder branch. https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder Systematically exercising returning and passing unique_ptr<T>, shared_ptr<T> with unique_ptr, shared_ptr holder. Observations: test_holder_unique_ptr: make_unique_pointee OK pass_unique_pointee BUILD_FAIL (as documented) make_shared_pointee Abort free(): double free detected pass_shared_pointee RuntimeError: Unable to load a custom holder type from a default-holder instance test_holder_shared_ptr: make_unique_pointee Segmentation fault (#1138) pass_unique_pointee BUILD_FAIL (as documented) make_shared_pointee OK pass_shared_pointee OK * Copying tests as-is from xxx_value_ptr_xxx_holder branch. https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder Systematically exercising casting between shared_ptr<base>, shared_ptr<derived>. * Demonstration of Undefined Behavior in handling of shared_ptr holder. Based on https://godbolt.org/z/4fdjaW by jorgbrown@ (thanks Jorg!). * Additional demonstration of Undefined Behavior in handling of shared_ptr holder. * fixing up-down mixup in comment * Demonstration of Undefined Behavior in handling of polymorphic pointers. (This demo does NOT involve smart pointers at all, unlike the otherwise similar test_smart_ptr_private_first_base.) * minor test_private_first_base.cpp simplification (after discovering that this can be wrapped with Boost.Python, using boost::noncopyable) * pybind11 equivalent of Boost.Python test similar to reproducer under #1333 * Snapshot of WIP, TODO: shared_ptr deleter with on/off switch * Adding vptr_deleter. * Adding from/as unique_ptr<T> and unique_ptr<T, D>. * Adding from_shared_ptr. Some polishing. * New tests/core/smart_holder_poc_test.cpp, using Catch2. * Adding in vptr_deleter_guard_flag. * Improved labeling of TEST_CASEs. * Shuffling existing TEST_CASEs into systematic matrix. * Implementing all [S]uccess tests. * Implementing all [E]xception tests. * Testing of exceptions not covered by the from-as matrix. * Adding top-level comment. * Converting from methods to factory functions (no functional change). * Removing obsolete and very incomplete test (replaced by Catch2-based test). * Removing stray file. * Adding type_caster_bare_interface_demo. * Adding shared_ptr<mpty>, shared_ptr<mpty const> casters. * Adding unique_ptr<mpty>, unique_ptr<mpty const> casters. * Pure copy of `class class_` implementation in pybind11.h (master commit 98f1bbb8004f654ba9e26717bdf5912fb899b05a). * classh.h: renaming of class_ to classh + namespace; forking test_classh_wip from test_type_caster_bare_interface_demo. * Hard-coding smart_holder into classh. * Adding mpty::mtxt string member. * Adding isinstance<mpty> in type_caster::load functions. * Adding rvalue_ref, renaming const_value_ref to lvalue_ref & removing const. * Retrieving smart_holder pointer in type_caster<mpty>::load, and using it cast_op operators. * Factoring out smart_holder_type_caster_load. * Retrieving smart_holder pointer in type_caster<std::shared_ptr<mpty[ const]>>::load, and using it cast_op operators. * Improved error messaging: Cannot disown nullptr (as_unique_ptr). * Retrieving smart_holder pointer in type_caster<std::unique_ptr<mpty[ const]>>::load, and using it cast_op operators. * Pure `clang-format --style=file -i` change. * Pure `clang-format --style=file -i` change, with two `clang-format off` directives. * Fixing oversight (discovered by flake8). * flake8 cleanup * Systematically setting mtxt for all rtrn_mpty_* functions (preparation, the values are not actually used yet). * static cast handle for rtrn_cptr works by simply dropping in code from type_caster_base (marked with comments). * static cast handle for rtrn_cref works by simply dropping in code from type_caster_base (marked with comments). rtrn_mref and rtrn_mptr work via const_cast (to add const). * static cast handle for rtrn_valu works by simply dropping in code from type_caster_base (marked with comments). rtrn_rref raises a RuntimeError, to be investigated. * Copying type_caster_generic::cast into type_caster<mpty> as-is (preparation for handling smart pointers). * Pure clang-format change (applied to original type_caster_generic::cast). * Adding comment re potential use_count data race. * static handle cast implementations for rtrn_shmp, rtrn_shcp. * Adding MISSING comments in operator std::unique_ptr<mpty[ const]>. * static handle cast implementations for rtrn_uqmp, rtrn_uqcp. * Bug fix: vptr_deleter_armed_flag_ptr has to live on the heap. See new bullet point in comment section near the top. The variable was also renamed to reflect its function more accurately. * Fixing bugs discovered by ASAN. The code is now ASAN, MSAN, UBSAN clean. * Making test_type_caster_bare_interface_demo.cpp slightly more realistic, ASAN, MSAN, UBSAN clean. * Calling deregister_instance after disowning via unique_ptr. * Removing enable_shared_from_this stub, simplifying existing code, clang-format. Open question, with respect to the original code: https://github.com/pybind/pybind11/blob/76a160070b369f8d82b945c97924227e8b835c94/include/pybind11/pybind11.h#L1510 To me it looks like the exact situation marked as `std::shared_ptr<Good> gp1 = not_so_good.getptr();` here: https://en.cppreference.com/w/cpp/memory/enable_shared_from_this The comment there is: `// undefined behavior (until C++17) and std::bad_weak_ptr thrown (since C++17)` Does the existing code have UB pre C++17? I'll leave handling of enable_shared_from_this for later, as the need arises. * Cosmetical change around helper functions. * Using type_caster_base<mpty>::src_and_type directly, removing copy. Also renaming one cast to cast_const_raw_ptr, for clarity. * Fixing clang-format oversight. * Using factored-out make_constructor (PR #2798), removing duplicate code. * Inserting additional assert to ensure a returned unique_ptr is always a new Python instance. * Adding minor comment (change to internals needed to distinguish uninitialized/disowned in error message). * Factoring out find_existing_python_instance(). * Moving factored-out make_constructor to test_classh_wip.cpp, restoring previous version of cast.h. This is currently the most practical approach. See PR #2798 for background. * Copying classh type_casters from test_classh_wip.cpp UNMODIFIED, as a baseline for generalizing the code. * Using pybind11/detail/classh_type_casters.h from test_classh_wip.cpp. * Adding & using PYBIND11_CLASSH_TYPE_CASTERS define. * Adding test_classh_inheritance, currently failing (passes with class_). * Removing .clang-format before git rebase master (where the file was added). * Bringing back .clang-format, the previous rm was a bad idea. * Folding in modified_type_caster_generic_load_impl, just enough to pass test_class_wip. test_classh_inheritance is still failing, but with a different error: [RuntimeError: Incompatible type (as_raw_ptr_unowned).] * Minimal changes needed to pass test_classh_inheritance. * First pass adjusting try_implicit_casts and try_load_foreign_module_local to capture loaded_v_h, but untested and guarded with pybind11_failure("Untested"). This was done mainly to determine general feasibility. Note the TODO in pybind11.h, where type_caster_generic::local_load is currently hard-coded. test_classh_wip and test_classh_inheritance still pass, as before. * Decoupling generic_type from type_caster_generic. * Changes and tests covering classh_type_casters try_implicit_casts. * Minimal test covering classh_type_casters load_impl Case 2b. * Removing stray isinstance<T>(src): it interferes with the py::module_local feature. Adding missing #includes. * Tests for classh py::module_local() feature. * Pure renaming of function names in test_classh_inheritance, similar to the systematic approach used in test_class_wip. NO functional changes. * Pure renaming of function and variable names, for better generalization when convoluting with inheritance. NO functional changes. * Adopting systematic naming scheme from test_classh_wip. NO functional changes. * Moving const after type name, for functions that cover a systematic scheme. NO functional changes. * Adding smart_holder_type_caster_load::loaded_as_shared_ptr, currently bypassing smart_holder shared_ptr tracking completely, but the tests pass and are sanitizer clean. * Removing rtti_held from smart_holder. See updated comment. * Cleaning up loaded_as_raw_ptr_unowned, loaded_as_shared_ptr. * Factoring out convert_type and folding into loaded_as_unique_ptr. * Folding convert_type into lvalue_ref and rvalue_ref paths. Some smart_holder_type_caster_load cleanup. * Using unique_ptr in local_load to replace static variable. Also adding local_load_safety_guard. * Converting test_unique_ptr_member to using classh: fully working, ASAN, MSAN, UBSAN clean. * Removing debugging comments (GET_STACK, GET_INT_STACK). cast.h is identical to current master again, pybind11.h only has the generic_type::initialize(..., &type_caster_generic::local_load) change. * Purging obsolete pybind11/vptr_holder.h and associated test. * Moving several tests to github.com/rwgk/rwgk_tbx/tree/main/pybind11_tests https://github.com/rwgk/rwgk_tbx/commit/a2c2f88174a30f5de80d7d26e0f77c7b60f5fb85 These tests are from experimenting, and for demonstrating UB in pybind11 multiple inheritance handling ("first_base"), to be fixed later. * Adding py::smart_holder support to py::class_, purging py::classh completely. * Renaming files in include directory, creating pybind11/smart_holder.h. * Renaming all "classh" to "smart_holder" in pybind11/detail/smart_holder_type_casters.h. The user-facing macro is now PYBIND11_SMART_HOLDER_TYPE_CASTERS. * Systematically renaming tests to use "class_sh" in the name. * Renaming test_type_caster_bare_interface_demo to test_type_caster_bare_interface. * Renaming new tests/core subdirectory to tests/pure_cpp. * Adding new tests to CMake config, resetting CI config. * Changing CMake file so that test_class_sh_module_local.py actually runs. * clang-tidy fixes. * 32-bit compatibility. * Reusing type_caster_base make_copy_constructor, make_move_constructor with a trick. * CMake COMPARE NATURAL is not available with older versions. * Adding copyright notices to new header files. * Explicitly define copy/move constructors/assignments. * Adding new header files to tests/extra_python_package/test_files.py. * Adding tests/pure_cpp/CMakeLists.txt. * Making use of the new find_existing_python_instance() function factored out with PR #2822. * Moving define PYBIND11_SMART_HOLDER_TYPE_CASTERS(T) down in the file. NO functional changes. Preparation for follow-up work (to keep that diff smaller). * Reintroducing py::classh, this time as a simple alias for py::class_<U, py::smart_holder>. * Replacing detail::is_smart_holder<H> in cast.h with detail::is_smart_holder_type_caster<T>. Moving get_local_load_function_ptr, init_instance_for_type to smart_holder_type_caster_class_hooks. Expanding static_assert in py::type::handle_of<> to accommodate smart_holder_type_casters. * Fixing oversight. * Adding classu alias for class_<U, std::unique_ptr<U>>. * Giving up on idea to use legacy init_instance only if is_base_of<type_caster_generic, type_caster<T>. There are use cases in the wild that define both a custom type_caster and class_. * Removing test_type_caster_bare_interface, which was moved to the separate PR #2834. * Moving up is_smart_holder_type_caster, to also use in cast_is_temporary_value_reference. * Adding smart_holder_type_casters for unique_ptr with custom deleter. SEVERE CODE DUPLICATION. This commit is to establish a baseline for consolidating the unique_ptr code. * Unification of unique_ptr, unique_ptr_with_deleter code in smart_holder_poc.h. Leads to more fitting error messages. Enables use of unique_ptr<T, D> smart_holder_type_casters also for unique_ptr<T>. * Copying files as-is from branch test_unique_ptr_member (PR #2672). * Adding comment, simplifying naming, cmake addition. * Introducing PYBIND11_USE_SMART_HOLDER_AS_DEFAULT macro (tested only undefined; there are many errors with the macro defined). * Removing test_type_caster_bare_interface, which was moved to the separate PR #2834. * Fixing oversight introduced with commit 95425f13d6c14fcb6ee479b62b602dc8a605ec49. * Setting record.default_holder correctly for PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. With this test_class.cpp builds and even mostly runs, except `test_multiple_instances_with_same_pointer`, which segfaults because it is using a `unique_ptr` holder but `smart_holder` `type_caster`. Also adding `static_assert`s to generate build errors for such situations, but guarding with `#if 0` to first pivot to test_factory_constructors.cpp. * Fixing up cast.h and smart_holder.h after rebase. * Removing detail/smart_holder_type_casters.h in separate commit. * Commenting out const in def_buffer(... const). With this, test_buffers builds and runs with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. Explanation why the const needs to be removed, or fix elsewhere, is still needed, but left for later. * Adding test_class_sh_factory_constructors, reproducing test_factory_constructors failure. Using py::class_ in this commit, to be changed to py::classh for debugging. * Removing include/pybind11/detail/smart_holder_type_casters.h from CMakeLists.txt, test_files.py (since it does not exist in this branch). * Adding // DANGER ZONE reminders. * Converting as many py::class_ to py::classh as possible, not breaking tests. * Adding initimpl::construct() overloads, resulting in test_class_sh_factory_constructors feature parity for py::class_ and py::classh. * Adding enable_if !is_smart_holder_type_caster to existing initimpl::construct(). With this test_factory_constructors.cpp builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. * Disabling shared_ptr&, shared_ptr* tests when building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT for now, pending work on smart_holder_type_caster<shared_ptr>. * Factoring out struct and class definitions into anonymous namespace. Preparation for building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. * Simplifying from_unique_ptr(): typename D = std::default_delete<T> is not needed. Factoring out is_std_default_delete<T>() for consistentcy between ensure_compatible_rtti_uqp_del() and from_unique_ptr(). * Introducing PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. Using it in test_smart_ptr.cpp. With this test_smart_ptr builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT and all but one test run successfully. * Introducing 1. type_caster_for_class_, used in PYBIND11_MAKE_OPAQUE, and 2. default_holder_type, used in stl_bind.h. * Using __VA_ARGS__ in PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. * Replacing condense_for_macro with much simpler approach. * Softening static_assert, to only check specifically that smart_holder is not mixed with type_caster_base, and unique_ptr/shared_ptr holders are not mixed with smart_holder_type_casters. * Adding PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS in test_class.cpp (with this all but one test succeed with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT). * Adding remaining PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. static_assert for "necessary conditions" for both types of default holder, static_assert for "strict conditions" guarded by new PYBIND11_STRICT_ASSERTS_CLASS_HOLDER_VS_TYPE_CASTER_MIX. All tests build & run as before with unique_ptr as the default holder, all tests build for smart_holder as the default holder, even with the strict static_assert. * Introducing check_is_smart_holder_type_caster() function for runtime check, and reinterpreting record.default_holder as "uses_unique_ptr_holder". With this test_smart_ptr succeeds. (All 42 tests build, 35 tests succeed, 5 run but have some failures, 2 segfault.) * Bug fix: Adding have_value() to smart_holder_type_caster_load. With this test_builtin_casters succeeds. (All 42 tests build, 36 tests succeed, 5 run but have some failures, 1 segfault.) * Adding unowned_void_ptr_from_direct_conversion to modified_type_caster_generic_load_impl. This fixes the last remaining segfault (test_numpy_dtypes). New stats for all tests combined: 12 failed, 458 passed. * Adding "Lazy allocation for unallocated values" (for old-style __init__) into load_value_and_holder. Deferring destruction of disowned holder until clear_instance, to remain inspectable for "uninitialized" or "disowned" detection. New stats for all tests combined: 5 failed, 465 passed. * Changing std::shared_ptr pointer/reference to const pointer/reference. New stats for all tests combined: 4 failed, 466 passed. * Adding return_value_policy::move to permissible policies for unique_ptr returns. New stats for all tests combined: 3 failed, 467 passed. * Overlooked flake8 fixes. * Manipulating failing ConstructorStats test to pass, to be able to run all tests with ASAN. This version of the code is ASAN clean with unique_ptr or smart_holder as the default. This change needs to be reverted after adopting the existing move-only-if-refcount-is-1 logic used by type_caster_base. * Adding copy constructor and move constructor tracking to atyp. Preparation for a follow-up change in smart_holder_type_caster, to make this test sensitive to the changing behavior. [skip ci] * Removing `operator T&&() &&` from smart_holder_type_caster, for compatibility with the behavior of type_caster_base. Enables reverting 2 of 3 test manipulations applied under commit 249df7cbdb09817fed0ddf80f01ba5af12466820. The manipulation in test_factory_constructors.py is NOT reverted in this commit. [skip ci] * Fixing unfortunate editing mishap. This reverts the last remaining test manipulation in commit 249df7cbdb09817fed0ddf80f01ba5af12466820 and makes all existing unit tests pass with smart_holder as default holder. * GitHub CI clang-tidy fixes. * Adding messages to terse `static_assert`s, for pre-C++17 compatibility. * Using @pytest.mark.parametrize to run each assert separately (to see all errors, not just the first). * Systematically removing _atyp from function names, to make the test code simpler. * Using re.match to accommodate variable number of intermediate MvCtor. * Also removing `operator T()` from smart_holder_type_caster, to fix gcc compilation errors. The only loss is pass_rref in test_class_sh_basic. * Systematically replacing `detail::enable_if_t<...smart_holder...>` with `typename std::enable_if<...smart_holder...>::type`. Attempt to work around MSVC 2015 issues, to be tested via GitHub CI. The idea for this change originates from this comment: https://github.com/pybind/pybind11/issues/1616#issuecomment-444536813 * Importing re before pytest after observing a PyPy CI flake when importing pytest first. * Copying MSVC 2015 compatibility change from branch pr2672_use_smart_holder_as_default. * Introducing is_smart_holder_type_caster_base_tag, to keep smart_holder code more disconnected. * Working around MSVC 2015 bug. * Expanding comment for MSVC 2015 workaround. * Systematically changing std::enable_if back to detail::enable_if_t, effectively reverting commit 5d4b6890a337ae1bbaec4091f4195606f89a3b06. * Removing unused smart_holder_type_caster_load::loaded_as_rvalue_ref (it was an oversight that it was not removed with commit 23036a45eb4731a06b488ec1fdf83bca677b7f67). * Removing py::classu, because it does not seem useful enough. * Reverting commit 63495313066119dcf7510c2ae8b468b46c12ef8f by un-commenting `const` in `def_buffer(...)`. To make this possible, `operator T const&` and `operator T const*` in `smart_holder_type_caster` need to be marked as `const` member functions. * Adding construct() overloads for constructing smart_holder from alias unique_ptr, shared_ptr returns. * Adding test_class_sh_factory_constructors.cpp to tests/CMakeLists.txt (fixes oversight, this should have been added long before). * Compatibility with old clang versions (clang 3.6, 3.7 C++11). * Cleaning up changes to existing unit tests. * Systematically adding SMART_HOLDER_WIP tag. Removing minor UNTESTED tags (only the throw are not actually exercised, investing time there has a high cost but very little benefit). * Splitting out smart_holder_type_casters again, into new detail/smart_holder_type_casters_inline_include.h. * Splitting out smart_holder_init_inline_include.h. * Adding additional new include files to CMakeLists.txt, tests/extra_python_package/test_files.py. * clang-format cleanup of most smart_holder code. * Adding source code comments in response to review. * Simple micro-benchmark ("ubench") comparing runtime performance for several holders. Tested using github.com/rwgk/pybind11_scons and Google-internal build system. Sorry, no cmake support at the moment. First results: https://docs.google.com/spreadsheets/d/1InapCYws2Gt-stmFf_Bwl33eOMo3aLE_gc9adveY7RU/edit#gid=0 * Breaking out number_bucket.h, adding hook for also collecting performance data for PyCLIF. * Accounting for ubench in MANIFEST.in (simply prune, for now). * Smarter determination of call_repetitions. [skip ci] * Also scaling performance data to PyCLIF. [skip ci] * Adding ubench/python/number_bucket.clif here for general visibility. * Fix after rebase * Merging detail/smart_holder_init_inline_include.h into detail/init.h. * Renaming detail/is_smart_holder_type_caster.h -> detail/smart_holder_sfinae_hooks_only.h. * Renaming is_smart_holder_type_caster -> type_uses_smart_holder_type_caster for clarity. * Renaming type_caster_type_is_smart_holder_type_caster -> wrapped_type_uses_smart_holder_type_caster for clarity. * Renaming is_smart_holder_type_caster_base_tag -> smart_holder_type_caster_base_tag for simplicity. * Adding copyright notices and minor colateral cleanup. * iwyu cleanup (comprehensive only for cast.h and smart_holder*.h files). * Fixing `git rebase master` accident. * Moving large `pragma warning` block from pybind11.h to detail/common.h. * Fixing another `git rebase master` accident.
2021-02-24 05:50:42 +00:00
// clang-format on
// Helpers for testing.
std::string get_mtxt(atyp const &obj) { return obj.mtxt; }
std::ptrdiff_t get_ptr(atyp const &obj) { return reinterpret_cast<std::ptrdiff_t>(&obj); }
Adding py::smart_holder (for smart-pointer interoperability). (#2672) * Adding test_unique_ptr_member (for desired PyCLIF behavior). See also: https://github.com/pybind/pybind11/issues/2583 Does not build with upstream master or https://github.com/pybind/pybind11/pull/2047, but builds with https://github.com/RobotLocomotion/pybind11 and almost runs: ``` Running tests in directory "/usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests": ================================================================================= test session starts ================================================================================= platform linux -- Python 3.8.5, pytest-5.4.3, py-1.9.0, pluggy-0.13.1 rootdir: /usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests, inifile: pytest.ini collected 2 items test_unique_ptr_member.py .F [100%] ====================================================================================== FAILURES ======================================================================================= _____________________________________________________________________________ test_pointee_and_ptr_owner ______________________________________________________________________________ def test_pointee_and_ptr_owner(): obj = m.pointee() assert obj.get_int() == 213 m.ptr_owner(obj) with pytest.raises(ValueError) as exc_info: > obj.get_int() E Failed: DID NOT RAISE <class 'ValueError'> test_unique_ptr_member.py:17: Failed ============================================================================= 1 failed, 1 passed in 0.06s ============================================================================= ``` * unique_ptr or shared_ptr return * new test_variant_unique_shared with vptr_holder prototype * moving prototype code to pybind11/vptr_holder.h, adding type_caster specialization to make the bindings involving unique_ptr passing compile, but load and cast implementations are missing * disabling GitHub Actions on pull_request (for this PR) * disabling AppVeyor (for this PR) * TRIGGER_SEGSEV macro, annotations for GET_STACK (vptr::get), GET_INT_STACK (pointee) * adding test_promotion_of_disowned_to_shared * Copying tests as-is from xxx_value_ptr_xxx_holder branch. https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder Systematically exercising returning and passing unique_ptr<T>, shared_ptr<T> with unique_ptr, shared_ptr holder. Observations: test_holder_unique_ptr: make_unique_pointee OK pass_unique_pointee BUILD_FAIL (as documented) make_shared_pointee Abort free(): double free detected pass_shared_pointee RuntimeError: Unable to load a custom holder type from a default-holder instance test_holder_shared_ptr: make_unique_pointee Segmentation fault (#1138) pass_unique_pointee BUILD_FAIL (as documented) make_shared_pointee OK pass_shared_pointee OK * Copying tests as-is from xxx_value_ptr_xxx_holder branch. https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder Systematically exercising casting between shared_ptr<base>, shared_ptr<derived>. * Demonstration of Undefined Behavior in handling of shared_ptr holder. Based on https://godbolt.org/z/4fdjaW by jorgbrown@ (thanks Jorg!). * Additional demonstration of Undefined Behavior in handling of shared_ptr holder. * fixing up-down mixup in comment * Demonstration of Undefined Behavior in handling of polymorphic pointers. (This demo does NOT involve smart pointers at all, unlike the otherwise similar test_smart_ptr_private_first_base.) * minor test_private_first_base.cpp simplification (after discovering that this can be wrapped with Boost.Python, using boost::noncopyable) * pybind11 equivalent of Boost.Python test similar to reproducer under #1333 * Snapshot of WIP, TODO: shared_ptr deleter with on/off switch * Adding vptr_deleter. * Adding from/as unique_ptr<T> and unique_ptr<T, D>. * Adding from_shared_ptr. Some polishing. * New tests/core/smart_holder_poc_test.cpp, using Catch2. * Adding in vptr_deleter_guard_flag. * Improved labeling of TEST_CASEs. * Shuffling existing TEST_CASEs into systematic matrix. * Implementing all [S]uccess tests. * Implementing all [E]xception tests. * Testing of exceptions not covered by the from-as matrix. * Adding top-level comment. * Converting from methods to factory functions (no functional change). * Removing obsolete and very incomplete test (replaced by Catch2-based test). * Removing stray file. * Adding type_caster_bare_interface_demo. * Adding shared_ptr<mpty>, shared_ptr<mpty const> casters. * Adding unique_ptr<mpty>, unique_ptr<mpty const> casters. * Pure copy of `class class_` implementation in pybind11.h (master commit 98f1bbb8004f654ba9e26717bdf5912fb899b05a). * classh.h: renaming of class_ to classh + namespace; forking test_classh_wip from test_type_caster_bare_interface_demo. * Hard-coding smart_holder into classh. * Adding mpty::mtxt string member. * Adding isinstance<mpty> in type_caster::load functions. * Adding rvalue_ref, renaming const_value_ref to lvalue_ref & removing const. * Retrieving smart_holder pointer in type_caster<mpty>::load, and using it cast_op operators. * Factoring out smart_holder_type_caster_load. * Retrieving smart_holder pointer in type_caster<std::shared_ptr<mpty[ const]>>::load, and using it cast_op operators. * Improved error messaging: Cannot disown nullptr (as_unique_ptr). * Retrieving smart_holder pointer in type_caster<std::unique_ptr<mpty[ const]>>::load, and using it cast_op operators. * Pure `clang-format --style=file -i` change. * Pure `clang-format --style=file -i` change, with two `clang-format off` directives. * Fixing oversight (discovered by flake8). * flake8 cleanup * Systematically setting mtxt for all rtrn_mpty_* functions (preparation, the values are not actually used yet). * static cast handle for rtrn_cptr works by simply dropping in code from type_caster_base (marked with comments). * static cast handle for rtrn_cref works by simply dropping in code from type_caster_base (marked with comments). rtrn_mref and rtrn_mptr work via const_cast (to add const). * static cast handle for rtrn_valu works by simply dropping in code from type_caster_base (marked with comments). rtrn_rref raises a RuntimeError, to be investigated. * Copying type_caster_generic::cast into type_caster<mpty> as-is (preparation for handling smart pointers). * Pure clang-format change (applied to original type_caster_generic::cast). * Adding comment re potential use_count data race. * static handle cast implementations for rtrn_shmp, rtrn_shcp. * Adding MISSING comments in operator std::unique_ptr<mpty[ const]>. * static handle cast implementations for rtrn_uqmp, rtrn_uqcp. * Bug fix: vptr_deleter_armed_flag_ptr has to live on the heap. See new bullet point in comment section near the top. The variable was also renamed to reflect its function more accurately. * Fixing bugs discovered by ASAN. The code is now ASAN, MSAN, UBSAN clean. * Making test_type_caster_bare_interface_demo.cpp slightly more realistic, ASAN, MSAN, UBSAN clean. * Calling deregister_instance after disowning via unique_ptr. * Removing enable_shared_from_this stub, simplifying existing code, clang-format. Open question, with respect to the original code: https://github.com/pybind/pybind11/blob/76a160070b369f8d82b945c97924227e8b835c94/include/pybind11/pybind11.h#L1510 To me it looks like the exact situation marked as `std::shared_ptr<Good> gp1 = not_so_good.getptr();` here: https://en.cppreference.com/w/cpp/memory/enable_shared_from_this The comment there is: `// undefined behavior (until C++17) and std::bad_weak_ptr thrown (since C++17)` Does the existing code have UB pre C++17? I'll leave handling of enable_shared_from_this for later, as the need arises. * Cosmetical change around helper functions. * Using type_caster_base<mpty>::src_and_type directly, removing copy. Also renaming one cast to cast_const_raw_ptr, for clarity. * Fixing clang-format oversight. * Using factored-out make_constructor (PR #2798), removing duplicate code. * Inserting additional assert to ensure a returned unique_ptr is always a new Python instance. * Adding minor comment (change to internals needed to distinguish uninitialized/disowned in error message). * Factoring out find_existing_python_instance(). * Moving factored-out make_constructor to test_classh_wip.cpp, restoring previous version of cast.h. This is currently the most practical approach. See PR #2798 for background. * Copying classh type_casters from test_classh_wip.cpp UNMODIFIED, as a baseline for generalizing the code. * Using pybind11/detail/classh_type_casters.h from test_classh_wip.cpp. * Adding & using PYBIND11_CLASSH_TYPE_CASTERS define. * Adding test_classh_inheritance, currently failing (passes with class_). * Removing .clang-format before git rebase master (where the file was added). * Bringing back .clang-format, the previous rm was a bad idea. * Folding in modified_type_caster_generic_load_impl, just enough to pass test_class_wip. test_classh_inheritance is still failing, but with a different error: [RuntimeError: Incompatible type (as_raw_ptr_unowned).] * Minimal changes needed to pass test_classh_inheritance. * First pass adjusting try_implicit_casts and try_load_foreign_module_local to capture loaded_v_h, but untested and guarded with pybind11_failure("Untested"). This was done mainly to determine general feasibility. Note the TODO in pybind11.h, where type_caster_generic::local_load is currently hard-coded. test_classh_wip and test_classh_inheritance still pass, as before. * Decoupling generic_type from type_caster_generic. * Changes and tests covering classh_type_casters try_implicit_casts. * Minimal test covering classh_type_casters load_impl Case 2b. * Removing stray isinstance<T>(src): it interferes with the py::module_local feature. Adding missing #includes. * Tests for classh py::module_local() feature. * Pure renaming of function names in test_classh_inheritance, similar to the systematic approach used in test_class_wip. NO functional changes. * Pure renaming of function and variable names, for better generalization when convoluting with inheritance. NO functional changes. * Adopting systematic naming scheme from test_classh_wip. NO functional changes. * Moving const after type name, for functions that cover a systematic scheme. NO functional changes. * Adding smart_holder_type_caster_load::loaded_as_shared_ptr, currently bypassing smart_holder shared_ptr tracking completely, but the tests pass and are sanitizer clean. * Removing rtti_held from smart_holder. See updated comment. * Cleaning up loaded_as_raw_ptr_unowned, loaded_as_shared_ptr. * Factoring out convert_type and folding into loaded_as_unique_ptr. * Folding convert_type into lvalue_ref and rvalue_ref paths. Some smart_holder_type_caster_load cleanup. * Using unique_ptr in local_load to replace static variable. Also adding local_load_safety_guard. * Converting test_unique_ptr_member to using classh: fully working, ASAN, MSAN, UBSAN clean. * Removing debugging comments (GET_STACK, GET_INT_STACK). cast.h is identical to current master again, pybind11.h only has the generic_type::initialize(..., &type_caster_generic::local_load) change. * Purging obsolete pybind11/vptr_holder.h and associated test. * Moving several tests to github.com/rwgk/rwgk_tbx/tree/main/pybind11_tests https://github.com/rwgk/rwgk_tbx/commit/a2c2f88174a30f5de80d7d26e0f77c7b60f5fb85 These tests are from experimenting, and for demonstrating UB in pybind11 multiple inheritance handling ("first_base"), to be fixed later. * Adding py::smart_holder support to py::class_, purging py::classh completely. * Renaming files in include directory, creating pybind11/smart_holder.h. * Renaming all "classh" to "smart_holder" in pybind11/detail/smart_holder_type_casters.h. The user-facing macro is now PYBIND11_SMART_HOLDER_TYPE_CASTERS. * Systematically renaming tests to use "class_sh" in the name. * Renaming test_type_caster_bare_interface_demo to test_type_caster_bare_interface. * Renaming new tests/core subdirectory to tests/pure_cpp. * Adding new tests to CMake config, resetting CI config. * Changing CMake file so that test_class_sh_module_local.py actually runs. * clang-tidy fixes. * 32-bit compatibility. * Reusing type_caster_base make_copy_constructor, make_move_constructor with a trick. * CMake COMPARE NATURAL is not available with older versions. * Adding copyright notices to new header files. * Explicitly define copy/move constructors/assignments. * Adding new header files to tests/extra_python_package/test_files.py. * Adding tests/pure_cpp/CMakeLists.txt. * Making use of the new find_existing_python_instance() function factored out with PR #2822. * Moving define PYBIND11_SMART_HOLDER_TYPE_CASTERS(T) down in the file. NO functional changes. Preparation for follow-up work (to keep that diff smaller). * Reintroducing py::classh, this time as a simple alias for py::class_<U, py::smart_holder>. * Replacing detail::is_smart_holder<H> in cast.h with detail::is_smart_holder_type_caster<T>. Moving get_local_load_function_ptr, init_instance_for_type to smart_holder_type_caster_class_hooks. Expanding static_assert in py::type::handle_of<> to accommodate smart_holder_type_casters. * Fixing oversight. * Adding classu alias for class_<U, std::unique_ptr<U>>. * Giving up on idea to use legacy init_instance only if is_base_of<type_caster_generic, type_caster<T>. There are use cases in the wild that define both a custom type_caster and class_. * Removing test_type_caster_bare_interface, which was moved to the separate PR #2834. * Moving up is_smart_holder_type_caster, to also use in cast_is_temporary_value_reference. * Adding smart_holder_type_casters for unique_ptr with custom deleter. SEVERE CODE DUPLICATION. This commit is to establish a baseline for consolidating the unique_ptr code. * Unification of unique_ptr, unique_ptr_with_deleter code in smart_holder_poc.h. Leads to more fitting error messages. Enables use of unique_ptr<T, D> smart_holder_type_casters also for unique_ptr<T>. * Copying files as-is from branch test_unique_ptr_member (PR #2672). * Adding comment, simplifying naming, cmake addition. * Introducing PYBIND11_USE_SMART_HOLDER_AS_DEFAULT macro (tested only undefined; there are many errors with the macro defined). * Removing test_type_caster_bare_interface, which was moved to the separate PR #2834. * Fixing oversight introduced with commit 95425f13d6c14fcb6ee479b62b602dc8a605ec49. * Setting record.default_holder correctly for PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. With this test_class.cpp builds and even mostly runs, except `test_multiple_instances_with_same_pointer`, which segfaults because it is using a `unique_ptr` holder but `smart_holder` `type_caster`. Also adding `static_assert`s to generate build errors for such situations, but guarding with `#if 0` to first pivot to test_factory_constructors.cpp. * Fixing up cast.h and smart_holder.h after rebase. * Removing detail/smart_holder_type_casters.h in separate commit. * Commenting out const in def_buffer(... const). With this, test_buffers builds and runs with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. Explanation why the const needs to be removed, or fix elsewhere, is still needed, but left for later. * Adding test_class_sh_factory_constructors, reproducing test_factory_constructors failure. Using py::class_ in this commit, to be changed to py::classh for debugging. * Removing include/pybind11/detail/smart_holder_type_casters.h from CMakeLists.txt, test_files.py (since it does not exist in this branch). * Adding // DANGER ZONE reminders. * Converting as many py::class_ to py::classh as possible, not breaking tests. * Adding initimpl::construct() overloads, resulting in test_class_sh_factory_constructors feature parity for py::class_ and py::classh. * Adding enable_if !is_smart_holder_type_caster to existing initimpl::construct(). With this test_factory_constructors.cpp builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. * Disabling shared_ptr&, shared_ptr* tests when building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT for now, pending work on smart_holder_type_caster<shared_ptr>. * Factoring out struct and class definitions into anonymous namespace. Preparation for building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. * Simplifying from_unique_ptr(): typename D = std::default_delete<T> is not needed. Factoring out is_std_default_delete<T>() for consistentcy between ensure_compatible_rtti_uqp_del() and from_unique_ptr(). * Introducing PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. Using it in test_smart_ptr.cpp. With this test_smart_ptr builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT and all but one test run successfully. * Introducing 1. type_caster_for_class_, used in PYBIND11_MAKE_OPAQUE, and 2. default_holder_type, used in stl_bind.h. * Using __VA_ARGS__ in PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. * Replacing condense_for_macro with much simpler approach. * Softening static_assert, to only check specifically that smart_holder is not mixed with type_caster_base, and unique_ptr/shared_ptr holders are not mixed with smart_holder_type_casters. * Adding PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS in test_class.cpp (with this all but one test succeed with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT). * Adding remaining PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. static_assert for "necessary conditions" for both types of default holder, static_assert for "strict conditions" guarded by new PYBIND11_STRICT_ASSERTS_CLASS_HOLDER_VS_TYPE_CASTER_MIX. All tests build & run as before with unique_ptr as the default holder, all tests build for smart_holder as the default holder, even with the strict static_assert. * Introducing check_is_smart_holder_type_caster() function for runtime check, and reinterpreting record.default_holder as "uses_unique_ptr_holder". With this test_smart_ptr succeeds. (All 42 tests build, 35 tests succeed, 5 run but have some failures, 2 segfault.) * Bug fix: Adding have_value() to smart_holder_type_caster_load. With this test_builtin_casters succeeds. (All 42 tests build, 36 tests succeed, 5 run but have some failures, 1 segfault.) * Adding unowned_void_ptr_from_direct_conversion to modified_type_caster_generic_load_impl. This fixes the last remaining segfault (test_numpy_dtypes). New stats for all tests combined: 12 failed, 458 passed. * Adding "Lazy allocation for unallocated values" (for old-style __init__) into load_value_and_holder. Deferring destruction of disowned holder until clear_instance, to remain inspectable for "uninitialized" or "disowned" detection. New stats for all tests combined: 5 failed, 465 passed. * Changing std::shared_ptr pointer/reference to const pointer/reference. New stats for all tests combined: 4 failed, 466 passed. * Adding return_value_policy::move to permissible policies for unique_ptr returns. New stats for all tests combined: 3 failed, 467 passed. * Overlooked flake8 fixes. * Manipulating failing ConstructorStats test to pass, to be able to run all tests with ASAN. This version of the code is ASAN clean with unique_ptr or smart_holder as the default. This change needs to be reverted after adopting the existing move-only-if-refcount-is-1 logic used by type_caster_base. * Adding copy constructor and move constructor tracking to atyp. Preparation for a follow-up change in smart_holder_type_caster, to make this test sensitive to the changing behavior. [skip ci] * Removing `operator T&&() &&` from smart_holder_type_caster, for compatibility with the behavior of type_caster_base. Enables reverting 2 of 3 test manipulations applied under commit 249df7cbdb09817fed0ddf80f01ba5af12466820. The manipulation in test_factory_constructors.py is NOT reverted in this commit. [skip ci] * Fixing unfortunate editing mishap. This reverts the last remaining test manipulation in commit 249df7cbdb09817fed0ddf80f01ba5af12466820 and makes all existing unit tests pass with smart_holder as default holder. * GitHub CI clang-tidy fixes. * Adding messages to terse `static_assert`s, for pre-C++17 compatibility. * Using @pytest.mark.parametrize to run each assert separately (to see all errors, not just the first). * Systematically removing _atyp from function names, to make the test code simpler. * Using re.match to accommodate variable number of intermediate MvCtor. * Also removing `operator T()` from smart_holder_type_caster, to fix gcc compilation errors. The only loss is pass_rref in test_class_sh_basic. * Systematically replacing `detail::enable_if_t<...smart_holder...>` with `typename std::enable_if<...smart_holder...>::type`. Attempt to work around MSVC 2015 issues, to be tested via GitHub CI. The idea for this change originates from this comment: https://github.com/pybind/pybind11/issues/1616#issuecomment-444536813 * Importing re before pytest after observing a PyPy CI flake when importing pytest first. * Copying MSVC 2015 compatibility change from branch pr2672_use_smart_holder_as_default. * Introducing is_smart_holder_type_caster_base_tag, to keep smart_holder code more disconnected. * Working around MSVC 2015 bug. * Expanding comment for MSVC 2015 workaround. * Systematically changing std::enable_if back to detail::enable_if_t, effectively reverting commit 5d4b6890a337ae1bbaec4091f4195606f89a3b06. * Removing unused smart_holder_type_caster_load::loaded_as_rvalue_ref (it was an oversight that it was not removed with commit 23036a45eb4731a06b488ec1fdf83bca677b7f67). * Removing py::classu, because it does not seem useful enough. * Reverting commit 63495313066119dcf7510c2ae8b468b46c12ef8f by un-commenting `const` in `def_buffer(...)`. To make this possible, `operator T const&` and `operator T const*` in `smart_holder_type_caster` need to be marked as `const` member functions. * Adding construct() overloads for constructing smart_holder from alias unique_ptr, shared_ptr returns. * Adding test_class_sh_factory_constructors.cpp to tests/CMakeLists.txt (fixes oversight, this should have been added long before). * Compatibility with old clang versions (clang 3.6, 3.7 C++11). * Cleaning up changes to existing unit tests. * Systematically adding SMART_HOLDER_WIP tag. Removing minor UNTESTED tags (only the throw are not actually exercised, investing time there has a high cost but very little benefit). * Splitting out smart_holder_type_casters again, into new detail/smart_holder_type_casters_inline_include.h. * Splitting out smart_holder_init_inline_include.h. * Adding additional new include files to CMakeLists.txt, tests/extra_python_package/test_files.py. * clang-format cleanup of most smart_holder code. * Adding source code comments in response to review. * Simple micro-benchmark ("ubench") comparing runtime performance for several holders. Tested using github.com/rwgk/pybind11_scons and Google-internal build system. Sorry, no cmake support at the moment. First results: https://docs.google.com/spreadsheets/d/1InapCYws2Gt-stmFf_Bwl33eOMo3aLE_gc9adveY7RU/edit#gid=0 * Breaking out number_bucket.h, adding hook for also collecting performance data for PyCLIF. * Accounting for ubench in MANIFEST.in (simply prune, for now). * Smarter determination of call_repetitions. [skip ci] * Also scaling performance data to PyCLIF. [skip ci] * Adding ubench/python/number_bucket.clif here for general visibility. * Fix after rebase * Merging detail/smart_holder_init_inline_include.h into detail/init.h. * Renaming detail/is_smart_holder_type_caster.h -> detail/smart_holder_sfinae_hooks_only.h. * Renaming is_smart_holder_type_caster -> type_uses_smart_holder_type_caster for clarity. * Renaming type_caster_type_is_smart_holder_type_caster -> wrapped_type_uses_smart_holder_type_caster for clarity. * Renaming is_smart_holder_type_caster_base_tag -> smart_holder_type_caster_base_tag for simplicity. * Adding copyright notices and minor colateral cleanup. * iwyu cleanup (comprehensive only for cast.h and smart_holder*.h files). * Fixing `git rebase master` accident. * Moving large `pragma warning` block from pybind11.h to detail/common.h. * Fixing another `git rebase master` accident.
2021-02-24 05:50:42 +00:00
std::unique_ptr<atyp> unique_ptr_roundtrip(std::unique_ptr<atyp> obj) { return obj; }
std::string pass_unique_ptr_cref(const std::unique_ptr<atyp> &obj) { return obj->mtxt; }
const std::unique_ptr<atyp> &rtrn_unique_ptr_cref(const std::string &mtxt) {
static std::unique_ptr<atyp> obj{new atyp{"static_ctor_arg"}};
if (!mtxt.empty()) {
obj->mtxt = mtxt;
}
return obj;
}
const std::unique_ptr<atyp> &unique_ptr_cref_roundtrip(const std::unique_ptr<atyp> &obj) {
return obj;
}
struct SharedPtrStash {
std::vector<std::shared_ptr<const atyp>> stash;
void Add(const std::shared_ptr<const atyp> &obj) { stash.push_back(obj); }
};
Adding py::smart_holder (for smart-pointer interoperability). (#2672) * Adding test_unique_ptr_member (for desired PyCLIF behavior). See also: https://github.com/pybind/pybind11/issues/2583 Does not build with upstream master or https://github.com/pybind/pybind11/pull/2047, but builds with https://github.com/RobotLocomotion/pybind11 and almost runs: ``` Running tests in directory "/usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests": ================================================================================= test session starts ================================================================================= platform linux -- Python 3.8.5, pytest-5.4.3, py-1.9.0, pluggy-0.13.1 rootdir: /usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests, inifile: pytest.ini collected 2 items test_unique_ptr_member.py .F [100%] ====================================================================================== FAILURES ======================================================================================= _____________________________________________________________________________ test_pointee_and_ptr_owner ______________________________________________________________________________ def test_pointee_and_ptr_owner(): obj = m.pointee() assert obj.get_int() == 213 m.ptr_owner(obj) with pytest.raises(ValueError) as exc_info: > obj.get_int() E Failed: DID NOT RAISE <class 'ValueError'> test_unique_ptr_member.py:17: Failed ============================================================================= 1 failed, 1 passed in 0.06s ============================================================================= ``` * unique_ptr or shared_ptr return * new test_variant_unique_shared with vptr_holder prototype * moving prototype code to pybind11/vptr_holder.h, adding type_caster specialization to make the bindings involving unique_ptr passing compile, but load and cast implementations are missing * disabling GitHub Actions on pull_request (for this PR) * disabling AppVeyor (for this PR) * TRIGGER_SEGSEV macro, annotations for GET_STACK (vptr::get), GET_INT_STACK (pointee) * adding test_promotion_of_disowned_to_shared * Copying tests as-is from xxx_value_ptr_xxx_holder branch. https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder Systematically exercising returning and passing unique_ptr<T>, shared_ptr<T> with unique_ptr, shared_ptr holder. Observations: test_holder_unique_ptr: make_unique_pointee OK pass_unique_pointee BUILD_FAIL (as documented) make_shared_pointee Abort free(): double free detected pass_shared_pointee RuntimeError: Unable to load a custom holder type from a default-holder instance test_holder_shared_ptr: make_unique_pointee Segmentation fault (#1138) pass_unique_pointee BUILD_FAIL (as documented) make_shared_pointee OK pass_shared_pointee OK * Copying tests as-is from xxx_value_ptr_xxx_holder branch. https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder Systematically exercising casting between shared_ptr<base>, shared_ptr<derived>. * Demonstration of Undefined Behavior in handling of shared_ptr holder. Based on https://godbolt.org/z/4fdjaW by jorgbrown@ (thanks Jorg!). * Additional demonstration of Undefined Behavior in handling of shared_ptr holder. * fixing up-down mixup in comment * Demonstration of Undefined Behavior in handling of polymorphic pointers. (This demo does NOT involve smart pointers at all, unlike the otherwise similar test_smart_ptr_private_first_base.) * minor test_private_first_base.cpp simplification (after discovering that this can be wrapped with Boost.Python, using boost::noncopyable) * pybind11 equivalent of Boost.Python test similar to reproducer under #1333 * Snapshot of WIP, TODO: shared_ptr deleter with on/off switch * Adding vptr_deleter. * Adding from/as unique_ptr<T> and unique_ptr<T, D>. * Adding from_shared_ptr. Some polishing. * New tests/core/smart_holder_poc_test.cpp, using Catch2. * Adding in vptr_deleter_guard_flag. * Improved labeling of TEST_CASEs. * Shuffling existing TEST_CASEs into systematic matrix. * Implementing all [S]uccess tests. * Implementing all [E]xception tests. * Testing of exceptions not covered by the from-as matrix. * Adding top-level comment. * Converting from methods to factory functions (no functional change). * Removing obsolete and very incomplete test (replaced by Catch2-based test). * Removing stray file. * Adding type_caster_bare_interface_demo. * Adding shared_ptr<mpty>, shared_ptr<mpty const> casters. * Adding unique_ptr<mpty>, unique_ptr<mpty const> casters. * Pure copy of `class class_` implementation in pybind11.h (master commit 98f1bbb8004f654ba9e26717bdf5912fb899b05a). * classh.h: renaming of class_ to classh + namespace; forking test_classh_wip from test_type_caster_bare_interface_demo. * Hard-coding smart_holder into classh. * Adding mpty::mtxt string member. * Adding isinstance<mpty> in type_caster::load functions. * Adding rvalue_ref, renaming const_value_ref to lvalue_ref & removing const. * Retrieving smart_holder pointer in type_caster<mpty>::load, and using it cast_op operators. * Factoring out smart_holder_type_caster_load. * Retrieving smart_holder pointer in type_caster<std::shared_ptr<mpty[ const]>>::load, and using it cast_op operators. * Improved error messaging: Cannot disown nullptr (as_unique_ptr). * Retrieving smart_holder pointer in type_caster<std::unique_ptr<mpty[ const]>>::load, and using it cast_op operators. * Pure `clang-format --style=file -i` change. * Pure `clang-format --style=file -i` change, with two `clang-format off` directives. * Fixing oversight (discovered by flake8). * flake8 cleanup * Systematically setting mtxt for all rtrn_mpty_* functions (preparation, the values are not actually used yet). * static cast handle for rtrn_cptr works by simply dropping in code from type_caster_base (marked with comments). * static cast handle for rtrn_cref works by simply dropping in code from type_caster_base (marked with comments). rtrn_mref and rtrn_mptr work via const_cast (to add const). * static cast handle for rtrn_valu works by simply dropping in code from type_caster_base (marked with comments). rtrn_rref raises a RuntimeError, to be investigated. * Copying type_caster_generic::cast into type_caster<mpty> as-is (preparation for handling smart pointers). * Pure clang-format change (applied to original type_caster_generic::cast). * Adding comment re potential use_count data race. * static handle cast implementations for rtrn_shmp, rtrn_shcp. * Adding MISSING comments in operator std::unique_ptr<mpty[ const]>. * static handle cast implementations for rtrn_uqmp, rtrn_uqcp. * Bug fix: vptr_deleter_armed_flag_ptr has to live on the heap. See new bullet point in comment section near the top. The variable was also renamed to reflect its function more accurately. * Fixing bugs discovered by ASAN. The code is now ASAN, MSAN, UBSAN clean. * Making test_type_caster_bare_interface_demo.cpp slightly more realistic, ASAN, MSAN, UBSAN clean. * Calling deregister_instance after disowning via unique_ptr. * Removing enable_shared_from_this stub, simplifying existing code, clang-format. Open question, with respect to the original code: https://github.com/pybind/pybind11/blob/76a160070b369f8d82b945c97924227e8b835c94/include/pybind11/pybind11.h#L1510 To me it looks like the exact situation marked as `std::shared_ptr<Good> gp1 = not_so_good.getptr();` here: https://en.cppreference.com/w/cpp/memory/enable_shared_from_this The comment there is: `// undefined behavior (until C++17) and std::bad_weak_ptr thrown (since C++17)` Does the existing code have UB pre C++17? I'll leave handling of enable_shared_from_this for later, as the need arises. * Cosmetical change around helper functions. * Using type_caster_base<mpty>::src_and_type directly, removing copy. Also renaming one cast to cast_const_raw_ptr, for clarity. * Fixing clang-format oversight. * Using factored-out make_constructor (PR #2798), removing duplicate code. * Inserting additional assert to ensure a returned unique_ptr is always a new Python instance. * Adding minor comment (change to internals needed to distinguish uninitialized/disowned in error message). * Factoring out find_existing_python_instance(). * Moving factored-out make_constructor to test_classh_wip.cpp, restoring previous version of cast.h. This is currently the most practical approach. See PR #2798 for background. * Copying classh type_casters from test_classh_wip.cpp UNMODIFIED, as a baseline for generalizing the code. * Using pybind11/detail/classh_type_casters.h from test_classh_wip.cpp. * Adding & using PYBIND11_CLASSH_TYPE_CASTERS define. * Adding test_classh_inheritance, currently failing (passes with class_). * Removing .clang-format before git rebase master (where the file was added). * Bringing back .clang-format, the previous rm was a bad idea. * Folding in modified_type_caster_generic_load_impl, just enough to pass test_class_wip. test_classh_inheritance is still failing, but with a different error: [RuntimeError: Incompatible type (as_raw_ptr_unowned).] * Minimal changes needed to pass test_classh_inheritance. * First pass adjusting try_implicit_casts and try_load_foreign_module_local to capture loaded_v_h, but untested and guarded with pybind11_failure("Untested"). This was done mainly to determine general feasibility. Note the TODO in pybind11.h, where type_caster_generic::local_load is currently hard-coded. test_classh_wip and test_classh_inheritance still pass, as before. * Decoupling generic_type from type_caster_generic. * Changes and tests covering classh_type_casters try_implicit_casts. * Minimal test covering classh_type_casters load_impl Case 2b. * Removing stray isinstance<T>(src): it interferes with the py::module_local feature. Adding missing #includes. * Tests for classh py::module_local() feature. * Pure renaming of function names in test_classh_inheritance, similar to the systematic approach used in test_class_wip. NO functional changes. * Pure renaming of function and variable names, for better generalization when convoluting with inheritance. NO functional changes. * Adopting systematic naming scheme from test_classh_wip. NO functional changes. * Moving const after type name, for functions that cover a systematic scheme. NO functional changes. * Adding smart_holder_type_caster_load::loaded_as_shared_ptr, currently bypassing smart_holder shared_ptr tracking completely, but the tests pass and are sanitizer clean. * Removing rtti_held from smart_holder. See updated comment. * Cleaning up loaded_as_raw_ptr_unowned, loaded_as_shared_ptr. * Factoring out convert_type and folding into loaded_as_unique_ptr. * Folding convert_type into lvalue_ref and rvalue_ref paths. Some smart_holder_type_caster_load cleanup. * Using unique_ptr in local_load to replace static variable. Also adding local_load_safety_guard. * Converting test_unique_ptr_member to using classh: fully working, ASAN, MSAN, UBSAN clean. * Removing debugging comments (GET_STACK, GET_INT_STACK). cast.h is identical to current master again, pybind11.h only has the generic_type::initialize(..., &type_caster_generic::local_load) change. * Purging obsolete pybind11/vptr_holder.h and associated test. * Moving several tests to github.com/rwgk/rwgk_tbx/tree/main/pybind11_tests https://github.com/rwgk/rwgk_tbx/commit/a2c2f88174a30f5de80d7d26e0f77c7b60f5fb85 These tests are from experimenting, and for demonstrating UB in pybind11 multiple inheritance handling ("first_base"), to be fixed later. * Adding py::smart_holder support to py::class_, purging py::classh completely. * Renaming files in include directory, creating pybind11/smart_holder.h. * Renaming all "classh" to "smart_holder" in pybind11/detail/smart_holder_type_casters.h. The user-facing macro is now PYBIND11_SMART_HOLDER_TYPE_CASTERS. * Systematically renaming tests to use "class_sh" in the name. * Renaming test_type_caster_bare_interface_demo to test_type_caster_bare_interface. * Renaming new tests/core subdirectory to tests/pure_cpp. * Adding new tests to CMake config, resetting CI config. * Changing CMake file so that test_class_sh_module_local.py actually runs. * clang-tidy fixes. * 32-bit compatibility. * Reusing type_caster_base make_copy_constructor, make_move_constructor with a trick. * CMake COMPARE NATURAL is not available with older versions. * Adding copyright notices to new header files. * Explicitly define copy/move constructors/assignments. * Adding new header files to tests/extra_python_package/test_files.py. * Adding tests/pure_cpp/CMakeLists.txt. * Making use of the new find_existing_python_instance() function factored out with PR #2822. * Moving define PYBIND11_SMART_HOLDER_TYPE_CASTERS(T) down in the file. NO functional changes. Preparation for follow-up work (to keep that diff smaller). * Reintroducing py::classh, this time as a simple alias for py::class_<U, py::smart_holder>. * Replacing detail::is_smart_holder<H> in cast.h with detail::is_smart_holder_type_caster<T>. Moving get_local_load_function_ptr, init_instance_for_type to smart_holder_type_caster_class_hooks. Expanding static_assert in py::type::handle_of<> to accommodate smart_holder_type_casters. * Fixing oversight. * Adding classu alias for class_<U, std::unique_ptr<U>>. * Giving up on idea to use legacy init_instance only if is_base_of<type_caster_generic, type_caster<T>. There are use cases in the wild that define both a custom type_caster and class_. * Removing test_type_caster_bare_interface, which was moved to the separate PR #2834. * Moving up is_smart_holder_type_caster, to also use in cast_is_temporary_value_reference. * Adding smart_holder_type_casters for unique_ptr with custom deleter. SEVERE CODE DUPLICATION. This commit is to establish a baseline for consolidating the unique_ptr code. * Unification of unique_ptr, unique_ptr_with_deleter code in smart_holder_poc.h. Leads to more fitting error messages. Enables use of unique_ptr<T, D> smart_holder_type_casters also for unique_ptr<T>. * Copying files as-is from branch test_unique_ptr_member (PR #2672). * Adding comment, simplifying naming, cmake addition. * Introducing PYBIND11_USE_SMART_HOLDER_AS_DEFAULT macro (tested only undefined; there are many errors with the macro defined). * Removing test_type_caster_bare_interface, which was moved to the separate PR #2834. * Fixing oversight introduced with commit 95425f13d6c14fcb6ee479b62b602dc8a605ec49. * Setting record.default_holder correctly for PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. With this test_class.cpp builds and even mostly runs, except `test_multiple_instances_with_same_pointer`, which segfaults because it is using a `unique_ptr` holder but `smart_holder` `type_caster`. Also adding `static_assert`s to generate build errors for such situations, but guarding with `#if 0` to first pivot to test_factory_constructors.cpp. * Fixing up cast.h and smart_holder.h after rebase. * Removing detail/smart_holder_type_casters.h in separate commit. * Commenting out const in def_buffer(... const). With this, test_buffers builds and runs with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. Explanation why the const needs to be removed, or fix elsewhere, is still needed, but left for later. * Adding test_class_sh_factory_constructors, reproducing test_factory_constructors failure. Using py::class_ in this commit, to be changed to py::classh for debugging. * Removing include/pybind11/detail/smart_holder_type_casters.h from CMakeLists.txt, test_files.py (since it does not exist in this branch). * Adding // DANGER ZONE reminders. * Converting as many py::class_ to py::classh as possible, not breaking tests. * Adding initimpl::construct() overloads, resulting in test_class_sh_factory_constructors feature parity for py::class_ and py::classh. * Adding enable_if !is_smart_holder_type_caster to existing initimpl::construct(). With this test_factory_constructors.cpp builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. * Disabling shared_ptr&, shared_ptr* tests when building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT for now, pending work on smart_holder_type_caster<shared_ptr>. * Factoring out struct and class definitions into anonymous namespace. Preparation for building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. * Simplifying from_unique_ptr(): typename D = std::default_delete<T> is not needed. Factoring out is_std_default_delete<T>() for consistentcy between ensure_compatible_rtti_uqp_del() and from_unique_ptr(). * Introducing PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. Using it in test_smart_ptr.cpp. With this test_smart_ptr builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT and all but one test run successfully. * Introducing 1. type_caster_for_class_, used in PYBIND11_MAKE_OPAQUE, and 2. default_holder_type, used in stl_bind.h. * Using __VA_ARGS__ in PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. * Replacing condense_for_macro with much simpler approach. * Softening static_assert, to only check specifically that smart_holder is not mixed with type_caster_base, and unique_ptr/shared_ptr holders are not mixed with smart_holder_type_casters. * Adding PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS in test_class.cpp (with this all but one test succeed with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT). * Adding remaining PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. static_assert for "necessary conditions" for both types of default holder, static_assert for "strict conditions" guarded by new PYBIND11_STRICT_ASSERTS_CLASS_HOLDER_VS_TYPE_CASTER_MIX. All tests build & run as before with unique_ptr as the default holder, all tests build for smart_holder as the default holder, even with the strict static_assert. * Introducing check_is_smart_holder_type_caster() function for runtime check, and reinterpreting record.default_holder as "uses_unique_ptr_holder". With this test_smart_ptr succeeds. (All 42 tests build, 35 tests succeed, 5 run but have some failures, 2 segfault.) * Bug fix: Adding have_value() to smart_holder_type_caster_load. With this test_builtin_casters succeeds. (All 42 tests build, 36 tests succeed, 5 run but have some failures, 1 segfault.) * Adding unowned_void_ptr_from_direct_conversion to modified_type_caster_generic_load_impl. This fixes the last remaining segfault (test_numpy_dtypes). New stats for all tests combined: 12 failed, 458 passed. * Adding "Lazy allocation for unallocated values" (for old-style __init__) into load_value_and_holder. Deferring destruction of disowned holder until clear_instance, to remain inspectable for "uninitialized" or "disowned" detection. New stats for all tests combined: 5 failed, 465 passed. * Changing std::shared_ptr pointer/reference to const pointer/reference. New stats for all tests combined: 4 failed, 466 passed. * Adding return_value_policy::move to permissible policies for unique_ptr returns. New stats for all tests combined: 3 failed, 467 passed. * Overlooked flake8 fixes. * Manipulating failing ConstructorStats test to pass, to be able to run all tests with ASAN. This version of the code is ASAN clean with unique_ptr or smart_holder as the default. This change needs to be reverted after adopting the existing move-only-if-refcount-is-1 logic used by type_caster_base. * Adding copy constructor and move constructor tracking to atyp. Preparation for a follow-up change in smart_holder_type_caster, to make this test sensitive to the changing behavior. [skip ci] * Removing `operator T&&() &&` from smart_holder_type_caster, for compatibility with the behavior of type_caster_base. Enables reverting 2 of 3 test manipulations applied under commit 249df7cbdb09817fed0ddf80f01ba5af12466820. The manipulation in test_factory_constructors.py is NOT reverted in this commit. [skip ci] * Fixing unfortunate editing mishap. This reverts the last remaining test manipulation in commit 249df7cbdb09817fed0ddf80f01ba5af12466820 and makes all existing unit tests pass with smart_holder as default holder. * GitHub CI clang-tidy fixes. * Adding messages to terse `static_assert`s, for pre-C++17 compatibility. * Using @pytest.mark.parametrize to run each assert separately (to see all errors, not just the first). * Systematically removing _atyp from function names, to make the test code simpler. * Using re.match to accommodate variable number of intermediate MvCtor. * Also removing `operator T()` from smart_holder_type_caster, to fix gcc compilation errors. The only loss is pass_rref in test_class_sh_basic. * Systematically replacing `detail::enable_if_t<...smart_holder...>` with `typename std::enable_if<...smart_holder...>::type`. Attempt to work around MSVC 2015 issues, to be tested via GitHub CI. The idea for this change originates from this comment: https://github.com/pybind/pybind11/issues/1616#issuecomment-444536813 * Importing re before pytest after observing a PyPy CI flake when importing pytest first. * Copying MSVC 2015 compatibility change from branch pr2672_use_smart_holder_as_default. * Introducing is_smart_holder_type_caster_base_tag, to keep smart_holder code more disconnected. * Working around MSVC 2015 bug. * Expanding comment for MSVC 2015 workaround. * Systematically changing std::enable_if back to detail::enable_if_t, effectively reverting commit 5d4b6890a337ae1bbaec4091f4195606f89a3b06. * Removing unused smart_holder_type_caster_load::loaded_as_rvalue_ref (it was an oversight that it was not removed with commit 23036a45eb4731a06b488ec1fdf83bca677b7f67). * Removing py::classu, because it does not seem useful enough. * Reverting commit 63495313066119dcf7510c2ae8b468b46c12ef8f by un-commenting `const` in `def_buffer(...)`. To make this possible, `operator T const&` and `operator T const*` in `smart_holder_type_caster` need to be marked as `const` member functions. * Adding construct() overloads for constructing smart_holder from alias unique_ptr, shared_ptr returns. * Adding test_class_sh_factory_constructors.cpp to tests/CMakeLists.txt (fixes oversight, this should have been added long before). * Compatibility with old clang versions (clang 3.6, 3.7 C++11). * Cleaning up changes to existing unit tests. * Systematically adding SMART_HOLDER_WIP tag. Removing minor UNTESTED tags (only the throw are not actually exercised, investing time there has a high cost but very little benefit). * Splitting out smart_holder_type_casters again, into new detail/smart_holder_type_casters_inline_include.h. * Splitting out smart_holder_init_inline_include.h. * Adding additional new include files to CMakeLists.txt, tests/extra_python_package/test_files.py. * clang-format cleanup of most smart_holder code. * Adding source code comments in response to review. * Simple micro-benchmark ("ubench") comparing runtime performance for several holders. Tested using github.com/rwgk/pybind11_scons and Google-internal build system. Sorry, no cmake support at the moment. First results: https://docs.google.com/spreadsheets/d/1InapCYws2Gt-stmFf_Bwl33eOMo3aLE_gc9adveY7RU/edit#gid=0 * Breaking out number_bucket.h, adding hook for also collecting performance data for PyCLIF. * Accounting for ubench in MANIFEST.in (simply prune, for now). * Smarter determination of call_repetitions. [skip ci] * Also scaling performance data to PyCLIF. [skip ci] * Adding ubench/python/number_bucket.clif here for general visibility. * Fix after rebase * Merging detail/smart_holder_init_inline_include.h into detail/init.h. * Renaming detail/is_smart_holder_type_caster.h -> detail/smart_holder_sfinae_hooks_only.h. * Renaming is_smart_holder_type_caster -> type_uses_smart_holder_type_caster for clarity. * Renaming type_caster_type_is_smart_holder_type_caster -> wrapped_type_uses_smart_holder_type_caster for clarity. * Renaming is_smart_holder_type_caster_base_tag -> smart_holder_type_caster_base_tag for simplicity. * Adding copyright notices and minor colateral cleanup. * iwyu cleanup (comprehensive only for cast.h and smart_holder*.h files). * Fixing `git rebase master` accident. * Moving large `pragma warning` block from pybind11.h to detail/common.h. * Fixing another `git rebase master` accident.
2021-02-24 05:50:42 +00:00
class LocalUnusualOpRef : UnusualOpRef {}; // To avoid clashing with `py::class_<UnusualOpRef>`.
py::object CastUnusualOpRefConstRef(const LocalUnusualOpRef &cref) { return py::cast(cref); }
py::object CastUnusualOpRefMovable(LocalUnusualOpRef &&mvbl) { return py::cast(std::move(mvbl)); }
Adding py::smart_holder (for smart-pointer interoperability). (#2672) * Adding test_unique_ptr_member (for desired PyCLIF behavior). See also: https://github.com/pybind/pybind11/issues/2583 Does not build with upstream master or https://github.com/pybind/pybind11/pull/2047, but builds with https://github.com/RobotLocomotion/pybind11 and almost runs: ``` Running tests in directory "/usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests": ================================================================================= test session starts ================================================================================= platform linux -- Python 3.8.5, pytest-5.4.3, py-1.9.0, pluggy-0.13.1 rootdir: /usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests, inifile: pytest.ini collected 2 items test_unique_ptr_member.py .F [100%] ====================================================================================== FAILURES ======================================================================================= _____________________________________________________________________________ test_pointee_and_ptr_owner ______________________________________________________________________________ def test_pointee_and_ptr_owner(): obj = m.pointee() assert obj.get_int() == 213 m.ptr_owner(obj) with pytest.raises(ValueError) as exc_info: > obj.get_int() E Failed: DID NOT RAISE <class 'ValueError'> test_unique_ptr_member.py:17: Failed ============================================================================= 1 failed, 1 passed in 0.06s ============================================================================= ``` * unique_ptr or shared_ptr return * new test_variant_unique_shared with vptr_holder prototype * moving prototype code to pybind11/vptr_holder.h, adding type_caster specialization to make the bindings involving unique_ptr passing compile, but load and cast implementations are missing * disabling GitHub Actions on pull_request (for this PR) * disabling AppVeyor (for this PR) * TRIGGER_SEGSEV macro, annotations for GET_STACK (vptr::get), GET_INT_STACK (pointee) * adding test_promotion_of_disowned_to_shared * Copying tests as-is from xxx_value_ptr_xxx_holder branch. https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder Systematically exercising returning and passing unique_ptr<T>, shared_ptr<T> with unique_ptr, shared_ptr holder. Observations: test_holder_unique_ptr: make_unique_pointee OK pass_unique_pointee BUILD_FAIL (as documented) make_shared_pointee Abort free(): double free detected pass_shared_pointee RuntimeError: Unable to load a custom holder type from a default-holder instance test_holder_shared_ptr: make_unique_pointee Segmentation fault (#1138) pass_unique_pointee BUILD_FAIL (as documented) make_shared_pointee OK pass_shared_pointee OK * Copying tests as-is from xxx_value_ptr_xxx_holder branch. https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder Systematically exercising casting between shared_ptr<base>, shared_ptr<derived>. * Demonstration of Undefined Behavior in handling of shared_ptr holder. Based on https://godbolt.org/z/4fdjaW by jorgbrown@ (thanks Jorg!). * Additional demonstration of Undefined Behavior in handling of shared_ptr holder. * fixing up-down mixup in comment * Demonstration of Undefined Behavior in handling of polymorphic pointers. (This demo does NOT involve smart pointers at all, unlike the otherwise similar test_smart_ptr_private_first_base.) * minor test_private_first_base.cpp simplification (after discovering that this can be wrapped with Boost.Python, using boost::noncopyable) * pybind11 equivalent of Boost.Python test similar to reproducer under #1333 * Snapshot of WIP, TODO: shared_ptr deleter with on/off switch * Adding vptr_deleter. * Adding from/as unique_ptr<T> and unique_ptr<T, D>. * Adding from_shared_ptr. Some polishing. * New tests/core/smart_holder_poc_test.cpp, using Catch2. * Adding in vptr_deleter_guard_flag. * Improved labeling of TEST_CASEs. * Shuffling existing TEST_CASEs into systematic matrix. * Implementing all [S]uccess tests. * Implementing all [E]xception tests. * Testing of exceptions not covered by the from-as matrix. * Adding top-level comment. * Converting from methods to factory functions (no functional change). * Removing obsolete and very incomplete test (replaced by Catch2-based test). * Removing stray file. * Adding type_caster_bare_interface_demo. * Adding shared_ptr<mpty>, shared_ptr<mpty const> casters. * Adding unique_ptr<mpty>, unique_ptr<mpty const> casters. * Pure copy of `class class_` implementation in pybind11.h (master commit 98f1bbb8004f654ba9e26717bdf5912fb899b05a). * classh.h: renaming of class_ to classh + namespace; forking test_classh_wip from test_type_caster_bare_interface_demo. * Hard-coding smart_holder into classh. * Adding mpty::mtxt string member. * Adding isinstance<mpty> in type_caster::load functions. * Adding rvalue_ref, renaming const_value_ref to lvalue_ref & removing const. * Retrieving smart_holder pointer in type_caster<mpty>::load, and using it cast_op operators. * Factoring out smart_holder_type_caster_load. * Retrieving smart_holder pointer in type_caster<std::shared_ptr<mpty[ const]>>::load, and using it cast_op operators. * Improved error messaging: Cannot disown nullptr (as_unique_ptr). * Retrieving smart_holder pointer in type_caster<std::unique_ptr<mpty[ const]>>::load, and using it cast_op operators. * Pure `clang-format --style=file -i` change. * Pure `clang-format --style=file -i` change, with two `clang-format off` directives. * Fixing oversight (discovered by flake8). * flake8 cleanup * Systematically setting mtxt for all rtrn_mpty_* functions (preparation, the values are not actually used yet). * static cast handle for rtrn_cptr works by simply dropping in code from type_caster_base (marked with comments). * static cast handle for rtrn_cref works by simply dropping in code from type_caster_base (marked with comments). rtrn_mref and rtrn_mptr work via const_cast (to add const). * static cast handle for rtrn_valu works by simply dropping in code from type_caster_base (marked with comments). rtrn_rref raises a RuntimeError, to be investigated. * Copying type_caster_generic::cast into type_caster<mpty> as-is (preparation for handling smart pointers). * Pure clang-format change (applied to original type_caster_generic::cast). * Adding comment re potential use_count data race. * static handle cast implementations for rtrn_shmp, rtrn_shcp. * Adding MISSING comments in operator std::unique_ptr<mpty[ const]>. * static handle cast implementations for rtrn_uqmp, rtrn_uqcp. * Bug fix: vptr_deleter_armed_flag_ptr has to live on the heap. See new bullet point in comment section near the top. The variable was also renamed to reflect its function more accurately. * Fixing bugs discovered by ASAN. The code is now ASAN, MSAN, UBSAN clean. * Making test_type_caster_bare_interface_demo.cpp slightly more realistic, ASAN, MSAN, UBSAN clean. * Calling deregister_instance after disowning via unique_ptr. * Removing enable_shared_from_this stub, simplifying existing code, clang-format. Open question, with respect to the original code: https://github.com/pybind/pybind11/blob/76a160070b369f8d82b945c97924227e8b835c94/include/pybind11/pybind11.h#L1510 To me it looks like the exact situation marked as `std::shared_ptr<Good> gp1 = not_so_good.getptr();` here: https://en.cppreference.com/w/cpp/memory/enable_shared_from_this The comment there is: `// undefined behavior (until C++17) and std::bad_weak_ptr thrown (since C++17)` Does the existing code have UB pre C++17? I'll leave handling of enable_shared_from_this for later, as the need arises. * Cosmetical change around helper functions. * Using type_caster_base<mpty>::src_and_type directly, removing copy. Also renaming one cast to cast_const_raw_ptr, for clarity. * Fixing clang-format oversight. * Using factored-out make_constructor (PR #2798), removing duplicate code. * Inserting additional assert to ensure a returned unique_ptr is always a new Python instance. * Adding minor comment (change to internals needed to distinguish uninitialized/disowned in error message). * Factoring out find_existing_python_instance(). * Moving factored-out make_constructor to test_classh_wip.cpp, restoring previous version of cast.h. This is currently the most practical approach. See PR #2798 for background. * Copying classh type_casters from test_classh_wip.cpp UNMODIFIED, as a baseline for generalizing the code. * Using pybind11/detail/classh_type_casters.h from test_classh_wip.cpp. * Adding & using PYBIND11_CLASSH_TYPE_CASTERS define. * Adding test_classh_inheritance, currently failing (passes with class_). * Removing .clang-format before git rebase master (where the file was added). * Bringing back .clang-format, the previous rm was a bad idea. * Folding in modified_type_caster_generic_load_impl, just enough to pass test_class_wip. test_classh_inheritance is still failing, but with a different error: [RuntimeError: Incompatible type (as_raw_ptr_unowned).] * Minimal changes needed to pass test_classh_inheritance. * First pass adjusting try_implicit_casts and try_load_foreign_module_local to capture loaded_v_h, but untested and guarded with pybind11_failure("Untested"). This was done mainly to determine general feasibility. Note the TODO in pybind11.h, where type_caster_generic::local_load is currently hard-coded. test_classh_wip and test_classh_inheritance still pass, as before. * Decoupling generic_type from type_caster_generic. * Changes and tests covering classh_type_casters try_implicit_casts. * Minimal test covering classh_type_casters load_impl Case 2b. * Removing stray isinstance<T>(src): it interferes with the py::module_local feature. Adding missing #includes. * Tests for classh py::module_local() feature. * Pure renaming of function names in test_classh_inheritance, similar to the systematic approach used in test_class_wip. NO functional changes. * Pure renaming of function and variable names, for better generalization when convoluting with inheritance. NO functional changes. * Adopting systematic naming scheme from test_classh_wip. NO functional changes. * Moving const after type name, for functions that cover a systematic scheme. NO functional changes. * Adding smart_holder_type_caster_load::loaded_as_shared_ptr, currently bypassing smart_holder shared_ptr tracking completely, but the tests pass and are sanitizer clean. * Removing rtti_held from smart_holder. See updated comment. * Cleaning up loaded_as_raw_ptr_unowned, loaded_as_shared_ptr. * Factoring out convert_type and folding into loaded_as_unique_ptr. * Folding convert_type into lvalue_ref and rvalue_ref paths. Some smart_holder_type_caster_load cleanup. * Using unique_ptr in local_load to replace static variable. Also adding local_load_safety_guard. * Converting test_unique_ptr_member to using classh: fully working, ASAN, MSAN, UBSAN clean. * Removing debugging comments (GET_STACK, GET_INT_STACK). cast.h is identical to current master again, pybind11.h only has the generic_type::initialize(..., &type_caster_generic::local_load) change. * Purging obsolete pybind11/vptr_holder.h and associated test. * Moving several tests to github.com/rwgk/rwgk_tbx/tree/main/pybind11_tests https://github.com/rwgk/rwgk_tbx/commit/a2c2f88174a30f5de80d7d26e0f77c7b60f5fb85 These tests are from experimenting, and for demonstrating UB in pybind11 multiple inheritance handling ("first_base"), to be fixed later. * Adding py::smart_holder support to py::class_, purging py::classh completely. * Renaming files in include directory, creating pybind11/smart_holder.h. * Renaming all "classh" to "smart_holder" in pybind11/detail/smart_holder_type_casters.h. The user-facing macro is now PYBIND11_SMART_HOLDER_TYPE_CASTERS. * Systematically renaming tests to use "class_sh" in the name. * Renaming test_type_caster_bare_interface_demo to test_type_caster_bare_interface. * Renaming new tests/core subdirectory to tests/pure_cpp. * Adding new tests to CMake config, resetting CI config. * Changing CMake file so that test_class_sh_module_local.py actually runs. * clang-tidy fixes. * 32-bit compatibility. * Reusing type_caster_base make_copy_constructor, make_move_constructor with a trick. * CMake COMPARE NATURAL is not available with older versions. * Adding copyright notices to new header files. * Explicitly define copy/move constructors/assignments. * Adding new header files to tests/extra_python_package/test_files.py. * Adding tests/pure_cpp/CMakeLists.txt. * Making use of the new find_existing_python_instance() function factored out with PR #2822. * Moving define PYBIND11_SMART_HOLDER_TYPE_CASTERS(T) down in the file. NO functional changes. Preparation for follow-up work (to keep that diff smaller). * Reintroducing py::classh, this time as a simple alias for py::class_<U, py::smart_holder>. * Replacing detail::is_smart_holder<H> in cast.h with detail::is_smart_holder_type_caster<T>. Moving get_local_load_function_ptr, init_instance_for_type to smart_holder_type_caster_class_hooks. Expanding static_assert in py::type::handle_of<> to accommodate smart_holder_type_casters. * Fixing oversight. * Adding classu alias for class_<U, std::unique_ptr<U>>. * Giving up on idea to use legacy init_instance only if is_base_of<type_caster_generic, type_caster<T>. There are use cases in the wild that define both a custom type_caster and class_. * Removing test_type_caster_bare_interface, which was moved to the separate PR #2834. * Moving up is_smart_holder_type_caster, to also use in cast_is_temporary_value_reference. * Adding smart_holder_type_casters for unique_ptr with custom deleter. SEVERE CODE DUPLICATION. This commit is to establish a baseline for consolidating the unique_ptr code. * Unification of unique_ptr, unique_ptr_with_deleter code in smart_holder_poc.h. Leads to more fitting error messages. Enables use of unique_ptr<T, D> smart_holder_type_casters also for unique_ptr<T>. * Copying files as-is from branch test_unique_ptr_member (PR #2672). * Adding comment, simplifying naming, cmake addition. * Introducing PYBIND11_USE_SMART_HOLDER_AS_DEFAULT macro (tested only undefined; there are many errors with the macro defined). * Removing test_type_caster_bare_interface, which was moved to the separate PR #2834. * Fixing oversight introduced with commit 95425f13d6c14fcb6ee479b62b602dc8a605ec49. * Setting record.default_holder correctly for PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. With this test_class.cpp builds and even mostly runs, except `test_multiple_instances_with_same_pointer`, which segfaults because it is using a `unique_ptr` holder but `smart_holder` `type_caster`. Also adding `static_assert`s to generate build errors for such situations, but guarding with `#if 0` to first pivot to test_factory_constructors.cpp. * Fixing up cast.h and smart_holder.h after rebase. * Removing detail/smart_holder_type_casters.h in separate commit. * Commenting out const in def_buffer(... const). With this, test_buffers builds and runs with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. Explanation why the const needs to be removed, or fix elsewhere, is still needed, but left for later. * Adding test_class_sh_factory_constructors, reproducing test_factory_constructors failure. Using py::class_ in this commit, to be changed to py::classh for debugging. * Removing include/pybind11/detail/smart_holder_type_casters.h from CMakeLists.txt, test_files.py (since it does not exist in this branch). * Adding // DANGER ZONE reminders. * Converting as many py::class_ to py::classh as possible, not breaking tests. * Adding initimpl::construct() overloads, resulting in test_class_sh_factory_constructors feature parity for py::class_ and py::classh. * Adding enable_if !is_smart_holder_type_caster to existing initimpl::construct(). With this test_factory_constructors.cpp builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. * Disabling shared_ptr&, shared_ptr* tests when building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT for now, pending work on smart_holder_type_caster<shared_ptr>. * Factoring out struct and class definitions into anonymous namespace. Preparation for building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. * Simplifying from_unique_ptr(): typename D = std::default_delete<T> is not needed. Factoring out is_std_default_delete<T>() for consistentcy between ensure_compatible_rtti_uqp_del() and from_unique_ptr(). * Introducing PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. Using it in test_smart_ptr.cpp. With this test_smart_ptr builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT and all but one test run successfully. * Introducing 1. type_caster_for_class_, used in PYBIND11_MAKE_OPAQUE, and 2. default_holder_type, used in stl_bind.h. * Using __VA_ARGS__ in PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. * Replacing condense_for_macro with much simpler approach. * Softening static_assert, to only check specifically that smart_holder is not mixed with type_caster_base, and unique_ptr/shared_ptr holders are not mixed with smart_holder_type_casters. * Adding PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS in test_class.cpp (with this all but one test succeed with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT). * Adding remaining PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. static_assert for "necessary conditions" for both types of default holder, static_assert for "strict conditions" guarded by new PYBIND11_STRICT_ASSERTS_CLASS_HOLDER_VS_TYPE_CASTER_MIX. All tests build & run as before with unique_ptr as the default holder, all tests build for smart_holder as the default holder, even with the strict static_assert. * Introducing check_is_smart_holder_type_caster() function for runtime check, and reinterpreting record.default_holder as "uses_unique_ptr_holder". With this test_smart_ptr succeeds. (All 42 tests build, 35 tests succeed, 5 run but have some failures, 2 segfault.) * Bug fix: Adding have_value() to smart_holder_type_caster_load. With this test_builtin_casters succeeds. (All 42 tests build, 36 tests succeed, 5 run but have some failures, 1 segfault.) * Adding unowned_void_ptr_from_direct_conversion to modified_type_caster_generic_load_impl. This fixes the last remaining segfault (test_numpy_dtypes). New stats for all tests combined: 12 failed, 458 passed. * Adding "Lazy allocation for unallocated values" (for old-style __init__) into load_value_and_holder. Deferring destruction of disowned holder until clear_instance, to remain inspectable for "uninitialized" or "disowned" detection. New stats for all tests combined: 5 failed, 465 passed. * Changing std::shared_ptr pointer/reference to const pointer/reference. New stats for all tests combined: 4 failed, 466 passed. * Adding return_value_policy::move to permissible policies for unique_ptr returns. New stats for all tests combined: 3 failed, 467 passed. * Overlooked flake8 fixes. * Manipulating failing ConstructorStats test to pass, to be able to run all tests with ASAN. This version of the code is ASAN clean with unique_ptr or smart_holder as the default. This change needs to be reverted after adopting the existing move-only-if-refcount-is-1 logic used by type_caster_base. * Adding copy constructor and move constructor tracking to atyp. Preparation for a follow-up change in smart_holder_type_caster, to make this test sensitive to the changing behavior. [skip ci] * Removing `operator T&&() &&` from smart_holder_type_caster, for compatibility with the behavior of type_caster_base. Enables reverting 2 of 3 test manipulations applied under commit 249df7cbdb09817fed0ddf80f01ba5af12466820. The manipulation in test_factory_constructors.py is NOT reverted in this commit. [skip ci] * Fixing unfortunate editing mishap. This reverts the last remaining test manipulation in commit 249df7cbdb09817fed0ddf80f01ba5af12466820 and makes all existing unit tests pass with smart_holder as default holder. * GitHub CI clang-tidy fixes. * Adding messages to terse `static_assert`s, for pre-C++17 compatibility. * Using @pytest.mark.parametrize to run each assert separately (to see all errors, not just the first). * Systematically removing _atyp from function names, to make the test code simpler. * Using re.match to accommodate variable number of intermediate MvCtor. * Also removing `operator T()` from smart_holder_type_caster, to fix gcc compilation errors. The only loss is pass_rref in test_class_sh_basic. * Systematically replacing `detail::enable_if_t<...smart_holder...>` with `typename std::enable_if<...smart_holder...>::type`. Attempt to work around MSVC 2015 issues, to be tested via GitHub CI. The idea for this change originates from this comment: https://github.com/pybind/pybind11/issues/1616#issuecomment-444536813 * Importing re before pytest after observing a PyPy CI flake when importing pytest first. * Copying MSVC 2015 compatibility change from branch pr2672_use_smart_holder_as_default. * Introducing is_smart_holder_type_caster_base_tag, to keep smart_holder code more disconnected. * Working around MSVC 2015 bug. * Expanding comment for MSVC 2015 workaround. * Systematically changing std::enable_if back to detail::enable_if_t, effectively reverting commit 5d4b6890a337ae1bbaec4091f4195606f89a3b06. * Removing unused smart_holder_type_caster_load::loaded_as_rvalue_ref (it was an oversight that it was not removed with commit 23036a45eb4731a06b488ec1fdf83bca677b7f67). * Removing py::classu, because it does not seem useful enough. * Reverting commit 63495313066119dcf7510c2ae8b468b46c12ef8f by un-commenting `const` in `def_buffer(...)`. To make this possible, `operator T const&` and `operator T const*` in `smart_holder_type_caster` need to be marked as `const` member functions. * Adding construct() overloads for constructing smart_holder from alias unique_ptr, shared_ptr returns. * Adding test_class_sh_factory_constructors.cpp to tests/CMakeLists.txt (fixes oversight, this should have been added long before). * Compatibility with old clang versions (clang 3.6, 3.7 C++11). * Cleaning up changes to existing unit tests. * Systematically adding SMART_HOLDER_WIP tag. Removing minor UNTESTED tags (only the throw are not actually exercised, investing time there has a high cost but very little benefit). * Splitting out smart_holder_type_casters again, into new detail/smart_holder_type_casters_inline_include.h. * Splitting out smart_holder_init_inline_include.h. * Adding additional new include files to CMakeLists.txt, tests/extra_python_package/test_files.py. * clang-format cleanup of most smart_holder code. * Adding source code comments in response to review. * Simple micro-benchmark ("ubench") comparing runtime performance for several holders. Tested using github.com/rwgk/pybind11_scons and Google-internal build system. Sorry, no cmake support at the moment. First results: https://docs.google.com/spreadsheets/d/1InapCYws2Gt-stmFf_Bwl33eOMo3aLE_gc9adveY7RU/edit#gid=0 * Breaking out number_bucket.h, adding hook for also collecting performance data for PyCLIF. * Accounting for ubench in MANIFEST.in (simply prune, for now). * Smarter determination of call_repetitions. [skip ci] * Also scaling performance data to PyCLIF. [skip ci] * Adding ubench/python/number_bucket.clif here for general visibility. * Fix after rebase * Merging detail/smart_holder_init_inline_include.h into detail/init.h. * Renaming detail/is_smart_holder_type_caster.h -> detail/smart_holder_sfinae_hooks_only.h. * Renaming is_smart_holder_type_caster -> type_uses_smart_holder_type_caster for clarity. * Renaming type_caster_type_is_smart_holder_type_caster -> wrapped_type_uses_smart_holder_type_caster for clarity. * Renaming is_smart_holder_type_caster_base_tag -> smart_holder_type_caster_base_tag for simplicity. * Adding copyright notices and minor colateral cleanup. * iwyu cleanup (comprehensive only for cast.h and smart_holder*.h files). * Fixing `git rebase master` accident. * Moving large `pragma warning` block from pybind11.h to detail/common.h. * Fixing another `git rebase master` accident.
2021-02-24 05:50:42 +00:00
} // namespace class_sh_basic
} // namespace pybind11_tests
PYBIND11_SMART_HOLDER_TYPE_CASTERS(pybind11_tests::class_sh_basic::atyp)
PYBIND11_SMART_HOLDER_TYPE_CASTERS(pybind11_tests::class_sh_basic::uconsumer)
PYBIND11_SMART_HOLDER_TYPE_CASTERS(pybind11_tests::class_sh_basic::SharedPtrStash)
PYBIND11_SMART_HOLDER_TYPE_CASTERS(pybind11_tests::class_sh_basic::LocalUnusualOpRef)
Adding py::smart_holder (for smart-pointer interoperability). (#2672) * Adding test_unique_ptr_member (for desired PyCLIF behavior). See also: https://github.com/pybind/pybind11/issues/2583 Does not build with upstream master or https://github.com/pybind/pybind11/pull/2047, but builds with https://github.com/RobotLocomotion/pybind11 and almost runs: ``` Running tests in directory "/usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests": ================================================================================= test session starts ================================================================================= platform linux -- Python 3.8.5, pytest-5.4.3, py-1.9.0, pluggy-0.13.1 rootdir: /usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests, inifile: pytest.ini collected 2 items test_unique_ptr_member.py .F [100%] ====================================================================================== FAILURES ======================================================================================= _____________________________________________________________________________ test_pointee_and_ptr_owner ______________________________________________________________________________ def test_pointee_and_ptr_owner(): obj = m.pointee() assert obj.get_int() == 213 m.ptr_owner(obj) with pytest.raises(ValueError) as exc_info: > obj.get_int() E Failed: DID NOT RAISE <class 'ValueError'> test_unique_ptr_member.py:17: Failed ============================================================================= 1 failed, 1 passed in 0.06s ============================================================================= ``` * unique_ptr or shared_ptr return * new test_variant_unique_shared with vptr_holder prototype * moving prototype code to pybind11/vptr_holder.h, adding type_caster specialization to make the bindings involving unique_ptr passing compile, but load and cast implementations are missing * disabling GitHub Actions on pull_request (for this PR) * disabling AppVeyor (for this PR) * TRIGGER_SEGSEV macro, annotations for GET_STACK (vptr::get), GET_INT_STACK (pointee) * adding test_promotion_of_disowned_to_shared * Copying tests as-is from xxx_value_ptr_xxx_holder branch. https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder Systematically exercising returning and passing unique_ptr<T>, shared_ptr<T> with unique_ptr, shared_ptr holder. Observations: test_holder_unique_ptr: make_unique_pointee OK pass_unique_pointee BUILD_FAIL (as documented) make_shared_pointee Abort free(): double free detected pass_shared_pointee RuntimeError: Unable to load a custom holder type from a default-holder instance test_holder_shared_ptr: make_unique_pointee Segmentation fault (#1138) pass_unique_pointee BUILD_FAIL (as documented) make_shared_pointee OK pass_shared_pointee OK * Copying tests as-is from xxx_value_ptr_xxx_holder branch. https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder Systematically exercising casting between shared_ptr<base>, shared_ptr<derived>. * Demonstration of Undefined Behavior in handling of shared_ptr holder. Based on https://godbolt.org/z/4fdjaW by jorgbrown@ (thanks Jorg!). * Additional demonstration of Undefined Behavior in handling of shared_ptr holder. * fixing up-down mixup in comment * Demonstration of Undefined Behavior in handling of polymorphic pointers. (This demo does NOT involve smart pointers at all, unlike the otherwise similar test_smart_ptr_private_first_base.) * minor test_private_first_base.cpp simplification (after discovering that this can be wrapped with Boost.Python, using boost::noncopyable) * pybind11 equivalent of Boost.Python test similar to reproducer under #1333 * Snapshot of WIP, TODO: shared_ptr deleter with on/off switch * Adding vptr_deleter. * Adding from/as unique_ptr<T> and unique_ptr<T, D>. * Adding from_shared_ptr. Some polishing. * New tests/core/smart_holder_poc_test.cpp, using Catch2. * Adding in vptr_deleter_guard_flag. * Improved labeling of TEST_CASEs. * Shuffling existing TEST_CASEs into systematic matrix. * Implementing all [S]uccess tests. * Implementing all [E]xception tests. * Testing of exceptions not covered by the from-as matrix. * Adding top-level comment. * Converting from methods to factory functions (no functional change). * Removing obsolete and very incomplete test (replaced by Catch2-based test). * Removing stray file. * Adding type_caster_bare_interface_demo. * Adding shared_ptr<mpty>, shared_ptr<mpty const> casters. * Adding unique_ptr<mpty>, unique_ptr<mpty const> casters. * Pure copy of `class class_` implementation in pybind11.h (master commit 98f1bbb8004f654ba9e26717bdf5912fb899b05a). * classh.h: renaming of class_ to classh + namespace; forking test_classh_wip from test_type_caster_bare_interface_demo. * Hard-coding smart_holder into classh. * Adding mpty::mtxt string member. * Adding isinstance<mpty> in type_caster::load functions. * Adding rvalue_ref, renaming const_value_ref to lvalue_ref & removing const. * Retrieving smart_holder pointer in type_caster<mpty>::load, and using it cast_op operators. * Factoring out smart_holder_type_caster_load. * Retrieving smart_holder pointer in type_caster<std::shared_ptr<mpty[ const]>>::load, and using it cast_op operators. * Improved error messaging: Cannot disown nullptr (as_unique_ptr). * Retrieving smart_holder pointer in type_caster<std::unique_ptr<mpty[ const]>>::load, and using it cast_op operators. * Pure `clang-format --style=file -i` change. * Pure `clang-format --style=file -i` change, with two `clang-format off` directives. * Fixing oversight (discovered by flake8). * flake8 cleanup * Systematically setting mtxt for all rtrn_mpty_* functions (preparation, the values are not actually used yet). * static cast handle for rtrn_cptr works by simply dropping in code from type_caster_base (marked with comments). * static cast handle for rtrn_cref works by simply dropping in code from type_caster_base (marked with comments). rtrn_mref and rtrn_mptr work via const_cast (to add const). * static cast handle for rtrn_valu works by simply dropping in code from type_caster_base (marked with comments). rtrn_rref raises a RuntimeError, to be investigated. * Copying type_caster_generic::cast into type_caster<mpty> as-is (preparation for handling smart pointers). * Pure clang-format change (applied to original type_caster_generic::cast). * Adding comment re potential use_count data race. * static handle cast implementations for rtrn_shmp, rtrn_shcp. * Adding MISSING comments in operator std::unique_ptr<mpty[ const]>. * static handle cast implementations for rtrn_uqmp, rtrn_uqcp. * Bug fix: vptr_deleter_armed_flag_ptr has to live on the heap. See new bullet point in comment section near the top. The variable was also renamed to reflect its function more accurately. * Fixing bugs discovered by ASAN. The code is now ASAN, MSAN, UBSAN clean. * Making test_type_caster_bare_interface_demo.cpp slightly more realistic, ASAN, MSAN, UBSAN clean. * Calling deregister_instance after disowning via unique_ptr. * Removing enable_shared_from_this stub, simplifying existing code, clang-format. Open question, with respect to the original code: https://github.com/pybind/pybind11/blob/76a160070b369f8d82b945c97924227e8b835c94/include/pybind11/pybind11.h#L1510 To me it looks like the exact situation marked as `std::shared_ptr<Good> gp1 = not_so_good.getptr();` here: https://en.cppreference.com/w/cpp/memory/enable_shared_from_this The comment there is: `// undefined behavior (until C++17) and std::bad_weak_ptr thrown (since C++17)` Does the existing code have UB pre C++17? I'll leave handling of enable_shared_from_this for later, as the need arises. * Cosmetical change around helper functions. * Using type_caster_base<mpty>::src_and_type directly, removing copy. Also renaming one cast to cast_const_raw_ptr, for clarity. * Fixing clang-format oversight. * Using factored-out make_constructor (PR #2798), removing duplicate code. * Inserting additional assert to ensure a returned unique_ptr is always a new Python instance. * Adding minor comment (change to internals needed to distinguish uninitialized/disowned in error message). * Factoring out find_existing_python_instance(). * Moving factored-out make_constructor to test_classh_wip.cpp, restoring previous version of cast.h. This is currently the most practical approach. See PR #2798 for background. * Copying classh type_casters from test_classh_wip.cpp UNMODIFIED, as a baseline for generalizing the code. * Using pybind11/detail/classh_type_casters.h from test_classh_wip.cpp. * Adding & using PYBIND11_CLASSH_TYPE_CASTERS define. * Adding test_classh_inheritance, currently failing (passes with class_). * Removing .clang-format before git rebase master (where the file was added). * Bringing back .clang-format, the previous rm was a bad idea. * Folding in modified_type_caster_generic_load_impl, just enough to pass test_class_wip. test_classh_inheritance is still failing, but with a different error: [RuntimeError: Incompatible type (as_raw_ptr_unowned).] * Minimal changes needed to pass test_classh_inheritance. * First pass adjusting try_implicit_casts and try_load_foreign_module_local to capture loaded_v_h, but untested and guarded with pybind11_failure("Untested"). This was done mainly to determine general feasibility. Note the TODO in pybind11.h, where type_caster_generic::local_load is currently hard-coded. test_classh_wip and test_classh_inheritance still pass, as before. * Decoupling generic_type from type_caster_generic. * Changes and tests covering classh_type_casters try_implicit_casts. * Minimal test covering classh_type_casters load_impl Case 2b. * Removing stray isinstance<T>(src): it interferes with the py::module_local feature. Adding missing #includes. * Tests for classh py::module_local() feature. * Pure renaming of function names in test_classh_inheritance, similar to the systematic approach used in test_class_wip. NO functional changes. * Pure renaming of function and variable names, for better generalization when convoluting with inheritance. NO functional changes. * Adopting systematic naming scheme from test_classh_wip. NO functional changes. * Moving const after type name, for functions that cover a systematic scheme. NO functional changes. * Adding smart_holder_type_caster_load::loaded_as_shared_ptr, currently bypassing smart_holder shared_ptr tracking completely, but the tests pass and are sanitizer clean. * Removing rtti_held from smart_holder. See updated comment. * Cleaning up loaded_as_raw_ptr_unowned, loaded_as_shared_ptr. * Factoring out convert_type and folding into loaded_as_unique_ptr. * Folding convert_type into lvalue_ref and rvalue_ref paths. Some smart_holder_type_caster_load cleanup. * Using unique_ptr in local_load to replace static variable. Also adding local_load_safety_guard. * Converting test_unique_ptr_member to using classh: fully working, ASAN, MSAN, UBSAN clean. * Removing debugging comments (GET_STACK, GET_INT_STACK). cast.h is identical to current master again, pybind11.h only has the generic_type::initialize(..., &type_caster_generic::local_load) change. * Purging obsolete pybind11/vptr_holder.h and associated test. * Moving several tests to github.com/rwgk/rwgk_tbx/tree/main/pybind11_tests https://github.com/rwgk/rwgk_tbx/commit/a2c2f88174a30f5de80d7d26e0f77c7b60f5fb85 These tests are from experimenting, and for demonstrating UB in pybind11 multiple inheritance handling ("first_base"), to be fixed later. * Adding py::smart_holder support to py::class_, purging py::classh completely. * Renaming files in include directory, creating pybind11/smart_holder.h. * Renaming all "classh" to "smart_holder" in pybind11/detail/smart_holder_type_casters.h. The user-facing macro is now PYBIND11_SMART_HOLDER_TYPE_CASTERS. * Systematically renaming tests to use "class_sh" in the name. * Renaming test_type_caster_bare_interface_demo to test_type_caster_bare_interface. * Renaming new tests/core subdirectory to tests/pure_cpp. * Adding new tests to CMake config, resetting CI config. * Changing CMake file so that test_class_sh_module_local.py actually runs. * clang-tidy fixes. * 32-bit compatibility. * Reusing type_caster_base make_copy_constructor, make_move_constructor with a trick. * CMake COMPARE NATURAL is not available with older versions. * Adding copyright notices to new header files. * Explicitly define copy/move constructors/assignments. * Adding new header files to tests/extra_python_package/test_files.py. * Adding tests/pure_cpp/CMakeLists.txt. * Making use of the new find_existing_python_instance() function factored out with PR #2822. * Moving define PYBIND11_SMART_HOLDER_TYPE_CASTERS(T) down in the file. NO functional changes. Preparation for follow-up work (to keep that diff smaller). * Reintroducing py::classh, this time as a simple alias for py::class_<U, py::smart_holder>. * Replacing detail::is_smart_holder<H> in cast.h with detail::is_smart_holder_type_caster<T>. Moving get_local_load_function_ptr, init_instance_for_type to smart_holder_type_caster_class_hooks. Expanding static_assert in py::type::handle_of<> to accommodate smart_holder_type_casters. * Fixing oversight. * Adding classu alias for class_<U, std::unique_ptr<U>>. * Giving up on idea to use legacy init_instance only if is_base_of<type_caster_generic, type_caster<T>. There are use cases in the wild that define both a custom type_caster and class_. * Removing test_type_caster_bare_interface, which was moved to the separate PR #2834. * Moving up is_smart_holder_type_caster, to also use in cast_is_temporary_value_reference. * Adding smart_holder_type_casters for unique_ptr with custom deleter. SEVERE CODE DUPLICATION. This commit is to establish a baseline for consolidating the unique_ptr code. * Unification of unique_ptr, unique_ptr_with_deleter code in smart_holder_poc.h. Leads to more fitting error messages. Enables use of unique_ptr<T, D> smart_holder_type_casters also for unique_ptr<T>. * Copying files as-is from branch test_unique_ptr_member (PR #2672). * Adding comment, simplifying naming, cmake addition. * Introducing PYBIND11_USE_SMART_HOLDER_AS_DEFAULT macro (tested only undefined; there are many errors with the macro defined). * Removing test_type_caster_bare_interface, which was moved to the separate PR #2834. * Fixing oversight introduced with commit 95425f13d6c14fcb6ee479b62b602dc8a605ec49. * Setting record.default_holder correctly for PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. With this test_class.cpp builds and even mostly runs, except `test_multiple_instances_with_same_pointer`, which segfaults because it is using a `unique_ptr` holder but `smart_holder` `type_caster`. Also adding `static_assert`s to generate build errors for such situations, but guarding with `#if 0` to first pivot to test_factory_constructors.cpp. * Fixing up cast.h and smart_holder.h after rebase. * Removing detail/smart_holder_type_casters.h in separate commit. * Commenting out const in def_buffer(... const). With this, test_buffers builds and runs with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. Explanation why the const needs to be removed, or fix elsewhere, is still needed, but left for later. * Adding test_class_sh_factory_constructors, reproducing test_factory_constructors failure. Using py::class_ in this commit, to be changed to py::classh for debugging. * Removing include/pybind11/detail/smart_holder_type_casters.h from CMakeLists.txt, test_files.py (since it does not exist in this branch). * Adding // DANGER ZONE reminders. * Converting as many py::class_ to py::classh as possible, not breaking tests. * Adding initimpl::construct() overloads, resulting in test_class_sh_factory_constructors feature parity for py::class_ and py::classh. * Adding enable_if !is_smart_holder_type_caster to existing initimpl::construct(). With this test_factory_constructors.cpp builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. * Disabling shared_ptr&, shared_ptr* tests when building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT for now, pending work on smart_holder_type_caster<shared_ptr>. * Factoring out struct and class definitions into anonymous namespace. Preparation for building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. * Simplifying from_unique_ptr(): typename D = std::default_delete<T> is not needed. Factoring out is_std_default_delete<T>() for consistentcy between ensure_compatible_rtti_uqp_del() and from_unique_ptr(). * Introducing PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. Using it in test_smart_ptr.cpp. With this test_smart_ptr builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT and all but one test run successfully. * Introducing 1. type_caster_for_class_, used in PYBIND11_MAKE_OPAQUE, and 2. default_holder_type, used in stl_bind.h. * Using __VA_ARGS__ in PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. * Replacing condense_for_macro with much simpler approach. * Softening static_assert, to only check specifically that smart_holder is not mixed with type_caster_base, and unique_ptr/shared_ptr holders are not mixed with smart_holder_type_casters. * Adding PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS in test_class.cpp (with this all but one test succeed with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT). * Adding remaining PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. static_assert for "necessary conditions" for both types of default holder, static_assert for "strict conditions" guarded by new PYBIND11_STRICT_ASSERTS_CLASS_HOLDER_VS_TYPE_CASTER_MIX. All tests build & run as before with unique_ptr as the default holder, all tests build for smart_holder as the default holder, even with the strict static_assert. * Introducing check_is_smart_holder_type_caster() function for runtime check, and reinterpreting record.default_holder as "uses_unique_ptr_holder". With this test_smart_ptr succeeds. (All 42 tests build, 35 tests succeed, 5 run but have some failures, 2 segfault.) * Bug fix: Adding have_value() to smart_holder_type_caster_load. With this test_builtin_casters succeeds. (All 42 tests build, 36 tests succeed, 5 run but have some failures, 1 segfault.) * Adding unowned_void_ptr_from_direct_conversion to modified_type_caster_generic_load_impl. This fixes the last remaining segfault (test_numpy_dtypes). New stats for all tests combined: 12 failed, 458 passed. * Adding "Lazy allocation for unallocated values" (for old-style __init__) into load_value_and_holder. Deferring destruction of disowned holder until clear_instance, to remain inspectable for "uninitialized" or "disowned" detection. New stats for all tests combined: 5 failed, 465 passed. * Changing std::shared_ptr pointer/reference to const pointer/reference. New stats for all tests combined: 4 failed, 466 passed. * Adding return_value_policy::move to permissible policies for unique_ptr returns. New stats for all tests combined: 3 failed, 467 passed. * Overlooked flake8 fixes. * Manipulating failing ConstructorStats test to pass, to be able to run all tests with ASAN. This version of the code is ASAN clean with unique_ptr or smart_holder as the default. This change needs to be reverted after adopting the existing move-only-if-refcount-is-1 logic used by type_caster_base. * Adding copy constructor and move constructor tracking to atyp. Preparation for a follow-up change in smart_holder_type_caster, to make this test sensitive to the changing behavior. [skip ci] * Removing `operator T&&() &&` from smart_holder_type_caster, for compatibility with the behavior of type_caster_base. Enables reverting 2 of 3 test manipulations applied under commit 249df7cbdb09817fed0ddf80f01ba5af12466820. The manipulation in test_factory_constructors.py is NOT reverted in this commit. [skip ci] * Fixing unfortunate editing mishap. This reverts the last remaining test manipulation in commit 249df7cbdb09817fed0ddf80f01ba5af12466820 and makes all existing unit tests pass with smart_holder as default holder. * GitHub CI clang-tidy fixes. * Adding messages to terse `static_assert`s, for pre-C++17 compatibility. * Using @pytest.mark.parametrize to run each assert separately (to see all errors, not just the first). * Systematically removing _atyp from function names, to make the test code simpler. * Using re.match to accommodate variable number of intermediate MvCtor. * Also removing `operator T()` from smart_holder_type_caster, to fix gcc compilation errors. The only loss is pass_rref in test_class_sh_basic. * Systematically replacing `detail::enable_if_t<...smart_holder...>` with `typename std::enable_if<...smart_holder...>::type`. Attempt to work around MSVC 2015 issues, to be tested via GitHub CI. The idea for this change originates from this comment: https://github.com/pybind/pybind11/issues/1616#issuecomment-444536813 * Importing re before pytest after observing a PyPy CI flake when importing pytest first. * Copying MSVC 2015 compatibility change from branch pr2672_use_smart_holder_as_default. * Introducing is_smart_holder_type_caster_base_tag, to keep smart_holder code more disconnected. * Working around MSVC 2015 bug. * Expanding comment for MSVC 2015 workaround. * Systematically changing std::enable_if back to detail::enable_if_t, effectively reverting commit 5d4b6890a337ae1bbaec4091f4195606f89a3b06. * Removing unused smart_holder_type_caster_load::loaded_as_rvalue_ref (it was an oversight that it was not removed with commit 23036a45eb4731a06b488ec1fdf83bca677b7f67). * Removing py::classu, because it does not seem useful enough. * Reverting commit 63495313066119dcf7510c2ae8b468b46c12ef8f by un-commenting `const` in `def_buffer(...)`. To make this possible, `operator T const&` and `operator T const*` in `smart_holder_type_caster` need to be marked as `const` member functions. * Adding construct() overloads for constructing smart_holder from alias unique_ptr, shared_ptr returns. * Adding test_class_sh_factory_constructors.cpp to tests/CMakeLists.txt (fixes oversight, this should have been added long before). * Compatibility with old clang versions (clang 3.6, 3.7 C++11). * Cleaning up changes to existing unit tests. * Systematically adding SMART_HOLDER_WIP tag. Removing minor UNTESTED tags (only the throw are not actually exercised, investing time there has a high cost but very little benefit). * Splitting out smart_holder_type_casters again, into new detail/smart_holder_type_casters_inline_include.h. * Splitting out smart_holder_init_inline_include.h. * Adding additional new include files to CMakeLists.txt, tests/extra_python_package/test_files.py. * clang-format cleanup of most smart_holder code. * Adding source code comments in response to review. * Simple micro-benchmark ("ubench") comparing runtime performance for several holders. Tested using github.com/rwgk/pybind11_scons and Google-internal build system. Sorry, no cmake support at the moment. First results: https://docs.google.com/spreadsheets/d/1InapCYws2Gt-stmFf_Bwl33eOMo3aLE_gc9adveY7RU/edit#gid=0 * Breaking out number_bucket.h, adding hook for also collecting performance data for PyCLIF. * Accounting for ubench in MANIFEST.in (simply prune, for now). * Smarter determination of call_repetitions. [skip ci] * Also scaling performance data to PyCLIF. [skip ci] * Adding ubench/python/number_bucket.clif here for general visibility. * Fix after rebase * Merging detail/smart_holder_init_inline_include.h into detail/init.h. * Renaming detail/is_smart_holder_type_caster.h -> detail/smart_holder_sfinae_hooks_only.h. * Renaming is_smart_holder_type_caster -> type_uses_smart_holder_type_caster for clarity. * Renaming type_caster_type_is_smart_holder_type_caster -> wrapped_type_uses_smart_holder_type_caster for clarity. * Renaming is_smart_holder_type_caster_base_tag -> smart_holder_type_caster_base_tag for simplicity. * Adding copyright notices and minor colateral cleanup. * iwyu cleanup (comprehensive only for cast.h and smart_holder*.h files). * Fixing `git rebase master` accident. * Moving large `pragma warning` block from pybind11.h to detail/common.h. * Fixing another `git rebase master` accident.
2021-02-24 05:50:42 +00:00
namespace pybind11_tests {
namespace class_sh_basic {
TEST_SUBMODULE(class_sh_basic, m) {
m.attr("defined_PYBIND11_SMART_HOLDER_ENABLED") =
#ifndef PYBIND11_SMART_HOLDER_ENABLED
[smart_holder] Bake smart_holder functionality into `class_` and `type_caster_base` (#5257) * Put bakein branch @ 18b72c0ffa6ff2747ed6c4b869a80adfb8e762c9 on top of smart_holder branch: Commands used: ``` git checkout bakein git diff smart_holder > ~/zd git checkout smart_holder git checkout -b bakein_sh patch -p 1 < ~/zd git checkout smart_holder \ MANIFEST.in \ README.rst \ README_smart_holder.rst \ docs/advanced/smart_ptrs.rst \ ubench/holder_comparison.cpp \ ubench/holder_comparison.py \ ubench/holder_comparison_extract_sheet_data.py \ ubench/number_bucket.h \ ubench/python/number_bucket.clif git add -A ``` * Add back README_smart_holder.rst in tests/extra_python_package/test_files.py * Restore smart_holder_poc.h as-is on smart_holder branch (i.e. undo `PYBIND11_SMART_HOLDER_PADDING`, which was meant for stress-testing only). * Insert `std::move()` as suggested by @laramiel * `property_cpp_function_sh_*` named specializations, as suggested by @laramiel (https://github.com/pybind/pybind11/pull/5257#discussion_r1688346807) * Call `property_cpp_function_classic` member functions, rather than inlining the implementations. * Use `PYBIND11_HAVE_INTERNALS_WITH_SMART_HOLDER_SUPPORT` in holder_comparison.cpp (holder_comparison.py is NOT changed accordingly in this commit, i.e. can still only be run if the smart_holder functionality is available). * Systematically rename `loaded_as` to `load_as` (`shared_ptr`, `unique_ptr`) as suggested by @laramiel * Make change as suggested by @laramiel. This makes it much more obvious that the latest implementation of `smart_holder_from_unique_ptr()` accepts all existing `return_value_policy` enum values except `copy`. * Resolve `BAKEIN_WIP: Rewrite comment.` for `property_cpp_function_*` specializations. * Resolve `BAKEIN_WIP: Add comment to explain: This is meant for stress-testing only.` * Resolve all remaining BAKEIN_WIP (in pybind11/cast.h). Leave only two pairs of SMART_HOLDER_BAKEIN_FOLLOW_ON comments: refactoring of copyable_holder_caster, move_only_holder_caster. This is best left until after the smart_holder branch is merged into the master branch. * Remove obsolete `using holder_type = smart_holder;` in `load_helper` * Add SMART_HOLDER_BAKEIN_FOLLOW_ON comment for `internals::default_holder` * README_smart_holder.rst update (line count reduced from 356 to 123).
2024-07-31 13:17:31 +00:00
false;
#else
true;
Adding py::smart_holder (for smart-pointer interoperability). (#2672) * Adding test_unique_ptr_member (for desired PyCLIF behavior). See also: https://github.com/pybind/pybind11/issues/2583 Does not build with upstream master or https://github.com/pybind/pybind11/pull/2047, but builds with https://github.com/RobotLocomotion/pybind11 and almost runs: ``` Running tests in directory "/usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests": ================================================================================= test session starts ================================================================================= platform linux -- Python 3.8.5, pytest-5.4.3, py-1.9.0, pluggy-0.13.1 rootdir: /usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests, inifile: pytest.ini collected 2 items test_unique_ptr_member.py .F [100%] ====================================================================================== FAILURES ======================================================================================= _____________________________________________________________________________ test_pointee_and_ptr_owner ______________________________________________________________________________ def test_pointee_and_ptr_owner(): obj = m.pointee() assert obj.get_int() == 213 m.ptr_owner(obj) with pytest.raises(ValueError) as exc_info: > obj.get_int() E Failed: DID NOT RAISE <class 'ValueError'> test_unique_ptr_member.py:17: Failed ============================================================================= 1 failed, 1 passed in 0.06s ============================================================================= ``` * unique_ptr or shared_ptr return * new test_variant_unique_shared with vptr_holder prototype * moving prototype code to pybind11/vptr_holder.h, adding type_caster specialization to make the bindings involving unique_ptr passing compile, but load and cast implementations are missing * disabling GitHub Actions on pull_request (for this PR) * disabling AppVeyor (for this PR) * TRIGGER_SEGSEV macro, annotations for GET_STACK (vptr::get), GET_INT_STACK (pointee) * adding test_promotion_of_disowned_to_shared * Copying tests as-is from xxx_value_ptr_xxx_holder branch. https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder Systematically exercising returning and passing unique_ptr<T>, shared_ptr<T> with unique_ptr, shared_ptr holder. Observations: test_holder_unique_ptr: make_unique_pointee OK pass_unique_pointee BUILD_FAIL (as documented) make_shared_pointee Abort free(): double free detected pass_shared_pointee RuntimeError: Unable to load a custom holder type from a default-holder instance test_holder_shared_ptr: make_unique_pointee Segmentation fault (#1138) pass_unique_pointee BUILD_FAIL (as documented) make_shared_pointee OK pass_shared_pointee OK * Copying tests as-is from xxx_value_ptr_xxx_holder branch. https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder Systematically exercising casting between shared_ptr<base>, shared_ptr<derived>. * Demonstration of Undefined Behavior in handling of shared_ptr holder. Based on https://godbolt.org/z/4fdjaW by jorgbrown@ (thanks Jorg!). * Additional demonstration of Undefined Behavior in handling of shared_ptr holder. * fixing up-down mixup in comment * Demonstration of Undefined Behavior in handling of polymorphic pointers. (This demo does NOT involve smart pointers at all, unlike the otherwise similar test_smart_ptr_private_first_base.) * minor test_private_first_base.cpp simplification (after discovering that this can be wrapped with Boost.Python, using boost::noncopyable) * pybind11 equivalent of Boost.Python test similar to reproducer under #1333 * Snapshot of WIP, TODO: shared_ptr deleter with on/off switch * Adding vptr_deleter. * Adding from/as unique_ptr<T> and unique_ptr<T, D>. * Adding from_shared_ptr. Some polishing. * New tests/core/smart_holder_poc_test.cpp, using Catch2. * Adding in vptr_deleter_guard_flag. * Improved labeling of TEST_CASEs. * Shuffling existing TEST_CASEs into systematic matrix. * Implementing all [S]uccess tests. * Implementing all [E]xception tests. * Testing of exceptions not covered by the from-as matrix. * Adding top-level comment. * Converting from methods to factory functions (no functional change). * Removing obsolete and very incomplete test (replaced by Catch2-based test). * Removing stray file. * Adding type_caster_bare_interface_demo. * Adding shared_ptr<mpty>, shared_ptr<mpty const> casters. * Adding unique_ptr<mpty>, unique_ptr<mpty const> casters. * Pure copy of `class class_` implementation in pybind11.h (master commit 98f1bbb8004f654ba9e26717bdf5912fb899b05a). * classh.h: renaming of class_ to classh + namespace; forking test_classh_wip from test_type_caster_bare_interface_demo. * Hard-coding smart_holder into classh. * Adding mpty::mtxt string member. * Adding isinstance<mpty> in type_caster::load functions. * Adding rvalue_ref, renaming const_value_ref to lvalue_ref & removing const. * Retrieving smart_holder pointer in type_caster<mpty>::load, and using it cast_op operators. * Factoring out smart_holder_type_caster_load. * Retrieving smart_holder pointer in type_caster<std::shared_ptr<mpty[ const]>>::load, and using it cast_op operators. * Improved error messaging: Cannot disown nullptr (as_unique_ptr). * Retrieving smart_holder pointer in type_caster<std::unique_ptr<mpty[ const]>>::load, and using it cast_op operators. * Pure `clang-format --style=file -i` change. * Pure `clang-format --style=file -i` change, with two `clang-format off` directives. * Fixing oversight (discovered by flake8). * flake8 cleanup * Systematically setting mtxt for all rtrn_mpty_* functions (preparation, the values are not actually used yet). * static cast handle for rtrn_cptr works by simply dropping in code from type_caster_base (marked with comments). * static cast handle for rtrn_cref works by simply dropping in code from type_caster_base (marked with comments). rtrn_mref and rtrn_mptr work via const_cast (to add const). * static cast handle for rtrn_valu works by simply dropping in code from type_caster_base (marked with comments). rtrn_rref raises a RuntimeError, to be investigated. * Copying type_caster_generic::cast into type_caster<mpty> as-is (preparation for handling smart pointers). * Pure clang-format change (applied to original type_caster_generic::cast). * Adding comment re potential use_count data race. * static handle cast implementations for rtrn_shmp, rtrn_shcp. * Adding MISSING comments in operator std::unique_ptr<mpty[ const]>. * static handle cast implementations for rtrn_uqmp, rtrn_uqcp. * Bug fix: vptr_deleter_armed_flag_ptr has to live on the heap. See new bullet point in comment section near the top. The variable was also renamed to reflect its function more accurately. * Fixing bugs discovered by ASAN. The code is now ASAN, MSAN, UBSAN clean. * Making test_type_caster_bare_interface_demo.cpp slightly more realistic, ASAN, MSAN, UBSAN clean. * Calling deregister_instance after disowning via unique_ptr. * Removing enable_shared_from_this stub, simplifying existing code, clang-format. Open question, with respect to the original code: https://github.com/pybind/pybind11/blob/76a160070b369f8d82b945c97924227e8b835c94/include/pybind11/pybind11.h#L1510 To me it looks like the exact situation marked as `std::shared_ptr<Good> gp1 = not_so_good.getptr();` here: https://en.cppreference.com/w/cpp/memory/enable_shared_from_this The comment there is: `// undefined behavior (until C++17) and std::bad_weak_ptr thrown (since C++17)` Does the existing code have UB pre C++17? I'll leave handling of enable_shared_from_this for later, as the need arises. * Cosmetical change around helper functions. * Using type_caster_base<mpty>::src_and_type directly, removing copy. Also renaming one cast to cast_const_raw_ptr, for clarity. * Fixing clang-format oversight. * Using factored-out make_constructor (PR #2798), removing duplicate code. * Inserting additional assert to ensure a returned unique_ptr is always a new Python instance. * Adding minor comment (change to internals needed to distinguish uninitialized/disowned in error message). * Factoring out find_existing_python_instance(). * Moving factored-out make_constructor to test_classh_wip.cpp, restoring previous version of cast.h. This is currently the most practical approach. See PR #2798 for background. * Copying classh type_casters from test_classh_wip.cpp UNMODIFIED, as a baseline for generalizing the code. * Using pybind11/detail/classh_type_casters.h from test_classh_wip.cpp. * Adding & using PYBIND11_CLASSH_TYPE_CASTERS define. * Adding test_classh_inheritance, currently failing (passes with class_). * Removing .clang-format before git rebase master (where the file was added). * Bringing back .clang-format, the previous rm was a bad idea. * Folding in modified_type_caster_generic_load_impl, just enough to pass test_class_wip. test_classh_inheritance is still failing, but with a different error: [RuntimeError: Incompatible type (as_raw_ptr_unowned).] * Minimal changes needed to pass test_classh_inheritance. * First pass adjusting try_implicit_casts and try_load_foreign_module_local to capture loaded_v_h, but untested and guarded with pybind11_failure("Untested"). This was done mainly to determine general feasibility. Note the TODO in pybind11.h, where type_caster_generic::local_load is currently hard-coded. test_classh_wip and test_classh_inheritance still pass, as before. * Decoupling generic_type from type_caster_generic. * Changes and tests covering classh_type_casters try_implicit_casts. * Minimal test covering classh_type_casters load_impl Case 2b. * Removing stray isinstance<T>(src): it interferes with the py::module_local feature. Adding missing #includes. * Tests for classh py::module_local() feature. * Pure renaming of function names in test_classh_inheritance, similar to the systematic approach used in test_class_wip. NO functional changes. * Pure renaming of function and variable names, for better generalization when convoluting with inheritance. NO functional changes. * Adopting systematic naming scheme from test_classh_wip. NO functional changes. * Moving const after type name, for functions that cover a systematic scheme. NO functional changes. * Adding smart_holder_type_caster_load::loaded_as_shared_ptr, currently bypassing smart_holder shared_ptr tracking completely, but the tests pass and are sanitizer clean. * Removing rtti_held from smart_holder. See updated comment. * Cleaning up loaded_as_raw_ptr_unowned, loaded_as_shared_ptr. * Factoring out convert_type and folding into loaded_as_unique_ptr. * Folding convert_type into lvalue_ref and rvalue_ref paths. Some smart_holder_type_caster_load cleanup. * Using unique_ptr in local_load to replace static variable. Also adding local_load_safety_guard. * Converting test_unique_ptr_member to using classh: fully working, ASAN, MSAN, UBSAN clean. * Removing debugging comments (GET_STACK, GET_INT_STACK). cast.h is identical to current master again, pybind11.h only has the generic_type::initialize(..., &type_caster_generic::local_load) change. * Purging obsolete pybind11/vptr_holder.h and associated test. * Moving several tests to github.com/rwgk/rwgk_tbx/tree/main/pybind11_tests https://github.com/rwgk/rwgk_tbx/commit/a2c2f88174a30f5de80d7d26e0f77c7b60f5fb85 These tests are from experimenting, and for demonstrating UB in pybind11 multiple inheritance handling ("first_base"), to be fixed later. * Adding py::smart_holder support to py::class_, purging py::classh completely. * Renaming files in include directory, creating pybind11/smart_holder.h. * Renaming all "classh" to "smart_holder" in pybind11/detail/smart_holder_type_casters.h. The user-facing macro is now PYBIND11_SMART_HOLDER_TYPE_CASTERS. * Systematically renaming tests to use "class_sh" in the name. * Renaming test_type_caster_bare_interface_demo to test_type_caster_bare_interface. * Renaming new tests/core subdirectory to tests/pure_cpp. * Adding new tests to CMake config, resetting CI config. * Changing CMake file so that test_class_sh_module_local.py actually runs. * clang-tidy fixes. * 32-bit compatibility. * Reusing type_caster_base make_copy_constructor, make_move_constructor with a trick. * CMake COMPARE NATURAL is not available with older versions. * Adding copyright notices to new header files. * Explicitly define copy/move constructors/assignments. * Adding new header files to tests/extra_python_package/test_files.py. * Adding tests/pure_cpp/CMakeLists.txt. * Making use of the new find_existing_python_instance() function factored out with PR #2822. * Moving define PYBIND11_SMART_HOLDER_TYPE_CASTERS(T) down in the file. NO functional changes. Preparation for follow-up work (to keep that diff smaller). * Reintroducing py::classh, this time as a simple alias for py::class_<U, py::smart_holder>. * Replacing detail::is_smart_holder<H> in cast.h with detail::is_smart_holder_type_caster<T>. Moving get_local_load_function_ptr, init_instance_for_type to smart_holder_type_caster_class_hooks. Expanding static_assert in py::type::handle_of<> to accommodate smart_holder_type_casters. * Fixing oversight. * Adding classu alias for class_<U, std::unique_ptr<U>>. * Giving up on idea to use legacy init_instance only if is_base_of<type_caster_generic, type_caster<T>. There are use cases in the wild that define both a custom type_caster and class_. * Removing test_type_caster_bare_interface, which was moved to the separate PR #2834. * Moving up is_smart_holder_type_caster, to also use in cast_is_temporary_value_reference. * Adding smart_holder_type_casters for unique_ptr with custom deleter. SEVERE CODE DUPLICATION. This commit is to establish a baseline for consolidating the unique_ptr code. * Unification of unique_ptr, unique_ptr_with_deleter code in smart_holder_poc.h. Leads to more fitting error messages. Enables use of unique_ptr<T, D> smart_holder_type_casters also for unique_ptr<T>. * Copying files as-is from branch test_unique_ptr_member (PR #2672). * Adding comment, simplifying naming, cmake addition. * Introducing PYBIND11_USE_SMART_HOLDER_AS_DEFAULT macro (tested only undefined; there are many errors with the macro defined). * Removing test_type_caster_bare_interface, which was moved to the separate PR #2834. * Fixing oversight introduced with commit 95425f13d6c14fcb6ee479b62b602dc8a605ec49. * Setting record.default_holder correctly for PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. With this test_class.cpp builds and even mostly runs, except `test_multiple_instances_with_same_pointer`, which segfaults because it is using a `unique_ptr` holder but `smart_holder` `type_caster`. Also adding `static_assert`s to generate build errors for such situations, but guarding with `#if 0` to first pivot to test_factory_constructors.cpp. * Fixing up cast.h and smart_holder.h after rebase. * Removing detail/smart_holder_type_casters.h in separate commit. * Commenting out const in def_buffer(... const). With this, test_buffers builds and runs with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. Explanation why the const needs to be removed, or fix elsewhere, is still needed, but left for later. * Adding test_class_sh_factory_constructors, reproducing test_factory_constructors failure. Using py::class_ in this commit, to be changed to py::classh for debugging. * Removing include/pybind11/detail/smart_holder_type_casters.h from CMakeLists.txt, test_files.py (since it does not exist in this branch). * Adding // DANGER ZONE reminders. * Converting as many py::class_ to py::classh as possible, not breaking tests. * Adding initimpl::construct() overloads, resulting in test_class_sh_factory_constructors feature parity for py::class_ and py::classh. * Adding enable_if !is_smart_holder_type_caster to existing initimpl::construct(). With this test_factory_constructors.cpp builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. * Disabling shared_ptr&, shared_ptr* tests when building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT for now, pending work on smart_holder_type_caster<shared_ptr>. * Factoring out struct and class definitions into anonymous namespace. Preparation for building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. * Simplifying from_unique_ptr(): typename D = std::default_delete<T> is not needed. Factoring out is_std_default_delete<T>() for consistentcy between ensure_compatible_rtti_uqp_del() and from_unique_ptr(). * Introducing PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. Using it in test_smart_ptr.cpp. With this test_smart_ptr builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT and all but one test run successfully. * Introducing 1. type_caster_for_class_, used in PYBIND11_MAKE_OPAQUE, and 2. default_holder_type, used in stl_bind.h. * Using __VA_ARGS__ in PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. * Replacing condense_for_macro with much simpler approach. * Softening static_assert, to only check specifically that smart_holder is not mixed with type_caster_base, and unique_ptr/shared_ptr holders are not mixed with smart_holder_type_casters. * Adding PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS in test_class.cpp (with this all but one test succeed with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT). * Adding remaining PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. static_assert for "necessary conditions" for both types of default holder, static_assert for "strict conditions" guarded by new PYBIND11_STRICT_ASSERTS_CLASS_HOLDER_VS_TYPE_CASTER_MIX. All tests build & run as before with unique_ptr as the default holder, all tests build for smart_holder as the default holder, even with the strict static_assert. * Introducing check_is_smart_holder_type_caster() function for runtime check, and reinterpreting record.default_holder as "uses_unique_ptr_holder". With this test_smart_ptr succeeds. (All 42 tests build, 35 tests succeed, 5 run but have some failures, 2 segfault.) * Bug fix: Adding have_value() to smart_holder_type_caster_load. With this test_builtin_casters succeeds. (All 42 tests build, 36 tests succeed, 5 run but have some failures, 1 segfault.) * Adding unowned_void_ptr_from_direct_conversion to modified_type_caster_generic_load_impl. This fixes the last remaining segfault (test_numpy_dtypes). New stats for all tests combined: 12 failed, 458 passed. * Adding "Lazy allocation for unallocated values" (for old-style __init__) into load_value_and_holder. Deferring destruction of disowned holder until clear_instance, to remain inspectable for "uninitialized" or "disowned" detection. New stats for all tests combined: 5 failed, 465 passed. * Changing std::shared_ptr pointer/reference to const pointer/reference. New stats for all tests combined: 4 failed, 466 passed. * Adding return_value_policy::move to permissible policies for unique_ptr returns. New stats for all tests combined: 3 failed, 467 passed. * Overlooked flake8 fixes. * Manipulating failing ConstructorStats test to pass, to be able to run all tests with ASAN. This version of the code is ASAN clean with unique_ptr or smart_holder as the default. This change needs to be reverted after adopting the existing move-only-if-refcount-is-1 logic used by type_caster_base. * Adding copy constructor and move constructor tracking to atyp. Preparation for a follow-up change in smart_holder_type_caster, to make this test sensitive to the changing behavior. [skip ci] * Removing `operator T&&() &&` from smart_holder_type_caster, for compatibility with the behavior of type_caster_base. Enables reverting 2 of 3 test manipulations applied under commit 249df7cbdb09817fed0ddf80f01ba5af12466820. The manipulation in test_factory_constructors.py is NOT reverted in this commit. [skip ci] * Fixing unfortunate editing mishap. This reverts the last remaining test manipulation in commit 249df7cbdb09817fed0ddf80f01ba5af12466820 and makes all existing unit tests pass with smart_holder as default holder. * GitHub CI clang-tidy fixes. * Adding messages to terse `static_assert`s, for pre-C++17 compatibility. * Using @pytest.mark.parametrize to run each assert separately (to see all errors, not just the first). * Systematically removing _atyp from function names, to make the test code simpler. * Using re.match to accommodate variable number of intermediate MvCtor. * Also removing `operator T()` from smart_holder_type_caster, to fix gcc compilation errors. The only loss is pass_rref in test_class_sh_basic. * Systematically replacing `detail::enable_if_t<...smart_holder...>` with `typename std::enable_if<...smart_holder...>::type`. Attempt to work around MSVC 2015 issues, to be tested via GitHub CI. The idea for this change originates from this comment: https://github.com/pybind/pybind11/issues/1616#issuecomment-444536813 * Importing re before pytest after observing a PyPy CI flake when importing pytest first. * Copying MSVC 2015 compatibility change from branch pr2672_use_smart_holder_as_default. * Introducing is_smart_holder_type_caster_base_tag, to keep smart_holder code more disconnected. * Working around MSVC 2015 bug. * Expanding comment for MSVC 2015 workaround. * Systematically changing std::enable_if back to detail::enable_if_t, effectively reverting commit 5d4b6890a337ae1bbaec4091f4195606f89a3b06. * Removing unused smart_holder_type_caster_load::loaded_as_rvalue_ref (it was an oversight that it was not removed with commit 23036a45eb4731a06b488ec1fdf83bca677b7f67). * Removing py::classu, because it does not seem useful enough. * Reverting commit 63495313066119dcf7510c2ae8b468b46c12ef8f by un-commenting `const` in `def_buffer(...)`. To make this possible, `operator T const&` and `operator T const*` in `smart_holder_type_caster` need to be marked as `const` member functions. * Adding construct() overloads for constructing smart_holder from alias unique_ptr, shared_ptr returns. * Adding test_class_sh_factory_constructors.cpp to tests/CMakeLists.txt (fixes oversight, this should have been added long before). * Compatibility with old clang versions (clang 3.6, 3.7 C++11). * Cleaning up changes to existing unit tests. * Systematically adding SMART_HOLDER_WIP tag. Removing minor UNTESTED tags (only the throw are not actually exercised, investing time there has a high cost but very little benefit). * Splitting out smart_holder_type_casters again, into new detail/smart_holder_type_casters_inline_include.h. * Splitting out smart_holder_init_inline_include.h. * Adding additional new include files to CMakeLists.txt, tests/extra_python_package/test_files.py. * clang-format cleanup of most smart_holder code. * Adding source code comments in response to review. * Simple micro-benchmark ("ubench") comparing runtime performance for several holders. Tested using github.com/rwgk/pybind11_scons and Google-internal build system. Sorry, no cmake support at the moment. First results: https://docs.google.com/spreadsheets/d/1InapCYws2Gt-stmFf_Bwl33eOMo3aLE_gc9adveY7RU/edit#gid=0 * Breaking out number_bucket.h, adding hook for also collecting performance data for PyCLIF. * Accounting for ubench in MANIFEST.in (simply prune, for now). * Smarter determination of call_repetitions. [skip ci] * Also scaling performance data to PyCLIF. [skip ci] * Adding ubench/python/number_bucket.clif here for general visibility. * Fix after rebase * Merging detail/smart_holder_init_inline_include.h into detail/init.h. * Renaming detail/is_smart_holder_type_caster.h -> detail/smart_holder_sfinae_hooks_only.h. * Renaming is_smart_holder_type_caster -> type_uses_smart_holder_type_caster for clarity. * Renaming type_caster_type_is_smart_holder_type_caster -> wrapped_type_uses_smart_holder_type_caster for clarity. * Renaming is_smart_holder_type_caster_base_tag -> smart_holder_type_caster_base_tag for simplicity. * Adding copyright notices and minor colateral cleanup. * iwyu cleanup (comprehensive only for cast.h and smart_holder*.h files). * Fixing `git rebase master` accident. * Moving large `pragma warning` block from pybind11.h to detail/common.h. * Fixing another `git rebase master` accident.
2021-02-24 05:50:42 +00:00
namespace py = pybind11;
py::classh<atyp>(m, "atyp").def(py::init<>()).def(py::init([](const std::string &mtxt) {
atyp obj;
obj.mtxt = mtxt;
return obj;
}));
m.def("rtrn_valu", rtrn_valu);
m.def("rtrn_rref", rtrn_rref);
m.def("rtrn_cref", rtrn_cref);
m.def("rtrn_mref", rtrn_mref);
m.def("rtrn_cptr", rtrn_cptr);
m.def("rtrn_mptr", rtrn_mptr);
m.def("pass_valu", pass_valu);
m.def("pass_cref", pass_cref);
m.def("pass_mref", pass_mref);
m.def("pass_cptr", pass_cptr);
m.def("pass_mptr", pass_mptr);
m.def("rtrn_shmp", rtrn_shmp);
m.def("rtrn_shcp", rtrn_shcp);
m.def("pass_shmp", pass_shmp);
m.def("pass_shcp", pass_shcp);
m.def("rtrn_uqmp", rtrn_uqmp);
m.def("rtrn_uqcp", rtrn_uqcp);
m.def("pass_uqmp", pass_uqmp);
m.def("pass_uqcp", pass_uqcp);
m.def("rtrn_udmp", rtrn_udmp);
m.def("rtrn_udcp", rtrn_udcp);
m.def("pass_udmp", pass_udmp);
m.def("pass_udcp", pass_udcp);
m.def("rtrn_udmp_del", rtrn_udmp_del);
m.def("rtrn_udcp_del", rtrn_udcp_del);
m.def("pass_udmp_del", pass_udmp_del);
m.def("pass_udcp_del", pass_udcp_del);
m.def("rtrn_udmp_del_nd", rtrn_udmp_del_nd);
m.def("rtrn_udcp_del_nd", rtrn_udcp_del_nd);
m.def("pass_udmp_del_nd", pass_udmp_del_nd);
m.def("pass_udcp_del_nd", pass_udcp_del_nd);
py::classh<uconsumer>(m, "uconsumer")
.def(py::init<>())
.def("valid", &uconsumer::valid)
.def("pass_valu", &uconsumer::pass_valu)
.def("pass_rref", &uconsumer::pass_rref)
.def("rtrn_valu", &uconsumer::rtrn_valu)
.def("rtrn_lref", &uconsumer::rtrn_lref)
.def("rtrn_cref", &uconsumer::rtrn_cref);
Adding py::smart_holder (for smart-pointer interoperability). (#2672) * Adding test_unique_ptr_member (for desired PyCLIF behavior). See also: https://github.com/pybind/pybind11/issues/2583 Does not build with upstream master or https://github.com/pybind/pybind11/pull/2047, but builds with https://github.com/RobotLocomotion/pybind11 and almost runs: ``` Running tests in directory "/usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests": ================================================================================= test session starts ================================================================================= platform linux -- Python 3.8.5, pytest-5.4.3, py-1.9.0, pluggy-0.13.1 rootdir: /usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests, inifile: pytest.ini collected 2 items test_unique_ptr_member.py .F [100%] ====================================================================================== FAILURES ======================================================================================= _____________________________________________________________________________ test_pointee_and_ptr_owner ______________________________________________________________________________ def test_pointee_and_ptr_owner(): obj = m.pointee() assert obj.get_int() == 213 m.ptr_owner(obj) with pytest.raises(ValueError) as exc_info: > obj.get_int() E Failed: DID NOT RAISE <class 'ValueError'> test_unique_ptr_member.py:17: Failed ============================================================================= 1 failed, 1 passed in 0.06s ============================================================================= ``` * unique_ptr or shared_ptr return * new test_variant_unique_shared with vptr_holder prototype * moving prototype code to pybind11/vptr_holder.h, adding type_caster specialization to make the bindings involving unique_ptr passing compile, but load and cast implementations are missing * disabling GitHub Actions on pull_request (for this PR) * disabling AppVeyor (for this PR) * TRIGGER_SEGSEV macro, annotations for GET_STACK (vptr::get), GET_INT_STACK (pointee) * adding test_promotion_of_disowned_to_shared * Copying tests as-is from xxx_value_ptr_xxx_holder branch. https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder Systematically exercising returning and passing unique_ptr<T>, shared_ptr<T> with unique_ptr, shared_ptr holder. Observations: test_holder_unique_ptr: make_unique_pointee OK pass_unique_pointee BUILD_FAIL (as documented) make_shared_pointee Abort free(): double free detected pass_shared_pointee RuntimeError: Unable to load a custom holder type from a default-holder instance test_holder_shared_ptr: make_unique_pointee Segmentation fault (#1138) pass_unique_pointee BUILD_FAIL (as documented) make_shared_pointee OK pass_shared_pointee OK * Copying tests as-is from xxx_value_ptr_xxx_holder branch. https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder Systematically exercising casting between shared_ptr<base>, shared_ptr<derived>. * Demonstration of Undefined Behavior in handling of shared_ptr holder. Based on https://godbolt.org/z/4fdjaW by jorgbrown@ (thanks Jorg!). * Additional demonstration of Undefined Behavior in handling of shared_ptr holder. * fixing up-down mixup in comment * Demonstration of Undefined Behavior in handling of polymorphic pointers. (This demo does NOT involve smart pointers at all, unlike the otherwise similar test_smart_ptr_private_first_base.) * minor test_private_first_base.cpp simplification (after discovering that this can be wrapped with Boost.Python, using boost::noncopyable) * pybind11 equivalent of Boost.Python test similar to reproducer under #1333 * Snapshot of WIP, TODO: shared_ptr deleter with on/off switch * Adding vptr_deleter. * Adding from/as unique_ptr<T> and unique_ptr<T, D>. * Adding from_shared_ptr. Some polishing. * New tests/core/smart_holder_poc_test.cpp, using Catch2. * Adding in vptr_deleter_guard_flag. * Improved labeling of TEST_CASEs. * Shuffling existing TEST_CASEs into systematic matrix. * Implementing all [S]uccess tests. * Implementing all [E]xception tests. * Testing of exceptions not covered by the from-as matrix. * Adding top-level comment. * Converting from methods to factory functions (no functional change). * Removing obsolete and very incomplete test (replaced by Catch2-based test). * Removing stray file. * Adding type_caster_bare_interface_demo. * Adding shared_ptr<mpty>, shared_ptr<mpty const> casters. * Adding unique_ptr<mpty>, unique_ptr<mpty const> casters. * Pure copy of `class class_` implementation in pybind11.h (master commit 98f1bbb8004f654ba9e26717bdf5912fb899b05a). * classh.h: renaming of class_ to classh + namespace; forking test_classh_wip from test_type_caster_bare_interface_demo. * Hard-coding smart_holder into classh. * Adding mpty::mtxt string member. * Adding isinstance<mpty> in type_caster::load functions. * Adding rvalue_ref, renaming const_value_ref to lvalue_ref & removing const. * Retrieving smart_holder pointer in type_caster<mpty>::load, and using it cast_op operators. * Factoring out smart_holder_type_caster_load. * Retrieving smart_holder pointer in type_caster<std::shared_ptr<mpty[ const]>>::load, and using it cast_op operators. * Improved error messaging: Cannot disown nullptr (as_unique_ptr). * Retrieving smart_holder pointer in type_caster<std::unique_ptr<mpty[ const]>>::load, and using it cast_op operators. * Pure `clang-format --style=file -i` change. * Pure `clang-format --style=file -i` change, with two `clang-format off` directives. * Fixing oversight (discovered by flake8). * flake8 cleanup * Systematically setting mtxt for all rtrn_mpty_* functions (preparation, the values are not actually used yet). * static cast handle for rtrn_cptr works by simply dropping in code from type_caster_base (marked with comments). * static cast handle for rtrn_cref works by simply dropping in code from type_caster_base (marked with comments). rtrn_mref and rtrn_mptr work via const_cast (to add const). * static cast handle for rtrn_valu works by simply dropping in code from type_caster_base (marked with comments). rtrn_rref raises a RuntimeError, to be investigated. * Copying type_caster_generic::cast into type_caster<mpty> as-is (preparation for handling smart pointers). * Pure clang-format change (applied to original type_caster_generic::cast). * Adding comment re potential use_count data race. * static handle cast implementations for rtrn_shmp, rtrn_shcp. * Adding MISSING comments in operator std::unique_ptr<mpty[ const]>. * static handle cast implementations for rtrn_uqmp, rtrn_uqcp. * Bug fix: vptr_deleter_armed_flag_ptr has to live on the heap. See new bullet point in comment section near the top. The variable was also renamed to reflect its function more accurately. * Fixing bugs discovered by ASAN. The code is now ASAN, MSAN, UBSAN clean. * Making test_type_caster_bare_interface_demo.cpp slightly more realistic, ASAN, MSAN, UBSAN clean. * Calling deregister_instance after disowning via unique_ptr. * Removing enable_shared_from_this stub, simplifying existing code, clang-format. Open question, with respect to the original code: https://github.com/pybind/pybind11/blob/76a160070b369f8d82b945c97924227e8b835c94/include/pybind11/pybind11.h#L1510 To me it looks like the exact situation marked as `std::shared_ptr<Good> gp1 = not_so_good.getptr();` here: https://en.cppreference.com/w/cpp/memory/enable_shared_from_this The comment there is: `// undefined behavior (until C++17) and std::bad_weak_ptr thrown (since C++17)` Does the existing code have UB pre C++17? I'll leave handling of enable_shared_from_this for later, as the need arises. * Cosmetical change around helper functions. * Using type_caster_base<mpty>::src_and_type directly, removing copy. Also renaming one cast to cast_const_raw_ptr, for clarity. * Fixing clang-format oversight. * Using factored-out make_constructor (PR #2798), removing duplicate code. * Inserting additional assert to ensure a returned unique_ptr is always a new Python instance. * Adding minor comment (change to internals needed to distinguish uninitialized/disowned in error message). * Factoring out find_existing_python_instance(). * Moving factored-out make_constructor to test_classh_wip.cpp, restoring previous version of cast.h. This is currently the most practical approach. See PR #2798 for background. * Copying classh type_casters from test_classh_wip.cpp UNMODIFIED, as a baseline for generalizing the code. * Using pybind11/detail/classh_type_casters.h from test_classh_wip.cpp. * Adding & using PYBIND11_CLASSH_TYPE_CASTERS define. * Adding test_classh_inheritance, currently failing (passes with class_). * Removing .clang-format before git rebase master (where the file was added). * Bringing back .clang-format, the previous rm was a bad idea. * Folding in modified_type_caster_generic_load_impl, just enough to pass test_class_wip. test_classh_inheritance is still failing, but with a different error: [RuntimeError: Incompatible type (as_raw_ptr_unowned).] * Minimal changes needed to pass test_classh_inheritance. * First pass adjusting try_implicit_casts and try_load_foreign_module_local to capture loaded_v_h, but untested and guarded with pybind11_failure("Untested"). This was done mainly to determine general feasibility. Note the TODO in pybind11.h, where type_caster_generic::local_load is currently hard-coded. test_classh_wip and test_classh_inheritance still pass, as before. * Decoupling generic_type from type_caster_generic. * Changes and tests covering classh_type_casters try_implicit_casts. * Minimal test covering classh_type_casters load_impl Case 2b. * Removing stray isinstance<T>(src): it interferes with the py::module_local feature. Adding missing #includes. * Tests for classh py::module_local() feature. * Pure renaming of function names in test_classh_inheritance, similar to the systematic approach used in test_class_wip. NO functional changes. * Pure renaming of function and variable names, for better generalization when convoluting with inheritance. NO functional changes. * Adopting systematic naming scheme from test_classh_wip. NO functional changes. * Moving const after type name, for functions that cover a systematic scheme. NO functional changes. * Adding smart_holder_type_caster_load::loaded_as_shared_ptr, currently bypassing smart_holder shared_ptr tracking completely, but the tests pass and are sanitizer clean. * Removing rtti_held from smart_holder. See updated comment. * Cleaning up loaded_as_raw_ptr_unowned, loaded_as_shared_ptr. * Factoring out convert_type and folding into loaded_as_unique_ptr. * Folding convert_type into lvalue_ref and rvalue_ref paths. Some smart_holder_type_caster_load cleanup. * Using unique_ptr in local_load to replace static variable. Also adding local_load_safety_guard. * Converting test_unique_ptr_member to using classh: fully working, ASAN, MSAN, UBSAN clean. * Removing debugging comments (GET_STACK, GET_INT_STACK). cast.h is identical to current master again, pybind11.h only has the generic_type::initialize(..., &type_caster_generic::local_load) change. * Purging obsolete pybind11/vptr_holder.h and associated test. * Moving several tests to github.com/rwgk/rwgk_tbx/tree/main/pybind11_tests https://github.com/rwgk/rwgk_tbx/commit/a2c2f88174a30f5de80d7d26e0f77c7b60f5fb85 These tests are from experimenting, and for demonstrating UB in pybind11 multiple inheritance handling ("first_base"), to be fixed later. * Adding py::smart_holder support to py::class_, purging py::classh completely. * Renaming files in include directory, creating pybind11/smart_holder.h. * Renaming all "classh" to "smart_holder" in pybind11/detail/smart_holder_type_casters.h. The user-facing macro is now PYBIND11_SMART_HOLDER_TYPE_CASTERS. * Systematically renaming tests to use "class_sh" in the name. * Renaming test_type_caster_bare_interface_demo to test_type_caster_bare_interface. * Renaming new tests/core subdirectory to tests/pure_cpp. * Adding new tests to CMake config, resetting CI config. * Changing CMake file so that test_class_sh_module_local.py actually runs. * clang-tidy fixes. * 32-bit compatibility. * Reusing type_caster_base make_copy_constructor, make_move_constructor with a trick. * CMake COMPARE NATURAL is not available with older versions. * Adding copyright notices to new header files. * Explicitly define copy/move constructors/assignments. * Adding new header files to tests/extra_python_package/test_files.py. * Adding tests/pure_cpp/CMakeLists.txt. * Making use of the new find_existing_python_instance() function factored out with PR #2822. * Moving define PYBIND11_SMART_HOLDER_TYPE_CASTERS(T) down in the file. NO functional changes. Preparation for follow-up work (to keep that diff smaller). * Reintroducing py::classh, this time as a simple alias for py::class_<U, py::smart_holder>. * Replacing detail::is_smart_holder<H> in cast.h with detail::is_smart_holder_type_caster<T>. Moving get_local_load_function_ptr, init_instance_for_type to smart_holder_type_caster_class_hooks. Expanding static_assert in py::type::handle_of<> to accommodate smart_holder_type_casters. * Fixing oversight. * Adding classu alias for class_<U, std::unique_ptr<U>>. * Giving up on idea to use legacy init_instance only if is_base_of<type_caster_generic, type_caster<T>. There are use cases in the wild that define both a custom type_caster and class_. * Removing test_type_caster_bare_interface, which was moved to the separate PR #2834. * Moving up is_smart_holder_type_caster, to also use in cast_is_temporary_value_reference. * Adding smart_holder_type_casters for unique_ptr with custom deleter. SEVERE CODE DUPLICATION. This commit is to establish a baseline for consolidating the unique_ptr code. * Unification of unique_ptr, unique_ptr_with_deleter code in smart_holder_poc.h. Leads to more fitting error messages. Enables use of unique_ptr<T, D> smart_holder_type_casters also for unique_ptr<T>. * Copying files as-is from branch test_unique_ptr_member (PR #2672). * Adding comment, simplifying naming, cmake addition. * Introducing PYBIND11_USE_SMART_HOLDER_AS_DEFAULT macro (tested only undefined; there are many errors with the macro defined). * Removing test_type_caster_bare_interface, which was moved to the separate PR #2834. * Fixing oversight introduced with commit 95425f13d6c14fcb6ee479b62b602dc8a605ec49. * Setting record.default_holder correctly for PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. With this test_class.cpp builds and even mostly runs, except `test_multiple_instances_with_same_pointer`, which segfaults because it is using a `unique_ptr` holder but `smart_holder` `type_caster`. Also adding `static_assert`s to generate build errors for such situations, but guarding with `#if 0` to first pivot to test_factory_constructors.cpp. * Fixing up cast.h and smart_holder.h after rebase. * Removing detail/smart_holder_type_casters.h in separate commit. * Commenting out const in def_buffer(... const). With this, test_buffers builds and runs with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. Explanation why the const needs to be removed, or fix elsewhere, is still needed, but left for later. * Adding test_class_sh_factory_constructors, reproducing test_factory_constructors failure. Using py::class_ in this commit, to be changed to py::classh for debugging. * Removing include/pybind11/detail/smart_holder_type_casters.h from CMakeLists.txt, test_files.py (since it does not exist in this branch). * Adding // DANGER ZONE reminders. * Converting as many py::class_ to py::classh as possible, not breaking tests. * Adding initimpl::construct() overloads, resulting in test_class_sh_factory_constructors feature parity for py::class_ and py::classh. * Adding enable_if !is_smart_holder_type_caster to existing initimpl::construct(). With this test_factory_constructors.cpp builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. * Disabling shared_ptr&, shared_ptr* tests when building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT for now, pending work on smart_holder_type_caster<shared_ptr>. * Factoring out struct and class definitions into anonymous namespace. Preparation for building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. * Simplifying from_unique_ptr(): typename D = std::default_delete<T> is not needed. Factoring out is_std_default_delete<T>() for consistentcy between ensure_compatible_rtti_uqp_del() and from_unique_ptr(). * Introducing PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. Using it in test_smart_ptr.cpp. With this test_smart_ptr builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT and all but one test run successfully. * Introducing 1. type_caster_for_class_, used in PYBIND11_MAKE_OPAQUE, and 2. default_holder_type, used in stl_bind.h. * Using __VA_ARGS__ in PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. * Replacing condense_for_macro with much simpler approach. * Softening static_assert, to only check specifically that smart_holder is not mixed with type_caster_base, and unique_ptr/shared_ptr holders are not mixed with smart_holder_type_casters. * Adding PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS in test_class.cpp (with this all but one test succeed with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT). * Adding remaining PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. static_assert for "necessary conditions" for both types of default holder, static_assert for "strict conditions" guarded by new PYBIND11_STRICT_ASSERTS_CLASS_HOLDER_VS_TYPE_CASTER_MIX. All tests build & run as before with unique_ptr as the default holder, all tests build for smart_holder as the default holder, even with the strict static_assert. * Introducing check_is_smart_holder_type_caster() function for runtime check, and reinterpreting record.default_holder as "uses_unique_ptr_holder". With this test_smart_ptr succeeds. (All 42 tests build, 35 tests succeed, 5 run but have some failures, 2 segfault.) * Bug fix: Adding have_value() to smart_holder_type_caster_load. With this test_builtin_casters succeeds. (All 42 tests build, 36 tests succeed, 5 run but have some failures, 1 segfault.) * Adding unowned_void_ptr_from_direct_conversion to modified_type_caster_generic_load_impl. This fixes the last remaining segfault (test_numpy_dtypes). New stats for all tests combined: 12 failed, 458 passed. * Adding "Lazy allocation for unallocated values" (for old-style __init__) into load_value_and_holder. Deferring destruction of disowned holder until clear_instance, to remain inspectable for "uninitialized" or "disowned" detection. New stats for all tests combined: 5 failed, 465 passed. * Changing std::shared_ptr pointer/reference to const pointer/reference. New stats for all tests combined: 4 failed, 466 passed. * Adding return_value_policy::move to permissible policies for unique_ptr returns. New stats for all tests combined: 3 failed, 467 passed. * Overlooked flake8 fixes. * Manipulating failing ConstructorStats test to pass, to be able to run all tests with ASAN. This version of the code is ASAN clean with unique_ptr or smart_holder as the default. This change needs to be reverted after adopting the existing move-only-if-refcount-is-1 logic used by type_caster_base. * Adding copy constructor and move constructor tracking to atyp. Preparation for a follow-up change in smart_holder_type_caster, to make this test sensitive to the changing behavior. [skip ci] * Removing `operator T&&() &&` from smart_holder_type_caster, for compatibility with the behavior of type_caster_base. Enables reverting 2 of 3 test manipulations applied under commit 249df7cbdb09817fed0ddf80f01ba5af12466820. The manipulation in test_factory_constructors.py is NOT reverted in this commit. [skip ci] * Fixing unfortunate editing mishap. This reverts the last remaining test manipulation in commit 249df7cbdb09817fed0ddf80f01ba5af12466820 and makes all existing unit tests pass with smart_holder as default holder. * GitHub CI clang-tidy fixes. * Adding messages to terse `static_assert`s, for pre-C++17 compatibility. * Using @pytest.mark.parametrize to run each assert separately (to see all errors, not just the first). * Systematically removing _atyp from function names, to make the test code simpler. * Using re.match to accommodate variable number of intermediate MvCtor. * Also removing `operator T()` from smart_holder_type_caster, to fix gcc compilation errors. The only loss is pass_rref in test_class_sh_basic. * Systematically replacing `detail::enable_if_t<...smart_holder...>` with `typename std::enable_if<...smart_holder...>::type`. Attempt to work around MSVC 2015 issues, to be tested via GitHub CI. The idea for this change originates from this comment: https://github.com/pybind/pybind11/issues/1616#issuecomment-444536813 * Importing re before pytest after observing a PyPy CI flake when importing pytest first. * Copying MSVC 2015 compatibility change from branch pr2672_use_smart_holder_as_default. * Introducing is_smart_holder_type_caster_base_tag, to keep smart_holder code more disconnected. * Working around MSVC 2015 bug. * Expanding comment for MSVC 2015 workaround. * Systematically changing std::enable_if back to detail::enable_if_t, effectively reverting commit 5d4b6890a337ae1bbaec4091f4195606f89a3b06. * Removing unused smart_holder_type_caster_load::loaded_as_rvalue_ref (it was an oversight that it was not removed with commit 23036a45eb4731a06b488ec1fdf83bca677b7f67). * Removing py::classu, because it does not seem useful enough. * Reverting commit 63495313066119dcf7510c2ae8b468b46c12ef8f by un-commenting `const` in `def_buffer(...)`. To make this possible, `operator T const&` and `operator T const*` in `smart_holder_type_caster` need to be marked as `const` member functions. * Adding construct() overloads for constructing smart_holder from alias unique_ptr, shared_ptr returns. * Adding test_class_sh_factory_constructors.cpp to tests/CMakeLists.txt (fixes oversight, this should have been added long before). * Compatibility with old clang versions (clang 3.6, 3.7 C++11). * Cleaning up changes to existing unit tests. * Systematically adding SMART_HOLDER_WIP tag. Removing minor UNTESTED tags (only the throw are not actually exercised, investing time there has a high cost but very little benefit). * Splitting out smart_holder_type_casters again, into new detail/smart_holder_type_casters_inline_include.h. * Splitting out smart_holder_init_inline_include.h. * Adding additional new include files to CMakeLists.txt, tests/extra_python_package/test_files.py. * clang-format cleanup of most smart_holder code. * Adding source code comments in response to review. * Simple micro-benchmark ("ubench") comparing runtime performance for several holders. Tested using github.com/rwgk/pybind11_scons and Google-internal build system. Sorry, no cmake support at the moment. First results: https://docs.google.com/spreadsheets/d/1InapCYws2Gt-stmFf_Bwl33eOMo3aLE_gc9adveY7RU/edit#gid=0 * Breaking out number_bucket.h, adding hook for also collecting performance data for PyCLIF. * Accounting for ubench in MANIFEST.in (simply prune, for now). * Smarter determination of call_repetitions. [skip ci] * Also scaling performance data to PyCLIF. [skip ci] * Adding ubench/python/number_bucket.clif here for general visibility. * Fix after rebase * Merging detail/smart_holder_init_inline_include.h into detail/init.h. * Renaming detail/is_smart_holder_type_caster.h -> detail/smart_holder_sfinae_hooks_only.h. * Renaming is_smart_holder_type_caster -> type_uses_smart_holder_type_caster for clarity. * Renaming type_caster_type_is_smart_holder_type_caster -> wrapped_type_uses_smart_holder_type_caster for clarity. * Renaming is_smart_holder_type_caster_base_tag -> smart_holder_type_caster_base_tag for simplicity. * Adding copyright notices and minor colateral cleanup. * iwyu cleanup (comprehensive only for cast.h and smart_holder*.h files). * Fixing `git rebase master` accident. * Moving large `pragma warning` block from pybind11.h to detail/common.h. * Fixing another `git rebase master` accident.
2021-02-24 05:50:42 +00:00
// Helpers for testing.
// These require selected functions above to work first, as indicated:
m.def("get_mtxt", get_mtxt); // pass_cref
m.def("get_ptr", get_ptr); // pass_cref
Adding py::smart_holder (for smart-pointer interoperability). (#2672) * Adding test_unique_ptr_member (for desired PyCLIF behavior). See also: https://github.com/pybind/pybind11/issues/2583 Does not build with upstream master or https://github.com/pybind/pybind11/pull/2047, but builds with https://github.com/RobotLocomotion/pybind11 and almost runs: ``` Running tests in directory "/usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests": ================================================================================= test session starts ================================================================================= platform linux -- Python 3.8.5, pytest-5.4.3, py-1.9.0, pluggy-0.13.1 rootdir: /usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests, inifile: pytest.ini collected 2 items test_unique_ptr_member.py .F [100%] ====================================================================================== FAILURES ======================================================================================= _____________________________________________________________________________ test_pointee_and_ptr_owner ______________________________________________________________________________ def test_pointee_and_ptr_owner(): obj = m.pointee() assert obj.get_int() == 213 m.ptr_owner(obj) with pytest.raises(ValueError) as exc_info: > obj.get_int() E Failed: DID NOT RAISE <class 'ValueError'> test_unique_ptr_member.py:17: Failed ============================================================================= 1 failed, 1 passed in 0.06s ============================================================================= ``` * unique_ptr or shared_ptr return * new test_variant_unique_shared with vptr_holder prototype * moving prototype code to pybind11/vptr_holder.h, adding type_caster specialization to make the bindings involving unique_ptr passing compile, but load and cast implementations are missing * disabling GitHub Actions on pull_request (for this PR) * disabling AppVeyor (for this PR) * TRIGGER_SEGSEV macro, annotations for GET_STACK (vptr::get), GET_INT_STACK (pointee) * adding test_promotion_of_disowned_to_shared * Copying tests as-is from xxx_value_ptr_xxx_holder branch. https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder Systematically exercising returning and passing unique_ptr<T>, shared_ptr<T> with unique_ptr, shared_ptr holder. Observations: test_holder_unique_ptr: make_unique_pointee OK pass_unique_pointee BUILD_FAIL (as documented) make_shared_pointee Abort free(): double free detected pass_shared_pointee RuntimeError: Unable to load a custom holder type from a default-holder instance test_holder_shared_ptr: make_unique_pointee Segmentation fault (#1138) pass_unique_pointee BUILD_FAIL (as documented) make_shared_pointee OK pass_shared_pointee OK * Copying tests as-is from xxx_value_ptr_xxx_holder branch. https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder Systematically exercising casting between shared_ptr<base>, shared_ptr<derived>. * Demonstration of Undefined Behavior in handling of shared_ptr holder. Based on https://godbolt.org/z/4fdjaW by jorgbrown@ (thanks Jorg!). * Additional demonstration of Undefined Behavior in handling of shared_ptr holder. * fixing up-down mixup in comment * Demonstration of Undefined Behavior in handling of polymorphic pointers. (This demo does NOT involve smart pointers at all, unlike the otherwise similar test_smart_ptr_private_first_base.) * minor test_private_first_base.cpp simplification (after discovering that this can be wrapped with Boost.Python, using boost::noncopyable) * pybind11 equivalent of Boost.Python test similar to reproducer under #1333 * Snapshot of WIP, TODO: shared_ptr deleter with on/off switch * Adding vptr_deleter. * Adding from/as unique_ptr<T> and unique_ptr<T, D>. * Adding from_shared_ptr. Some polishing. * New tests/core/smart_holder_poc_test.cpp, using Catch2. * Adding in vptr_deleter_guard_flag. * Improved labeling of TEST_CASEs. * Shuffling existing TEST_CASEs into systematic matrix. * Implementing all [S]uccess tests. * Implementing all [E]xception tests. * Testing of exceptions not covered by the from-as matrix. * Adding top-level comment. * Converting from methods to factory functions (no functional change). * Removing obsolete and very incomplete test (replaced by Catch2-based test). * Removing stray file. * Adding type_caster_bare_interface_demo. * Adding shared_ptr<mpty>, shared_ptr<mpty const> casters. * Adding unique_ptr<mpty>, unique_ptr<mpty const> casters. * Pure copy of `class class_` implementation in pybind11.h (master commit 98f1bbb8004f654ba9e26717bdf5912fb899b05a). * classh.h: renaming of class_ to classh + namespace; forking test_classh_wip from test_type_caster_bare_interface_demo. * Hard-coding smart_holder into classh. * Adding mpty::mtxt string member. * Adding isinstance<mpty> in type_caster::load functions. * Adding rvalue_ref, renaming const_value_ref to lvalue_ref & removing const. * Retrieving smart_holder pointer in type_caster<mpty>::load, and using it cast_op operators. * Factoring out smart_holder_type_caster_load. * Retrieving smart_holder pointer in type_caster<std::shared_ptr<mpty[ const]>>::load, and using it cast_op operators. * Improved error messaging: Cannot disown nullptr (as_unique_ptr). * Retrieving smart_holder pointer in type_caster<std::unique_ptr<mpty[ const]>>::load, and using it cast_op operators. * Pure `clang-format --style=file -i` change. * Pure `clang-format --style=file -i` change, with two `clang-format off` directives. * Fixing oversight (discovered by flake8). * flake8 cleanup * Systematically setting mtxt for all rtrn_mpty_* functions (preparation, the values are not actually used yet). * static cast handle for rtrn_cptr works by simply dropping in code from type_caster_base (marked with comments). * static cast handle for rtrn_cref works by simply dropping in code from type_caster_base (marked with comments). rtrn_mref and rtrn_mptr work via const_cast (to add const). * static cast handle for rtrn_valu works by simply dropping in code from type_caster_base (marked with comments). rtrn_rref raises a RuntimeError, to be investigated. * Copying type_caster_generic::cast into type_caster<mpty> as-is (preparation for handling smart pointers). * Pure clang-format change (applied to original type_caster_generic::cast). * Adding comment re potential use_count data race. * static handle cast implementations for rtrn_shmp, rtrn_shcp. * Adding MISSING comments in operator std::unique_ptr<mpty[ const]>. * static handle cast implementations for rtrn_uqmp, rtrn_uqcp. * Bug fix: vptr_deleter_armed_flag_ptr has to live on the heap. See new bullet point in comment section near the top. The variable was also renamed to reflect its function more accurately. * Fixing bugs discovered by ASAN. The code is now ASAN, MSAN, UBSAN clean. * Making test_type_caster_bare_interface_demo.cpp slightly more realistic, ASAN, MSAN, UBSAN clean. * Calling deregister_instance after disowning via unique_ptr. * Removing enable_shared_from_this stub, simplifying existing code, clang-format. Open question, with respect to the original code: https://github.com/pybind/pybind11/blob/76a160070b369f8d82b945c97924227e8b835c94/include/pybind11/pybind11.h#L1510 To me it looks like the exact situation marked as `std::shared_ptr<Good> gp1 = not_so_good.getptr();` here: https://en.cppreference.com/w/cpp/memory/enable_shared_from_this The comment there is: `// undefined behavior (until C++17) and std::bad_weak_ptr thrown (since C++17)` Does the existing code have UB pre C++17? I'll leave handling of enable_shared_from_this for later, as the need arises. * Cosmetical change around helper functions. * Using type_caster_base<mpty>::src_and_type directly, removing copy. Also renaming one cast to cast_const_raw_ptr, for clarity. * Fixing clang-format oversight. * Using factored-out make_constructor (PR #2798), removing duplicate code. * Inserting additional assert to ensure a returned unique_ptr is always a new Python instance. * Adding minor comment (change to internals needed to distinguish uninitialized/disowned in error message). * Factoring out find_existing_python_instance(). * Moving factored-out make_constructor to test_classh_wip.cpp, restoring previous version of cast.h. This is currently the most practical approach. See PR #2798 for background. * Copying classh type_casters from test_classh_wip.cpp UNMODIFIED, as a baseline for generalizing the code. * Using pybind11/detail/classh_type_casters.h from test_classh_wip.cpp. * Adding & using PYBIND11_CLASSH_TYPE_CASTERS define. * Adding test_classh_inheritance, currently failing (passes with class_). * Removing .clang-format before git rebase master (where the file was added). * Bringing back .clang-format, the previous rm was a bad idea. * Folding in modified_type_caster_generic_load_impl, just enough to pass test_class_wip. test_classh_inheritance is still failing, but with a different error: [RuntimeError: Incompatible type (as_raw_ptr_unowned).] * Minimal changes needed to pass test_classh_inheritance. * First pass adjusting try_implicit_casts and try_load_foreign_module_local to capture loaded_v_h, but untested and guarded with pybind11_failure("Untested"). This was done mainly to determine general feasibility. Note the TODO in pybind11.h, where type_caster_generic::local_load is currently hard-coded. test_classh_wip and test_classh_inheritance still pass, as before. * Decoupling generic_type from type_caster_generic. * Changes and tests covering classh_type_casters try_implicit_casts. * Minimal test covering classh_type_casters load_impl Case 2b. * Removing stray isinstance<T>(src): it interferes with the py::module_local feature. Adding missing #includes. * Tests for classh py::module_local() feature. * Pure renaming of function names in test_classh_inheritance, similar to the systematic approach used in test_class_wip. NO functional changes. * Pure renaming of function and variable names, for better generalization when convoluting with inheritance. NO functional changes. * Adopting systematic naming scheme from test_classh_wip. NO functional changes. * Moving const after type name, for functions that cover a systematic scheme. NO functional changes. * Adding smart_holder_type_caster_load::loaded_as_shared_ptr, currently bypassing smart_holder shared_ptr tracking completely, but the tests pass and are sanitizer clean. * Removing rtti_held from smart_holder. See updated comment. * Cleaning up loaded_as_raw_ptr_unowned, loaded_as_shared_ptr. * Factoring out convert_type and folding into loaded_as_unique_ptr. * Folding convert_type into lvalue_ref and rvalue_ref paths. Some smart_holder_type_caster_load cleanup. * Using unique_ptr in local_load to replace static variable. Also adding local_load_safety_guard. * Converting test_unique_ptr_member to using classh: fully working, ASAN, MSAN, UBSAN clean. * Removing debugging comments (GET_STACK, GET_INT_STACK). cast.h is identical to current master again, pybind11.h only has the generic_type::initialize(..., &type_caster_generic::local_load) change. * Purging obsolete pybind11/vptr_holder.h and associated test. * Moving several tests to github.com/rwgk/rwgk_tbx/tree/main/pybind11_tests https://github.com/rwgk/rwgk_tbx/commit/a2c2f88174a30f5de80d7d26e0f77c7b60f5fb85 These tests are from experimenting, and for demonstrating UB in pybind11 multiple inheritance handling ("first_base"), to be fixed later. * Adding py::smart_holder support to py::class_, purging py::classh completely. * Renaming files in include directory, creating pybind11/smart_holder.h. * Renaming all "classh" to "smart_holder" in pybind11/detail/smart_holder_type_casters.h. The user-facing macro is now PYBIND11_SMART_HOLDER_TYPE_CASTERS. * Systematically renaming tests to use "class_sh" in the name. * Renaming test_type_caster_bare_interface_demo to test_type_caster_bare_interface. * Renaming new tests/core subdirectory to tests/pure_cpp. * Adding new tests to CMake config, resetting CI config. * Changing CMake file so that test_class_sh_module_local.py actually runs. * clang-tidy fixes. * 32-bit compatibility. * Reusing type_caster_base make_copy_constructor, make_move_constructor with a trick. * CMake COMPARE NATURAL is not available with older versions. * Adding copyright notices to new header files. * Explicitly define copy/move constructors/assignments. * Adding new header files to tests/extra_python_package/test_files.py. * Adding tests/pure_cpp/CMakeLists.txt. * Making use of the new find_existing_python_instance() function factored out with PR #2822. * Moving define PYBIND11_SMART_HOLDER_TYPE_CASTERS(T) down in the file. NO functional changes. Preparation for follow-up work (to keep that diff smaller). * Reintroducing py::classh, this time as a simple alias for py::class_<U, py::smart_holder>. * Replacing detail::is_smart_holder<H> in cast.h with detail::is_smart_holder_type_caster<T>. Moving get_local_load_function_ptr, init_instance_for_type to smart_holder_type_caster_class_hooks. Expanding static_assert in py::type::handle_of<> to accommodate smart_holder_type_casters. * Fixing oversight. * Adding classu alias for class_<U, std::unique_ptr<U>>. * Giving up on idea to use legacy init_instance only if is_base_of<type_caster_generic, type_caster<T>. There are use cases in the wild that define both a custom type_caster and class_. * Removing test_type_caster_bare_interface, which was moved to the separate PR #2834. * Moving up is_smart_holder_type_caster, to also use in cast_is_temporary_value_reference. * Adding smart_holder_type_casters for unique_ptr with custom deleter. SEVERE CODE DUPLICATION. This commit is to establish a baseline for consolidating the unique_ptr code. * Unification of unique_ptr, unique_ptr_with_deleter code in smart_holder_poc.h. Leads to more fitting error messages. Enables use of unique_ptr<T, D> smart_holder_type_casters also for unique_ptr<T>. * Copying files as-is from branch test_unique_ptr_member (PR #2672). * Adding comment, simplifying naming, cmake addition. * Introducing PYBIND11_USE_SMART_HOLDER_AS_DEFAULT macro (tested only undefined; there are many errors with the macro defined). * Removing test_type_caster_bare_interface, which was moved to the separate PR #2834. * Fixing oversight introduced with commit 95425f13d6c14fcb6ee479b62b602dc8a605ec49. * Setting record.default_holder correctly for PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. With this test_class.cpp builds and even mostly runs, except `test_multiple_instances_with_same_pointer`, which segfaults because it is using a `unique_ptr` holder but `smart_holder` `type_caster`. Also adding `static_assert`s to generate build errors for such situations, but guarding with `#if 0` to first pivot to test_factory_constructors.cpp. * Fixing up cast.h and smart_holder.h after rebase. * Removing detail/smart_holder_type_casters.h in separate commit. * Commenting out const in def_buffer(... const). With this, test_buffers builds and runs with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. Explanation why the const needs to be removed, or fix elsewhere, is still needed, but left for later. * Adding test_class_sh_factory_constructors, reproducing test_factory_constructors failure. Using py::class_ in this commit, to be changed to py::classh for debugging. * Removing include/pybind11/detail/smart_holder_type_casters.h from CMakeLists.txt, test_files.py (since it does not exist in this branch). * Adding // DANGER ZONE reminders. * Converting as many py::class_ to py::classh as possible, not breaking tests. * Adding initimpl::construct() overloads, resulting in test_class_sh_factory_constructors feature parity for py::class_ and py::classh. * Adding enable_if !is_smart_holder_type_caster to existing initimpl::construct(). With this test_factory_constructors.cpp builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. * Disabling shared_ptr&, shared_ptr* tests when building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT for now, pending work on smart_holder_type_caster<shared_ptr>. * Factoring out struct and class definitions into anonymous namespace. Preparation for building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. * Simplifying from_unique_ptr(): typename D = std::default_delete<T> is not needed. Factoring out is_std_default_delete<T>() for consistentcy between ensure_compatible_rtti_uqp_del() and from_unique_ptr(). * Introducing PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. Using it in test_smart_ptr.cpp. With this test_smart_ptr builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT and all but one test run successfully. * Introducing 1. type_caster_for_class_, used in PYBIND11_MAKE_OPAQUE, and 2. default_holder_type, used in stl_bind.h. * Using __VA_ARGS__ in PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. * Replacing condense_for_macro with much simpler approach. * Softening static_assert, to only check specifically that smart_holder is not mixed with type_caster_base, and unique_ptr/shared_ptr holders are not mixed with smart_holder_type_casters. * Adding PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS in test_class.cpp (with this all but one test succeed with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT). * Adding remaining PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. static_assert for "necessary conditions" for both types of default holder, static_assert for "strict conditions" guarded by new PYBIND11_STRICT_ASSERTS_CLASS_HOLDER_VS_TYPE_CASTER_MIX. All tests build & run as before with unique_ptr as the default holder, all tests build for smart_holder as the default holder, even with the strict static_assert. * Introducing check_is_smart_holder_type_caster() function for runtime check, and reinterpreting record.default_holder as "uses_unique_ptr_holder". With this test_smart_ptr succeeds. (All 42 tests build, 35 tests succeed, 5 run but have some failures, 2 segfault.) * Bug fix: Adding have_value() to smart_holder_type_caster_load. With this test_builtin_casters succeeds. (All 42 tests build, 36 tests succeed, 5 run but have some failures, 1 segfault.) * Adding unowned_void_ptr_from_direct_conversion to modified_type_caster_generic_load_impl. This fixes the last remaining segfault (test_numpy_dtypes). New stats for all tests combined: 12 failed, 458 passed. * Adding "Lazy allocation for unallocated values" (for old-style __init__) into load_value_and_holder. Deferring destruction of disowned holder until clear_instance, to remain inspectable for "uninitialized" or "disowned" detection. New stats for all tests combined: 5 failed, 465 passed. * Changing std::shared_ptr pointer/reference to const pointer/reference. New stats for all tests combined: 4 failed, 466 passed. * Adding return_value_policy::move to permissible policies for unique_ptr returns. New stats for all tests combined: 3 failed, 467 passed. * Overlooked flake8 fixes. * Manipulating failing ConstructorStats test to pass, to be able to run all tests with ASAN. This version of the code is ASAN clean with unique_ptr or smart_holder as the default. This change needs to be reverted after adopting the existing move-only-if-refcount-is-1 logic used by type_caster_base. * Adding copy constructor and move constructor tracking to atyp. Preparation for a follow-up change in smart_holder_type_caster, to make this test sensitive to the changing behavior. [skip ci] * Removing `operator T&&() &&` from smart_holder_type_caster, for compatibility with the behavior of type_caster_base. Enables reverting 2 of 3 test manipulations applied under commit 249df7cbdb09817fed0ddf80f01ba5af12466820. The manipulation in test_factory_constructors.py is NOT reverted in this commit. [skip ci] * Fixing unfortunate editing mishap. This reverts the last remaining test manipulation in commit 249df7cbdb09817fed0ddf80f01ba5af12466820 and makes all existing unit tests pass with smart_holder as default holder. * GitHub CI clang-tidy fixes. * Adding messages to terse `static_assert`s, for pre-C++17 compatibility. * Using @pytest.mark.parametrize to run each assert separately (to see all errors, not just the first). * Systematically removing _atyp from function names, to make the test code simpler. * Using re.match to accommodate variable number of intermediate MvCtor. * Also removing `operator T()` from smart_holder_type_caster, to fix gcc compilation errors. The only loss is pass_rref in test_class_sh_basic. * Systematically replacing `detail::enable_if_t<...smart_holder...>` with `typename std::enable_if<...smart_holder...>::type`. Attempt to work around MSVC 2015 issues, to be tested via GitHub CI. The idea for this change originates from this comment: https://github.com/pybind/pybind11/issues/1616#issuecomment-444536813 * Importing re before pytest after observing a PyPy CI flake when importing pytest first. * Copying MSVC 2015 compatibility change from branch pr2672_use_smart_holder_as_default. * Introducing is_smart_holder_type_caster_base_tag, to keep smart_holder code more disconnected. * Working around MSVC 2015 bug. * Expanding comment for MSVC 2015 workaround. * Systematically changing std::enable_if back to detail::enable_if_t, effectively reverting commit 5d4b6890a337ae1bbaec4091f4195606f89a3b06. * Removing unused smart_holder_type_caster_load::loaded_as_rvalue_ref (it was an oversight that it was not removed with commit 23036a45eb4731a06b488ec1fdf83bca677b7f67). * Removing py::classu, because it does not seem useful enough. * Reverting commit 63495313066119dcf7510c2ae8b468b46c12ef8f by un-commenting `const` in `def_buffer(...)`. To make this possible, `operator T const&` and `operator T const*` in `smart_holder_type_caster` need to be marked as `const` member functions. * Adding construct() overloads for constructing smart_holder from alias unique_ptr, shared_ptr returns. * Adding test_class_sh_factory_constructors.cpp to tests/CMakeLists.txt (fixes oversight, this should have been added long before). * Compatibility with old clang versions (clang 3.6, 3.7 C++11). * Cleaning up changes to existing unit tests. * Systematically adding SMART_HOLDER_WIP tag. Removing minor UNTESTED tags (only the throw are not actually exercised, investing time there has a high cost but very little benefit). * Splitting out smart_holder_type_casters again, into new detail/smart_holder_type_casters_inline_include.h. * Splitting out smart_holder_init_inline_include.h. * Adding additional new include files to CMakeLists.txt, tests/extra_python_package/test_files.py. * clang-format cleanup of most smart_holder code. * Adding source code comments in response to review. * Simple micro-benchmark ("ubench") comparing runtime performance for several holders. Tested using github.com/rwgk/pybind11_scons and Google-internal build system. Sorry, no cmake support at the moment. First results: https://docs.google.com/spreadsheets/d/1InapCYws2Gt-stmFf_Bwl33eOMo3aLE_gc9adveY7RU/edit#gid=0 * Breaking out number_bucket.h, adding hook for also collecting performance data for PyCLIF. * Accounting for ubench in MANIFEST.in (simply prune, for now). * Smarter determination of call_repetitions. [skip ci] * Also scaling performance data to PyCLIF. [skip ci] * Adding ubench/python/number_bucket.clif here for general visibility. * Fix after rebase * Merging detail/smart_holder_init_inline_include.h into detail/init.h. * Renaming detail/is_smart_holder_type_caster.h -> detail/smart_holder_sfinae_hooks_only.h. * Renaming is_smart_holder_type_caster -> type_uses_smart_holder_type_caster for clarity. * Renaming type_caster_type_is_smart_holder_type_caster -> wrapped_type_uses_smart_holder_type_caster for clarity. * Renaming is_smart_holder_type_caster_base_tag -> smart_holder_type_caster_base_tag for simplicity. * Adding copyright notices and minor colateral cleanup. * iwyu cleanup (comprehensive only for cast.h and smart_holder*.h files). * Fixing `git rebase master` accident. * Moving large `pragma warning` block from pybind11.h to detail/common.h. * Fixing another `git rebase master` accident.
2021-02-24 05:50:42 +00:00
m.def("unique_ptr_roundtrip", unique_ptr_roundtrip); // pass_uqmp, rtrn_uqmp
m.def("pass_unique_ptr_cref", pass_unique_ptr_cref);
m.def("rtrn_unique_ptr_cref", rtrn_unique_ptr_cref);
m.def("unique_ptr_cref_roundtrip", unique_ptr_cref_roundtrip);
py::classh<SharedPtrStash>(m, "SharedPtrStash")
.def(py::init<>())
.def("Add", &SharedPtrStash::Add, py::arg("obj"));
Adding py::smart_holder (for smart-pointer interoperability). (#2672) * Adding test_unique_ptr_member (for desired PyCLIF behavior). See also: https://github.com/pybind/pybind11/issues/2583 Does not build with upstream master or https://github.com/pybind/pybind11/pull/2047, but builds with https://github.com/RobotLocomotion/pybind11 and almost runs: ``` Running tests in directory "/usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests": ================================================================================= test session starts ================================================================================= platform linux -- Python 3.8.5, pytest-5.4.3, py-1.9.0, pluggy-0.13.1 rootdir: /usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests, inifile: pytest.ini collected 2 items test_unique_ptr_member.py .F [100%] ====================================================================================== FAILURES ======================================================================================= _____________________________________________________________________________ test_pointee_and_ptr_owner ______________________________________________________________________________ def test_pointee_and_ptr_owner(): obj = m.pointee() assert obj.get_int() == 213 m.ptr_owner(obj) with pytest.raises(ValueError) as exc_info: > obj.get_int() E Failed: DID NOT RAISE <class 'ValueError'> test_unique_ptr_member.py:17: Failed ============================================================================= 1 failed, 1 passed in 0.06s ============================================================================= ``` * unique_ptr or shared_ptr return * new test_variant_unique_shared with vptr_holder prototype * moving prototype code to pybind11/vptr_holder.h, adding type_caster specialization to make the bindings involving unique_ptr passing compile, but load and cast implementations are missing * disabling GitHub Actions on pull_request (for this PR) * disabling AppVeyor (for this PR) * TRIGGER_SEGSEV macro, annotations for GET_STACK (vptr::get), GET_INT_STACK (pointee) * adding test_promotion_of_disowned_to_shared * Copying tests as-is from xxx_value_ptr_xxx_holder branch. https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder Systematically exercising returning and passing unique_ptr<T>, shared_ptr<T> with unique_ptr, shared_ptr holder. Observations: test_holder_unique_ptr: make_unique_pointee OK pass_unique_pointee BUILD_FAIL (as documented) make_shared_pointee Abort free(): double free detected pass_shared_pointee RuntimeError: Unable to load a custom holder type from a default-holder instance test_holder_shared_ptr: make_unique_pointee Segmentation fault (#1138) pass_unique_pointee BUILD_FAIL (as documented) make_shared_pointee OK pass_shared_pointee OK * Copying tests as-is from xxx_value_ptr_xxx_holder branch. https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder Systematically exercising casting between shared_ptr<base>, shared_ptr<derived>. * Demonstration of Undefined Behavior in handling of shared_ptr holder. Based on https://godbolt.org/z/4fdjaW by jorgbrown@ (thanks Jorg!). * Additional demonstration of Undefined Behavior in handling of shared_ptr holder. * fixing up-down mixup in comment * Demonstration of Undefined Behavior in handling of polymorphic pointers. (This demo does NOT involve smart pointers at all, unlike the otherwise similar test_smart_ptr_private_first_base.) * minor test_private_first_base.cpp simplification (after discovering that this can be wrapped with Boost.Python, using boost::noncopyable) * pybind11 equivalent of Boost.Python test similar to reproducer under #1333 * Snapshot of WIP, TODO: shared_ptr deleter with on/off switch * Adding vptr_deleter. * Adding from/as unique_ptr<T> and unique_ptr<T, D>. * Adding from_shared_ptr. Some polishing. * New tests/core/smart_holder_poc_test.cpp, using Catch2. * Adding in vptr_deleter_guard_flag. * Improved labeling of TEST_CASEs. * Shuffling existing TEST_CASEs into systematic matrix. * Implementing all [S]uccess tests. * Implementing all [E]xception tests. * Testing of exceptions not covered by the from-as matrix. * Adding top-level comment. * Converting from methods to factory functions (no functional change). * Removing obsolete and very incomplete test (replaced by Catch2-based test). * Removing stray file. * Adding type_caster_bare_interface_demo. * Adding shared_ptr<mpty>, shared_ptr<mpty const> casters. * Adding unique_ptr<mpty>, unique_ptr<mpty const> casters. * Pure copy of `class class_` implementation in pybind11.h (master commit 98f1bbb8004f654ba9e26717bdf5912fb899b05a). * classh.h: renaming of class_ to classh + namespace; forking test_classh_wip from test_type_caster_bare_interface_demo. * Hard-coding smart_holder into classh. * Adding mpty::mtxt string member. * Adding isinstance<mpty> in type_caster::load functions. * Adding rvalue_ref, renaming const_value_ref to lvalue_ref & removing const. * Retrieving smart_holder pointer in type_caster<mpty>::load, and using it cast_op operators. * Factoring out smart_holder_type_caster_load. * Retrieving smart_holder pointer in type_caster<std::shared_ptr<mpty[ const]>>::load, and using it cast_op operators. * Improved error messaging: Cannot disown nullptr (as_unique_ptr). * Retrieving smart_holder pointer in type_caster<std::unique_ptr<mpty[ const]>>::load, and using it cast_op operators. * Pure `clang-format --style=file -i` change. * Pure `clang-format --style=file -i` change, with two `clang-format off` directives. * Fixing oversight (discovered by flake8). * flake8 cleanup * Systematically setting mtxt for all rtrn_mpty_* functions (preparation, the values are not actually used yet). * static cast handle for rtrn_cptr works by simply dropping in code from type_caster_base (marked with comments). * static cast handle for rtrn_cref works by simply dropping in code from type_caster_base (marked with comments). rtrn_mref and rtrn_mptr work via const_cast (to add const). * static cast handle for rtrn_valu works by simply dropping in code from type_caster_base (marked with comments). rtrn_rref raises a RuntimeError, to be investigated. * Copying type_caster_generic::cast into type_caster<mpty> as-is (preparation for handling smart pointers). * Pure clang-format change (applied to original type_caster_generic::cast). * Adding comment re potential use_count data race. * static handle cast implementations for rtrn_shmp, rtrn_shcp. * Adding MISSING comments in operator std::unique_ptr<mpty[ const]>. * static handle cast implementations for rtrn_uqmp, rtrn_uqcp. * Bug fix: vptr_deleter_armed_flag_ptr has to live on the heap. See new bullet point in comment section near the top. The variable was also renamed to reflect its function more accurately. * Fixing bugs discovered by ASAN. The code is now ASAN, MSAN, UBSAN clean. * Making test_type_caster_bare_interface_demo.cpp slightly more realistic, ASAN, MSAN, UBSAN clean. * Calling deregister_instance after disowning via unique_ptr. * Removing enable_shared_from_this stub, simplifying existing code, clang-format. Open question, with respect to the original code: https://github.com/pybind/pybind11/blob/76a160070b369f8d82b945c97924227e8b835c94/include/pybind11/pybind11.h#L1510 To me it looks like the exact situation marked as `std::shared_ptr<Good> gp1 = not_so_good.getptr();` here: https://en.cppreference.com/w/cpp/memory/enable_shared_from_this The comment there is: `// undefined behavior (until C++17) and std::bad_weak_ptr thrown (since C++17)` Does the existing code have UB pre C++17? I'll leave handling of enable_shared_from_this for later, as the need arises. * Cosmetical change around helper functions. * Using type_caster_base<mpty>::src_and_type directly, removing copy. Also renaming one cast to cast_const_raw_ptr, for clarity. * Fixing clang-format oversight. * Using factored-out make_constructor (PR #2798), removing duplicate code. * Inserting additional assert to ensure a returned unique_ptr is always a new Python instance. * Adding minor comment (change to internals needed to distinguish uninitialized/disowned in error message). * Factoring out find_existing_python_instance(). * Moving factored-out make_constructor to test_classh_wip.cpp, restoring previous version of cast.h. This is currently the most practical approach. See PR #2798 for background. * Copying classh type_casters from test_classh_wip.cpp UNMODIFIED, as a baseline for generalizing the code. * Using pybind11/detail/classh_type_casters.h from test_classh_wip.cpp. * Adding & using PYBIND11_CLASSH_TYPE_CASTERS define. * Adding test_classh_inheritance, currently failing (passes with class_). * Removing .clang-format before git rebase master (where the file was added). * Bringing back .clang-format, the previous rm was a bad idea. * Folding in modified_type_caster_generic_load_impl, just enough to pass test_class_wip. test_classh_inheritance is still failing, but with a different error: [RuntimeError: Incompatible type (as_raw_ptr_unowned).] * Minimal changes needed to pass test_classh_inheritance. * First pass adjusting try_implicit_casts and try_load_foreign_module_local to capture loaded_v_h, but untested and guarded with pybind11_failure("Untested"). This was done mainly to determine general feasibility. Note the TODO in pybind11.h, where type_caster_generic::local_load is currently hard-coded. test_classh_wip and test_classh_inheritance still pass, as before. * Decoupling generic_type from type_caster_generic. * Changes and tests covering classh_type_casters try_implicit_casts. * Minimal test covering classh_type_casters load_impl Case 2b. * Removing stray isinstance<T>(src): it interferes with the py::module_local feature. Adding missing #includes. * Tests for classh py::module_local() feature. * Pure renaming of function names in test_classh_inheritance, similar to the systematic approach used in test_class_wip. NO functional changes. * Pure renaming of function and variable names, for better generalization when convoluting with inheritance. NO functional changes. * Adopting systematic naming scheme from test_classh_wip. NO functional changes. * Moving const after type name, for functions that cover a systematic scheme. NO functional changes. * Adding smart_holder_type_caster_load::loaded_as_shared_ptr, currently bypassing smart_holder shared_ptr tracking completely, but the tests pass and are sanitizer clean. * Removing rtti_held from smart_holder. See updated comment. * Cleaning up loaded_as_raw_ptr_unowned, loaded_as_shared_ptr. * Factoring out convert_type and folding into loaded_as_unique_ptr. * Folding convert_type into lvalue_ref and rvalue_ref paths. Some smart_holder_type_caster_load cleanup. * Using unique_ptr in local_load to replace static variable. Also adding local_load_safety_guard. * Converting test_unique_ptr_member to using classh: fully working, ASAN, MSAN, UBSAN clean. * Removing debugging comments (GET_STACK, GET_INT_STACK). cast.h is identical to current master again, pybind11.h only has the generic_type::initialize(..., &type_caster_generic::local_load) change. * Purging obsolete pybind11/vptr_holder.h and associated test. * Moving several tests to github.com/rwgk/rwgk_tbx/tree/main/pybind11_tests https://github.com/rwgk/rwgk_tbx/commit/a2c2f88174a30f5de80d7d26e0f77c7b60f5fb85 These tests are from experimenting, and for demonstrating UB in pybind11 multiple inheritance handling ("first_base"), to be fixed later. * Adding py::smart_holder support to py::class_, purging py::classh completely. * Renaming files in include directory, creating pybind11/smart_holder.h. * Renaming all "classh" to "smart_holder" in pybind11/detail/smart_holder_type_casters.h. The user-facing macro is now PYBIND11_SMART_HOLDER_TYPE_CASTERS. * Systematically renaming tests to use "class_sh" in the name. * Renaming test_type_caster_bare_interface_demo to test_type_caster_bare_interface. * Renaming new tests/core subdirectory to tests/pure_cpp. * Adding new tests to CMake config, resetting CI config. * Changing CMake file so that test_class_sh_module_local.py actually runs. * clang-tidy fixes. * 32-bit compatibility. * Reusing type_caster_base make_copy_constructor, make_move_constructor with a trick. * CMake COMPARE NATURAL is not available with older versions. * Adding copyright notices to new header files. * Explicitly define copy/move constructors/assignments. * Adding new header files to tests/extra_python_package/test_files.py. * Adding tests/pure_cpp/CMakeLists.txt. * Making use of the new find_existing_python_instance() function factored out with PR #2822. * Moving define PYBIND11_SMART_HOLDER_TYPE_CASTERS(T) down in the file. NO functional changes. Preparation for follow-up work (to keep that diff smaller). * Reintroducing py::classh, this time as a simple alias for py::class_<U, py::smart_holder>. * Replacing detail::is_smart_holder<H> in cast.h with detail::is_smart_holder_type_caster<T>. Moving get_local_load_function_ptr, init_instance_for_type to smart_holder_type_caster_class_hooks. Expanding static_assert in py::type::handle_of<> to accommodate smart_holder_type_casters. * Fixing oversight. * Adding classu alias for class_<U, std::unique_ptr<U>>. * Giving up on idea to use legacy init_instance only if is_base_of<type_caster_generic, type_caster<T>. There are use cases in the wild that define both a custom type_caster and class_. * Removing test_type_caster_bare_interface, which was moved to the separate PR #2834. * Moving up is_smart_holder_type_caster, to also use in cast_is_temporary_value_reference. * Adding smart_holder_type_casters for unique_ptr with custom deleter. SEVERE CODE DUPLICATION. This commit is to establish a baseline for consolidating the unique_ptr code. * Unification of unique_ptr, unique_ptr_with_deleter code in smart_holder_poc.h. Leads to more fitting error messages. Enables use of unique_ptr<T, D> smart_holder_type_casters also for unique_ptr<T>. * Copying files as-is from branch test_unique_ptr_member (PR #2672). * Adding comment, simplifying naming, cmake addition. * Introducing PYBIND11_USE_SMART_HOLDER_AS_DEFAULT macro (tested only undefined; there are many errors with the macro defined). * Removing test_type_caster_bare_interface, which was moved to the separate PR #2834. * Fixing oversight introduced with commit 95425f13d6c14fcb6ee479b62b602dc8a605ec49. * Setting record.default_holder correctly for PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. With this test_class.cpp builds and even mostly runs, except `test_multiple_instances_with_same_pointer`, which segfaults because it is using a `unique_ptr` holder but `smart_holder` `type_caster`. Also adding `static_assert`s to generate build errors for such situations, but guarding with `#if 0` to first pivot to test_factory_constructors.cpp. * Fixing up cast.h and smart_holder.h after rebase. * Removing detail/smart_holder_type_casters.h in separate commit. * Commenting out const in def_buffer(... const). With this, test_buffers builds and runs with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. Explanation why the const needs to be removed, or fix elsewhere, is still needed, but left for later. * Adding test_class_sh_factory_constructors, reproducing test_factory_constructors failure. Using py::class_ in this commit, to be changed to py::classh for debugging. * Removing include/pybind11/detail/smart_holder_type_casters.h from CMakeLists.txt, test_files.py (since it does not exist in this branch). * Adding // DANGER ZONE reminders. * Converting as many py::class_ to py::classh as possible, not breaking tests. * Adding initimpl::construct() overloads, resulting in test_class_sh_factory_constructors feature parity for py::class_ and py::classh. * Adding enable_if !is_smart_holder_type_caster to existing initimpl::construct(). With this test_factory_constructors.cpp builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. * Disabling shared_ptr&, shared_ptr* tests when building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT for now, pending work on smart_holder_type_caster<shared_ptr>. * Factoring out struct and class definitions into anonymous namespace. Preparation for building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. * Simplifying from_unique_ptr(): typename D = std::default_delete<T> is not needed. Factoring out is_std_default_delete<T>() for consistentcy between ensure_compatible_rtti_uqp_del() and from_unique_ptr(). * Introducing PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. Using it in test_smart_ptr.cpp. With this test_smart_ptr builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT and all but one test run successfully. * Introducing 1. type_caster_for_class_, used in PYBIND11_MAKE_OPAQUE, and 2. default_holder_type, used in stl_bind.h. * Using __VA_ARGS__ in PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. * Replacing condense_for_macro with much simpler approach. * Softening static_assert, to only check specifically that smart_holder is not mixed with type_caster_base, and unique_ptr/shared_ptr holders are not mixed with smart_holder_type_casters. * Adding PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS in test_class.cpp (with this all but one test succeed with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT). * Adding remaining PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. static_assert for "necessary conditions" for both types of default holder, static_assert for "strict conditions" guarded by new PYBIND11_STRICT_ASSERTS_CLASS_HOLDER_VS_TYPE_CASTER_MIX. All tests build & run as before with unique_ptr as the default holder, all tests build for smart_holder as the default holder, even with the strict static_assert. * Introducing check_is_smart_holder_type_caster() function for runtime check, and reinterpreting record.default_holder as "uses_unique_ptr_holder". With this test_smart_ptr succeeds. (All 42 tests build, 35 tests succeed, 5 run but have some failures, 2 segfault.) * Bug fix: Adding have_value() to smart_holder_type_caster_load. With this test_builtin_casters succeeds. (All 42 tests build, 36 tests succeed, 5 run but have some failures, 1 segfault.) * Adding unowned_void_ptr_from_direct_conversion to modified_type_caster_generic_load_impl. This fixes the last remaining segfault (test_numpy_dtypes). New stats for all tests combined: 12 failed, 458 passed. * Adding "Lazy allocation for unallocated values" (for old-style __init__) into load_value_and_holder. Deferring destruction of disowned holder until clear_instance, to remain inspectable for "uninitialized" or "disowned" detection. New stats for all tests combined: 5 failed, 465 passed. * Changing std::shared_ptr pointer/reference to const pointer/reference. New stats for all tests combined: 4 failed, 466 passed. * Adding return_value_policy::move to permissible policies for unique_ptr returns. New stats for all tests combined: 3 failed, 467 passed. * Overlooked flake8 fixes. * Manipulating failing ConstructorStats test to pass, to be able to run all tests with ASAN. This version of the code is ASAN clean with unique_ptr or smart_holder as the default. This change needs to be reverted after adopting the existing move-only-if-refcount-is-1 logic used by type_caster_base. * Adding copy constructor and move constructor tracking to atyp. Preparation for a follow-up change in smart_holder_type_caster, to make this test sensitive to the changing behavior. [skip ci] * Removing `operator T&&() &&` from smart_holder_type_caster, for compatibility with the behavior of type_caster_base. Enables reverting 2 of 3 test manipulations applied under commit 249df7cbdb09817fed0ddf80f01ba5af12466820. The manipulation in test_factory_constructors.py is NOT reverted in this commit. [skip ci] * Fixing unfortunate editing mishap. This reverts the last remaining test manipulation in commit 249df7cbdb09817fed0ddf80f01ba5af12466820 and makes all existing unit tests pass with smart_holder as default holder. * GitHub CI clang-tidy fixes. * Adding messages to terse `static_assert`s, for pre-C++17 compatibility. * Using @pytest.mark.parametrize to run each assert separately (to see all errors, not just the first). * Systematically removing _atyp from function names, to make the test code simpler. * Using re.match to accommodate variable number of intermediate MvCtor. * Also removing `operator T()` from smart_holder_type_caster, to fix gcc compilation errors. The only loss is pass_rref in test_class_sh_basic. * Systematically replacing `detail::enable_if_t<...smart_holder...>` with `typename std::enable_if<...smart_holder...>::type`. Attempt to work around MSVC 2015 issues, to be tested via GitHub CI. The idea for this change originates from this comment: https://github.com/pybind/pybind11/issues/1616#issuecomment-444536813 * Importing re before pytest after observing a PyPy CI flake when importing pytest first. * Copying MSVC 2015 compatibility change from branch pr2672_use_smart_holder_as_default. * Introducing is_smart_holder_type_caster_base_tag, to keep smart_holder code more disconnected. * Working around MSVC 2015 bug. * Expanding comment for MSVC 2015 workaround. * Systematically changing std::enable_if back to detail::enable_if_t, effectively reverting commit 5d4b6890a337ae1bbaec4091f4195606f89a3b06. * Removing unused smart_holder_type_caster_load::loaded_as_rvalue_ref (it was an oversight that it was not removed with commit 23036a45eb4731a06b488ec1fdf83bca677b7f67). * Removing py::classu, because it does not seem useful enough. * Reverting commit 63495313066119dcf7510c2ae8b468b46c12ef8f by un-commenting `const` in `def_buffer(...)`. To make this possible, `operator T const&` and `operator T const*` in `smart_holder_type_caster` need to be marked as `const` member functions. * Adding construct() overloads for constructing smart_holder from alias unique_ptr, shared_ptr returns. * Adding test_class_sh_factory_constructors.cpp to tests/CMakeLists.txt (fixes oversight, this should have been added long before). * Compatibility with old clang versions (clang 3.6, 3.7 C++11). * Cleaning up changes to existing unit tests. * Systematically adding SMART_HOLDER_WIP tag. Removing minor UNTESTED tags (only the throw are not actually exercised, investing time there has a high cost but very little benefit). * Splitting out smart_holder_type_casters again, into new detail/smart_holder_type_casters_inline_include.h. * Splitting out smart_holder_init_inline_include.h. * Adding additional new include files to CMakeLists.txt, tests/extra_python_package/test_files.py. * clang-format cleanup of most smart_holder code. * Adding source code comments in response to review. * Simple micro-benchmark ("ubench") comparing runtime performance for several holders. Tested using github.com/rwgk/pybind11_scons and Google-internal build system. Sorry, no cmake support at the moment. First results: https://docs.google.com/spreadsheets/d/1InapCYws2Gt-stmFf_Bwl33eOMo3aLE_gc9adveY7RU/edit#gid=0 * Breaking out number_bucket.h, adding hook for also collecting performance data for PyCLIF. * Accounting for ubench in MANIFEST.in (simply prune, for now). * Smarter determination of call_repetitions. [skip ci] * Also scaling performance data to PyCLIF. [skip ci] * Adding ubench/python/number_bucket.clif here for general visibility. * Fix after rebase * Merging detail/smart_holder_init_inline_include.h into detail/init.h. * Renaming detail/is_smart_holder_type_caster.h -> detail/smart_holder_sfinae_hooks_only.h. * Renaming is_smart_holder_type_caster -> type_uses_smart_holder_type_caster for clarity. * Renaming type_caster_type_is_smart_holder_type_caster -> wrapped_type_uses_smart_holder_type_caster for clarity. * Renaming is_smart_holder_type_caster_base_tag -> smart_holder_type_caster_base_tag for simplicity. * Adding copyright notices and minor colateral cleanup. * iwyu cleanup (comprehensive only for cast.h and smart_holder*.h files). * Fixing `git rebase master` accident. * Moving large `pragma warning` block from pybind11.h to detail/common.h. * Fixing another `git rebase master` accident.
2021-02-24 05:50:42 +00:00
m.def("py_type_handle_of_atyp", []() {
return py::type::handle_of<atyp>(); // Exercises static_cast in this function.
});
// Checks for type names used as arguments
m.def("args_shared_ptr", [](std::shared_ptr<atyp> p) { return p; });
m.def("args_shared_ptr_const", [](std::shared_ptr<atyp const> p) { return p; });
m.def("args_unique_ptr", [](std::unique_ptr<atyp> p) { return p; });
m.def("args_unique_ptr_const", [](std::unique_ptr<atyp const> p) { return p; });
// Make sure unique_ptr type caster accept automatic_reference return value policy.
m.def(
"rtrn_uq_automatic_reference",
[]() { return std::unique_ptr<atyp>(new atyp("rtrn_uq_automatic_reference")); },
pybind11::return_value_policy::automatic_reference);
[smart_holder] Bake smart_holder functionality into `class_` and `type_caster_base` (#5257) * Put bakein branch @ 18b72c0ffa6ff2747ed6c4b869a80adfb8e762c9 on top of smart_holder branch: Commands used: ``` git checkout bakein git diff smart_holder > ~/zd git checkout smart_holder git checkout -b bakein_sh patch -p 1 < ~/zd git checkout smart_holder \ MANIFEST.in \ README.rst \ README_smart_holder.rst \ docs/advanced/smart_ptrs.rst \ ubench/holder_comparison.cpp \ ubench/holder_comparison.py \ ubench/holder_comparison_extract_sheet_data.py \ ubench/number_bucket.h \ ubench/python/number_bucket.clif git add -A ``` * Add back README_smart_holder.rst in tests/extra_python_package/test_files.py * Restore smart_holder_poc.h as-is on smart_holder branch (i.e. undo `PYBIND11_SMART_HOLDER_PADDING`, which was meant for stress-testing only). * Insert `std::move()` as suggested by @laramiel * `property_cpp_function_sh_*` named specializations, as suggested by @laramiel (https://github.com/pybind/pybind11/pull/5257#discussion_r1688346807) * Call `property_cpp_function_classic` member functions, rather than inlining the implementations. * Use `PYBIND11_HAVE_INTERNALS_WITH_SMART_HOLDER_SUPPORT` in holder_comparison.cpp (holder_comparison.py is NOT changed accordingly in this commit, i.e. can still only be run if the smart_holder functionality is available). * Systematically rename `loaded_as` to `load_as` (`shared_ptr`, `unique_ptr`) as suggested by @laramiel * Make change as suggested by @laramiel. This makes it much more obvious that the latest implementation of `smart_holder_from_unique_ptr()` accepts all existing `return_value_policy` enum values except `copy`. * Resolve `BAKEIN_WIP: Rewrite comment.` for `property_cpp_function_*` specializations. * Resolve `BAKEIN_WIP: Add comment to explain: This is meant for stress-testing only.` * Resolve all remaining BAKEIN_WIP (in pybind11/cast.h). Leave only two pairs of SMART_HOLDER_BAKEIN_FOLLOW_ON comments: refactoring of copyable_holder_caster, move_only_holder_caster. This is best left until after the smart_holder branch is merged into the master branch. * Remove obsolete `using holder_type = smart_holder;` in `load_helper` * Add SMART_HOLDER_BAKEIN_FOLLOW_ON comment for `internals::default_holder` * README_smart_holder.rst update (line count reduced from 356 to 123).
2024-07-31 13:17:31 +00:00
m.def("pass_shared_ptr_ptr", [](std::shared_ptr<atyp> *) {});
py::classh<LocalUnusualOpRef>(m, "LocalUnusualOpRef");
m.def("CallCastUnusualOpRefConstRef",
[]() { return CastUnusualOpRefConstRef(LocalUnusualOpRef()); });
m.def("CallCastUnusualOpRefMovable",
[]() { return CastUnusualOpRefMovable(LocalUnusualOpRef()); });
#endif // PYBIND11_SMART_HOLDER_ENABLED
Adding py::smart_holder (for smart-pointer interoperability). (#2672) * Adding test_unique_ptr_member (for desired PyCLIF behavior). See also: https://github.com/pybind/pybind11/issues/2583 Does not build with upstream master or https://github.com/pybind/pybind11/pull/2047, but builds with https://github.com/RobotLocomotion/pybind11 and almost runs: ``` Running tests in directory "/usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests": ================================================================================= test session starts ================================================================================= platform linux -- Python 3.8.5, pytest-5.4.3, py-1.9.0, pluggy-0.13.1 rootdir: /usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests, inifile: pytest.ini collected 2 items test_unique_ptr_member.py .F [100%] ====================================================================================== FAILURES ======================================================================================= _____________________________________________________________________________ test_pointee_and_ptr_owner ______________________________________________________________________________ def test_pointee_and_ptr_owner(): obj = m.pointee() assert obj.get_int() == 213 m.ptr_owner(obj) with pytest.raises(ValueError) as exc_info: > obj.get_int() E Failed: DID NOT RAISE <class 'ValueError'> test_unique_ptr_member.py:17: Failed ============================================================================= 1 failed, 1 passed in 0.06s ============================================================================= ``` * unique_ptr or shared_ptr return * new test_variant_unique_shared with vptr_holder prototype * moving prototype code to pybind11/vptr_holder.h, adding type_caster specialization to make the bindings involving unique_ptr passing compile, but load and cast implementations are missing * disabling GitHub Actions on pull_request (for this PR) * disabling AppVeyor (for this PR) * TRIGGER_SEGSEV macro, annotations for GET_STACK (vptr::get), GET_INT_STACK (pointee) * adding test_promotion_of_disowned_to_shared * Copying tests as-is from xxx_value_ptr_xxx_holder branch. https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder Systematically exercising returning and passing unique_ptr<T>, shared_ptr<T> with unique_ptr, shared_ptr holder. Observations: test_holder_unique_ptr: make_unique_pointee OK pass_unique_pointee BUILD_FAIL (as documented) make_shared_pointee Abort free(): double free detected pass_shared_pointee RuntimeError: Unable to load a custom holder type from a default-holder instance test_holder_shared_ptr: make_unique_pointee Segmentation fault (#1138) pass_unique_pointee BUILD_FAIL (as documented) make_shared_pointee OK pass_shared_pointee OK * Copying tests as-is from xxx_value_ptr_xxx_holder branch. https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder Systematically exercising casting between shared_ptr<base>, shared_ptr<derived>. * Demonstration of Undefined Behavior in handling of shared_ptr holder. Based on https://godbolt.org/z/4fdjaW by jorgbrown@ (thanks Jorg!). * Additional demonstration of Undefined Behavior in handling of shared_ptr holder. * fixing up-down mixup in comment * Demonstration of Undefined Behavior in handling of polymorphic pointers. (This demo does NOT involve smart pointers at all, unlike the otherwise similar test_smart_ptr_private_first_base.) * minor test_private_first_base.cpp simplification (after discovering that this can be wrapped with Boost.Python, using boost::noncopyable) * pybind11 equivalent of Boost.Python test similar to reproducer under #1333 * Snapshot of WIP, TODO: shared_ptr deleter with on/off switch * Adding vptr_deleter. * Adding from/as unique_ptr<T> and unique_ptr<T, D>. * Adding from_shared_ptr. Some polishing. * New tests/core/smart_holder_poc_test.cpp, using Catch2. * Adding in vptr_deleter_guard_flag. * Improved labeling of TEST_CASEs. * Shuffling existing TEST_CASEs into systematic matrix. * Implementing all [S]uccess tests. * Implementing all [E]xception tests. * Testing of exceptions not covered by the from-as matrix. * Adding top-level comment. * Converting from methods to factory functions (no functional change). * Removing obsolete and very incomplete test (replaced by Catch2-based test). * Removing stray file. * Adding type_caster_bare_interface_demo. * Adding shared_ptr<mpty>, shared_ptr<mpty const> casters. * Adding unique_ptr<mpty>, unique_ptr<mpty const> casters. * Pure copy of `class class_` implementation in pybind11.h (master commit 98f1bbb8004f654ba9e26717bdf5912fb899b05a). * classh.h: renaming of class_ to classh + namespace; forking test_classh_wip from test_type_caster_bare_interface_demo. * Hard-coding smart_holder into classh. * Adding mpty::mtxt string member. * Adding isinstance<mpty> in type_caster::load functions. * Adding rvalue_ref, renaming const_value_ref to lvalue_ref & removing const. * Retrieving smart_holder pointer in type_caster<mpty>::load, and using it cast_op operators. * Factoring out smart_holder_type_caster_load. * Retrieving smart_holder pointer in type_caster<std::shared_ptr<mpty[ const]>>::load, and using it cast_op operators. * Improved error messaging: Cannot disown nullptr (as_unique_ptr). * Retrieving smart_holder pointer in type_caster<std::unique_ptr<mpty[ const]>>::load, and using it cast_op operators. * Pure `clang-format --style=file -i` change. * Pure `clang-format --style=file -i` change, with two `clang-format off` directives. * Fixing oversight (discovered by flake8). * flake8 cleanup * Systematically setting mtxt for all rtrn_mpty_* functions (preparation, the values are not actually used yet). * static cast handle for rtrn_cptr works by simply dropping in code from type_caster_base (marked with comments). * static cast handle for rtrn_cref works by simply dropping in code from type_caster_base (marked with comments). rtrn_mref and rtrn_mptr work via const_cast (to add const). * static cast handle for rtrn_valu works by simply dropping in code from type_caster_base (marked with comments). rtrn_rref raises a RuntimeError, to be investigated. * Copying type_caster_generic::cast into type_caster<mpty> as-is (preparation for handling smart pointers). * Pure clang-format change (applied to original type_caster_generic::cast). * Adding comment re potential use_count data race. * static handle cast implementations for rtrn_shmp, rtrn_shcp. * Adding MISSING comments in operator std::unique_ptr<mpty[ const]>. * static handle cast implementations for rtrn_uqmp, rtrn_uqcp. * Bug fix: vptr_deleter_armed_flag_ptr has to live on the heap. See new bullet point in comment section near the top. The variable was also renamed to reflect its function more accurately. * Fixing bugs discovered by ASAN. The code is now ASAN, MSAN, UBSAN clean. * Making test_type_caster_bare_interface_demo.cpp slightly more realistic, ASAN, MSAN, UBSAN clean. * Calling deregister_instance after disowning via unique_ptr. * Removing enable_shared_from_this stub, simplifying existing code, clang-format. Open question, with respect to the original code: https://github.com/pybind/pybind11/blob/76a160070b369f8d82b945c97924227e8b835c94/include/pybind11/pybind11.h#L1510 To me it looks like the exact situation marked as `std::shared_ptr<Good> gp1 = not_so_good.getptr();` here: https://en.cppreference.com/w/cpp/memory/enable_shared_from_this The comment there is: `// undefined behavior (until C++17) and std::bad_weak_ptr thrown (since C++17)` Does the existing code have UB pre C++17? I'll leave handling of enable_shared_from_this for later, as the need arises. * Cosmetical change around helper functions. * Using type_caster_base<mpty>::src_and_type directly, removing copy. Also renaming one cast to cast_const_raw_ptr, for clarity. * Fixing clang-format oversight. * Using factored-out make_constructor (PR #2798), removing duplicate code. * Inserting additional assert to ensure a returned unique_ptr is always a new Python instance. * Adding minor comment (change to internals needed to distinguish uninitialized/disowned in error message). * Factoring out find_existing_python_instance(). * Moving factored-out make_constructor to test_classh_wip.cpp, restoring previous version of cast.h. This is currently the most practical approach. See PR #2798 for background. * Copying classh type_casters from test_classh_wip.cpp UNMODIFIED, as a baseline for generalizing the code. * Using pybind11/detail/classh_type_casters.h from test_classh_wip.cpp. * Adding & using PYBIND11_CLASSH_TYPE_CASTERS define. * Adding test_classh_inheritance, currently failing (passes with class_). * Removing .clang-format before git rebase master (where the file was added). * Bringing back .clang-format, the previous rm was a bad idea. * Folding in modified_type_caster_generic_load_impl, just enough to pass test_class_wip. test_classh_inheritance is still failing, but with a different error: [RuntimeError: Incompatible type (as_raw_ptr_unowned).] * Minimal changes needed to pass test_classh_inheritance. * First pass adjusting try_implicit_casts and try_load_foreign_module_local to capture loaded_v_h, but untested and guarded with pybind11_failure("Untested"). This was done mainly to determine general feasibility. Note the TODO in pybind11.h, where type_caster_generic::local_load is currently hard-coded. test_classh_wip and test_classh_inheritance still pass, as before. * Decoupling generic_type from type_caster_generic. * Changes and tests covering classh_type_casters try_implicit_casts. * Minimal test covering classh_type_casters load_impl Case 2b. * Removing stray isinstance<T>(src): it interferes with the py::module_local feature. Adding missing #includes. * Tests for classh py::module_local() feature. * Pure renaming of function names in test_classh_inheritance, similar to the systematic approach used in test_class_wip. NO functional changes. * Pure renaming of function and variable names, for better generalization when convoluting with inheritance. NO functional changes. * Adopting systematic naming scheme from test_classh_wip. NO functional changes. * Moving const after type name, for functions that cover a systematic scheme. NO functional changes. * Adding smart_holder_type_caster_load::loaded_as_shared_ptr, currently bypassing smart_holder shared_ptr tracking completely, but the tests pass and are sanitizer clean. * Removing rtti_held from smart_holder. See updated comment. * Cleaning up loaded_as_raw_ptr_unowned, loaded_as_shared_ptr. * Factoring out convert_type and folding into loaded_as_unique_ptr. * Folding convert_type into lvalue_ref and rvalue_ref paths. Some smart_holder_type_caster_load cleanup. * Using unique_ptr in local_load to replace static variable. Also adding local_load_safety_guard. * Converting test_unique_ptr_member to using classh: fully working, ASAN, MSAN, UBSAN clean. * Removing debugging comments (GET_STACK, GET_INT_STACK). cast.h is identical to current master again, pybind11.h only has the generic_type::initialize(..., &type_caster_generic::local_load) change. * Purging obsolete pybind11/vptr_holder.h and associated test. * Moving several tests to github.com/rwgk/rwgk_tbx/tree/main/pybind11_tests https://github.com/rwgk/rwgk_tbx/commit/a2c2f88174a30f5de80d7d26e0f77c7b60f5fb85 These tests are from experimenting, and for demonstrating UB in pybind11 multiple inheritance handling ("first_base"), to be fixed later. * Adding py::smart_holder support to py::class_, purging py::classh completely. * Renaming files in include directory, creating pybind11/smart_holder.h. * Renaming all "classh" to "smart_holder" in pybind11/detail/smart_holder_type_casters.h. The user-facing macro is now PYBIND11_SMART_HOLDER_TYPE_CASTERS. * Systematically renaming tests to use "class_sh" in the name. * Renaming test_type_caster_bare_interface_demo to test_type_caster_bare_interface. * Renaming new tests/core subdirectory to tests/pure_cpp. * Adding new tests to CMake config, resetting CI config. * Changing CMake file so that test_class_sh_module_local.py actually runs. * clang-tidy fixes. * 32-bit compatibility. * Reusing type_caster_base make_copy_constructor, make_move_constructor with a trick. * CMake COMPARE NATURAL is not available with older versions. * Adding copyright notices to new header files. * Explicitly define copy/move constructors/assignments. * Adding new header files to tests/extra_python_package/test_files.py. * Adding tests/pure_cpp/CMakeLists.txt. * Making use of the new find_existing_python_instance() function factored out with PR #2822. * Moving define PYBIND11_SMART_HOLDER_TYPE_CASTERS(T) down in the file. NO functional changes. Preparation for follow-up work (to keep that diff smaller). * Reintroducing py::classh, this time as a simple alias for py::class_<U, py::smart_holder>. * Replacing detail::is_smart_holder<H> in cast.h with detail::is_smart_holder_type_caster<T>. Moving get_local_load_function_ptr, init_instance_for_type to smart_holder_type_caster_class_hooks. Expanding static_assert in py::type::handle_of<> to accommodate smart_holder_type_casters. * Fixing oversight. * Adding classu alias for class_<U, std::unique_ptr<U>>. * Giving up on idea to use legacy init_instance only if is_base_of<type_caster_generic, type_caster<T>. There are use cases in the wild that define both a custom type_caster and class_. * Removing test_type_caster_bare_interface, which was moved to the separate PR #2834. * Moving up is_smart_holder_type_caster, to also use in cast_is_temporary_value_reference. * Adding smart_holder_type_casters for unique_ptr with custom deleter. SEVERE CODE DUPLICATION. This commit is to establish a baseline for consolidating the unique_ptr code. * Unification of unique_ptr, unique_ptr_with_deleter code in smart_holder_poc.h. Leads to more fitting error messages. Enables use of unique_ptr<T, D> smart_holder_type_casters also for unique_ptr<T>. * Copying files as-is from branch test_unique_ptr_member (PR #2672). * Adding comment, simplifying naming, cmake addition. * Introducing PYBIND11_USE_SMART_HOLDER_AS_DEFAULT macro (tested only undefined; there are many errors with the macro defined). * Removing test_type_caster_bare_interface, which was moved to the separate PR #2834. * Fixing oversight introduced with commit 95425f13d6c14fcb6ee479b62b602dc8a605ec49. * Setting record.default_holder correctly for PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. With this test_class.cpp builds and even mostly runs, except `test_multiple_instances_with_same_pointer`, which segfaults because it is using a `unique_ptr` holder but `smart_holder` `type_caster`. Also adding `static_assert`s to generate build errors for such situations, but guarding with `#if 0` to first pivot to test_factory_constructors.cpp. * Fixing up cast.h and smart_holder.h after rebase. * Removing detail/smart_holder_type_casters.h in separate commit. * Commenting out const in def_buffer(... const). With this, test_buffers builds and runs with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. Explanation why the const needs to be removed, or fix elsewhere, is still needed, but left for later. * Adding test_class_sh_factory_constructors, reproducing test_factory_constructors failure. Using py::class_ in this commit, to be changed to py::classh for debugging. * Removing include/pybind11/detail/smart_holder_type_casters.h from CMakeLists.txt, test_files.py (since it does not exist in this branch). * Adding // DANGER ZONE reminders. * Converting as many py::class_ to py::classh as possible, not breaking tests. * Adding initimpl::construct() overloads, resulting in test_class_sh_factory_constructors feature parity for py::class_ and py::classh. * Adding enable_if !is_smart_holder_type_caster to existing initimpl::construct(). With this test_factory_constructors.cpp builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. * Disabling shared_ptr&, shared_ptr* tests when building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT for now, pending work on smart_holder_type_caster<shared_ptr>. * Factoring out struct and class definitions into anonymous namespace. Preparation for building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. * Simplifying from_unique_ptr(): typename D = std::default_delete<T> is not needed. Factoring out is_std_default_delete<T>() for consistentcy between ensure_compatible_rtti_uqp_del() and from_unique_ptr(). * Introducing PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. Using it in test_smart_ptr.cpp. With this test_smart_ptr builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT and all but one test run successfully. * Introducing 1. type_caster_for_class_, used in PYBIND11_MAKE_OPAQUE, and 2. default_holder_type, used in stl_bind.h. * Using __VA_ARGS__ in PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. * Replacing condense_for_macro with much simpler approach. * Softening static_assert, to only check specifically that smart_holder is not mixed with type_caster_base, and unique_ptr/shared_ptr holders are not mixed with smart_holder_type_casters. * Adding PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS in test_class.cpp (with this all but one test succeed with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT). * Adding remaining PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. static_assert for "necessary conditions" for both types of default holder, static_assert for "strict conditions" guarded by new PYBIND11_STRICT_ASSERTS_CLASS_HOLDER_VS_TYPE_CASTER_MIX. All tests build & run as before with unique_ptr as the default holder, all tests build for smart_holder as the default holder, even with the strict static_assert. * Introducing check_is_smart_holder_type_caster() function for runtime check, and reinterpreting record.default_holder as "uses_unique_ptr_holder". With this test_smart_ptr succeeds. (All 42 tests build, 35 tests succeed, 5 run but have some failures, 2 segfault.) * Bug fix: Adding have_value() to smart_holder_type_caster_load. With this test_builtin_casters succeeds. (All 42 tests build, 36 tests succeed, 5 run but have some failures, 1 segfault.) * Adding unowned_void_ptr_from_direct_conversion to modified_type_caster_generic_load_impl. This fixes the last remaining segfault (test_numpy_dtypes). New stats for all tests combined: 12 failed, 458 passed. * Adding "Lazy allocation for unallocated values" (for old-style __init__) into load_value_and_holder. Deferring destruction of disowned holder until clear_instance, to remain inspectable for "uninitialized" or "disowned" detection. New stats for all tests combined: 5 failed, 465 passed. * Changing std::shared_ptr pointer/reference to const pointer/reference. New stats for all tests combined: 4 failed, 466 passed. * Adding return_value_policy::move to permissible policies for unique_ptr returns. New stats for all tests combined: 3 failed, 467 passed. * Overlooked flake8 fixes. * Manipulating failing ConstructorStats test to pass, to be able to run all tests with ASAN. This version of the code is ASAN clean with unique_ptr or smart_holder as the default. This change needs to be reverted after adopting the existing move-only-if-refcount-is-1 logic used by type_caster_base. * Adding copy constructor and move constructor tracking to atyp. Preparation for a follow-up change in smart_holder_type_caster, to make this test sensitive to the changing behavior. [skip ci] * Removing `operator T&&() &&` from smart_holder_type_caster, for compatibility with the behavior of type_caster_base. Enables reverting 2 of 3 test manipulations applied under commit 249df7cbdb09817fed0ddf80f01ba5af12466820. The manipulation in test_factory_constructors.py is NOT reverted in this commit. [skip ci] * Fixing unfortunate editing mishap. This reverts the last remaining test manipulation in commit 249df7cbdb09817fed0ddf80f01ba5af12466820 and makes all existing unit tests pass with smart_holder as default holder. * GitHub CI clang-tidy fixes. * Adding messages to terse `static_assert`s, for pre-C++17 compatibility. * Using @pytest.mark.parametrize to run each assert separately (to see all errors, not just the first). * Systematically removing _atyp from function names, to make the test code simpler. * Using re.match to accommodate variable number of intermediate MvCtor. * Also removing `operator T()` from smart_holder_type_caster, to fix gcc compilation errors. The only loss is pass_rref in test_class_sh_basic. * Systematically replacing `detail::enable_if_t<...smart_holder...>` with `typename std::enable_if<...smart_holder...>::type`. Attempt to work around MSVC 2015 issues, to be tested via GitHub CI. The idea for this change originates from this comment: https://github.com/pybind/pybind11/issues/1616#issuecomment-444536813 * Importing re before pytest after observing a PyPy CI flake when importing pytest first. * Copying MSVC 2015 compatibility change from branch pr2672_use_smart_holder_as_default. * Introducing is_smart_holder_type_caster_base_tag, to keep smart_holder code more disconnected. * Working around MSVC 2015 bug. * Expanding comment for MSVC 2015 workaround. * Systematically changing std::enable_if back to detail::enable_if_t, effectively reverting commit 5d4b6890a337ae1bbaec4091f4195606f89a3b06. * Removing unused smart_holder_type_caster_load::loaded_as_rvalue_ref (it was an oversight that it was not removed with commit 23036a45eb4731a06b488ec1fdf83bca677b7f67). * Removing py::classu, because it does not seem useful enough. * Reverting commit 63495313066119dcf7510c2ae8b468b46c12ef8f by un-commenting `const` in `def_buffer(...)`. To make this possible, `operator T const&` and `operator T const*` in `smart_holder_type_caster` need to be marked as `const` member functions. * Adding construct() overloads for constructing smart_holder from alias unique_ptr, shared_ptr returns. * Adding test_class_sh_factory_constructors.cpp to tests/CMakeLists.txt (fixes oversight, this should have been added long before). * Compatibility with old clang versions (clang 3.6, 3.7 C++11). * Cleaning up changes to existing unit tests. * Systematically adding SMART_HOLDER_WIP tag. Removing minor UNTESTED tags (only the throw are not actually exercised, investing time there has a high cost but very little benefit). * Splitting out smart_holder_type_casters again, into new detail/smart_holder_type_casters_inline_include.h. * Splitting out smart_holder_init_inline_include.h. * Adding additional new include files to CMakeLists.txt, tests/extra_python_package/test_files.py. * clang-format cleanup of most smart_holder code. * Adding source code comments in response to review. * Simple micro-benchmark ("ubench") comparing runtime performance for several holders. Tested using github.com/rwgk/pybind11_scons and Google-internal build system. Sorry, no cmake support at the moment. First results: https://docs.google.com/spreadsheets/d/1InapCYws2Gt-stmFf_Bwl33eOMo3aLE_gc9adveY7RU/edit#gid=0 * Breaking out number_bucket.h, adding hook for also collecting performance data for PyCLIF. * Accounting for ubench in MANIFEST.in (simply prune, for now). * Smarter determination of call_repetitions. [skip ci] * Also scaling performance data to PyCLIF. [skip ci] * Adding ubench/python/number_bucket.clif here for general visibility. * Fix after rebase * Merging detail/smart_holder_init_inline_include.h into detail/init.h. * Renaming detail/is_smart_holder_type_caster.h -> detail/smart_holder_sfinae_hooks_only.h. * Renaming is_smart_holder_type_caster -> type_uses_smart_holder_type_caster for clarity. * Renaming type_caster_type_is_smart_holder_type_caster -> wrapped_type_uses_smart_holder_type_caster for clarity. * Renaming is_smart_holder_type_caster_base_tag -> smart_holder_type_caster_base_tag for simplicity. * Adding copyright notices and minor colateral cleanup. * iwyu cleanup (comprehensive only for cast.h and smart_holder*.h files). * Fixing `git rebase master` accident. * Moving large `pragma warning` block from pybind11.h to detail/common.h. * Fixing another `git rebase master` accident.
2021-02-24 05:50:42 +00:00
}
} // namespace class_sh_basic
} // namespace pybind11_tests