* Manual line breaks to pre-empt undesired `clang-format`ing.
Informed by work under https://github.com/pybind/pybind11/pull/3683:
60b7eb410f59572e6559
* Manual curation of clang-format diffs involving source code comments.
Very labor-intensive and dull.
* Pulling .clang-format change from @henryiii's 9057962d40
* Adding commonly used .clang-format `CommentPragmas:`
* Ensure short lambdas are allowed
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
* Adding readability-qualified-auto to .clang-tidy
Ported from @henryiii's 287527f705
* fix: support Python < 3.6
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
* Add `.keys` and `.values` to bind_map
Both of these implement views (rather than just iterators), and `.items`
is also upgraded to a view. In practical terms, this allows a view to be
iterated multiple times and have its size taken, neither of which works
with an iterator.
The views implement `__len__`, `__iter__`, and the keys view implements
`__contains__`. Testing membership also works in item and value views
because Python falls back to iteration. This won't be optimal
for item values since it's linear rather than O(log n) or O(1), but I
didn't fancy trying to get all the corner cases to match Python
behaviour (tuple of wrong types, wrong length tuple, not a tuple etc).
Missing relative to Python dictionary views is `__reversed__` (only
added to Python in 3.8). Implementing that could break code that binds
custom map classes which don't provide `rbegin`/`rend` (at least without
doing clever things with SFINAE), so I've not tried.
The size increase on my system is 131072 bytes, which is rather large
(5%) but also suspiciously round (2^17) and makes me suspect some
quantisation effect.
* bind_map: support any object in __contains__
Add extra overload of `__contains__` (for both the map itself and
KeysView) which takes an arbitrary object and returns false.
* Take py::object by const reference in __contains__
To keep clang-tidy happy.
* Removing stray `py::` (detected via interactive testing in Google environment).
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
* Adding test_unique_ptr_member (for desired PyCLIF behavior).
See also: https://github.com/pybind/pybind11/issues/2583
Does not build with upstream master or
https://github.com/pybind/pybind11/pull/2047, but builds with
https://github.com/RobotLocomotion/pybind11 and almost runs:
```
Running tests in directory "/usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests":
================================================================================= test session starts =================================================================================
platform linux -- Python 3.8.5, pytest-5.4.3, py-1.9.0, pluggy-0.13.1
rootdir: /usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests, inifile: pytest.ini
collected 2 items
test_unique_ptr_member.py .F [100%]
====================================================================================== FAILURES =======================================================================================
_____________________________________________________________________________ test_pointee_and_ptr_owner ______________________________________________________________________________
def test_pointee_and_ptr_owner():
obj = m.pointee()
assert obj.get_int() == 213
m.ptr_owner(obj)
with pytest.raises(ValueError) as exc_info:
> obj.get_int()
E Failed: DID NOT RAISE <class 'ValueError'>
test_unique_ptr_member.py:17: Failed
============================================================================= 1 failed, 1 passed in 0.06s =============================================================================
```
* unique_ptr or shared_ptr return
* new test_variant_unique_shared with vptr_holder prototype
* moving prototype code to pybind11/vptr_holder.h, adding type_caster specialization to make the bindings involving unique_ptr passing compile, but load and cast implementations are missing
* disabling GitHub Actions on pull_request (for this PR)
* disabling AppVeyor (for this PR)
* TRIGGER_SEGSEV macro, annotations for GET_STACK (vptr::get), GET_INT_STACK (pointee)
* adding test_promotion_of_disowned_to_shared
* Copying tests as-is from xxx_value_ptr_xxx_holder branch.
https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder
Systematically exercising returning and passing unique_ptr<T>, shared_ptr<T>
with unique_ptr, shared_ptr holder.
Observations:
test_holder_unique_ptr:
make_unique_pointee OK
pass_unique_pointee BUILD_FAIL (as documented)
make_shared_pointee Abort free(): double free detected
pass_shared_pointee RuntimeError: Unable to load a custom holder type from a default-holder instance
test_holder_shared_ptr:
make_unique_pointee Segmentation fault (#1138)
pass_unique_pointee BUILD_FAIL (as documented)
make_shared_pointee OK
pass_shared_pointee OK
* Copying tests as-is from xxx_value_ptr_xxx_holder branch.
https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder
Systematically exercising casting between shared_ptr<base>, shared_ptr<derived>.
* Demonstration of Undefined Behavior in handling of shared_ptr holder.
Based on https://godbolt.org/z/4fdjaW by jorgbrown@ (thanks Jorg!).
* Additional demonstration of Undefined Behavior in handling of shared_ptr holder.
* fixing up-down mixup in comment
* Demonstration of Undefined Behavior in handling of polymorphic pointers.
(This demo does NOT involve smart pointers at all, unlike the otherwise similar test_smart_ptr_private_first_base.)
* minor test_private_first_base.cpp simplification (after discovering that this can be wrapped with Boost.Python, using boost::noncopyable)
* pybind11 equivalent of Boost.Python test similar to reproducer under #1333
* Snapshot of WIP, TODO: shared_ptr deleter with on/off switch
* Adding vptr_deleter.
* Adding from/as unique_ptr<T> and unique_ptr<T, D>.
* Adding from_shared_ptr. Some polishing.
* New tests/core/smart_holder_poc_test.cpp, using Catch2.
* Adding in vptr_deleter_guard_flag.
* Improved labeling of TEST_CASEs.
* Shuffling existing TEST_CASEs into systematic matrix.
* Implementing all [S]uccess tests.
* Implementing all [E]xception tests.
* Testing of exceptions not covered by the from-as matrix.
* Adding top-level comment.
* Converting from methods to factory functions (no functional change).
* Removing obsolete and very incomplete test (replaced by Catch2-based test).
* Removing stray file.
* Adding type_caster_bare_interface_demo.
* Adding shared_ptr<mpty>, shared_ptr<mpty const> casters.
* Adding unique_ptr<mpty>, unique_ptr<mpty const> casters.
* Pure copy of `class class_` implementation in pybind11.h (master commit 98f1bbb800).
* classh.h: renaming of class_ to classh + namespace; forking test_classh_wip from test_type_caster_bare_interface_demo.
* Hard-coding smart_holder into classh.
* Adding mpty::mtxt string member.
* Adding isinstance<mpty> in type_caster::load functions.
* Adding rvalue_ref, renaming const_value_ref to lvalue_ref & removing const.
* Retrieving smart_holder pointer in type_caster<mpty>::load, and using it cast_op operators.
* Factoring out smart_holder_type_caster_load.
* Retrieving smart_holder pointer in type_caster<std::shared_ptr<mpty[ const]>>::load, and using it cast_op operators.
* Improved error messaging: Cannot disown nullptr (as_unique_ptr).
* Retrieving smart_holder pointer in type_caster<std::unique_ptr<mpty[ const]>>::load, and using it cast_op operators.
* Pure `clang-format --style=file -i` change.
* Pure `clang-format --style=file -i` change, with two `clang-format off` directives.
* Fixing oversight (discovered by flake8).
* flake8 cleanup
* Systematically setting mtxt for all rtrn_mpty_* functions (preparation, the values are not actually used yet).
* static cast handle for rtrn_cptr works by simply dropping in code from type_caster_base (marked with comments).
* static cast handle for rtrn_cref works by simply dropping in code from type_caster_base (marked with comments). rtrn_mref and rtrn_mptr work via const_cast (to add const).
* static cast handle for rtrn_valu works by simply dropping in code from type_caster_base (marked with comments). rtrn_rref raises a RuntimeError, to be investigated.
* Copying type_caster_generic::cast into type_caster<mpty> as-is (preparation for handling smart pointers).
* Pure clang-format change (applied to original type_caster_generic::cast).
* Adding comment re potential use_count data race.
* static handle cast implementations for rtrn_shmp, rtrn_shcp.
* Adding MISSING comments in operator std::unique_ptr<mpty[ const]>.
* static handle cast implementations for rtrn_uqmp, rtrn_uqcp.
* Bug fix: vptr_deleter_armed_flag_ptr has to live on the heap.
See new bullet point in comment section near the top.
The variable was also renamed to reflect its function more accurately.
* Fixing bugs discovered by ASAN. The code is now ASAN, MSAN, UBSAN clean.
* Making test_type_caster_bare_interface_demo.cpp slightly more realistic, ASAN, MSAN, UBSAN clean.
* Calling deregister_instance after disowning via unique_ptr.
* Removing enable_shared_from_this stub, simplifying existing code, clang-format.
Open question, with respect to the original code:
76a160070b/include/pybind11/pybind11.h (L1510)
To me it looks like the exact situation marked as `std::shared_ptr<Good> gp1 = not_so_good.getptr();` here: https://en.cppreference.com/w/cpp/memory/enable_shared_from_this
The comment there is: `// undefined behavior (until C++17) and std::bad_weak_ptr thrown (since C++17)`
Does the existing code have UB pre C++17?
I'll leave handling of enable_shared_from_this for later, as the need arises.
* Cosmetical change around helper functions.
* Using type_caster_base<mpty>::src_and_type directly, removing copy. Also renaming one cast to cast_const_raw_ptr, for clarity.
* Fixing clang-format oversight.
* Using factored-out make_constructor (PR #2798), removing duplicate code.
* Inserting additional assert to ensure a returned unique_ptr is always a new Python instance.
* Adding minor comment (change to internals needed to distinguish uninitialized/disowned in error message).
* Factoring out find_existing_python_instance().
* Moving factored-out make_constructor to test_classh_wip.cpp, restoring previous version of cast.h. This is currently the most practical approach. See PR #2798 for background.
* Copying classh type_casters from test_classh_wip.cpp UNMODIFIED, as a baseline for generalizing the code.
* Using pybind11/detail/classh_type_casters.h from test_classh_wip.cpp.
* Adding & using PYBIND11_CLASSH_TYPE_CASTERS define.
* Adding test_classh_inheritance, currently failing (passes with class_).
* Removing .clang-format before git rebase master (where the file was added).
* Bringing back .clang-format, the previous rm was a bad idea.
* Folding in modified_type_caster_generic_load_impl, just enough to pass test_class_wip. test_classh_inheritance is still failing, but with a different error: [RuntimeError: Incompatible type (as_raw_ptr_unowned).]
* Minimal changes needed to pass test_classh_inheritance.
* First pass adjusting try_implicit_casts and try_load_foreign_module_local to capture loaded_v_h, but untested and guarded with pybind11_failure("Untested"). This was done mainly to determine general feasibility. Note the TODO in pybind11.h, where type_caster_generic::local_load is currently hard-coded. test_classh_wip and test_classh_inheritance still pass, as before.
* Decoupling generic_type from type_caster_generic.
* Changes and tests covering classh_type_casters try_implicit_casts.
* Minimal test covering classh_type_casters load_impl Case 2b.
* Removing stray isinstance<T>(src): it interferes with the py::module_local feature. Adding missing #includes.
* Tests for classh py::module_local() feature.
* Pure renaming of function names in test_classh_inheritance, similar to the systematic approach used in test_class_wip. NO functional changes.
* Pure renaming of function and variable names, for better generalization when convoluting with inheritance. NO functional changes.
* Adopting systematic naming scheme from test_classh_wip. NO functional changes.
* Moving const after type name, for functions that cover a systematic scheme. NO functional changes.
* Adding smart_holder_type_caster_load::loaded_as_shared_ptr, currently bypassing smart_holder shared_ptr tracking completely, but the tests pass and are sanitizer clean.
* Removing rtti_held from smart_holder. See updated comment.
* Cleaning up loaded_as_raw_ptr_unowned, loaded_as_shared_ptr.
* Factoring out convert_type and folding into loaded_as_unique_ptr.
* Folding convert_type into lvalue_ref and rvalue_ref paths. Some smart_holder_type_caster_load cleanup.
* Using unique_ptr in local_load to replace static variable. Also adding local_load_safety_guard.
* Converting test_unique_ptr_member to using classh: fully working, ASAN, MSAN, UBSAN clean.
* Removing debugging comments (GET_STACK, GET_INT_STACK). cast.h is identical to current master again, pybind11.h only has the generic_type::initialize(..., &type_caster_generic::local_load) change.
* Purging obsolete pybind11/vptr_holder.h and associated test.
* Moving several tests to github.com/rwgk/rwgk_tbx/tree/main/pybind11_tests
a2c2f88174
These tests are from experimenting, and for demonstrating UB in pybind11 multiple inheritance handling ("first_base"), to be fixed later.
* Adding py::smart_holder support to py::class_, purging py::classh completely.
* Renaming files in include directory, creating pybind11/smart_holder.h.
* Renaming all "classh" to "smart_holder" in pybind11/detail/smart_holder_type_casters.h.
The user-facing macro is now PYBIND11_SMART_HOLDER_TYPE_CASTERS.
* Systematically renaming tests to use "class_sh" in the name.
* Renaming test_type_caster_bare_interface_demo to test_type_caster_bare_interface.
* Renaming new tests/core subdirectory to tests/pure_cpp.
* Adding new tests to CMake config, resetting CI config.
* Changing CMake file so that test_class_sh_module_local.py actually runs.
* clang-tidy fixes.
* 32-bit compatibility.
* Reusing type_caster_base make_copy_constructor, make_move_constructor with a trick.
* CMake COMPARE NATURAL is not available with older versions.
* Adding copyright notices to new header files.
* Explicitly define copy/move constructors/assignments.
* Adding new header files to tests/extra_python_package/test_files.py.
* Adding tests/pure_cpp/CMakeLists.txt.
* Making use of the new find_existing_python_instance() function factored out with PR #2822.
* Moving define PYBIND11_SMART_HOLDER_TYPE_CASTERS(T) down in the file. NO functional changes. Preparation for follow-up work (to keep that diff smaller).
* Reintroducing py::classh, this time as a simple alias for py::class_<U, py::smart_holder>.
* Replacing detail::is_smart_holder<H> in cast.h with detail::is_smart_holder_type_caster<T>.
Moving get_local_load_function_ptr, init_instance_for_type to smart_holder_type_caster_class_hooks.
Expanding static_assert in py::type::handle_of<> to accommodate smart_holder_type_casters.
* Fixing oversight.
* Adding classu alias for class_<U, std::unique_ptr<U>>.
* Giving up on idea to use legacy init_instance only if is_base_of<type_caster_generic, type_caster<T>. There are use cases in the wild that define both a custom type_caster and class_.
* Removing test_type_caster_bare_interface, which was moved to the separate PR #2834.
* Moving up is_smart_holder_type_caster, to also use in cast_is_temporary_value_reference.
* Adding smart_holder_type_casters for unique_ptr with custom deleter. SEVERE CODE DUPLICATION. This commit is to establish a baseline for consolidating the unique_ptr code.
* Unification of unique_ptr, unique_ptr_with_deleter code in smart_holder_poc.h. Leads to more fitting error messages. Enables use of unique_ptr<T, D> smart_holder_type_casters also for unique_ptr<T>.
* Copying files as-is from branch test_unique_ptr_member (PR #2672).
* Adding comment, simplifying naming, cmake addition.
* Introducing PYBIND11_USE_SMART_HOLDER_AS_DEFAULT macro (tested only undefined; there are many errors with the macro defined).
* Removing test_type_caster_bare_interface, which was moved to the separate PR #2834.
* Fixing oversight introduced with commit 95425f13d6.
* Setting record.default_holder correctly for PYBIND11_USE_SMART_HOLDER_AS_DEFAULT.
With this test_class.cpp builds and even mostly runs, except
`test_multiple_instances_with_same_pointer`, which segfaults because it is
using a `unique_ptr` holder but `smart_holder` `type_caster`.
Also adding `static_assert`s to generate build errors for such situations,
but guarding with `#if 0` to first pivot to test_factory_constructors.cpp.
* Fixing up cast.h and smart_holder.h after rebase.
* Removing detail/smart_holder_type_casters.h in separate commit.
* Commenting out const in def_buffer(... const). With this, test_buffers builds and runs with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. Explanation why the const needs to be removed, or fix elsewhere, is still needed, but left for later.
* Adding test_class_sh_factory_constructors, reproducing test_factory_constructors failure. Using py::class_ in this commit, to be changed to py::classh for debugging.
* Removing include/pybind11/detail/smart_holder_type_casters.h from CMakeLists.txt, test_files.py (since it does not exist in this branch).
* Adding // DANGER ZONE reminders.
* Converting as many py::class_ to py::classh as possible, not breaking tests.
* Adding initimpl::construct() overloads, resulting in test_class_sh_factory_constructors feature parity for py::class_ and py::classh.
* Adding enable_if !is_smart_holder_type_caster to existing initimpl::construct(). With this test_factory_constructors.cpp builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT.
* Disabling shared_ptr&, shared_ptr* tests when building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT for now, pending work on smart_holder_type_caster<shared_ptr>.
* Factoring out struct and class definitions into anonymous namespace. Preparation for building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT.
* Simplifying from_unique_ptr(): typename D = std::default_delete<T> is not needed. Factoring out is_std_default_delete<T>() for consistentcy between ensure_compatible_rtti_uqp_del() and from_unique_ptr().
* Introducing PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. Using it in test_smart_ptr.cpp. With this test_smart_ptr builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT and all but one test run successfully.
* Introducing 1. type_caster_for_class_, used in PYBIND11_MAKE_OPAQUE, and 2. default_holder_type, used in stl_bind.h.
* Using __VA_ARGS__ in PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS.
* Replacing condense_for_macro with much simpler approach.
* Softening static_assert, to only check specifically that smart_holder is not mixed with type_caster_base, and unique_ptr/shared_ptr holders are not mixed with smart_holder_type_casters.
* Adding PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS in test_class.cpp (with this all but one test succeed with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT).
* Adding remaining PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. static_assert for "necessary conditions" for both types of default holder, static_assert for "strict conditions" guarded by new PYBIND11_STRICT_ASSERTS_CLASS_HOLDER_VS_TYPE_CASTER_MIX. All tests build & run as before with unique_ptr as the default holder, all tests build for smart_holder as the default holder, even with the strict static_assert.
* Introducing check_is_smart_holder_type_caster() function for runtime check, and reinterpreting record.default_holder as "uses_unique_ptr_holder". With this test_smart_ptr succeeds. (All 42 tests build, 35 tests succeed, 5 run but have some failures, 2 segfault.)
* Bug fix: Adding have_value() to smart_holder_type_caster_load. With this test_builtin_casters succeeds. (All 42 tests build, 36 tests succeed, 5 run but have some failures, 1 segfault.)
* Adding unowned_void_ptr_from_direct_conversion to modified_type_caster_generic_load_impl. This fixes the last remaining segfault (test_numpy_dtypes). New stats for all tests combined: 12 failed, 458 passed.
* Adding "Lazy allocation for unallocated values" (for old-style __init__) into load_value_and_holder. Deferring destruction of disowned holder until clear_instance, to remain inspectable for "uninitialized" or "disowned" detection. New stats for all tests combined: 5 failed, 465 passed.
* Changing std::shared_ptr pointer/reference to const pointer/reference. New stats for all tests combined: 4 failed, 466 passed.
* Adding return_value_policy::move to permissible policies for unique_ptr returns. New stats for all tests combined: 3 failed, 467 passed.
* Overlooked flake8 fixes.
* Manipulating failing ConstructorStats test to pass, to be able to run all tests with ASAN.
This version of the code is ASAN clean with unique_ptr or smart_holder as the default.
This change needs to be reverted after adopting the existing move-only-if-refcount-is-1
logic used by type_caster_base.
* Adding copy constructor and move constructor tracking to atyp. Preparation for a follow-up change in smart_holder_type_caster, to make this test sensitive to the changing behavior.
[skip ci]
* Removing `operator T&&() &&` from smart_holder_type_caster, for compatibility with the behavior of type_caster_base. Enables reverting 2 of 3 test manipulations applied under commit 249df7cbdb. The manipulation in test_factory_constructors.py is NOT reverted in this commit.
[skip ci]
* Fixing unfortunate editing mishap. This reverts the last remaining test manipulation in commit 249df7cbdb and makes all existing unit tests pass with smart_holder as default holder.
* GitHub CI clang-tidy fixes.
* Adding messages to terse `static_assert`s, for pre-C++17 compatibility.
* Using @pytest.mark.parametrize to run each assert separately (to see all errors, not just the first).
* Systematically removing _atyp from function names, to make the test code simpler.
* Using re.match to accommodate variable number of intermediate MvCtor.
* Also removing `operator T()` from smart_holder_type_caster, to fix gcc compilation errors. The only loss is pass_rref in test_class_sh_basic.
* Systematically replacing `detail::enable_if_t<...smart_holder...>` with `typename std::enable_if<...smart_holder...>::type`. Attempt to work around MSVC 2015 issues, to be tested via GitHub CI. The idea for this change originates from this comment: https://github.com/pybind/pybind11/issues/1616#issuecomment-444536813
* Importing re before pytest after observing a PyPy CI flake when importing pytest first.
* Copying MSVC 2015 compatibility change from branch pr2672_use_smart_holder_as_default.
* Introducing is_smart_holder_type_caster_base_tag, to keep smart_holder code more disconnected.
* Working around MSVC 2015 bug.
* Expanding comment for MSVC 2015 workaround.
* Systematically changing std::enable_if back to detail::enable_if_t, effectively reverting commit 5d4b6890a3.
* Removing unused smart_holder_type_caster_load::loaded_as_rvalue_ref (it was an oversight that it was not removed with commit 23036a45eb).
* Removing py::classu, because it does not seem useful enough.
* Reverting commit 6349531306 by un-commenting `const` in `def_buffer(...)`. To make this possible, `operator T const&` and `operator T const*` in `smart_holder_type_caster` need to be marked as `const` member functions.
* Adding construct() overloads for constructing smart_holder from alias unique_ptr, shared_ptr returns.
* Adding test_class_sh_factory_constructors.cpp to tests/CMakeLists.txt (fixes oversight, this should have been added long before).
* Compatibility with old clang versions (clang 3.6, 3.7 C++11).
* Cleaning up changes to existing unit tests.
* Systematically adding SMART_HOLDER_WIP tag. Removing minor UNTESTED tags (only the throw are not actually exercised, investing time there has a high cost but very little benefit).
* Splitting out smart_holder_type_casters again, into new detail/smart_holder_type_casters_inline_include.h.
* Splitting out smart_holder_init_inline_include.h.
* Adding additional new include files to CMakeLists.txt, tests/extra_python_package/test_files.py.
* clang-format cleanup of most smart_holder code.
* Adding source code comments in response to review.
* Simple micro-benchmark ("ubench") comparing runtime performance for several holders.
Tested using github.com/rwgk/pybind11_scons and Google-internal build system.
Sorry, no cmake support at the moment.
First results: https://docs.google.com/spreadsheets/d/1InapCYws2Gt-stmFf_Bwl33eOMo3aLE_gc9adveY7RU/edit#gid=0
* Breaking out number_bucket.h, adding hook for also collecting performance data for PyCLIF.
* Accounting for ubench in MANIFEST.in (simply prune, for now).
* Smarter determination of call_repetitions.
[skip ci]
* Also scaling performance data to PyCLIF.
[skip ci]
* Adding ubench/python/number_bucket.clif here for general visibility.
* Fix after rebase
* Merging detail/smart_holder_init_inline_include.h into detail/init.h.
* Renaming detail/is_smart_holder_type_caster.h -> detail/smart_holder_sfinae_hooks_only.h.
* Renaming is_smart_holder_type_caster -> type_uses_smart_holder_type_caster for clarity.
* Renaming type_caster_type_is_smart_holder_type_caster -> wrapped_type_uses_smart_holder_type_caster for clarity.
* Renaming is_smart_holder_type_caster_base_tag -> smart_holder_type_caster_base_tag for simplicity.
* Adding copyright notices and minor colateral cleanup.
* iwyu cleanup (comprehensive only for cast.h and smart_holder*.h files).
* Fixing `git rebase master` accident.
* Moving large `pragma warning` block from pybind11.h to detail/common.h.
* Fixing another `git rebase master` accident.
* CI: Intel icc/icpc via oneAPI
Add testing for Intel icc/icpc via the oneAPI images.
Intel oneAPI is in a late beta stage, currently shipping
oneAPI beta09 with ICC 20.2.
CI: Skip Interpreter Tests for Intel
Cannot find how to add this, neiter the package `libc6-dev` nor
`intel-oneapi-mkl-devel` help when installed to solve this:
```
-- Looking for C++ include pthread.h
-- Looking for C++ include pthread.h - not found
CMake Error at /__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:165 (message):
Could NOT find Threads (missing: Threads_FOUND)
Call Stack (most recent call first):
/__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:458 (_FPHSA_FAILURE_MESSAGE)
/__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindThreads.cmake:234 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
tests/test_embed/CMakeLists.txt:17 (find_package)
```
CI: libc6-dev from GCC for ICC
CI: Run bare metal for oneAPI
CI: Ubuntu 18.04 for oneAPI
CI: Intel +Catch -Eigen
CI: CMake from Apt (ICC tests)
CI: Replace Intel Py with GCC Py
CI: Intel w/o GCC's Eigen
CI: ICC with verbose make
[Debug] Find core dump
tests: use arg{} instead of arg() for Intel
tests: adding a few more missing {}
fix: sync with @tobiasleibner's branch
fix: try ubuntu 20-04
fix: drop exit 1
docs: Apply suggestions from code review
Co-authored-by: Tobias Leibner <tobias.leibner@googlemail.com>
Workaround for ICC enable_if issues
Another workaround for ICC's enable_if issues
fix error in previous commit
Disable one test for the Intel compiler in C++17 mode
Add back one instance of py::arg().noconvert()
Add NOLINT to fix clang-tidy check
Work around for ICC internal error in PYBIND11_EXPAND_SIDE_EFFECTS in C++17 mode
CI: Intel ICC with C++17
docs: pybind11/numpy.h does not require numpy at build time. (#2720)
This is nice enough to be mentioned explicitly in the docs.
docs: Update warning about Python 3.9.0 UB, now that 3.9.1 has been released (#2719)
Adjusting `type_caster<std::reference_wrapper<T>>` to support const/non-const propagation in `cast_op`. (#2705)
* Allow type_caster of std::reference_wrapper<T> to be the same as a native reference.
Before, both std::reference_wrapper<T> and std::reference_wrapper<const T> would
invoke cast_op<type>. This doesn't allow the type_caster<> specialization for T
to distinguish reference_wrapper types from value types.
After, the type_caster<> specialization invokes cast_op<type&>, which allows
reference_wrapper to behave in the same way as a native reference type.
* Add tests/examples for std::reference_wrapper<const T>
* Add tests which use mutable/immutable variants
This test is a chimera; it blends the pybind11 casters with a custom
pytype implementation that supports immutable and mutable calls.
In order to detect the immutable/mutable state, the cast_op needs
to propagate it, even through e.g. std::reference<const T>
Note: This is still a work in progress; some things are crashing,
which likely means that I have a refcounting bug or something else
missing.
* Add/finish tests that distinguish const& from &
Fixes the bugs in my custom python type implementation,
demonstrate test that requires const& and reference_wrapper<const T>
being treated differently from Non-const.
* Add passing a const to non-const method.
* Demonstrate non-const conversion of reference_wrapper in tests.
Apply formatting presubmit check.
* Fix build errors from presubmit checks.
* Try and fix a few more CI errors
* More CI fixes.
* More CI fixups.
* Try and get PyPy to work.
* Additional minor fixups. Getting close to CI green.
* More ci fixes?
* fix clang-tidy warnings from presubmit
* fix more clang-tidy warnings
* minor comment and consistency cleanups
* PyDECREF -> Py_DECREF
* copy/move constructors
* Resolve codereview comments
* more review comment fixes
* review comments: remove spurious &
* Make the test fail even when the static_assert is commented out.
This expands the test_freezable_type_caster a bit by:
1/ adding accessors .is_immutable and .addr to compare identity
from python.
2/ Changing the default cast_op of the type_caster<> specialization
to return a non-const value. In normal codepaths this is a reasonable
default.
3/ adding roundtrip variants to exercise the by reference, by pointer
and by reference_wrapper in all call paths. In conjunction with 2/, this
demonstrates the failure case of the existing std::reference_wrpper conversion,
which now loses const in a similar way that happens when using the default cast_op_type<>.
* apply presubmit formatting
* Revert inclusion of test_freezable_type_caster
There's some concern that this test is a bit unwieldly because of the use
of the raw <Python.h> functions. Removing for now.
* Add a test that validates const references propagation.
This test verifies that cast_op may be used to correctly detect
const reference types when used with std::reference_wrapper.
* mend
* Review comments based changes.
1. std::add_lvalue_reference<type> -> type&
2. Simplify the test a little more; we're never returning the ConstRefCaster
type so the class_ definition can be removed.
* formatted files again.
* Move const_ref_caster test to builtin_casters
* Review comments: use cast_op and adjust some comments.
* Simplify ConstRefCasted test
I like this version better as it moves the assertion that matters
back into python.
ci: drop pypy2 linux, PGI 20.7, add Python 10 dev (#2724)
* ci: drop pypy2 linux, add Python 10 dev
* ci: fix mistake
* ci: commented-out PGI 20.11, drop 20.7
fix: regression with installed pybind11 overriding local one (#2716)
* fix: regression with installed pybind11 overriding discovered one
Closes#2709
* docs: wording incorrect
style: remove redundant instance->owned = true (#2723)
which was just before set to True in instance->allocate_layout()
fix: also throw in the move-constructor added by the PYBIND11_OBJECT macro, after the argument has been moved-out (if necessary) (#2701)
Make args_are_all_* ICC workarounds unconditional
Disable test_aligned on Intel ICC
Fix test_aligned on Intel ICC
Skip test_python_alreadyset_in_destructor on Intel ICC
Fix test_aligned again
ICC CI: Downgrade pytest
pytest 6 does not capture the `discard_as_unraisable` stderr and
just writes a warning with its content instead.
* refactor: simpler Intel workaround, suggested by @laramiel
* fix: try version with impl to see if it is easier to compile
* docs: update README for ICC
Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
* Modified Vector STL bind initialization from a buffer type with optimization for simple arrays
* Add subtests to demonstrate processing Python buffer protocol objects with step > 1
* Fixed memoryview step test to only run on Python 3+
* Modified Vector constructor from buffer to return by value for readability
* Change NAMESPACE_BEGIN and NAMESPACE_END macros into PYBIND11_NAMESPACE_BEGIN and PYBIND11_NAMESPACE_END
* Fix sudden HomeBrew 'python not installed' error
* Sweep difference in 'Class.__init__() must be called when overriding __init__' error message between CPython and PyPy under the rug
* Homebrew updated to 3.8 yesterday.
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
* test pair-copyability on C++17 upwards
The stdlib falsely detects containers like M=std::map<T, U>
as copyable, even when one of T and U is not copyable.
Therefore we cannot rely on the stdlib dismissing std::pair<T, M>
by itself, even on C++17.
* fix is_copy_assignable
bind_map used std::is_copy_assignable which suffers from the same problems
as std::is_copy_constructible, therefore the same fix has been applied.
* created tests for copyability
This adds a PYBIND11_NAMESPACE macro that expands to the `pybind11`
namespace with hidden visibility under gcc-type compilers, and otherwise
to the plain `pybind11`. This then forces hidden visibility on
everything in pybind, solving the visibility issues discussed at end
end of #949.
This commit adds a `py::module_local` attribute that lets you confine a
registered type to the module (more technically, the shared object) in
which it is defined, by registering it with:
py::class_<C>(m, "C", py::module_local())
This will allow the same C++ class `C` to be registered in different
modules with independent sets of class definitions. On the Python side,
two such types will be completely distinct; on the C++ side, the C++
type resolves to a different Python type in each module.
This applies `py::module_local` automatically to `stl_bind.h` bindings
when the container value type looks like something global: i.e. when it
is a converting type (for example, when binding a `std::vector<int>`),
or when it is a registered type itself bound with `py::module_local`.
This should help resolve potential future conflicts (e.g. if two
completely unrelated modules both try to bind a `std::vector<int>`.
Users can override the automatic selection by adding a
`py::module_local()` or `py::module_local(false)`.
Note that this does mildly break backwards compatibility: bound stl
containers of basic types like `std::vector<int>` cannot be bound in one
module and returned in a different module. (This can be re-enabled with
`py::module_local(false)` as described above, but with the potential for
eventual load conflicts).
Pre-C++17, std::pair can technically have an copy constructor even
though it can't actually be invoked without a compilation failure (due
to the underlying types being non-copyable). Most stls, including
libc++ since ~3.4, use the C++17 behaviour of not exposing an uncallable
copy constructor, but FreeBSD deliberately broke their libc++ to
preserve the nonsensical behaviour
(https://svnweb.freebsd.org/base?view=revision&revision=261801).
This updates pybind's internal `is_copy_constructible` to also detect
the std::pair case under pre-C++17.
This also everything (except for a couple cases in the internal version)
to use the internal `is_copy_constructible` rather than
`std::is_copy_constructible`.
The "extend" method for vectors defined in stl_bind.h used `reserve` to
allocate space for the extra growth. While this can sometimes make a
constant-factor improvement in performance, it can also cause
construction of a vector by repeated extension to take quadratic rather
than linear time, as memory is reallocated in small increments rather
than on an exponential schedule. For example, this Python code would
take time proportional to the square of the trip count:
```python
a = VectorInt([1, 2, 3])
b = VectorInt()
for i in range(100000):
b.extend(a)
```
This commit removes the `reserve` call. The alternative would be to try
to add some smarter heuristics, but the standard library may well have
its own heuristics (the iterators are random access iterators, so it can
easily determine the number of items being added) and trying to add more
heuristics on top of that seems like a bad idea.
We're current copy by creating an Eigen::Map into the input numpy
array, then assigning that to the basic eigen type, effectively having
Eigen do the copy. That doesn't work for negative strides, though:
Eigen doesn't allow them.
This commit makes numpy do the copying instead by allocating the eigen
type, then having numpy copy from the input array into a numpy reference
into the eigen object's data. This also saves a copy when type
conversion is required: numpy can do the conversion on-the-fly as part
of the copy.
Finally this commit also makes non-reference parameters respect the
convert flag, declining the load when called in a noconvert pass with a
convertible, but non-array input or an array with the wrong dtype.
`EigenConformable::stride_compatible` returns false if the strides are
negative. In this case, do not use `EigenConformable::stride`, as it
is {0,0}. We cannot write negative strides in this element, as Eigen
will throw an assertion if we do.
The `type_caster` specialization for regular, dense Eigen matrices now
does a second `array_t::ensure` to copy data in case of negative strides.
I'm not sure that this is the best way to implement this.
I have added "TODO" tags linking these changes to Eigen bug #747, which,
when fixed, will allow Eigen to accept negative strides.
Allows use of vectors as python buffers, so for example they can be adopted without a copy by numpy.asarray
Allows faster conversion of buffers to vectors by copying instead of individually casting the elements
This replaces the current `all_of_t<Pred, Ts...>` with `all_of<Ts...>`,
with previous use of `all_of_t<Pred, Ts...>` becoming
`all_of<Pred<Ts>...>` (and similarly for `any_of_t`). It also adds a
`none_of<Ts...>`, a shortcut for `negation<any_of<Ts...>>`.
This allows `all_of` and `any_of` to be used a bit more flexible, e.g.
in cases where several predicates need to be tested for the same type
instead of the same predicate for multiple types.
This commit replaces the implementation with a more efficient version
for non-MSVC. For MSVC, this changes the workaround to use the
built-in, recursive std::conjunction/std::disjunction instead.
This also removes the `count_t` since `any_of_t` and `all_of_t` were the
only things using it.
This commit also rearranges some of the future std imports to use actual
`std` implementations for C++14/17 features when under the appropriate
compiler mode, as we were already doing for a few things (like
index_sequence). Most of these aren't saving much (the implementation
for enable_if_t, for example, is trivial), but I think it makes the
intention of the code instantly clear. It also enables MSVC's native
std::index_sequence support.
This commit includes the following changes:
* Don't provide make_copy_constructor for non-copyable container
make_copy_constructor currently fails for various stl containers (e.g.
std::vector, std::unordered_map, std::deque, etc.) when the container's
value type (e.g. the "T" or the std::pair<K,T> for a map) is
non-copyable. This adds an override that, for types that look like
containers, also requires that the value_type be copyable.
* stl_bind.h: make bind_{vector,map} work for non-copy-constructible types
Most stl_bind modifiers require copying, so if the type isn't copy
constructible, we provide a read-only interface instead.
In practice, this means that if the type is non-copyable, it will be,
for all intents and purposes, read-only from the Python side (but
currently it simply fails to compile with such a container).
It is still possible for the caller to provide an interface manually
(by defining methods on the returned class_ object), but this isn't
something stl_bind can handle because the C++ code to construct values
is going to be highly dependent on the container value_type.
* stl_bind: copy only for arithmetic value types
For non-primitive types, we may well be copying some complex type, when
returning by reference is more appropriate. This commit returns by
internal reference for all but basic arithmetic types.
* Return by reference whenever possible
Only if we definitely can't--i.e. std::vector<bool>--because v[i]
returns something that isn't a T& do we copy; for everything else, we
return by reference.
For the map case, we can always return by reference (at least for the
default stl map/unordered_map).
