Commit Graph

1244 Commits

Author SHA1 Message Date
Ralf W. Grosse-Kunstleve
6c922614ed
Adding reclaim_disowned logic & miscellaneous naming and documentation improvements. (#2943)
* Using new smart_holder::reclaim_disowned in smart_holder_type_caster for unique_ptr.

* Systematically renaming was_disowned to is_disowned (because disowning is now reversible: reclaim_disowned).

* Systematically renaming virtual_overrider_self_life_support to trampoline_self_life_support (to reuse existing terminology instead of introducing new one).

* Systematically renaming test_class_sh_with_alias to test_class_sh_trampoline_basic.

* Adding a Trampolines and std::unique_ptr section to README_smart_holder.rst.

* MSVC compatibility.
2021-04-09 23:08:44 -07:00
Ralf W. Grosse-Kunstleve
2b4fbbd521
Bug fix for virtual_overrider_self_life_support ASAN heap-use-after-free failure. (#2942)
* Porting subset of absltest code from reproducer provided by @elkhrt. Baseline for debugging ASAN heap-use-after-free.

* Moving Py_DECREF to resolve ASAN heap-use-after-free failure.

* Fixing trivial formatting issue.

* Workaround for clang 3.6 and 3.7.
2021-04-08 22:56:46 -07:00
Ralf W. Grosse-Kunstleve
2f624af1ac Merge branch 'master' into smart_holder 2021-04-02 18:20:27 -07:00
Ralf W. Grosse-Kunstleve
ad6bf5cd39
Adding PyGILState_Check() in object_api<>::operator(). (#2919)
* Adding PyGILState_Check() in object_api<>::operator().

* Enabling PyGILState_Check() for Python >= 3.6 only.

Possibly, this explains why PyGILState_Check() cannot safely be used with Python 3.4 and 3.5:

https://github.com/python/cpython/pull/10267#issuecomment-434881587

* Adding simple micro benchmark.

* Reducing test time to minimum (purely for coverage, not for accurate results).

* Fixing silly oversight.

* Minor code organization improvement in test.

* Adding example runtimes.

* Removing capsys (just run with `-k test_callback_num_times -s` and using `.format()`.
2021-04-02 18:17:12 -07:00
Ralf W. Grosse-Kunstleve
488014076b Merge branch 'master' into smart_holder 2021-04-02 11:36:35 -07:00
Robert Haschke
c2db53da56
fix: catch missing self argument in overloads constructor (#2914) 2021-04-02 13:13:44 -04:00
Ralf W. Grosse-Kunstleve
5319ca3817
Using dynamic_cast<AliasType> to determine pointee_depends_on_holder_owner. (#2910)
* Adaption of PyCLIF virtual_py_cpp_mix test.

* Removing ValueError: Ownership of instance with virtual overrides in Python cannot be transferred to C++. TODO: static_assert alias class needs to inherit from virtual_overrider_self_life_support.

* Bringing back ValueError: "... instance cannot safely be transferred to C++.", but based on dynamic_cast<AliasType>.

* Fixing oversight: adding test_class_sh_virtual_py_cpp_mix.cpp to cmake file.

* clang <= 3.6 compatibility.

* Fixing oversight: dynamic_raw_ptr_cast_if_possible needs special handling for To = void. Adding corresponding missing test in test_class_sh_virtual_py_cpp_mix. Moving dynamic_raw_ptr_cast_if_possible to separate header.

* Changing py::detail::virtual_overrider_self_life_support to py::virtual_overrider_self_life_support.
2021-03-19 12:18:39 -07:00
Ralf W. Grosse-Kunstleve
62a8d35831 Fixing oversight: clang-format pybind11.h (affecting smart_holder code only). 2021-03-18 11:23:14 -07:00
Ralf W. Grosse-Kunstleve
245d31cb03
Renaming PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS to PYBIND11_TYPE_CASTER_BASE_HOLDER. (#2907) 2021-03-17 04:56:11 -07:00
Ralf W. Grosse-Kunstleve
2ada792085 Pure clang-format cleanup (after #2904), NO other changes. 2021-03-17 03:52:43 -07:00
Robert Haschke
784092dfd2
Missing cast from const unique_ptr& (#2904)
* Add roundtrip tests for unique_ptr

* Implementation for casting from const std::unique_ptr&

... forwarding to smart_holder_type_caster<T>::cast(T*)
2021-03-16 18:10:12 -07:00
Ralf W. Grosse-Kunstleve
469792032a
Adding virtual_overrider_self_life_support. (#2902)
* Initial version of virtual_overrider_self_life_support (enables safely passing unique_ptr to C++).

* Clang 3.6, 3.7 compatibility.

* Adding missing default constructor.

* Restoring test for exception for the case that virtual_overrider_self_life_support is not used.

* Fixing oversight: Adding missing holder().ensure_was_not_disowned().

* Adding unit tests for new `struct smart_holder` member functions.

* Moving virtual_overrider_self_life_support to separate include file, with iwyu cleanup.
2021-03-16 06:31:24 -07:00
Ralf W. Grosse-Kunstleve
d6cf6dfed3 Merge branch 'master' into smart_holder 2021-03-09 15:11:35 -08:00
Boris Staletic
f110889dde
Use correct duration representation when casting from datetime.timdelta to std::chrono::duration (#2870)
* Use correct duration representation when casting from datetime.timdelta to std::chrono::duration

* When asserting datetime/timedelta/date/time we can equality-compare whole objects
2021-03-07 23:42:01 -08:00
Ralf W. Grosse-Kunstleve
97a7fb722a
Porting/adapting Dustin's PR #2839 to smart_holder branch (#2886)
* WIP: test setup complete, AddInCppUniquePtr failing (reproduces PyCLIF smart_ptrs_test failure).

* Fully tested locally.

* Adding new tests to cmake file.
2021-03-03 17:58:42 -08:00
Ralf W. Grosse-Kunstleve
6a7e9f42fe
Changing all but one std::runtime_error to std::invalid_argument, which appears as ValueError in the Python interpreter. Adding test_cannot_disown_use_count_ne_1. (#2883) 2021-03-03 05:08:47 -08:00
Ralf W. Grosse-Kunstleve
3a336a2047
shared_ptr<bool> vptr_deleter_armed_flag_ptr (instead of unique_ptr) (#2882)
* shared_ptr<bool> vptr_deleter_armed_flag_ptr (instead of unique_ptr), to fix heap-use-after-free bug.

* Fixing  generated by some compilers in the pybind11 CI suite.
2021-03-02 17:43:25 -08:00
Ralf W. Grosse-Kunstleve
666fc0d8e8
Changing cast_error("... Python instance was disowned.") to value_error (which changes RuntimeError to ValueError). (#2880) 2021-02-26 23:19:23 -08:00
Ralf W. Grosse-Kunstleve
01e0045547
Enabling use of smart_holder for types with non-public destructors. (#2878)
* Enabling use of smart_holder for types with non-public destructors.

* Resolving clang-tidy error (GitHub CI).
2021-02-26 17:51:50 -08:00
Ralf W. Grosse-Kunstleve
1bafd5db5f
Adding py::smart_holder (for smart-pointer interoperability). (#2672)
* Adding test_unique_ptr_member (for desired PyCLIF behavior).

See also: https://github.com/pybind/pybind11/issues/2583

Does not build with upstream master or
https://github.com/pybind/pybind11/pull/2047, but builds with
https://github.com/RobotLocomotion/pybind11 and almost runs:

```
Running tests in directory "/usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests":
================================================================================= test session starts =================================================================================
platform linux -- Python 3.8.5, pytest-5.4.3, py-1.9.0, pluggy-0.13.1
rootdir: /usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests, inifile: pytest.ini
collected 2 items

test_unique_ptr_member.py .F                                                                                                                                                    [100%]

====================================================================================== FAILURES =======================================================================================
_____________________________________________________________________________ test_pointee_and_ptr_owner ______________________________________________________________________________

    def test_pointee_and_ptr_owner():
        obj = m.pointee()
        assert obj.get_int() == 213
        m.ptr_owner(obj)
        with pytest.raises(ValueError) as exc_info:
>           obj.get_int()
E           Failed: DID NOT RAISE <class 'ValueError'>

test_unique_ptr_member.py:17: Failed
============================================================================= 1 failed, 1 passed in 0.06s =============================================================================
```

* unique_ptr or shared_ptr return

* new test_variant_unique_shared with vptr_holder prototype

* moving prototype code to pybind11/vptr_holder.h, adding type_caster specialization to make the bindings involving unique_ptr passing compile, but load and cast implementations are missing

* disabling GitHub Actions on pull_request (for this PR)

* disabling AppVeyor (for this PR)

* TRIGGER_SEGSEV macro, annotations for GET_STACK (vptr::get), GET_INT_STACK (pointee)

* adding test_promotion_of_disowned_to_shared

* Copying tests as-is from xxx_value_ptr_xxx_holder branch.

https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder

Systematically exercising returning and passing unique_ptr<T>, shared_ptr<T>
with unique_ptr, shared_ptr holder.

Observations:

test_holder_unique_ptr:
  make_unique_pointee  OK
  pass_unique_pointee  BUILD_FAIL (as documented)
  make_shared_pointee  Abort free(): double free detected
  pass_shared_pointee  RuntimeError: Unable to load a custom holder type from a default-holder instance

test_holder_shared_ptr:
  make_unique_pointee  Segmentation fault (#1138)
  pass_unique_pointee  BUILD_FAIL (as documented)
  make_shared_pointee  OK
  pass_shared_pointee  OK

* Copying tests as-is from xxx_value_ptr_xxx_holder branch.

https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder

Systematically exercising casting between shared_ptr<base>, shared_ptr<derived>.

* Demonstration of Undefined Behavior in handling of shared_ptr holder.

Based on https://godbolt.org/z/4fdjaW by jorgbrown@ (thanks Jorg!).

* Additional demonstration of Undefined Behavior in handling of shared_ptr holder.

* fixing up-down mixup in comment

* Demonstration of Undefined Behavior in handling of polymorphic pointers.

(This demo does NOT involve smart pointers at all, unlike the otherwise similar test_smart_ptr_private_first_base.)

* minor test_private_first_base.cpp simplification (after discovering that this can be wrapped with Boost.Python, using boost::noncopyable)

* pybind11 equivalent of Boost.Python test similar to reproducer under #1333

* Snapshot of WIP, TODO: shared_ptr deleter with on/off switch

* Adding vptr_deleter.

* Adding from/as unique_ptr<T> and unique_ptr<T, D>.

* Adding from_shared_ptr. Some polishing.

* New tests/core/smart_holder_poc_test.cpp, using Catch2.

* Adding in vptr_deleter_guard_flag.

* Improved labeling of TEST_CASEs.

* Shuffling existing TEST_CASEs into systematic matrix.

* Implementing all [S]uccess tests.

* Implementing all [E]xception tests.

* Testing of exceptions not covered by the from-as matrix.

* Adding top-level comment.

* Converting from methods to factory functions (no functional change).

* Removing obsolete and very incomplete test (replaced by Catch2-based test).

* Removing stray file.

* Adding type_caster_bare_interface_demo.

* Adding shared_ptr<mpty>, shared_ptr<mpty const> casters.

* Adding unique_ptr<mpty>, unique_ptr<mpty const> casters.

* Pure copy of `class class_` implementation in pybind11.h (master commit 98f1bbb800).

* classh.h: renaming of class_ to classh + namespace; forking test_classh_wip from test_type_caster_bare_interface_demo.

* Hard-coding smart_holder into classh.

* Adding mpty::mtxt string member.

* Adding isinstance<mpty> in type_caster::load functions.

* Adding rvalue_ref, renaming const_value_ref to lvalue_ref & removing const.

* Retrieving smart_holder pointer in type_caster<mpty>::load, and using it cast_op operators.

* Factoring out smart_holder_type_caster_load.

* Retrieving smart_holder pointer in type_caster<std::shared_ptr<mpty[ const]>>::load, and using it cast_op operators.

* Improved error messaging: Cannot disown nullptr (as_unique_ptr).

* Retrieving smart_holder pointer in type_caster<std::unique_ptr<mpty[ const]>>::load, and using it cast_op operators.

* Pure `clang-format --style=file -i` change.

* Pure `clang-format --style=file -i` change, with two `clang-format off` directives.

* Fixing oversight (discovered by flake8).

* flake8 cleanup

* Systematically setting mtxt for all rtrn_mpty_* functions (preparation, the values are not actually used yet).

* static cast handle for rtrn_cptr works by simply dropping in code from type_caster_base (marked with comments).

* static cast handle for rtrn_cref works by simply dropping in code from type_caster_base (marked with comments). rtrn_mref and rtrn_mptr work via const_cast (to add const).

* static cast handle for rtrn_valu works by simply dropping in code from type_caster_base (marked with comments). rtrn_rref raises a RuntimeError, to be investigated.

* Copying type_caster_generic::cast into type_caster<mpty> as-is (preparation for handling smart pointers).

* Pure clang-format change (applied to original type_caster_generic::cast).

* Adding comment re potential use_count data race.

* static handle cast implementations for rtrn_shmp, rtrn_shcp.

* Adding MISSING comments in operator std::unique_ptr<mpty[ const]>.

* static handle cast implementations for rtrn_uqmp, rtrn_uqcp.

* Bug fix: vptr_deleter_armed_flag_ptr has to live on the heap.

See new bullet point in comment section near the top.

The variable was also renamed to reflect its function more accurately.

* Fixing bugs discovered by ASAN. The code is now ASAN, MSAN, UBSAN clean.

* Making test_type_caster_bare_interface_demo.cpp slightly more realistic, ASAN, MSAN, UBSAN clean.

* Calling deregister_instance after disowning via unique_ptr.

* Removing enable_shared_from_this stub, simplifying existing code, clang-format.

Open question, with respect to the original code:
76a160070b/include/pybind11/pybind11.h (L1510)
To me it looks like the exact situation marked as `std::shared_ptr<Good> gp1 = not_so_good.getptr();` here: https://en.cppreference.com/w/cpp/memory/enable_shared_from_this
The comment there is: `// undefined behavior (until C++17) and std::bad_weak_ptr thrown (since C++17)`
Does the existing code have UB pre C++17?

I'll leave handling of enable_shared_from_this for later, as the need arises.

* Cosmetical change around helper functions.

* Using type_caster_base<mpty>::src_and_type directly, removing copy. Also renaming one cast to cast_const_raw_ptr, for clarity.

* Fixing clang-format oversight.

* Using factored-out make_constructor (PR #2798), removing duplicate code.

* Inserting additional assert to ensure a returned unique_ptr is always a new Python instance.

* Adding minor comment (change to internals needed to distinguish uninitialized/disowned in error message).

* Factoring out find_existing_python_instance().

* Moving factored-out make_constructor to test_classh_wip.cpp, restoring previous version of cast.h. This is currently the most practical approach. See PR #2798 for background.

* Copying classh type_casters from test_classh_wip.cpp UNMODIFIED, as a baseline for generalizing the code.

* Using pybind11/detail/classh_type_casters.h from test_classh_wip.cpp.

* Adding & using PYBIND11_CLASSH_TYPE_CASTERS define.

* Adding test_classh_inheritance, currently failing (passes with class_).

* Removing .clang-format before git rebase master (where the file was added).

* Bringing back .clang-format, the previous rm was a bad idea.

* Folding in modified_type_caster_generic_load_impl, just enough to pass test_class_wip. test_classh_inheritance is still failing, but with a different error: [RuntimeError: Incompatible type (as_raw_ptr_unowned).]

* Minimal changes needed to pass test_classh_inheritance.

* First pass adjusting try_implicit_casts and try_load_foreign_module_local to capture loaded_v_h, but untested and guarded with pybind11_failure("Untested"). This was done mainly to determine general feasibility. Note the TODO in pybind11.h, where type_caster_generic::local_load is currently hard-coded. test_classh_wip and test_classh_inheritance still pass, as before.

* Decoupling generic_type from type_caster_generic.

* Changes and tests covering classh_type_casters try_implicit_casts.

* Minimal test covering classh_type_casters load_impl Case 2b.

* Removing stray isinstance<T>(src): it interferes with the py::module_local feature. Adding missing #includes.

* Tests for classh py::module_local() feature.

* Pure renaming of function names in test_classh_inheritance, similar to the systematic approach used in test_class_wip. NO functional changes.

* Pure renaming of function and variable names, for better generalization when convoluting with inheritance. NO functional changes.

* Adopting systematic naming scheme from test_classh_wip. NO functional changes.

* Moving const after type name, for functions that cover a systematic scheme. NO functional changes.

* Adding smart_holder_type_caster_load::loaded_as_shared_ptr, currently bypassing smart_holder shared_ptr tracking completely, but the tests pass and are sanitizer clean.

* Removing rtti_held from smart_holder. See updated comment.

* Cleaning up loaded_as_raw_ptr_unowned, loaded_as_shared_ptr.

* Factoring out convert_type and folding into loaded_as_unique_ptr.

* Folding convert_type into lvalue_ref and rvalue_ref paths. Some smart_holder_type_caster_load cleanup.

* Using unique_ptr in local_load to replace static variable. Also adding local_load_safety_guard.

* Converting test_unique_ptr_member to using classh: fully working, ASAN, MSAN, UBSAN clean.

* Removing debugging comments (GET_STACK, GET_INT_STACK). cast.h is identical to current master again, pybind11.h only has the generic_type::initialize(..., &type_caster_generic::local_load) change.

* Purging obsolete pybind11/vptr_holder.h and associated test.

* Moving several tests to github.com/rwgk/rwgk_tbx/tree/main/pybind11_tests

a2c2f88174

These tests are from experimenting, and for demonstrating UB in pybind11 multiple inheritance handling ("first_base"), to be fixed later.

* Adding py::smart_holder support to py::class_, purging py::classh completely.

* Renaming files in include directory, creating pybind11/smart_holder.h.

* Renaming all "classh" to "smart_holder" in pybind11/detail/smart_holder_type_casters.h.

The user-facing macro is now PYBIND11_SMART_HOLDER_TYPE_CASTERS.

* Systematically renaming tests to use "class_sh" in the name.

* Renaming test_type_caster_bare_interface_demo to test_type_caster_bare_interface.

* Renaming new tests/core subdirectory to tests/pure_cpp.

* Adding new tests to CMake config, resetting CI config.

* Changing CMake file so that test_class_sh_module_local.py actually runs.

* clang-tidy fixes.

* 32-bit compatibility.

* Reusing type_caster_base make_copy_constructor, make_move_constructor with a trick.

* CMake COMPARE NATURAL is not available with older versions.

* Adding copyright notices to new header files.

* Explicitly define copy/move constructors/assignments.

* Adding new header files to tests/extra_python_package/test_files.py.

* Adding tests/pure_cpp/CMakeLists.txt.

* Making use of the new find_existing_python_instance() function factored out with PR #2822.

* Moving define PYBIND11_SMART_HOLDER_TYPE_CASTERS(T) down in the file. NO functional changes. Preparation for follow-up work (to keep that diff smaller).

* Reintroducing py::classh, this time as a simple alias for py::class_<U, py::smart_holder>.

* Replacing detail::is_smart_holder<H> in cast.h with detail::is_smart_holder_type_caster<T>.
Moving get_local_load_function_ptr, init_instance_for_type to smart_holder_type_caster_class_hooks.
Expanding static_assert in py::type::handle_of<> to accommodate smart_holder_type_casters.

* Fixing oversight.

* Adding classu alias for class_<U, std::unique_ptr<U>>.

* Giving up on idea to use legacy init_instance only if is_base_of<type_caster_generic, type_caster<T>. There are use cases in the wild that define both a custom type_caster and class_.

* Removing test_type_caster_bare_interface, which was moved to the separate PR #2834.

* Moving up is_smart_holder_type_caster, to also use in cast_is_temporary_value_reference.

* Adding smart_holder_type_casters for unique_ptr with custom deleter. SEVERE CODE DUPLICATION. This commit is to establish a baseline for consolidating the unique_ptr code.

* Unification of unique_ptr, unique_ptr_with_deleter code in smart_holder_poc.h. Leads to more fitting error messages. Enables use of unique_ptr<T, D> smart_holder_type_casters also for unique_ptr<T>.

* Copying files as-is from branch test_unique_ptr_member (PR #2672).

* Adding comment, simplifying naming, cmake addition.

* Introducing PYBIND11_USE_SMART_HOLDER_AS_DEFAULT macro (tested only undefined; there are many errors with the macro defined).

* Removing test_type_caster_bare_interface, which was moved to the separate PR #2834.

* Fixing oversight introduced with commit 95425f13d6.

* Setting record.default_holder correctly for PYBIND11_USE_SMART_HOLDER_AS_DEFAULT.

With this test_class.cpp builds and even mostly runs, except
`test_multiple_instances_with_same_pointer`, which segfaults because it is
using a `unique_ptr` holder but `smart_holder` `type_caster`.

Also adding `static_assert`s to generate build errors for such situations,
but guarding with `#if 0` to first pivot to test_factory_constructors.cpp.

* Fixing up cast.h and smart_holder.h after rebase.

* Removing detail/smart_holder_type_casters.h in separate commit.

* Commenting out const in def_buffer(... const). With this, test_buffers builds and runs with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. Explanation why the const needs to be removed, or fix elsewhere, is still needed, but left for later.

* Adding test_class_sh_factory_constructors, reproducing test_factory_constructors failure. Using py::class_ in this commit, to be changed to py::classh for debugging.

* Removing include/pybind11/detail/smart_holder_type_casters.h from CMakeLists.txt, test_files.py (since it does not exist in this branch).

* Adding // DANGER ZONE reminders.

* Converting as many py::class_ to py::classh as possible, not breaking tests.

* Adding initimpl::construct() overloads, resulting in test_class_sh_factory_constructors feature parity for py::class_ and py::classh.

* Adding enable_if !is_smart_holder_type_caster to existing initimpl::construct(). With this test_factory_constructors.cpp builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT.

* Disabling shared_ptr&, shared_ptr* tests when building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT for now, pending work on smart_holder_type_caster<shared_ptr>.

* Factoring out struct and class definitions into anonymous namespace. Preparation for building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT.

* Simplifying from_unique_ptr(): typename D = std::default_delete<T> is not needed. Factoring out is_std_default_delete<T>() for consistentcy between ensure_compatible_rtti_uqp_del() and from_unique_ptr().

* Introducing PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. Using it in test_smart_ptr.cpp. With this test_smart_ptr builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT and all but one test run successfully.

* Introducing 1. type_caster_for_class_, used in PYBIND11_MAKE_OPAQUE, and 2. default_holder_type, used in stl_bind.h.

* Using __VA_ARGS__ in PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS.

* Replacing condense_for_macro with much simpler approach.

* Softening static_assert, to only check specifically that smart_holder is not mixed with type_caster_base, and unique_ptr/shared_ptr holders are not mixed with smart_holder_type_casters.

* Adding PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS in test_class.cpp (with this all but one test succeed with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT).

* Adding remaining PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. static_assert for "necessary conditions" for both types of default holder, static_assert for "strict conditions" guarded by new PYBIND11_STRICT_ASSERTS_CLASS_HOLDER_VS_TYPE_CASTER_MIX. All tests build & run as before with unique_ptr as the default holder, all tests build for smart_holder as the default holder, even with the strict static_assert.

* Introducing check_is_smart_holder_type_caster() function for runtime check, and reinterpreting record.default_holder as "uses_unique_ptr_holder". With this test_smart_ptr succeeds. (All 42 tests build, 35 tests succeed, 5 run but have some failures, 2 segfault.)

* Bug fix: Adding have_value() to smart_holder_type_caster_load. With this test_builtin_casters succeeds. (All 42 tests build, 36 tests succeed, 5 run but have some failures, 1 segfault.)

* Adding unowned_void_ptr_from_direct_conversion to modified_type_caster_generic_load_impl. This fixes the last remaining segfault (test_numpy_dtypes). New stats for all tests combined: 12 failed, 458 passed.

* Adding "Lazy allocation for unallocated values" (for old-style __init__) into load_value_and_holder. Deferring destruction of disowned holder until clear_instance, to remain inspectable for "uninitialized" or "disowned" detection. New stats for all tests combined: 5 failed, 465 passed.

* Changing std::shared_ptr pointer/reference to const pointer/reference. New stats for all tests combined: 4 failed, 466 passed.

* Adding return_value_policy::move to permissible policies for unique_ptr returns. New stats for all tests combined: 3 failed, 467 passed.

* Overlooked flake8 fixes.

* Manipulating failing ConstructorStats test to pass, to be able to run all tests with ASAN.

This version of the code is ASAN clean with unique_ptr or smart_holder as the default.

This change needs to be reverted after adopting the existing move-only-if-refcount-is-1
logic used by type_caster_base.

* Adding copy constructor and move constructor tracking to atyp. Preparation for a follow-up change in smart_holder_type_caster, to make this test sensitive to the changing behavior.

[skip ci]

* Removing `operator T&&() &&` from smart_holder_type_caster, for compatibility with the behavior of type_caster_base. Enables reverting 2 of 3 test manipulations applied under commit 249df7cbdb. The manipulation in test_factory_constructors.py is NOT reverted in this commit.

[skip ci]

* Fixing unfortunate editing mishap. This reverts the last remaining test manipulation in commit 249df7cbdb and makes all existing unit tests pass with smart_holder as default holder.

* GitHub CI clang-tidy fixes.

* Adding messages to terse `static_assert`s, for pre-C++17 compatibility.

* Using @pytest.mark.parametrize to run each assert separately (to see all errors, not just the first).

* Systematically removing _atyp from function names, to make the test code simpler.

* Using re.match to accommodate variable number of intermediate MvCtor.

* Also removing `operator T()` from smart_holder_type_caster, to fix gcc compilation errors. The only loss is pass_rref in test_class_sh_basic.

* Systematically replacing `detail::enable_if_t<...smart_holder...>` with `typename std::enable_if<...smart_holder...>::type`. Attempt to work around MSVC 2015 issues, to be tested via GitHub CI. The idea for this change originates from this comment: https://github.com/pybind/pybind11/issues/1616#issuecomment-444536813

* Importing re before pytest after observing a PyPy CI flake when importing pytest first.

* Copying MSVC 2015 compatibility change from branch pr2672_use_smart_holder_as_default.

* Introducing is_smart_holder_type_caster_base_tag, to keep smart_holder code more disconnected.

* Working around MSVC 2015 bug.

* Expanding comment for MSVC 2015 workaround.

* Systematically changing std::enable_if back to detail::enable_if_t, effectively reverting commit 5d4b6890a3.

* Removing unused smart_holder_type_caster_load::loaded_as_rvalue_ref (it was an oversight that it was not removed with commit 23036a45eb).

* Removing py::classu, because it does not seem useful enough.

* Reverting commit 6349531306 by un-commenting `const` in `def_buffer(...)`. To make this possible, `operator T const&` and `operator T const*` in `smart_holder_type_caster` need to be marked as `const` member functions.

* Adding construct() overloads for constructing smart_holder from alias unique_ptr, shared_ptr returns.

* Adding test_class_sh_factory_constructors.cpp to tests/CMakeLists.txt (fixes oversight, this should have been added long before).

* Compatibility with old clang versions (clang 3.6, 3.7 C++11).

* Cleaning up changes to existing unit tests.

* Systematically adding SMART_HOLDER_WIP tag. Removing minor UNTESTED tags (only the throw are not actually exercised, investing time there has a high cost but very little benefit).

* Splitting out smart_holder_type_casters again, into new detail/smart_holder_type_casters_inline_include.h.

* Splitting out smart_holder_init_inline_include.h.

* Adding additional new include files to CMakeLists.txt, tests/extra_python_package/test_files.py.

* clang-format cleanup of most smart_holder code.

* Adding source code comments in response to review.

* Simple micro-benchmark ("ubench") comparing runtime performance for several holders.

Tested using github.com/rwgk/pybind11_scons and Google-internal build system.
Sorry, no cmake support at the moment.

First results: https://docs.google.com/spreadsheets/d/1InapCYws2Gt-stmFf_Bwl33eOMo3aLE_gc9adveY7RU/edit#gid=0

* Breaking out number_bucket.h, adding hook for also collecting performance data for PyCLIF.

* Accounting for ubench in MANIFEST.in (simply prune, for now).

* Smarter determination of call_repetitions.

[skip ci]

* Also scaling performance data to PyCLIF.

[skip ci]

* Adding ubench/python/number_bucket.clif here for general visibility.

* Fix after rebase

* Merging detail/smart_holder_init_inline_include.h into detail/init.h.

* Renaming detail/is_smart_holder_type_caster.h -> detail/smart_holder_sfinae_hooks_only.h.

* Renaming is_smart_holder_type_caster -> type_uses_smart_holder_type_caster for clarity.

* Renaming type_caster_type_is_smart_holder_type_caster -> wrapped_type_uses_smart_holder_type_caster for clarity.

* Renaming is_smart_holder_type_caster_base_tag -> smart_holder_type_caster_base_tag for simplicity.

* Adding copyright notices and minor colateral cleanup.

* iwyu cleanup (comprehensive only for cast.h and smart_holder*.h files).

* Fixing `git rebase master` accident.

* Moving large `pragma warning` block from pybind11.h to detail/common.h.

* Fixing another `git rebase master` accident.
2021-02-23 21:50:42 -08:00
Bertrand MICHEL
74a767d429
Dtype kind vs char (#2864)
* [dtype]: add type() method to access type attribute of PyArray_Descr (eq. to dtype.char in Python)

* [dtype] change type() name method to char_() to be compliant with Python numpy interface

* [dtype] fix by pre-commit

* [dtype] Change comments and solutions format for test

* Clarify documentation and move note about dtype.char vs PyArray_Descr::type to a plain, non-doxygen comment

* Fix and extend tests

* Fix the supposedly fixed tests

* Fix the fixed tests again

Co-authored-by: Bertrand MICHEL <bertrand.michel@onera.fr>
Co-authored-by: Yannick Jadoul <yannick.jadoul@belgacom.net>
2021-02-23 10:57:28 +01:00
Dustin Spicuzza
c0fbb02c9f
Extract gil management functions to separate header (#2845) 2021-02-22 19:15:40 -08:00
Ralf W. Grosse-Kunstleve
0c42250a4e
Splitting out detail/type_caster_base.h from cast.h, with iwyu cleanup. (#2841)
* Splitting out detail/type_caster_base.h from cast.h.
* Include cleanup guided by include-what-you-use 0.12 based on clang version 9.0.1-11.
2021-02-22 18:38:18 -08:00
Vikram Pal
417067eeb8
Add pybind11::bytearray (#2799)
* Add initial implementation

* Add few more methods

* Add tests

* Fix a typo

* Use std::string constructor which takes size

* Fix implicit sign conversion error

* Add size method and test

* Remove implicit conversion

* Fix bytearray constructors and operator std::string()

* Make implicit bytearray constructor explicit

* Rerun tests

* Add null check

* Rerun tests

* Rerun tests - 2

* Remove NULL check
2021-02-14 15:51:13 +01:00
Karthik Nishanth
e791ec4e27
fix: add null pointer check with std::localtime (#2846) 2021-02-08 15:38:22 -05:00
Yannick Jadoul
6cf6bf203e
Fix confusing weakref constructor overload (#2832)
* Demonstrate issue with weakref constructor overloads

* Fix weakref constructor to convert on being passed a non-weakref object

* Improve on nonlocal-scoped variable in test_weakref

* Keep backwards-compatibility by introducing PYBIND11_OBJECT_CVT_DEFAULT macro

* Simplify test_weakref
2021-01-31 23:13:31 +01:00
Ralf W. Grosse-Kunstleve
932769b038
Adding holder_caster typename SFINAE = void hooks to help work around the current lack of smart-pointer interoperability (#2833)
* Adding move_only_holder_caster `typename SFINAE = void` to enable external specializations.

* Adding SFINAE hook also to copyable_holder_caster, for uniformity, with comment to explain the purpose.
2021-01-30 12:02:24 -08:00
Edward Lockhart
23c3edcf21
When determining if a shared_ptr already exists, use a test on the we… (#2819)
* When determining if a shared_ptr already exists, use a test on the weak_ptr instead of a try/catch block.

* When determining if a shared_ptr already exists, use a test on the weak_ptr instead of a try/catch block.

* weak_from_this is only available in C++17 and later

* Switch to use feature flag instead of C++ version flag.

* Add Microsoft-specific check.

* Avoid undefined preprocessor macro warning treated as error.

* Simplify shared_from_this in init_holder

* Include <version> in detail/common.h (~stolen~ borrowed from @bstaletic's #2816)

* Move class_::get_shared_from_this to detail::try_get_shared_from_this

* Simplify try_get_shared_from_this by using weak_ptr::lock()

Co-authored-by: Yannick Jadoul <yannick.jadoul@belgacom.net>
2021-01-30 20:05:13 +01:00
Ralf W. Grosse-Kunstleve
0432ae7c52
Changing pybind11::str to exclusively hold PyUnicodeObject (#2409)
* Changing pybind11::str to exclusively hold PyUnicodeObject
2021-01-29 09:41:42 -08:00
Yannick Jadoul
587d5f840a
Update breathe to 4.26.1, add make_tuple, make_iterator, and make_key_iterator (#2828) 2021-01-28 12:28:16 +01:00
Ralf W. Grosse-Kunstleve
9b7bfef833
Factoring out find_registered_python_instance() from type_caster_generic::cast. (#2822)
Factoring out find_registered_python_instance() from type_caster_generic::cast.
2021-01-26 21:08:46 -08:00
Henry Fredrick Schreiner
87954e7a54 fix: corrected dev versioning 2021-01-26 22:42:14 -05:00
Henry Fredrick Schreiner
4a5b81b1b7 chore: get back to work 2021-01-26 22:28:09 -05:00
Henry Schreiner
8de7772cc7
chore: prepare for the 2.6.2 release (#2821) 2021-01-26 21:26:45 -05:00
Henry Fredrick Schreiner
732bf88d19 fix: avoid changing class outside of GIL 2021-01-26 20:59:27 -05:00
Yannick Jadoul
0bb8ca2639
Always call PyNumber_Index when casting from Python to a C++ integral type, also pre-3.8 (#2801)
* Always call PyNumber_Index when casting from Python to a C++ integral type, also pre-3.8

* Fixed on PyPy

* Simplify use of PyNumber_Index, following @rwgk's idea, and ignore warnings in >=3.8

* Reproduce mismatch between pre-3.8 and post-3.8 behavior on __index__ throwing TypeError

* Fix tests on 3.6 <= Python < 3.8

* No, I don't have an uninitialized variable

* Fix use of __index__ on Python 2

* Make types in test_int_convert more ~boring~ descriptive
2021-01-25 21:05:17 +01:00
crimsoncor
9ea39dc356
Force the builtin module key to be the correct type. (#2814)
* Force the builtin module key to be the correct type.

Previously it was always going to be a std::string which converted into
unicode. Python 2 appears to want module keys to be normal str types, so
this was breaking code that expected plain string types in the
builtins.keys() data structure

* Add a simple unit test to ensure all built-in keys are str

* Update the unit test so it will also run on pypy

* Run pre-commit.

Co-authored-by: Jesse Clemens <jesse.clemens@sony.com>
2021-01-24 09:17:28 -08:00
Henry Fredrick Schreiner
5b43ac42a0 docs: fix missing line from #2310 2021-01-19 19:56:22 -05:00
Henry Schreiner
2db0264aca
style: add clang-format file (#2310)
* style: adding clang-format as manual hook

* docs: adding a Clang-Format section. (#2803)

Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
2021-01-19 19:10:26 -05:00
Michael Kuron
48534089f7
fix: Intel ICC C++17 compatibility (#2729)
* CI: Intel icc/icpc via oneAPI

Add testing for Intel icc/icpc via the oneAPI images.
Intel oneAPI is in a late beta stage, currently shipping
oneAPI beta09 with ICC 20.2.

CI: Skip Interpreter Tests for Intel

Cannot find how to add this, neiter the package `libc6-dev` nor
`intel-oneapi-mkl-devel` help when installed to solve this:
```
-- Looking for C++ include pthread.h
-- Looking for C++ include pthread.h - not found
CMake Error at /__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:165 (message):
  Could NOT find Threads (missing: Threads_FOUND)
Call Stack (most recent call first):
  /__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:458 (_FPHSA_FAILURE_MESSAGE)
  /__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindThreads.cmake:234 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
  tests/test_embed/CMakeLists.txt:17 (find_package)
```

CI: libc6-dev from GCC for ICC

CI: Run bare metal for oneAPI

CI: Ubuntu 18.04 for oneAPI

CI: Intel +Catch -Eigen

CI: CMake from Apt (ICC tests)

CI: Replace Intel Py with GCC Py

CI: Intel w/o GCC's Eigen

CI: ICC with verbose make

[Debug] Find core dump

tests: use arg{} instead of arg() for Intel

tests: adding a few more missing {}

fix: sync with @tobiasleibner's branch

fix: try ubuntu 20-04

fix: drop exit 1

docs: Apply suggestions from code review

Co-authored-by: Tobias Leibner <tobias.leibner@googlemail.com>

Workaround for ICC enable_if issues

Another workaround for ICC's enable_if issues

fix error in previous commit

Disable one test for the Intel compiler in C++17 mode

Add back one instance of py::arg().noconvert()

Add NOLINT to fix clang-tidy check

Work around for ICC internal error in PYBIND11_EXPAND_SIDE_EFFECTS in C++17 mode

CI: Intel ICC with C++17

docs: pybind11/numpy.h does not require numpy at build time. (#2720)

This is nice enough to be mentioned explicitly in the docs.

docs: Update warning about Python 3.9.0 UB, now that 3.9.1 has been released (#2719)

Adjusting `type_caster<std::reference_wrapper<T>>` to support const/non-const propagation in `cast_op`. (#2705)

* Allow type_caster of std::reference_wrapper<T> to be the same as a native reference.

Before, both std::reference_wrapper<T> and std::reference_wrapper<const T> would
invoke cast_op<type>. This doesn't allow the type_caster<> specialization for T
to distinguish reference_wrapper types from value types.

After, the type_caster<> specialization invokes cast_op<type&>, which allows
reference_wrapper to behave in the same way as a native reference type.

* Add tests/examples for std::reference_wrapper<const T>

* Add tests which use mutable/immutable variants

This test is a chimera; it blends the pybind11 casters with a custom
pytype implementation that supports immutable and mutable calls.

In order to detect the immutable/mutable state, the cast_op needs
to propagate it, even through e.g. std::reference<const T>

Note: This is still a work in progress; some things are crashing,
which likely means that I have a refcounting bug or something else
missing.

* Add/finish tests that distinguish const& from &

Fixes the bugs in my custom python type implementation,
demonstrate test that requires const& and reference_wrapper<const T>
being treated differently from Non-const.

* Add passing a const to non-const method.

* Demonstrate non-const conversion of reference_wrapper in tests.

Apply formatting presubmit check.

* Fix build errors from presubmit checks.

* Try and fix a few more CI errors

* More CI fixes.

* More CI fixups.

* Try and get PyPy to work.

* Additional minor fixups. Getting close to CI green.

* More ci fixes?

* fix clang-tidy warnings from presubmit

* fix more clang-tidy warnings

* minor comment and consistency cleanups

* PyDECREF -> Py_DECREF

* copy/move constructors

* Resolve codereview comments

* more review comment fixes

* review comments: remove spurious &

* Make the test fail even when the static_assert is commented out.

This expands the test_freezable_type_caster a bit by:
1/ adding accessors .is_immutable and .addr to compare identity
from python.
2/ Changing the default cast_op of the type_caster<> specialization
to return a non-const value. In normal codepaths this is a reasonable
default.
3/ adding roundtrip variants to exercise the by reference, by pointer
and by reference_wrapper in all call paths.  In conjunction with 2/, this
demonstrates the failure case of the existing std::reference_wrpper conversion,
which now loses const in a similar way that happens when using the default cast_op_type<>.

* apply presubmit formatting

* Revert inclusion of test_freezable_type_caster

There's some concern that this test is a bit unwieldly because of the use
of the raw <Python.h> functions. Removing for now.

* Add a test that validates const references propagation.

This test verifies that cast_op may be used to correctly detect
const reference types when used with std::reference_wrapper.

* mend

* Review comments based changes.

1. std::add_lvalue_reference<type> -> type&
2. Simplify the test a little more; we're never returning the ConstRefCaster
type so the class_ definition can be removed.

* formatted files again.

* Move const_ref_caster test to builtin_casters

* Review comments: use cast_op and adjust some comments.

* Simplify ConstRefCasted test

I like this version better as it moves the assertion that matters
back into python.

ci: drop pypy2 linux, PGI 20.7, add Python 10 dev (#2724)

* ci: drop pypy2 linux, add Python 10 dev

* ci: fix mistake

* ci: commented-out PGI 20.11, drop 20.7

fix: regression with installed pybind11 overriding local one (#2716)

* fix: regression with installed pybind11 overriding discovered one

Closes #2709

* docs: wording incorrect

style: remove redundant instance->owned = true (#2723)

which was just before set to True in instance->allocate_layout()

fix: also throw in the move-constructor added by the PYBIND11_OBJECT macro, after the argument has been moved-out (if necessary) (#2701)

Make args_are_all_* ICC workarounds unconditional

Disable test_aligned on Intel ICC

Fix test_aligned on Intel ICC

Skip test_python_alreadyset_in_destructor on Intel ICC

Fix test_aligned again

ICC CI: Downgrade pytest

pytest 6 does not capture the `discard_as_unraisable` stderr and
just writes a warning with its content instead.

* refactor: simpler Intel workaround, suggested by @laramiel

* fix: try version with impl to see if it is easier to compile

* docs: update README for ICC

Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
2021-01-17 19:53:07 -05:00
Yannick Jadoul
8449a8089c
fix: only allow integer type_caster to call __int__ method when conversion is allowed; always call __index__ (#2698)
* Only allow integer type_caster to call __int__ or __index__ method when conversion is allowed

* Remove tests for __index__ as this seems to only be used to convert to int in 3.8+

* Take both `int` and `long` types into account for Python 2

* Add test_numpy_int_convert to assert tests currently fail, even though np.intc has an __index__ method

* Also consider __index__ as noconvert to a C++ integer

* New-style classes for Python 2.7; sigh

* Add some tests on types with custom __index__ method

* Ignore some tests in Python <3.8

* Update comment about conversion from np.float32 to C++ int

* Workaround difference between CPython and PyPy's different PyIndex_Check (unnoticed because we currently don't have PyPy >= 3.8)

* Avoid ICC segfault with py::arg()
2021-01-16 20:52:14 -05:00
Yannick Jadoul
0855146357
Plug leaking function_records in cpp_function initialization in case of exceptions (found by Valgrind in #2746) (#2756)
* Plug leaking function_record objects when exceptions are thrown

* Plug leak of strdup'ed strings in function_record

* Some extra comments about the function_record ownership dance

* Clean up the function_record better, in case of exceptions

* Demonstrate some extra function_record leaks

* Change  DeleteStrings template argument to free_strings runtime argument in destruct(function_record *)

* Zero-state unique_function_record deleter object

* Clarify rvalue reference to unique_ptr parameter in initialize_generic

* Use push_back with const char * instead of emplace_back
2021-01-14 19:34:32 +01:00
Andy Maloney
40931961e3
docs: fix spelling in some comments/docs (#2777)
Found with codespell
2021-01-13 23:15:58 -05:00
Andy Maloney
df8494dc86
fix: a clang warning [-Wshadow-field-in-constructor-modified] (#2780)
* Fix a clang warning from [-Wshadow-field-in-constructor-modified]

warning: modifying constructor parameter 'flag' that shadows a field of 'set_flag' [-Wshadow-field-in-constructor-modified]

* Change name based on review
2021-01-13 23:15:27 -05:00
Changming Sun
210c8c218f
fix: a warning found by static code analyzer (#2783)
* Update attr.h: fix a warning found by static code analyzer

Update attr.h: fix a warning found by Visual Studio static code analyzer

Severity:Warning
Code:C6323
Description: Use of arithmetic operator on Boolean type(s).
Location:	C:\src\onnxruntime\debug\pybind11\src\pybind11\include\pybind11\attr.h:547

* Update include/pybind11/attr.h

Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>

* Update attr.h

Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
2021-01-13 23:14:06 -05:00
Yannick Jadoul
e57dd4717e
Fix various minor memory leaks in the tests (found by Valgrind in #2746) (#2758)
* Fix leak in the test_copy_move::test_move_fallback

* Fix leaking PyMethodDef in test_class::test_implicit_conversion_life_support

* Plumb leak in test_buffer, occuring when a mutable buffer is requested for a read-only object, and enable test_buffer.py

* Fix weird return_value_policy::reference in test_stl_binders, and enable those tests

* Cleanup nodelete holder objects in test_smart_ptr, and enable those tests
2021-01-01 17:05:22 +01:00
Yannick Jadoul
e612043d43
Fix invalid access when reinterpret_casting a non-pybind11 PyObject* to instance* (found by Valgrind in #2746) (#2755) 2020-12-31 17:10:11 +01:00
Eric Cousineau
2110d2d8ba
enum: add missing Enum.value property (#2739)
* enum: Add Enum.value property

* simplify

* address review
2020-12-31 11:08:15 -05:00
Qifan Lu
d587a2fd17
fix: do not set docstring for function when empty (#2745)
* Do not set docstring for function when it's empty

* No need to check pointer for `free`

* Use ternary operator to conditionally set `ml_doc`
2020-12-27 22:56:30 -05:00
Nikita Shulga
79cb013f1f
fix: allow users to avoid thread termination in scoped_released (#2657)
* Avoid thread termination in scoped_released

Do not call `PyEval_RestoreThread()` from `~gil_scoped_release()` if python runtime is finalizing, as it will result in thread termination in Python runtime newer than 3.6, as documented in https://docs.python.org/3/c-api/init.html#c.PyEval_RestoreThread
Similarly do not call `PyThreadState_DeleteCurrent` from `~gil_scoped_acquire()` if runtime is finalizing.

Discovered while debugging PyTorch crash using Python-3.9 described in  https://github.com/pytorch/pytorch/issues/47776

* Simplify _Py_IsFinalizing() availability check

* Fix typo

* Add version agnostic `detail::finalization_guard()`

* Move `finalization_guard` to detail/common.h

And rename it to `is_finalizing`

* Move `is_finalizing()` back to pybind11.h

* Simplify `is_finalizing()` check

One should follow documentation rather than make any assumptions

* feat: disarm

* docs: fix comment

Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
2020-12-19 15:45:19 -05:00
Yannick Jadoul
30eb39ed79
fix: also throw in the move-constructor added by the PYBIND11_OBJECT macro, after the argument has been moved-out (if necessary) (#2701) 2020-12-15 23:22:53 -05:00
Robert Haschke
d9fa70561e
style: remove redundant instance->owned = true (#2723)
which was just before set to True in instance->allocate_layout()
2020-12-15 23:21:40 -05:00
Laramie Leavitt
5469c238c8
Adjusting type_caster<std::reference_wrapper<T>> to support const/non-const propagation in cast_op. (#2705)
* Allow type_caster of std::reference_wrapper<T> to be the same as a native reference.

Before, both std::reference_wrapper<T> and std::reference_wrapper<const T> would
invoke cast_op<type>. This doesn't allow the type_caster<> specialization for T
to distinguish reference_wrapper types from value types.

After, the type_caster<> specialization invokes cast_op<type&>, which allows
reference_wrapper to behave in the same way as a native reference type.

* Add tests/examples for std::reference_wrapper<const T>

* Add tests which use mutable/immutable variants

This test is a chimera; it blends the pybind11 casters with a custom
pytype implementation that supports immutable and mutable calls.

In order to detect the immutable/mutable state, the cast_op needs
to propagate it, even through e.g. std::reference<const T>

Note: This is still a work in progress; some things are crashing,
which likely means that I have a refcounting bug or something else
missing.

* Add/finish tests that distinguish const& from &

Fixes the bugs in my custom python type implementation,
demonstrate test that requires const& and reference_wrapper<const T>
being treated differently from Non-const.

* Add passing a const to non-const method.

* Demonstrate non-const conversion of reference_wrapper in tests.

Apply formatting presubmit check.

* Fix build errors from presubmit checks.

* Try and fix a few more CI errors

* More CI fixes.

* More CI fixups.

* Try and get PyPy to work.

* Additional minor fixups. Getting close to CI green.

* More ci fixes?

* fix clang-tidy warnings from presubmit

* fix more clang-tidy warnings

* minor comment and consistency cleanups

* PyDECREF -> Py_DECREF

* copy/move constructors

* Resolve codereview comments

* more review comment fixes

* review comments: remove spurious &

* Make the test fail even when the static_assert is commented out.

This expands the test_freezable_type_caster a bit by:
1/ adding accessors .is_immutable and .addr to compare identity
from python.
2/ Changing the default cast_op of the type_caster<> specialization
to return a non-const value. In normal codepaths this is a reasonable
default.
3/ adding roundtrip variants to exercise the by reference, by pointer
and by reference_wrapper in all call paths.  In conjunction with 2/, this
demonstrates the failure case of the existing std::reference_wrpper conversion,
which now loses const in a similar way that happens when using the default cast_op_type<>.

* apply presubmit formatting

* Revert inclusion of test_freezable_type_caster

There's some concern that this test is a bit unwieldly because of the use
of the raw <Python.h> functions. Removing for now.

* Add a test that validates const references propagation.

This test verifies that cast_op may be used to correctly detect
const reference types when used with std::reference_wrapper.

* mend

* Review comments based changes.

1. std::add_lvalue_reference<type> -> type&
2. Simplify the test a little more; we're never returning the ConstRefCaster
type so the class_ definition can be removed.

* formatted files again.

* Move const_ref_caster test to builtin_casters

* Review comments: use cast_op and adjust some comments.

* Simplify ConstRefCasted test

I like this version better as it moves the assertion that matters
back into python.
2020-12-15 16:53:55 -08:00
Boris Staletic
8adef2c7f6
fix: workaround for #2682 and #2422 by simply clearing the TypeError (#2685) 2020-11-23 14:02:25 -05:00
albanD
087b07c8b7
Remove workaround code that is not needed since #1211 (#2683) 2020-11-23 14:00:35 -05:00
Tobias Leibner
7bd4b39754
fix: define PYBIND11_CPP14 for recent intel compilers (#2679) 2020-11-23 09:11:04 -05:00
nickbridgechess
2fa4747cd4
pythonbuf fix (#2675)
* Added test_thread testing for ostream_redirect segfault recreation

* fix: scoped_ostream_redirect str created outside gil

* Moved threading tests into test_iostream. Cleaned up some formatting. Deleted test_thread.{cpp,py}

* CI: few formatting fixes

* CI: yet another formatting fix

* CI: more formatting fixes. Removed unecessary comment

* Ignore 'warning C4702: unreachable code' in MSVC 2015

Co-authored-by: Nick Bridge <nick.bridge.chess@gmail.com>
Co-authored-by: Nick Bridge <nbridge@jumptrading.com>
Co-authored-by: Yannick Jadoul <yannick.jadoul@belgacom.net>
2020-11-19 12:09:33 +01:00
Henry Schreiner
b7c741b540 docs: back to work after 2.6.1 2020-11-11 19:27:41 -05:00
Henry Schreiner
f1abf5d915
docs: changelog update (#2652) 2020-11-11 16:33:21 -05:00
Yannick Jadoul
c58758d049
fix: add reasonable argument names to enum_ methods (#2637)
* Add argument names to enum_ methods

* Add test_enum::test_docstring_signatures
2020-11-10 12:49:42 -05:00
Robert Haschke
b72cebeb22
style: clang-tidy: modernize-use-using (#2645)
* style: clang-tidy: modernize-use-using

* style: more clang-tidy checking

Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
2020-11-09 10:10:19 -08:00
Boris Staletic
06b673a0da
Allow NULL value in pybind11_meta_setattro (#2629) 2020-11-05 23:54:37 +01:00
Yannick Jadoul
7d6713a46d
Use weakref to clean up captured function object in def_buffer (#2634) 2020-11-02 18:39:40 +01:00
Thomas Köppe
f2e799863b
[common.h] Mark another entry point as "unused". (#2625) 2020-10-29 12:47:59 +01:00
Yannick Jadoul
3a37d33830
Add __builtins__ to globals argument of py::exec and py::eval if not present (#2616)
* Add __builtins__ to globals argument of `py::exec` and `py::eval` if not present

* Refactor into inline ensure_builtins_in_globals function
2020-10-27 19:58:27 +01:00
Wenzel Jakob
86d3e9eb43 begin working towards a future v2.6.1 patch release 2020-10-21 18:31:10 +02:00
Wenzel Jakob
59a2ac2745 v2.6.0 release 2020-10-21 18:07:48 +02:00
Boris Staletic
4038542b15
Remove friend declaration of py::class_ in py::detail::generic_type (#2613)
This line had two bugs:

1. It declares `py::detail::class_` as `friend`.
2. After fixing that, we would have to change it to
   `template <typename, typename...>`

The first one was introduced ~5 years ago, when a large refactoring was
made, probably as an intermediate step during refactoring.

The second was made when `generic_type` was made to be agnostic with
respect to the order of `py::class_` template parameters.

&nbsp;

We're removing the declaration altogether, because it was never relied
on. This is what makes me think that it was an intermediate step in
refactoring that shouldn't have ended up in commit history.
2020-10-21 13:14:56 +02:00
Mana Borwornpadungkitti
6edd0e6d90
fix: Reject keyword argument None with .none(false) (#2611)
* demo kwarg with none(false)

* Reorder and extend tests for arg::none(false) in test_methods_and_attributes.py::test_accepts_none

* Fix arg::none() for keyword arguments

* Add changelog note

* Fix names of no_none_kw test functions

Co-authored-by: Yannick Jadoul <yannick.jadoul@belgacom.net>
2020-10-20 23:57:22 +02:00
Henry Schreiner
c16da99309 chore: bump to 2.6.0rc3 2020-10-16 17:34:53 -04:00
Henry Schreiner
064362fbb7
fix: allow the ABI string to be forced (#2602) 2020-10-16 17:23:36 -04:00
Yannick Jadoul
6364b732e9
fix: test_factory_constructors.py failure triggered by test_register_duplicate_class (#2564)
* Demonstrate test_factory_constructors.py failure without functional changes from #2335

* Revert "Demonstrate test_factory_constructors.py failure without functional changes from #2335"

This reverts commit ca33a8021fc2a3617c3356b188796528f4594419.

* Fix test crash where registered Python type gets garbage collected

* Clean up some more internal structures when class objects go out of scope

* Reduce length of std::erase_if-in-C++20 comment

* Clean up code for cleaning up type internals

* Move cleaning up of type info in internals to tp_dealloc on pybind11_metaclass
2020-10-16 16:38:51 -04:00
Henry Schreiner
6bb71c48d5 fix: more intp asserts, reinterpret_cast 2020-10-15 17:38:49 -04:00
Henry Schreiner
9ac604a3e8 fix: PyPy windows issue 2020-10-15 17:38:49 -04:00
Henry Schreiner
b72c79f964 fix: std::is_pod -> pybind11::detail::is_pod 2020-10-15 17:38:49 -04:00
Henry Schreiner
c4a8b5bb91 chore: bump to 2.6.0rc2 2020-10-14 16:37:25 -04:00
Yannick Jadoul
493649f965
fix: valgrind-detected after-freeing access of PyMethodDef (macOS Python 3.9.0 segfaults) (#2576)
* Check if valgrind-detected after-freeing access of PyMethodDef causes macOS Python 3.9 segfaults

* fix: only apply leak on 3.9.0

* fix: faster check

* fix: better naming thanks to @bstaletic

Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
2020-10-14 14:11:09 -04:00
Yannick Jadoul
99773fc5f6
fix: throw error_already_set in py::len on failing PyObject_Length (#2575)
* Throw error_already_set in py::len on failing PyObject_Length

* Fix tests to mach error message on PyPy
2020-10-12 23:00:54 -04:00
Yannick Jadoul
1914b7d3a7
Shorten PYBIND11_EMBEDDED_MODULE macro implementation by using PYBIND11_CATCH_INIT_EXCEPTIONS (#2579) 2020-10-12 23:10:18 +02:00
Henry Schreiner
993495c96c
fix: Intel 18+ required (#2577)
* fix: Intel 18+ fully supported

* fix: Intel compiler workaround no longer needed

Followup on #94 now that Intel 18+ is required.
2020-10-12 16:31:44 -04:00
Henry Schreiner
fecef38864
refactor: simpler followup to #2569 (#2572) 2020-10-09 17:12:05 -04:00
Henry Schreiner
f385eccdb7 chore: bump to 2.6.0rc1 2020-10-09 16:41:42 -04:00
Ahmed Sobhy
8d8d48c528
fix: do not use abi::__forced_unwind with libc++, even with gcc instead of clang (#2569)
* add support for LLVM compiler

* Update include/pybind11/pybind11.h

__forced_unwind libc++ vs libstdc++

Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>

Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
2020-10-09 16:10:23 -04:00
Yannick Jadoul
0c5cc031ee
feat: deprecate public constructors of module_ class (#2552)
* Deprecated public constructors of module

* Turn documentation comment of module_::add_object into valid doxygen documentation

* Move definition of PYBIND11_DETAIL_MODULE_STATIC_DEF and PYBIND11_DETAIL_MODULE_CREATE macros up

* Move detail::create_top_level_module to module_::create_extension_module, and unify Python 2 and 3 signature again

* Throw error_already_set if module creation fails in module_::create_extension_module

* Mention module_::create_extension_module in deprecation warning message of module_::module_
2020-10-09 10:46:11 -04:00
Yannick Jadoul
71aea49b8b
Check scope's __dict__ instead of using hasattr when registering classes and exceptions (#2335)
* Check scope's __dict__ instead of using hasattr when registering classes and exceptions, to allow registering the same name in a derived class scope

* Extend test_base_and_derived_nested_scope test

* Add tests on error being thrown registering duplicate classes

* Circumvent bug with combination of test_class.py::test_register_duplicate_class and test_factory_constructors.py::test_init_factory_alias
2020-10-09 01:09:56 +02:00
Henry Schreiner
00edc3001b
fix: PYBIND11_OBJECT required pybind11 namespace (regression) (#2553)
* fix: PYBIND11_OBJECT could only be used inside the pybind11 namespace (regression)

* docs: add changelog for conversion protection change

* ci: update to Python 3.9
2020-10-06 10:04:13 -04:00
Henry Schreiner
9a0c96dd4c
feat: py::prepend tag (#1131)
* feat: add a priority overload with py::prepend

* doc: fix wording as suggested by rwgk

* feat: add get_pointer

* refactor: is_prepended -> prepend (internal)

* docs: suggestion from @wjakob

* tests: add test covering get_pointer/set_pointer
2020-10-05 22:36:33 -04:00
Yannick Jadoul
f537093a2f
Fail on passing py::object with wrong Python type to py::object subclass using PYBIND11_OBJECT macro (#2349)
* Fail on passing py::object with wrong Python type to py::object subclass using PYBIND11_OBJECT macro

* Split off test_non_converting_constructors from test_constructors

* Fix test_as_type, as py::type constructor now throws an error itself if the argument is not a type

* Replace tp_name access by pybind11::detail::get_fully_qualified_tp_name

* Move forward-declaration of get_fully_qualified_tp_name to detail/common.h

* Don't add the builtins module name in get_fully_qualified_tp_name for PyPy

* Add PYBIND11_BUILTINS_MODULE macro, and use it in get_fully_qualified_tp_name
2020-10-05 22:48:54 +02:00
Yannick Jadoul
1411207711
chore: drop support for PyPy < 7.3.1 and clean up old PyPy workarounds (#2456)
* Remove code inside 'PYPY_VERSION_NUM < 0x06000000' preprocessor if branch

* fix: more cleanup

* Remove more references to PyPy 5.7 and 5.9 in the docs

* Update comment on PyUnicode_UTF* in PyPy

Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
2020-10-05 10:43:27 -04:00
Fritz Reese
e8ad33bb30
Fix buffer_info for ctypes buffers (pybind#2502) (#2503)
* tests: New test for ctypes buffers (pybind#2502)

* fix: fix buffer_info segfault on views with no stride (pybind11#2502)

* Explicit conversions in buffer_info to make clang happy (pybind#2502)

* Another explicit cast in buffer_info constructor for clang (pybind#2502)

* Simpler implementation of buffer_info constructor from Py_buffer.

* Move test_ctypes_buffer into test_buffers

* Comment on why view->strides may be NULL (and fix some whitespace)

* Use c_strides() instead of zero when view->strides is NULL.

c_strides and f_strides are moved from numpy.h (py::array)
to buffer_info.h (py::detail) so they can be used from the
buffer_info Py_buffer constructor.

* Increase ctypes buffer test coverage in test_buffers.

* Split ctypes tests and skip one which is broken in PyPy2.
2020-10-03 23:09:14 +02:00
Henry Schreiner
6bcd220c8d
refactor: module -> module_ with typedef (#2544)
* WIP: module -> module_ without typedef

* refactor: allow py::module to work again
2020-10-03 13:38:03 -04:00
Hyrum Wright
961b2e6205
fix: ensure the GIL is held when copying a function. (#2545)
Co-authored-by: Hyrum Wright <hwright@google.com>
2020-10-02 17:00:45 -04:00
Yannick Jadoul
9796fe98fc
feat: vectorize functions with void return type (#1969)
* Allow function/functor passed to py::vectorize to return void

* Stealing @sizmailov's test and fixing unused argument warning

* Add missing std::move()

RVO doesn't work here because function return type is different from
actual returned type

* remove extra EOL

* docs: add a few details

* chore: pre-commit autoupdate

* Remove array_iterator, array_begin, and array_end (in detail namespace)

Co-authored-by: Sergei Izmailov <sergei.a.izmailov@gmail.com>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
2020-10-02 15:30:34 -04:00
Yannick Jadoul
56784c4f42
Add unchecked_reference::operator() and operator[] to overload resolution of unchecked_mutable_reference (#2514) 2020-10-02 19:07:04 +02:00
Riyaz Haque
2b6b98e28f
Bugfix/Check actual value when deregistering pybind11 instance (#2252)
* Add tests demonstrating the problem with deregistering pybind11 instances

* Fix deregistering of different pybind11 instance from internals

Co-authored-by: Yannick Jadoul <yannick.jadoul@belgacom.net>
Co-authored-by: Blistic <wots_wot@hotmail.com>
2020-10-02 19:06:04 +02:00
Yannick Jadoul
07b069a55b
Unify Python 2 & 3 py::module constructor, and make contructor with pre-allocated PyModuleDef private (#2534) 2020-10-02 10:01:24 -04:00
Boris Staletic
5ebc78164d
Allow raw unions without base classes in is_accessible_base_of (#2320)
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
2020-10-02 09:39:22 -04:00
Yannick Jadoul
c72708a746
Moving tp_class access, and consistent fully-qualified naming for PyPy, to detail::get_tp_name (#2520)
* Moving tp_class access, and consistent fully-qualified naming for PyPy, to detail::get_tp_name

* Change get_tp_name to get_fully_qualified_tp_name
2020-10-01 22:57:25 -04:00
Henry Schreiner
0fc5774c0c
fix: beta does not have a dot in front (#2539) 2020-09-30 16:32:19 -04:00
Henry Schreiner
2190246243 chore: bump to 2.6.0b1, beta release 2020-09-30 16:10:26 -04:00
Ralf W. Grosse-Kunstleve
d159a56338
static allocation for PyModuleDef, to avoid leak check errors. (#2413)
* Initializing PyModuleDef object with PyModuleDef_HEAD_INIT.

Python 3.8 documentation: m_base - Always initialize this member to PyModuleDef_HEAD_INIT.

Long-standing (since first github commit in 2015), inconsequential bug.

Also removing inconsequential Py_INCREF(def): PyModule_Create() resets the reference count to 1.

* git rebase master

* moving static PyModuleDef declaration to global scope, as requested by @wjakob

* renaming the two new macros, to start with PYBIND11_DETAIL_MODULE
2020-09-26 22:55:11 -07:00
Henry Schreiner
3c7ef56bb6
fix: warning on latest AppleClang (#2522)
Fixed in #2510 but reintroduced on one line by #2126
2020-09-22 23:36:25 +02:00
David Vo
5e6ec49652
Add enum value to enum repr (#2126)
This changes enum reprs to look like `<Enum.name: value>` similarly to
the Python enum module.

This keeps the str of enums as `Enum.name`, like the Python enum module.
2020-09-20 01:12:19 +02:00
Rickard Hallerbäck
5a8ec8e505
Include what is used in pybind11.h (#2499)
* Include what is used

* Separated the C++ standard library headers and the project headers

* Reordering includes
2020-09-19 20:25:46 +02:00
Henry Schreiner
d0ed035cc5
fix: AppleClang 12 warnings (#2510)
* fix: AppleClang 12 new warning

* Fix: AppleClang X.X.0 will not trigger this warning
2020-09-19 20:23:47 +02:00
Henry Fredrick Schreiner
11f756f5a9 fix: type bug intruduced in #2492
This now tests the old form too, and fixes the bug introduced.
2020-09-16 23:21:47 -04:00
Boris Staletic
d3c999c774
fix: rename pybind11::module to pybind11::module_ (#2489)
Support C++20. For backwards compatibility, we provide an alias for the old name.
This change is necessary to easily avoid errors when a compiler thinks
`module` is used as a keyword.
2020-09-16 17:15:42 -04:00
Henry Schreiner
fd61f5038e
feat: setup.py redesign and helpers (#2433)
* feat: setup.py redesign and helpers

* refactor: simpler design with two outputs

* refactor: helper file update and Windows support

* fix: review points from @YannickJadoul

* refactor: fixes to naming and more docs

* feat: more customization points

* feat: add entry point pybind11-config

* refactor: Try Extension-focused method

* refactor: rename alt/inplace to global

* fix: allow usage with git modules, better docs

* feat: global as an extra (@YannickJadoul's suggestion)

* feat: single version location

* fix: remove the requirement that setuptools must be imported first

* fix: some review points from @wjacob

* fix: use .in, add procedure to docs

* refactor: avoid monkeypatch copy

* docs: minor typos corrected

* fix: minor points from @YannickJadoul

* fix: typo on Windows C++ mode

* fix: MSVC 15 update 3+ have c++14 flag

See <https://docs.microsoft.com/en-us/cpp/build/reference/std-specify-language-standard-version?view=vs-2019>

* docs: discuss making SDists by hand

* ci: use pep517.build instead of manual setup.py

* refactor: more comments from @YannickJadoul

* docs: updates from @ktbarrett

* fix: change to newly recommended tool instead of pep517.build

This was intended as a proof of concept; build seems to be the correct replacement.

See https://github.com/pypa/pep517/pull/83

* docs: updates from @wjakob

* refactor: dual version locations

* docs: typo spotted by @wjakob
2020-09-16 17:13:41 -04:00
Henry Schreiner
41aa92601e
refactor: replace .get_type with type::handle_of (#2492)
* refactor: replace .get_type with type::handle_of

* refactor: use impl for handle_of

* fix: deprecate h.get_type()
2020-09-16 11:32:17 -04:00
Yannick Jadoul
16f199f8d9
Change base parameter type in register_exception and exception constructor from PyObject* to handle (#2467)
* Change base parameter type in register_exception and excepion constructor from PyObject* to handle

* Fix compilation error passing `handle` to `PyObject*`
2020-09-15 10:24:39 -04:00
Henry Schreiner
e7bafc8ec1 style: clang-tidy: default checks and fix bug in iostream deconstruction
```
/pybind11/include/pybind11/iostream.h:71:9: warning: Call to virtual method 'pythonbuf::sync' during destruction bypasses virtual dispatch [clang-analyzer-optin.cplusplus.VirtualCall]
        sync();
        ^
/pybind11/tests/test_iostream.cpp:72:5: note: Calling '~scoped_ostream_redirect'
    });
```
2020-09-15 09:56:59 -04:00
Henry Schreiner
4d78640830 style: clang-tidy: modernize-use-emplace 2020-09-15 09:56:59 -04:00
Henry Schreiner
ce88e940ce style: clang-tidy: modernize-use-auto 2020-09-15 09:56:59 -04:00
Henry Schreiner
b491b465c7 style: clang-tidy: modernize-use-equals-default 2020-09-15 09:56:59 -04:00
Henry Schreiner
b342c37388 style: clang-tidy: modernize-use-using 2020-09-15 09:56:59 -04:00
Henry Schreiner
96e6a8d554 style: clang-tidy: readability-container-size-empty 2020-09-15 09:56:59 -04:00
Henry Schreiner
5dfbe6f903 style: clang-tidy: modernize-use-override 2020-09-15 09:56:59 -04:00
Henry Schreiner
8dc31c7b29 style: clang-tidy: llvm-namespace-comment 2020-09-15 09:56:59 -04:00
Yannick Jadoul
d65e34d61d
Resolve empty statement warning when using PYBIND11_OVERLOAD_PURE_NAME and PYBIND11_OVERLOAD_PURE (#2325)
* Wrap PYBIND11_OVERLOAD_NAME and PYBIND11_OVERLOAD_PURE_NAME in do { ... } while (false), and resolve trailing semicolon

* Deprecate PYBIND11_OVERLOAD_* and get_overload in favor of PYBIND11_OVERRIDE_* and get_override

* Correct erroneous usage of 'overload' instead of 'override' in the implementation and internals

* Fix tests to use non-deprecated PYBIND11_OVERRIDE_* macros

* Update docs to use override instead of overload where appropriate, and add warning about deprecated aliases

* Add semicolons to deprecated PYBIND11_OVERLOAD macros to match original behavior

* Remove deprecation of PYBIND11_OVERLOAD_* macros and get_overload

* Add note to changelog and upgrade guide
2020-09-15 14:56:20 +02:00
Yannick Jadoul
9df13835c8
Stop py::array_t arguments from accepting arrays that do not match the C- or F-contiguity flags (#2484)
* Stop py::array_t arguments from accepting arrays that do not match the C- or F-contiguity flags

* Add trivially-contiguous arrays to the tests
2020-09-15 14:50:51 +02:00
Henry Schreiner
f12ec00d70
feat: py::type::of<T>() and py::type::of(h) (#2364)
* feat: type<T>()

* refactor: using py::type as class

* refactor: py::object as base

* wip: tigher api

* refactor: fix conversion and limit API further

* docs: some added notes from @EricCousineau-TRI

* refactor: use py::type::of
2020-09-14 18:06:26 -04:00
Boris Staletic
cc982ac1cd
fix: allow assignment of time points of resolutions other than that of a system clock (#2481) 2020-09-13 10:24:00 -04:00
andriish
38370a87f4
fix: support NVIDIA-PGI HPC SDK (#2475)
* Added guards to the includes

Added new CI config

Added new trigger

Changed CI workflow name

Debug CI

Debug CI

Debug CI

Debug CI

Added flags fro PGI

Disable Eigen

Removed tests that fail

Uncomment lines

* fix: missing include

fix: minor style cleanup

tests: support skipping

ci: remove and tighten a bit

fix: try msvc workaround for pgic

* tests: split up prealoc tests

* fix: PGI compiler fix

* fix: PGI void_t only

* fix: try to appease nvcc

* ci: better ordering for slow tests

* ci: minor improvements to testing

* ci: Add NumPy to testing

* ci: Eigen generates CUDA warnings / PGI errors

* Added CentOS7 back for a moment

* Fix YAML

* ci: runs-on missing

* centos7 is missing pytest

* ci: use C++11 on CentOS 7

* ci: test something else

* Try just adding flags on CentOS 7

* fix: CentOS 7

* refactor: move include to shared location

* Added verbose flag

* Try to use system cmake3 on CI

* Try to use system cmake3 on CI, attempt2

* Try to use system cmake3 on CI, attempt3

* tests: not finding pytest should be a warning, not a fatal error

* tests: cleanup

* Weird issue?

* fix: final polish

Co-authored-by: Andrii Verbytskyi <andrii.verbytskyi@mpp.mpg.de>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Andrii Verbytskyi <averbyts@cern.ch>
2020-09-11 22:06:52 -04:00
Yannick Jadoul
fe9ee86ba8
Add check if str(handle) correctly converted the object, and throw py::error_already_set if not (bis) (#2477)
* Add check if `str(handle)` correctly converted the object, and throw py::error_already_set if not

* Fix tests on Python 3

* Apply @rwgk's fixes to cherry-picked commits from #2392
2020-09-11 19:53:04 +02:00
Ciro Santilli
b47efd35fb
Use defined for some preprocessor variables that might be undefined (#2476)
The variables PYBIND11_HAS_OPTIONAL, PYBIND11_HAS_EXP_OPTIONAL, PYBIND11_HAS_VARIANT,
__clang__, __APPLE__ were not checked for defined in a minortity of instances.

If the project using pybind11 sets -Wundef, the warnings will show.

The test build is also modified to catch the problem.
2020-09-10 13:58:26 -04:00
Henry Schreiner
621906b3e7
fix: support nvcc and test (#2461)
* fix: support nvcc and test

* fixup! fix: support nvcc and test

* docs: mention what compilers fail

* fix: much simpler logic

* refactor: slightly faster / clearer
2020-09-10 11:49:26 -04:00
Henry Schreiner
0dbda6e80b
feat: py::pos_only (#2459)
* feat: py::pos_only

* fix: review points from @YannickJadoul

* fix: review points from @bstaletic

* refactor: kwonly -> kw_only
2020-09-04 20:02:05 -04:00
Ralf W. Grosse-Kunstleve
3c061f2168 Fixing pybind11::bytes() ambiguous conversion issue.
Adding missing `bytes` type to `test_constructors()`, to exercise the code change.

The changes in the PR were cherry-picked from PR #2409 (with a very minor
modification in test_pytypes.py related to flake8). Via PR #2409, these
changes were extensively tested in the Google environment, as summarized here:
https://docs.google.com/document/d/1TPL-J__mph_yHa1quDvsO12E_F5OZnvBaZlW9IIrz8M/
The changes in this PR did not cause an issues at all.

Note that `test_constructors()` before this PR passes for Python 2 only
because `pybind11::str` can hold `PyUnicodeObject` or `PyBytesObject`. As a
side-effect of this PR, `test_constructors()` no longer relies on this
permissive `pybind11::str` behavior. However, the permissive behavior is still
exercised/exposed via the existing `test_pybind11_str_raw_str()`.

The test code change is designed to enable easy removal later, when Python 2
support is dropped.

For completeness: confusingly, the non-test code changes travelled through PR

Example `ambiguous conversion` error fixed by this PR:
```
pybind11/tests/test_pytypes.cpp:214:23: error: ambiguous conversion for functional-style cast from 'pybind11::detail::item_accessor' (aka 'accessor<accessor_policies::generic_item>') to 'py::bytes'
            "bytes"_a=py::bytes(d["bytes"]),
                      ^~~~~~~~~~~~~~~~~~~~
pybind11/include/pybind11/detail/../pytypes.h:957:21: note: candidate constructor
    PYBIND11_OBJECT(bytes, object, PYBIND11_BYTES_CHECK)
                    ^
pybind11/include/pybind11/detail/../pytypes.h:957:21: note: candidate constructor
pybind11/include/pybind11/detail/../pytypes.h:987:15: note: candidate constructor
inline bytes::bytes(const pybind11::str &s) {
              ^
1 error generated.
```
2020-08-28 11:52:51 -07:00
Yannick Jadoul
6a192781fc
Fix bug roundtripping datetime.time objects after midnight in eastern hemisphere timezones (#2417) (#2438)
* Fix bug roundtripping datetime.time objects after midnight in eastern hemisphere timezones (#2417)

* tests: check more timezones

* Fix review remarks: remove useless comment and skip setting TZ environment variable on Windows
2020-08-28 15:21:43 +02:00
Dekken
1abc4a9de5
fix: doc typo, drop second use of 'without' (#2439) 2020-08-27 00:55:18 +02:00
Yannick Jadoul
a2bb297b32
Throw exception on returning a unique_ptr or shared_ptr nullptr (or any other holder type) from py::init, rather than crashing (#2430) 2020-08-25 18:51:06 +02:00
Henry Schreiner
56df3c4649
fix: a couple more places where pybind11 is missing 11 (#2421) 2020-08-21 15:27:21 -04:00
Henry Schreiner
cf0a64596e
fix: throwing repr caused a segfault (#2389)
* fix: throwing repr caused a segfault

* fixup! ci: include Python 3.9 RC1 (#2387)
2020-08-18 07:14:34 -04:00
James R. Barlow
3618bea2aa Add and document py::error_already_set::discard_as_unraisable()
To deal with exceptions that hit destructors or other noexcept functions.

Includes fixes to support Python 2.7 and extends documentation on
error handling.

@virtuald and @YannickJadoul both contributed to this PR.
2020-08-16 10:05:03 -07:00
Michael Goulding
fb042d692f
Fix warning C26817 on copying in for (auto vh : value_and_holder(...)) (#2382)
* Fix warning C26817: Potentially expensive copy of variable 'vh' in range-for loop. Consider making it a const reference (es.71).

* Replace another instance of `for (auto vh : values_and_holders(...))` with `auto vh &` (found by @bstaletic)

Co-authored-by: Michael Goulding <Michael.Goulding@microsoft.com>
Co-authored-by: Yannick Jadoul <yannick.jadoul@belgacom.net>
2020-08-14 18:15:50 +02:00
Henry Schreiner
2e2de8c87a
fix: add missing signature (#2363)
* fix: add missing signature

* fix: add to array_t too
2020-08-13 20:13:16 -04:00
marc-chiesa
830adda850
Modified Vector STL bind initialization from a buffer type with optimization for simple arrays (#2298)
* Modified Vector STL bind initialization from a buffer type with optimization for simple arrays

* Add subtests to demonstrate processing Python buffer protocol objects with step > 1

* Fixed memoryview step test to only run on Python 3+

* Modified Vector constructor from buffer to return by value for readability
2020-08-13 22:47:23 +02:00
Yannick Jadoul
3e448c0b5e
Enable py::ellipsis on Python 2 (#2360)
* Enable py::ellipsis on Python 2

* Enable py::ellipsis tests on Python 2 and mention `Ellipsis` in the docs
2020-08-04 14:45:55 +02:00
jbarlow83
4d90f1a199
Add error_scope to py::class_::dealloc() to protect destructor calls (#2342)
Fixes issue #1878
2020-07-31 17:46:12 -07:00
Henry Schreiner
1651c32492 update: address review points 2020-07-30 20:27:55 -04:00
Henry Schreiner
6ec1775fff feat: drop CMake 3.6 and below, modernize CMake
fix: include PYTHON_IS_DEBUG
2020-07-30 20:27:55 -04:00
Marcin Wojdyr
8e40e389fd
cast pointer to std::tuple and std::pair (#2334) 2020-07-28 21:44:19 +02:00
Sergei Izmailov
7b067cc387
Set __hash__ to None for types that defines __eq__, but not __hash__ (#2291)
fixes #2191
2020-07-27 01:44:25 +02:00
Henry Schreiner
4470671796 fix: too many braces on clang 3.6 2020-07-26 09:25:27 -04:00
Henry Schreiner
e428a7f6b8 ci: fix clang warnings 2020-07-26 09:25:27 -04:00
Boris Staletic
2819ce64a4
Avoid attr("__repr__") in initialize_generic (#2317)
If the default argument value is a class, and not an instance of a
class, `a.value.attr("__repr__")` raises a `ValueError`. Switching to
`repr(a.value)` makes this use case work.

Fixes #2028
2020-07-24 18:43:59 +02:00
Kota Yamaguchi
e248869893
Fix undefined memoryview format (#2223)
* Fix undefined memoryview format

* Add missing <algorithm> header

* Add workaround for py27 array compatibility

* Workaround py27 memoryview behavior

* Fix memoryview constructor from buffer_info

* Workaround PyMemoryView_FromMemory availability in py27

* Fix up memoryview tests

* Update memoryview test from buffer to check signedness

* Use static factory method to create memoryview

* Remove ndim arg from memoryview::frombuffer and add tests

* Allow ndim=0 memoryview and documentation fixup

* Use void* to align to frombuffer method signature

* Add const variants of frombuffer and frommemory

* Add memory view section in doc

* Fix docs

* Add test for null buffer

* Workaround py27 nullptr behavior in test

* Rename frombuffer to from_buffer
2020-07-15 08:50:43 -07:00
Boris Staletic
aa982e131d
Small fixes in numpy.h (#2293)
- `PyArray_NewFromDescr_` should have been using `Py_intptr_t const *`
  - 18a6e3e505/numpy/core/src/multiarray/ctors.h (L5-L8)
- `PyArray_GetArrayParamsFromObject_` should be using `NPY_BOOL`
  - 18a6e3e505/numpy/core/src/multiarray/ctors.c (L1350-L1355)
  - https://docs.scipy.org/doc/numpy-1.13.0/reference/c-api.dtype.html#c.npy_bool
- `PyArray_DescrNewFromType` is at offset 96
  - 36e017194c/numpy/core/code_generators/numpy_api.py (L141)
- `array_t` constructor that takes a size and a pointer should take `ssize_t`.
  - Fixes #1599
2020-07-12 16:45:13 +02:00
Florian Apolloner
fe1392d089
Silence Clang 10 compiler warnings. Fixes #2225. (#2294) 2020-07-12 14:16:19 +02:00
Boris Staletic
8b9eb964d9
Check for NULL in raw_str on Python3 (#2290) 2020-07-11 17:20:22 +02:00