* Add make_value_iterator (#3271)
* Add make_value_iterator
This is the counterpart to make_key_iterator, and will allow
implementing a `value` method in `bind_map` (although doing so is left
for a subsequent PR).
I made a few design changes to reduce copy-and-paste boilerplate.
Previously detail::iterator_state had a boolean template parameter to
indicate whether it was being used for make_iterator or
make_key_iterator. I replaced the boolean with a class that determines
how to dereference the iterator. This allows for a generic
implementation of `__next__`.
I also added the ValueType and Extra... parameters to the iterator_state
template args, because I think it was a bug that they were missing: if
make_iterator is called twice with different values of these, only the
first set has effect (because the state class is only registered once).
There is still a potential issue in that the *values* of the extra
arguments are latched on the first call, but since most policies are
empty classes this should be even less common.
* Add some remove_cv_t to appease clang-tidy
* Make iterator_access and friends take reference
For some reason I'd accidentally made it take a const value, which
caused some issues with third-party packages.
* Another attempt to remove remove_cv_t from iterators
Some of the return types were const (non-reference) types because of the
pecularities of decltype: `decltype((*it).first)` is the *declared* type
of the member of the pair, rather than the type of the expression. So if
the reference type of the iterator is `pair<const int, int> &`, then the
decltype is `const int`. Wrapping an extra set of parentheses to form
`decltype(((*it).first))` would instead give `const int &`.
This means that the existing make_key_iterator actually returns by value
from `__next__`, rather than by reference. Since for mapping types, keys
are always const, this probably hasn't been noticed, but it will affect
make_value_iterator if the Python code tries to mutate the returned
objects. I've changed things to use double parentheses so that
make_iterator, make_key_iterator and make_value_iterator should now all
return the reference type of the iterator. I'll still need to add a test
for that; for now I'm just checking whether I can keep Clang-Tidy happy.
* Add back some NOLINTNEXTLINE to appease Clang-Tidy
This is favoured over using remove_cv_t because in some cases a const
value return type is deliberate (particularly for Eigen).
* Add a unit test for iterator referencing
Ensure that make_iterator, make_key_iterator and make_value_iterator
return references to the container elements, rather than copies. The
test for make_key_iterator fails to compile on master, which gives me
confidence that this branch has fixed it.
* Make the iterator_access etc operator() const
I'm actually a little surprised it compiled at all given that the
operator() is called on a temporary, but I don't claim to fully
understand all the different value types in C++11.
* Attempt to work around compiler bugs
https://godbolt.org/ shows an example where ICC gets the wrong result
for a decltype used as the default for a template argument, and CI also
showed problems with PGI. This is a shot in the dark to see if it fixes
things.
* Make a test constructor explicit (Clang-Tidy)
* Fix unit test on GCC 4.8.5
It seems to require the arguments to the std::pair constructor to be
implicitly convertible to the types in the pair, rather than just
requiring is_constructible.
* Remove DOXYGEN_SHOULD_SKIP_THIS guards
Now that a complex decltype expression has been replaced by a simpler
nested type, I'm hoping Doxygen will be able to build it without issues.
* Add comment to explain iterator_state template params
* fix: regression in #3271
Co-authored-by: Bruce Merry <1963944+bmerry@users.noreply.github.com>
* Adding nullptr slices
Using example from #1095
Some fixes from @wjakob's review
Stop clang-tidy from complaining
New proposal for py::slice constructor
Eric's suggested changes: simplify testing; shift def's
* chore: drop MSVC pragma (hopefully unneeded)
* Apply suggestions from code review
* Add make_value_iterator
This is the counterpart to make_key_iterator, and will allow
implementing a `value` method in `bind_map` (although doing so is left
for a subsequent PR).
I made a few design changes to reduce copy-and-paste boilerplate.
Previously detail::iterator_state had a boolean template parameter to
indicate whether it was being used for make_iterator or
make_key_iterator. I replaced the boolean with a class that determines
how to dereference the iterator. This allows for a generic
implementation of `__next__`.
I also added the ValueType and Extra... parameters to the iterator_state
template args, because I think it was a bug that they were missing: if
make_iterator is called twice with different values of these, only the
first set has effect (because the state class is only registered once).
There is still a potential issue in that the *values* of the extra
arguments are latched on the first call, but since most policies are
empty classes this should be even less common.
* Add some remove_cv_t to appease clang-tidy
* Make iterator_access and friends take reference
For some reason I'd accidentally made it take a const value, which
caused some issues with third-party packages.
* Another attempt to remove remove_cv_t from iterators
Some of the return types were const (non-reference) types because of the
pecularities of decltype: `decltype((*it).first)` is the *declared* type
of the member of the pair, rather than the type of the expression. So if
the reference type of the iterator is `pair<const int, int> &`, then the
decltype is `const int`. Wrapping an extra set of parentheses to form
`decltype(((*it).first))` would instead give `const int &`.
This means that the existing make_key_iterator actually returns by value
from `__next__`, rather than by reference. Since for mapping types, keys
are always const, this probably hasn't been noticed, but it will affect
make_value_iterator if the Python code tries to mutate the returned
objects. I've changed things to use double parentheses so that
make_iterator, make_key_iterator and make_value_iterator should now all
return the reference type of the iterator. I'll still need to add a test
for that; for now I'm just checking whether I can keep Clang-Tidy happy.
* Add back some NOLINTNEXTLINE to appease Clang-Tidy
This is favoured over using remove_cv_t because in some cases a const
value return type is deliberate (particularly for Eigen).
* Add a unit test for iterator referencing
Ensure that make_iterator, make_key_iterator and make_value_iterator
return references to the container elements, rather than copies. The
test for make_key_iterator fails to compile on master, which gives me
confidence that this branch has fixed it.
* Make the iterator_access etc operator() const
I'm actually a little surprised it compiled at all given that the
operator() is called on a temporary, but I don't claim to fully
understand all the different value types in C++11.
* Attempt to work around compiler bugs
https://godbolt.org/ shows an example where ICC gets the wrong result
for a decltype used as the default for a template argument, and CI also
showed problems with PGI. This is a shot in the dark to see if it fixes
things.
* Make a test constructor explicit (Clang-Tidy)
* Fix unit test on GCC 4.8.5
It seems to require the arguments to the std::pair constructor to be
implicitly convertible to the types in the pair, rather than just
requiring is_constructible.
* Remove DOXYGEN_SHOULD_SKIP_THIS guards
Now that a complex decltype expression has been replaced by a simpler
nested type, I'm hoping Doxygen will be able to build it without issues.
* Add comment to explain iterator_state template params
* Enable defining custom __new__
* See if xfail needed
* Qualify auto self
* Unconditionally defining PYBIND11_DISABLE_NEW_STYLE_INIT_WARNING. Returning pointer from "__init__" instead of reference.
* Use new style __init__
* Simplify __new__ creation
* Reviewer suggestions
* Match indentation
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
* Eliminate duplicate TLS keys for loader_life_support stack
This revises the existing fix for
https://github.com/pybind/pybind11/issues/2765 in
https://github.com/pybind/pybind11/pull/3237 to reduce the amount of
TLS storage used.
The shared TLS key is stored in two different ways, depending on
`PYBIND11_INTERNALS_VERSION`. If `PYBIND11_INTERNALS_VERSION ==
4` (as is currently set), the TLS key is stored in the
`internal::shared_data` map to avoid breaking ABI compatibility. If
`PYBIND11_INTERNALS_VERSION > 4`, the TLS key is stored directly in
the `internals` struct.
* Fix test_pytypes.py::test_issue2361 failure on PyPy3.7
* Add github actions tests for unstable ABI
* Restoring `const` removed from pytypes.h in PR #3254, adding tests reflective of user code that breaks when those `const` are removed.
* clang-tidy NOLINTs (and one collateral fix).
* Inserting PYBIND11_CONST_FOR_STRICT_PLATFORMS
* Trying `defined(__APPLE__)`
* Trying again: `auto it` for strict platforms.
* Adding NOLINTNEXTLINE(bugprone-macro-parentheses), expanding comments.
* Labeling all changes with `PR #3263`, for easy reference, and to make it easy to undo these changes if we decide to do so in the future.
* Fix thread safety for pybind11 loader_life_support
Fixes issue: https://github.com/pybind/pybind11/issues/2765
This converts the vector of PyObjects to either a single void* or
a per-thread void* depending on the WITH_THREAD define.
The new field is used by each thread to construct a stack
of loader_life_support frames that can extend the life of python
objects.
The pointer is updated when the loader_life_support object is allocated
(which happens before a call) as well as on release.
Each loader_life_support maintains a set of PyObject references
that need to be lifetime extended; this is done by storing them
in a c++ std::unordered_set and clearing the references when the
method completes.
* Also update the internals version as the internal struct is no longer compatible
* Add test demonstrating threading works correctly.
It may be appropriate to run this under msan/tsan/etc.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update test to use lifetime-extended references rather than
std::string_view, as that's a C++ 17 feature.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Make loader_life_support members private
* Update version to dev2
* Update test to use python threading rather than concurrent.futures
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove unnecessary env in test
* Remove unnecessary pytest in test
* Use native C++ thread_local in place of python per-thread data structures to retain compatability
* clang-format test_thread.cpp
* Add a note about debugging the py::cast() error
* thread_test.py now propagates exceptions on join() calls.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* remove unused sys / merge
* Update include order in test_thread.cpp
* Remove spurious whitespace
* Update comment / whitespace.
* Address review comments
* lint cleanup
* Fix test IntStruct constructor.
* Add explicit to constructor
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
* Set __file__ constant when using eval_file
* Use const ref
* Use a move instead
* Revert
* Improve test
* Guard test with Python version
* Fix tests
* Dont support Python2 API
* Drop Python2 eval __file__ support
* Hack
* Semisupport Python2
* Take2
* Remove Python2 support
* Minor tweaks.
* Restoring tests/pybind11_tests.h version from master, removing just the comment and empty line that was added in PR #3087; those were made obsolete by the pragma cleanup that concluded with PR #3186.
* More-to-the-point test for Python 3.
* Minor tweaks.
* Restoring tests/pybind11_tests.h version from master, removing just the comment and empty line that was added in PR #3087; those were made obsolete by the pragma cleanup that concluded with PR #3186.
* [ci skip] Restoring tests/test_enum.py from master.
* fix: memory leak in cpp_function (#3228)
* add a test case to check objects are deconstructed in cpp_function
* update the test case about cpp_function
* fix the test case about cpp_function: remove "noexcept"
* Actually calling func. CHECK(stat.alive() == 2); Manually verified that the new tests fails without the change in pybind11.h
* Moving new test to test_callbacks.cpp,py, with small enhancements.
* Removing new test from test_interpreter.cpp (after it was moved to test_callbacks.cpp,py). This restores test_interpreter.cpp to the current state on master.
* Using py::detail::silence_unused_warnings(py_func); to make the intent clear.
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
* Trivial change to avoid (ssize_t) cast.
* Demo for safe_ssize_t idea.
* Removing safe_ssize_t.cpp (proof-of-concept code) to not upset the GHA Format workflow.
* Completing changes in pytypes.h
* New ssize_t_cast (better replacement for safe_ssize_t).
* clang-format-diff (no manual changes).
* bytes_ssize_t -Wnarrowing reproducer (see PR #2692).
* Backing out tuple(), list() ssize_t support, for compatibility with older compilers (to resolve link failures).
* Bug fix: missing `py::` for `py::ssize_t`
* Restoring tuple(), list() ssize_t support, but passing `size` by value, for compatibility with older compilers (to resolve link failures).
* Full test coverage of all functions with modified signatures.
* Use equivalent_integer for enum's Scalar decision
* Add test for char underlying enum
* Support translating bool type in enum's Scalar
* Add test for bool underlying enum
* Fix comment in test
* Switch from `PYBIND11_CPP20` macro to `PYBIND11_HAS_U8STRING`
* Refine tests
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
* Call PySys_SetArgv when initializing interpreter.
* Document argc/argv parameters in initialize_interpreter.
* Remove manual memory management from set_interpreter_argv in favor of smart pointers.
* Use size_t for indexers in set_interpreter_argv.
* Minimize macros for flow control in set_interpreter_argv.
* Fix 'unused variable' warning on Py2
* whitespace
* Define wide_char_arg_deleter outside set_interpreter_argv.
* Do sys.path workaround in C++ rather than eval.
* Factor out wchar conversion to a separate function.
* Restore widened_argv variable declaration.
* Fix undeclared widened_arg variable on some paths.
* Use delete[] to match new wchar_t[].
* Fix compiler errors
* Use PY_VERSION_HEX for a cleaner CVE-2008-5983 mode check.
* Fix typo
* Use explicit type for deleter so delete[] works cross-compiler.
* Always use PySys_SetArgvEx because pybind11 doesn't support pythons that don't include it.
* Remove pointless ternary operator.
* Use unique_ptr.reset instead of a second initialization.
* Rename add_program_dir_to_path parameter to clarify intent.
* Add defined() check before evaluating HAVE_BROKEN_MBSTOWCS.
* Apply clang-tidy fixes
* Pre-commit
* refactor: use const for set_interpreter_argv
* Try to fix const issue and allocate vector properly
* fix: copy strings on Python 2
* Applying clang-format-diff relative to master.
The only manual change is an added empty line between pybind11 and system `#include`s.
```
git diff -U0 --no-color master | python3 $HOME/clone/llvm-project/clang/tools/clang-format/clang-format-diff.py -p1 -style=file -i
```
Co-authored-by: Boris Staletic <boris.staletic@gmail.com>
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
* Add py::raise_from to enable chaining exceptions on Python 3.3+
* Use 'raise from' in initialization
* Documenting the exact base version of _PyErr_FormatVFromCause, adding back `assert`s.
Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com>
* extend msys2 CI
- add 32-bit job
- add c++11/17 c++/interface tests copied from standard ci
- add numpy/scipy
* account for padding of PartialStruct in numpy dtypes test with mingw32
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* msys2 ci: add c++14 tests
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Removing all MSVC C4127 warning suppression pragmas.
* Removing MSVC /WX (WERROR). To get a full list of all warnings.
* Inserting PYBIND11_SILENCE_MSVC_C4127. Changing one runtime if to #if.
* Changing PYBIND11_SILENCE_MSVC_C4127 macro to use absolute namespace (for use outside pybind11 include directory).
* Restoring MSVC /WX (WERROR).
* Removing globally-scoped suppression for clang -Wunsequenced. Based on an experiment under PR #3202 it is obsolete and can simply be removed.
The only manual intervention was to remove the merge conflict markers in pybind11.h.
TODO in separate commit: complete removal of the obsolete detail/pragma_warning_block.h.
* Apply isort
* Tweak isort config
* Add env.py as a known_first_party
* Add one missing known first party
* Make config compat with older isort versions
* Add another comment
* Revert pyproject setting
* test_eval: Show example of working closure
* Extend test_eval_closure with weirder examples of closures for py::eval
Co-authored-by: Yannick Jadoul <yannick.jadoul@belgacom.net>
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
* mingw64 platform string is like mingw_xxx not "mingw"
See https://github.com/msys2/MINGW-packages/blob/master/mingw-w64-python/0099-Change-the-get_platform-method-in-sysconfig-and-dist.patch
* Mingw: Do not dllexport exceptions
This is a fix for errors like:
D:/a/pybind11/pybind11/include/pybind11/detail/common.h:735:23: error: 'dllexport' implies default visibility, but 'class pybind11::builtin_exception' has already been declared with a different visibility
735 | class PYBIND11_EXPORT builtin_exception : public std::runtime_error {
| ^~~~~~~~~~~~~~~~~
* GHA: Test Mingw64 build
* fix: avoid thin binaries on mingw
* fix: drop lto on MinGW
* Mingw64: disable PYBIND11_DEPRECATED
It trigger many warnings for unknown reasons
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
* Fixing `pragma warning pop` for `__INTEL_COMPILER`.
* Adding push/pop to 3 tests. Removing #878 from top of pybind11.h (it was/is only needed for 1 test).
* Trying again after CI failure, moving the push to the top of 2 tests.
* Trying more after CI failure, adding push/pop to pybind11_tests.h, constructor_stats.h.
* Moving ICC #2196 suppression to CMakeLists.txt
* Fixing condition for `pragma GCC diagnostic push` in pybind11.h
* Moving `pragma warning disable 2196` to common.h
* Revising #ifdef to be more conservative.
* Undoing insertion of notes that will hopefully soon be completely obsolete anyway.
* maint(clang-tidy) Improve code readability
* Fix minor typos
* Revert optimization that removed test case
* Fix comment formatting
* Revert another optimization to repro an issue
* Remove make_unique since it C++14 and newer only
* eformat comments
* Fix unsignedness of comparison
* Update comment
* Allow python builtins to be used as callbacks
* Try to fix pypy segfault
* Add expected fail for PyPy
* Fix typo
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add more info to xfail
* Add env
* Try returning false
* Try removing the move for pypy
* Fix bugs
* Try removing move
* Just keep ignoring for PyPy
* Add back xfail
* Fix ctors
* Revert change of std::move
* Change to skip
* Fix bug and edit comments
* Remove clang-tidy bugprone fix skip bug
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Create a module_internals struct
Since we now have two things that are going to be module local, it felt
correct to add a struct to manage them.
* Add local exception translators
These are added via the register_local_exception_translator function
and are then applied before the global translators
* Add unit tests to show the local exception translator works
* Fix a bug in the unit test with the string value of KeyError
* Fix a formatting issue
* Rename registered_local_types_cpp()
Rename it to get_registered_local_types_cpp() to disambiguate from the
new member of module_internals
* Add additional comments to new local exception code path
* Add a register_local_exception function
* Add additional unit tests for register_local_exception
* Use get_local_internals like get_internals
* Update documentation for new local exception feature
* Add back a missing space
* Clean-up some issues in the docs
* Remove the code duplication when translating exceptions
Separated out the exception processing into a standalone function in the
details namespace.
Clean-up some comments as per PR notes as well
* Remove the code duplication in register_exception
* Cleanup some formatting things caught by clang-format
* Remove the templates from exception translators
But I added a using declaration to alias the type.
* Remove the extra local from local_internals variable names
* Add an extra explanatory comment to local_internals
* Fix a typo in the code
* Raise codec errors when casting to std::string
Allow the codec's exception to be raised instead of RuntimeError when
casting from py::str to std::string.
PY2 allows ucs surrogates in UTF-8 conversion
Signed-off-by: Shane Loretz <sloretz@openrobotics.org>
Signed-off-by: Shane Loretz <sloretz@osrfoundation.org>
* Attempt to fix py2 error
* Revert all unicode literals
* Fixed
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
* Manually moving `// NOLINT` comments so that clang-format does not move them to the wrong places.
* Manually reformatting comments related to `static_assert`s so that clang-format does not need two passes.
* Empty lines between #includes, to prevent clang-format from shuffling the order and thereby confusing MSVC 2015.
* git diff -U0 --no-color HEAD^ | python3 $HOME/clone/llvm-project/clang/tools/clang-format/clang-format-diff.py -p1 -style=file -i
For single-file extensions, a convenient pattern offered by cython
is to place the source files directly in the python source tree
(`foo/__init__.py`, `foo/ext.pyx`), deriving the package names from
their filesystem location. Adapt this pattern for pybind11, using an
`intree_extensions` helper, which should be thought of as the moral
equivalent to `cythonize`.
Differences with cythonize: I chose not to include globbing support
(`intree_extensions(glob.glob("**/*.cpp"))` seems sufficient), nor to
provide extension-customization kwargs (directly setting the attributes
on the resulting Pybind11Extension objects seems sufficient).
We could choose to have `intree_extension` (singular instead) and make
users write `[*map(intree_extension, glob.glob("**/*.cpp"))]`; no strong
opinion here.
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
* Adding test_return_vector_bool_raw_ptr to test_stl.py.
* First attempt to make the documentation more accurate, but not trying to be comprehensive, to not bloat the reference table with too many details.
* Fixing minor oversights.
* Applying reviewer suggestion.
* Adding iostream.h thread-safety documentation.
* Restoring `TestThread` code with added `std::lock_guard<std::mutex>`.
* Updating new comments to reflect new information.
* Fixing up `git rebase -X theirs` accidents.
* * Rollback of PR #3068.
* Using latest pytest main branch for 3.9 and 3.10.
* WORKAROUND_ENABLING_ROLLBACK_OF_PR3068 in test_class_sh_trampoline_shared_from_this.py
First experiment combining two potential fixes: latest pytest, workaround.
If this succeeds the next step will be to try only latest pytest without the workaround.
Note: the workaround is known to resolve the MSAN error reported under
https://github.com/pybind/pybind11/pull/3068#issuecomment-877658470
* WORKAROUND_ENABLING_ROLLBACK_OF_PR3068 = False
* Narrowing down WORKAROUND_ENABLING_ROLLBACK_OF_PR3068 to Python 3.9
* Splitting out pybind11/stl/filesystem.h.
To solve breakages like: https://github.com/deepmind/open_spiel/runs/2999582108
Mostly following the suggestion here: https://github.com/pybind/pybind11/pull/2730#issuecomment-750507575
Except using pybind11/stl/filesystem.h instead of pybind11/stlfs.h, as decided via chat.
stl.h restored to the exact state before merging PR #2730 via:
```
git checkout 733f8de24f stl.h
```
* Properly including new stl subdirectory in pip wheel config.
This now passes interactively:
```
pytest tests/extra_python_package/
```
* iwyu cleanup.
iwyuh.py -c -std=c++17 -DPYBIND11_TEST_BOOST -Ipybind11/include -I/usr/include/python3.9 -I/usr/include/eigen3 include/pybind11/stl/filesystem.h
* Adding PYBIND11_HAS_FILESYSTEM_IS_OPTIONAL.
* Eliminating else after return.
* Unify cast_error message thrown by [simple|unpacking]_collector
simple_collector and unpacking_collector throw different error messages
when the casting of an argument failed: While the former mentions make_tuple(),
the latter emphasises the call argument (and its name/position).
* Consolidating "Unable to convert call argument" error reporting code to guarantee uniformity.
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
* pickle setstate: setattr __dict__ only if not empty, to not force use of py::dynamic_attr() unnecessarily.
* Adding unit test.
* Clang 3.6 & 3.7 compatibility.
* PyPy compatibility.
* Minor iwyu fix, additional comment.
* Addressing reviewer requests.
* Applying clang-tidy suggested fixes.
* Adding check_dynamic_cast_SimpleCppDerived, related to issue #3062.
Now also passes the open_spiel iterated_prisoners_dilemma_test ASAN clean, in addition to all pybind11 and PyCLIF unit tests.
The problem was that calling `std::shared_ptr<void>::reset()` with a `void` pointer cannot possibly update the `shared_from_this` `weak_ptr`.
The solution is to store a `shd_ptr_reset` function pointer in `guarded_deleter` (similar in idea to the stored function pointer for calling `delete`).
This commit still includes all debugging code, i.e. is "dirty". The code will be cleaned up after the GitHub CI is fully successful.
Example:
```
test_class_sh_unique_ptr_member.cpp:17:5: error: deleted member function should be public [modernize-use-equals-delete,-warnings-as-errors]
pointee(const pointee &) = delete;
^
```
* Add const T to docstring generation.
* Change order.
* See if existing test triggers for a const type.
* Add tests.
* Fix test.
* Remove experiment.
* Reformat.
* More tests, checks run.
* Adding `test_fmt_desc_` prefix to new test functions.
* Using pytest.mark.parametrize to 1. condense test; 2. exercise all functions even if one fails; 3. be less platform-specific (e.g. C++ float is not necessarily float32).
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
* Crash when printing Unicode to redirected cout
Add failing tests
* Fix Unicode crashes redirected cout
* pythonbuf::utf8_remainder check end iterator
* Remove trailing whitespace and formatting iostream
* Avoid buffer overflow if ostream redirect races
This doesn't solve the actual race, but at least it now has a much lower
probability of reading past the end of the buffer even when data races
do occur.
* Bug fix: adding back `!is_alias<Class>(ptr)` that were accidentally omitted.
* Introducing PYBIND11_SH_AVL, PYBIND11_SH_DEF macros. Applying PYBIND11_SH_DEF to test_factory_constructors.py to complete test coverage.
* Using PYBIND11_SH_DEF in test_methods_and_attributes.cpp, for more complete test coverage.
* Using PYBIND11_SH_DEF in test_multiple_inheritance.cpp, for more complete test coverage.
* Cleaning up test_classh_mock.cpp.
* Better explanations for PYBIND11_SH_AVL, PYBIND11_SH_DEF.
* Disabling 3.10-dev builds.
* Using new smart_holder::reclaim_disowned in smart_holder_type_caster for unique_ptr.
* Systematically renaming was_disowned to is_disowned (because disowning is now reversible: reclaim_disowned).
* Systematically renaming virtual_overrider_self_life_support to trampoline_self_life_support (to reuse existing terminology instead of introducing new one).
* Systematically renaming test_class_sh_with_alias to test_class_sh_trampoline_basic.
* Adding a Trampolines and std::unique_ptr section to README_smart_holder.rst.
* MSVC compatibility.
* Porting subset of absltest code from reproducer provided by @elkhrt. Baseline for debugging ASAN heap-use-after-free.
* Moving Py_DECREF to resolve ASAN heap-use-after-free failure.
* Fixing trivial formatting issue.
* Workaround for clang 3.6 and 3.7.
It was only useful for easily harvest this from the GitHub Actions CI results, mostly out of curiosity:
52 C++ function argument 1 is evaluated first.
90 C++ function argument 2 is evaluated first.
These results came for the final CI run for PR #2936; that PR has nothing else to do with the results.
Pushing directly. This tiny change is not worth a PR.
[skip actions]
* Adding PyGILState_Check() in object_api<>::operator().
* Enabling PyGILState_Check() for Python >= 3.6 only.
Possibly, this explains why PyGILState_Check() cannot safely be used with Python 3.4 and 3.5:
https://github.com/python/cpython/pull/10267#issuecomment-434881587
* Adding simple micro benchmark.
* Reducing test time to minimum (purely for coverage, not for accurate results).
* Fixing silly oversight.
* Minor code organization improvement in test.
* Adding example runtimes.
* Removing capsys (just run with `-k test_callback_num_times -s` and using `.format()`.
* Adding test_class_sh_disowning.
* Fixing minor namespace naming inconsistency between test_class_sh_*.cpp files.
* Replacing py::overload_cast with plain cast for C++11 compatibility.
* Accommodate that the C++ order of evaluation of function arguments is unspecified.
* Adaption of PyCLIF virtual_py_cpp_mix test.
* Removing ValueError: Ownership of instance with virtual overrides in Python cannot be transferred to C++. TODO: static_assert alias class needs to inherit from virtual_overrider_self_life_support.
* Bringing back ValueError: "... instance cannot safely be transferred to C++.", but based on dynamic_cast<AliasType>.
* Fixing oversight: adding test_class_sh_virtual_py_cpp_mix.cpp to cmake file.
* clang <= 3.6 compatibility.
* Fixing oversight: dynamic_raw_ptr_cast_if_possible needs special handling for To = void. Adding corresponding missing test in test_class_sh_virtual_py_cpp_mix. Moving dynamic_raw_ptr_cast_if_possible to separate header.
* Changing py::detail::virtual_overrider_self_life_support to py::virtual_overrider_self_life_support.
* Initial version of virtual_overrider_self_life_support (enables safely passing unique_ptr to C++).
* Clang 3.6, 3.7 compatibility.
* Adding missing default constructor.
* Restoring test for exception for the case that virtual_overrider_self_life_support is not used.
* Fixing oversight: Adding missing holder().ensure_was_not_disowned().
* Adding unit tests for new `struct smart_holder` member functions.
* Moving virtual_overrider_self_life_support to separate include file, with iwyu cleanup.
* Use correct duration representation when casting from datetime.timdelta to std::chrono::duration
* When asserting datetime/timedelta/date/time we can equality-compare whole objects
* shared_ptr<bool> vptr_deleter_armed_flag_ptr (instead of unique_ptr), to fix heap-use-after-free bug.
* Fixing generated by some compilers in the pybind11 CI suite.
* Adding test_unique_ptr_member (for desired PyCLIF behavior).
See also: https://github.com/pybind/pybind11/issues/2583
Does not build with upstream master or
https://github.com/pybind/pybind11/pull/2047, but builds with
https://github.com/RobotLocomotion/pybind11 and almost runs:
```
Running tests in directory "/usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests":
================================================================================= test session starts =================================================================================
platform linux -- Python 3.8.5, pytest-5.4.3, py-1.9.0, pluggy-0.13.1
rootdir: /usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests, inifile: pytest.ini
collected 2 items
test_unique_ptr_member.py .F [100%]
====================================================================================== FAILURES =======================================================================================
_____________________________________________________________________________ test_pointee_and_ptr_owner ______________________________________________________________________________
def test_pointee_and_ptr_owner():
obj = m.pointee()
assert obj.get_int() == 213
m.ptr_owner(obj)
with pytest.raises(ValueError) as exc_info:
> obj.get_int()
E Failed: DID NOT RAISE <class 'ValueError'>
test_unique_ptr_member.py:17: Failed
============================================================================= 1 failed, 1 passed in 0.06s =============================================================================
```
* unique_ptr or shared_ptr return
* new test_variant_unique_shared with vptr_holder prototype
* moving prototype code to pybind11/vptr_holder.h, adding type_caster specialization to make the bindings involving unique_ptr passing compile, but load and cast implementations are missing
* disabling GitHub Actions on pull_request (for this PR)
* disabling AppVeyor (for this PR)
* TRIGGER_SEGSEV macro, annotations for GET_STACK (vptr::get), GET_INT_STACK (pointee)
* adding test_promotion_of_disowned_to_shared
* Copying tests as-is from xxx_value_ptr_xxx_holder branch.
https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder
Systematically exercising returning and passing unique_ptr<T>, shared_ptr<T>
with unique_ptr, shared_ptr holder.
Observations:
test_holder_unique_ptr:
make_unique_pointee OK
pass_unique_pointee BUILD_FAIL (as documented)
make_shared_pointee Abort free(): double free detected
pass_shared_pointee RuntimeError: Unable to load a custom holder type from a default-holder instance
test_holder_shared_ptr:
make_unique_pointee Segmentation fault (#1138)
pass_unique_pointee BUILD_FAIL (as documented)
make_shared_pointee OK
pass_shared_pointee OK
* Copying tests as-is from xxx_value_ptr_xxx_holder branch.
https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder
Systematically exercising casting between shared_ptr<base>, shared_ptr<derived>.
* Demonstration of Undefined Behavior in handling of shared_ptr holder.
Based on https://godbolt.org/z/4fdjaW by jorgbrown@ (thanks Jorg!).
* Additional demonstration of Undefined Behavior in handling of shared_ptr holder.
* fixing up-down mixup in comment
* Demonstration of Undefined Behavior in handling of polymorphic pointers.
(This demo does NOT involve smart pointers at all, unlike the otherwise similar test_smart_ptr_private_first_base.)
* minor test_private_first_base.cpp simplification (after discovering that this can be wrapped with Boost.Python, using boost::noncopyable)
* pybind11 equivalent of Boost.Python test similar to reproducer under #1333
* Snapshot of WIP, TODO: shared_ptr deleter with on/off switch
* Adding vptr_deleter.
* Adding from/as unique_ptr<T> and unique_ptr<T, D>.
* Adding from_shared_ptr. Some polishing.
* New tests/core/smart_holder_poc_test.cpp, using Catch2.
* Adding in vptr_deleter_guard_flag.
* Improved labeling of TEST_CASEs.
* Shuffling existing TEST_CASEs into systematic matrix.
* Implementing all [S]uccess tests.
* Implementing all [E]xception tests.
* Testing of exceptions not covered by the from-as matrix.
* Adding top-level comment.
* Converting from methods to factory functions (no functional change).
* Removing obsolete and very incomplete test (replaced by Catch2-based test).
* Removing stray file.
* Adding type_caster_bare_interface_demo.
* Adding shared_ptr<mpty>, shared_ptr<mpty const> casters.
* Adding unique_ptr<mpty>, unique_ptr<mpty const> casters.
* Pure copy of `class class_` implementation in pybind11.h (master commit 98f1bbb800).
* classh.h: renaming of class_ to classh + namespace; forking test_classh_wip from test_type_caster_bare_interface_demo.
* Hard-coding smart_holder into classh.
* Adding mpty::mtxt string member.
* Adding isinstance<mpty> in type_caster::load functions.
* Adding rvalue_ref, renaming const_value_ref to lvalue_ref & removing const.
* Retrieving smart_holder pointer in type_caster<mpty>::load, and using it cast_op operators.
* Factoring out smart_holder_type_caster_load.
* Retrieving smart_holder pointer in type_caster<std::shared_ptr<mpty[ const]>>::load, and using it cast_op operators.
* Improved error messaging: Cannot disown nullptr (as_unique_ptr).
* Retrieving smart_holder pointer in type_caster<std::unique_ptr<mpty[ const]>>::load, and using it cast_op operators.
* Pure `clang-format --style=file -i` change.
* Pure `clang-format --style=file -i` change, with two `clang-format off` directives.
* Fixing oversight (discovered by flake8).
* flake8 cleanup
* Systematically setting mtxt for all rtrn_mpty_* functions (preparation, the values are not actually used yet).
* static cast handle for rtrn_cptr works by simply dropping in code from type_caster_base (marked with comments).
* static cast handle for rtrn_cref works by simply dropping in code from type_caster_base (marked with comments). rtrn_mref and rtrn_mptr work via const_cast (to add const).
* static cast handle for rtrn_valu works by simply dropping in code from type_caster_base (marked with comments). rtrn_rref raises a RuntimeError, to be investigated.
* Copying type_caster_generic::cast into type_caster<mpty> as-is (preparation for handling smart pointers).
* Pure clang-format change (applied to original type_caster_generic::cast).
* Adding comment re potential use_count data race.
* static handle cast implementations for rtrn_shmp, rtrn_shcp.
* Adding MISSING comments in operator std::unique_ptr<mpty[ const]>.
* static handle cast implementations for rtrn_uqmp, rtrn_uqcp.
* Bug fix: vptr_deleter_armed_flag_ptr has to live on the heap.
See new bullet point in comment section near the top.
The variable was also renamed to reflect its function more accurately.
* Fixing bugs discovered by ASAN. The code is now ASAN, MSAN, UBSAN clean.
* Making test_type_caster_bare_interface_demo.cpp slightly more realistic, ASAN, MSAN, UBSAN clean.
* Calling deregister_instance after disowning via unique_ptr.
* Removing enable_shared_from_this stub, simplifying existing code, clang-format.
Open question, with respect to the original code:
76a160070b/include/pybind11/pybind11.h (L1510)
To me it looks like the exact situation marked as `std::shared_ptr<Good> gp1 = not_so_good.getptr();` here: https://en.cppreference.com/w/cpp/memory/enable_shared_from_this
The comment there is: `// undefined behavior (until C++17) and std::bad_weak_ptr thrown (since C++17)`
Does the existing code have UB pre C++17?
I'll leave handling of enable_shared_from_this for later, as the need arises.
* Cosmetical change around helper functions.
* Using type_caster_base<mpty>::src_and_type directly, removing copy. Also renaming one cast to cast_const_raw_ptr, for clarity.
* Fixing clang-format oversight.
* Using factored-out make_constructor (PR #2798), removing duplicate code.
* Inserting additional assert to ensure a returned unique_ptr is always a new Python instance.
* Adding minor comment (change to internals needed to distinguish uninitialized/disowned in error message).
* Factoring out find_existing_python_instance().
* Moving factored-out make_constructor to test_classh_wip.cpp, restoring previous version of cast.h. This is currently the most practical approach. See PR #2798 for background.
* Copying classh type_casters from test_classh_wip.cpp UNMODIFIED, as a baseline for generalizing the code.
* Using pybind11/detail/classh_type_casters.h from test_classh_wip.cpp.
* Adding & using PYBIND11_CLASSH_TYPE_CASTERS define.
* Adding test_classh_inheritance, currently failing (passes with class_).
* Removing .clang-format before git rebase master (where the file was added).
* Bringing back .clang-format, the previous rm was a bad idea.
* Folding in modified_type_caster_generic_load_impl, just enough to pass test_class_wip. test_classh_inheritance is still failing, but with a different error: [RuntimeError: Incompatible type (as_raw_ptr_unowned).]
* Minimal changes needed to pass test_classh_inheritance.
* First pass adjusting try_implicit_casts and try_load_foreign_module_local to capture loaded_v_h, but untested and guarded with pybind11_failure("Untested"). This was done mainly to determine general feasibility. Note the TODO in pybind11.h, where type_caster_generic::local_load is currently hard-coded. test_classh_wip and test_classh_inheritance still pass, as before.
* Decoupling generic_type from type_caster_generic.
* Changes and tests covering classh_type_casters try_implicit_casts.
* Minimal test covering classh_type_casters load_impl Case 2b.
* Removing stray isinstance<T>(src): it interferes with the py::module_local feature. Adding missing #includes.
* Tests for classh py::module_local() feature.
* Pure renaming of function names in test_classh_inheritance, similar to the systematic approach used in test_class_wip. NO functional changes.
* Pure renaming of function and variable names, for better generalization when convoluting with inheritance. NO functional changes.
* Adopting systematic naming scheme from test_classh_wip. NO functional changes.
* Moving const after type name, for functions that cover a systematic scheme. NO functional changes.
* Adding smart_holder_type_caster_load::loaded_as_shared_ptr, currently bypassing smart_holder shared_ptr tracking completely, but the tests pass and are sanitizer clean.
* Removing rtti_held from smart_holder. See updated comment.
* Cleaning up loaded_as_raw_ptr_unowned, loaded_as_shared_ptr.
* Factoring out convert_type and folding into loaded_as_unique_ptr.
* Folding convert_type into lvalue_ref and rvalue_ref paths. Some smart_holder_type_caster_load cleanup.
* Using unique_ptr in local_load to replace static variable. Also adding local_load_safety_guard.
* Converting test_unique_ptr_member to using classh: fully working, ASAN, MSAN, UBSAN clean.
* Removing debugging comments (GET_STACK, GET_INT_STACK). cast.h is identical to current master again, pybind11.h only has the generic_type::initialize(..., &type_caster_generic::local_load) change.
* Purging obsolete pybind11/vptr_holder.h and associated test.
* Moving several tests to github.com/rwgk/rwgk_tbx/tree/main/pybind11_tests
a2c2f88174
These tests are from experimenting, and for demonstrating UB in pybind11 multiple inheritance handling ("first_base"), to be fixed later.
* Adding py::smart_holder support to py::class_, purging py::classh completely.
* Renaming files in include directory, creating pybind11/smart_holder.h.
* Renaming all "classh" to "smart_holder" in pybind11/detail/smart_holder_type_casters.h.
The user-facing macro is now PYBIND11_SMART_HOLDER_TYPE_CASTERS.
* Systematically renaming tests to use "class_sh" in the name.
* Renaming test_type_caster_bare_interface_demo to test_type_caster_bare_interface.
* Renaming new tests/core subdirectory to tests/pure_cpp.
* Adding new tests to CMake config, resetting CI config.
* Changing CMake file so that test_class_sh_module_local.py actually runs.
* clang-tidy fixes.
* 32-bit compatibility.
* Reusing type_caster_base make_copy_constructor, make_move_constructor with a trick.
* CMake COMPARE NATURAL is not available with older versions.
* Adding copyright notices to new header files.
* Explicitly define copy/move constructors/assignments.
* Adding new header files to tests/extra_python_package/test_files.py.
* Adding tests/pure_cpp/CMakeLists.txt.
* Making use of the new find_existing_python_instance() function factored out with PR #2822.
* Moving define PYBIND11_SMART_HOLDER_TYPE_CASTERS(T) down in the file. NO functional changes. Preparation for follow-up work (to keep that diff smaller).
* Reintroducing py::classh, this time as a simple alias for py::class_<U, py::smart_holder>.
* Replacing detail::is_smart_holder<H> in cast.h with detail::is_smart_holder_type_caster<T>.
Moving get_local_load_function_ptr, init_instance_for_type to smart_holder_type_caster_class_hooks.
Expanding static_assert in py::type::handle_of<> to accommodate smart_holder_type_casters.
* Fixing oversight.
* Adding classu alias for class_<U, std::unique_ptr<U>>.
* Giving up on idea to use legacy init_instance only if is_base_of<type_caster_generic, type_caster<T>. There are use cases in the wild that define both a custom type_caster and class_.
* Removing test_type_caster_bare_interface, which was moved to the separate PR #2834.
* Moving up is_smart_holder_type_caster, to also use in cast_is_temporary_value_reference.
* Adding smart_holder_type_casters for unique_ptr with custom deleter. SEVERE CODE DUPLICATION. This commit is to establish a baseline for consolidating the unique_ptr code.
* Unification of unique_ptr, unique_ptr_with_deleter code in smart_holder_poc.h. Leads to more fitting error messages. Enables use of unique_ptr<T, D> smart_holder_type_casters also for unique_ptr<T>.
* Copying files as-is from branch test_unique_ptr_member (PR #2672).
* Adding comment, simplifying naming, cmake addition.
* Introducing PYBIND11_USE_SMART_HOLDER_AS_DEFAULT macro (tested only undefined; there are many errors with the macro defined).
* Removing test_type_caster_bare_interface, which was moved to the separate PR #2834.
* Fixing oversight introduced with commit 95425f13d6.
* Setting record.default_holder correctly for PYBIND11_USE_SMART_HOLDER_AS_DEFAULT.
With this test_class.cpp builds and even mostly runs, except
`test_multiple_instances_with_same_pointer`, which segfaults because it is
using a `unique_ptr` holder but `smart_holder` `type_caster`.
Also adding `static_assert`s to generate build errors for such situations,
but guarding with `#if 0` to first pivot to test_factory_constructors.cpp.
* Fixing up cast.h and smart_holder.h after rebase.
* Removing detail/smart_holder_type_casters.h in separate commit.
* Commenting out const in def_buffer(... const). With this, test_buffers builds and runs with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. Explanation why the const needs to be removed, or fix elsewhere, is still needed, but left for later.
* Adding test_class_sh_factory_constructors, reproducing test_factory_constructors failure. Using py::class_ in this commit, to be changed to py::classh for debugging.
* Removing include/pybind11/detail/smart_holder_type_casters.h from CMakeLists.txt, test_files.py (since it does not exist in this branch).
* Adding // DANGER ZONE reminders.
* Converting as many py::class_ to py::classh as possible, not breaking tests.
* Adding initimpl::construct() overloads, resulting in test_class_sh_factory_constructors feature parity for py::class_ and py::classh.
* Adding enable_if !is_smart_holder_type_caster to existing initimpl::construct(). With this test_factory_constructors.cpp builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT.
* Disabling shared_ptr&, shared_ptr* tests when building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT for now, pending work on smart_holder_type_caster<shared_ptr>.
* Factoring out struct and class definitions into anonymous namespace. Preparation for building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT.
* Simplifying from_unique_ptr(): typename D = std::default_delete<T> is not needed. Factoring out is_std_default_delete<T>() for consistentcy between ensure_compatible_rtti_uqp_del() and from_unique_ptr().
* Introducing PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. Using it in test_smart_ptr.cpp. With this test_smart_ptr builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT and all but one test run successfully.
* Introducing 1. type_caster_for_class_, used in PYBIND11_MAKE_OPAQUE, and 2. default_holder_type, used in stl_bind.h.
* Using __VA_ARGS__ in PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS.
* Replacing condense_for_macro with much simpler approach.
* Softening static_assert, to only check specifically that smart_holder is not mixed with type_caster_base, and unique_ptr/shared_ptr holders are not mixed with smart_holder_type_casters.
* Adding PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS in test_class.cpp (with this all but one test succeed with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT).
* Adding remaining PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. static_assert for "necessary conditions" for both types of default holder, static_assert for "strict conditions" guarded by new PYBIND11_STRICT_ASSERTS_CLASS_HOLDER_VS_TYPE_CASTER_MIX. All tests build & run as before with unique_ptr as the default holder, all tests build for smart_holder as the default holder, even with the strict static_assert.
* Introducing check_is_smart_holder_type_caster() function for runtime check, and reinterpreting record.default_holder as "uses_unique_ptr_holder". With this test_smart_ptr succeeds. (All 42 tests build, 35 tests succeed, 5 run but have some failures, 2 segfault.)
* Bug fix: Adding have_value() to smart_holder_type_caster_load. With this test_builtin_casters succeeds. (All 42 tests build, 36 tests succeed, 5 run but have some failures, 1 segfault.)
* Adding unowned_void_ptr_from_direct_conversion to modified_type_caster_generic_load_impl. This fixes the last remaining segfault (test_numpy_dtypes). New stats for all tests combined: 12 failed, 458 passed.
* Adding "Lazy allocation for unallocated values" (for old-style __init__) into load_value_and_holder. Deferring destruction of disowned holder until clear_instance, to remain inspectable for "uninitialized" or "disowned" detection. New stats for all tests combined: 5 failed, 465 passed.
* Changing std::shared_ptr pointer/reference to const pointer/reference. New stats for all tests combined: 4 failed, 466 passed.
* Adding return_value_policy::move to permissible policies for unique_ptr returns. New stats for all tests combined: 3 failed, 467 passed.
* Overlooked flake8 fixes.
* Manipulating failing ConstructorStats test to pass, to be able to run all tests with ASAN.
This version of the code is ASAN clean with unique_ptr or smart_holder as the default.
This change needs to be reverted after adopting the existing move-only-if-refcount-is-1
logic used by type_caster_base.
* Adding copy constructor and move constructor tracking to atyp. Preparation for a follow-up change in smart_holder_type_caster, to make this test sensitive to the changing behavior.
[skip ci]
* Removing `operator T&&() &&` from smart_holder_type_caster, for compatibility with the behavior of type_caster_base. Enables reverting 2 of 3 test manipulations applied under commit 249df7cbdb. The manipulation in test_factory_constructors.py is NOT reverted in this commit.
[skip ci]
* Fixing unfortunate editing mishap. This reverts the last remaining test manipulation in commit 249df7cbdb and makes all existing unit tests pass with smart_holder as default holder.
* GitHub CI clang-tidy fixes.
* Adding messages to terse `static_assert`s, for pre-C++17 compatibility.
* Using @pytest.mark.parametrize to run each assert separately (to see all errors, not just the first).
* Systematically removing _atyp from function names, to make the test code simpler.
* Using re.match to accommodate variable number of intermediate MvCtor.
* Also removing `operator T()` from smart_holder_type_caster, to fix gcc compilation errors. The only loss is pass_rref in test_class_sh_basic.
* Systematically replacing `detail::enable_if_t<...smart_holder...>` with `typename std::enable_if<...smart_holder...>::type`. Attempt to work around MSVC 2015 issues, to be tested via GitHub CI. The idea for this change originates from this comment: https://github.com/pybind/pybind11/issues/1616#issuecomment-444536813
* Importing re before pytest after observing a PyPy CI flake when importing pytest first.
* Copying MSVC 2015 compatibility change from branch pr2672_use_smart_holder_as_default.
* Introducing is_smart_holder_type_caster_base_tag, to keep smart_holder code more disconnected.
* Working around MSVC 2015 bug.
* Expanding comment for MSVC 2015 workaround.
* Systematically changing std::enable_if back to detail::enable_if_t, effectively reverting commit 5d4b6890a3.
* Removing unused smart_holder_type_caster_load::loaded_as_rvalue_ref (it was an oversight that it was not removed with commit 23036a45eb).
* Removing py::classu, because it does not seem useful enough.
* Reverting commit 6349531306 by un-commenting `const` in `def_buffer(...)`. To make this possible, `operator T const&` and `operator T const*` in `smart_holder_type_caster` need to be marked as `const` member functions.
* Adding construct() overloads for constructing smart_holder from alias unique_ptr, shared_ptr returns.
* Adding test_class_sh_factory_constructors.cpp to tests/CMakeLists.txt (fixes oversight, this should have been added long before).
* Compatibility with old clang versions (clang 3.6, 3.7 C++11).
* Cleaning up changes to existing unit tests.
* Systematically adding SMART_HOLDER_WIP tag. Removing minor UNTESTED tags (only the throw are not actually exercised, investing time there has a high cost but very little benefit).
* Splitting out smart_holder_type_casters again, into new detail/smart_holder_type_casters_inline_include.h.
* Splitting out smart_holder_init_inline_include.h.
* Adding additional new include files to CMakeLists.txt, tests/extra_python_package/test_files.py.
* clang-format cleanup of most smart_holder code.
* Adding source code comments in response to review.
* Simple micro-benchmark ("ubench") comparing runtime performance for several holders.
Tested using github.com/rwgk/pybind11_scons and Google-internal build system.
Sorry, no cmake support at the moment.
First results: https://docs.google.com/spreadsheets/d/1InapCYws2Gt-stmFf_Bwl33eOMo3aLE_gc9adveY7RU/edit#gid=0
* Breaking out number_bucket.h, adding hook for also collecting performance data for PyCLIF.
* Accounting for ubench in MANIFEST.in (simply prune, for now).
* Smarter determination of call_repetitions.
[skip ci]
* Also scaling performance data to PyCLIF.
[skip ci]
* Adding ubench/python/number_bucket.clif here for general visibility.
* Fix after rebase
* Merging detail/smart_holder_init_inline_include.h into detail/init.h.
* Renaming detail/is_smart_holder_type_caster.h -> detail/smart_holder_sfinae_hooks_only.h.
* Renaming is_smart_holder_type_caster -> type_uses_smart_holder_type_caster for clarity.
* Renaming type_caster_type_is_smart_holder_type_caster -> wrapped_type_uses_smart_holder_type_caster for clarity.
* Renaming is_smart_holder_type_caster_base_tag -> smart_holder_type_caster_base_tag for simplicity.
* Adding copyright notices and minor colateral cleanup.
* iwyu cleanup (comprehensive only for cast.h and smart_holder*.h files).
* Fixing `git rebase master` accident.
* Moving large `pragma warning` block from pybind11.h to detail/common.h.
* Fixing another `git rebase master` accident.
* [dtype]: add type() method to access type attribute of PyArray_Descr (eq. to dtype.char in Python)
* [dtype] change type() name method to char_() to be compliant with Python numpy interface
* [dtype] fix by pre-commit
* [dtype] Change comments and solutions format for test
* Clarify documentation and move note about dtype.char vs PyArray_Descr::type to a plain, non-doxygen comment
* Fix and extend tests
* Fix the supposedly fixed tests
* Fix the fixed tests again
Co-authored-by: Bertrand MICHEL <bertrand.michel@onera.fr>
Co-authored-by: Yannick Jadoul <yannick.jadoul@belgacom.net>
* Make sure all warnings in pytest get turned into errors
* Suppress DeprecationWarnings in test_int_convert and test_numpy_int_convert
* PyLong_AsLong only shouts "Deprecated!" on Python>=3.8
* Fix remaining warnings on PyPy and CPython 3.10-dev
* Demonstrate issue with weakref constructor overloads
* Fix weakref constructor to convert on being passed a non-weakref object
* Improve on nonlocal-scoped variable in test_weakref
* Keep backwards-compatibility by introducing PYBIND11_OBJECT_CVT_DEFAULT macro
* Simplify test_weakref
* Always call PyNumber_Index when casting from Python to a C++ integral type, also pre-3.8
* Fixed on PyPy
* Simplify use of PyNumber_Index, following @rwgk's idea, and ignore warnings in >=3.8
* Reproduce mismatch between pre-3.8 and post-3.8 behavior on __index__ throwing TypeError
* Fix tests on 3.6 <= Python < 3.8
* No, I don't have an uninitialized variable
* Fix use of __index__ on Python 2
* Make types in test_int_convert more ~boring~ descriptive
* Force the builtin module key to be the correct type.
Previously it was always going to be a std::string which converted into
unicode. Python 2 appears to want module keys to be normal str types, so
this was breaking code that expected plain string types in the
builtins.keys() data structure
* Add a simple unit test to ensure all built-in keys are str
* Update the unit test so it will also run on pypy
* Run pre-commit.
Co-authored-by: Jesse Clemens <jesse.clemens@sony.com>
* CI: Intel icc/icpc via oneAPI
Add testing for Intel icc/icpc via the oneAPI images.
Intel oneAPI is in a late beta stage, currently shipping
oneAPI beta09 with ICC 20.2.
CI: Skip Interpreter Tests for Intel
Cannot find how to add this, neiter the package `libc6-dev` nor
`intel-oneapi-mkl-devel` help when installed to solve this:
```
-- Looking for C++ include pthread.h
-- Looking for C++ include pthread.h - not found
CMake Error at /__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:165 (message):
Could NOT find Threads (missing: Threads_FOUND)
Call Stack (most recent call first):
/__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:458 (_FPHSA_FAILURE_MESSAGE)
/__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindThreads.cmake:234 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
tests/test_embed/CMakeLists.txt:17 (find_package)
```
CI: libc6-dev from GCC for ICC
CI: Run bare metal for oneAPI
CI: Ubuntu 18.04 for oneAPI
CI: Intel +Catch -Eigen
CI: CMake from Apt (ICC tests)
CI: Replace Intel Py with GCC Py
CI: Intel w/o GCC's Eigen
CI: ICC with verbose make
[Debug] Find core dump
tests: use arg{} instead of arg() for Intel
tests: adding a few more missing {}
fix: sync with @tobiasleibner's branch
fix: try ubuntu 20-04
fix: drop exit 1
docs: Apply suggestions from code review
Co-authored-by: Tobias Leibner <tobias.leibner@googlemail.com>
Workaround for ICC enable_if issues
Another workaround for ICC's enable_if issues
fix error in previous commit
Disable one test for the Intel compiler in C++17 mode
Add back one instance of py::arg().noconvert()
Add NOLINT to fix clang-tidy check
Work around for ICC internal error in PYBIND11_EXPAND_SIDE_EFFECTS in C++17 mode
CI: Intel ICC with C++17
docs: pybind11/numpy.h does not require numpy at build time. (#2720)
This is nice enough to be mentioned explicitly in the docs.
docs: Update warning about Python 3.9.0 UB, now that 3.9.1 has been released (#2719)
Adjusting `type_caster<std::reference_wrapper<T>>` to support const/non-const propagation in `cast_op`. (#2705)
* Allow type_caster of std::reference_wrapper<T> to be the same as a native reference.
Before, both std::reference_wrapper<T> and std::reference_wrapper<const T> would
invoke cast_op<type>. This doesn't allow the type_caster<> specialization for T
to distinguish reference_wrapper types from value types.
After, the type_caster<> specialization invokes cast_op<type&>, which allows
reference_wrapper to behave in the same way as a native reference type.
* Add tests/examples for std::reference_wrapper<const T>
* Add tests which use mutable/immutable variants
This test is a chimera; it blends the pybind11 casters with a custom
pytype implementation that supports immutable and mutable calls.
In order to detect the immutable/mutable state, the cast_op needs
to propagate it, even through e.g. std::reference<const T>
Note: This is still a work in progress; some things are crashing,
which likely means that I have a refcounting bug or something else
missing.
* Add/finish tests that distinguish const& from &
Fixes the bugs in my custom python type implementation,
demonstrate test that requires const& and reference_wrapper<const T>
being treated differently from Non-const.
* Add passing a const to non-const method.
* Demonstrate non-const conversion of reference_wrapper in tests.
Apply formatting presubmit check.
* Fix build errors from presubmit checks.
* Try and fix a few more CI errors
* More CI fixes.
* More CI fixups.
* Try and get PyPy to work.
* Additional minor fixups. Getting close to CI green.
* More ci fixes?
* fix clang-tidy warnings from presubmit
* fix more clang-tidy warnings
* minor comment and consistency cleanups
* PyDECREF -> Py_DECREF
* copy/move constructors
* Resolve codereview comments
* more review comment fixes
* review comments: remove spurious &
* Make the test fail even when the static_assert is commented out.
This expands the test_freezable_type_caster a bit by:
1/ adding accessors .is_immutable and .addr to compare identity
from python.
2/ Changing the default cast_op of the type_caster<> specialization
to return a non-const value. In normal codepaths this is a reasonable
default.
3/ adding roundtrip variants to exercise the by reference, by pointer
and by reference_wrapper in all call paths. In conjunction with 2/, this
demonstrates the failure case of the existing std::reference_wrpper conversion,
which now loses const in a similar way that happens when using the default cast_op_type<>.
* apply presubmit formatting
* Revert inclusion of test_freezable_type_caster
There's some concern that this test is a bit unwieldly because of the use
of the raw <Python.h> functions. Removing for now.
* Add a test that validates const references propagation.
This test verifies that cast_op may be used to correctly detect
const reference types when used with std::reference_wrapper.
* mend
* Review comments based changes.
1. std::add_lvalue_reference<type> -> type&
2. Simplify the test a little more; we're never returning the ConstRefCaster
type so the class_ definition can be removed.
* formatted files again.
* Move const_ref_caster test to builtin_casters
* Review comments: use cast_op and adjust some comments.
* Simplify ConstRefCasted test
I like this version better as it moves the assertion that matters
back into python.
ci: drop pypy2 linux, PGI 20.7, add Python 10 dev (#2724)
* ci: drop pypy2 linux, add Python 10 dev
* ci: fix mistake
* ci: commented-out PGI 20.11, drop 20.7
fix: regression with installed pybind11 overriding local one (#2716)
* fix: regression with installed pybind11 overriding discovered one
Closes#2709
* docs: wording incorrect
style: remove redundant instance->owned = true (#2723)
which was just before set to True in instance->allocate_layout()
fix: also throw in the move-constructor added by the PYBIND11_OBJECT macro, after the argument has been moved-out (if necessary) (#2701)
Make args_are_all_* ICC workarounds unconditional
Disable test_aligned on Intel ICC
Fix test_aligned on Intel ICC
Skip test_python_alreadyset_in_destructor on Intel ICC
Fix test_aligned again
ICC CI: Downgrade pytest
pytest 6 does not capture the `discard_as_unraisable` stderr and
just writes a warning with its content instead.
* refactor: simpler Intel workaround, suggested by @laramiel
* fix: try version with impl to see if it is easier to compile
* docs: update README for ICC
Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
* Only allow integer type_caster to call __int__ or __index__ method when conversion is allowed
* Remove tests for __index__ as this seems to only be used to convert to int in 3.8+
* Take both `int` and `long` types into account for Python 2
* Add test_numpy_int_convert to assert tests currently fail, even though np.intc has an __index__ method
* Also consider __index__ as noconvert to a C++ integer
* New-style classes for Python 2.7; sigh
* Add some tests on types with custom __index__ method
* Ignore some tests in Python <3.8
* Update comment about conversion from np.float32 to C++ int
* Workaround difference between CPython and PyPy's different PyIndex_Check (unnoticed because we currently don't have PyPy >= 3.8)
* Avoid ICC segfault with py::arg()
* CI: Intel icc/icpc via oneAPI
Add testing for Intel icc/icpc via the oneAPI images.
Intel oneAPI is in a late beta stage, currently shipping
oneAPI beta09 with ICC 20.2.
* CI: Skip Interpreter Tests for Intel
Cannot find how to add this, neiter the package `libc6-dev` nor
`intel-oneapi-mkl-devel` help when installed to solve this:
```
-- Looking for C++ include pthread.h
-- Looking for C++ include pthread.h - not found
CMake Error at /__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:165 (message):
Could NOT find Threads (missing: Threads_FOUND)
Call Stack (most recent call first):
/__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:458 (_FPHSA_FAILURE_MESSAGE)
/__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindThreads.cmake:234 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
tests/test_embed/CMakeLists.txt:17 (find_package)
```
* CI: libc6-dev from GCC for ICC
* CI: Run bare metal for oneAPI
* CI: Ubuntu 18.04 for oneAPI
* CI: Intel +Catch -Eigen
* CI: CMake from Apt (ICC tests)
* CI: Replace Intel Py with GCC Py
* CI: Intel w/o GCC's Eigen
* CI: ICC with verbose make
* [Debug] Find core dump
* tests: use arg{} instead of arg() for Intel
* tests: adding a few more missing {}
* fix: sync with @tobiasleibner's branch
* fix: try ubuntu 20-04
* fix: drop exit 1
* style: clang tidy fix
* style: fix missing NOLINT
* ICC: Update Compiler Name
Changed upstream with the last oneAPI release.
* ICC CI: Downgrade pytest
pytest 6 does not capture the `discard_as_unraisable` stderr and
just writes a warning with its content instead.
* Use new test pinning requirements.txt
* tests: add notes about intel, cleanup
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
* Adding a valgrind build on debug Python 3.9
Co-authored-by: Boris Staletic <boris.staletic@gmail.com>
* Add Valgrind suppression files
- Introduce suppression file, populate it with a first suppression taken from CPython, and fix one leak in the tests
- Suppress leak in NumPy
- More clean tests!
- Tests with names a-e passing (except for test_buffer)
- Suppress multiprocessing errors
- Merge multiprocessing suppressions into other suppression files
- Numpy seems to be spelled with a big P
- Append single entry from valgrind-misc.supp to valgrind-python.supp, and make clear valgrind-python.supp is only CPython
Co-authored-by: Boris Staletic <boris.staletic@gmail.com>
* Enable test_virtual_functions with a workaround
* Add a memcheck cmake target
- Add a memcheck cmake target
- Reformat cmake
- Appease the formatting overlords - they are angry
- Format CMake valgrind target decently
* Update CI config to new action versions
* fix: separate memcheck from pytest
* ci: cleanup
* Merge Valgrind and other deadsnakes builds
Co-authored-by: Boris Staletic <boris.staletic@gmail.com>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
* Plug leaking function_record objects when exceptions are thrown
* Plug leak of strdup'ed strings in function_record
* Some extra comments about the function_record ownership dance
* Clean up the function_record better, in case of exceptions
* Demonstrate some extra function_record leaks
* Change DeleteStrings template argument to free_strings runtime argument in destruct(function_record *)
* Zero-state unique_function_record deleter object
* Clarify rvalue reference to unique_ptr parameter in initialize_generic
* Use push_back with const char * instead of emplace_back
* Ignore old-style __init__ warnings
* Simplify ignoreOldStyleInitWarnings with py::exec
* Only wrap single class_::defs to ignore DeprecationWarnings about old-style __init__
* Fix leak in the test_copy_move::test_move_fallback
* Fix leaking PyMethodDef in test_class::test_implicit_conversion_life_support
* Plumb leak in test_buffer, occuring when a mutable buffer is requested for a read-only object, and enable test_buffer.py
* Fix weird return_value_policy::reference in test_stl_binders, and enable those tests
* Cleanup nodelete holder objects in test_smart_ptr, and enable those tests
* Update pytest to 6.2.1 in tests/requirements.txt
* Pin pytest to last supported version for 3.5
* Suppress PytestUnraisableExceptionWarning and use sys.__unraisablehook__ instead of sys.unraisablehook
* Fix filterwarnings mark on old pytest and old Python versions
* Cleanup ignore_pytest_unraisable_warning decorator
* minor cleanup: fixing or silencing flake8 errors
* ci: lock CMake to non-Universal version
* Update .github/workflows/ci.yml
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
* Allow type_caster of std::reference_wrapper<T> to be the same as a native reference.
Before, both std::reference_wrapper<T> and std::reference_wrapper<const T> would
invoke cast_op<type>. This doesn't allow the type_caster<> specialization for T
to distinguish reference_wrapper types from value types.
After, the type_caster<> specialization invokes cast_op<type&>, which allows
reference_wrapper to behave in the same way as a native reference type.
* Add tests/examples for std::reference_wrapper<const T>
* Add tests which use mutable/immutable variants
This test is a chimera; it blends the pybind11 casters with a custom
pytype implementation that supports immutable and mutable calls.
In order to detect the immutable/mutable state, the cast_op needs
to propagate it, even through e.g. std::reference<const T>
Note: This is still a work in progress; some things are crashing,
which likely means that I have a refcounting bug or something else
missing.
* Add/finish tests that distinguish const& from &
Fixes the bugs in my custom python type implementation,
demonstrate test that requires const& and reference_wrapper<const T>
being treated differently from Non-const.
* Add passing a const to non-const method.
* Demonstrate non-const conversion of reference_wrapper in tests.
Apply formatting presubmit check.
* Fix build errors from presubmit checks.
* Try and fix a few more CI errors
* More CI fixes.
* More CI fixups.
* Try and get PyPy to work.
* Additional minor fixups. Getting close to CI green.
* More ci fixes?
* fix clang-tidy warnings from presubmit
* fix more clang-tidy warnings
* minor comment and consistency cleanups
* PyDECREF -> Py_DECREF
* copy/move constructors
* Resolve codereview comments
* more review comment fixes
* review comments: remove spurious &
* Make the test fail even when the static_assert is commented out.
This expands the test_freezable_type_caster a bit by:
1/ adding accessors .is_immutable and .addr to compare identity
from python.
2/ Changing the default cast_op of the type_caster<> specialization
to return a non-const value. In normal codepaths this is a reasonable
default.
3/ adding roundtrip variants to exercise the by reference, by pointer
and by reference_wrapper in all call paths. In conjunction with 2/, this
demonstrates the failure case of the existing std::reference_wrpper conversion,
which now loses const in a similar way that happens when using the default cast_op_type<>.
* apply presubmit formatting
* Revert inclusion of test_freezable_type_caster
There's some concern that this test is a bit unwieldly because of the use
of the raw <Python.h> functions. Removing for now.
* Add a test that validates const references propagation.
This test verifies that cast_op may be used to correctly detect
const reference types when used with std::reference_wrapper.
* mend
* Review comments based changes.
1. std::add_lvalue_reference<type> -> type&
2. Simplify the test a little more; we're never returning the ConstRefCaster
type so the class_ definition can be removed.
* formatted files again.
* Move const_ref_caster test to builtin_casters
* Review comments: use cast_op and adjust some comments.
* Simplify ConstRefCasted test
I like this version better as it moves the assertion that matters
back into python.
* Adding missing virtual destructors, to silence clang -Wnon-virtual-dtor warnings.
Tested with clang version 9.0.1-12 under an Ubuntu-like OS.
Originally discovered in the Google-internal environment.
* adding -Wnon-virtual-dtor for GNU|Intel|Clang
