* Add a test showing a flaw in make_key_iterator/make_value_iterator
If the iterator dereference operator returns a value rather than a
reference (and that pair also does not *contain* references),
make_key_iterator and make_value_iterator will return a reference to a
temporary, causing a segfault.
* Fix make_key_iterator/make_value_iterator for prvalue iterators
If an iterator returns a pair<T1, T2> rather than a reference to a pair
or a pair of references, make_key_iterator and make_value_iterator would
return a reference to a temporary, typically leading to a segfault. This
is because the value category of member access to a prvalue is an
xvalue, not a prvalue, so decltype produces an rvalue reference type.
Fix the type calculation to handle this case.
I also removed some decltype parentheses that weren't needed, either
because the expression isn't one of the special cases for decltype or
because decltype was only used for SFINAE. Hopefully that makes the code
a bit more readable.
Closes#3347
* Attempt a workaround for nvcc
* Add `.keys` and `.values` to bind_map
Both of these implement views (rather than just iterators), and `.items`
is also upgraded to a view. In practical terms, this allows a view to be
iterated multiple times and have its size taken, neither of which works
with an iterator.
The views implement `__len__`, `__iter__`, and the keys view implements
`__contains__`. Testing membership also works in item and value views
because Python falls back to iteration. This won't be optimal
for item values since it's linear rather than O(log n) or O(1), but I
didn't fancy trying to get all the corner cases to match Python
behaviour (tuple of wrong types, wrong length tuple, not a tuple etc).
Missing relative to Python dictionary views is `__reversed__` (only
added to Python in 3.8). Implementing that could break code that binds
custom map classes which don't provide `rbegin`/`rend` (at least without
doing clever things with SFINAE), so I've not tried.
The size increase on my system is 131072 bytes, which is rather large
(5%) but also suspiciously round (2^17) and makes me suspect some
quantisation effect.
* bind_map: support any object in __contains__
Add extra overload of `__contains__` (for both the map itself and
KeysView) which takes an arbitrary object and returns false.
* Take py::object by const reference in __contains__
To keep clang-tidy happy.
* Removing stray `py::` (detected via interactive testing in Google environment).
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
* Docs: Demonstrate non-enum internal types in example
Previously example only demonstrated internal enumeration type.
To show that it works for other internal types the same way the example was updated with an additional struct Pet::Attributes type.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix `pybind11::object::operator=` to be safe if `*this` is accessible from Python
* Add `custom_type_setup` attribute
This allows for custom modifications to the PyHeapTypeObject prior to
calling `PyType_Ready`. This may be used, for example, to define
`tp_traverse` and `tp_clear` functions.
The new FindPython-based variant of the CMake scripts caches information
about the chosen Python version that can become stale. For example,
suppose I configure a simple pybind11-based project as follows
```
cmake -S . -B build -GNinja -DPython_ROOT=<path to python 3.8>
```
which will generate `my_extension.cpython-38-x86_64-linux-gnu.so`.
A subsequent change to the python version like
```
cmake -S . -B build -GNinja -DPython_ROOT=<path to python 3.9>
```
does not update all necessary build system information. In particular,
the compiled file is still called
`my_extension.cpython-38-x86_64-linux-gnu.so`.
This commit fixes the problem by detecting changes in
`Python_EXECUTABLE` and re-running Python as needed.
Note that the previous way of detecting Python does not seem to be
affected, it always specifies the right suffix.
* Add make_value_iterator (#3271)
* Add make_value_iterator
This is the counterpart to make_key_iterator, and will allow
implementing a `value` method in `bind_map` (although doing so is left
for a subsequent PR).
I made a few design changes to reduce copy-and-paste boilerplate.
Previously detail::iterator_state had a boolean template parameter to
indicate whether it was being used for make_iterator or
make_key_iterator. I replaced the boolean with a class that determines
how to dereference the iterator. This allows for a generic
implementation of `__next__`.
I also added the ValueType and Extra... parameters to the iterator_state
template args, because I think it was a bug that they were missing: if
make_iterator is called twice with different values of these, only the
first set has effect (because the state class is only registered once).
There is still a potential issue in that the *values* of the extra
arguments are latched on the first call, but since most policies are
empty classes this should be even less common.
* Add some remove_cv_t to appease clang-tidy
* Make iterator_access and friends take reference
For some reason I'd accidentally made it take a const value, which
caused some issues with third-party packages.
* Another attempt to remove remove_cv_t from iterators
Some of the return types were const (non-reference) types because of the
pecularities of decltype: `decltype((*it).first)` is the *declared* type
of the member of the pair, rather than the type of the expression. So if
the reference type of the iterator is `pair<const int, int> &`, then the
decltype is `const int`. Wrapping an extra set of parentheses to form
`decltype(((*it).first))` would instead give `const int &`.
This means that the existing make_key_iterator actually returns by value
from `__next__`, rather than by reference. Since for mapping types, keys
are always const, this probably hasn't been noticed, but it will affect
make_value_iterator if the Python code tries to mutate the returned
objects. I've changed things to use double parentheses so that
make_iterator, make_key_iterator and make_value_iterator should now all
return the reference type of the iterator. I'll still need to add a test
for that; for now I'm just checking whether I can keep Clang-Tidy happy.
* Add back some NOLINTNEXTLINE to appease Clang-Tidy
This is favoured over using remove_cv_t because in some cases a const
value return type is deliberate (particularly for Eigen).
* Add a unit test for iterator referencing
Ensure that make_iterator, make_key_iterator and make_value_iterator
return references to the container elements, rather than copies. The
test for make_key_iterator fails to compile on master, which gives me
confidence that this branch has fixed it.
* Make the iterator_access etc operator() const
I'm actually a little surprised it compiled at all given that the
operator() is called on a temporary, but I don't claim to fully
understand all the different value types in C++11.
* Attempt to work around compiler bugs
https://godbolt.org/ shows an example where ICC gets the wrong result
for a decltype used as the default for a template argument, and CI also
showed problems with PGI. This is a shot in the dark to see if it fixes
things.
* Make a test constructor explicit (Clang-Tidy)
* Fix unit test on GCC 4.8.5
It seems to require the arguments to the std::pair constructor to be
implicitly convertible to the types in the pair, rather than just
requiring is_constructible.
* Remove DOXYGEN_SHOULD_SKIP_THIS guards
Now that a complex decltype expression has been replaced by a simpler
nested type, I'm hoping Doxygen will be able to build it without issues.
* Add comment to explain iterator_state template params
* fix: regression in #3271
Co-authored-by: Bruce Merry <1963944+bmerry@users.noreply.github.com>
* Adding nullptr slices
Using example from #1095
Some fixes from @wjakob's review
Stop clang-tidy from complaining
New proposal for py::slice constructor
Eric's suggested changes: simplify testing; shift def's
* chore: drop MSVC pragma (hopefully unneeded)
* Apply suggestions from code review
* Add make_value_iterator
This is the counterpart to make_key_iterator, and will allow
implementing a `value` method in `bind_map` (although doing so is left
for a subsequent PR).
I made a few design changes to reduce copy-and-paste boilerplate.
Previously detail::iterator_state had a boolean template parameter to
indicate whether it was being used for make_iterator or
make_key_iterator. I replaced the boolean with a class that determines
how to dereference the iterator. This allows for a generic
implementation of `__next__`.
I also added the ValueType and Extra... parameters to the iterator_state
template args, because I think it was a bug that they were missing: if
make_iterator is called twice with different values of these, only the
first set has effect (because the state class is only registered once).
There is still a potential issue in that the *values* of the extra
arguments are latched on the first call, but since most policies are
empty classes this should be even less common.
* Add some remove_cv_t to appease clang-tidy
* Make iterator_access and friends take reference
For some reason I'd accidentally made it take a const value, which
caused some issues with third-party packages.
* Another attempt to remove remove_cv_t from iterators
Some of the return types were const (non-reference) types because of the
pecularities of decltype: `decltype((*it).first)` is the *declared* type
of the member of the pair, rather than the type of the expression. So if
the reference type of the iterator is `pair<const int, int> &`, then the
decltype is `const int`. Wrapping an extra set of parentheses to form
`decltype(((*it).first))` would instead give `const int &`.
This means that the existing make_key_iterator actually returns by value
from `__next__`, rather than by reference. Since for mapping types, keys
are always const, this probably hasn't been noticed, but it will affect
make_value_iterator if the Python code tries to mutate the returned
objects. I've changed things to use double parentheses so that
make_iterator, make_key_iterator and make_value_iterator should now all
return the reference type of the iterator. I'll still need to add a test
for that; for now I'm just checking whether I can keep Clang-Tidy happy.
* Add back some NOLINTNEXTLINE to appease Clang-Tidy
This is favoured over using remove_cv_t because in some cases a const
value return type is deliberate (particularly for Eigen).
* Add a unit test for iterator referencing
Ensure that make_iterator, make_key_iterator and make_value_iterator
return references to the container elements, rather than copies. The
test for make_key_iterator fails to compile on master, which gives me
confidence that this branch has fixed it.
* Make the iterator_access etc operator() const
I'm actually a little surprised it compiled at all given that the
operator() is called on a temporary, but I don't claim to fully
understand all the different value types in C++11.
* Attempt to work around compiler bugs
https://godbolt.org/ shows an example where ICC gets the wrong result
for a decltype used as the default for a template argument, and CI also
showed problems with PGI. This is a shot in the dark to see if it fixes
things.
* Make a test constructor explicit (Clang-Tidy)
* Fix unit test on GCC 4.8.5
It seems to require the arguments to the std::pair constructor to be
implicitly convertible to the types in the pair, rather than just
requiring is_constructible.
* Remove DOXYGEN_SHOULD_SKIP_THIS guards
Now that a complex decltype expression has been replaced by a simpler
nested type, I'm hoping Doxygen will be able to build it without issues.
* Add comment to explain iterator_state template params
* Enable defining custom __new__
* See if xfail needed
* Qualify auto self
* Unconditionally defining PYBIND11_DISABLE_NEW_STYLE_INIT_WARNING. Returning pointer from "__init__" instead of reference.
* Use new style __init__
* Simplify __new__ creation
* Reviewer suggestions
* Match indentation
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
* Eliminate duplicate TLS keys for loader_life_support stack
This revises the existing fix for
https://github.com/pybind/pybind11/issues/2765 in
https://github.com/pybind/pybind11/pull/3237 to reduce the amount of
TLS storage used.
The shared TLS key is stored in two different ways, depending on
`PYBIND11_INTERNALS_VERSION`. If `PYBIND11_INTERNALS_VERSION ==
4` (as is currently set), the TLS key is stored in the
`internal::shared_data` map to avoid breaking ABI compatibility. If
`PYBIND11_INTERNALS_VERSION > 4`, the TLS key is stored directly in
the `internals` struct.
* Fix test_pytypes.py::test_issue2361 failure on PyPy3.7
* Add github actions tests for unstable ABI
* Restoring `const` removed from pytypes.h in PR #3254, adding tests reflective of user code that breaks when those `const` are removed.
* clang-tidy NOLINTs (and one collateral fix).
* Inserting PYBIND11_CONST_FOR_STRICT_PLATFORMS
* Trying `defined(__APPLE__)`
* Trying again: `auto it` for strict platforms.
* Adding NOLINTNEXTLINE(bugprone-macro-parentheses), expanding comments.
* Labeling all changes with `PR #3263`, for easy reference, and to make it easy to undo these changes if we decide to do so in the future.
* Fix thread safety for pybind11 loader_life_support
Fixes issue: https://github.com/pybind/pybind11/issues/2765
This converts the vector of PyObjects to either a single void* or
a per-thread void* depending on the WITH_THREAD define.
The new field is used by each thread to construct a stack
of loader_life_support frames that can extend the life of python
objects.
The pointer is updated when the loader_life_support object is allocated
(which happens before a call) as well as on release.
Each loader_life_support maintains a set of PyObject references
that need to be lifetime extended; this is done by storing them
in a c++ std::unordered_set and clearing the references when the
method completes.
* Also update the internals version as the internal struct is no longer compatible
* Add test demonstrating threading works correctly.
It may be appropriate to run this under msan/tsan/etc.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update test to use lifetime-extended references rather than
std::string_view, as that's a C++ 17 feature.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Make loader_life_support members private
* Update version to dev2
* Update test to use python threading rather than concurrent.futures
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove unnecessary env in test
* Remove unnecessary pytest in test
* Use native C++ thread_local in place of python per-thread data structures to retain compatability
* clang-format test_thread.cpp
* Add a note about debugging the py::cast() error
* thread_test.py now propagates exceptions on join() calls.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* remove unused sys / merge
* Update include order in test_thread.cpp
* Remove spurious whitespace
* Update comment / whitespace.
* Address review comments
* lint cleanup
* Fix test IntStruct constructor.
* Add explicit to constructor
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
* Set __file__ constant when using eval_file
* Use const ref
* Use a move instead
* Revert
* Improve test
* Guard test with Python version
* Fix tests
* Dont support Python2 API
* Drop Python2 eval __file__ support
* Hack
* Semisupport Python2
* Take2
* Remove Python2 support
* Minor tweaks.
* Restoring tests/pybind11_tests.h version from master, removing just the comment and empty line that was added in PR #3087; those were made obsolete by the pragma cleanup that concluded with PR #3186.
* More-to-the-point test for Python 3.
