* Add return_as_bytes policy
* Fix format
* Fix test failures
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix std::variant
* Resolve comments
* Note this policy experimental
* Add tests for `return_as_bytes` with `def_property`.
* Change comment for the new return_as_bytes enum to note that the policy is not available on master.
* Applying pr3838_sh.patch (exactly as used Google-internally since 2022-03-31).
* Add `case return_as_bytes` to `switch`es in detail/type_caster_base.h and eigen.h
Based on systematic review under https://github.com/pybind/pybind11/pull/3838#issuecomment-1094390333
* Add missing break (clang-tidy).
* More clang-tidy fixes (this time around clang-tidy was run interactively to pre-empt repeat trips through the CI).
* Underscore prefix: _return_as_bytes
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
* `#error BYE_BYE_GOLDEN_SNAKE`
* Removing everything related to 2.7 from ci.yml
* Commenting-out Centos7
* Removing `PYTHON: 27` from .appveyor.yml
* "PY2" removal, mainly from tests. C++ code is not touched.
* Systematic removal of `u` prefix from `u"..."` and `u'...'` literals. Collateral cleanup of a couple minor other things.
* Cleaning up around case-insensitive hits for `[^a-z]py.*2` in tests/.
* Removing obsolete Python 2 mention in compiling.rst
* Proper `#error` for Python 2.
* Using PY_VERSION_HEX to guard `#error "PYTHON 2 IS NO LONGER SUPPORTED.`
* chore: bump pre-commit
* style: run pre-commit for pyupgrade 3+
* tests: use sys.version_info, not PY
* chore: more Python 2 removal
* Uncommenting Centos7 block (PR #3691 showed that it is working again).
* Update pre-commit hooks
* Fix pre-commit hook
* refactor: remove Python 2 from CMake
* refactor: remove Python 2 from setup code
* refactor: simplify, better static typing
* feat: fail with nice messages
* refactor: drop Python 2 C++ code
* docs: cleanup for Python 3
* revert: intree
revert: intree
* docs: minor touchup to py2 statement
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
* Adding readability-qualified-auto to .clang-tidy
Ported from @henryiii's 287527f705
* fix: support Python < 3.6
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
* Expand string_view support to str, bytes, memoryview
1. Allows constructing a str or bytes implicitly from a string_view;
this is essentially a small shortcut allowing a caller to write
`py::bytes{sv}` rather than `py::bytes{sv.data(), sv.size()}`.
2. Allows implicit conversion *to* string_view from py::bytes -- this
saves a fair bit more as currently there is no simple way to get such
a view of the bytes without copying it (or resorting to Python API
calls).
(This is not done for `str` because when the str contains unicode we
have to allocate to a temporary and so there might not be some string
data we can properly view without owning.)
3. Allows `memoryview::from_memory` to accept a string_view. As with
the other from_memory calls, it's entirely your responsibility to
keep it alive.
This also required moving the string_view availability detection into
detail/common.h because this PR needs it in pytypes.h, which is higher
up the include chain than cast.h where it was being detected currently.
* Move string_view include to pytypes.h
* CI-testing a fix for the "ambiguous conversion" issue.
This change is known to fix the `tensorflow::tstring` issue reported under https://github.com/pybind/pybind11/pull/3521#issuecomment-985100965
TODO: Minimal reproducer for the `tensorflow::tstring` issue.
* Make clang-tidy happy (hopefully).
* Adding minimal reproducer for the `tensorflow::tstring` issue.
Error without the enable_if trick:
```
/usr/local/google/home/rwgk/forked/pybind11/tests/test_builtin_casters.cpp:169:16: error: ambiguous conversion for functional-style cast from 'TypeWithBothOperatorStringAndStringView' to 'py::bytes'
return py::bytes(TypeWithBothOperatorStringAndStringView());
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/google/home/rwgk/forked/pybind11/include/pybind11/detail/../pytypes.h:1174:5: note: candidate constructor
bytes(const std::string &s) : bytes(s.data(), s.size()) { }
^
/usr/local/google/home/rwgk/forked/pybind11/include/pybind11/detail/../pytypes.h:1191:5: note: candidate constructor
bytes(std::string_view s) : bytes(s.data(), s.size()) { }
^
```
* Adding missing NOLINTNEXTLINE
* Also apply ambiguous conversion workaround to str()
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
* Removing all MSVC C4127 warning suppression pragmas.
* Removing MSVC /WX (WERROR). To get a full list of all warnings.
* Inserting PYBIND11_SILENCE_MSVC_C4127. Changing one runtime if to #if.
* Changing PYBIND11_SILENCE_MSVC_C4127 macro to use absolute namespace (for use outside pybind11 include directory).
* Restoring MSVC /WX (WERROR).
* Removing globally-scoped suppression for clang -Wunsequenced. Based on an experiment under PR #3202 it is obsolete and can simply be removed.
* CI: Intel icc/icpc via oneAPI
Add testing for Intel icc/icpc via the oneAPI images.
Intel oneAPI is in a late beta stage, currently shipping
oneAPI beta09 with ICC 20.2.
CI: Skip Interpreter Tests for Intel
Cannot find how to add this, neiter the package `libc6-dev` nor
`intel-oneapi-mkl-devel` help when installed to solve this:
```
-- Looking for C++ include pthread.h
-- Looking for C++ include pthread.h - not found
CMake Error at /__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:165 (message):
Could NOT find Threads (missing: Threads_FOUND)
Call Stack (most recent call first):
/__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:458 (_FPHSA_FAILURE_MESSAGE)
/__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindThreads.cmake:234 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
tests/test_embed/CMakeLists.txt:17 (find_package)
```
CI: libc6-dev from GCC for ICC
CI: Run bare metal for oneAPI
CI: Ubuntu 18.04 for oneAPI
CI: Intel +Catch -Eigen
CI: CMake from Apt (ICC tests)
CI: Replace Intel Py with GCC Py
CI: Intel w/o GCC's Eigen
CI: ICC with verbose make
[Debug] Find core dump
tests: use arg{} instead of arg() for Intel
tests: adding a few more missing {}
fix: sync with @tobiasleibner's branch
fix: try ubuntu 20-04
fix: drop exit 1
docs: Apply suggestions from code review
Co-authored-by: Tobias Leibner <tobias.leibner@googlemail.com>
Workaround for ICC enable_if issues
Another workaround for ICC's enable_if issues
fix error in previous commit
Disable one test for the Intel compiler in C++17 mode
Add back one instance of py::arg().noconvert()
Add NOLINT to fix clang-tidy check
Work around for ICC internal error in PYBIND11_EXPAND_SIDE_EFFECTS in C++17 mode
CI: Intel ICC with C++17
docs: pybind11/numpy.h does not require numpy at build time. (#2720)
This is nice enough to be mentioned explicitly in the docs.
docs: Update warning about Python 3.9.0 UB, now that 3.9.1 has been released (#2719)
Adjusting `type_caster<std::reference_wrapper<T>>` to support const/non-const propagation in `cast_op`. (#2705)
* Allow type_caster of std::reference_wrapper<T> to be the same as a native reference.
Before, both std::reference_wrapper<T> and std::reference_wrapper<const T> would
invoke cast_op<type>. This doesn't allow the type_caster<> specialization for T
to distinguish reference_wrapper types from value types.
After, the type_caster<> specialization invokes cast_op<type&>, which allows
reference_wrapper to behave in the same way as a native reference type.
* Add tests/examples for std::reference_wrapper<const T>
* Add tests which use mutable/immutable variants
This test is a chimera; it blends the pybind11 casters with a custom
pytype implementation that supports immutable and mutable calls.
In order to detect the immutable/mutable state, the cast_op needs
to propagate it, even through e.g. std::reference<const T>
Note: This is still a work in progress; some things are crashing,
which likely means that I have a refcounting bug or something else
missing.
* Add/finish tests that distinguish const& from &
Fixes the bugs in my custom python type implementation,
demonstrate test that requires const& and reference_wrapper<const T>
being treated differently from Non-const.
* Add passing a const to non-const method.
* Demonstrate non-const conversion of reference_wrapper in tests.
Apply formatting presubmit check.
* Fix build errors from presubmit checks.
* Try and fix a few more CI errors
* More CI fixes.
* More CI fixups.
* Try and get PyPy to work.
* Additional minor fixups. Getting close to CI green.
* More ci fixes?
* fix clang-tidy warnings from presubmit
* fix more clang-tidy warnings
* minor comment and consistency cleanups
* PyDECREF -> Py_DECREF
* copy/move constructors
* Resolve codereview comments
* more review comment fixes
* review comments: remove spurious &
* Make the test fail even when the static_assert is commented out.
This expands the test_freezable_type_caster a bit by:
1/ adding accessors .is_immutable and .addr to compare identity
from python.
2/ Changing the default cast_op of the type_caster<> specialization
to return a non-const value. In normal codepaths this is a reasonable
default.
3/ adding roundtrip variants to exercise the by reference, by pointer
and by reference_wrapper in all call paths. In conjunction with 2/, this
demonstrates the failure case of the existing std::reference_wrpper conversion,
which now loses const in a similar way that happens when using the default cast_op_type<>.
* apply presubmit formatting
* Revert inclusion of test_freezable_type_caster
There's some concern that this test is a bit unwieldly because of the use
of the raw <Python.h> functions. Removing for now.
* Add a test that validates const references propagation.
This test verifies that cast_op may be used to correctly detect
const reference types when used with std::reference_wrapper.
* mend
* Review comments based changes.
1. std::add_lvalue_reference<type> -> type&
2. Simplify the test a little more; we're never returning the ConstRefCaster
type so the class_ definition can be removed.
* formatted files again.
* Move const_ref_caster test to builtin_casters
* Review comments: use cast_op and adjust some comments.
* Simplify ConstRefCasted test
I like this version better as it moves the assertion that matters
back into python.
ci: drop pypy2 linux, PGI 20.7, add Python 10 dev (#2724)
* ci: drop pypy2 linux, add Python 10 dev
* ci: fix mistake
* ci: commented-out PGI 20.11, drop 20.7
fix: regression with installed pybind11 overriding local one (#2716)
* fix: regression with installed pybind11 overriding discovered one
Closes#2709
* docs: wording incorrect
style: remove redundant instance->owned = true (#2723)
which was just before set to True in instance->allocate_layout()
fix: also throw in the move-constructor added by the PYBIND11_OBJECT macro, after the argument has been moved-out (if necessary) (#2701)
Make args_are_all_* ICC workarounds unconditional
Disable test_aligned on Intel ICC
Fix test_aligned on Intel ICC
Skip test_python_alreadyset_in_destructor on Intel ICC
Fix test_aligned again
ICC CI: Downgrade pytest
pytest 6 does not capture the `discard_as_unraisable` stderr and
just writes a warning with its content instead.
* refactor: simpler Intel workaround, suggested by @laramiel
* fix: try version with impl to see if it is easier to compile
* docs: update README for ICC
Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
* Only allow integer type_caster to call __int__ or __index__ method when conversion is allowed
* Remove tests for __index__ as this seems to only be used to convert to int in 3.8+
* Take both `int` and `long` types into account for Python 2
* Add test_numpy_int_convert to assert tests currently fail, even though np.intc has an __index__ method
* Also consider __index__ as noconvert to a C++ integer
* New-style classes for Python 2.7; sigh
* Add some tests on types with custom __index__ method
* Ignore some tests in Python <3.8
* Update comment about conversion from np.float32 to C++ int
* Workaround difference between CPython and PyPy's different PyIndex_Check (unnoticed because we currently don't have PyPy >= 3.8)
* Avoid ICC segfault with py::arg()
* CI: Intel icc/icpc via oneAPI
Add testing for Intel icc/icpc via the oneAPI images.
Intel oneAPI is in a late beta stage, currently shipping
oneAPI beta09 with ICC 20.2.
* CI: Skip Interpreter Tests for Intel
Cannot find how to add this, neiter the package `libc6-dev` nor
`intel-oneapi-mkl-devel` help when installed to solve this:
```
-- Looking for C++ include pthread.h
-- Looking for C++ include pthread.h - not found
CMake Error at /__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:165 (message):
Could NOT find Threads (missing: Threads_FOUND)
Call Stack (most recent call first):
/__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:458 (_FPHSA_FAILURE_MESSAGE)
/__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindThreads.cmake:234 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
tests/test_embed/CMakeLists.txt:17 (find_package)
```
* CI: libc6-dev from GCC for ICC
* CI: Run bare metal for oneAPI
* CI: Ubuntu 18.04 for oneAPI
* CI: Intel +Catch -Eigen
* CI: CMake from Apt (ICC tests)
* CI: Replace Intel Py with GCC Py
* CI: Intel w/o GCC's Eigen
* CI: ICC with verbose make
* [Debug] Find core dump
* tests: use arg{} instead of arg() for Intel
* tests: adding a few more missing {}
* fix: sync with @tobiasleibner's branch
* fix: try ubuntu 20-04
* fix: drop exit 1
* style: clang tidy fix
* style: fix missing NOLINT
* ICC: Update Compiler Name
Changed upstream with the last oneAPI release.
* ICC CI: Downgrade pytest
pytest 6 does not capture the `discard_as_unraisable` stderr and
just writes a warning with its content instead.
* Use new test pinning requirements.txt
* tests: add notes about intel, cleanup
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
* Allow type_caster of std::reference_wrapper<T> to be the same as a native reference.
Before, both std::reference_wrapper<T> and std::reference_wrapper<const T> would
invoke cast_op<type>. This doesn't allow the type_caster<> specialization for T
to distinguish reference_wrapper types from value types.
After, the type_caster<> specialization invokes cast_op<type&>, which allows
reference_wrapper to behave in the same way as a native reference type.
* Add tests/examples for std::reference_wrapper<const T>
* Add tests which use mutable/immutable variants
This test is a chimera; it blends the pybind11 casters with a custom
pytype implementation that supports immutable and mutable calls.
In order to detect the immutable/mutable state, the cast_op needs
to propagate it, even through e.g. std::reference<const T>
Note: This is still a work in progress; some things are crashing,
which likely means that I have a refcounting bug or something else
missing.
* Add/finish tests that distinguish const& from &
Fixes the bugs in my custom python type implementation,
demonstrate test that requires const& and reference_wrapper<const T>
being treated differently from Non-const.
* Add passing a const to non-const method.
* Demonstrate non-const conversion of reference_wrapper in tests.
Apply formatting presubmit check.
* Fix build errors from presubmit checks.
* Try and fix a few more CI errors
* More CI fixes.
* More CI fixups.
* Try and get PyPy to work.
* Additional minor fixups. Getting close to CI green.
* More ci fixes?
* fix clang-tidy warnings from presubmit
* fix more clang-tidy warnings
* minor comment and consistency cleanups
* PyDECREF -> Py_DECREF
* copy/move constructors
* Resolve codereview comments
* more review comment fixes
* review comments: remove spurious &
* Make the test fail even when the static_assert is commented out.
This expands the test_freezable_type_caster a bit by:
1/ adding accessors .is_immutable and .addr to compare identity
from python.
2/ Changing the default cast_op of the type_caster<> specialization
to return a non-const value. In normal codepaths this is a reasonable
default.
3/ adding roundtrip variants to exercise the by reference, by pointer
and by reference_wrapper in all call paths. In conjunction with 2/, this
demonstrates the failure case of the existing std::reference_wrpper conversion,
which now loses const in a similar way that happens when using the default cast_op_type<>.
* apply presubmit formatting
* Revert inclusion of test_freezable_type_caster
There's some concern that this test is a bit unwieldly because of the use
of the raw <Python.h> functions. Removing for now.
* Add a test that validates const references propagation.
This test verifies that cast_op may be used to correctly detect
const reference types when used with std::reference_wrapper.
* mend
* Review comments based changes.
1. std::add_lvalue_reference<type> -> type&
2. Simplify the test a little more; we're never returning the ConstRefCaster
type so the class_ definition can be removed.
* formatted files again.
* Move const_ref_caster test to builtin_casters
* Review comments: use cast_op and adjust some comments.
* Simplify ConstRefCasted test
I like this version better as it moves the assertion that matters
back into python.
Pybind11 provides a cast operator between opaque void* pointers on the
C++ side and capsules on the Python side. The py::cast<void *>
expression was not aware of this possibility and incorrectly triggered a
compile-time assertion ("Unable to cast type to reference: value is
local to type caster") that is now fixed.
Pybind11's default conversion to int always produces a long on Python 2 (`int`s and `long`s were unified in Python 3). This patch fixes `int` handling to match Python 2 on Python 2; for short types (`size_t` or smaller), the number will be returned as an `int` if possible, otherwise `long`. Requires Python 2.5+.
This is needed for things like `sys.exit`, which refuse to accept a `long`.
This changes the caster to return a reference to a (new) local `CharT`
type caster member so that binding lvalue-reference char arguments
works (currently it results in a compilation failure).
Fixes#1116
This adds support for implicit conversions to bool from Python types
with `__bool__` (Python 3) or `__nonzero__` (Python 2) attributes, and
adds direct (i.e. non-converting) support for numpy bools.
This updates the std::tuple, std::pair and `stl.h` type casters to
forward their contained value according to whether the container being
cast is an lvalue or rvalue reference. This fixes an issue where
subcaster casts were always called with a const lvalue which meant
nested type casters didn't have the desired `cast()` overload invoked.
For example, this caused Eigen values in a tuple to end up with a
readonly flag (issue #935) and made it impossible to return a container
of move-only types (issue #853).
This fixes both issues by adding templated universal reference `cast()`
methods to the various container types that forward container elements
according to the container reference type.
The std::pair caster can be written as a special case of the std::tuple
caster; this combines them via a base `tuple_caster` class (which is
essentially identical to the previous std::tuple caster).
This also removes the special empty tuple base case: returning an empty
tuple is relatively rare, and the base case still works perfectly well
even when the tuple types is an empty list.
When casting to an unsigned type from a python 2 `int`, we currently
cast using `(unsigned long long) PyLong_AsUnsignedLong(src.ptr())`.
If the Python cast fails, it returns (unsigned long) -1, but then we
cast this to `unsigned long long`, which means we get 4294967295, but
because that isn't equal to `(unsigned long long) -1`, we don't detect
the failure.
This commit moves the unsigned casting into a `detail::as_unsigned`
function which, upon error, casts -1 to the final type, and otherwise
casts the return value to the final type to avoid the problematic double
cast when an error occurs.
The error most commonly shows up wherever `long` is 32-bits (e.g. under
both 32- and 64-bit Windows, and under 32-bit linux) when passing a
negative value to a bound function taking an `unsigned long`.
Fixes#929.
The added tests also trigger a latent segfault under PyPy: when casting
to an integer smaller than `long` (e.g. casting to a `uint32_t` on a
64-bit `long` architecture) we check both for a Python error and also
that the resulting intermediate value will fit in the final type. If
there is no conversion error, but we get a value that would overflow, we
end up calling `PyErr_ExceptionMatches()` illegally: that call is only
allowed when there is a current exception. Under PyPy, this segfaults
the test suite. It doesn't appear to segfault under CPython, but the
documentation suggests that it *could* do so. The fix is to only check
for the exception match if we actually got an error.
