* Add support for array_t<handle> and array_t<object>
* style: pre-commit fixes
* Remove loops that aren't strictly needed
* Fix compiler warning
* Disable GC-dependent checks when running on pypy or graalpy
* style: pre-commit fixes
* Remove PyValueHolder counter again
* Move tests to templates to avoid code duplication
* Rerun pre-commit
* Restore import that was erroneously removed by pre-commit
* Reduce code duplication with more template magic
* Bring back `.attr("value")` in `return_array_cpp_loop()`
This was meant to further stress-test correctness of refcount handling.
All modified test functions were manually leak-checked (`while True`, top command, Python 3.12.3, Ubuntu 24.01, gcc 13.2.0).
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
* Add `npy_format_descriptor<PyObject *>` to enable `py::array_t<PyObject *>` to/from-python conversions.
* resolve clang-tidy warning
* Use existing constructor instead of adding a static method. Thanks @Skylion007 for pointing out.
* Add `format_descriptor<PyObject *>`
Trivial addition, but still in search for a meaningful test.
* Add test_format_descriptor_format
* Ensure the Eigen `type_caster`s do not segfault when loading arrays with dtype=object
* Use `static_assert()` `!std::is_pointer<>` to replace runtime guards.
* Add comments to explain how to check for ref-count bugs. (NO code changes.)
* Make the "Pointer types ... are not supported" message Eigen-specific, as suggested by @Lalaland. Move to new pybind11/eigen/common.h header.
* Change "format_descriptor_format" implementation as suggested by @Lalaland. Additional tests meant to ensure consistency between py::format_descriptor<>, np.array, np.format_parser turn out to be useful only to highlight long-standing inconsistencies.
* resolve clang-tidy warning
* Account for np.float128, np.complex256 not being available on Windows, in a future-proof way.
* Fully address i|q|l ambiguity (hopefully).
* Remove the new `np.format_parser()`-based test, it's much more distracting than useful.
* Use bi.itemsize to disambiguate "l" or "L"
* Use `py::detail::compare_buffer_info<T>::compare()` to validate the `format_descriptor<T>::format()` strings.
* Add `buffer_info::compare<T>` to make `detail::compare_buffer_info<T>::compare` more visible & accessible.
* silence clang-tidy warning
* pytest-compatible access to np.float128, np.complex256
* Revert "pytest-compatible access to np.float128, np.complex256"
This reverts commit e9a289c50f.
* Use `sizeof(long double) == sizeof(double)` instead of `std::is_same<>`
* Report skipped `long double` tests.
* Change the name of the new `buffer_info` member function to `item_type_is_equivalent_to`. Add comment defining "equivalent" by example.
* Change `item_type_is_equivalent_to<>()` from `static` function to member function, as suggested by @Lalaland
* First draft of Eigen::Tensor support
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix build errors
* Weird allocator stuff?
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove unused + additional allocator junk
* Disable warning
* Use constexpr
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* clang tidy fixes
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Resolve comments
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove auto constexpr function
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Try again for older C++
* Oops forgot constexpr
* Move to new files as suggested
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix weird tests
* Fix nits
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Oops, forgot import
* Fix clang 3.6 bug
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* More comprehensive test suite
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Refactor allocators to make things more clear
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Switch to std::copy
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Switch to DSizes instead of array
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address feedback
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix python + dummy c++ change to trigger build
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Alignment
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add include guard
* Forgot inline
* Fix compiler warning
* Remove bad test
* Better type signatures
* Add guards to make compiler requirements more explicit
* style: pre-commit fixes
* Force rerun of tests due to flake
* style: pre-commit fixes
* Keep pragmas & all related comments together, add PLEASE KEEP IN SYNC
* Move headers out of detail
* style: pre-commit fixes
* Fix cmake
* Improve casting
* style: pre-commit fixes
* Add a ton more tests + refactor
* Improve names
* style: pre-commit fixes
* Update include/pybind11/eigen/tensor.h
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
* Fix tests
* style: pre-commit fixes
* Update
* Add a test to verify that strange numpy arrays work
* Fix dumb compiler warning
* Better tests
* Better tests
* Fix tests
* style: pre-commit fixes
* More test fixes
* style: pre-commit fixes
* A ton more test coverage
* Fix tests
* style: pre-commit fixes
* style: pre-commit fixes
* Add back constexpr
* Another test
* style: pre-commit fixes
* Improve tests
* Whoops
* Less magic numbers
* Update tests/test_eigen_tensor.py
Co-authored-by: Sergiu Deitsch <sergiud@users.noreply.github.com>
* Update tests/test_eigen_tensor.py
Co-authored-by: Sergiu Deitsch <sergiud@users.noreply.github.com>
* style: pre-commit fixes
* Fix tests
* style: pre-commit fixes
* Fix memory leak
* style: pre-commit fixes
* Fix order
* style: pre-commit fixes
* Add test to make sure unsafe casts fail
* Minor bug fix to work on 32 bit machines
* Implement convert flag
* style: pre-commit fixes
* Switch to correct TensorMap const use
* style: pre-commit fixes
* Support older versions of eigen
* Weird c++ compilers
* Fix Eigen bug
* Fix another eigen bug
* Yet another eigen bug
* Potential flakes?
* style: pre-commit fixes
* Rerun tests with dummy exception to find out what is going on
* Another dummy test run
* Ablate more
* Found the broken test?
* One step closer
* one step further
* Double check
* one thing at a time
* Give up and disable the test
* Clang lies about being gcc
* Oops, fix matrix test
* style: pre-commit fixes
* Add tests to verify scalar conversions
* style: pre-commit fixes
* Fix nits
* Support no_array
* Fix tests
* style: pre-commit fixes
* Silence compiler warning
* Improve build system for ancient compilers
* Make clang happy
* Make gcc happy
* Implement Skylion's suggestions
* Fix warning
* Inline const pointer check
* Implement suggestions
* style: pre-commit fixes
* Improve tests
* Typo
* style: pre-commit fixes
* Support Google's build environment
* style: pre-commit fixes
* Update include/pybind11/eigen/tensor.h
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
* style: pre-commit fixes
* Test cleanup per Skylion
* Switch to remvove_cv_t
* Cleaner test
* style: pre-commit fixes
* Remove tensor from eigen.h, update tests
* style: pre-commit fixes
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com>
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
Co-authored-by: Sergiu Deitsch <sergiud@users.noreply.github.com>
* Manual line breaks to pre-empt undesired `clang-format`ing.
Informed by work under https://github.com/pybind/pybind11/pull/3683:
60b7eb410f59572e6559
* Manual curation of clang-format diffs involving source code comments.
Very labor-intensive and dull.
* Pulling .clang-format change from @henryiii's 9057962d40
* Adding commonly used .clang-format `CommentPragmas:`
* Ensure short lambdas are allowed
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
* Adding readability-qualified-auto to .clang-tidy
Ported from @henryiii's 287527f705
* fix: support Python < 3.6
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
* Add const T to docstring generation.
* Change order.
* See if existing test triggers for a const type.
* Add tests.
* Fix test.
* Remove experiment.
* Reformat.
* More tests, checks run.
* Adding `test_fmt_desc_` prefix to new test functions.
* Using pytest.mark.parametrize to 1. condense test; 2. exercise all functions even if one fails; 3. be less platform-specific (e.g. C++ float is not necessarily float32).
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
* CI: Intel icc/icpc via oneAPI
Add testing for Intel icc/icpc via the oneAPI images.
Intel oneAPI is in a late beta stage, currently shipping
oneAPI beta09 with ICC 20.2.
* CI: Skip Interpreter Tests for Intel
Cannot find how to add this, neiter the package `libc6-dev` nor
`intel-oneapi-mkl-devel` help when installed to solve this:
```
-- Looking for C++ include pthread.h
-- Looking for C++ include pthread.h - not found
CMake Error at /__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:165 (message):
Could NOT find Threads (missing: Threads_FOUND)
Call Stack (most recent call first):
/__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:458 (_FPHSA_FAILURE_MESSAGE)
/__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindThreads.cmake:234 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
tests/test_embed/CMakeLists.txt:17 (find_package)
```
* CI: libc6-dev from GCC for ICC
* CI: Run bare metal for oneAPI
* CI: Ubuntu 18.04 for oneAPI
* CI: Intel +Catch -Eigen
* CI: CMake from Apt (ICC tests)
* CI: Replace Intel Py with GCC Py
* CI: Intel w/o GCC's Eigen
* CI: ICC with verbose make
* [Debug] Find core dump
* tests: use arg{} instead of arg() for Intel
* tests: adding a few more missing {}
* fix: sync with @tobiasleibner's branch
* fix: try ubuntu 20-04
* fix: drop exit 1
* style: clang tidy fix
* style: fix missing NOLINT
* ICC: Update Compiler Name
Changed upstream with the last oneAPI release.
* ICC CI: Downgrade pytest
pytest 6 does not capture the `discard_as_unraisable` stderr and
just writes a warning with its content instead.
* Use new test pinning requirements.txt
* tests: add notes about intel, cleanup
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
I found that the numpy array tests already contained an empty-shaped
array test, but none with data in it.
Following PEP 3118, scalars have an empty shape and ndim 0. This
works already and is now also documented/covered by a test.
This PR adds a new py::ellipsis() method which can be used in
conjunction with NumPy's generalized slicing support. For instance,
the following is now valid (where "a" is a NumPy array):
py::array b = a[py::make_tuple(0, py::ellipsis(), 0)];
This udpates all the remaining tests to the new test suite code and
comment styles started in #898. For the most part, the test coverage
here is unchanged, with a few minor exceptions as noted below.
- test_constants_and_functions: this adds more overload tests with
overloads with different number of arguments for more comprehensive
overload_cast testing. The test style conversion broke the overload
tests under MSVC 2015, prompting the additional tests while looking
for a workaround.
- test_eigen: this dropped the unused functions `get_cm_corners` and
`get_cm_corners_const`--these same tests were duplicates of the same
things provided (and used) via ReturnTester methods.
- test_opaque_types: this test had a hidden dependence on ExampleMandA
which is now fixed by using the global UserType which suffices for the
relevant test.
- test_methods_and_attributes: this required some additions to UserType
to make it usable as a replacement for the test's previous SimpleType:
UserType gained a value mutator, and the `value` property is not
mutable (it was previously readonly). Some overload tests were also
added to better test overload_cast (as described above).
- test_numpy_array: removed the untemplated mutate_data/mutate_data_t:
the templated versions with an empty parameter pack expand to the same
thing.
- test_stl: this was already mostly in the new style; this just tweaks
things a bit, localizing a class, and adding some missing
`// test_whatever` comments.
- test_virtual_functions: like `test_stl`, this was mostly in the new
test style already, but needed some `// test_whatever` comments.
This commit also moves the inherited virtual example code to the end
of the file, after the main set of tests (since it is less important
than the other tests, and rather length); it also got renamed to
`test_inherited_virtuals` (from `test_inheriting_repeat`) because it
tests both inherited virtual approaches, not just the repeat approach.
We're current copy by creating an Eigen::Map into the input numpy
array, then assigning that to the basic eigen type, effectively having
Eigen do the copy. That doesn't work for negative strides, though:
Eigen doesn't allow them.
This commit makes numpy do the copying instead by allocating the eigen
type, then having numpy copy from the input array into a numpy reference
into the eigen object's data. This also saves a copy when type
conversion is required: numpy can do the conversion on-the-fly as part
of the copy.
Finally this commit also makes non-reference parameters respect the
convert flag, declining the load when called in a noconvert pass with a
convertible, but non-array input or an array with the wrong dtype.
This removes the convert-from-arithemtic-scalar constructor of
any_container as it can result in ambiguous calls, as in:
py::array_t<float>({ 1, 2 })
which could be intepreted as either of:
py::array_t<float>(py::array_t<float>(1, 2))
py::array_t<float>(py::detail::any_container({ 1, 2 }))
Removing the convert-from-arithmetic constructor reduces the number of
implicit conversions, avoiding the ambiguity for array and array_t.
This also re-adds the array/array_t constructors taking a scalar
argument for backwards compatibility.
This adds support for constructing `buffer_info` and `array`s using
arbitrary containers or iterator pairs instead of requiring a vector.
This is primarily needed by PR #782 (which makes strides signed to
properly support negative strides, and will likely also make shape and
itemsize to avoid mixed integer issues), but also needs to preserve
backwards compatibility with 2.1 and earlier which accepts the strides
parameter as a vector of size_t's.
Rather than adding nearly duplicate constructors for each stride-taking
constructor, it seems nicer to simply allow any type of container (or
iterator pairs). This works by replacing the existing vector arguments
with a new `detail::any_container` class that handles implicit
conversion of arbitrary containers into a vector of the desired type.
It can also be explicitly instantiated with a pair of iterators (e.g.
by passing {begin, end} instead of the container).
When attempting to get a raw array pointer we return nullptr if given a
nullptr, which triggers an error_already_set(), but we haven't set an
exception message, which results in "Unknown internal error".
Callers that want explicit allowing of a nullptr here already handle it
(by clearing the exception after the call).
The extends the previous unchecked support with the ability to
determine the dimensions at runtime. This incurs a small performance
hit when used (versus the compile-time fixed alternative), but is still considerably
faster than the full checks on every call that happen with
`.at()`/`.mutable_at()`.
This adds bounds-unchecked access to arrays through a `a.unchecked<Type,
Dimensions>()` method. (For `array_t<T>`, the `Type` template parameter
is omitted). The mutable version (which requires the array have the
`writeable` flag) is available as `a.mutable_unchecked<...>()`.
Specifying the Dimensions as a template parameter allows storage of an
std::array; having the strides and sizes stored that way (as opposed to
storing a copy of the array's strides/shape pointers) allows the
compiler to make significant optimizations of the shape() method that it
can't make with a pointer; testing with nested loops of the form:
for (size_t i0 = 0; i0 < r.shape(0); i0++)
for (size_t i1 = 0; i1 < r.shape(1); i1++)
...
r(i0, i1, ...) += 1;
over a 10 million element array gives around a 25% speedup (versus using
a pointer) for the 1D case, 33% for 2D, and runs more than twice as fast
with a 5D array.
* Add value_type member alias to py::array_t (resolve#632)
* Use numpy scalar name in py::array_t function signatures (e.g. float32/64 instead of just float)
This makes array_t respect overload resolution and noconvert by failing
to load when `convert = false` if the src isn't already an array of the
correct type.
* Make string conversion stricter
The string conversion logic added in PR #624 for all std::basic_strings
was derived from the old std::wstring logic, but that was underused and
turns out to have had a bug in accepting almost anything convertible to
unicode, while the previous std::string logic was much stricter. This
restores the previous std::string logic by only allowing actual unicode
or string types.
Fixes#685.
* Added missing 'requires numpy' decorator
(I forgot that the change to a global decorator here is in the
not-yet-merged Eigen PR)
* `array_t(const object &)` now throws on error
* `array_t::ensure()` is intended for casters —- old constructor is
deprecated
* `array` and `array_t` get default constructors (empty array)
* `array` gets a converting constructor
* `py::isinstance<array_T<T>>()` checks the type (but not flags)
There is only one special thing which must remain: `array_t` gets
its own `type_caster` specialization which uses `ensure` instead
of a simple check.
The current integer caster was unnecessarily strict and rejected
various kinds of NumPy integer types when calling C++ functions
expecting normal integers. This relaxes the current behavior.
This patch adds an extra base handle parameter to most ``py::array`` and
``py::array_t<>`` constructors. If specified along with a pointer to
data, the base object will be registered within NumPy, which increases
the base's reference count. This feature is useful to create shallow
copies of C++ or Python arrays while ensuring that the owners of the
underlying can't be garbage collected while referenced by NumPy.
The commit also adds a simple test function involving a ``wrap()``
function that creates shallow copies of various N-D arrays.
