`function_signature_t` extracts the function type from a function,
function pointer, or lambda.
`is_lambda` (which is really
`is_not_a_function_or_pointer_or_member_pointer`, but that name is a
bit too long) checks whether the type is (in the approprate context) a
lambda.
`is_function_pointer` checks whether the type is a pointer to a
function.
This adds a PYBIND11_NAMESPACE macro that expands to the `pybind11`
namespace with hidden visibility under gcc-type compilers, and otherwise
to the plain `pybind11`. This then forces hidden visibility on
everything in pybind, solving the visibility issues discussed at end
end of #949.
CLion slows to a crawl when evaluating the intricate `PYBIND11_NUMPY_DTYPE`
macro. This commit replaces the macro cascade with a simple `(void)0`
to ease IDE evaluation.
This extends py::vectorize to automatically pass through
non-vectorizable arguments. This removes the need for the documented
"explicitly exclude an argument" workaround.
Vectorization now applies to arithmetic, std::complex, and POD types,
passed as plain value or by const lvalue reference (previously only
pass-by-value types were supported). Non-const lvalue references and
any other types are passed through as-is.
Functions with rvalue reference arguments (whether vectorizable or not)
are explicitly prohibited: an rvalue reference is inherently not
something that can be passed multiple times and is thus unsuitable to
being in a vectorized function.
The vectorize returned value is also now more sensitive to inputs:
previously it would return by value when all inputs are of size 1; this
is now amended to having all inputs of size 1 *and* 0 dimensions. Thus
if you pass in, for example, [[1]], you get back a 1x1, 2D array, while
previously you got back just the resulting single value.
Vectorization of member function specializations is now also supported
via `py::vectorize(&Class::method)`; this required passthrough support
for the initial object pointer on the wrapping function pointer.
Currently if you construct an `array_t<T, array::f_style>` with a shape
but not strides you get a C-style array; the only way to get F-style
strides was to calculate the strides manually. This commit fixes that
by adding logic to use f_style strides when the flag is set.
This also simplifies the existing c_style stride logic.
This changes javadoc-style documenting comments from:
/** Text starts here
* and continues here
*/
to:
/**
* Test starts here
* and continues here
*/
which looks a little better, and also matches the javadoc-recommended
way of writing documenting comments.
This exposed a few underlying issues:
1. is_pod_struct was too strict to allow this. I've relaxed it to
require only trivially copyable and standard layout, rather than POD
(which additionally requires a trivial constructor, which std::complex
violates).
2. format_descriptor<std::complex<T>>::format() returned numpy format
strings instead of PEP3118 format strings, but register_dtype
feeds format codes of its fields to _dtype_from_pep3118. I've changed it
to return PEP3118 format codes. format_descriptor is a public type, so
this may be considered an incompatible change.
3. register_structured_dtype tried to be smart about whether to mark
fields as unaligned (with ^). However, it's examining the C++ alignment,
rather than what numpy (or possibly PEP3118) thinks the alignment should
be. For complex values those are different. I've made it mark all fields
as ^ unconditionally, which should always be safe even if they are
aligned, because we explicitly mark the padding.
Resolves#800.
Both C++ arrays and std::array are supported, including mixtures like
std::array<int, 2>[4]. In a multi-dimensional array of char, the last
dimension is used to construct a numpy string type.
We're current copy by creating an Eigen::Map into the input numpy
array, then assigning that to the basic eigen type, effectively having
Eigen do the copy. That doesn't work for negative strides, though:
Eigen doesn't allow them.
This commit makes numpy do the copying instead by allocating the eigen
type, then having numpy copy from the input array into a numpy reference
into the eigen object's data. This also saves a copy when type
conversion is required: numpy can do the conversion on-the-fly as part
of the copy.
Finally this commit also makes non-reference parameters respect the
convert flag, declining the load when called in a noconvert pass with a
convertible, but non-array input or an array with the wrong dtype.
`EigenConformable::stride_compatible` returns false if the strides are
negative. In this case, do not use `EigenConformable::stride`, as it
is {0,0}. We cannot write negative strides in this element, as Eigen
will throw an assertion if we do.
The `type_caster` specialization for regular, dense Eigen matrices now
does a second `array_t::ensure` to copy data in case of negative strides.
I'm not sure that this is the best way to implement this.
I have added "TODO" tags linking these changes to Eigen bug #747, which,
when fixed, will allow Eigen to accept negative strides.
This removes the convert-from-arithemtic-scalar constructor of
any_container as it can result in ambiguous calls, as in:
py::array_t<float>({ 1, 2 })
which could be intepreted as either of:
py::array_t<float>(py::array_t<float>(1, 2))
py::array_t<float>(py::detail::any_container({ 1, 2 }))
Removing the convert-from-arithmetic constructor reduces the number of
implicit conversions, avoiding the ambiguity for array and array_t.
This also re-adds the array/array_t constructors taking a scalar
argument for backwards compatibility.
The numpy API constants can check past the end of the API array if the
numpy version is too old thus causing a segfault. The current list of
functions requires numpy >= 1.7.0, so this adds a check and exception if
numpy is too old.
The added feature version API element was added in numpy 1.4.0, so this
could still segfault if loaded in 1.3.0 or earlier, but given that
1.4.0 was released at the end of 2009, it seems reasonable enough to
not worry about that case. (1.7.0 was released in early 2013).
This further reduces the constructors required in buffer_info/numpy by
removing the need for the constructors that take a single size_t and
just forward it on via an initializer_list to the container-accepting
constructor.
Unfortunately, in `array` one of the constructors runs into an ambiguity
problem with the deprecated `array(handle, bool)` constructor (because
both the bool constructor and the any_container constructor involve an
implicit conversion, so neither has precedence), so a forwarding
constructor is kept there (until the deprecated constructor is
eventually removed).
This adds support for constructing `buffer_info` and `array`s using
arbitrary containers or iterator pairs instead of requiring a vector.
This is primarily needed by PR #782 (which makes strides signed to
properly support negative strides, and will likely also make shape and
itemsize to avoid mixed integer issues), but also needs to preserve
backwards compatibility with 2.1 and earlier which accepts the strides
parameter as a vector of size_t's.
Rather than adding nearly duplicate constructors for each stride-taking
constructor, it seems nicer to simply allow any type of container (or
iterator pairs). This works by replacing the existing vector arguments
with a new `detail::any_container` class that handles implicit
conversion of arbitrary containers into a vector of the desired type.
It can also be explicitly instantiated with a pair of iterators (e.g.
by passing {begin, end} instead of the container).
When attempting to get a raw array pointer we return nullptr if given a
nullptr, which triggers an error_already_set(), but we haven't set an
exception message, which results in "Unknown internal error".
Callers that want explicit allowing of a nullptr here already handle it
(by clearing the exception after the call).
The constexpr static instances can cause linking failures if the
compiler doesn't optimize away the reference, as reported in #770.
There's no particularly nice way of fixing this in C++11/14: we can't
inline definitions to match the declaration aren't permitted for
non-templated static variables (C++17 *does* allows "inline" on
variables, but that obviously doesn't help us.)
One solution that could work around it is to add an extra inherited
subclass to `object`'s hierarchy, but that's a bit of a messy solution
and was decided against in #771 in favour of just deprecating (and
eventually dropping) the constexpr statics.
Fixes#770.
The extends the previous unchecked support with the ability to
determine the dimensions at runtime. This incurs a small performance
hit when used (versus the compile-time fixed alternative), but is still considerably
faster than the full checks on every call that happen with
`.at()`/`.mutable_at()`.
This adds bounds-unchecked access to arrays through a `a.unchecked<Type,
Dimensions>()` method. (For `array_t<T>`, the `Type` template parameter
is omitted). The mutable version (which requires the array have the
`writeable` flag) is available as `a.mutable_unchecked<...>()`.
Specifying the Dimensions as a template parameter allows storage of an
std::array; having the strides and sizes stored that way (as opposed to
storing a copy of the array's strides/shape pointers) allows the
compiler to make significant optimizations of the shape() method that it
can't make with a pointer; testing with nested loops of the form:
for (size_t i0 = 0; i0 < r.shape(0); i0++)
for (size_t i1 = 0; i1 < r.shape(1); i1++)
...
r(i0, i1, ...) += 1;
over a 10 million element array gives around a 25% speedup (versus using
a pointer) for the 1D case, 33% for 2D, and runs more than twice as fast
with a 5D array.
This extends the trivial handling to support trivial handling for
Fortran-order arrays (i.e. column major): if inputs aren't all
C-contiguous, but *are* all F-contiguous, the resulting array will be
F-contiguous and we can do trivial processing.
For anything else (e.g. C-contiguous, or inputs requiring non-trivial
processing), the result is in (numpy-default) C-contiguous layout.
The only part of the vectorize code that actually needs c-contiguous is
the "trivial" broadcast; for non-trivial arguments, the code already
uses strides properly (and so handles C-style, F-style, neither, slices,
etc.)
This commit rewrites `broadcast` to additionally check for C-contiguous
storage, then takes off the `c_style` flag for the arguments, which
will keep the functionality more or less the same, except for no longer
requiring an array copy for non-c-contiguous input arrays.
Additionally, if we're given a singleton slice (e.g. a[0::4, 0::4] for a
4x4 or smaller array), we no longer fail triviality because the trivial
code path never actually uses the strides on a singleton.
* Add value_type member alias to py::array_t (resolve#632)
* Use numpy scalar name in py::array_t function signatures (e.g. float32/64 instead of just float)
This makes array_t respect overload resolution and noconvert by failing
to load when `convert = false` if the src isn't already an array of the
correct type.
Numpy raises ValueError when attempting to modify an array, while
py::array is raising a RuntimeError. This changes the exception to a
std::domain_error, which gets mapped to the expected ValueError in
python.
numpy arrays aren't currently properly setting base: by setting `->base`
directly, the base doesn't follow what numpy expects and documents (that
is, following chained array bases to the root array).
This fixes the behaviour by using numpy's PyArray_SetBaseObject to set
the base instead, and then updates the tests to reflect the fixed
behaviour.
A few of pybind's numpy constants are using the numpy-deprecated names
(without "ARRAY_" in them); updated our names to be consistent with
current numpy code.
noexcept deduction, added in PR #555, doesn't work with clang's
-std=c++1z; and while it works with g++, it isn't entirely clear to me
that it is required to work in C++17.
What should work, however, is that C++17 allows implicit conversion of a
`noexcept(true)` function pointer to a `noexcept(false)` (i.e. default,
noexcept-not-specified) function pointer. That was breaking in pybind11
because the cpp_function template used for lambdas provided a better
match (i.e. without requiring an implicit conversion), but it then
failed.
This commit takes a different approach of using SFINAE on the lambda
function to prevent it from matching a non-lambda object, which then
gets implicit conversion from a `noexcept` function pointer to a
`noexcept(false)` function pointer. This much nicer solution also gets
rid of the C++17 NOEXCEPT macros, and works in both clang and g++.
* Avoid C-style const casts
Replace C-style casts that discard `const` with `const_cast` (and, where
necessary, `reinterpret_cast` as well).
* Warn about C-style const-discarding casts
Change pybind11_enable_warnings to also enable `-Wcast-qual` (warn if a
C-style cast discards `const`) by default. The previous commit should
have gotten rid of all of these (at least, all the ones that tripped in
my build, which included the tests), and this should discourage more
from newly appearing.
* Clarify PYBIND11_NUMPY_DTYPE documentation
The current documentation and example reads as though
PYBIND11_NUMPY_DTYPE is a declarative macro along the same lines as
PYBIND11_DECLARE_HOLDER_TYPE, but it isn't. The changes the
documentation and docs example to make it clear that you need to "call"
the macro.
* Add satisfies_{all,any,none}_of<T, Preds>
`satisfies_all_of<T, Pred1, Pred2, Pred3>` is a nice legibility-enhanced
shortcut for `is_all<Pred1<T>, Pred2<T>, Pred3<T>>`.
* Give better error message for non-POD dtype attempts
If you try to use a non-POD data type, you get difficult-to-interpret
compilation errors (about ::name() not being a member of an internal
pybind11 struct, among others), for which isn't at all obvious what the
problem is.
This adds a static_assert for such cases.
It also changes the base case from an empty struct to the is_pod_struct
case by no longer using `enable_if<is_pod_struct>` but instead using a
static_assert: thus specializations avoid the base class, POD types
work, and non-POD types (and unimplemented POD types like std::array)
get a more informative static_assert failure.
* Prefix macros with PYBIND11_
numpy.h uses unprefixed macros, which seems undesirable. This prefixes
them with PYBIND11_ to match all the other macros in numpy.h (and
elsewhere).
* Add long double support
This adds long double and std::complex<long double> support for numpy
arrays.
This allows some simplification of the code used to generate format
descriptors; the new code uses fewer macros, instead putting the code as
different templated options; the template conditions end up simpler with
this because we are now supporting all basic C++ arithmetic types (and
so can use is_arithmetic instead of is_integral + multiple
different specializations).
In addition to testing that it is indeed working in the test script, it
also adds various offset and size calculations there, which
fixes the test failures under x86 compilations.
When compiling in C++17 mode the noexcept specifier is part of the
function type. This causes a failure in pybind11 because, by omitting
a noexcept specifier when deducing function return and argument types,
we are implicitly making `noexcept(false)` part of the type.
This means that functions with `noexcept` fail to match the function
templates in cpp_function (and other places), and we get compilation
failure (we end up trying to fit it into the lambda function version,
which fails since a function pointer has no `operator()`).
We can, however, deduce the true/false `B` in noexcept(B), so we don't
need to add a whole other set of overloads, but need to deduce the extra
argument when under C++17. That will *not* work under pre-C++17,
however.
This commit adds two macros to fix the problem: under C++17 (with the
appropriate feature macro set) they provide an extra `bool NoExceptions`
template argument and provide the `noexcept(NoExceptions)` deduced
specifier. Under pre-C++17 they expand to nothing.
This is needed to compile pybind11 with gcc7 under -std=c++17.
Newer standard libraries use compiler intrinsics for std::index_sequence
which makes it ‘free’. This prevents hitting instantiation limits for
recursive templates (-ftemplate-depth).
* `array_t(const object &)` now throws on error
* `array_t::ensure()` is intended for casters —- old constructor is
deprecated
* `array` and `array_t` get default constructors (empty array)
* `array` gets a converting constructor
* `py::isinstance<array_T<T>>()` checks the type (but not flags)
There is only one special thing which must remain: `array_t` gets
its own `type_caster` specialization which uses `ensure` instead
of a simple check.
The pytype converting constructors are convenient and safe for user
code, but for library internals the additional type checks and possible
conversions are sometimes not desired. `reinterpret_borrow<T>()` and
`reinterpret_steal<T>()` serve as the low-level unsafe counterparts
of `cast<T>()`.
This deprecates the `object(handle, bool)` constructor.
Renamed `borrowed` parameter to `is_borrowed` to avoid shadowing
warnings on MSVC.
* Deprecate the `py::object::str()` member function since `py::str(obj)`
is now equivalent and preferred
* Make `py::repr()` a free function
* Make sure obj.cast<T>() works as expected when T is a Python type
`obj.cast<T>()` should be the same as `T(obj)`, i.e. it should convert
the given object to a different Python type. However, `obj.cast<T>()`
usually calls `type_caster::load()` which only checks the type without
doing any actual conversion. That causes a very unexpected `cast_error`.
This commit makes it so that `obj.cast<T>()` and `T(obj)` are the same
when T is a Python type.
* Simplify pytypes converting constructor implementation
It's not necessary to maintain a full set of converting constructors
and assignment operators + const& and &&. A single converting const&
constructor will work and there is no impact on binary size. On the
other hand, the conversion functions can be significantly simplified.
Allows checking the Python types before creating an object instead of
after. For example:
```c++
auto l = list(ptr, true);
if (l.check())
// ...
```
The above is replaced with:
```c++
if (isinstance<list>(ptr)) {
auto l = reinterpret_borrow(ptr);
// ...
}
```
This deprecates `py::object::check()`. `py::isinstance()` covers the
same use case, but it can also check for user-defined types:
```c++
class Pet { ... };
py::class_<Pet>(...);
m.def("is_pet", [](py::object obj) {
return py::isinstance<Pet>(obj); // works as expected
});
```
We have various classes that have non-explicit constructors that accept
a single argument, which is implicitly making them implicitly
convertible from the argument. In a few cases, this is desirable (e.g.
implicit conversion of std::string to py::str, or conversion of double
to py::float_); in many others, however, it is unintended (e.g. implicit
conversion of size_t to some pre-declared py::array_t<T> type).
This disables most of the unwanted implicit conversions by marking them
`explicit`, and comments the ones that are deliberately left implicit.
This convenience function ensures that a py::object is either a
py::array, or the implementation will try to convert it into one. Layout
requirements (such as c_style or f_style) can be also be provided.
This patch adds an extra base handle parameter to most ``py::array`` and
``py::array_t<>`` constructors. If specified along with a pointer to
data, the base object will be registered within NumPy, which increases
the base's reference count. This feature is useful to create shallow
copies of C++ or Python arrays while ensuring that the owners of the
underlying can't be garbage collected while referenced by NumPy.
The commit also adds a simple test function involving a ``wrap()``
function that creates shallow copies of various N-D arrays.
This also adds the `hasattr` and `getattr` functions which are needed
with the new attribute behavior. The new functions behave exactly like
their Python counterparts.
Similarly `object` gets a `contains` method which calls `__contains__`,
i.e. it's the same as the `in` keyword in Python.
This is required since format descriptors for string types that
were using PYBIND11_DESCR were causing problems on C++14 on Linux.
Although this is technically a breaking change, it shouldn't cause
problems since the only use of format strings is passing them to
buffer_info constructor which expects std::string.
Note: for non-structured types, the const char * value is still
accessible via ::value for compatibility purpose.
