Currently, `py::int_(1).cast<variant<double, int>>()` fills the `double`
slot of the variant. This commit switches the loader to a 2-pass scheme
in order to correctly fill the `int` slot.
Many of our `is_none()` checks in type caster loading return true, but
this should really be considered a deferral so that, for example, an
overload with a `py::none` argument would win over one that takes
`py::none` as a null option.
This keeps None-accepting for the `!convert` pass only for std::optional
and void casters. (The `char` caster already deferred None; this just
extends that behaviour to other casters).
Under gcc 7 with -std=c++11, compilation results in several of the
following warnings:
In file included from /home/jagerman/src/pybind11/tests/test_sequences_and_iterators.cpp:13:0:
/home/jagerman/src/pybind11/include/pybind11/operators.h: In function ‘pybind11::detail::op_<(pybind11::detail::op_id)0, (pybind11::detail::op_type)0, pybind11::detail::self_t, pybind11::detail::self_t> pybind11::detail::operator+(const pybind11::detail::self_t&, const pybind11::detail::self_t&)’:
/home/jagerman/src/pybind11/include/pybind11/operators.h:78:76: warning: inline declaration of ‘pybind11::detail::op_<(pybind11::detail::op_id)0, (pybind11::detail::op_type)0, pybind11::detail::self_t, pybind11::detail::self_t> pybind11::detail::operator+(const pybind11::detail::self_t&, const pybind11::detail::self_t&)’ follows declaration with attribute noinline [-Wattributes]
inline op_<op_##id, op_l, self_t, self_t> op(const self_t &, const self_t &) { \
^
/home/jagerman/src/pybind11/include/pybind11/operators.h:109:1: note: in expansion of macro ‘PYBIND11_BINARY_OPERATOR’
PYBIND11_BINARY_OPERATOR(add, radd, operator+, l + r)
^~~~~~~~~~~~~~~~~~~~~~~~
In file included from /home/jagerman/src/pybind11/include/pybind11/cast.h:15:0,
from /home/jagerman/src/pybind11/include/pybind11/attr.h:13,
from /home/jagerman/src/pybind11/include/pybind11/pybind11.h:36,
from /home/jagerman/src/pybind11/tests/pybind11_tests.h:2,
from /home/jagerman/src/pybind11/tests/test_sequences_and_iterators.cpp:11:
/home/jagerman/src/pybind11/include/pybind11/descr.h:116:36: note: previous definition of ‘pybind11::detail::descr pybind11::detail::operator+(pybind11::detail::descr&&, pybind11::detail::descr&&)’ was here
PYBIND11_NOINLINE descr friend operator+(descr &&d1, descr &&d2) {
^~~~~~~~
This appears to be happening because gcc is considering implicit
construction of `descr` in some places using addition of two
`descr`-compatible arguments in the `descr.h` c++11 fallback code.
There's no particular reason that this operator needs to be a friend
function: this commit changes it to an rvalue-context member function
operator, which avoids the warning.
This exposed a few underlying issues:
1. is_pod_struct was too strict to allow this. I've relaxed it to
require only trivially copyable and standard layout, rather than POD
(which additionally requires a trivial constructor, which std::complex
violates).
2. format_descriptor<std::complex<T>>::format() returned numpy format
strings instead of PEP3118 format strings, but register_dtype
feeds format codes of its fields to _dtype_from_pep3118. I've changed it
to return PEP3118 format codes. format_descriptor is a public type, so
this may be considered an incompatible change.
3. register_structured_dtype tried to be smart about whether to mark
fields as unaligned (with ^). However, it's examining the C++ alignment,
rather than what numpy (or possibly PEP3118) thinks the alignment should
be. For complex values those are different. I've made it mark all fields
as ^ unconditionally, which should always be safe even if they are
aligned, because we explicitly mark the padding.
Resolves#800.
Both C++ arrays and std::array are supported, including mixtures like
std::array<int, 2>[4]. In a multi-dimensional array of char, the last
dimension is used to construct a numpy string type.
The PYBIND11_CPP14 macro started out as a guard for the compile-time
path code in `descr.h`, but has since come to mean other things. This
means that while the `descr.h` check has just checked the
`PYBIND11_CPP14` macro, various other places now check `PYBIND11_CPP14
|| _MSC_VER`. This reverses that by now setting the CPP14 macro when
MSVC is trying to support C++14, but disabling the `descr.h` C++14 code
(which still fails under MSVC 2017).
The CPP17 macro also gets enabled when MSVC 2017 is compiling with
/std:c++latest (the default is /std:c++14), which enables
`std::optional` and `std::variant` support under MSVC.
GCC 7 generates (when compiling in C++11/14 mode) warnings such as:
mangled name for ‘pybind11::class_<type_, options>&
pybind11::class_<type_, options>::def(const char*, Func&&, const Extra&
...) [with Func = int (test_exc_sp::C::*)(int) noexcept; Extra = {};
type_ = test_exc_sp::C; options = {}]’ will change in C++17 because the
exception specification is part of a function type [-Wnoexcept-type]
There's nothing we can actually do in the code to avoid this, so just
disable the warning.
GCC supports `deprecated(msg)` since v4.5 and VS supports the standard
[[deprecated(msg)]] since 2015 RTM.
The deprecated constructor change from `= default` to `{}` is
a workaround for a VS2015 bug.
We're current copy by creating an Eigen::Map into the input numpy
array, then assigning that to the basic eigen type, effectively having
Eigen do the copy. That doesn't work for negative strides, though:
Eigen doesn't allow them.
This commit makes numpy do the copying instead by allocating the eigen
type, then having numpy copy from the input array into a numpy reference
into the eigen object's data. This also saves a copy when type
conversion is required: numpy can do the conversion on-the-fly as part
of the copy.
Finally this commit also makes non-reference parameters respect the
convert flag, declining the load when called in a noconvert pass with a
convertible, but non-array input or an array with the wrong dtype.
`EigenConformable::stride_compatible` returns false if the strides are
negative. In this case, do not use `EigenConformable::stride`, as it
is {0,0}. We cannot write negative strides in this element, as Eigen
will throw an assertion if we do.
The `type_caster` specialization for regular, dense Eigen matrices now
does a second `array_t::ensure` to copy data in case of negative strides.
I'm not sure that this is the best way to implement this.
I have added "TODO" tags linking these changes to Eigen bug #747, which,
when fixed, will allow Eigen to accept negative strides.
If a bound std::function is invoked with a bound method, the implicit
bound self is lost because we use `detail::get_function` to unbox the
function. This commit amends the code to use py::function and only
unboxes in the special is-really-a-c-function case. This makes bound
methods stay bound rather than unbinding them by forcing extraction of
the c function.
Enumerations on Python 2.7 were not always implicitly converted to
integers (depending on the target size). This patch adds a __long__
conversion function (only enabled on 2.7) which fixes this issue.
The attached test case fails without this patch.
This removes the convert-from-arithemtic-scalar constructor of
any_container as it can result in ambiguous calls, as in:
py::array_t<float>({ 1, 2 })
which could be intepreted as either of:
py::array_t<float>(py::array_t<float>(1, 2))
py::array_t<float>(py::detail::any_container({ 1, 2 }))
Removing the convert-from-arithmetic constructor reduces the number of
implicit conversions, avoiding the ambiguity for array and array_t.
This also re-adds the array/array_t constructors taking a scalar
argument for backwards compatibility.
Python 3's `PyInstanceMethod_Type` hides itself via its `tp_descr_get`,
which prevents aliasing methods via `cls.attr("m2") = cls.attr("m1")`:
instead the `tp_descr_get` returns a plain function, when called on a
class, or a `PyMethod`, when called on an instance. Override that
behaviour for pybind11 types with a special bypass for
`PyInstanceMethod_Types`.
The Unicode support added in 2.1 (PR #624) inadvertently broke accepting
`bytes` as std::string/char* arguments. This restores it with a
separate path that does a plain conversion (i.e. completely bypassing
all the encoding/decoding code), but only for single-byte string types.
The numpy API constants can check past the end of the API array if the
numpy version is too old thus causing a segfault. The current list of
functions requires numpy >= 1.7.0, so this adds a check and exception if
numpy is too old.
The added feature version API element was added in numpy 1.4.0, so this
could still segfault if loaded in 1.3.0 or earlier, but given that
1.4.0 was released at the end of 2009, it seems reasonable enough to
not worry about that case. (1.7.0 was released in early 2013).
This commits adds base class pointers of offset base classes (i.e. due
to multiple inheritance) to `registered_instances` so that if such a
pointer is returned we properly recognize it as an existing instance.
Without this, returning a base class pointer will cast to the existing
instance if the pointer happens to coincide with the instance pointer,
but constructs a new instance (quite possibly with a segfault, if
ownership is applied) for unequal base class pointers due to multiple
inheritance.
When we are returned a base class pointer (either directly or via
shared_from_this()) we detect its runtime type (using `typeid`), then
end up essentially reinterpret_casting the pointer to the derived type.
This is invalid when the base class pointer was a non-first base, and we
end up with an invalid pointer. We could dynamic_cast to the
most-derived type, but if *that* type isn't pybind11-registered, the
resulting pointer given to the base `cast` implementation isn't necessarily valid
to be reinterpret_cast'ed back to the backup type.
This commit removes the "backup" type argument from the many-argument
`cast(...)` and instead does the derived-or-pointer type decision and
type lookup in type_caster_base, where the dynamic_cast has to be to
correctly get the derived pointer, but also has to do the type lookup to
ensure that we don't pass the wrong (derived) pointer when the backup
type (i.e. the type caster intrinsic type) pointer is needed.
Since the lookup is needed before calling the base cast(), this also
changes the input type to a detail::type_info rather than doing a
(second) lookup in cast().
This breaks up the instance management functions in class_support.h a
little bit so that other pybind11 code can use it. In particular:
- added make_new_instance() which does what pybind11_object_new does,
but also allows instance allocation without `value` allocation. This
lets `cast.h` use the same instance allocation rather than having its
own separate implementation.
- instance registration is now moved to a
`register_instance()`/deregister_instance()` pair (rather than having
individual code add or remove things from `registered_instances`
directory).
- clear_instance() does everything `pybind11_object_dealloc()` needs
except for the deallocation; this is helpful for factory construction
which needs to be able to replace the internals of an instance without
deallocating it.
- clear_instance() now also calls `dealloc` when `holder_constructed`
is true, even if `value` is false. This can happen in factory
construction when the pointer is moved from one instance to another,
but the holder itself is only copied (i.e. for a shared_ptr holder).
I got some unexpected errors from code using `overload_cast` until I
realized that I'd configured the build with -std=c++11.
This commit adds a fake `overload_cast` class in C++11 mode that
triggers a static_assert failure indicating that C++14 is needed.
We currently fail at runtime when trying to call a method that is
overloaded with both static and non-static methods. This is something
python won't allow: the object is either a function or an instance, and
can't be both.
This further reduces the constructors required in buffer_info/numpy by
removing the need for the constructors that take a single size_t and
just forward it on via an initializer_list to the container-accepting
constructor.
Unfortunately, in `array` one of the constructors runs into an ambiguity
problem with the deprecated `array(handle, bool)` constructor (because
both the bool constructor and the any_container constructor involve an
implicit conversion, so neither has precedence), so a forwarding
constructor is kept there (until the deprecated constructor is
eventually removed).
This adds support for constructing `buffer_info` and `array`s using
arbitrary containers or iterator pairs instead of requiring a vector.
This is primarily needed by PR #782 (which makes strides signed to
properly support negative strides, and will likely also make shape and
itemsize to avoid mixed integer issues), but also needs to preserve
backwards compatibility with 2.1 and earlier which accepts the strides
parameter as a vector of size_t's.
Rather than adding nearly duplicate constructors for each stride-taking
constructor, it seems nicer to simply allow any type of container (or
iterator pairs). This works by replacing the existing vector arguments
with a new `detail::any_container` class that handles implicit
conversion of arbitrary containers into a vector of the desired type.
It can also be explicitly instantiated with a pair of iterators (e.g.
by passing {begin, end} instead of the container).
Upcoming changes to buffer_info make it need some things declared in
common.h; it also feels a bit misplaced in common.h (which is arguably
too large already), so move it out. (Separating this and the subsequent
changes into separate commits to make the changes easier to distinguish
from the move.)
When attempting to get a raw array pointer we return nullptr if given a
nullptr, which triggers an error_already_set(), but we haven't set an
exception message, which results in "Unknown internal error".
Callers that want explicit allowing of a nullptr here already handle it
(by clearing the exception after the call).
Many of the Eigen type casters' name() methods weren't wrapping the type
description in a `type_descr` object, which thus wasn't adding the
"{...}" annotation used to identify an argument which broke the help
output by skipping eigen arguments.
The test code I had added even had some (unnoticed) broken output (with
the "arg0: " showing up in the return value).
This commit also adds test code to ensure that named eigen arguments
actually work properly, despite the invalid help output. (The added
tests pass without the rest of this commit).
The holder casters assume but don't check that a `holder<type>`'s `type`
is really a `type_caster_base<type>`; this adds a static_assert to make
sure this is really the case, to turn things like
`std::shared_ptr<array>` into a compilation failure.
Fixes#785
Fixes#775.
Assignments of the form `Type.static_prop = value` should be translated to
`Type.static_prop.__set__(value)` except when `isinstance(value, static_prop)`.
PR #771 deprecated them as they can cause linking failures (#770), but
the deprecation tags cause warnings on GCC 5.x through 6.2.x. Removing
them entirely will break backwards-compatibility consequences, but the
effects should be minimal (only code that was inheriting from `object`
could get at them at all as they are protected).
Fixes#777
When make_tuple fails (for example, when print() is called with a
non-convertible argument, as in #778) the error message a less helpful
than it could be:
make_tuple(): unable to convert arguments of types 'std::tuple<type1, type2>' to Python object
There is no actual std::tuple involved (only a parameter pack and a
Python tuple), but it also doesn't immediately reveal which type caused
the problem.
This commit changes the debugging mode output to show just the
problematic type:
make_tuple(): unable to convert argument of type 'type2' to Python object
This commit adds `error_already_set::matches()` convenience method to
check if the exception trapped by `error_already_set` matches a given
Python exception type. This will address #700 by providing a less
verbose way to check exceptions.
The constexpr static instances can cause linking failures if the
compiler doesn't optimize away the reference, as reported in #770.
There's no particularly nice way of fixing this in C++11/14: we can't
inline definitions to match the declaration aren't permitted for
non-templated static variables (C++17 *does* allows "inline" on
variables, but that obviously doesn't help us.)
One solution that could work around it is to add an extra inherited
subclass to `object`'s hierarchy, but that's a bit of a messy solution
and was decided against in #771 in favour of just deprecating (and
eventually dropping) the constexpr statics.
Fixes#770.
The extends the previous unchecked support with the ability to
determine the dimensions at runtime. This incurs a small performance
hit when used (versus the compile-time fixed alternative), but is still considerably
faster than the full checks on every call that happen with
`.at()`/`.mutable_at()`.
This adds bounds-unchecked access to arrays through a `a.unchecked<Type,
Dimensions>()` method. (For `array_t<T>`, the `Type` template parameter
is omitted). The mutable version (which requires the array have the
`writeable` flag) is available as `a.mutable_unchecked<...>()`.
Specifying the Dimensions as a template parameter allows storage of an
std::array; having the strides and sizes stored that way (as opposed to
storing a copy of the array's strides/shape pointers) allows the
compiler to make significant optimizations of the shape() method that it
can't make with a pointer; testing with nested loops of the form:
for (size_t i0 = 0; i0 < r.shape(0); i0++)
for (size_t i1 = 0; i1 < r.shape(1); i1++)
...
r(i0, i1, ...) += 1;
over a 10 million element array gives around a 25% speedup (versus using
a pointer) for the 1D case, 33% for 2D, and runs more than twice as fast
with a 5D array.
This extends the trivial handling to support trivial handling for
Fortran-order arrays (i.e. column major): if inputs aren't all
C-contiguous, but *are* all F-contiguous, the resulting array will be
F-contiguous and we can do trivial processing.
For anything else (e.g. C-contiguous, or inputs requiring non-trivial
processing), the result is in (numpy-default) C-contiguous layout.
The only part of the vectorize code that actually needs c-contiguous is
the "trivial" broadcast; for non-trivial arguments, the code already
uses strides properly (and so handles C-style, F-style, neither, slices,
etc.)
This commit rewrites `broadcast` to additionally check for C-contiguous
storage, then takes off the `c_style` flag for the arguments, which
will keep the functionality more or less the same, except for no longer
requiring an array copy for non-c-contiguous input arrays.
Additionally, if we're given a singleton slice (e.g. a[0::4, 0::4] for a
4x4 or smaller array), we no longer fail triviality because the trivial
code path never actually uses the strides on a singleton.
Instead of a segfault. Fixes#751.
This covers the case of loading a custom holder from a default-holder
instance. Attempting to load one custom holder from a different custom
holder (i.e. not `std::unique_ptr`) yields undefined behavior, just as
#588 established for inheritance.
py::arg() doesn't only specify named arguments anymore, so the error
message was misleading (e.g. when using `py::arg().noconvert()` and
forgetting `py::arg()` for a second positional argument).
We now require (and enforce at compile time):
- GCC 4.8+
- clang 3.3+ (5.0+ for Apple's renumbered clang)
- MSVC 2015u3+
- ICC 15+
This also updates the versions listed in the README, and removes a
now-redundant MSVC version check.
This adds brief API documentation for make_iterator/make_key_iterator,
specifically mentioning that it requires InputIterators.
Closes#734.
[skip ci] (no code change here)
We can't support this for classes from imported modules (which is the
primary purpose of a ctor argument base class) because we *have* to
have both parent and derived to properly extract a multiple-inheritance
base class pointer from a derived class pointer.
We could support this for actual `class_<Base, ...> instances, but since
in that case the `Base` is already present in the code, it seems more
consistent to simply always require MI to go via template options.
This puts the fold expressions behind the feature macro instead of a
general C++17 macro.
It also adds a fold expression optimization to constexpr_sum (guarded
by the same feature macro).
Fixes#738
The current check for conformability fails when given a 2D, 1xN or Nx1
input to a row-major or column-major, respectively, Eigen::Ref, leading
to a copy-required state in the type_caster, but this later failed
because the copy was also non-conformable because it had the same shape
and strides (because a 1xN or Nx1 is both F and C contiguous).
In such cases we can safely ignore the stride on the "1" dimension since
it'll never be used: only the "N" dimension stride needs to match the
Eigen::Ref stride, which both fixes the non-conformable copy problem,
but also avoids a copy entirely as long as the "N" dimension has a
compatible stride.
Allows use of vectors as python buffers, so for example they can be adopted without a copy by numpy.asarray
Allows faster conversion of buffers to vectors by copying instead of individually casting the elements
* Add value_type member alias to py::array_t (resolve#632)
* Use numpy scalar name in py::array_t function signatures (e.g. float32/64 instead of just float)
The `decltype(...)` in the template parameter that gives us SFINAE
matching for a lambda makes MSVC 2017 ICE; this works around if by
changing the test to an explicit not-a-function-or-pointer test, which
seems to work everywhere.
Some versions of Python 2.7 reportedly (#713) have issues with
PyUnicode_Decode being passed the encoding string, so just skip it
entirely by calling the PyUnicode_DecodeUTF* function directly. This
will also be slightly more efficient by avoiding having to check the
encoding string, and (for python 2) going through the unicode class's
decode (python 3 fast-tracks this for all utf-{8,16,32} encodings;
python 2 only fast-tracked for the exact string "utf-8", which we
weren't passing anyway (we had "utf8")).
This doesn't work for PyPy, however: its `PyUnicode_DecodeUTF{8,16,32}`
appear rather broken: the UTF8 one segfaults, while the 16/32 require
recasting into a non-const `char *` (and might segfault; I didn't get
far enough to find out). Just avoid the whole thing by keeping the
encoding-passed-as-string version for PyPy, which seems to work
reliably.
The duration calculation was using %, but that's only supported on
duration objects when the arithmetic type supports %, and hence fails
for floats. Fixed by subtracting off the calculated values instead.
When using pybind::options to disable function signatures, user-defined
docstrings only get appended if they exist, but newlines were getting
appended unconditionally, so the docstring could end up with blank lines
(depending on which overloads, in particular, provided docstrings).
This commit suppresses the empty lines by only adding newlines for
overloads when needed.
This makes array_t respect overload resolution and noconvert by failing
to load when `convert = false` if the src isn't already an array of the
correct type.
Added in 6fb48490ef
The second constructor can't be doing anything--the signatures are
exactly the same, and so the first is always going to be the one
invoked by the dispatcher.
Commit 11a337f1 added major and minor python version
checking to cast.h but does not use the macros defined
via the Python.h inclusion. This may be due to an
intention to use the variables defined by the cmake
module FindPythonInterpreter, but nothing in the
pybind11 repo does anything to convert the cmake
variables to preprocessor defines.
* The definition of `PySequence_Fast` is more restrictive on PyPy, so
use the slow path instead.
* `PyDict_Next` has been fixed in PyPy -> remove workaround.
Before this, `py::iterator` didn't do any error handling, so code like:
```c++
for (auto item : py::int_(1)) {
// ...
}
```
would just silently skip the loop. The above now throws `TypeError` as
expected. This is a breaking behavior change, but any code which relied
on the silent skip was probably broken anyway.
Also, errors returned by `PyIter_Next()` are now properly handled.
This commit largely rewrites the Eigen dense matrix support to avoid
copying in many cases: Eigen arguments can now reference numpy data, and
numpy objects can now reference Eigen data (given compatible types).
Eigen::Ref<...> arguments now also make use of the new `convert`
argument use (added in PR #634) to avoid conversion, allowing
`py::arg().noconvert()` to be used when binding a function to prohibit
copying when invoking the function. Respecting `convert` also means
Eigen overloads that avoid copying will be preferred during overload
resolution to ones that require copying.
This commit also rewrites the Eigen documentation and test suite to
explain and test the new capabilities.
Eigen::Ref objects, when returned, are almost always returned as
rvalues; what's important is the data they reference, not the outer
shell, and so we want to be able to use `::copy`,
`::reference_internal`, etc. to refer to the data the Eigen::Ref
references (in the following commits), rather than the Eigen::Ref
instance itself.
This moves the policy override into a struct so that code that wants to
avoid it (or wants to provide some other Return-type-conditional
override) can create a specialization of
return_value_policy_override<Return> in order to override the override.
This lets an Eigen::Ref-returning function be bound with `rvp::copy`,
for example, to specify that the data should be copied into a new numpy
array rather than referenced, or `rvp::reference_internal` to indicate
that it should be referenced, but a keep-alive used (actually, we used
the array's `base` rather than a py::keep_alive in such a case, but it
accomplishes the same thing).
Numpy raises ValueError when attempting to modify an array, while
py::array is raising a RuntimeError. This changes the exception to a
std::domain_error, which gets mapped to the expected ValueError in
python.
numpy arrays aren't currently properly setting base: by setting `->base`
directly, the base doesn't follow what numpy expects and documents (that
is, following chained array bases to the root array).
This fixes the behaviour by using numpy's PyArray_SetBaseObject to set
the base instead, and then updates the tests to reflect the fixed
behaviour.
A few of pybind's numpy constants are using the numpy-deprecated names
(without "ARRAY_" in them); updated our names to be consistent with
current numpy code.
`is_template_base_of<T>` fails when `T` is `const` (because its
implementation relies on being able to convert a `T*` to a `Base<U>*`,
which won't work when `T` is const).
(This also agrees with std::is_base_of, which ignores cv qualification.)
Currently when we do a conversion between a numpy array and an Eigen
Vector, we allow the conversion only if the Eigen type is a
compile-time vector (i.e. at least one dimension is fixed at 1 at
compile time), or if the type is dynamic on *both* dimensions.
This means we can run into cases where MatrixXd allow things that
conforming, compile-time sizes does not: for example,
`Matrix<double,4,Dynamic>` is currently not allowed, even when assigning
from a 4-element vector, but it *is* allowed for a
`Matrix<double,Dynamic,Dynamic>`.
This commit also reverts the current behaviour of using the matrix's
storage order to determine the structure when the Matrix is fully
dynamic (i.e. in both dimensions). Currently we assign to an eigen row
if the storage order is row-major, and column otherwise: this seems
wrong (the storage order has nothing to do with the shape!). While
numpy doesn't distinguish between a row/column vector, Eigen does, but
it makes more sense to consistently choose one than to produce
something with a different shape based on the intended storage layout.
With the previous commit, output can be very confusing because you only
see positional arguments in the "invoked with" line, but you can have a
failure from kwargs as well (in particular, when a value is invalidly
specified via both via positional and kwargs). This commits adds
kwargs to the output, and updates the associated tests to match.
* Make string conversion stricter
The string conversion logic added in PR #624 for all std::basic_strings
was derived from the old std::wstring logic, but that was underused and
turns out to have had a bug in accepting almost anything convertible to
unicode, while the previous std::string logic was much stricter. This
restores the previous std::string logic by only allowing actual unicode
or string types.
Fixes#685.
* Added missing 'requires numpy' decorator
(I forgot that the change to a global decorator here is in the
not-yet-merged Eigen PR)
Now that only one shared metaclass is ever allocated, it's extremely
cheap to enable it for all pybind11 types.
* Deprecate the default py::metaclass() since it's not needed anymore.
* Allow users to specify a custom metaclass via py::metaclass(handle).
In order to fully satisfy Python's inheritance type layout requirements,
all types should have a common 'solid' base. A solid base is one which
has the same instance size as the derived type (not counting the space
required for the optional `dict_ptr` and `weakrefs_ptr`). Thus, `object`
does not qualify as a solid base for pybind11 types and this can lead to
issues with multiple inheritance.
To get around this, new base types are created: one per unique instance
size. There is going to be very few of these bases. They ensure Python's
MRO checks will pass when multiple bases are involved.
Instead of creating a new unique metaclass for each type, the builtin
`property` type is subclassed to support static properties. The new
setter/getters always pass types instead of instances in their `self`
argument. A metaclass is still required to support this behavior, but
it doesn't store any data anymore, so a new one doesn't need to be
created for each class. There is now only one common metaclass which
is shared by all pybind11 types.
* Fixed compilation error when defining function accepting some forms of std::function.
The compilation error happens only when the functional.h header is
present, and the build is done in debug mode, with NDEBUG being
undefined. In addition, the std::function must accept an abstract
base class by reference.
The compilation error occurred in cast.h, when trying to construct a
std::tuple<AbstractBase>, rather than a std::tuple<AbstractBase&>.
This was caused by functional.h using std::move rather than
std::forward, changing the signature of the function being used.
This commit contains the fix, along with a test that exhibits the
issue when compiled in debug mode without the fix applied.
* Moved new std::function tests into test_callbacks, added callback_with_movable test.
noexcept deduction, added in PR #555, doesn't work with clang's
-std=c++1z; and while it works with g++, it isn't entirely clear to me
that it is required to work in C++17.
What should work, however, is that C++17 allows implicit conversion of a
`noexcept(true)` function pointer to a `noexcept(false)` (i.e. default,
noexcept-not-specified) function pointer. That was breaking in pybind11
because the cpp_function template used for lambdas provided a better
match (i.e. without requiring an implicit conversion), but it then
failed.
This commit takes a different approach of using SFINAE on the lambda
function to prevent it from matching a non-lambda object, which then
gets implicit conversion from a `noexcept` function pointer to a
`noexcept(false)` function pointer. This much nicer solution also gets
rid of the C++17 NOEXCEPT macros, and works in both clang and g++.
* Propagate unicode conversion failure
If returning a std::string with invalid utf-8 data, we currently fail
with an uninformative TypeError instead of propagating the
UnicodeDecodeError that Python sets on failure.
* Add support for u16/u32strings and literals
This adds support for wchar{16,32}_t character literals and the
associated std::u{16,32}string types. It also folds the
character/string conversion into a single type_caster template, since
the type casters for string and wstring were mostly the same anyway.
* Added too-long and too-big character conversion errors
With this commit, when casting to a single character, as opposed to a
C-style string, we make sure the input wasn't a multi-character string
or a single character with codepoint too large for the character type.
This also changes the character cast op to CharT instead of CharT& (we
need to be able to return a temporary decoded char value, but also
because there's little gained by bothering with an lvalue return here).
Finally it changes the char caster to 'has-a-string-caster' instead of
'is-a-string-caster' because, with the cast_op change above, there's
nothing at all gained from inheritance. This also lets us remove the
`success` from the string caster (which was only there for the char
caster) into the char caster itself. (I also renamed it to 'none' and
inverted its value to better reflect its purpose). The None -> nullptr
loading also now takes place only under a `convert = true` load pass.
Although it's unlikely that a function taking a char also has overloads
that can take a None, it seems marginally more correct to treat it as a
conversion.
This commit simplifies the size assumptions about character sizes with
static_asserts to back them up.
* Avoid C-style const casts
Replace C-style casts that discard `const` with `const_cast` (and, where
necessary, `reinterpret_cast` as well).
* Warn about C-style const-discarding casts
Change pybind11_enable_warnings to also enable `-Wcast-qual` (warn if a
C-style cast discards `const`) by default. The previous commit should
have gotten rid of all of these (at least, all the ones that tripped in
my build, which included the tests), and this should discourage more
from newly appearing.
Fixes#656.
Before this commit, the problematic sequence was:
1. `catch (const std::exception &e)` gets a Python exception,
i.e. `error_already_set`.
2. `PyErr_SetString(PyExc_ImportError, e.what())` sets an `ImportError`.
3. `~error_already_set()` now runs, but `gil_scoped_acquire` fails due
to an unhandled `ImportError` (which was just set in step 2).
This commit adds a separate catch block for Python exceptions which just
clears the Python error state a little earlier and replaces it with an
`ImportError`, thus making sure that there is only a single Python
exception in flight at a time. (After step 2 in the sequence above,
there were effectively two Python expections set.)
* Fix debugging output for nameless py::arg annotations
This fixes a couple bugs with nameless py::arg() (introduced in #634)
annotations:
- the argument name was being used in debug mode without checking that
it exists (which would result in the std::string construction throwing
an exception for being invoked with a nullptr)
- the error output says "keyword arguments", but py::arg_v() can now
also be used for positional argument defaults.
- the debugging output "in function named 'blah'" was overly verbose:
changed it to just "in function 'blah'".
* Fix missing space in debug test string
* Moved tests from issues to methods_and_attributes
This changes the function dispatching code for overloaded functions into
a two-pass procedure where we first try all overloads with
`convert=false` for all arguments. If no function calls succeeds in the
first pass, we then try a second pass where we allow arguments to have
`convert=true` (unless, of course, the argument was explicitly specified
with `py::arg().noconvert()`).
For non-overloaded methods, the two-pass procedure is skipped (we just
make the overload-allowed call). The second pass is also skipped if it
would result in the same thing (i.e. where all arguments are
`.noconvert()` arguments).
This adds support for controlling the `convert` flag of arguments
through the py::arg annotation. This then allows arguments to be
flagged as non-converting, which the type_caster is able to use to
request different behaviour.
Currently, AFAICS `convert` is only used for type converters of regular
pybind11-registered types; all of the other core type_casters ignore it.
We can, however, repurpose it to control internal conversion of
converters like Eigen and `array`: most usefully to give callers a way
to disable the conversion that would otherwise occur when a
`Eigen::Ref<const Eigen::Matrix>` argument is passed a numpy array that
requires conversion (either because it has an incompatible stride or the
wrong dtype).
Specifying a noconvert looks like one of these:
m.def("f1", &f, "a"_a.noconvert() = "default"); // Named, default, noconvert
m.def("f2", &f, "a"_a.noconvert()); // Named, no default, no converting
m.def("f3", &f, py::arg().noconvert()); // Unnamed, no default, no converting
(The last part--being able to declare a py::arg without a name--is new:
previous py::arg() only accepted named keyword arguments).
Such an non-convert argument is then passed `convert = false` by the
type caster when loading the argument. Whether this has an effect is up
to the type caster itself, but as mentioned above, this would be
extremely helpful for the Eigen support to give a nicer way to specify
a "no-copy" mode than the custom wrapper in the current PR, and
moreover isn't an Eigen-specific hack.
Arithmetic and complex casters now only do a converting cast when
`convert=true`; previously they would convert always (e.g. when passing
an int to a float-accepting function, or a float to complex-accepting
function).
This cleans up the previous commit slightly by further reducing the
function call arguments to a single struct (containing the
function_record, arguments vector, and parent).
Although this doesn't currently change anything, it does allow for
future functionality to have a place for precalls to store temporary
objects that need to be destroyed after a function call (whether or not
the call succeeds).
As a concrete example, with this change #625 could be easily implemented
(I think) by adding a std::unique_ptr<gil_scoped_release> member to the
`function_call` struct with a precall that actually constructs it.
Without this, the precall can't do that: the postcall won't be invoked
if the call throws an exception.
This doesn't seems to affect the .so size noticeably (either way).
This commit rewrites the function dispatcher code to support mixing
regular arguments with py::args/py::kwargs arguments. It also
simplifies the argument loader noticeably as it no longer has to worry
about args/kwargs: all of that is now sorted out in the dispatcher,
which now simply appends a tuple/dict if the function takes
py::args/py::kwargs, then passes all the arguments in a vector.
When the argument loader hit a py::args or py::kwargs, it doesn't do
anything special: it just calls the appropriate type_caster just like it
does for any other argument (thus removing the previous special cases
for args/kwargs).
Switching to passing arguments in a single std::vector instead of a pair
of tuples also makes things simpler, both in the dispatch and the
argument_loader: since this argument list is strictly pybind-internal
(i.e. it never goes to Python) we have no particular reason to use a
Python tuple here.
Some (intentional) restrictions:
- you may not bind a function that has args/kwargs somewhere other than
the end (this somewhat matches Python, and keeps the dispatch code a
little cleaner by being able to not worry about where to inject the
args/kwargs in the argument list).
- If you specify an argument both positionally and via a keyword
argument, you get a TypeError alerting you to this (as you do in
Python).
* Abstract away some holder functionality (resolve#585)
Custom holder types which don't have `.get()` can select the correct
function to call by specializing `holder_traits`.
* Add support for move-only holders (fix#605)
* Clarify PYBIND11_NUMPY_DTYPE documentation
The current documentation and example reads as though
PYBIND11_NUMPY_DTYPE is a declarative macro along the same lines as
PYBIND11_DECLARE_HOLDER_TYPE, but it isn't. The changes the
documentation and docs example to make it clear that you need to "call"
the macro.
* Add satisfies_{all,any,none}_of<T, Preds>
`satisfies_all_of<T, Pred1, Pred2, Pred3>` is a nice legibility-enhanced
shortcut for `is_all<Pred1<T>, Pred2<T>, Pred3<T>>`.
* Give better error message for non-POD dtype attempts
If you try to use a non-POD data type, you get difficult-to-interpret
compilation errors (about ::name() not being a member of an internal
pybind11 struct, among others), for which isn't at all obvious what the
problem is.
This adds a static_assert for such cases.
It also changes the base case from an empty struct to the is_pod_struct
case by no longer using `enable_if<is_pod_struct>` but instead using a
static_assert: thus specializations avoid the base class, POD types
work, and non-POD types (and unimplemented POD types like std::array)
get a more informative static_assert failure.
* Prefix macros with PYBIND11_
numpy.h uses unprefixed macros, which seems undesirable. This prefixes
them with PYBIND11_ to match all the other macros in numpy.h (and
elsewhere).
* Add long double support
This adds long double and std::complex<long double> support for numpy
arrays.
This allows some simplification of the code used to generate format
descriptors; the new code uses fewer macros, instead putting the code as
different templated options; the template conditions end up simpler with
this because we are now supporting all basic C++ arithmetic types (and
so can use is_arithmetic instead of is_integral + multiple
different specializations).
In addition to testing that it is indeed working in the test script, it
also adds various offset and size calculations there, which
fixes the test failures under x86 compilations.
* Make 'any' the default markup role for Sphinx docs
* Automate generation of reference docs with doxygen and breathe
* Improve reference docs coverage
* Fixed a regression that was introduced in the PyPy patch: use ht_qualname_meta instead of ht_qualname to fix PyHeapTypeObject->ht_qualname field.
* Added a qualname/repr test that works in both Python 3.3+ and previous versions
This commit includes modifications that are needed to get pybind11 to work with PyPy. The full test suite compiles and runs except for a last few functions that are commented out (due to problems in PyPy that were reported on the PyPy bugtracker).
Two somewhat intrusive changes were needed to make it possible: two new tags ``py::buffer_protocol()`` and ``py::metaclass()`` must now be specified to the ``class_`` constructor if the class uses the buffer protocol and/or requires a metaclass (e.g. for static properties).
Note that this is only for the PyPy version based on Python 2.7 for now. When the PyPy 3.x has caught up in terms of cpyext compliance, a PyPy 3.x patch will follow.
This replaces the current `all_of_t<Pred, Ts...>` with `all_of<Ts...>`,
with previous use of `all_of_t<Pred, Ts...>` becoming
`all_of<Pred<Ts>...>` (and similarly for `any_of_t`). It also adds a
`none_of<Ts...>`, a shortcut for `negation<any_of<Ts...>>`.
This allows `all_of` and `any_of` to be used a bit more flexible, e.g.
in cases where several predicates need to be tested for the same type
instead of the same predicate for multiple types.
This commit replaces the implementation with a more efficient version
for non-MSVC. For MSVC, this changes the workaround to use the
built-in, recursive std::conjunction/std::disjunction instead.
This also removes the `count_t` since `any_of_t` and `all_of_t` were the
only things using it.
This commit also rearranges some of the future std imports to use actual
`std` implementations for C++14/17 features when under the appropriate
compiler mode, as we were already doing for a few things (like
index_sequence). Most of these aren't saving much (the implementation
for enable_if_t, for example, is trivial), but I think it makes the
intention of the code instantly clear. It also enables MSVC's native
std::index_sequence support.
When compiling in C++17 mode the noexcept specifier is part of the
function type. This causes a failure in pybind11 because, by omitting
a noexcept specifier when deducing function return and argument types,
we are implicitly making `noexcept(false)` part of the type.
This means that functions with `noexcept` fail to match the function
templates in cpp_function (and other places), and we get compilation
failure (we end up trying to fit it into the lambda function version,
which fails since a function pointer has no `operator()`).
We can, however, deduce the true/false `B` in noexcept(B), so we don't
need to add a whole other set of overloads, but need to deduce the extra
argument when under C++17. That will *not* work under pre-C++17,
however.
This commit adds two macros to fix the problem: under C++17 (with the
appropriate feature macro set) they provide an extra `bool NoExceptions`
template argument and provide the `noexcept(NoExceptions)` deduced
specifier. Under pre-C++17 they expand to nothing.
This is needed to compile pybind11 with gcc7 under -std=c++17.
Since the argument loader split off from the tuple converter, it is
never called with a `convert` argument set to anything but true. This
removes the argument entirely, passing a literal `true` from within
`argument_loader` to the individual value casters.
This adds automatic casting when assigning to python types like dict,
list, and attributes. Instead of:
dict["key"] = py::cast(val);
m.attr("foo") = py::cast(true);
list.append(py::cast(42));
you can now simply write:
dict["key"] = val;
m.attr("foo") = true;
list.append(42);
Casts needing extra parameters (e.g. for a non-default rvp) still
require the py::cast() call. set::add() is also supported.
All usage is channeled through a SFINAE implementation which either just returns or casts.
Combined non-converting handle and autocasting template methods via a
helper method that either just returns (handle) or casts (C++ type).
* Added ternary support with descr args
Current the `_<bool>(a, b)` ternary support only works for `char[]` `a`
and `b`; this commit allows it to work for `descr` `a` and `b` arguments
as well.
* Add support for std::valarray to stl.h
This abstracts the std::array into a `array_caster` which can then be
used with either std::array or std::valarray, the main difference being
that std::valarray is resizable. (It also lets the array_caster be
potentially used for other std::array-like interfaces, much as the
list_caster and map_caster currently provide).
* Small stl.h cleanups
- Remove redundant `type` typedefs
- make internal list_caster methods private
Newer standard libraries use compiler intrinsics for std::index_sequence
which makes it ‘free’. This prevents hitting instantiation limits for
recursive templates (-ftemplate-depth).
This is needed in order to allow the tuple caster to accept any sequence
while keeping the argument loader fast. There is also very little overlap
between the two classes which makes the separation clean. It’s also good
practice not to have completely new functionality in a specialization.
Using a complicated declval here was pointlessly complicated: we
already know the type, because that's what cast_op_type<T> is in the
first place. (The declval also broke MSVC).
This adds a `detail::cast_op<T>(caster)` function which handles the
rather verbose:
caster.operator typename CasterType::template cast_op_type<T>()
which allows various places to use the shorter and clearer:
cast_op<T>(caster)
instead of the full verbose cast operator invocation.
stl casters were using a value cast to (Value) or (Key), but that isn't
always appropriate. This changes it to use the appropriate value
converter's cast_op_type.
C++ exceptions are destructed in the context of the code that catches
them. At this point, the Python GIL may not be held, which could lead
to crashes with the previous implementation.
PyErr_Fetch and PyErr_Restore should always occur in pairs, which was
not the case for the previous implementation. To clear the exception,
the new approach uses PyErr_Restore && PyErr_Clear instead of simply
decreasing the reference counts of the exception objects.
Fixes#509.
The move policy was already set for rvalues in PR #473, but this only
applied to directly cast user-defined types. The problem is that STL
containers cast values indirectly and the rvalue information is lost.
Therefore the move policy was not set correctly. This commit fixes it.
This also makes an additional adjustment to remove the `copy` policy
exception: rvalues now always use the `move` policy. This is also safe
for copy-only rvalues because the `move` policy has an internal fallback
to copying.
Following commit 90d278, the object code generated by the python
bindings of nanogui (github.com/wjakob/nanogui) went up by a whopping
12%. It turns out that that project has quite a few enums where we don't
really care about arithmetic operators.
This commit thus partially reverts the effects of #503 by introducing
an additional attribute py::arithmetic() that must be specified if the
arithmetic operators are desired.
* `array_t(const object &)` now throws on error
* `array_t::ensure()` is intended for casters —- old constructor is
deprecated
* `array` and `array_t` get default constructors (empty array)
* `array` gets a converting constructor
* `py::isinstance<array_T<T>>()` checks the type (but not flags)
There is only one special thing which must remain: `array_t` gets
its own `type_caster` specialization which uses `ensure` instead
of a simple check.
The pytype converting constructors are convenient and safe for user
code, but for library internals the additional type checks and possible
conversions are sometimes not desired. `reinterpret_borrow<T>()` and
`reinterpret_steal<T>()` serve as the low-level unsafe counterparts
of `cast<T>()`.
This deprecates the `object(handle, bool)` constructor.
Renamed `borrowed` parameter to `is_borrowed` to avoid shadowing
warnings on MSVC.
* Deprecate the `py::object::str()` member function since `py::str(obj)`
is now equivalent and preferred
* Make `py::repr()` a free function
* Make sure obj.cast<T>() works as expected when T is a Python type
`obj.cast<T>()` should be the same as `T(obj)`, i.e. it should convert
the given object to a different Python type. However, `obj.cast<T>()`
usually calls `type_caster::load()` which only checks the type without
doing any actual conversion. That causes a very unexpected `cast_error`.
This commit makes it so that `obj.cast<T>()` and `T(obj)` are the same
when T is a Python type.
* Simplify pytypes converting constructor implementation
It's not necessary to maintain a full set of converting constructors
and assignment operators + const& and &&. A single converting const&
constructor will work and there is no impact on binary size. On the
other hand, the conversion functions can be significantly simplified.
Allows checking the Python types before creating an object instead of
after. For example:
```c++
auto l = list(ptr, true);
if (l.check())
// ...
```
The above is replaced with:
```c++
if (isinstance<list>(ptr)) {
auto l = reinterpret_borrow(ptr);
// ...
}
```
This deprecates `py::object::check()`. `py::isinstance()` covers the
same use case, but it can also check for user-defined types:
```c++
class Pet { ... };
py::class_<Pet>(...);
m.def("is_pet", [](py::object obj) {
return py::isinstance<Pet>(obj); // works as expected
});
```
This commit includes the following changes:
* Don't provide make_copy_constructor for non-copyable container
make_copy_constructor currently fails for various stl containers (e.g.
std::vector, std::unordered_map, std::deque, etc.) when the container's
value type (e.g. the "T" or the std::pair<K,T> for a map) is
non-copyable. This adds an override that, for types that look like
containers, also requires that the value_type be copyable.
* stl_bind.h: make bind_{vector,map} work for non-copy-constructible types
Most stl_bind modifiers require copying, so if the type isn't copy
constructible, we provide a read-only interface instead.
In practice, this means that if the type is non-copyable, it will be,
for all intents and purposes, read-only from the Python side (but
currently it simply fails to compile with such a container).
It is still possible for the caller to provide an interface manually
(by defining methods on the returned class_ object), but this isn't
something stl_bind can handle because the C++ code to construct values
is going to be highly dependent on the container value_type.
* stl_bind: copy only for arithmetic value types
For non-primitive types, we may well be copying some complex type, when
returning by reference is more appropriate. This commit returns by
internal reference for all but basic arithmetic types.
* Return by reference whenever possible
Only if we definitely can't--i.e. std::vector<bool>--because v[i]
returns something that isn't a T& do we copy; for everything else, we
return by reference.
For the map case, we can always return by reference (at least for the
default stl map/unordered_map).
If we need to initialize a holder around an unowned instance, and the
holder type is non-copyable (i.e. a unique_ptr), we currently construct
the holder type around the value pointer, but then never actually
destruct the holder: the holder destructor is called only for the
instance that actually has `inst->owned = true` set.
This seems no pointer, however, in creating such a holder around an
unowned instance: we never actually intend to use anything that the
unique_ptr gives us: and, in fact, do not want the unique_ptr (because
if it ever actually got destroyed, it would cause destruction of the
wrapped pointer, despite the fact that that wrapped pointer isn't
owned).
This commit changes the logic to only create a unique_ptr holder if we
actually own the instance, and to destruct via the constructed holder
whenever we have a constructed holder--which will now only be the case
for owned-unique-holder or shared-holder types.
Other changes include:
* Added test for non-movable holder constructor/destructor counts
The three alive assertions now pass, before #478 they fail with counts
of 2/2/1 respectively, because of the unique_ptr that we don't want and
don't destroy (because we don't *want* its destructor to run).
* Return cstats reference; fix ConstructStats doc
Small cleanup to the #478 test code, and fix to the ConstructStats
documentation (the static method definition should use `reference` not
`reference_internal`).
* Rename inst->constructed to inst->holder_constructed
This makes it clearer exactly what it's referring to.
There are now more places than just descr.h that make use of these.
The new macro isn't quite the same: the old one only tested for a
couple features, while the new one checks for the __cplusplus version
(but doesn't even try to enable C++14 for MSVC/ICC).
g++ 7 adds <optional>, but including it in C++14 mode isn't allowed
(just as including <experimental/optional> isn't allowed in C++11 mode).
(This wasn't triggered in g++-6 because it doesn't provide <optional>
yet.)
* Add type caster for std::experimental::optional
* Add tests for std::experimental::optional
* Support both <optional> / <experimental/optional>
* Mention std{::experimental,}::optional in the docs
* Make reference(_internal) the default return value policy for properties
Before this, all `def_property*` functions used `automatic` as their
default return value policy. This commit makes it so that:
* Non-static properties use `reference_interal` by default, thus
matching `def_readonly` and `def_readwrite`.
* Static properties use `reference` by default, thus matching
`def_readonly_static` and `def_readwrite_static`.
In case `cpp_function` is passed to any `def_property*`, its policy will
be used instead of any defaults. User-defined arguments in `extras`
still have top priority and will override both the default policies and
the ones from `cpp_function`.
Resolves#436.
* Almost always use return_value_policy::move for rvalues
For functions which return rvalues or rvalue references, the only viable
return value policies are `copy` and `move`. `reference(_internal)` and
`take_ownership` would take the address of a temporary which is always
an error.
This commit prevents possible user errors by overriding the bad rvalue
policies with `move`. Besides `move`, only `copy` is allowed, and only
if it's explicitly selected by the user.
This is also a necessary safety feature to support the new default
return value policies for properties: `reference(_internal)`.
The current integer caster was unnecessarily strict and rejected
various kinds of NumPy integer types when calling C++ functions
expecting normal integers. This relaxes the current behavior.
Currently pybind11 doesn't check when you define a new object (e.g. a
class, function, or exception) that overwrites an existing one. If the
thing being overwritten is a class, this leads to a segfault (because
pybind still thinks the type is defined, even though Python no longer
has the type). In other cases this is harmless (e.g. replacing a
function with an exception), but even in that case it's most likely a
bug.
This code doesn't prevent you from actively doing something harmful,
like deliberately overwriting a previous definition, but detects
overwriting with a run-time error if it occurs in the standard
class/function/exception/def registration interfaces.
All of the additions are in non-template code; the result is actually a
tiny decrease in .so size compared to master without the new test code
(977304 to 977272 bytes), and about 4K higher with the new tests.
