Currently pybind11 always translates values returned by Python functions
invoked from C++ code by copying, even when moving is feasible--and,
more importantly, even when moving is required.
The first, and relatively minor, concern is that moving may be
considerably more efficient for some types. The second problem,
however, is more serious: there's currently no way python code can
return a non-copyable type to C++ code.
I ran into this while trying to add a PYBIND11_OVERLOAD of a virtual
method that returns just such a type: it simply fails to compile because
this:
overload = ...
overload(args).template cast<ret_type>();
involves a copy: overload(args) returns an object instance, and the
invoked object::cast() loads the returned value, then returns a copy of
the loaded value.
We can, however, safely move that returned value *if* the object has the
only reference to it (i.e. if ref_count() == 1) and the object is
itself temporary (i.e. if it's an rvalue).
This commit does that by adding an rvalue-qualified object::cast()
method that allows the returned value to be move-constructed out of the
stored instance when feasible.
This basically comes down to three cases:
- For objects that are movable but not copyable, we always try the move,
with a runtime exception raised if this would involve moving a value
with multiple references.
- When the type is both movable and non-trivially copyable, the move
happens only if the invoked object has a ref_count of 1, otherwise the
object is copied. (Trivially copyable types are excluded from this
case because they are typically just collections of primitive types,
which can be copied just as easily as they can be moved.)
- Non-movable and trivially copy constructible objects are simply
copied.
This also adds examples to example-virtual-functions that shows both a
non-copyable object and a movable/copyable object in action: the former
raises an exception if returned while holding a reference, the latter
invokes a move constructor if unreferenced, or a copy constructor if
referenced.
Basically this allows code such as:
class MyClass(Pybind11Class):
def somemethod(self, whatever):
mt = MovableType(whatever)
# ...
return mt
which allows the MovableType instance to be returned to the C++ code
via its move constructor.
Of course if you attempt to violate this by doing something like:
self.value = MovableType(whatever)
return self.value
you get an exception--but right now, the pybind11-side of that code
won't compile at all.
Example signatures (old => new):
foo(int) => foo(arg0: int)
bar(Object, int) => bar(self: Object, arg0: int)
The change makes the signatures uniform for named and unnamed arguments
and it helps static analysis tools reconstruct function signatures from
docstrings.
This also tweaks the signature whitespace style to better conform to
PEP 8 for annotations and default arguments:
" : " => ": "
" = " => "="
Functions returning specialized Eigen matrices like Eigen::DiagonalMatrix and
Eigen::SelfAdjointView--which inherit from EigenBase but not
DenseBase--isn't currently allowed; such classes are explicitly copyable
into a Matrix (by definition), and so we can support functions that
return them by copying the value into a Matrix then casting that
resulting dense Matrix into a numpy.ndarray. This commit does exactly
that.
Some Eigen objects, such as those returned by matrix.diagonal() and
matrix.block() have non-standard stride values because they are
basically just maps onto the underlying matrix without copying it (for
example, the primary diagonal of a 3x3 matrix is a vector-like object
with .src equal to the full matrix data, but with stride 4). Returning
such an object from a pybind11 method breaks, however, because pybind11
assumes vectors have stride 1, and that matrices have strides equal to
the number of rows/columns or 1 (depending on whether the matrix is
stored column-major or row-major).
This commit fixes the issue by making pybind11 use Eigen's stride
methods when copying the data.
PR #309 broke scoped enums, which failed to compile because the added:
value == value2
comparison isn't valid for a scoped enum (they aren't implicitly
convertible to the underlying type). This commit fixes it by
explicitly converting the enum value to its underlying type before
doing the comparison.
It also adds a scoped enum example to the constants-and-functions
example that triggers the problem fixed in this commit.
Eigen::Ref is a common way to pass eigen dense types without needing a
template, e.g. the single definition `void
func(Eigen::Ref<Eigen::MatrixXd> x)` can be called with any double
matrix-like object.
The current pybind11 eigen support fails with internal errors if
attempting to bind a function with an Eigen::Ref<...> argument because
Eigen::Ref<...> satisfies the "is_eigen_dense" requirement, but can't
compile if actually used: Eigen::Ref<...> itself is not default
constructible, and so the argument std::tuple containing an
Eigen::Ref<...> isn't constructible, which results in compilation
failure.
This commit adds support for Eigen::Ref<...> by giving it its own
type_caster implementation which consists of an internal type_caster of
the referenced type, load/cast methods that dispatch to the internal
type_caster, and a unique_ptr to an Eigen::Ref<> instance that gets
set during load().
There is, of course, no performance advantage for pybind11-using code of
using Eigen::Ref<...>--we are allocating a matrix of the derived type
when loading it--but this has the advantage of allowing pybind11 to bind
transparently to C++ methods taking Eigen::Refs.
This changes the exception error message of a bad-arguments error to
suppress the constructor argument when the failure is a constructor.
This changes both the "Invoked with: " output to omit the object
instances, and rewrites the constructor signature to make it look
like a constructor (changing the first argument to the object name, and
removing the ' -> NoneType' return type.
GCC-6 adds a -Wplacement-new warning that warns for placement-new into a
space that is too small, which is sometimes being triggered here (e.g.
example5 always generates the warning under g++-6). It's a false
warning, however: the line immediately before just checked the size, and
so this line is never going to actually be reached in the cases where
the GCC warning is being triggered.
This localizes the warning disabling just to this one spot as there are
other placement-new uses in pybind11 where this warning could warn about
legitimate future problems.
This commit adds an additional _ template function for compile-time
selection between two description strings. This in turn allows the
elimination of needing two name() methods in type_caster<arithmetic
types> and type_caster<eigen types>, which allows them to start using
PYBIND11_TYPE_CASTER instead, simplifying their code by eliminating all
the code that they are duplicating from the macro.
In eigen.h, type_caster<Type>::load(): For the 'ndim == 1' case, use
the 'InnerStride' type because there is only an inner stride for a
vector. Choose between (n_elts x 1) or (1 x n_elts) according to
whether we're constructing a Vector or a RowVector.
Sergey Lyskov pointed out that the trampoline mechanism used to override
virtual methods from within Python caused unnecessary overheads when
instantiating the original (i.e. non-extended) class.
This commit removes this inefficiency, but some syntax changes were
needed to achieve this. Projects using this features will need to make a
few changes:
In particular, the example below shows the old syntax to instantiate a
class with a trampoline:
class_<TrampolineClass>("MyClass")
.alias<MyClass>()
....
This is what should be used now:
class_<MyClass, std::unique_ptr<MyClass, TrampolineClass>("MyClass")
....
Importantly, the trampoline class is now specified as the *third*
argument to the class_ template, and the alias<..>() call is gone. The
second argument with the unique pointer is simply the default holder
type used by pybind11.
args was derived from list, but cpp_function::dispatcher sends a tuple to it->impl (line #346 and #392 in pybind11.h). As a result args::size() and args::operator[] don't work at all. On my mac args::size() returns -1. Making args a subclass of tuple fixes it.
This somewhat heavyweight solution will avoid size_t/long long/long/int
mismatches on various platforms once and for all. The previous template
overloads could e.g. not handle size_t on Darwin.
One gotcha: the 'format_descriptor<T>::value()' syntax changed to just
'format_descriptor<T>::value'
This fixes a build error compiling with `nvcc/7.5` + `gcc/4.9.2`
causing a
```
./include/pybind11/pybind11.h(952): here
./include/pybind11/pytypes.h: In member function ‘pybind11::str pybind11::handle::str() const’:
./include/pybind11/pytypes.h:269:8: error: expected primary-expression before ‘class’
return pybind11::str(str, false);
^
./include/pybind11/pytypes.h:269:8: error: expected ‘;’ before ‘class’
./include/pybind11/pytypes.h:269:8: error: expected primary-expression before ‘class’
```
- new pybind11::base<> attribute to indicate a subclass relationship
- unified infrastructure for parsing variadic arguments in class_ and cpp_function
- use 'handle' and 'object' more consistently everywhere
Previously, pybind11 required classes using std::shared_ptr<> to derive
from std::enable_shared_from_this<> (or compilation failures would ensue).
Everything now also works for classes that don't do this, assuming that
some basic rules are followed (e.g. never passing "raw" pointers of
instances manged by shared pointers). The safer
std::enable_shared_from_this<> approach continues to be supported.
This modification taps into some newer C++14 features (if present) to
generate function signatures considerably more efficiently at compile
time rather than at run time.
With this change, pybind11 binaries are now *2.1 times* smaller compared
to the Boost.Python baseline in the benchmark. Compilation times get a
nice improvement as well.
Visual Studio 2015 unfortunately doesn't implement 'constexpr' well
enough yet to support this change and uses a runtime fallback.
The cpp_function class accepts a variadic argument, which was formerly
processed twice -- once at registration time, and once in the dispatch
lambda function. This is not only unnecessarily slow but also leads to
code bloat since it adds to the object code generated for every bound
function. This change removes the second pass at dispatch time.
One noteworthy change of this commit is that default arguments are now
constructed (and converted to Python objects) right at declaration time.
Consider the following example:
py::class_<MyClass>("MyClass")
.def("myFunction", py::arg("arg") = SomeType(123));
In this case, the change means that pybind11 must already be set up to
deal with values of the type 'SomeType', or an exception will be thrown.
Another change is that the "preview" of the default argument in the
function signature is generated using the __repr__ special method. If
it is not available in this type, the signature may not be very helpful,
i.e.:
| myFunction(...)
| Signature : (MyClass, arg : SomeType = <SomeType object at 0x101b7b080>) -> None
One workaround (other than defining SomeType.__repr__) is to specify the
human-readable preview of the default argument manually using the more
cumbersome arg_t notation:
py::class_<MyClass>("MyClass")
.def("myFunction", py::arg_t<SomeType>("arg", SomeType(123), "SomeType(123)"));
Using object class to hold converted object automatically deallocates
object if an exception is thrown or scope is left before constructing
complete Python object.
Additionally added method object::release() that allows to release
ownership of python object without decreasing its reference count.
The array(const buffer_info &info) constructor fails when given
complex types since their format string is 'Zd' or 'Zf' which has
a length of two and causes an error here:
if (info.format.size() != 1)
throw std::runtime_error("Unsupported buffer format!");
Fixed by allowing format sizes of one and two.
- Get a descriptive string after a Python exception (without changing the exception status)
- Convenience function to map from a C++ object to a Python handle
- Convenience to check if a pybind::function is defined by an underlying
C++ implementation
- Get the type object of a pybind::handle
