Currently pybind11 only supports std::unique_ptr<T> holders by default
(other holders can, of course, be declared using the macro). PR #368
added a `py::nodelete` that is intended to be used as:
py::class_<Type, std::unique_ptr<Type, py::nodelete>> c("Type");
but this doesn't work out of the box. (You could add an explicit
holder type declaration, but this doesn't appear to have been the
intention of the commit).
This commit fixes it by generalizing the unique_ptr type_caster to take
both the type and deleter as template arguments, so that *any*
unique_ptr instances are now automatically handled by pybind. It also
adds a test to test_smart_ptr, testing both that py::nodelete (now)
works, and that the object is indeed not deleted as intended.
Problem
=======
The template trampoline pattern documented in PR #322 has a problem with
virtual method overloads in intermediate classes in the inheritance
chain between the trampoline class and the base class.
For example, consider the following inheritance structure, where `B` is
the actual class, `PyB<B>` is the trampoline class, and `PyA<B>` is an
intermediate class adding A's methods into the trampoline:
PyB<B> -> PyA<B> -> B -> A
Suppose PyA<B> has a method `some_method()` with a PYBIND11_OVERLOAD in
it to overload the virtual `A::some_method()`. If a Python class `C` is
defined that inherits from the pybind11-registered `B` and tries to
provide an overriding `some_method()`, the PYBIND11_OVERLOADs declared
in PyA<B> fails to find this overloaded method, and thus never invoke it
(or, if pure virtual and not overridden in PyB<B>, raises an exception).
This happens because the base (internal) `PYBIND11_OVERLOAD_INT` macro
simply calls `get_overload(this, name)`; `get_overload()` then uses the
inferred type of `this` to do a type lookup in `registered_types_cpp`.
This is where it fails: `this` will be a `PyA<B> *`, but `PyA<B>` is
neither the base type (`B`) nor the trampoline type (`PyB<B>`). As a
result, the overload fails and we get a failed overload lookup.
The fix
=======
The fix is relatively simple: we can cast `this` passed to
`get_overload()` to a `const B *`, which lets get_overload look up the
correct class. Since trampoline classes should be derived from `B`
classes anyway, this cast should be perfectly safe.
This does require adding the class name as an argument to the
PYBIND11_OVERLOAD_INT macro, but leaves the public macro signatures
unchanged.
- ICPC can't handle the NCVirt trampoline which returns a non-copyable
type, which is likely due to a constexpr/SFINAE issue. This disables
the test on that compiler so that at least the rest can be tested.
For example keep_alive<0,1>() should work where the return value may sometimes be None. At present a "Could not allocate weak reference!" exception is thrown.
Update documentation to clarify behaviour of keep_alive when nurse is None or does not support weak references.
This is required since format descriptors for string types that
were using PYBIND11_DESCR were causing problems on C++14 on Linux.
Although this is technically a breaking change, it shouldn't cause
problems since the only use of format strings is passing them to
buffer_info constructor which expects std::string.
Note: for non-structured types, the const char * value is still
accessible via ::value for compatibility purpose.
The format strings that are known at compile time are now accessible
via both ::value and ::format(), and format strings for everything
else is accessible via ::format(). This makes it backwards compatible.
This allows exposing a dict-like interface to python code, allowing
iteration over keys via:
for k in custommapping:
...
while still allowing iteration over pairs, so that you can also
implement 'dict.items()' functionality which returns a pair iterator,
allowing:
for k, v in custommapping.items():
...
example-sequences-and-iterators is updated with a custom class providing
both types of iteration.
PR #329 generates the following warning under MSVC:
...\cast.h(202): warning C4456: declaration of 'it' hides previous local declaration
This renames the second iterator to silence it.
reference_internal requires an `instance` field to track the returned
reference's parent, but that's just a duplication of what
keep_alive<0,1> does, so use a keep alive to do this to eliminate the
duplication.
The pointer to the first member of a class instance is the same as the
pointer to instance itself; pybind11 has some workarounds for this to
not track registered instances that have a registered parent with the
same address. This doesn't work everywhere, however: issue #328 is a
failure of this for a mutator operator which resolves its argument to
the parent rather than the child, as is needed in #328.
This commit resolves the issue (and restores tracking of same-address
instances) by changing registered_instances from an unordered_map to an
unordered_multimap that allows duplicate instances for the same pointer
to be recorded, then resolves these differences by checking the type of
each matched instance when looking up an instance. (A
unordered_multimap seems cleaner for this than a unordered_map<list> or
similar because, the vast majority of the time, the instance will be
unique).
Currently pybind11 always translates values returned by Python functions
invoked from C++ code by copying, even when moving is feasible--and,
more importantly, even when moving is required.
The first, and relatively minor, concern is that moving may be
considerably more efficient for some types. The second problem,
however, is more serious: there's currently no way python code can
return a non-copyable type to C++ code.
I ran into this while trying to add a PYBIND11_OVERLOAD of a virtual
method that returns just such a type: it simply fails to compile because
this:
overload = ...
overload(args).template cast<ret_type>();
involves a copy: overload(args) returns an object instance, and the
invoked object::cast() loads the returned value, then returns a copy of
the loaded value.
We can, however, safely move that returned value *if* the object has the
only reference to it (i.e. if ref_count() == 1) and the object is
itself temporary (i.e. if it's an rvalue).
This commit does that by adding an rvalue-qualified object::cast()
method that allows the returned value to be move-constructed out of the
stored instance when feasible.
This basically comes down to three cases:
- For objects that are movable but not copyable, we always try the move,
with a runtime exception raised if this would involve moving a value
with multiple references.
- When the type is both movable and non-trivially copyable, the move
happens only if the invoked object has a ref_count of 1, otherwise the
object is copied. (Trivially copyable types are excluded from this
case because they are typically just collections of primitive types,
which can be copied just as easily as they can be moved.)
- Non-movable and trivially copy constructible objects are simply
copied.
This also adds examples to example-virtual-functions that shows both a
non-copyable object and a movable/copyable object in action: the former
raises an exception if returned while holding a reference, the latter
invokes a move constructor if unreferenced, or a copy constructor if
referenced.
Basically this allows code such as:
class MyClass(Pybind11Class):
def somemethod(self, whatever):
mt = MovableType(whatever)
# ...
return mt
which allows the MovableType instance to be returned to the C++ code
via its move constructor.
Of course if you attempt to violate this by doing something like:
self.value = MovableType(whatever)
return self.value
you get an exception--but right now, the pybind11-side of that code
won't compile at all.
Example signatures (old => new):
foo(int) => foo(arg0: int)
bar(Object, int) => bar(self: Object, arg0: int)
The change makes the signatures uniform for named and unnamed arguments
and it helps static analysis tools reconstruct function signatures from
docstrings.
This also tweaks the signature whitespace style to better conform to
PEP 8 for annotations and default arguments:
" : " => ": "
" = " => "="
Functions returning specialized Eigen matrices like Eigen::DiagonalMatrix and
Eigen::SelfAdjointView--which inherit from EigenBase but not
DenseBase--isn't currently allowed; such classes are explicitly copyable
into a Matrix (by definition), and so we can support functions that
return them by copying the value into a Matrix then casting that
resulting dense Matrix into a numpy.ndarray. This commit does exactly
that.
