new move value policy

docs/advanced.rst

@@ -411,38 +411,63 @@ For this reason, pybind11 provides a several `return value policy` annotations
 that can be passed to the :func:`module::def` and :func:`class_::def`
 functions. The default policy is :enum:`return_value_policy::automatic`.
 
-
++--------------------------------------------------+----------------------------------------------------------------------------+
-+--------------------------------------------------+---------------------------------------------------------------------------+
 | Return value policy                              | Description                                                                |
-+==================================================+===========================================================================+
++==================================================+============================================================================+
-| :enum:`return_value_policy::automatic`           | Automatic: copy objects returned as values and take ownership of          |
+| :enum:`return_value_policy::automatic`           | This is the default return value policy, which falls back to the policy    |
-|                                                  | objects returned as pointers                                              |
+|                                                  | :enum:`return_value_policy::take_ownership` when the return value is a     |
-+--------------------------------------------------+---------------------------------------------------------------------------+
+|                                                  | pointer. Otherwise, it uses :enum::`return_value::move` or                 |
-| :enum:`return_value_policy::automatic_reference` | Automatic variant 2 : copy objects returned as values and reference       |
+|                                                  | :enum::`return_value::copy` for rvalue and lvalue references, respectively.|
-|                                                  | objects returned as pointers                                              |
+|                                                  | See below for a description of what all of these different policies do.    |
-+--------------------------------------------------+---------------------------------------------------------------------------+
++--------------------------------------------------+----------------------------------------------------------------------------+
-| :enum:`return_value_policy::copy`                | Create a new copy of the returned object, which will be owned by Python   |
+| :enum:`return_value_policy::automatic_reference` | As above, but use policy :enum:`return_value_policy::reference` when the   |
-+--------------------------------------------------+---------------------------------------------------------------------------+
+|                                                  | return value is a pointer.                                                 |
-| :enum:`return_value_policy::take_ownership`      | Reference the existing object and take ownership. Python will call        |
++--------------------------------------------------+----------------------------------------------------------------------------+
-|                                                  | the destructor and delete operator when the reference count reaches zero  |
+| :enum:`return_value_policy::take_ownership`      | Reference an existing object (i.e. do not create a new copy) and take      |
-+--------------------------------------------------+---------------------------------------------------------------------------+
+|                                                  | ownership. Python will call the destructor and delete operator when the    |
-| :enum:`return_value_policy::reference`           | Reference the object, but do not take ownership and defer responsibility  |
+|                                                  | object's reference count reaches zero. Undefined behavior ensues when the  |
-|                                                  | for deleting it to C++ (dangerous when C++ code at some point decides to  |
+|                                                  | C++ side does the same..                                                   |
-|                                                  | delete it while Python still has a nonzero reference count)               |
++--------------------------------------------------+----------------------------------------------------------------------------+
-+--------------------------------------------------+---------------------------------------------------------------------------+
+| :enum:`return_value_policy::copy`                | Create a new copy of the returned object, which will be owned by Python.   |
+|                                                  | This policy is comparably safe because the lifetimes of the two instances  |
+|                                                  | are decoupled.                                                             |
++--------------------------------------------------+----------------------------------------------------------------------------+
+| :enum:`return_value_policy::move`                | Use ``std::move`` to move the return value contents into a new instance    |
+|                                                  | that will be owned by Python. This policy is comparably safe because the   |
+|                                                  | lifetimes of the two instances (move source and destination) are decoupled.|
++--------------------------------------------------+----------------------------------------------------------------------------+
+| :enum:`return_value_policy::reference`           | Reference an existing object, but do not take ownership. The C++ side is   |
+|                                                  | responsible for managing the object's lifetime and deallocating it when    |
+|                                                  | it is no longer used. Warning: undefined behavior will ensue when the C++  |
+|                                                  | side deletes an object that is still referenced by Python.                 |
++--------------------------------------------------+----------------------------------------------------------------------------+
 | :enum:`return_value_policy::reference_internal`  | Reference the object, but do not take ownership. The object is considered  |
 |                                                  | be owned by the C++ instance whose method or property returned it. The     |
 |                                                  | Python object will increase the reference count of this 'parent' by 1      |
 |                                                  | to ensure that it won't be deallocated while Python is using the 'child'   |
-+--------------------------------------------------+---------------------------------------------------------------------------+
++--------------------------------------------------+----------------------------------------------------------------------------+
 
 .. warning::
 
-    Code with invalid call policies might access unitialized memory and free
+    Code with invalid call policies might access unitialized memory or free
     data structures multiple times, which can lead to hard-to-debug
     non-determinism and segmentation faults, hence it is worth spending the
     time to understand all the different options above.
 
+.. note::
+
+    The next section on :ref:`call_policies` discusses *call policies* that can be
+    specified *in addition* to a return value policy from the list above. Call
+    policies indicate reference relationships that can involve both return values
+    and parameters of functions.
+
+.. note::
+
+   As an alternative to elaborate call policies and lifetime management logic,
+   consider using smart pointers (see :ref:`smart_pointers` for details) that
+   can be used to share reference count information between C++ and Python.
+
+
 See below for an example that uses the
 :enum:`return_value_policy::reference_internal` policy.
 
@@ -466,6 +491,8 @@ See below for an example that uses the
     }
 
 
+.. _call_policies:
+
 Additional call policies
 ========================
 
@@ -557,6 +584,8 @@ The above signature would imply that Python needs to give up ownership of an
 object that is passed to this function, which is generally not possible (for
 instance, the object might be referenced elsewhere).
 
+.. _smart_pointers:
+
 Smart pointers
 ==============
 

example/example1.ref

@@ -7,8 +7,8 @@ Called Example1 destructor (32)
 Instance 1: Example1[value=320]
 Instance 2: Example1[value=32]
 Called Example1 copy constructor with value 320..
-Called Example1 copy constructor with value 320..
+Called Example1 move constructor with value 320..
-Called Example1 destructor (320)
+Called Example1 destructor (0)
 Example1[value=320]
 Called Example1 destructor (320)
 Example1[value=320]

example/example3.ref

@@ -3,52 +3,52 @@ Value constructor
 v1    = [1.000000, 2.000000]
 v2    = [3.000000, -1.000000]
 Value constructor
-Copy constructor
+Move constructor
 Destructor.
 Destructor.
 v1+v2 = [4.000000, 1.000000]
 Value constructor
-Copy constructor
+Move constructor
 Destructor.
 Destructor.
 v1-v2 = [-2.000000, 3.000000]
 Value constructor
-Copy constructor
+Move constructor
 Destructor.
 Destructor.
 v1-8  = [-7.000000, -6.000000]
 Value constructor
-Copy constructor
+Move constructor
 Destructor.
 Destructor.
 v1+8  = [9.000000, 10.000000]
 Value constructor
-Copy constructor
+Move constructor
 Destructor.
 Destructor.
 v1*8  = [8.000000, 16.000000]
 Value constructor
-Copy constructor
+Move constructor
 Destructor.
 Destructor.
 v1/8  = [0.125000, 0.250000]
 Value constructor
-Copy constructor
+Move constructor
 Destructor.
 Destructor.
 8-v1  = [7.000000, 6.000000]
 Value constructor
-Copy constructor
+Move constructor
 Destructor.
 Destructor.
 8+v1  = [9.000000, 10.000000]
 Value constructor
-Copy constructor
+Move constructor
 Destructor.
 Destructor.
 8*v1  = [8.000000, 16.000000]
 Value constructor
-Copy constructor
+Move constructor
 Destructor.
 Destructor.
 8/v1  = [8.000000, 4.000000]

example/example6.ref

@@ -1,13 +1,13 @@
 Value constructor: Creating a sequence with 5 entries
-s = <example.Sequence object at 0x1033bd8d0>
+s = <example.Sequence object at 0x10c786c70>
 len(s) = 5
 s[0], s[3] = 0.000000 0.000000
 12.34 in s: False
 12.34 in s: True
 s[0], s[3] = 12.340000 56.779999
 Value constructor: Creating a sequence with 5 entries
-Copy constructor: Creating a sequence with 5 entries
+Move constructor: Creating a sequence with 5 entries
-Freeing a sequence with 5 entries
+Freeing a sequence with 0 entries
 Value constructor: Creating a sequence with 5 entries
 rev[0], rev[1], rev[2], rev[3], rev[4] = 0.000000 56.779999 0.000000 0.000000 12.340000
 0.0 56.7799987793 0.0 0.0 12.3400001526 

include/pybind11/cast.h

@@ -158,6 +158,7 @@ public:
                                          const std::type_info *type_info,
                                          const std::type_info *type_info_backup,
                                          void *(*copy_constructor)(const void *),
+                                         void *(*move_constructor)(const void *),
                                          const void *existing_holder = nullptr) {
         void *src = const_cast<void *>(_src);
         if (src == nullptr)
@@ -204,6 +205,12 @@ public:
             wrapper->value = copy_constructor(wrapper->value);
             if (wrapper->value == nullptr)
                 throw cast_error("return_value_policy = copy, but the object is non-copyable!");
+        } else if (policy == return_value_policy::move) {
+            wrapper->value = move_constructor(wrapper->value);
+            if (wrapper->value == nullptr)
+                wrapper->value = copy_constructor(wrapper->value);
+            if (wrapper->value == nullptr)
+                throw cast_error("return_value_policy = move, but the object is neither movable nor copyable!");
         } else if (policy == return_value_policy::reference) {
             wrapper->owned = false;
         } else if (policy == return_value_policy::reference_internal) {
@@ -243,8 +250,16 @@ public:
         return cast(&src, policy, parent);
     }
 
+    static handle cast(type &&src, return_value_policy policy, handle parent) {
+        if (policy == return_value_policy::automatic || policy == return_value_policy::automatic_reference)
+            policy = return_value_policy::move;
+        return cast(&src, policy, parent);
+    }
+
     static handle cast(const type *src, return_value_policy policy, handle parent) {
-        return type_caster_generic::cast(src, policy, parent, src ? &typeid(*src) : nullptr, &typeid(type), &copy_constructor);
+        return type_caster_generic::cast(
+            src, policy, parent, src ? &typeid(*src) : nullptr, &typeid(type),
+            &copy_constructor, &move_constructor);
     }
 
     template <typename T> using cast_op_type = pybind11::detail::cast_op_type<T>;
@@ -253,11 +268,13 @@ public:
     operator type&() { return *((type *) value); }
 protected:
     template <typename T = type, typename std::enable_if<detail::is_copy_constructible<T>::value, int>::type = 0>
-    static void *copy_constructor(const void *arg) {
+    static void *copy_constructor(const void *arg) { return (void *) new type(*((const type *) arg)); }
-        return (void *) new type(*((const type *) arg));
-    }
     template <typename T = type, typename std::enable_if<!detail::is_copy_constructible<T>::value, int>::type = 0>
     static void *copy_constructor(const void *) { return nullptr; }
+    template <typename T = type, typename std::enable_if<detail::is_move_constructible<T>::value, int>::type = 0>
+    static void *move_constructor(const void *arg) { return (void *) new type(std::move(*((type *) arg))); }
+    template <typename T = type, typename std::enable_if<!detail::is_move_constructible<T>::value, int>::type = 0>
+    static void *move_constructor(const void *) { return nullptr; }
 };
 
 template <typename type> class type_caster<std::reference_wrapper<type>> : public type_caster<type> {
@@ -662,6 +679,7 @@ public:
     using type_caster<type>::value;
     using type_caster<type>::temp;
     using type_caster<type>::copy_constructor;
+    using type_caster<type>::move_constructor;
 
     bool load(handle src, bool convert) {
         if (!src || !typeinfo) {
@@ -702,7 +720,7 @@ public:
         return type_caster_generic::cast(
             src.get(), policy, parent,
             src.get() ? &typeid(*src.get()) : nullptr, &typeid(type),
-            &copy_constructor, &src);
+            &copy_constructor, &move_constructor, &src);
     }
 
 protected:

include/pybind11/common.h

@@ -141,29 +141,46 @@ typedef Py_ssize_t ssize_t;
 
 /// Approach used to cast a previously unknown C++ instance into a Python object
 enum class return_value_policy : int {
-    /** Automatic: copy objects returned as values and take ownership of objects
+    /** This is the default return value policy, which falls back to the policy
-        returned as pointers */
+        return_value_policy::take_ownership when the return value is a pointer.
+        Otherwise, it uses return_value::move or return_value::copy for rvalue
+        and lvalue references, respectively. See below for a description of what
+        all of these different policies do. */
     automatic = 0,
 
-    /** Automatic variant 2: copy objects returned as values and reference objects
+    /** As above, but use policy return_value_policy::reference when the return
-        returned as pointers */
+        value is a pointer. */
     automatic_reference,
 
-    /** Reference the object and take ownership. Python will call the
+    /** Reference an existing object (i.e. do not create a new copy) and take
-        destructor and delete operator when the reference count reaches zero */
+        ownership. Python will call the destructor and delete operator when the
+        object’s reference count reaches zero. Undefined behavior ensues when
+        the C++ side does the same.. */
     take_ownership,
 
-    /** Reference the object, but do not take ownership (dangerous when C++ code
+    /** Create a new copy of the returned object, which will be owned by
-        deletes it and Python still has a nonzero reference count) */
+        Python. This policy is comparably safe because the lifetimes of the two
+        instances are decoupled. */
+    copy,
+
+    /** Use std::move to move the return value contents into a new instance
+        that will be owned by Python. This policy is comparably safe because the
+        lifetimes of the two instances (move source and destination) are
+        decoupled. */
+    move,
+
+    /** Reference an existing object, but do not take ownership. The C++ side
+        is responsible for managing the object’s lifetime and deallocating it
+        when it is no longer used. Warning: undefined behavior will ensue when
+        the C++ side deletes an object that is still referenced by Python. */
     reference,
 
-    /** Reference the object, but do not take ownership. The object is considered
+    /** Reference the object, but do not take ownership. The object is
-        be owned by the C++ instance whose method or property returned it. The
+        considered be owned by the C++ instance whose method or property
-        Python object will increase the reference count of this 'parent' by 1 */
+        returned it. The Python object will increase the reference count of this
-    reference_internal,
+        ‘parent’ by 1 to ensure that it won’t be deallocated while Python is
-
+        using the ‘child’ */
-    /// Create a new copy of the returned object, which will be owned by Python
+    reference_internal
-    copy
 };
 
 /// Format strings for basic number types
@@ -276,6 +293,12 @@ template <typename T>  struct is_copy_constructible {
     static const bool value = std::is_same<std::true_type, decltype(test<T>(nullptr))>::value;
 };
 
+template <typename T>  struct is_move_constructible {
+    template <typename T2> static std::true_type test(decltype(new T2(std::declval<typename std::add_rvalue_reference<T2>::type>())) *);
+    template <typename T2> static std::false_type test(...);
+    static const bool value = std::is_same<std::true_type, decltype(test<T>(nullptr))>::value;
+};
+
 /// Helper type to replace 'void' in some expressions
 struct void_type { };