Allow binding factory functions as constructors

This allows you to use:

    cls.def(py::init(&factory_function));

where `factory_function` returns a pointer, holder, or value of the
class type (or a derived type).  Various compile-time checks
(static_asserts) are performed to ensure the function is valid, and
various run-time type checks where necessary.

Some other details of this feature:
- The `py::init` name doesn't conflict with the templated no-argument
  `py::init<...>()`, but keeps the naming consistent: the existing
  templated, no-argument one wraps constructors, the no-template,
  function-argument one wraps factory functions.
- If returning a CppClass (whether by value or pointer) when an CppAlias
  is required (i.e. python-side inheritance and a declared alias), a
  dynamic_cast to the alias is attempted (for the pointer version); if
  it fails, or if returned by value, an Alias(Class &&) constructor
  is invoked.  If this constructor doesn't exist, a runtime error occurs.
- for holder returns when an alias is required, we try a dynamic_cast of
  the wrapped pointer to the alias to see if it is already an alias
  instance; if it isn't, we raise an error.
- `py::init(class_factory, alias_factory)` is also available that takes
  two factories: the first is called when an alias is not needed, the
  second when it is.
- Reimplement factory instance clearing.  The previous implementation
  failed under python-side multiple inheritance: *each* inherited
  type's factory init would clear the instance instead of only setting
  its own type value.  The new implementation here clears just the
  relevant value pointer.
- dealloc is updated to explicitly set the leftover value pointer to
  nullptr and the `holder_constructed` flag to false so that it can be
  used to clear preallocated value without needing to rebuild the
  instance internals data.
- Added various tests to test out new allocation/deallocation code.
- With preallocation now done lazily, init factory holders can
  completely avoid the extra overhead of needing an extra
  allocation/deallocation.
- Updated documentation to make factory constructors the default
  advanced constructor style.
- If an `__init__` is called a second time, we have two choices: we can
  throw away the first instance, replacing it with the second; or we can
  ignore the second call.  The latter is slightly easier, so do that.

CMakeLists.txt

@@ -39,6 +39,7 @@ set(PYBIND11_HEADERS
   include/pybind11/detail/class.h
   include/pybind11/detail/common.h
   include/pybind11/detail/descr.h
+  include/pybind11/detail/init.h
   include/pybind11/detail/typeid.h
   include/pybind11/attr.h
   include/pybind11/buffer_info.h

docs/advanced/classes.rst

@@ -322,6 +322,8 @@ can now create a python class that inherits from ``Dog``:
     See the file :file:`tests/test_virtual_functions.cpp` for complete examples
     using both the duplication and templated trampoline approaches.
 
+.. _extended_aliases:
+
 Extended trampoline class functionality
 =======================================
 
@@ -358,16 +360,124 @@ Custom constructors
 ===================
 
 The syntax for binding constructors was previously introduced, but it only
-works when a constructor with the given parameters actually exists on the C++
+works when a constructor of the appropriate arguments actually exists on the
-side. To extend this to more general cases, let's take a look at what actually
+C++ side.  To extend this to more general cases, pybind11 offers two different
-happens under the hood: the following statement
+approaches: binding factory functions, and placement-new creation.
+
+Factory function constructors
+-----------------------------
+
+It is possible to expose a Python-side constructor from a C++ function that
+returns a new object by value or pointer.  For example, suppose you have a
+class like this:
+
+.. code-block:: cpp
+
+    class Example {
+    private:
+        Example(int); // private constructor
+    public:
+        // Factory function:
+        static Example create(int a) { return Example(a); }
+    };
+
+While it is possible to expose the ``create`` method to Python, it is often
+preferrable to expose it on the Python side as a constructor rather than a
+named static method.  You can do this by calling ``.def(py::init(...))`` with
+the function reference returning the new instance passed as an argument.  It is
+also possible to use this approach to bind a function returning a new instance
+by raw pointer or by the holder (e.g. ``std::unique_ptr``).
+
+The following example shows the different approaches:
+
+.. code-block:: cpp
+
+    class Example {
+    private:
+        Example(int); // private constructor
+    public:
+        // Factory function - returned by value:
+        static Example create(int a) { return Example(a); }
+
+        // These constructors are publicly callable:
+        Example(double);
+        Example(int, int);
+        Example(std::string);
+    };
+
+    py::class_<Example>(m, "Example")
+        // Bind the factory function as a constructor:
+        .def(py::init(&Example::create))
+        // Bind a lambda function returning a pointer wrapped in a holder:
+        .def(py::init([](std::string arg) {
+            return std::unique_ptr<Example>(new Example(arg));
+        }))
+        // Return a raw pointer:
+        .def(py::init([](int a, int b) { return new Example(a, b); }))
+        // You can mix the above with regular C++ constructor bindings as well:
+        .def(py::init<double>())
+        ;
+
+When the constructor is invoked from Python, pybind11 will call the factory
+function and store the resulting C++ instance in the Python instance.
+
+When combining factory functions constructors with :ref:`overriding_virtuals`
+there are two approaches.  The first is to add a constructor to the alias class
+that takes a base value by rvalue-reference.  If such a constructor is
+available, it will be used to construct an alias instance from the value
+returned by the factory function.  The second option is to provide two factory
+functions to ``py::init()``: the first will be invoked when no alias class is
+required (i.e. when the class is being used but not inherited from in Python),
+and the second will be invoked when an alias is required.
+
+You can also specify a single factory function that always returns an alias
+instance: this will result in behaviour similar to ``py::init_alias<...>()``,
+as described in :ref:`extended_aliases`.
+
+The following example shows the different factory approaches for a class with
+an alias:
+
+.. code-block:: cpp
+
+    #include <pybind11/factory.h>
+    class Example {
+    public:
+        // ...
+        virtual ~Example() = default;
+    };
+    class PyExample : public Example {
+    public:
+        using Example::Example;
+        PyExample(Example &&base) : Example(std::move(base)) {}
+    };
+    py::class_<Example, PyExample>(m, "Example")
+        // Returns an Example pointer.  If a PyExample is needed, the Example
+        // instance will be moved via the extra constructor in PyExample, above.
+        .def(py::init([]() { return new Example(); }))
+        // Two callbacks:
+        .def(py::init([]() { return new Example(); } /* no alias needed */,
+                      []() { return new PyExample(); } /* alias needed */))
+        // *Always* returns an alias instance (like py::init_alias<>())
+        .def(py::init([]() { return new PyExample(); }))
+        ;
+
+Low-level placement-new construction
+------------------------------------
+
+A second approach for creating new instances use C++ placement new to construct
+an object in-place in preallocated memory.  To do this, you simply bind a
+method name ``__init__`` that takes the class instance as the first argument by
+pointer or reference, then uses a placement-new constructor to construct the
+object in the pre-allocated (but uninitialized) memory.
+
+For example, instead of:
 
 .. code-block:: cpp
 
     py::class_<Example>(m, "Example")
         .def(py::init<int>());
 
-is short hand notation for
+you could equivalently write:
 
 .. code-block:: cpp
 
@@ -378,9 +488,7 @@ is short hand notation for
             }
         );
 
-In other words, :func:`init` creates an anonymous function that invokes an
+which will invoke the constructor in-place at the pre-allocated memory.
-in-place constructor. Memory allocation etc. is already take care of beforehand
-within pybind11.
 
 .. _classes_with_non_public_destructors:
 

include/pybind11/attr.h

@@ -223,7 +223,7 @@ struct type_record {
     void (*init_instance)(instance *, const void *) = nullptr;
 
     /// Function pointer to class_<..>::dealloc
-    void (*dealloc)(const detail::value_and_holder &) = nullptr;
+    void (*dealloc)(detail::value_and_holder &) = nullptr;
 
     /// List of base classes of the newly created type
     list bases;

include/pybind11/cast.h

@@ -45,7 +45,7 @@ struct type_info {
     size_t type_size, holder_size_in_ptrs;
     void *(*operator_new)(size_t);
     void (*init_instance)(instance *, const void *);
-    void (*dealloc)(const value_and_holder &v_h);
+    void (*dealloc)(value_and_holder &v_h);
     std::vector<PyObject *(*)(PyObject *, PyTypeObject *)> implicit_conversions;
     std::vector<std::pair<const std::type_info *, void *(*)(void *)>> implicit_casts;
     std::vector<bool (*)(PyObject *, void *&)> *direct_conversions;
@@ -301,11 +301,15 @@ struct value_and_holder {
     const detail::type_info *type;
     void **vh;
 
+    // Main constructor for a found value/holder:
     value_and_holder(instance *i, const detail::type_info *type, size_t vpos, size_t index) :
         inst{i}, index{index}, type{type},
         vh{inst->simple_layout ? inst->simple_value_holder : &inst->nonsimple.values_and_holders[vpos]}
     {}
 
+    // Default constructor (used to signal a value-and-holder not found by get_value_and_holder())
+    value_and_holder() : inst{nullptr} {}
+
     // Used for past-the-end iterator
     value_and_holder(size_t index) : index{index} {}
 
@@ -323,22 +327,26 @@ struct value_and_holder {
             ? inst->simple_holder_constructed
             : inst->nonsimple.status[index] & instance::status_holder_constructed;
     }
-    void set_holder_constructed() {
+    void set_holder_constructed(bool v = true) {
         if (inst->simple_layout)
-            inst->simple_holder_constructed = true;
+            inst->simple_holder_constructed = v;
-        else
+        else if (v)
             inst->nonsimple.status[index] |= instance::status_holder_constructed;
+        else
+            inst->nonsimple.status[index] &= (uint8_t) ~instance::status_holder_constructed;
     }
     bool instance_registered() const {
         return inst->simple_layout
             ? inst->simple_instance_registered
             : inst->nonsimple.status[index] & instance::status_instance_registered;
     }
-    void set_instance_registered() {
+    void set_instance_registered(bool v = true) {
         if (inst->simple_layout)
-            inst->simple_instance_registered = true;
+            inst->simple_instance_registered = v;
-        else
+        else if (v)
             inst->nonsimple.status[index] |= instance::status_instance_registered;
+        else
+            inst->nonsimple.status[index] &= (uint8_t) ~instance::status_instance_registered;
     }
 };
 
@@ -403,7 +411,7 @@ public:
  * The returned object should be short-lived: in particular, it must not outlive the called-upon
  * instance.
  */
-PYBIND11_NOINLINE inline value_and_holder instance::get_value_and_holder(const type_info *find_type /*= nullptr default in common.h*/) {
+PYBIND11_NOINLINE inline value_and_holder instance::get_value_and_holder(const type_info *find_type /*= nullptr default in common.h*/, bool throw_if_missing /*= true in common.h*/) {
     // Optimize common case:
     if (!find_type || Py_TYPE(this) == find_type->type)
         return value_and_holder(this, find_type, 0, 0);
@@ -413,6 +421,9 @@ PYBIND11_NOINLINE inline value_and_holder instance::get_value_and_holder(const t
     if (it != vhs.end())
         return *it;
 
+    if (!throw_if_missing)
+        return value_and_holder();
+
 #if defined(NDEBUG)
     pybind11_fail("pybind11::detail::instance::get_value_and_holder: "
             "type is not a pybind11 base of the given instance "
@@ -1454,7 +1465,7 @@ protected:
             throw cast_error("Unable to load a custom holder type from a default-holder instance");
     }
 
-    bool load_value(const value_and_holder &v_h) {
+    bool load_value(value_and_holder &&v_h) {
         if (v_h.holder_constructed()) {
             value = v_h.value_ptr();
             holder = v_h.holder<holder_type>();

include/pybind11/detail/common.h

@@ -441,8 +441,9 @@ struct instance {
     void deallocate_layout();
 
     /// Returns the value_and_holder wrapper for the given type (or the first, if `find_type`
-    /// omitted)
+    /// omitted).  Returns a default-constructed (with `.inst = nullptr`) object on failure if
-    value_and_holder get_value_and_holder(const type_info *find_type = nullptr);
+    /// `throw_if_missing` is false.
+    value_and_holder get_value_and_holder(const type_info *find_type = nullptr, bool throw_if_missing = true);
 
     /// Bit values for the non-simple status flags
     static constexpr uint8_t status_holder_constructed  = 1;

include/pybind11/detail/init.h

@@ -0,0 +1,274 @@
+/*
+    pybind11/detail/init.h: init factory function implementation and support code.
+
+    Copyright (c) 2017 Jason Rhinelander <jason@imaginary.ca>
+
+    All rights reserved. Use of this source code is governed by a
+    BSD-style license that can be found in the LICENSE file.
+*/
+
+#pragma once
+
+#include "class.h"
+
+NAMESPACE_BEGIN(PYBIND11_NAMESPACE)
+NAMESPACE_BEGIN(detail)
+NAMESPACE_BEGIN(initimpl)
+
+inline void no_nullptr(void *ptr) {
+    if (!ptr) throw type_error("pybind11::init(): factory function returned nullptr");
+}
+
+// Makes sure the `value` for the given value_and_holder is not preallocated (e.g. by a previous
+// old-style placement new `__init__` that requires a preallocated, uninitialized value).  If
+// preallocated, deallocate.  Returns the (null) value pointer reference ready for allocation.
+inline void *&deallocate(value_and_holder &v_h) {
+    if (v_h) v_h.type->dealloc(v_h);
+    return v_h.value_ptr();
+}
+
+PYBIND11_NOINLINE inline value_and_holder load_v_h(handle self_, type_info *tinfo) {
+    if (!self_ || !tinfo)
+        throw type_error("__init__(self, ...) called with invalid `self` argument");
+
+    auto *inst = reinterpret_cast<instance *>(self_.ptr());
+    auto result = inst->get_value_and_holder(tinfo, false);
+    if (!result.inst)
+        throw type_error("__init__(self, ...) called with invalid `self` argument");
+
+    return result;
+}
+
+
+// Implementing functions for py::init(...)
+template <typename Class> using Cpp = typename Class::type;
+template <typename Class> using Alias = typename Class::type_alias;
+template <typename Class> using Holder = typename Class::holder_type;
+
+template <typename Class> using is_alias_constructible = std::is_constructible<Alias<Class>, Cpp<Class> &&>;
+
+// Takes a Cpp pointer and returns true if it actually is a polymorphic Alias instance.
+template <typename Class, enable_if_t<Class::has_alias, int> = 0>
+bool is_alias(Cpp<Class> *ptr) {
+    return dynamic_cast<Alias<Class> *>(ptr) != nullptr;
+}
+// Failing fallback version of the above for a no-alias class (always returns false)
+template <typename /*Class*/>
+constexpr bool is_alias(void *) { return false; }
+
+// Attempts to constructs an alias using a `Alias(Cpp &&)` constructor.  This allows types with
+// an alias to provide only a single Cpp factory function as long as the Alias can be
+// constructed from an rvalue reference of the base Cpp type.  This means that Alias classes
+// can, when appropriate, simply define a `Alias(Cpp &&)` constructor rather than needing to
+// inherit all the base class constructors.
+template <typename Class>
+void construct_alias_from_cpp(std::true_type /*is_alias_constructible*/,
+                              value_and_holder &v_h, Cpp<Class> &&base) {
+    deallocate(v_h) = new Alias<Class>(std::move(base));
+}
+template <typename Class>
+[[noreturn]] void construct_alias_from_cpp(std::false_type /*!is_alias_constructible*/,
+                                           value_and_holder &, Cpp<Class> &&) {
+    throw type_error("pybind11::init(): unable to convert returned instance to required "
+                     "alias class: no `Alias<Class>(Class &&)` constructor available");
+}
+
+// Error-generating fallback for factories that don't match one of the below construction
+// mechanisms.
+template <typename Class>
+void construct(...) {
+    static_assert(!std::is_same<Class, Class>::value /* always false */,
+            "pybind11::init(): init function must return a compatible pointer, "
+            "holder, or value");
+}
+
+// Pointer return v1: the factory function returns a class pointer for a registered class.
+// If we don't need an alias (because this class doesn't have one, or because the final type is
+// inherited on the Python side) we can simply take over ownership.  Otherwise we need to try to
+// construct an Alias from the returned base instance.
+template <typename Class>
+void construct(value_and_holder &v_h, Cpp<Class> *ptr, bool need_alias) {
+    no_nullptr(ptr);
+    if (Class::has_alias && need_alias && !is_alias<Class>(ptr)) {
+        // We're going to try to construct an alias by moving the cpp type.  Whether or not
+        // that succeeds, we still need to destroy the original cpp pointer (either the
+        // moved away leftover, if the alias construction works, or the value itself if we
+        // throw an error), but we can't just call `delete ptr`: it might have a special
+        // deleter, or might be shared_from_this.  So we construct a holder around it as if
+        // it was a normal instance, then steal the holder away into a local variable; thus
+        // the holder and destruction happens when we leave the C++ scope, and the holder
+        // class gets to handle the destruction however it likes.
+        deallocate(v_h) = ptr;
+        v_h.set_instance_registered(true); // To prevent init_instance from registering it
+        v_h.type->init_instance(v_h.inst, nullptr); // Set up the holder
+        Holder<Class> temp_holder(std::move(v_h.holder<Holder<Class>>())); // Steal the holder
+        v_h.type->dealloc(v_h); // Destroys the moved-out holder remains, resets value ptr to null
+        v_h.set_instance_registered(false);
+
+        construct_alias_from_cpp<Class>(is_alias_constructible<Class>{}, v_h, std::move(*ptr));
+    }
+    else {
+        // Otherwise the type isn't inherited, so we don't need an Alias and can just store the Cpp
+        // pointer directory:
+        deallocate(v_h) = ptr;
+    }
+}
+
+// Pointer return v2: a factory that always returns an alias instance ptr.  We simply take over
+// ownership of the pointer.
+template <typename Class, enable_if_t<Class::has_alias, int> = 0>
+void construct(value_and_holder &v_h, Alias<Class> *alias_ptr, bool) {
+    no_nullptr(alias_ptr);
+    deallocate(v_h) = static_cast<Cpp<Class> *>(alias_ptr);
+}
+
+// Holder return: copy its pointer, and move or copy the returned holder into the new instance's
+// holder.  This also handles types like std::shared_ptr<T> and std::unique_ptr<T> where T is a
+// derived type (through those holder's implicit conversion from derived class holder constructors).
+template <typename Class>
+void construct(value_and_holder &v_h, Holder<Class> holder, bool need_alias) {
+    auto *ptr = holder_helper<Holder<Class>>::get(holder);
+    // If we need an alias, check that the held pointer is actually an alias instance
+    if (Class::has_alias && need_alias && !is_alias<Class>(ptr))
+        throw type_error("pybind11::init(): construction failed: returned holder-wrapped instance "
+                         "is not an alias instance");
+
+    deallocate(v_h) = ptr;
+    v_h.type->init_instance(v_h.inst, &holder);
+}
+
+// return-by-value version 1: returning a cpp class by value.  If the class has an alias and an
+// alias is required the alias must have an `Alias(Cpp &&)` constructor so that we can construct
+// the alias from the base when needed (i.e. because of Python-side inheritance).  When we don't
+// need it, we simply move-construct the cpp value into a new instance.
+template <typename Class>
+void construct(value_and_holder &v_h, Cpp<Class> &&result, bool need_alias) {
+    static_assert(std::is_move_constructible<Cpp<Class>>::value,
+        "pybind11::init() return-by-value factory function requires a movable class");
+    if (Class::has_alias && need_alias)
+        construct_alias_from_cpp<Class>(is_alias_constructible<Class>{}, v_h, std::move(result));
+    else
+        deallocate(v_h) = new Cpp<Class>(std::move(result));
+}
+
+// return-by-value version 2: returning a value of the alias type itself.  We move-construct an
+// Alias instance (even if no the python-side inheritance is involved).  The is intended for
+// cases where Alias initialization is always desired.
+template <typename Class>
+void construct(value_and_holder &v_h, Alias<Class> &&result, bool) {
+    static_assert(std::is_move_constructible<Alias<Class>>::value,
+        "pybind11::init() return-by-alias-value factory function requires a movable alias class");
+    deallocate(v_h) = new Alias<Class>(std::move(result));
+}
+
+// Implementation class for py::init(Func) and py::init(Func, AliasFunc)
+template <typename CFunc, typename AFuncIn, typename... Args> struct factory {
+private:
+    using CFuncType = typename std::remove_reference<CFunc>::type;
+    using AFunc = conditional_t<std::is_void<AFuncIn>::value, void_type, AFuncIn>;
+    using AFuncType = typename std::remove_reference<AFunc>::type;
+
+    CFuncType class_factory;
+    AFuncType alias_factory;
+
+public:
+    // Constructor with a single function/lambda to call; for classes without aliases or with
+    // aliases that can be move constructed from the base.
+    factory(CFunc &&f) : class_factory(std::forward<CFunc>(f)) {}
+
+    // Constructor with two functions/lambdas, for a class with distinct class/alias factories: the
+    // first is called when an alias is not needed, the second when the alias is needed.  Requires
+    // non-void AFunc.
+    factory(CFunc &&c, AFunc &&a) :
+        class_factory(std::forward<CFunc>(c)), alias_factory(std::forward<AFunc>(a)) {}
+
+    // Add __init__ definition for a class that either has no alias or has no separate alias
+    // factory; this always constructs the class itself.  If the class is registered with an alias
+    // type and an alias instance is needed (i.e. because the final type is a Python class
+    // inheriting from the C++ type) the returned value needs to either already be an alias
+    // instance, or the alias needs to be constructible from a `Class &&` argument.
+    template <typename Class, typename... Extra,
+              enable_if_t<!Class::has_alias || std::is_void<AFuncIn>::value, int> = 0>
+    void execute(Class &cl, const Extra&... extra) && {
+        auto *cl_type = get_type_info(typeid(Cpp<Class>));
+        #if defined(PYBIND11_CPP14)
+        cl.def("__init__", [cl_type, func = std::move(class_factory)]
+        #else
+        CFuncType &func = class_factory;
+        cl.def("__init__", [cl_type, func]
+        #endif
+        (handle self_, Args... args) {
+            auto v_h = load_v_h(self_, cl_type);
+            // If this value is already registered it must mean __init__ is invoked multiple times;
+            // we really can't support that in C++, so just ignore the second __init__.
+            if (v_h.instance_registered()) return;
+
+            construct<Class>(v_h, func(std::forward<Args>(args)...), Py_TYPE(v_h.inst) != cl_type->type);
+        }, extra...);
+    }
+
+    // Add __init__ definition for a class with an alias *and* distinct alias factory; the former is
+    // called when the `self` type passed to `__init__` is the direct class (i.e. not inherited), the latter
+    // when `self` is a Python-side subtype.
+    template <typename Class, typename... Extra,
+              enable_if_t<Class::has_alias && !std::is_void<AFuncIn>::value, int> = 0>
+    void execute(Class &cl, const Extra&... extra) && {
+        auto *cl_type = get_type_info(typeid(Cpp<Class>));
+
+        #if defined(PYBIND11_CPP14)
+        cl.def("__init__", [cl_type, class_func = std::move(class_factory), alias_func = std::move(alias_factory)]
+        #else
+        CFuncType &class_func = class_factory;
+        AFuncType &alias_func = alias_factory;
+        cl.def("__init__", [cl_type, class_func, alias_func]
+        #endif
+        (handle self_, Args... args) {
+            auto v_h = load_v_h(self_, cl_type);
+            if (v_h.instance_registered()) return; // (see comment above)
+
+            if (Py_TYPE(v_h.inst) == cl_type->type)
+                // If the instance type equals the registered type we don't have inheritance, so
+                // don't need the alias and can construct using the class function:
+                construct<Class>(v_h, class_func(std::forward<Args>(args)...), false);
+            else
+                construct<Class>(v_h, alias_func(std::forward<Args>(args)...), true);
+        }, extra...);
+    }
+};
+
+template <typename Func> using functype =
+    conditional_t<std::is_function<remove_reference_t<Func>>::value, remove_reference_t<Func> *,
+    conditional_t<is_function_pointer<remove_reference_t<Func>>::value, remove_reference_t<Func>,
+    Func>>;
+
+// Helper definition to infer the detail::initimpl::factory template types from a callable object
+template <typename Func, typename Return, typename... Args>
+factory<functype<Func>, void, Args...> func_decltype(Return (*)(Args...));
+
+// metatemplate that ensures the Class and Alias factories take identical arguments: we need to be
+// able to call either one with the given arguments (depending on the final instance type).
+template <typename Return1, typename Return2, typename... Args1, typename... Args2>
+inline constexpr bool require_matching_arguments(Return1 (*)(Args1...), Return2 (*)(Args2...)) {
+    static_assert(sizeof...(Args1) == sizeof...(Args2),
+        "pybind11::init(class_factory, alias_factory): class and alias factories must have identical argument signatures");
+    static_assert(all_of<std::is_same<Args1, Args2>...>::value,
+        "pybind11::init(class_factory, alias_factory): class and alias factories must have identical argument signatures");
+    return true;
+}
+
+// Unimplemented function provided only for its type signature (via `decltype`), which resolves to
+// the appropriate specialization of the above `init` struct with the appropriate function, argument
+// and return types.
+template <typename CFunc, typename AFunc,
+          typename CReturn, typename... CArgs, typename AReturn, typename... AArgs,
+          bool = require_matching_arguments((CReturn (*)(CArgs...)) nullptr, (AReturn (*)(AArgs...)) nullptr)>
+factory<functype<CFunc>, functype<AFunc>, CArgs...> func_decltype(CReturn (*)(CArgs...), AReturn (*)(AArgs...));
+
+// Resolves to the appropriate specialization of the `pybind11::detail::initimpl::factory<...>` for a
+// given init function or pair of class/alias init functions.
+template <typename... Func> using factory_t = decltype(func_decltype<Func...>(
+    (function_signature_t<Func> *) nullptr...));
+
+NAMESPACE_END(initimpl)
+NAMESPACE_END(detail)
+NAMESPACE_END(pybind11)

include/pybind11/pybind11.h

@@ -43,6 +43,7 @@
 #include "attr.h"
 #include "options.h"
 #include "detail/class.h"
+#include "detail/init.h"
 
 NAMESPACE_BEGIN(PYBIND11_NAMESPACE)
 
@@ -198,6 +199,8 @@ protected:
                 a.descr = strdup(a.value.attr("__repr__")().cast<std::string>().c_str());
         }
 
+        rec->is_constructor = !strcmp(rec->name, "__init__") || !strcmp(rec->name, "__setstate__");
+
         /* Generate a proper function signature */
         std::string signature;
         size_t type_depth = 0, char_index = 0, type_index = 0, arg_index = 0;
@@ -239,6 +242,12 @@ protected:
                                      .cast<std::string>() + ".";
 #endif
                     signature += tinfo->type->tp_name;
+                } else if (rec->is_constructor && arg_index == 0 && detail::same_type(typeid(handle), *t) && rec->scope) {
+                    // A py::init(...) constructor takes `self` as a `handle`; rewrite it to the type
+#if defined(PYPY_VERSION)
+                    signature += rec->scope.attr("__module__").cast<std::string>() + ".";
+#endif
+                    signature += ((PyTypeObject *) rec->scope.ptr())->tp_name;
                 } else {
                     std::string tname(t->name());
                     detail::clean_type_id(tname);
@@ -267,7 +276,6 @@ protected:
 #endif
         rec->signature = strdup(signature.c_str());
         rec->args.shrink_to_fit();
-        rec->is_constructor = !strcmp(rec->name, "__init__") || !strcmp(rec->name, "__setstate__");
         rec->nargs = (std::uint16_t) args;
 
         if (rec->sibling && PYBIND11_INSTANCE_METHOD_CHECK(rec->sibling.ptr()))
@@ -709,7 +717,11 @@ protected:
         } else {
             if (overloads->is_constructor) {
                 auto tinfo = get_type_info((PyTypeObject *) overloads->scope.ptr());
-                tinfo->init_instance(reinterpret_cast<instance *>(parent.ptr()), nullptr);
+                auto *pi = reinterpret_cast<instance *>(parent.ptr());
+                auto v_h = pi->get_value_and_holder(tinfo);
+                if (!v_h.holder_constructed()) {
+                    tinfo->init_instance(pi, nullptr);
+                }
             }
             return result.ptr();
         }
@@ -1045,6 +1057,12 @@ public:
         return *this;
     }
 
+    template <typename... Args, typename... Extra>
+    class_ &def(detail::initimpl::factory<Args...> &&init, const Extra&... extra) {
+        std::move(init).execute(*this, extra...);
+        return *this;
+    }
+
     template <typename Func> class_& def_buffer(Func &&func) {
         struct capture { Func func; };
         capture *ptr = new capture { std::forward<Func>(func) };
@@ -1225,12 +1243,16 @@ private:
     }
 
     /// Deallocates an instance; via holder, if constructed; otherwise via operator delete.
-    static void dealloc(const detail::value_and_holder &v_h) {
+    static void dealloc(detail::value_and_holder &v_h) {
-        if (v_h.holder_constructed())
+        if (v_h.holder_constructed()) {
             v_h.holder<holder_type>().~holder_type();
-        else
+            v_h.set_holder_constructed(false);
+        }
+        else {
             detail::call_operator_delete(v_h.value_ptr<type>(), v_h.type->type_size);
         }
+        v_h.value_ptr() = nullptr;
+    }
 
     static detail::function_record *get_function_record(handle h) {
         h = detail::get_function(h);
@@ -1328,6 +1350,23 @@ private:
     handle m_parent;
 };
 
+/// Binds an existing constructor taking arguments Args...
+template <typename... Args> detail::init<Args...> init() { return detail::init<Args...>(); }
+/// Like `init<Args...>()`, but the instance is always constructed through the alias class (even
+/// when not inheriting on the Python side).
+template <typename... Args> detail::init_alias<Args...> init_alias() { return detail::init_alias<Args...>(); }
+
+/// Binds a factory function as a constructor
+template <typename Func, typename Ret = detail::initimpl::factory_t<Func>>
+Ret init(Func &&f) { return {std::forward<Func>(f)}; }
+
+/// Dual-argument factory function: the first function is called when no alias is needed, the second
+/// when an alias is needed (i.e. due to python-side inheritance).  Arguments must be identical.
+template <typename CFunc, typename AFunc, typename Ret = detail::initimpl::factory_t<CFunc, AFunc>>
+Ret init(CFunc &&c, AFunc &&a) {
+    return {std::forward<CFunc>(c), std::forward<AFunc>(a)};
+}
+
 NAMESPACE_BEGIN(detail)
 template <typename... Args> struct init {
     template <typename Class, typename... Extra, enable_if_t<!Class::has_alias, int> = 0>
@@ -1431,9 +1470,6 @@ struct iterator_state {
 
 NAMESPACE_END(detail)
 
-template <typename... Args> detail::init<Args...> init() { return detail::init<Args...>(); }
-template <typename... Args> detail::init_alias<Args...> init_alias() { return detail::init_alias<Args...>(); }
-
 /// Makes a python iterator from a first and past-the-end C++ InputIterator.
 template <return_value_policy Policy = return_value_policy::reference_internal,
           typename Iterator,

setup.py

@@ -15,6 +15,7 @@ else:
         'include/pybind11/detail/class.h',
         'include/pybind11/detail/common.h',
         'include/pybind11/detail/descr.h',
+        'include/pybind11/detail/init.h',
         'include/pybind11/detail/typeid.h'
         'include/pybind11/attr.h',
         'include/pybind11/buffer_info.h',

tests/CMakeLists.txt

@@ -39,6 +39,7 @@ set(PYBIND11_TEST_FILES
   test_enum.cpp
   test_eval.cpp
   test_exceptions.cpp
+  test_factory_constructors.cpp
   test_kwargs_and_defaults.cpp
   test_local_bindings.cpp
   test_methods_and_attributes.cpp

tests/conftest.py

@@ -211,6 +211,8 @@ def pytest_namespace():
         'requires_eigen_and_scipy': skipif(not have_eigen or not scipy,
                                            reason="eigen and/or scipy are not installed"),
         'unsupported_on_pypy': skipif(pypy, reason="unsupported on PyPy"),
+        'unsupported_on_py2': skipif(sys.version_info.major < 3,
+                                     reason="unsupported on Python 2.x"),
         'gc_collect': gc_collect
     }
 

tests/test_factory_constructors.cpp

@@ -0,0 +1,336 @@
+/*
+    tests/test_factory_constructors.cpp -- tests construction from a factory function
+                                           via py::init_factory()
+
+    Copyright (c) 2017 Jason Rhinelander <jason@imaginary.ca>
+
+    All rights reserved. Use of this source code is governed by a
+    BSD-style license that can be found in the LICENSE file.
+*/
+
+#include "pybind11_tests.h"
+#include "constructor_stats.h"
+#include <cmath>
+
+// Classes for testing python construction via C++ factory function:
+// Not publically constructible, copyable, or movable:
+class TestFactory1 {
+    friend class TestFactoryHelper;
+    TestFactory1() : value("(empty)") { print_default_created(this); }
+    TestFactory1(int v) : value(std::to_string(v)) { print_created(this, value); }
+    TestFactory1(std::string v) : value(std::move(v)) { print_created(this, value); }
+    TestFactory1(TestFactory1 &&) = delete;
+    TestFactory1(const TestFactory1 &) = delete;
+    TestFactory1 &operator=(TestFactory1 &&) = delete;
+    TestFactory1 &operator=(const TestFactory1 &) = delete;
+public:
+    std::string value;
+    ~TestFactory1() { print_destroyed(this); }
+};
+// Non-public construction, but moveable:
+class TestFactory2 {
+    friend class TestFactoryHelper;
+    TestFactory2() : value("(empty2)") { print_default_created(this); }
+    TestFactory2(int v) : value(std::to_string(v)) { print_created(this, value); }
+    TestFactory2(std::string v) : value(std::move(v)) { print_created(this, value); }
+public:
+    TestFactory2(TestFactory2 &&m) { value = std::move(m.value); print_move_created(this); }
+    TestFactory2 &operator=(TestFactory2 &&m) { value = std::move(m.value); print_move_assigned(this); return *this; }
+    std::string value;
+    ~TestFactory2() { print_destroyed(this); }
+};
+// Mixed direct/factory construction:
+class TestFactory3 {
+protected:
+    friend class TestFactoryHelper;
+    TestFactory3() : value("(empty3)") { print_default_created(this); }
+    TestFactory3(int v) : value(std::to_string(v)) { print_created(this, value); }
+public:
+    TestFactory3(std::string v) : value(std::move(v)) { print_created(this, value); }
+    TestFactory3(TestFactory3 &&m) { value = std::move(m.value); print_move_created(this); }
+    TestFactory3 &operator=(TestFactory3 &&m) { value = std::move(m.value); print_move_assigned(this); return *this; }
+    std::string value;
+    virtual ~TestFactory3() { print_destroyed(this); }
+};
+// Inheritance test
+class TestFactory4 : public TestFactory3 {
+public:
+    TestFactory4() : TestFactory3() { print_default_created(this); }
+    TestFactory4(int v) : TestFactory3(v) { print_created(this, v); }
+    virtual ~TestFactory4() { print_destroyed(this); }
+};
+// Another class for an invalid downcast test
+class TestFactory5 : public TestFactory3 {
+public:
+    TestFactory5(int i) : TestFactory3(i) { print_created(this, i); }
+    virtual ~TestFactory5() { print_destroyed(this); }
+};
+
+class TestFactory6 {
+protected:
+    int value;
+    bool alias = false;
+public:
+    TestFactory6(int i) : value{i} { print_created(this, i); }
+    TestFactory6(TestFactory6 &&f) { print_move_created(this); value = f.value; alias = f.alias; }
+    TestFactory6(const TestFactory6 &f) { print_copy_created(this); value = f.value; alias = f.alias; }
+    virtual ~TestFactory6() { print_destroyed(this); }
+    virtual int get() { return value; }
+    bool has_alias() { return alias; }
+};
+class PyTF6 : public TestFactory6 {
+public:
+    // Special constructor that allows the factory to construct a PyTF6 from a TestFactory6 only
+    // when an alias is needed:
+    PyTF6(TestFactory6 &&base) : TestFactory6(std::move(base)) { alias = true; print_created(this, "move", value); }
+    PyTF6(int i) : TestFactory6(i) { alias = true; print_created(this, i); }
+    PyTF6(PyTF6 &&f) : TestFactory6(std::move(f)) { print_move_created(this); }
+    PyTF6(const PyTF6 &f) : TestFactory6(f) { print_copy_created(this); }
+    PyTF6(std::string s) : TestFactory6((int) s.size()) { alias = true; print_created(this, s); }
+    virtual ~PyTF6() { print_destroyed(this); }
+    int get() override { PYBIND11_OVERLOAD(int, TestFactory6, get, /*no args*/); }
+};
+
+class TestFactory7 {
+protected:
+    int value;
+    bool alias = false;
+public:
+    TestFactory7(int i) : value{i} { print_created(this, i); }
+    TestFactory7(TestFactory7 &&f) { print_move_created(this); value = f.value; alias = f.alias; }
+    TestFactory7(const TestFactory7 &f) { print_copy_created(this); value = f.value; alias = f.alias; }
+    virtual ~TestFactory7() { print_destroyed(this); }
+    virtual int get() { return value; }
+    bool has_alias() { return alias; }
+};
+class PyTF7 : public TestFactory7 {
+public:
+    PyTF7(int i) : TestFactory7(i) { alias = true; print_created(this, i); }
+    PyTF7(PyTF7 &&f) : TestFactory7(std::move(f)) { print_move_created(this); }
+    PyTF7(const PyTF7 &f) : TestFactory7(f) { print_copy_created(this); }
+    virtual ~PyTF7() { print_destroyed(this); }
+    int get() override { PYBIND11_OVERLOAD(int, TestFactory7, get, /*no args*/); }
+};
+
+
+class TestFactoryHelper {
+public:
+    // Non-movable, non-copyable type:
+    // Return via pointer:
+    static TestFactory1 *construct1() { return new TestFactory1(); }
+    // Holder:
+    static std::unique_ptr<TestFactory1> construct1(int a) { return std::unique_ptr<TestFactory1>(new TestFactory1(a)); }
+    // pointer again
+    static TestFactory1 *construct1_string(std::string a) { return new TestFactory1(a); }
+
+    // Moveable type:
+    // pointer:
+    static TestFactory2 *construct2() { return new TestFactory2(); }
+    // holder:
+    static std::unique_ptr<TestFactory2> construct2(int a) { return std::unique_ptr<TestFactory2>(new TestFactory2(a)); }
+    // by value moving:
+    static TestFactory2 construct2(std::string a) { return TestFactory2(a); }
+
+    // shared_ptr holder type:
+    // pointer:
+    static TestFactory3 *construct3() { return new TestFactory3(); }
+    // holder:
+    static std::shared_ptr<TestFactory3> construct3(int a) { return std::shared_ptr<TestFactory3>(new TestFactory3(a)); }
+};
+
+TEST_SUBMODULE(factory_constructors, m) {
+
+    // Define various trivial types to allow simpler overload resolution:
+    py::module m_tag = m.def_submodule("tag");
+#define MAKE_TAG_TYPE(Name) \
+    struct Name##_tag {}; \
+    py::class_<Name##_tag>(m_tag, #Name "_tag").def(py::init<>()); \
+    m_tag.attr(#Name) = py::cast(Name##_tag{})
+    MAKE_TAG_TYPE(pointer);
+    MAKE_TAG_TYPE(unique_ptr);
+    MAKE_TAG_TYPE(move);
+    MAKE_TAG_TYPE(shared_ptr);
+    MAKE_TAG_TYPE(derived);
+    MAKE_TAG_TYPE(TF4);
+    MAKE_TAG_TYPE(TF5);
+    MAKE_TAG_TYPE(null_ptr);
+    MAKE_TAG_TYPE(base);
+    MAKE_TAG_TYPE(invalid_base);
+    MAKE_TAG_TYPE(alias);
+    MAKE_TAG_TYPE(unaliasable);
+    MAKE_TAG_TYPE(mixed);
+
+    // test_init_factory_basic, test_bad_type
+    py::class_<TestFactory1>(m, "TestFactory1")
+        .def(py::init([](unique_ptr_tag, int v) { return TestFactoryHelper::construct1(v); }))
+        .def(py::init(&TestFactoryHelper::construct1_string)) // raw function pointer
+        .def(py::init([](pointer_tag) { return TestFactoryHelper::construct1(); }))
+        .def(py::init([](py::handle, int v, py::handle) { return TestFactoryHelper::construct1(v); }))
+        .def_readwrite("value", &TestFactory1::value)
+        ;
+    py::class_<TestFactory2>(m, "TestFactory2")
+        .def(py::init([](pointer_tag, int v) { return TestFactoryHelper::construct2(v); }))
+        .def(py::init([](unique_ptr_tag, std::string v) { return TestFactoryHelper::construct2(v); }))
+        .def(py::init([](move_tag) { return TestFactoryHelper::construct2(); }))
+        .def_readwrite("value", &TestFactory2::value)
+        ;
+
+    // Stateful & reused:
+    int c = 1;
+    auto c4a = [c](pointer_tag, TF4_tag, int a) { return new TestFactory4(a);};
+
+    // test_init_factory_basic, test_init_factory_casting
+    py::class_<TestFactory3, std::shared_ptr<TestFactory3>>(m, "TestFactory3")
+        .def(py::init([](pointer_tag, int v) { return TestFactoryHelper::construct3(v); }))
+        .def(py::init([](shared_ptr_tag) { return TestFactoryHelper::construct3(); }))
+        .def("__init__", [](TestFactory3 &self, std::string v) { new (&self) TestFactory3(v); }) // placement-new ctor
+
+        // factories returning a derived type:
+        .def(py::init(c4a)) // derived ptr
+        .def(py::init([](pointer_tag, TF5_tag, int a) { return new TestFactory5(a); }))
+        // derived shared ptr:
+        .def(py::init([](shared_ptr_tag, TF4_tag, int a) { return std::make_shared<TestFactory4>(a); }))
+        .def(py::init([](shared_ptr_tag, TF5_tag, int a) { return std::make_shared<TestFactory5>(a); }))
+
+        // Returns nullptr:
+        .def(py::init([](null_ptr_tag) { return (TestFactory3 *) nullptr; }))
+
+        .def_readwrite("value", &TestFactory3::value)
+        ;
+
+    // test_init_factory_casting
+    py::class_<TestFactory4, TestFactory3, std::shared_ptr<TestFactory4>>(m, "TestFactory4")
+        .def(py::init(c4a)) // pointer
+        ;
+
+    // Doesn't need to be registered, but registering makes getting ConstructorStats easier:
+    py::class_<TestFactory5, TestFactory3, std::shared_ptr<TestFactory5>>(m, "TestFactory5");
+
+    // test_init_factory_alias
+    // Alias testing
+    py::class_<TestFactory6, PyTF6>(m, "TestFactory6")
+        .def(py::init([](base_tag, int i) { return TestFactory6(i); }))
+        .def(py::init([](alias_tag, int i) { return PyTF6(i); }))
+        .def(py::init([](alias_tag, std::string s) { return PyTF6(s); }))
+        .def(py::init([](alias_tag, pointer_tag, int i) { return new PyTF6(i); }))
+        .def(py::init([](base_tag, pointer_tag, int i) { return new TestFactory6(i); }))
+        .def(py::init([](base_tag, alias_tag, pointer_tag, int i) { return (TestFactory6 *) new PyTF6(i); }))
+
+        .def("get", &TestFactory6::get)
+        .def("has_alias", &TestFactory6::has_alias)
+
+        .def_static("get_cstats", &ConstructorStats::get<TestFactory6>, py::return_value_policy::reference)
+        .def_static("get_alias_cstats", &ConstructorStats::get<PyTF6>, py::return_value_policy::reference)
+        ;
+
+    // test_init_factory_dual
+    // Separate alias constructor testing
+    py::class_<TestFactory7, PyTF7, std::shared_ptr<TestFactory7>>(m, "TestFactory7")
+        .def(py::init(
+            [](int i) { return TestFactory7(i); },
+            [](int i) { return PyTF7(i); }))
+        .def(py::init(
+            [](pointer_tag, int i) { return new TestFactory7(i); },
+            [](pointer_tag, int i) { return new PyTF7(i); }))
+        .def(py::init(
+            [](mixed_tag, int i) { return new TestFactory7(i); },
+            [](mixed_tag, int i) { return PyTF7(i); }))
+        .def(py::init(
+            [](mixed_tag, std::string s) { return TestFactory7((int) s.size()); },
+            [](mixed_tag, std::string s) { return new PyTF7((int) s.size()); }))
+        .def(py::init(
+            [](base_tag, pointer_tag, int i) { return new TestFactory7(i); },
+            [](base_tag, pointer_tag, int i) { return (TestFactory7 *) new PyTF7(i); }))
+        .def(py::init(
+            [](alias_tag, pointer_tag, int i) { return new PyTF7(i); },
+            [](alias_tag, pointer_tag, int i) { return new PyTF7(10*i); }))
+        .def(py::init(
+            [](shared_ptr_tag, base_tag, int i) { return std::make_shared<TestFactory7>(i); },
+            [](shared_ptr_tag, base_tag, int i) { auto *p = new PyTF7(i); return std::shared_ptr<TestFactory7>(p); }))
+        .def(py::init(
+            [](shared_ptr_tag, invalid_base_tag, int i) { return std::make_shared<TestFactory7>(i); },
+            [](shared_ptr_tag, invalid_base_tag, int i) { return std::make_shared<TestFactory7>(i); })) // <-- invalid alias factory
+
+        .def("get", &TestFactory7::get)
+        .def("has_alias", &TestFactory7::has_alias)
+
+        .def_static("get_cstats", &ConstructorStats::get<TestFactory7>, py::return_value_policy::reference)
+        .def_static("get_alias_cstats", &ConstructorStats::get<PyTF7>, py::return_value_policy::reference)
+        ;
+
+    // test_placement_new_alternative
+    // Class with a custom new operator but *without* a placement new operator (issue #948)
+    class NoPlacementNew {
+    public:
+        NoPlacementNew(int i) : i(i) { }
+        static void *operator new(std::size_t s) {
+            auto *p = ::operator new(s);
+            py::print("operator new called, returning", reinterpret_cast<uintptr_t>(p));
+            return p;
+        }
+        static void operator delete(void *p) {
+            py::print("operator delete called on", reinterpret_cast<uintptr_t>(p));
+            ::operator delete(p);
+        }
+        int i;
+    };
+    // Workaround for a `py::init<args>` on a class without placement new support
+    py::class_<NoPlacementNew>(m, "NoPlacementNew")
+        .def(py::init([]() { return new NoPlacementNew(100); }))
+        .def_readwrite("i", &NoPlacementNew::i)
+        ;
+
+
+    // test_reallocations
+    // Class that has verbose operator_new/operator_delete calls
+    struct NoisyAlloc {
+        NoisyAlloc(int i) { py::print(py::str("NoisyAlloc(int {})").format(i)); }
+        NoisyAlloc(double d) { py::print(py::str("NoisyAlloc(double {})").format(d)); }
+        ~NoisyAlloc() { py::print("~NoisyAlloc()"); }
+
+        static void *operator new(size_t s) { py::print("noisy new"); return ::operator new(s); }
+        static void *operator new(size_t, void *p) { py::print("noisy placement new"); return p; }
+        static void operator delete(void *p, size_t) { py::print("noisy delete"); ::operator delete(p); }
+        static void operator delete(void *, void *) { py::print("noisy placement delete"); }
+#if defined(_MSC_VER) && _MSC_VER < 1910
+        // MSVC 2015 bug: the above "noisy delete" isn't invoked (fixed in MSVC 2017)
+        static void operator delete(void *p) { py::print("noisy delete"); ::operator delete(p); }
+#endif
+    };
+    py::class_<NoisyAlloc>(m, "NoisyAlloc")
+        // Since these overloads have the same number of arguments, the dispatcher will try each of
+        // them until the arguments convert.  Thus we can get a pre-allocation here when passing a
+        // single non-integer:
+        .def("__init__", [](NoisyAlloc *a, int i) { new (a) NoisyAlloc(i); }) // Regular constructor, runs first, requires preallocation
+        .def(py::init([](double d) { return new NoisyAlloc(d); }))
+
+        // The two-argument version: first the factory pointer overload.
+        .def(py::init([](int i, int) { return new NoisyAlloc(i); }))
+        // Return-by-value:
+        .def(py::init([](double d, int) { return NoisyAlloc(d); }))
+        // Old-style placement new init; requires preallocation
+        .def("__init__", [](NoisyAlloc &a, double d, double) { new (&a) NoisyAlloc(d); })
+        // Requires deallocation of previous overload preallocated value:
+        .def(py::init([](int i, double) { return new NoisyAlloc(i); }))
+        // Regular again: requires yet another preallocation
+        .def("__init__", [](NoisyAlloc &a, int i, std::string) { new (&a) NoisyAlloc(i); })
+        ;
+
+
+
+
+    // static_assert testing (the following def's should all fail with appropriate compilation errors):
+#if 0
+    struct BadF1Base {};
+    struct BadF1 : BadF1Base {};
+    struct PyBadF1 : BadF1 {};
+    py::class_<BadF1, PyBadF1, std::shared_ptr<BadF1>> bf1(m, "BadF1");
+    // wrapped factory function must return a compatible pointer, holder, or value
+    bf1.def(py::init([]() { return 3; }));
+    // incompatible factory function pointer return type
+    bf1.def(py::init([]() { static int three = 3; return &three; }));
+    // incompatible factory function std::shared_ptr<T> return type: cannot convert shared_ptr<T> to holder
+    // (non-polymorphic base)
+    bf1.def(py::init([]() { return std::shared_ptr<BadF1Base>(new BadF1()); }));
+#endif
+}

tests/test_factory_constructors.py

@@ -0,0 +1,448 @@
+import pytest
+import re
+
+from pybind11_tests import factory_constructors as m
+from pybind11_tests.factory_constructors import tag
+from pybind11_tests import ConstructorStats
+
+
+def test_init_factory_basic():
+    """Tests py::init_factory() wrapper around various ways of returning the object"""
+
+    cstats = [ConstructorStats.get(c) for c in [m.TestFactory1, m.TestFactory2, m.TestFactory3]]
+    cstats[0].alive()  # force gc
+    n_inst = ConstructorStats.detail_reg_inst()
+
+    x1 = m.TestFactory1(tag.unique_ptr, 3)
+    assert x1.value == "3"
+    y1 = m.TestFactory1(tag.pointer)
+    assert y1.value == "(empty)"
+    z1 = m.TestFactory1("hi!")
+    assert z1.value == "hi!"
+
+    assert ConstructorStats.detail_reg_inst() == n_inst + 3
+
+    x2 = m.TestFactory2(tag.move)
+    assert x2.value == "(empty2)"
+    y2 = m.TestFactory2(tag.pointer, 7)
+    assert y2.value == "7"
+    z2 = m.TestFactory2(tag.unique_ptr, "hi again")
+    assert z2.value == "hi again"
+
+    assert ConstructorStats.detail_reg_inst() == n_inst + 6
+
+    x3 = m.TestFactory3(tag.shared_ptr)
+    assert x3.value == "(empty3)"
+    y3 = m.TestFactory3(tag.pointer, 42)
+    assert y3.value == "42"
+    z3 = m.TestFactory3("bye")
+    assert z3.value == "bye"
+
+    with pytest.raises(TypeError) as excinfo:
+        m.TestFactory3(tag.null_ptr)
+    assert (str(excinfo.value) ==
+            "pybind11::init(): factory function returned nullptr")
+
+    assert [i.alive() for i in cstats] == [3, 3, 3]
+    assert ConstructorStats.detail_reg_inst() == n_inst + 9
+
+    del x1, y2, y3, z3
+    assert [i.alive() for i in cstats] == [2, 2, 1]
+    assert ConstructorStats.detail_reg_inst() == n_inst + 5
+    del x2, x3, y1, z1, z2
+    assert [i.alive() for i in cstats] == [0, 0, 0]
+    assert ConstructorStats.detail_reg_inst() == n_inst
+
+    assert [i.values() for i in cstats] == [
+        ["3", "hi!"],
+        ["7", "hi again"],
+        ["42", "bye"]
+    ]
+    assert [i.default_constructions for i in cstats] == [1, 1, 1]
+
+
+def test_init_factory_signature(msg):
+    with pytest.raises(TypeError) as excinfo:
+        m.TestFactory1("invalid", "constructor", "arguments")
+    assert msg(excinfo.value) == """
+        __init__(): incompatible constructor arguments. The following argument types are supported:
+            1. m.factory_constructors.TestFactory1(arg0: m.factory_constructors.tag.unique_ptr_tag, arg1: int)
+            2. m.factory_constructors.TestFactory1(arg0: str)
+            3. m.factory_constructors.TestFactory1(arg0: m.factory_constructors.tag.pointer_tag)
+            4. m.factory_constructors.TestFactory1(arg0: handle, arg1: int, arg2: handle)
+
+        Invoked with: 'invalid', 'constructor', 'arguments'
+    """  # noqa: E501 line too long
+
+    assert msg(m.TestFactory1.__init__.__doc__) == """
+        __init__(*args, **kwargs)
+        Overloaded function.
+
+        1. __init__(self: m.factory_constructors.TestFactory1, arg0: m.factory_constructors.tag.unique_ptr_tag, arg1: int) -> None
+
+        2. __init__(self: m.factory_constructors.TestFactory1, arg0: str) -> None
+
+        3. __init__(self: m.factory_constructors.TestFactory1, arg0: m.factory_constructors.tag.pointer_tag) -> None
+
+        4. __init__(self: m.factory_constructors.TestFactory1, arg0: handle, arg1: int, arg2: handle) -> None
+    """  # noqa: E501 line too long
+
+
+def test_init_factory_casting():
+    """Tests py::init_factory() wrapper with various upcasting and downcasting returns"""
+
+    cstats = [ConstructorStats.get(c) for c in [m.TestFactory3, m.TestFactory4, m.TestFactory5]]
+    cstats[0].alive()  # force gc
+    n_inst = ConstructorStats.detail_reg_inst()
+
+    # Construction from derived references:
+    a = m.TestFactory3(tag.pointer, tag.TF4, 4)
+    assert a.value == "4"
+    b = m.TestFactory3(tag.shared_ptr, tag.TF4, 5)
+    assert b.value == "5"
+    c = m.TestFactory3(tag.pointer, tag.TF5, 6)
+    assert c.value == "6"
+    d = m.TestFactory3(tag.shared_ptr, tag.TF5, 7)
+    assert d.value == "7"
+
+    assert ConstructorStats.detail_reg_inst() == n_inst + 4
+
+    # Shared a lambda with TF3:
+    e = m.TestFactory4(tag.pointer, tag.TF4, 8)
+    assert e.value == "8"
+
+    assert ConstructorStats.detail_reg_inst() == n_inst + 5
+    assert [i.alive() for i in cstats] == [5, 3, 2]
+
+    del a
+    assert [i.alive() for i in cstats] == [4, 2, 2]
+    assert ConstructorStats.detail_reg_inst() == n_inst + 4
+
+    del b, c, e
+    assert [i.alive() for i in cstats] == [1, 0, 1]
+    assert ConstructorStats.detail_reg_inst() == n_inst + 1
+
+    del d
+    assert [i.alive() for i in cstats] == [0, 0, 0]
+    assert ConstructorStats.detail_reg_inst() == n_inst
+
+    assert [i.values() for i in cstats] == [
+        ["4", "5", "6", "7", "8"],
+        ["4", "5", "8"],
+        ["6", "7"]
+    ]
+
+
+def test_init_factory_alias():
+    """Tests py::init_factory() wrapper with value conversions and alias types"""
+
+    cstats = [m.TestFactory6.get_cstats(), m.TestFactory6.get_alias_cstats()]
+    cstats[0].alive()  # force gc
+    n_inst = ConstructorStats.detail_reg_inst()
+
+    a = m.TestFactory6(tag.base, 1)
+    assert a.get() == 1
+    assert not a.has_alias()
+    b = m.TestFactory6(tag.alias, "hi there")
+    assert b.get() == 8
+    assert b.has_alias()
+    c = m.TestFactory6(tag.alias, 3)
+    assert c.get() == 3
+    assert c.has_alias()
+    d = m.TestFactory6(tag.alias, tag.pointer, 4)
+    assert d.get() == 4
+    assert d.has_alias()
+    e = m.TestFactory6(tag.base, tag.pointer, 5)
+    assert e.get() == 5
+    assert not e.has_alias()
+    f = m.TestFactory6(tag.base, tag.alias, tag.pointer, 6)
+    assert f.get() == 6
+    assert f.has_alias()
+
+    assert ConstructorStats.detail_reg_inst() == n_inst + 6
+    assert [i.alive() for i in cstats] == [6, 4]
+
+    del a, b, e
+    assert [i.alive() for i in cstats] == [3, 3]
+    assert ConstructorStats.detail_reg_inst() == n_inst + 3
+    del f, c, d
+    assert [i.alive() for i in cstats] == [0, 0]
+    assert ConstructorStats.detail_reg_inst() == n_inst
+
+    class MyTest(m.TestFactory6):
+        def __init__(self, *args):
+            m.TestFactory6.__init__(self, *args)
+
+        def get(self):
+            return -5 + m.TestFactory6.get(self)
+
+    # Return Class by value, moved into new alias:
+    z = MyTest(tag.base, 123)
+    assert z.get() == 118
+    assert z.has_alias()
+
+    # Return alias by value, moved into new alias:
+    y = MyTest(tag.alias, "why hello!")
+    assert y.get() == 5
+    assert y.has_alias()
+
+    # Return Class by pointer, moved into new alias then original destroyed:
+    x = MyTest(tag.base, tag.pointer, 47)
+    assert x.get() == 42
+    assert x.has_alias()
+
+    assert ConstructorStats.detail_reg_inst() == n_inst + 3
+    assert [i.alive() for i in cstats] == [3, 3]
+    del x, y, z
+    assert [i.alive() for i in cstats] == [0, 0]
+    assert ConstructorStats.detail_reg_inst() == n_inst
+
+    assert [i.values() for i in cstats] == [
+        ["1", "8", "3", "4", "5", "6", "123", "10", "47"],
+        ["hi there", "3", "4", "6", "move", "123", "why hello!", "move", "47"]
+    ]
+
+
+def test_init_factory_dual():
+    """Tests init factory functions with dual main/alias factory functions"""
+    from pybind11_tests.factory_constructors import TestFactory7
+
+    cstats = [TestFactory7.get_cstats(), TestFactory7.get_alias_cstats()]
+    cstats[0].alive()  # force gc
+    n_inst = ConstructorStats.detail_reg_inst()
+
+    class PythFactory7(TestFactory7):
+        def get(self):
+            return 100 + TestFactory7.get(self)
+
+    a1 = TestFactory7(1)
+    a2 = PythFactory7(2)
+    assert a1.get() == 1
+    assert a2.get() == 102
+    assert not a1.has_alias()
+    assert a2.has_alias()
+
+    b1 = TestFactory7(tag.pointer, 3)
+    b2 = PythFactory7(tag.pointer, 4)
+    assert b1.get() == 3
+    assert b2.get() == 104
+    assert not b1.has_alias()
+    assert b2.has_alias()
+
+    c1 = TestFactory7(tag.mixed, 5)
+    c2 = PythFactory7(tag.mixed, 6)
+    assert c1.get() == 5
+    assert c2.get() == 106
+    assert not c1.has_alias()
+    assert c2.has_alias()
+
+    d1 = TestFactory7(tag.base, tag.pointer, 7)
+    d2 = PythFactory7(tag.base, tag.pointer, 8)
+    assert d1.get() == 7
+    assert d2.get() == 108
+    assert not d1.has_alias()
+    assert d2.has_alias()
+
+    # Both return an alias; the second multiplies the value by 10:
+    e1 = TestFactory7(tag.alias, tag.pointer, 9)
+    e2 = PythFactory7(tag.alias, tag.pointer, 10)
+    assert e1.get() == 9
+    assert e2.get() == 200
+    assert e1.has_alias()
+    assert e2.has_alias()
+
+    f1 = TestFactory7(tag.shared_ptr, tag.base, 11)
+    f2 = PythFactory7(tag.shared_ptr, tag.base, 12)
+    assert f1.get() == 11
+    assert f2.get() == 112
+    assert not f1.has_alias()
+    assert f2.has_alias()
+
+    g1 = TestFactory7(tag.shared_ptr, tag.invalid_base, 13)
+    assert g1.get() == 13
+    assert not g1.has_alias()
+    with pytest.raises(TypeError) as excinfo:
+        PythFactory7(tag.shared_ptr, tag.invalid_base, 14)
+    assert (str(excinfo.value) ==
+            "pybind11::init(): construction failed: returned holder-wrapped instance is not an "
+            "alias instance")
+
+    assert [i.alive() for i in cstats] == [13, 7]
+    assert ConstructorStats.detail_reg_inst() == n_inst + 13
+
+    del a1, a2, b1, d1, e1, e2
+    assert [i.alive() for i in cstats] == [7, 4]
+    assert ConstructorStats.detail_reg_inst() == n_inst + 7
+    del b2, c1, c2, d2, f1, f2, g1
+    assert [i.alive() for i in cstats] == [0, 0]
+    assert ConstructorStats.detail_reg_inst() == n_inst
+
+    assert [i.values() for i in cstats] == [
+        ["1", "2", "3", "4", "5", "6", "7", "8", "9", "100", "11", "12", "13", "14"],
+        ["2", "4", "6", "8", "9", "100", "12"]
+    ]
+
+
+def test_no_placement_new(capture):
+    """Tests a workaround for `py::init<...>` with a class that doesn't support placement new."""
+    with capture:
+        b = m.NoPlacementNew()
+
+    found = re.search(r'^operator new called, returning (\d+)\n$', str(capture))
+    assert found
+    assert b.i == 100
+    with capture:
+        del b
+        pytest.gc_collect()
+    assert capture == "operator delete called on " + found.group(1)
+
+
+def test_multiple_inheritance():
+    class MITest(m.TestFactory1, m.TestFactory2):
+        def __init__(self):
+            m.TestFactory1.__init__(self, tag.unique_ptr, 33)
+            m.TestFactory2.__init__(self, tag.move)
+
+    a = MITest()
+    assert m.TestFactory1.value.fget(a) == "33"
+    assert m.TestFactory2.value.fget(a) == "(empty2)"
+
+
+def create_and_destroy(*args):
+    a = m.NoisyAlloc(*args)
+    print("---")
+    del a
+    pytest.gc_collect()
+
+
+def strip_comments(s):
+    return re.sub(r'\s+#.*', '', s)
+
+
+def test_reallocations(capture, msg):
+    """When the constructor is overloaded, previous overloads can require a preallocated value.
+    This test makes sure that such preallocated values only happen when they might be necessary,
+    and that they are deallocated properly"""
+
+    pytest.gc_collect()
+
+    with capture:
+        create_and_destroy(1)
+    assert msg(capture) == """
+        noisy new
+        noisy placement new
+        NoisyAlloc(int 1)
+        ---
+        ~NoisyAlloc()
+        noisy delete
+    """
+    with capture:
+        create_and_destroy(1.5)
+    assert msg(capture) == strip_comments("""
+        noisy new               # allocation required to attempt first overload
+        noisy new               # pointer factory calling "new", part 1: allocation
+        NoisyAlloc(double 1.5)  # ... part two, invoking constructor
+        noisy delete            # have to dealloc before stashing factory-generated pointer
+        ---
+        ~NoisyAlloc()  # Destructor
+        noisy delete   # operator delete
+    """)
+
+    with capture:
+        create_and_destroy(2, 3)
+    assert msg(capture) == strip_comments("""
+        noisy new          # pointer factory calling "new", allocation
+        NoisyAlloc(int 2)  # constructor
+        ---
+        ~NoisyAlloc()  # Destructor
+        noisy delete   # operator delete
+    """)
+
+    with capture:
+        create_and_destroy(2.5, 3)
+    assert msg(capture) == strip_comments("""
+        NoisyAlloc(double 2.5)  # construction (local func variable: operator_new not called)
+        noisy new               # return-by-value "new" part 1: allocation
+        ~NoisyAlloc()           # moved-away local func variable destruction
+        ---
+        ~NoisyAlloc()  # Destructor
+        noisy delete   # operator delete
+    """)
+
+    with capture:
+        create_and_destroy(3.5, 4.5)
+    assert msg(capture) == strip_comments("""
+        noisy new               # preallocation needed before invoking placement-new overload
+        noisy placement new     # Placement new
+        NoisyAlloc(double 3.5)  # construction
+        ---
+        ~NoisyAlloc()  # Destructor
+        noisy delete   # operator delete
+    """)
+
+    with capture:
+        create_and_destroy(4, 0.5)
+    assert msg(capture) == strip_comments("""
+        noisy new          # preallocation needed before invoking placement-new overload
+        noisy new          # Factory pointer allocation
+        NoisyAlloc(int 4)  # factory pointer construction
+        noisy delete       # deallocation of preallocated storage
+        ---
+        ~NoisyAlloc()  # Destructor
+        noisy delete   # operator delete
+    """)
+
+    with capture:
+        create_and_destroy(5, "hi")
+    assert msg(capture) == strip_comments("""
+        noisy new            # preallocation needed before invoking first placement new
+        noisy placement new  # Placement new in the second placement new overload
+        NoisyAlloc(int 5)    # construction
+        ---
+        ~NoisyAlloc()  # Destructor
+        noisy delete   # operator delete
+    """)
+
+
+@pytest.unsupported_on_py2
+def test_invalid_self():
+    """Tests invocation of the pybind-registered base class with an invalid `self` argument.  You
+    can only actually do this on Python 3: Python 2 raises an exception itself if you try."""
+    class NotPybindDerived(object):
+        pass
+
+    # Attempts to initialize with an invalid type passed as `self`:
+    class BrokenTF1(m.TestFactory1):
+        def __init__(self, bad):
+            if bad == 1:
+                a = m.TestFactory2(tag.pointer, 1)
+                m.TestFactory1.__init__(a, tag.pointer)
+            elif bad == 2:
+                a = NotPybindDerived()
+                m.TestFactory1.__init__(a, tag.pointer)
+
+    # Same as above, but for a class with an alias:
+    class BrokenTF6(m.TestFactory6):
+        def __init__(self, bad):
+            if bad == 1:
+                a = m.TestFactory2(tag.pointer, 1)
+                m.TestFactory6.__init__(a, tag.base, 1)
+            elif bad == 2:
+                a = m.TestFactory2(tag.pointer, 1)
+                m.TestFactory6.__init__(a, tag.alias, 1)
+            elif bad == 3:
+                m.TestFactory6.__init__(NotPybindDerived.__new__(NotPybindDerived), tag.base, 1)
+            elif bad == 4:
+                m.TestFactory6.__init__(NotPybindDerived.__new__(NotPybindDerived), tag.alias, 1)
+
+    for arg in (1, 2):
+        with pytest.raises(TypeError) as excinfo:
+            BrokenTF1(arg)
+        assert (str(excinfo.value) ==
+                "__init__(self, ...) called with invalid `self` argument")
+
+    for arg in (1, 2, 3, 4):
+        with pytest.raises(TypeError) as excinfo:
+            BrokenTF6(arg)
+        assert (str(excinfo.value) ==
+                "__init__(self, ...) called with invalid `self` argument")

tests/test_multiple_inheritance.py

@@ -71,9 +71,9 @@ def test_multiple_inheritance_python():
             MI2.__init__(self, i, j)
 
     class MI4(MI3, m.Base2):
-        def __init__(self, i, j, k):
+        def __init__(self, i, j):
-            MI3.__init__(self, j, k)
+            MI3.__init__(self, i, j)
-            m.Base2.__init__(self, i)
+            # m.Base2 is already initialized (via MI2)
 
     class MI5(m.Base2, B1, m.Base1):
         def __init__(self, i, j):
@@ -127,10 +127,10 @@ def test_multiple_inheritance_python():
     assert mi3.foo() == 5
     assert mi3.bar() == 6
 
-    mi4 = MI4(7, 8, 9)
+    mi4 = MI4(7, 8)
     assert mi4.v() == 1
-    assert mi4.foo() == 8
+    assert mi4.foo() == 7
-    assert mi4.bar() == 7
+    assert mi4.bar() == 8
 
     mi5 = MI5(10, 11)
     assert mi5.v() == 1

tests/test_virtual_functions.cpp

@@ -234,6 +234,7 @@ TEST_SUBMODULE(virtual_functions, m) {
 
     py::class_<A2, PyA2>(m, "A2")
         .def(py::init_alias<>())
+        .def(py::init([](int) { return new PyA2(); }))
         .def("f", &A2::f);
 
     m.def("call_f", [](A2 *a2) { a2->f(); });
@@ -444,4 +445,3 @@ void initialize_inherited_virtuals(py::module &m) {
         .def(py::init<>());
 
 };
-

tests/test_virtual_functions.py

@@ -128,11 +128,19 @@ def test_alias_delay_initialization2(capture):
         m.call_f(a2)
         del a2
         pytest.gc_collect()
+        a3 = m.A2(1)
+        m.call_f(a3)
+        del a3
+        pytest.gc_collect()
     assert capture == """
         PyA2.PyA2()
         PyA2.f()
         A2.f()
         PyA2.~PyA2()
+        PyA2.PyA2()
+        PyA2.f()
+        A2.f()
+        PyA2.~PyA2()
     """
 
     # Python subclass version