Merge pull request #322 from jagerman/document-inherited-virtuals

Added advanced doc section on virtual methods + inheritance

docs/advanced.rst

@@ -232,7 +232,7 @@ code).
 
     class Dog : public Animal {
     public:
-        std::string go(int n_times) {
+        std::string go(int n_times) override {
             std::string result;
             for (int i=0; i<n_times; ++i)
                 result += "woof! ";
@@ -283,7 +283,7 @@ helper class that is defined as follows:
         using Animal::Animal;
 
         /* Trampoline (need one for each virtual function) */
-        std::string go(int n_times) {
+        std::string go(int n_times) override {
             PYBIND11_OVERLOAD_PURE(
                 std::string, /* Return type */
                 Animal,      /* Parent class */
@@ -328,6 +328,11 @@ by specifying it as the *third* template argument to :class:`class_`. The
 second argument with the unique pointer is simply the default holder type used
 by pybind11. Following this, we are able to define a constructor as usual.
 
+Note, however, that the above is sufficient for allowing python classes to
+extend ``Animal``, but not ``Dog``: see ref:`virtual_and_inheritance` for the
+necessary steps required to providing proper overload support for inherited
+classes.
+
 The Python session below shows how to override ``Animal::go`` and invoke it via
 a virtual method call.
 
@@ -353,6 +358,117 @@ Please take a look at the :ref:`macro_notes` before using this feature.
     example that demonstrates how to override virtual functions using pybind11
     in more detail.
 
+.. _virtual_and_inheritance:
+
+Combining virtual functions and inheritance
+===========================================
+
+When combining virtual methods with inheritance, you need to be sure to provide
+an override for each method for which you want to allow overrides from derived
+python classes.  For example, suppose we extend the above ``Animal``/``Dog``
+example as follows:
+
+.. code-block:: cpp
+    class Animal {
+    public:
+        virtual std::string go(int n_times) = 0;
+        virtual std::string name() { return "unknown"; }
+    };
+    class Dog : public class Animal {
+    public:
+        std::string go(int n_times) override {
+            std::string result;
+            for (int i=0; i<n_times; ++i)
+                result += bark() + " ";
+            return result;
+        }
+        virtual std::string bark() { return "woof!"; }
+    };
+
+then the trampoline class for ``Animal`` must, as described in the previous
+section, override ``go()`` and ``name()``, but in order to allow python code to
+inherit properly from ``Dog``, we also need a trampoline class for ``Dog`` that
+overrides both the added ``bark()`` method *and* the ``go()`` and ``name()``
+methods inherited from ``Animal`` (even though ``Dog`` doesn't directly
+override the ``name()`` method):
+
+.. code-block:: cpp
+    class PyAnimal : public Animal {
+    public:
+        using Animal::Animal; // Inherit constructors
+        std::string go(int n_times) override { PYBIND11_OVERLOAD_PURE(std::string, Animal, go, n_times); }
+        std::string name() override { PYBIND11_OVERLOAD(std::string, Animal, name, ); }
+    };
+    class PyDog : public Dog {
+    public:
+        using Dog::Dog; // Inherit constructors
+        std::string go(int n_times) override { PYBIND11_OVERLOAD_PURE(std::string, Dog, go, n_times); }
+        std::string name() override { PYBIND11_OVERLOAD(std::string, Dog, name, ); }
+        std::string bark() override { PYBIND11_OVERLOAD(std::string, Dog, bark, ); }
+    };
+
+A registered class derived from a pybind11-registered class with virtual
+methods requires a similar trampoline class, *even if* it doesn't explicitly
+declare or override any virtual methods itself:
+
+.. code-block:: cpp
+    class Husky : public Dog {};
+    class PyHusky : public Husky {
+        using Dog::Dog; // Inherit constructors
+        std::string go(int n_times) override { PYBIND11_OVERLOAD_PURE(std::string, Husky, go, n_times); }
+        std::string name() override { PYBIND11_OVERLOAD(std::string, Husky, name, ); }
+        std::string bark() override { PYBIND11_OVERLOAD(std::string, Husky, bark, ); }
+    };
+
+There is, however, a technique that can be used to avoid this duplication
+(which can be especially helpful for a base class with several virtual
+methods).  The technique involves using template trampoline classes, as
+follows:
+
+.. code-block:: cpp
+    template <class AnimalBase = Animal> class PyAnimal : public AnimalBase {
+        using AnimalBase::AnimalBase; // Inherit constructors
+        std::string go(int n_times) override { PYBIND11_OVERLOAD_PURE(std::string, AnimalBase, go, n_times); }
+        std::string name() override { PYBIND11_OVERLOAD(std::string, AnimalBase, name, ); }
+    };
+    template <class DogBase = Dog> class PyDog : public PyAnimal<DogBase> {
+        using PyAnimal<DogBase>::PyAnimal; // Inherit constructors
+        // Override PyAnimal's pure virtual go() with a non-pure one:
+        std::string go(int n_times) override { PYBIND11_OVERLOAD(std::string, DogBase, go, n_times); }
+        std::string bark() override { PYBIND11_OVERLOAD(std::string, DogBase, bark, ); }
+    };
+
+This technique has the advantage of requiring just one trampoline method to be
+declared per virtual method and pure virtual method override.  It does,
+however, require the compiler to generate at least as many methods (and
+possibly more, if both pure virtual and overridden pure virtual methods are
+exposed, as above).
+
+The classes are then registered with pybind11 using:
+
+.. code-block:: cpp
+    py::class_<Animal, std::unique_ptr<Animal>, PyAnimal<>> animal(m, "Animal");
+    py::class_<Dog, std::unique_ptr<Dog>, PyDog<>> dog(m, "Dog");
+    py::class_<Husky, std::unique_ptr<Husky>, PyDog<Husky>> husky(m, "Husky");
+    // ... add animal, dog, husky definitions
+
+Note that ``Husky`` did not require a dedicated trampoline template class at
+all, since it neither declares any new virtual methods nor provides any pure
+virtual method implementations.
+
+With either the repeated-virtuals or templated trampoline methods in place, you
+can now create a python class that inherits from ``Dog``:
+
+.. code-block:: python
+
+    class ShihTzu(Dog):
+        def bark(self):
+            return "yip!"
+
+.. seealso::
+
+    See the file :file:`example-virtual-functions.cpp` for complete examples
+    using both the duplication and templated trampoline approaches.
 
 .. _macro_notes:
 

example/example-virtual-functions.cpp

@@ -81,6 +81,154 @@ void runExampleVirtVirtual(ExampleVirt *ex) {
     ex->pure_virtual();
 }
 
+
+// Inheriting virtual methods.  We do two versions here: the repeat-everything version and the
+// templated trampoline versions mentioned in docs/advanced.rst.
+//
+// These base classes are exactly the same, but we technically need distinct
+// classes for this example code because we need to be able to bind them
+// properly (pybind11, sensibly, doesn't allow us to bind the same C++ class to
+// multiple python classes).
+class A_Repeat {
+#define A_METHODS \
+public: \
+    virtual int unlucky_number() = 0; \
+    virtual void say_something(unsigned times) { \
+        for (unsigned i = 0; i < times; i++) std::cout << "hi"; \
+        std::cout << std::endl; \
+    }
+A_METHODS
+};
+class B_Repeat : public A_Repeat {
+#define B_METHODS \
+public: \
+    int unlucky_number() override { return 13; } \
+    void say_something(unsigned times) override { \
+        std::cout << "B says hi " << times << " times" << std::endl; \
+    } \
+    virtual double lucky_number() { return 7.0; }
+B_METHODS
+};
+class C_Repeat : public B_Repeat {
+#define C_METHODS \
+public: \
+    int unlucky_number() override { return 4444; } \
+    double lucky_number() override { return 888; }
+C_METHODS
+};
+class D_Repeat : public C_Repeat {
+#define D_METHODS // Nothing overridden.
+D_METHODS
+};
+
+// Base classes for templated inheritance trampolines.  Identical to the repeat-everything version:
+class A_Tpl { A_METHODS };
+class B_Tpl : public A_Tpl { B_METHODS };
+class C_Tpl : public B_Tpl { C_METHODS };
+class D_Tpl : public C_Tpl { D_METHODS };
+
+
+// Inheritance approach 1: each trampoline gets every virtual method (11 in total)
+class PyA_Repeat : public A_Repeat {
+public:
+    using A_Repeat::A_Repeat;
+    int unlucky_number() override { PYBIND11_OVERLOAD_PURE(int, A_Repeat, unlucky_number, ); }
+    void say_something(unsigned times) override { PYBIND11_OVERLOAD(void, A_Repeat, say_something, times); }
+};
+class PyB_Repeat : public B_Repeat {
+public:
+    using B_Repeat::B_Repeat;
+    int unlucky_number() override { PYBIND11_OVERLOAD(int, B_Repeat, unlucky_number, ); }
+    void say_something(unsigned times) override { PYBIND11_OVERLOAD(void, B_Repeat, say_something, times); }
+    double lucky_number() override { PYBIND11_OVERLOAD(double, B_Repeat, lucky_number, ); }
+};
+class PyC_Repeat : public C_Repeat {
+public:
+    using C_Repeat::C_Repeat;
+    int unlucky_number() override { PYBIND11_OVERLOAD(int, C_Repeat, unlucky_number, ); }
+    void say_something(unsigned times) override { PYBIND11_OVERLOAD(void, C_Repeat, say_something, times); }
+    double lucky_number() override { PYBIND11_OVERLOAD(double, C_Repeat, lucky_number, ); }
+};
+class PyD_Repeat : public D_Repeat {
+public:
+    using D_Repeat::D_Repeat;
+    int unlucky_number() override { PYBIND11_OVERLOAD(int, D_Repeat, unlucky_number, ); }
+    void say_something(unsigned times) override { PYBIND11_OVERLOAD(void, D_Repeat, say_something, times); }
+    double lucky_number() override { PYBIND11_OVERLOAD(double, D_Repeat, lucky_number, ); }
+};
+
+// Inheritance approach 2: templated trampoline classes.
+//
+// Advantages:
+// - we have only 2 (template) class and 4 method declarations (one per virtual method, plus one for
+//   any override of a pure virtual method), versus 4 classes and 6 methods (MI) or 4 classes and 11
+//   methods (repeat).
+// - Compared to MI, we also don't have to change the non-trampoline inheritance to virtual, and can
+//   properly inherit constructors.
+//
+// Disadvantage:
+// - the compiler must still generate and compile 14 different methods (more, even, than the 11
+//   required for the repeat approach) instead of the 6 required for MI.  (If there was no pure
+//   method (or no pure method override), the number would drop down to the same 11 as the repeat
+//   approach).
+template <class Base = A_Tpl>
+class PyA_Tpl : public Base {
+public:
+    using Base::Base; // Inherit constructors
+    int unlucky_number() override { PYBIND11_OVERLOAD_PURE(int, Base, unlucky_number, ); }
+    void say_something(unsigned times) override { PYBIND11_OVERLOAD(void, Base, say_something, times); }
+};
+template <class Base = B_Tpl>
+class PyB_Tpl : public PyA_Tpl<Base> {
+public:
+    using PyA_Tpl<Base>::PyA_Tpl; // Inherit constructors (via PyA_Tpl's inherited constructors)
+    int unlucky_number() override { PYBIND11_OVERLOAD(int, Base, unlucky_number, ); }
+    double lucky_number() { PYBIND11_OVERLOAD(double, Base, lucky_number, ); }
+};
+// Since C_Tpl and D_Tpl don't declare any new virtual methods, we don't actually need these (we can
+// use PyB_Tpl<C_Tpl> and PyB_Tpl<D_Tpl> for the trampoline classes instead):
+/*
+template <class Base = C_Tpl> class PyC_Tpl : public PyB_Tpl<Base> {
+public:
+    using PyB_Tpl<Base>::PyB_Tpl;
+};
+template <class Base = D_Tpl> class PyD_Tpl : public PyC_Tpl<Base> {
+public:
+    using PyC_Tpl<Base>::PyC_Tpl;
+};
+*/
+
+
+void initialize_inherited_virtuals(py::module &m) {
+    // Method 1: repeat
+    py::class_<A_Repeat, std::unique_ptr<A_Repeat>, PyA_Repeat>(m, "A_Repeat")
+        .def(py::init<>())
+        .def("unlucky_number", &A_Repeat::unlucky_number)
+        .def("say_something", &A_Repeat::say_something);
+    py::class_<B_Repeat, std::unique_ptr<B_Repeat>, PyB_Repeat>(m, "B_Repeat", py::base<A_Repeat>())
+        .def(py::init<>())
+        .def("lucky_number", &B_Repeat::lucky_number);
+    py::class_<C_Repeat, std::unique_ptr<C_Repeat>, PyC_Repeat>(m, "C_Repeat", py::base<B_Repeat>())
+        .def(py::init<>());
+    py::class_<D_Repeat, std::unique_ptr<D_Repeat>, PyD_Repeat>(m, "D_Repeat", py::base<C_Repeat>())
+        .def(py::init<>());
+
+    // Method 2: Templated trampolines
+    py::class_<A_Tpl, std::unique_ptr<A_Tpl>, PyA_Tpl<>>(m, "A_Tpl")
+        .def(py::init<>())
+        .def("unlucky_number", &A_Tpl::unlucky_number)
+        .def("say_something", &A_Tpl::say_something);
+    py::class_<B_Tpl, std::unique_ptr<B_Tpl>, PyB_Tpl<>>(m, "B_Tpl", py::base<A_Tpl>())
+        .def(py::init<>())
+        .def("lucky_number", &B_Tpl::lucky_number);
+    py::class_<C_Tpl, std::unique_ptr<C_Tpl>, PyB_Tpl<C_Tpl>>(m, "C_Tpl", py::base<B_Tpl>())
+        .def(py::init<>());
+    py::class_<D_Tpl, std::unique_ptr<D_Tpl>, PyB_Tpl<D_Tpl>>(m, "D_Tpl", py::base<C_Tpl>())
+        .def(py::init<>());
+
+};
+
+
 void init_ex_virtual_functions(py::module &m) {
     /* Important: indicate the trampoline class PyExampleVirt using the third
        argument to py::class_. The second argument with the unique pointer
@@ -95,4 +243,6 @@ void init_ex_virtual_functions(py::module &m) {
     m.def("runExampleVirt", &runExampleVirt);
     m.def("runExampleVirtBool", &runExampleVirtBool);
     m.def("runExampleVirtVirtual", &runExampleVirtVirtual);
+
+    initialize_inherited_virtuals(m);
 }

example/example-virtual-functions.py

@@ -4,7 +4,7 @@ import sys
 sys.path.append('.')
 
 from example import ExampleVirt, runExampleVirt, runExampleVirtVirtual, runExampleVirtBool
-
+from example import A_Repeat, B_Repeat, C_Repeat, D_Repeat, A_Tpl, B_Tpl, C_Tpl, D_Tpl
 
 class ExtendedExampleVirt(ExampleVirt):
     def __init__(self, state):
@@ -34,3 +34,56 @@ ex12p = ExtendedExampleVirt(10)
 print(runExampleVirt(ex12p, 20))
 print(runExampleVirtBool(ex12p))
 runExampleVirtVirtual(ex12p)
+
+sys.stdout.flush()
+
+class VI_AR(A_Repeat):
+    def unlucky_number(self):
+        return 99
+class VI_AT(A_Tpl):
+    def unlucky_number(self):
+        return 999
+
+class VI_CR(C_Repeat):
+    def lucky_number(self):
+        return C_Repeat.lucky_number(self) + 1.25
+class VI_CT(C_Tpl):
+    pass
+class VI_CCR(VI_CR):
+    def lucky_number(self):
+        return VI_CR.lucky_number(self) * 10
+class VI_CCT(VI_CT):
+    def lucky_number(self):
+        return VI_CT.lucky_number(self) * 1000
+
+
+class VI_DR(D_Repeat):
+    def unlucky_number(self):
+        return 123
+    def lucky_number(self):
+        return 42.0
+class VI_DT(D_Tpl):
+    def say_something(self, times):
+        print("VI_DT says:" + (' quack' * times))
+    def unlucky_number(self):
+        return 1234
+    def lucky_number(self):
+        return -4.25
+
+classes = [
+    # A_Repeat, A_Tpl, # abstract (they have a pure virtual unlucky_number)
+    VI_AR, VI_AT,
+    B_Repeat, B_Tpl,
+    C_Repeat, C_Tpl,
+    VI_CR, VI_CT, VI_CCR, VI_CCT,
+    D_Repeat, D_Tpl, VI_DR, VI_DT
+]
+
+for cl in classes:
+    print("\n%s:" % cl.__name__)
+    obj = cl()
+    obj.say_something(3)
+    print("Unlucky = %d" % obj.unlucky_number())
+    if hasattr(obj, "lucky_number"):
+        print("Lucky = %.2f" % obj.lucky_number())
+

example/example-virtual-functions.ref

@@ -9,5 +9,73 @@ Original implementation of ExampleVirt::run(state=11, value=21)
 ExtendedExampleVirt::run_bool()
 False
 ExtendedExampleVirt::pure_virtual(): Hello world
+
+VI_AR:
+hihihi
+Unlucky = 99
+
+VI_AT:
+hihihi
+Unlucky = 999
+
+B_Repeat:
+B says hi 3 times
+Unlucky = 13
+Lucky = 7.00
+
+B_Tpl:
+B says hi 3 times
+Unlucky = 13
+Lucky = 7.00
+
+C_Repeat:
+B says hi 3 times
+Unlucky = 4444
+Lucky = 888.00
+
+C_Tpl:
+B says hi 3 times
+Unlucky = 4444
+Lucky = 888.00
+
+VI_CR:
+B says hi 3 times
+Unlucky = 4444
+Lucky = 889.25
+
+VI_CT:
+B says hi 3 times
+Unlucky = 4444
+Lucky = 888.00
+
+VI_CCR:
+B says hi 3 times
+Unlucky = 4444
+Lucky = 8892.50
+
+VI_CCT:
+B says hi 3 times
+Unlucky = 4444
+Lucky = 888000.00
+
+D_Repeat:
+B says hi 3 times
+Unlucky = 4444
+Lucky = 888.00
+
+D_Tpl:
+B says hi 3 times
+Unlucky = 4444
+Lucky = 888.00
+
+VI_DR:
+B says hi 3 times
+Unlucky = 123
+Lucky = 42.00
+
+VI_DT:
+VI_DT says: quack quack quack
+Unlucky = 1234
+Lucky = -4.25
 Destructing ExampleVirt..
 Destructing ExampleVirt..