Added advanced doc section on virtual methods + inheritance

As discussed in #320.

The adds a documentation block that mentions that the trampoline classes
must provide overrides for both the classes' own virtual methods *and*
any inherited virtual methods.  It also provides a templated solution to
avoiding method duplication.

The example includes a third method (only mentioned in the "see also"
section of the documentation addition), using multiple inheritance.
While this approach works, and avoids code generation in deep
hierarchies, it is intrusive by requiring that the wrapped classes use
virtual inheritance, which itself is more instrusive if any of the
virtual base classes need anything other than default constructors.  As
per the discussion in #320, it is kept as an example, but not suggested
in the documentation.
This commit is contained in:
Jason Rhinelander 2016-08-05 17:02:33 -04:00
parent 5289af5fc0
commit 0ca96e2915
4 changed files with 497 additions and 3 deletions

View File

@ -232,7 +232,7 @@ code).
class Dog : public Animal {
public:
std::string go(int n_times) {
std::string go(int n_times) override {
std::string result;
for (int i=0; i<n_times; ++i)
result += "woof! ";
@ -283,7 +283,7 @@ helper class that is defined as follows:
using Animal::Animal;
/* Trampoline (need one for each virtual function) */
std::string go(int n_times) {
std::string go(int n_times) override {
PYBIND11_OVERLOAD_PURE(
std::string, /* Return type */
Animal, /* Parent class */
@ -328,6 +328,11 @@ by specifying it as the *third* template argument to :class:`class_`. The
second argument with the unique pointer is simply the default holder type used
by pybind11. Following this, we are able to define a constructor as usual.
Note, however, that the above is sufficient for allowing python classes to
extend ``Animal``, but not ``Dog``: see ref:`virtual_and_inheritance` for the
necessary steps required to providing proper overload support for inherited
classes.
The Python session below shows how to override ``Animal::go`` and invoke it via
a virtual method call.
@ -353,6 +358,121 @@ Please take a look at the :ref:`macro_notes` before using this feature.
example that demonstrates how to override virtual functions using pybind11
in more detail.
.. _virtual_and_inheritance:
Combining virtual functions and inheritance
===========================================
When combining virtual methods with inheritance, you need to be sure to provide
an override for each method for which you want to allow overrides from derived
python classes. For example, suppose we extend the above ``Animal``/``Dog``
example as follows:
.. code-block:: cpp
class Animal {
public:
virtual std::string go(int n_times) = 0;
virtual std::string name() { return "unknown"; }
};
class Dog : public class Animal {
public:
std::string go(int n_times) override {
std::string result;
for (int i=0; i<n_times; ++i)
result += bark() + " ";
return result;
}
virtual std::string bark() { return "woof!"; }
};
then the trampoline class for ``Animal`` must, as described in the previous
section, override ``go()`` and ``name()``, but in order to allow python code to
inherit properly from ``Dog``, we also need a trampoline class for ``Dog`` that
overrides both the added ``bark()`` method *and* the ``go()`` and ``name()``
methods inherited from ``Animal`` (even though ``Dog`` doesn't directly
override the ``name()`` method):
.. code-block:: cpp
class PyAnimal : public Animal {
public:
using Animal::Animal; // Inherit constructors
std::string go(int n_times) override { PYBIND11_OVERLOAD_PURE(std::string, Animal, go, n_times); }
std::string name() override { PYBIND11_OVERLOAD(std::string, Animal, name, ); }
};
class PyDog : public Dog {
public:
using Dog::Dog; // Inherit constructors
std::string go(int n_times) override { PYBIND11_OVERLOAD_PURE(std::string, Dog, go, n_times); }
std::string name() override { PYBIND11_OVERLOAD(std::string, Dog, name, ); }
std::string bark() override { PYBIND11_OVERLOAD(std::string, Dog, bark, ); }
};
A registered class derived from a pybind11-registered class with virtual
methods requires a similar trampoline class, *even if* it doesn't explicitly
declare or override any virtual methods itself:
.. code-block:: cpp
class Husky : public Dog {};
class PyHusky : public Husky {
using Dog::Dog; // Inherit constructors
std::string go(int n_times) override { PYBIND11_OVERLOAD_PURE(std::string, Husky, go, n_times); }
std::string name() override { PYBIND11_OVERLOAD(std::string, Husky, name, ); }
std::string bark() override { PYBIND11_OVERLOAD(std::string, Husky, bark, ); }
};
There is, however, a technique that can be used to avoid this duplication
(which can be especially helpful for a base class with several virtual
methods). The technique involves using template trampoline classes, as
follows:
.. code-block:: cpp
template <class AnimalBase = Animal> class PyAnimal : public AnimalBase {
using AnimalBase::AnimalBase; // Inherit constructors
std::string go(int n_times) override { PYBIND11_OVERLOAD_PURE(std::string, AnimalBase, go, n_times); }
std::string name() override { PYBIND11_OVERLOAD(std::string, AnimalBase, name, ); }
};
template <class DogBase = Dog> class PyDog : public PyAnimal<DogBase> {
using PyAnimal<DogBase>::PyAnimal; // Inherit constructors
// Override PyAnimal's pure virtual go() with a non-pure one:
std::string go(int n_times) override { PYBIND11_OVERLOAD(std::string, DogBase, go, n_times); }
std::string bark() override { PYBIND11_OVERLOAD(std::string, DogBase, bark, ); }
};
This technique has the advantage of requiring just one trampoline method to be
declared per virtual method and pure virtual method override. It does,
however, require the compiler to generate at least as many methods (and
possibly more, if both pure virtual and overridden pure virtual methods are
exposed, as above).
The classes are then registered with pybind11 using:
.. code-block:: cpp
py::class_<Animal, std::unique_ptr<Animal>, PyAnimal<>> animal(m, "Animal");
py::class_<Dog, std::unique_ptr<Dog>, PyDog<>> dog(m, "Dog");
py::class_<Husky, std::unique_ptr<Husky>, PyDog<Husky>> husky(m, "Husky");
// ... add animal, dog, husky definitions
Note that ``Husky`` did not require a dedicated trampoline template class at
all, since it neither declares any new virtual methods nor provides any pure
virtual method implementations.
With either the repeated-virtuals or templated trampoline methods in place, you
can now create a python class that inherits from ``Dog``:
.. code-block:: python
class ShihTzu(Dog):
def bark(self):
return "yip!"
.. seealso::
See the file :file:`example-virtual-functions.cpp` for complete examples
using both the duplication and templated trampoline approaches.
The file also contains a more intrusive approach using multiple
inheritance, which may be useful in special situations with deep class
hierarchies to avoid code generation.
.. _macro_notes:

View File

@ -81,6 +81,207 @@ void runExampleVirtVirtual(ExampleVirt *ex) {
ex->pure_virtual();
}
// Inheriting virtual methods. We do three versions here: the repeat-everything version and the
// templated trampoline versions mentioned in docs/advanced.rst, and also another version using
// multiple inheritance.
//
// This latter approach has the advantage of generating substantially less code for deep
// hierarchies, but it requires intrusive changes of changing the wrapped classes to using virtual
// inheritance, which itself requires constructor rewriting for any non-default virtual base class
// constructors (most problematic is that constructors cannot be effectively inherited from a
// virtual base class). The example is kept here because it does work, and may be useful in limited
// cases, but the non-instrusive templated version is generally preferred.
//
// These base classes are all exactly the same (aside from the virtual inheritance for the MI
// version), but we technically need distinct classes for this example code because we need to be
// able to bind them properly (pybind11, sensibly, doesn't allow us to bind the same C++ class to
// multiple python classes).
class A_Repeat {
#define A_METHODS \
public: \
virtual int unlucky_number() = 0; \
virtual void say_something(unsigned times) { \
for (unsigned i = 0; i < times; i++) std::cout << "hi"; \
std::cout << std::endl; \
}
A_METHODS
};
class B_Repeat : public A_Repeat {
#define B_METHODS \
public: \
int unlucky_number() override { return 13; } \
void say_something(unsigned times) override { \
std::cout << "B says hi " << times << " times" << std::endl; \
} \
virtual double lucky_number() { return 7.0; }
B_METHODS
};
class C_Repeat : public B_Repeat {
#define C_METHODS \
public: \
int unlucky_number() override { return 4444; } \
double lucky_number() override { return 888; }
C_METHODS
};
class D_Repeat : public C_Repeat {
#define D_METHODS // Nothing overridden.
D_METHODS
};
// Base classes for multiple inheritance trampolines; note the added "virtual" inheritance. The
// classes are otherwise identical.
class A_MI { A_METHODS };
class B_MI : virtual public A_MI { B_METHODS };
class C_MI : virtual public B_MI { C_METHODS };
class D_MI : virtual public C_MI { D_METHODS };
// Base classes for templated inheritance trampolines. Identical to the repeat-everything version:
class A_Tpl { A_METHODS };
class B_Tpl : public A_Tpl { B_METHODS };
class C_Tpl : public B_Tpl { C_METHODS };
class D_Tpl : public C_Tpl { D_METHODS };
// Inheritance approach 1: each trampoline gets every virtual method (11 in total)
class PyA_Repeat : public A_Repeat {
public:
using A_Repeat::A_Repeat;
int unlucky_number() override { PYBIND11_OVERLOAD_PURE(int, A_Repeat, unlucky_number, ); }
void say_something(unsigned times) override { PYBIND11_OVERLOAD(void, A_Repeat, say_something, times); }
};
class PyB_Repeat : public B_Repeat {
public:
using B_Repeat::B_Repeat;
int unlucky_number() override { PYBIND11_OVERLOAD(int, B_Repeat, unlucky_number, ); }
void say_something(unsigned times) override { PYBIND11_OVERLOAD(void, B_Repeat, say_something, times); }
double lucky_number() override { PYBIND11_OVERLOAD(double, B_Repeat, lucky_number, ); }
};
class PyC_Repeat : public C_Repeat {
public:
using C_Repeat::C_Repeat;
int unlucky_number() override { PYBIND11_OVERLOAD(int, C_Repeat, unlucky_number, ); }
void say_something(unsigned times) override { PYBIND11_OVERLOAD(void, C_Repeat, say_something, times); }
double lucky_number() override { PYBIND11_OVERLOAD(double, C_Repeat, lucky_number, ); }
};
class PyD_Repeat : public D_Repeat {
public:
using D_Repeat::D_Repeat;
int unlucky_number() override { PYBIND11_OVERLOAD(int, D_Repeat, unlucky_number, ); }
void say_something(unsigned times) override { PYBIND11_OVERLOAD(void, D_Repeat, say_something, times); }
double lucky_number() override { PYBIND11_OVERLOAD(double, D_Repeat, lucky_number, ); }
};
// Inheritance approach 2: templated trampoline classes.
//
// Advantages:
// - we have only 2 (template) class and 4 method declarations (one per virtual method, plus one for
// any override of a pure virtual method), versus 4 classes and 6 methods (MI) or 4 classes and 11
// methods (repeat).
// - Compared to MI, we also don't have to change the non-trampoline inheritance to virtual, and can
// properly inherit constructors.
//
// Disadvantage:
// - the compiler must still generate and compile 14 different methods (more, even, than the 11
// required for the repeat approach) instead of the 6 required for MI. (If there was no pure
// method (or no pure method override), the number would drop down to the same 11 as the repeat
// approach).
template <class Base = A_Tpl>
class PyA_Tpl : public Base {
public:
using Base::Base; // Inherit constructors
int unlucky_number() override { PYBIND11_OVERLOAD_PURE(int, Base, unlucky_number, ); }
void say_something(unsigned times) override { PYBIND11_OVERLOAD(void, Base, say_something, times); }
};
template <class Base = B_Tpl>
class PyB_Tpl : public PyA_Tpl<Base> {
public:
using PyA_Tpl<Base>::PyA_Tpl; // Inherit constructors (via PyA_Tpl's inherited constructors)
int unlucky_number() override { PYBIND11_OVERLOAD(int, Base, unlucky_number, ); }
double lucky_number() { PYBIND11_OVERLOAD(double, Base, lucky_number, ); }
};
// Since C_Tpl and D_Tpl don't declare any new virtual methods, we don't actually need these (we can
// use PyB_Tpl<C_Tpl> and PyB_Tpl<D_Tpl> for the trampoline classes instead):
/*
template <class Base = C_Tpl> class PyC_Tpl : public PyB_Tpl<Base> {
public:
using PyB_Tpl<Base>::PyB_Tpl;
};
template <class Base = D_Tpl> class PyD_Tpl : public PyC_Tpl<Base> {
public:
using PyC_Tpl<Base>::PyC_Tpl;
};
*/
// Inheritance approach 3: multiple inheritance with virtual base class inheritance. This requires
// declaration and compilation of exactly 7 methods (one per virtual method declaration or
// override), versus the 11 required above. On the other hand, if we need anything other than
// default constructors, this quickly becomes painful, and so this is of limited use.
class PyA_MI : virtual public A_MI {
public:
// Can't inherit constructors: we would have to duplicate constructors with an explicit
// initializer for each virtual base, which is a pain for anything but the default constructor.
int unlucky_number() override { PYBIND11_OVERLOAD_PURE(int, A_MI, unlucky_number, ); }
void say_something(unsigned times) override { PYBIND11_OVERLOAD(void, A_MI, say_something, times); }
};
class PyB_MI : public PyA_MI, virtual public B_MI {
public:
void say_something(unsigned times) override { PYBIND11_OVERLOAD(void, B_MI, say_something, times); }
double lucky_number() { PYBIND11_OVERLOAD(double, B_MI, lucky_number, ); }
int unlucky_number() { PYBIND11_OVERLOAD(int, B_MI, unlucky_number, ); }
};
class PyC_MI : public PyB_MI, virtual public C_MI {
public:
int unlucky_number() override { PYBIND11_OVERLOAD(int, C_MI, unlucky_number, ); }
double lucky_number() override { PYBIND11_OVERLOAD(double, C_MI, lucky_number, ); }
};
class PyD_MI : public PyC_MI, virtual public D_MI {
};
void initialize_inherited_virtuals(py::module &m) {
// Method 1: repeat
py::class_<A_Repeat, std::unique_ptr<A_Repeat>, PyA_Repeat>(m, "A_Repeat")
.def(py::init<>())
.def("unlucky_number", &A_Repeat::unlucky_number)
.def("say_something", &A_Repeat::say_something);
py::class_<B_Repeat, std::unique_ptr<B_Repeat>, PyB_Repeat>(m, "B_Repeat", py::base<A_Repeat>())
.def(py::init<>())
.def("lucky_number", &B_Repeat::lucky_number);
py::class_<C_Repeat, std::unique_ptr<C_Repeat>, PyC_Repeat>(m, "C_Repeat", py::base<B_Repeat>())
.def(py::init<>());
py::class_<D_Repeat, std::unique_ptr<D_Repeat>, PyD_Repeat>(m, "D_Repeat", py::base<C_Repeat>())
.def(py::init<>());
// Method 2: Templated trampolines
py::class_<A_Tpl, std::unique_ptr<A_Tpl>, PyA_Tpl<>>(m, "A_Tpl")
.def(py::init<>())
.def("unlucky_number", &A_Tpl::unlucky_number)
.def("say_something", &A_Tpl::say_something);
py::class_<B_Tpl, std::unique_ptr<B_Tpl>, PyB_Tpl<>>(m, "B_Tpl", py::base<A_Tpl>())
.def(py::init<>())
.def("lucky_number", &B_Tpl::lucky_number);
py::class_<C_Tpl, std::unique_ptr<C_Tpl>, PyB_Tpl<C_Tpl>>(m, "C_Tpl", py::base<B_Tpl>())
.def(py::init<>());
py::class_<D_Tpl, std::unique_ptr<D_Tpl>, PyB_Tpl<D_Tpl>>(m, "D_Tpl", py::base<C_Tpl>())
.def(py::init<>());
// Method 3: MI
py::class_<A_MI, std::unique_ptr<A_MI>, PyA_MI>(m, "A_MI")
.def(py::init<>())
.def("unlucky_number", &A_MI::unlucky_number)
.def("say_something", &A_MI::say_something);
py::class_<B_MI, std::unique_ptr<B_MI>, PyB_MI>(m, "B_MI", py::base<A_MI>())
.def(py::init<>())
.def("lucky_number", &B_MI::lucky_number);
py::class_<C_MI, std::unique_ptr<C_MI>, PyC_MI>(m, "C_MI", py::base<B_MI>())
.def(py::init<>());
py::class_<D_MI, std::unique_ptr<D_MI>, PyD_MI>(m, "D_MI", py::base<C_MI>())
.def(py::init<>());
};
void init_ex_virtual_functions(py::module &m) {
/* Important: indicate the trampoline class PyExampleVirt using the third
argument to py::class_. The second argument with the unique pointer
@ -95,4 +296,6 @@ void init_ex_virtual_functions(py::module &m) {
m.def("runExampleVirt", &runExampleVirt);
m.def("runExampleVirtBool", &runExampleVirtBool);
m.def("runExampleVirtVirtual", &runExampleVirtVirtual);
initialize_inherited_virtuals(m);
}

View File

@ -4,7 +4,7 @@ import sys
sys.path.append('.')
from example import ExampleVirt, runExampleVirt, runExampleVirtVirtual, runExampleVirtBool
from example import A_Repeat, B_Repeat, C_Repeat, D_Repeat, A_MI, B_MI, C_MI, D_MI, A_Tpl, B_Tpl, C_Tpl, D_Tpl
class ExtendedExampleVirt(ExampleVirt):
def __init__(self, state):
@ -34,3 +34,72 @@ ex12p = ExtendedExampleVirt(10)
print(runExampleVirt(ex12p, 20))
print(runExampleVirtBool(ex12p))
runExampleVirtVirtual(ex12p)
sys.stdout.flush()
class VI_AR(A_Repeat):
def unlucky_number(self):
return 99
class VI_AMI(A_MI):
def unlucky_number(self):
return 990
def say_something(self, times):
return A_MI.say_something(self, 2*times)
class VI_AT(A_Tpl):
def unlucky_number(self):
return 999
class VI_CR(C_Repeat):
def lucky_number(self):
return C_Repeat.lucky_number(self) + 1.25
class VI_CMI(C_MI):
def lucky_number(self):
return 1.75
class VI_CT(C_Tpl):
pass
class VI_CCR(VI_CR):
def lucky_number(self):
return VI_CR.lucky_number(self) * 10
class VI_CCMI(VI_CMI):
def lucky_number(self):
return VI_CMI.lucky_number(self) * 100
class VI_CCT(VI_CT):
def lucky_number(self):
return VI_CT.lucky_number(self) * 1000
class VI_DR(D_Repeat):
def unlucky_number(self):
return 123
def lucky_number(self):
return 42.0
class VI_DMI(D_MI):
def unlucky_number(self):
return 1230
def lucky_number(self):
return -9.5
class VI_DT(D_Tpl):
def say_something(self, times):
print("VI_DT says:" + (' quack' * times))
def unlucky_number(self):
return 1234
def lucky_number(self):
return -4.25
classes = [
# A_Repeat, A_MI, A_Tpl, # abstract (they have a pure virtual unlucky_number)
VI_AR, VI_AMI, VI_AT,
B_Repeat, B_MI, B_Tpl,
C_Repeat, C_MI, C_Tpl,
VI_CR, VI_CMI, VI_CT, VI_CCR, VI_CCMI, VI_CCT,
D_Repeat, D_MI, D_Tpl, VI_DR, VI_DMI, VI_DT
]
for cl in classes:
print("\n%s:" % cl.__name__)
obj = cl()
obj.say_something(3)
print("Unlucky = %d" % obj.unlucky_number())
if hasattr(obj, "lucky_number"):
print("Lucky = %.2f" % obj.lucky_number())

View File

@ -9,5 +9,107 @@ Original implementation of ExampleVirt::run(state=11, value=21)
ExtendedExampleVirt::run_bool()
False
ExtendedExampleVirt::pure_virtual(): Hello world
VI_AR:
hihihi
Unlucky = 99
VI_AMI:
hihihihihihi
Unlucky = 990
VI_AT:
hihihi
Unlucky = 999
B_Repeat:
B says hi 3 times
Unlucky = 13
Lucky = 7.00
B_MI:
B says hi 3 times
Unlucky = 13
Lucky = 7.00
B_Tpl:
B says hi 3 times
Unlucky = 13
Lucky = 7.00
C_Repeat:
B says hi 3 times
Unlucky = 4444
Lucky = 888.00
C_MI:
B says hi 3 times
Unlucky = 4444
Lucky = 888.00
C_Tpl:
B says hi 3 times
Unlucky = 4444
Lucky = 888.00
VI_CR:
B says hi 3 times
Unlucky = 4444
Lucky = 889.25
VI_CMI:
B says hi 3 times
Unlucky = 4444
Lucky = 1.75
VI_CT:
B says hi 3 times
Unlucky = 4444
Lucky = 888.00
VI_CCR:
B says hi 3 times
Unlucky = 4444
Lucky = 8892.50
VI_CCMI:
B says hi 3 times
Unlucky = 4444
Lucky = 175.00
VI_CCT:
B says hi 3 times
Unlucky = 4444
Lucky = 888000.00
D_Repeat:
B says hi 3 times
Unlucky = 4444
Lucky = 888.00
D_MI:
B says hi 3 times
Unlucky = 4444
Lucky = 888.00
D_Tpl:
B says hi 3 times
Unlucky = 4444
Lucky = 888.00
VI_DR:
B says hi 3 times
Unlucky = 123
Lucky = 42.00
VI_DMI:
B says hi 3 times
Unlucky = 1230
Lucky = -9.50
VI_DT:
VI_DT says: quack quack quack
Unlucky = 1234
Lucky = -4.25
Destructing ExampleVirt..
Destructing ExampleVirt..