remainder of documentation

This commit is contained in:
Wenzel Jakob 2015-10-13 23:21:54 +02:00
parent b50872acf2
commit 9329669683
7 changed files with 568 additions and 41 deletions

View File

@ -3,23 +3,506 @@
Advanced topics
###############
For brevity, the rest of this chapter assumes that the following two lines are
present:
.. code-block:: cpp
#include <pybind/pybind.h>
namespace py = pybind;
Operator overloading
====================
Suppose that we're given the following ``Vector2`` class with a vector addition
and scalar multiplication operation, all implemented using overloaded operators
in C++.
.. code-block:: cpp
class Vector2 {
public:
Vector2(float x, float y) : x(x), y(y) { }
std::string toString() const { return "[" + std::to_string(x) + ", " + std::to_string(y) + "]"; }
Vector2 operator+(const Vector2 &v) const { return Vector2(x + v.x, y + v.y); }
Vector2 operator*(float value) const { return Vector2(x * value, y * value); }
Vector2& operator+=(const Vector2 &v) { x += v.x; y += v.y; return *this; }
Vector2& operator*=(float v) { x *= v; y *= v; return *this; }
friend Vector2 operator*(float f, const Vector2 &v) { return Vector2(f * v.x, f * v.y); }
private:
float x, y;
};
The following snippet shows how the above operators can be conveniently exposed
to Python.
.. code-block:: cpp
#include <pybind/operators.h>
PYBIND_PLUGIN(example) {
py::module m("example", "pybind example plugin");
py::class_<Vector2>(m, "Vector2")
.def(py::init<float, float>())
.def(py::self + py::self)
.def(py::self += py::self)
.def(py::self *= float())
.def(float() * py::self)
.def("__repr__", &Vector2::toString);
return m.ptr();
}
Note that a line like
.. code-block:: cpp
.def(py::self * float())
is really just short hand notation for
.. code-block:: cpp
.def("__mul__", [](const Vector2 &a, float b) {
return a * b;
})
This can be useful for exposing additional operators that don't exist on the
C++ side, or to perform other types of customization.
.. note::
To use the more convenient ``py::self`` notation, the additional
header file :file:`pybind/operators.h` must be included.
.. seealso::
The file :file:`example/example3.cpp` contains a complete example that
demonstrates how to work with overloaded operators in more detail.
Callbacks and passing anonymous functions
=========================================
The C++11 standard brought lambda functions and the generic polymorphic
function wrapper ``std::function<>`` to the C++ programming language, which
enable powerful new ways of working with functions. Lambda functions come in
two flavors: stateless lambda function resemble classic function pointers that
link to an anonymous piece of code, while stateful lambda functions
additionally depend on captured variables that are stored in an anonymous
*lambda closure object*.
Here is a simple example of a C++ function that takes an arbitrary function
(stateful or stateless) with signature ``int -> int`` as an argument and runs
it with the value 10.
.. code-block:: cpp
int func_arg(const std::function<int(int)> &f) {
return f(10);
}
The example below is more involved: it takes a function of signature ``int -> int``
and returns another function of the same kind. The return value is a stateful
lambda function, which stores the value ``f`` in the capture object and adds 1 to
its return value upon execution.
.. code-block:: cpp
std::function<int(int)> func_ret(const std::function<int(int)> &f) {
return [f](int i) {
return f(i) + 1;
};
}
After including the extra header file :file:`pybind/functional.h`, it is almost
trivial to generate binding code for both of these functions.
.. code-block:: cpp
#include <pybind/functional.h>
PYBIND_PLUGIN(example) {
py::module m("example", "pybind example plugin");
m.def("func_arg", &func_arg);
m.def("func_ret", &func_ret);
return m.ptr();
}
The following interactive session shows how to call them from Python.
.. code-block:: python
$ python
>>> import example
>>> def square(i):
... return i * i
...
>>> example.func_arg(square)
100L
>>> square_plus_1 = example.func_ret(square)
>>> square_plus_1(4)
17L
>>>
.. note::
This functionality is very useful when generating bindings for callbacks in
C++ libraries (e.g. a graphical user interface library).
The file :file:`example/example5.cpp` contains a complete example that
demonstrates how to work with callbacks and anonymous functions in more detail.
Overriding virtual functions in Python
======================================
Passing anonymous functions
Suppose that a C++ class or interface has a virtual function that we'd like to
to override from within Python (we'll focus on the class ``Animal``; ``Dog`` is
given as a specific example of how one would do this with traditional C++
code).
.. code-block:: cpp
class Animal {
public:
virtual ~Animal() { }
virtual std::string go(int n_times) = 0;
};
class Dog : public Animal {
public:
std::string go(int n_times) {
std::string result;
for (int i=0; i<n_times; ++i)
result += "woof! ";
return result;
}
};
Let's also suppose that we are given a plain function which calls the
function ``go()`` on an arbitrary ``Animal`` instance.
.. code-block:: cpp
std::string call_go(Animal *animal) {
return animal->go(3);
}
Normally, the binding code for these classes would look as follows:
.. code-block:: cpp
PYBIND_PLUGIN(example) {
py::module m("example", "pybind example plugin");
py::class_<Animal> animal(m, "Animal");
animal
.def("go", &Animal::go);
py::class_<Dog>(m, "Dog", animal)
.def(py::init<>());
m.def("call_go", &call_go);
return m.ptr();
}
However, these bindings are impossible to extend: ``Animal`` is not
constructible, and we clearly require some kind of "trampoline" that
redirects virtual calls back to Python.
Defining a new type of ``Animal`` from within Python is possible but requires a
helper class that is defined as follows:
.. code-block:: cpp
class PyAnimal : public Animal {
public:
/* Inherit the constructors */
using Animal::Animal;
/* Trampoline (need one for each virtual function) */
std::string go(int n_times) {
PYBIND_OVERLOAD_PURE(
std::string, /* Return type */
Animal, /* Parent class */
go, /* Name of function */
n_times /* Argument(s) */
);
}
};
The macro :func:`PYBIND_OVERLOAD_PURE` should be used for pure virtual
functions, and :func:`PYBIND_OVERLOAD` should be used for functions which have
a default implementation. The binding code also needs a few minor adaptations
(highlighted):
.. code-block:: cpp
:emphasize-lines: 4,6,7
PYBIND_PLUGIN(example) {
py::module m("example", "pybind example plugin");
py::class_<PyAnimal> animal(m, "Animal");
animal
.alias<Animal>()
.def(py::init<>())
.def("go", &Animal::go);
py::class_<Dog>(m, "Dog", animal)
.def(py::init<>());
m.def("call_go", &call_go);
return m.ptr();
}
Importantly, the trampoline helper class is used as the template argument to
:class:`class_`, and a call to :func:`class_::alias` informs the binding
generator that this is merely an alias for the underlying type ``Animal``.
Following this, we are able to define a constructor as usual.
The Python session below shows how to override ``Animal::go`` and invoke it via
a virtual method call.
.. code-block:: cpp
>>> from example import *
>>> d = Dog()
>>> call_go(d)
u'woof! woof! woof! '
>>> class Cat(Animal):
... def go(self, n_times):
... return "meow! " * n_times
...
>>> c = Cat()
>>> call_go(c)
u'meow! meow! meow! '
.. seealso::
The file :file:`example/example12.cpp` contains a complete example that
demonstrates how to override virtual functions using pybind11 in more
detail.
Passing STL data structures
===========================
When including the additional header file :file:`pybind/stl.h`, conversions
between ``std::vector<>`` and ``std::map<>`` and the Python ``list`` and
``dict`` data structures are automatically enabled. The types ``std::pair<>``
and ``std::tuple<>`` are already supported out of the box with just the core
:file:`pybind/pybind.h` header.
.. note::
Arbitrary nesting of any of these types is explicitly permitted.
.. seealso::
The file :file:`example/example2.cpp` contains a complete example that
demonstrates how to pass STL data types in more detail.
Binding sequence data types, the slicing protocol, etc.
=======================================================
Please refer to the supplemental example for details.
.. seealso::
The file :file:`example/example6.cpp` contains a complete example that
shows how to bind a sequence data type, including length queries
(``__len__``), iterators (``__iter__``), the slicing protocol and other
kinds of useful operations.
Return value policies
=====================
Functions taking Python objects as arguments
============================================
Python and C++ use wildly different ways of managing the memory and lifetime of
objects managed by them. This can lead to issues when creating bindings for
functions that return a non-trivial type. Just by looking at the type
information, it is not clear whether Python should take charge of the returned
value and eventually free its resources, or if this is handled on the C++ side.
For this reason, pybind11 provides a several `return value policy` annotations
that can be passed to the :func:`module::def` and :func:`class_::def`
functions. The default policy is :enum:`return_value_policy::automatic``.
Callbacks
=========
+--------------------------------------------------+---------------------------------------------------------------------------+
| Return value policy | Description |
+==================================================+===========================================================================+
| :enum:`return_value_policy::automatic` | Automatic: copy objects returned as values and take ownership of |
| | objects returned as pointers |
+--------------------------------------------------+---------------------------------------------------------------------------+
| :enum:`return_value_policy::copy` | Create a new copy of the returned object, which will be owned by Python |
+--------------------------------------------------+---------------------------------------------------------------------------+
| :enum:`return_value_policy::take_ownership` | Reference the existing object and take ownership. Python will call |
| | the destructor and delete operator when the reference count reaches zero |
+--------------------------------------------------+---------------------------------------------------------------------------+
| :enum:`return_value_policy::reference` | Reference the object, but do not take ownership and defer responsibility |
| | for deleting it to C++ (dangerous when C++ code at some point decides to |
| | delete it while Python still has a nonzero reference count) |
+--------------------------------------------------+---------------------------------------------------------------------------+
| :enum:`return_value_policy::reference_internal` | Reference the object, but do not take ownership. The object is considered |
| | be owned by the C++ instance whose method or property returned it. The |
| | Python object will increase the reference count of this 'parent' by 1 |
| | to ensure that it won't be deallocated while Python is using the 'child' |
+--------------------------------------------------+---------------------------------------------------------------------------+
.. warning::
Code with invalid call policies might access unitialized memory and free
data structures multiple times, which can lead to hard-to-debug
non-determinism and segmentation faults, hence it is worth spending the
time to understand all the different options above.
See below for an example that uses the
:enum:`return_value_policy::reference_internal` policy.
.. code-block:: cpp
class Example {
public:
Internal &get_internal() { return internal; }
private:
Internal internal;
};
PYBIND_PLUGIN(example) {
py::module m("example", "pybind example plugin");
py::class_<Example>(m, "Example")
.def(py::init<>())
.def("get_internal", &Example::get_internal, "Return the internal data", py::return_value_policy::reference_internal)
return m.ptr();
}
Implicit type conversions
=========================
Suppose that instances of two types ``A`` and ``B`` are used in a project, and
that an ``A`` can easily be converted into a an instance of type ``B`` (examples of this
could be a fixed and an arbitrary precision number type).
.. code-block:: cpp
py::class_<A>(m, "A")
/// ... members ...
py::class_<B>(m, "B")
.def(py::init<A>())
/// ... members ...
m.def("func",
[](const B &) { /* .... */ }
);
To invoke the function ``func`` using a variable ``a`` containing an ``A``
instance, we'd have to write ``func(B(a))`` in Python. On the other hand, C++
will automatically apply an implicit type conversion, which makes it possible
to directly write ``func(a)``.
In this situation (i.e. where ``B`` has a constructor that converts from
``A``), the following statement enables similar implicit conversions on the
Python side:
.. code-block:: cpp
py::implicitly_convertible<A, B>();
Smart pointers
==============
The binding generator for classes (:class:`class_`) takes an optional second
template type, which denotes a special *holder* type that is used to manage
references to the object. When wrapping a type named ``Type``, the default
value of this template parameter is ``std::unique_ptr<Type>``, which means that
the object is deallocated when Python's reference count goes to zero.
It is possible to switch to other types of smart pointers, which is useful in
codebases that rely on them. For instance, the following snippet causes
``std::shared_ptr`` to be used instead.
.. code-block:: cpp
py::class_<Example, std::shared_ptr<Example>> obj(m, "Example");
.. seealso::
The file :file:`example/example8.cpp` contains a complete example that
demonstrates how to work with custom smart pointer types in more detail.
.. _custom_constructors:
Custom constructors
===================
The syntax for binding constructors was previously introduced, but it only
works when a constructor with the given parameters actually exists on the C++
side. To extend this to more general cases, let's take a look at what actually
happens under the hood: the following statement
.. code-block:: cpp
py::class_<Example>(m, "Example")
.def(py::init<int>());
is short hand notation for
.. code-block:: cpp
py::class_<Example>(m, "Example")
.def("__init__",
[](Example &instance, int arg) {
new (&instance) Example(arg);
}
);
In other words, :func:`init` creates an anonymous function that invokes an
in-place constructor. Memory allocation etc. is already take care of beforehand
within pybind11.
Catching and throwing exceptions
================================
When C++ code invoked from Python throws an ``std::exception``, it is
automatically converted into a Python ``Exception``. pybind11 defines multiple
special exception classes that will map to different types of Python
exceptions:
+----------------------------+------------------------------+
| C++ exception type | Python exception type |
+============================+==============================+
| :class:`std::exception` | ``Exception`` |
+----------------------------+------------------------------+
| :class:`stop_iteration` | ``StopIteration`` (used to |
| | implement custom iterators) |
+----------------------------+------------------------------+
| :class:`index_error` | ``IndexError`` (used to |
| | indicate out of bounds |
| | accesses in ``__getitem__``, |
| | ``__setitem__``, etc.) |
+----------------------------+------------------------------+
| :class:`error_already_set` | Indicates that the Python |
| | exception flag has already |
| | been initialized. |
+----------------------------+------------------------------+
When a Python function invoked from C++ throws an exception, it is converted
into a C++ exception of type :class:`error_already_set` whose string payload
contains a textual summary.
There is also a special exception :class:`cast_error` that is thrown by
:func:`handle::call` when the input arguments cannot be converted to Python
objects.
Buffer protocol
===============
@ -114,6 +597,11 @@ objects (e.g. a NumPy matrix).
}
});
.. seealso::
The file :file:`example/example7.cpp` contains a complete example that
demonstrates using the buffer protocol with pybind11 in more detail.
NumPy support
=============
@ -122,13 +610,13 @@ restrict the function so that it only accepts NumPy arrays (rather than any
type of Python object satisfying the buffer object protocol).
In many situations, we want to define a function which only accepts a NumPy
array of a certain data type. This is possible via the ``py::array_dtype<T>``
array of a certain data type. This is possible via the ``py::array_t<T>``
template. For instance, the following function requires the argument to be a
dense array of doubles in C-style ordering.
.. code-block:: cpp
void f(py::array_dtype<double> array);
void f(py::array_t<double> array);
When it is invoked with a different type (e.g. an integer), the binding code
will attempt to cast the input into a NumPy array of the requested type.
@ -181,22 +669,41 @@ This can be done with a stateful Lambda closure:
// Vectorize a lambda function with a capture object (e.g. to exclude some arguments from the vectorization)
m.def("vectorized_func",
[](py::array_dtype<int> x, py::array_dtype<float> y, my_custom_type *z) {
[](py::array_t<int> x, py::array_t<float> y, my_custom_type *z) {
auto stateful_closure = [z](int x, float y) { return my_func(x, y, z); };
return py::vectorize(stateful_closure)(x, y);
}
);
Throwing exceptions
===================
.. seealso::
STL data structures
===================
The file :file:`example/example10.cpp` contains a complete example that
demonstrates using :func:`vectorize` in more detail.
Smart pointers
==============
Functions taking Python objects as arguments
============================================
.. _custom_constructors:
pybind11 exposes all major Python types using thin C++ wrapper classes. These
wrapper classes can also be used as parameters of functions in bindings, which
makes it possible to directly work with native Python types on the C++ side.
For instance, the following statement iterates over a Python ``dict``:
.. code-block:: cpp
void print_dict(py::dict dict) {
/* Easily interact with Python types */
for (auto item : dict)
std::cout << "key=" << item.first << ", "
<< "value=" << item.second << std::endl;
}
Available types include :class:`handle`, :class:`object`, :class:`bool_`,
:class:`int_`, :class:`float_`, :class:`str`, :class:`tuple`, :class:`list`,
:class:`dict`, :class:`slice`, :class:`capsule`, :class:`function`,
:class:`buffer`, :class:`array`, and :class:`array_t`.
.. seealso::
The file :file:`example/example2.cpp` contains a complete example that
demonstrates passing native Python types in more detail.
Custom constructors
===================

View File

@ -87,7 +87,7 @@ a file named :file:`example.cpp` with the following contents:
namespace py = pybind;
PYTHON_PLUGIN(example) {
PYBIND_PLUGIN(example) {
py::module m("example", "pybind example plugin");
m.def("add", &add, "A function which adds two numbers");
@ -95,7 +95,7 @@ a file named :file:`example.cpp` with the following contents:
return m.ptr();
}
The :func:`PYTHON_PLUGIN` macro creates a function that will be called when an
The :func:`PYBIND_PLUGIN` macro creates a function that will be called when an
``import`` statement is issued from within Python. The next line creates a
module named ``example`` (with the supplied docstring). The method
:func:`module::def` generates binding code that exposes the
@ -130,13 +130,14 @@ shows how to load and execute the example.
.. code-block:: python
% python
$ python
Python 2.7.10 (default, Aug 22 2015, 20:33:39)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.59.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import example
>>> example.add(1, 2)
3L
>>>
.. _keyword_args:
@ -219,7 +220,7 @@ Supported data types
The following basic data types are supported out of the box (some may require
an additional extension header to be included). To pass other data structures
as arguments and return values, refer to the section on :ref:`classes`.
as arguments and return values, refer to the section on binding :ref:`classes`.
+------------------------+--------------------------+---------------------+
| Data type | Description | Header file |

View File

@ -27,7 +27,7 @@ The binding code for ``Pet`` looks as follows:
namespace py = pybind;
PYTHON_PLUGIN(example) {
PYBIND_PLUGIN(example) {
py::module m("example", "pybind example plugin");
py::class_<Pet>(m, "Pet")
@ -140,7 +140,8 @@ that can only be accessed via setters and getters.
In this case, the method :func:`class_::def_property`
(:func:`class_::def_property_readonly` for read-only data) can be used to
provide an interface that is indistinguishable from within Python:
provide a field-like interface within Python that will transparently call
the setter and getter functions:
.. code-block:: cpp
@ -249,13 +250,18 @@ The overload signatures are also visible in the method's docstring:
| 2. Signature : (Pet, str) -> None
|
| Set the pet's name
|
.. note::
To define multiple overloaded constructors, simply declare one after the
other using the ``.def(py::init<...>())`` syntax. The existing machinery
for specifying keyword and default arguments also works.
Enumerations and internal types
===============================
Let's now suppose that the example class also contains an internal enumeration
type.
Let's now suppose that the example class contains an internal enumeration type,
e.g.:
.. code-block:: cpp
@ -288,9 +294,9 @@ The binding code for this example looks as follows:
To ensure that the ``Kind`` type is created within the scope of ``Pet``, the
``pet`` :class:`class_` instance must be supplied to the :class:`enum_`.
constructor. The :func:`enum_::export_values` function ensures that the enum
entries are exported into the parent scope; skip this call for new C++11-style
strongly typed enums.
constructor. The :func:`enum_::export_values` function exports the enum entries
into the parent scope, which should be skipped for newer C++11-style strongly
typed enums.
.. code-block:: python
@ -301,4 +307,4 @@ strongly typed enums.
1L
.. [#f1] (those with an empty pair of brackets ``[]`` as the capture object)
.. [#f1] Stateless closures are those with an empty pair of brackets ``[]`` as the capture object.

View File

@ -116,6 +116,11 @@ if not on_rtd: # only import and set the theme if we're building docs locally
import sphinx_rtd_theme
html_theme = 'sphinx_rtd_theme'
html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]
html_context = {
'css_files': [
'_static/theme_overrides.css',
]
}
#import alabaster

View File

@ -21,7 +21,9 @@ everything stripped away that isn't relevant for binding generation. The whole
codebase requires less than 3000 lines of code and only depends on Python (2.7
or 3.x) and the C++ standard library. This compact implementation was possible
thanks to some of the new C++11 language features (tuples, lambda functions and
variadic templates).
variadic templates). Since its creation, this library has grown beyond
Boost.Python in many ways, leading to dramatically simpler binding code in many
common situations.
Core features
*************

View File

@ -1,12 +1,18 @@
.. _reference:
.. warning::
Please be advised that the reference documentation discussing pybind11
internals is currently incomplete. Please refer to the previous sections
and the pybind header files for the nitty gritty details.
Reference
#########
Macros
======
.. function:: PYTHON_PLUGIN(const char *name)
.. function:: PYBIND_PLUGIN(const char *name)
This macro creates the entry point that will be invoked when the Python
interpreter imports a plugin library. Please create a
@ -15,7 +21,7 @@ Macros
.. code-block:: cpp
PYTHON_PLUGIN(example) {
PYBIND_PLUGIN(example) {
pybind::module m("example", "pybind example plugin");
/// Set up bindings here
return m.ptr();
@ -122,7 +128,7 @@ Without reference counting
Assuming the Python object is a function or implements the ``__call__``
protocol, ``call()`` invokes the underlying function, passing an arbitrary
set of parameters. The result is returned as a :class:`object` and may need
to be converted back into a Python object using :func:`template <typename T> handle::cast`.
to be converted back into a Python object using :func:`handle::cast`.
When some of the arguments cannot be converted to Python objects, the
function will throw a :class:`cast_error` exception. When the Python