Reorganize documentation

docs/advanced/cast/chrono.rst

@@ -0,0 +1,81 @@
+Chrono
+======
+
+When including the additional header file :file:`pybind11/chrono.h` conversions
+from C++11 chrono datatypes to python datetime objects are automatically enabled.
+This header also enables conversions of python floats (often from sources such
+as `time.monotonic()`, `time.perf_counter()` and `time.process_time()`) into
+durations.
+
+An overview of clocks in C++11
+------------------------------
+
+A point of confusion when using these conversions is the differences between
+clocks provided in C++11. There are three clock types defined by the C++11
+standard and users can define their own if needed. Each of these clocks have
+different properties and when converting to and from python will give different
+results.
+
+The first clock defined by the standard is ``std::chrono::system_clock``. This
+clock measures the current date and time. However, this clock changes with to
+updates to the operating system time. For example, if your time is synchronised
+with a time server this clock will change. This makes this clock a poor choice
+for timing purposes but good for measuring the wall time.
+
+The second clock defined in the standard is ``std::chrono::steady_clock``.
+This clock ticks at a steady rate and is never adjusted. This makes it excellent
+for timing purposes, however the value in this clock does not correspond to the
+current date and time. Often this clock will be the amount of time your system
+has been on, although it does not have to be. This clock will never be the same
+clock as the system clock as the system clock can change but steady clocks
+cannot.
+
+The third clock defined in the standard is ``std::chrono::high_resolution_clock``.
+This clock is the clock that has the highest resolution out of the clocks in the
+system. It is normally a typedef to either the system clock or the steady clock
+but can be its own independent clock. This is important as when using these
+conversions as the types you get in python for this clock might be different
+depending on the system.
+If it is a typedef of the system clock, python will get datetime objects, but if
+it is a different clock they will be timedelta objects.
+
+Conversions Provided
+--------------------
+
+.. rubric:: C++ to Python
+
+- ``std::chrono::system_clock::time_point`` → ``datetime.datetime``
+    System clock times are converted to python datetime instances. They are
+    in the local timezone, but do not have any timezone information attached
+    to them (they are naive datetime objects).
+
+- ``std::chrono::duration`` → ``datetime.timedelta``
+    Durations are converted to timedeltas, any precision in the duration
+    greater than microseconds is lost by rounding towards zero.
+
+- ``std::chrono::[other_clocks]::time_point`` → ``datetime.timedelta``
+    Any clock time that is not the system clock is converted to a time delta.
+    This timedelta measures the time from the clocks epoch to now.
+
+.. rubric:: Python to C++
+
+- ``datetime.datetime`` → ``std::chrono::system_clock::time_point``
+    Date/time objects are converted into system clock timepoints. Any
+    timezone information is ignored and the type is treated as a naive
+    object.
+
+- ``datetime.timedelta`` → ``std::chrono::duration``
+    Time delta are converted into durations with microsecond precision.
+
+- ``datetime.timedelta`` → ``std::chrono::[other_clocks]::time_point``
+    Time deltas that are converted into clock timepoints are treated as
+    the amount of time from the start of the clocks epoch.
+
+- ``float`` → ``std::chrono::duration``
+    Floats that are passed to C++ as durations be interpreted as a number of
+    seconds. These will be converted to the duration using ``duration_cast``
+    from the float.
+
+- ``float`` → ``std::chrono::[other_clocks]::time_point``
+    Floats that are passed to C++ as time points will be interpreted as the
+    number of seconds from the start of the clocks epoch.

docs/advanced/cast/custom.rst

@@ -0,0 +1,79 @@
+Custom type casters
+===================
+
+In very rare cases, applications may require custom type casters that cannot be
+expressed using the abstractions provided by pybind11, thus requiring raw
+Python C API calls. This is fairly advanced usage and should only be pursued by
+experts who are familiar with the intricacies of Python reference counting.
+
+The following snippets demonstrate how this works for a very simple ``inty``
+type that that should be convertible from Python types that provide a
+``__int__(self)`` method.
+
+.. code-block:: cpp
+
+    struct inty { long long_value; };
+
+    void print(inty s) {
+        std::cout << s.long_value << std::endl;
+    }
+
+The following Python snippet demonstrates the intended usage from the Python side:
+
+.. code-block:: python
+
+    class A:
+        def __int__(self):
+            return 123
+
+    from example import print
+    print(A())
+
+To register the necessary conversion routines, it is necessary to add
+a partial overload to the ``pybind11::detail::type_caster<T>`` template.
+Although this is an implementation detail, adding partial overloads to this
+type is explicitly allowed.
+
+.. code-block:: cpp
+
+    namespace pybind11 { namespace detail {
+        template <> struct type_caster<inty> {
+        public:
+            /**
+             * This macro establishes the name 'inty' in
+             * function signatures and declares a local variable
+             * 'value' of type inty
+             */
+            PYBIND11_TYPE_CASTER(inty, _("inty"));
+
+            /**
+             * Conversion part 1 (Python->C++): convert a PyObject into a inty
+             * instance or return false upon failure. The second argument
+             * indicates whether implicit conversions should be applied.
+             */
+            bool load(handle src, bool) {
+                /* Extract PyObject from handle */
+                PyObject *source = src.ptr();
+                /* Try converting into a Python integer value */
+                PyObject *tmp = PyNumber_Long(source);
+                if (!tmp)
+                    return false;
+                /* Now try to convert into a C++ int */
+                value.long_value = PyLong_AsLong(tmp);
+                Py_DECREF(tmp);
+                /* Ensure return code was OK (to avoid out-of-range errors etc) */
+                return !(value.long_value == -1 && !PyErr_Occurred());
+            }
+
+            /**
+             * Conversion part 2 (C++ -> Python): convert an inty instance into
+             * a Python object. The second and third arguments are used to
+             * indicate the return value policy and parent object (for
+             * ``return_value_policy::reference_internal``) and are generally
+             * ignored by implicit casters.
+             */
+            static handle cast(inty src, return_value_policy /* policy */, handle /* parent */) {
+                return PyLong_FromLong(src.long_value);
+            }
+        };
+    }} // namespace pybind11::detail

docs/advanced/cast/eigen.rst

@@ -0,0 +1,50 @@
+Eigen
+=====
+
+`Eigen <http://eigen.tuxfamily.org>`_ is C++ header-based library for dense and
+sparse linear algebra. Due to its popularity and widespread adoption, pybind11
+provides transparent conversion support between Eigen and Scientific Python linear
+algebra data types.
+
+Specifically, when including the optional header file :file:`pybind11/eigen.h`,
+pybind11 will automatically and transparently convert
+
+1. Static and dynamic Eigen dense vectors and matrices to instances of
+   ``numpy.ndarray`` (and vice versa).
+
+2. Returned matrix expressions such as blocks (including columns or rows) and
+   diagonals will be converted to ``numpy.ndarray`` of the expression
+   values.
+
+3. Returned matrix-like objects such as Eigen::DiagonalMatrix or
+   Eigen::SelfAdjointView will be converted to ``numpy.ndarray`` containing the
+   expressed value.
+
+4. Eigen sparse vectors and matrices to instances of
+   ``scipy.sparse.csr_matrix``/``scipy.sparse.csc_matrix`` (and vice versa).
+
+This makes it possible to bind most kinds of functions that rely on these types.
+One major caveat are functions that take Eigen matrices *by reference* and modify
+them somehow, in which case the information won't be propagated to the caller.
+
+.. code-block:: cpp
+
+    /* The Python bindings of these functions won't replicate
+       the intended effect of modifying the function arguments */
+    void scale_by_2(Eigen::Vector3f &v) {
+        v *= 2;
+    }
+    void scale_by_2(Eigen::Ref<Eigen::MatrixXd> &v) {
+        v *= 2;
+    }
+
+To see why this is, refer to the section on :ref:`opaque` (although that
+section specifically covers STL data types, the underlying issue is the same).
+The :ref:`numpy` sections discuss an efficient alternative for exposing the
+underlying native Eigen types as opaque objects in a way that still integrates
+with NumPy and SciPy.
+
+.. seealso::
+
+    The file :file:`tests/test_eigen.cpp` contains a complete example that
+    shows how to pass Eigen sparse and dense data types in more detail.

docs/advanced/cast/functional.rst

@@ -0,0 +1,113 @@
+Functional
+##########
+
+The following features must be enabled by including :file:`pybind11/functional.h`.
+
+
+Callbacks and passing anonymous functions
+=========================================
+
+The C++11 standard brought lambda functions and the generic polymorphic
+function wrapper ``std::function<>`` to the C++ programming language, which
+enable powerful new ways of working with functions. Lambda functions come in
+two flavors: stateless lambda function resemble classic function pointers that
+link to an anonymous piece of code, while stateful lambda functions
+additionally depend on captured variables that are stored in an anonymous
+*lambda closure object*.
+
+Here is a simple example of a C++ function that takes an arbitrary function
+(stateful or stateless) with signature ``int -> int`` as an argument and runs
+it with the value 10.
+
+.. code-block:: cpp
+
+    int func_arg(const std::function<int(int)> &f) {
+        return f(10);
+    }
+
+The example below is more involved: it takes a function of signature ``int -> int``
+and returns another function of the same kind. The return value is a stateful
+lambda function, which stores the value ``f`` in the capture object and adds 1 to
+its return value upon execution.
+
+.. code-block:: cpp
+
+    std::function<int(int)> func_ret(const std::function<int(int)> &f) {
+        return [f](int i) {
+            return f(i) + 1;
+        };
+    }
+
+This example demonstrates using python named parameters in C++ callbacks which
+requires using ``py::cpp_function`` as a wrapper. Usage is similar to defining
+methods of classes:
+
+.. code-block:: cpp
+
+    py::cpp_function func_cpp() {
+        return py::cpp_function([](int i) { return i+1; },
+           py::arg("number"));
+    }
+
+After including the extra header file :file:`pybind11/functional.h`, it is almost
+trivial to generate binding code for all of these functions.
+
+.. code-block:: cpp
+
+    #include <pybind11/functional.h>
+
+    PYBIND11_PLUGIN(example) {
+        py::module m("example", "pybind11 example plugin");
+
+        m.def("func_arg", &func_arg);
+        m.def("func_ret", &func_ret);
+        m.def("func_cpp", &func_cpp);
+
+        return m.ptr();
+    }
+
+The following interactive session shows how to call them from Python.
+
+.. code-block:: pycon
+
+    $ python
+    >>> import example
+    >>> def square(i):
+    ...     return i * i
+    ...
+    >>> example.func_arg(square)
+    100L
+    >>> square_plus_1 = example.func_ret(square)
+    >>> square_plus_1(4)
+    17L
+    >>> plus_1 = func_cpp()
+    >>> plus_1(number=43)
+    44L
+
+.. warning::
+
+    Keep in mind that passing a function from C++ to Python (or vice versa)
+    will instantiate a piece of wrapper code that translates function
+    invocations between the two languages. Naturally, this translation
+    increases the computational cost of each function call somewhat. A
+    problematic situation can arise when a function is copied back and forth
+    between Python and C++ many times in a row, in which case the underlying
+    wrappers will accumulate correspondingly. The resulting long sequence of
+    C++ -> Python -> C++ -> ... roundtrips can significantly decrease
+    performance.
+
+    There is one exception: pybind11 detects case where a stateless function
+    (i.e. a function pointer or a lambda function without captured variables)
+    is passed as an argument to another C++ function exposed in Python. In this
+    case, there is no overhead. Pybind11 will extract the underlying C++
+    function pointer from the wrapped function to sidestep a potential C++ ->
+    Python -> C++ roundtrip. This is demonstrated in :file:`tests/test_callbacks.cpp`.
+
+.. note::
+
+    This functionality is very useful when generating bindings for callbacks in
+    C++ libraries (e.g. GUI libraries, asynchronous networking libraries, etc.).
+
+    The file :file:`tests/test_callbacks.cpp` contains a complete example
+    that demonstrates how to work with callbacks and anonymous functions in
+    more detail.

docs/advanced/cast/index.rst

@@ -0,0 +1,79 @@
+Casting data types
+##################
+
+.. toctree::
+   :maxdepth: 1
+
+   stl
+   functional
+   chrono
+   eigen
+   custom
+
+The following basic data types are supported out of the box (some may require
+an additional extension header to be included). To pass other data structures
+as arguments and return values, refer to the section on binding :ref:`classes`.
+
++---------------------------------+--------------------------+-------------------------------+
+|  Data type                      |  Description             | Header file                   |
++=================================+==========================+===============================+
+| ``int8_t``, ``uint8_t``         | 8-bit integers           | :file:`pybind11/pybind11.h`   |
++---------------------------------+--------------------------+-------------------------------+
+| ``int16_t``, ``uint16_t``       | 16-bit integers          | :file:`pybind11/pybind11.h`   |
++---------------------------------+--------------------------+-------------------------------+
+| ``int32_t``, ``uint32_t``       | 32-bit integers          | :file:`pybind11/pybind11.h`   |
++---------------------------------+--------------------------+-------------------------------+
+| ``int64_t``, ``uint64_t``       | 64-bit integers          | :file:`pybind11/pybind11.h`   |
++---------------------------------+--------------------------+-------------------------------+
+| ``ssize_t``, ``size_t``         | Platform-dependent size  | :file:`pybind11/pybind11.h`   |
++---------------------------------+--------------------------+-------------------------------+
+| ``float``, ``double``           | Floating point types     | :file:`pybind11/pybind11.h`   |
++---------------------------------+--------------------------+-------------------------------+
+| ``bool``                        | Two-state Boolean type   | :file:`pybind11/pybind11.h`   |
++---------------------------------+--------------------------+-------------------------------+
+| ``char``                        | Character literal        | :file:`pybind11/pybind11.h`   |
++---------------------------------+--------------------------+-------------------------------+
+| ``wchar_t``                     | Wide character literal   | :file:`pybind11/pybind11.h`   |
++---------------------------------+--------------------------+-------------------------------+
+| ``const char *``                | UTF-8 string literal     | :file:`pybind11/pybind11.h`   |
++---------------------------------+--------------------------+-------------------------------+
+| ``const wchar_t *``             | Wide string literal      | :file:`pybind11/pybind11.h`   |
++---------------------------------+--------------------------+-------------------------------+
+| ``std::string``                 | STL dynamic UTF-8 string | :file:`pybind11/pybind11.h`   |
++---------------------------------+--------------------------+-------------------------------+
+| ``std::wstring``                | STL dynamic wide string  | :file:`pybind11/pybind11.h`   |
++---------------------------------+--------------------------+-------------------------------+
+| ``std::pair<T1, T2>``           | Pair of two custom types | :file:`pybind11/pybind11.h`   |
++---------------------------------+--------------------------+-------------------------------+
+| ``std::tuple<...>``             | Arbitrary tuple of types | :file:`pybind11/pybind11.h`   |
++---------------------------------+--------------------------+-------------------------------+
+| ``std::reference_wrapper<...>`` | Reference type wrapper   | :file:`pybind11/pybind11.h`   |
++---------------------------------+--------------------------+-------------------------------+
+| ``std::complex<T>``             | Complex numbers          | :file:`pybind11/complex.h`    |
++---------------------------------+--------------------------+-------------------------------+
+| ``std::array<T, Size>``         | STL static array         | :file:`pybind11/stl.h`        |
++---------------------------------+--------------------------+-------------------------------+
+| ``std::vector<T>``              | STL dynamic array        | :file:`pybind11/stl.h`        |
++---------------------------------+--------------------------+-------------------------------+
+| ``std::list<T>``                | STL linked list          | :file:`pybind11/stl.h`        |
++---------------------------------+--------------------------+-------------------------------+
+| ``std::map<T1, T2>``            | STL ordered map          | :file:`pybind11/stl.h`        |
++---------------------------------+--------------------------+-------------------------------+
+| ``std::unordered_map<T1, T2>``  | STL unordered map        | :file:`pybind11/stl.h`        |
++---------------------------------+--------------------------+-------------------------------+
+| ``std::set<T>``                 | STL ordered set          | :file:`pybind11/stl.h`        |
++---------------------------------+--------------------------+-------------------------------+
+| ``std::unordered_set<T>``       | STL unordered set        | :file:`pybind11/stl.h`        |
++---------------------------------+--------------------------+-------------------------------+
+| ``std::function<...>``          | STL polymorphic function | :file:`pybind11/functional.h` |
++---------------------------------+--------------------------+-------------------------------+
+| ``std::chrono::duration<...>``  | STL time duration        | :file:`pybind11/chrono.h`     |
++---------------------------------+--------------------------+-------------------------------+
+| ``std::chrono::time_point<...>``| STL date/time            | :file:`pybind11/chrono.h`     |
++---------------------------------+--------------------------+-------------------------------+
+| ``Eigen::Matrix<...>``          | Eigen: dense matrix      | :file:`pybind11/eigen.h`      |
++---------------------------------+--------------------------+-------------------------------+
+| ``Eigen::Map<...>``             | Eigen: mapped memory     | :file:`pybind11/eigen.h`      |
++---------------------------------+--------------------------+-------------------------------+
+| ``Eigen::SparseMatrix<...>``    | Eigen: sparse matrix     | :file:`pybind11/eigen.h`      |
++---------------------------------+--------------------------+-------------------------------+

docs/advanced/cast/stl.rst

@@ -0,0 +1,154 @@
+STL containers
+##############
+
+Automatic conversion
+====================
+
+When including the additional header file :file:`pybind11/stl.h`, conversions
+between ``std::vector<>``, ``std::list<>``, ``std::set<>``, and ``std::map<>``
+and the Python ``list``, ``set`` and ``dict`` data structures are automatically
+enabled. The types ``std::pair<>`` and ``std::tuple<>`` are already supported
+out of the box with just the core :file:`pybind11/pybind11.h` header.
+
+The major downside of these implicit conversions is that containers must be
+converted (i.e. copied) on every Python->C++ and C++->Python transition, which
+can have implications on the program semantics and performance. Please read the
+next sections for more details and alternative approaches that avoid this.
+
+.. note::
+
+    Arbitrary nesting of any of these types is possible.
+
+.. seealso::
+
+    The file :file:`tests/test_python_types.cpp` contains a complete
+    example that demonstrates how to pass STL data types in more detail.
+
+.. _opaque:
+
+Making opaque types
+===================
+
+pybind11 heavily relies on a template matching mechanism to convert parameters
+and return values that are constructed from STL data types such as vectors,
+linked lists, hash tables, etc. This even works in a recursive manner, for
+instance to deal with lists of hash maps of pairs of elementary and custom
+types, etc.
+
+However, a fundamental limitation of this approach is that internal conversions
+between Python and C++ types involve a copy operation that prevents
+pass-by-reference semantics. What does this mean?
+
+Suppose we bind the following function
+
+.. code-block:: cpp
+
+    void append_1(std::vector<int> &v) {
+       v.push_back(1);
+    }
+
+and call it from Python, the following happens:
+
+.. code-block:: pycon
+
+   >>> v = [5, 6]
+   >>> append_1(v)
+   >>> print(v)
+   [5, 6]
+
+As you can see, when passing STL data structures by reference, modifications
+are not propagated back the Python side. A similar situation arises when
+exposing STL data structures using the ``def_readwrite`` or ``def_readonly``
+functions:
+
+.. code-block:: cpp
+
+    /* ... definition ... */
+
+    class MyClass {
+        std::vector<int> contents;
+    };
+
+    /* ... binding code ... */
+
+    py::class_<MyClass>(m, "MyClass")
+        .def(py::init<>)
+        .def_readwrite("contents", &MyClass::contents);
+
+In this case, properties can be read and written in their entirety. However, an
+``append`` operation involving such a list type has no effect:
+
+.. code-block:: pycon
+
+   >>> m = MyClass()
+   >>> m.contents = [5, 6]
+   >>> print(m.contents)
+   [5, 6]
+   >>> m.contents.append(7)
+   >>> print(m.contents)
+   [5, 6]
+
+Finally, the involved copy operations can be costly when dealing with very
+large lists. To deal with all of the above situations, pybind11 provides a
+macro named ``PYBIND11_MAKE_OPAQUE(T)`` that disables the template-based
+conversion machinery of types, thus rendering them *opaque*. The contents of
+opaque objects are never inspected or extracted, hence they *can* be passed by
+reference. For instance, to turn ``std::vector<int>`` into an opaque type, add
+the declaration
+
+.. code-block:: cpp
+
+    PYBIND11_MAKE_OPAQUE(std::vector<int>);
+
+before any binding code (e.g. invocations to ``class_::def()``, etc.). This
+macro must be specified at the top level (and outside of any namespaces), since
+it instantiates a partial template overload. If your binding code consists of
+multiple compilation units, it must be present in every file preceding any
+usage of ``std::vector<int>``. Opaque types must also have a corresponding
+``class_`` declaration to associate them with a name in Python, and to define a
+set of available operations, e.g.:
+
+.. code-block:: cpp
+
+    py::class_<std::vector<int>>(m, "IntVector")
+        .def(py::init<>())
+        .def("clear", &std::vector<int>::clear)
+        .def("pop_back", &std::vector<int>::pop_back)
+        .def("__len__", [](const std::vector<int> &v) { return v.size(); })
+        .def("__iter__", [](std::vector<int> &v) {
+           return py::make_iterator(v.begin(), v.end());
+        }, py::keep_alive<0, 1>()) /* Keep vector alive while iterator is used */
+        // ....
+
+The ability to expose STL containers as native Python objects is a fairly
+common request, hence pybind11 also provides an optional header file named
+:file:`pybind11/stl_bind.h` that does exactly this. The mapped containers try
+to match the behavior of their native Python counterparts as much as possible.
+
+The following example showcases usage of :file:`pybind11/stl_bind.h`:
+
+.. code-block:: cpp
+
+    // Don't forget this
+    #include <pybind11/stl_bind.h>
+
+    PYBIND11_MAKE_OPAQUE(std::vector<int>);
+    PYBIND11_MAKE_OPAQUE(std::map<std::string, double>);
+
+    // ...
+
+    // later in binding code:
+    py::bind_vector<std::vector<int>>(m, "VectorInt");
+    py::bind_map<std::map<std::string, double>>(m, "MapStringDouble");
+
+Please take a look at the :ref:`macro_notes` before using the
+``PYBIND11_MAKE_OPAQUE`` macro.
+
+.. seealso::
+
+    The file :file:`tests/test_opaque_types.cpp` contains a complete
+    example that demonstrates how to create and expose opaque types using
+    pybind11 in more detail.
+
+    The file :file:`tests/test_stl_binders.cpp` shows how to use the
+    convenience STL container wrappers.

docs/advanced/classes.rst

@@ -0,0 +1,634 @@
+Classes
+#######
+
+This section presents advanced binding code for classes and it is assumed
+that you are already familiar with the basics from :doc:`/classes`.
+
+.. _overriding_virtuals:
+
+Overriding virtual functions in Python
+======================================
+
+Suppose that a C++ class or interface has a virtual function that we'd like to
+to override from within Python (we'll focus on the class ``Animal``; ``Dog`` is
+given as a specific example of how one would do this with traditional C++
+code).
+
+.. code-block:: cpp
+
+    class Animal {
+    public:
+        virtual ~Animal() { }
+        virtual std::string go(int n_times) = 0;
+    };
+
+    class Dog : public Animal {
+    public:
+        std::string go(int n_times) override {
+            std::string result;
+            for (int i=0; i<n_times; ++i)
+                result += "woof! ";
+            return result;
+        }
+    };
+
+Let's also suppose that we are given a plain function which calls the
+function ``go()`` on an arbitrary ``Animal`` instance.
+
+.. code-block:: cpp
+
+    std::string call_go(Animal *animal) {
+        return animal->go(3);
+    }
+
+Normally, the binding code for these classes would look as follows:
+
+.. code-block:: cpp
+
+    PYBIND11_PLUGIN(example) {
+        py::module m("example", "pybind11 example plugin");
+
+        py::class_<Animal> animal(m, "Animal");
+        animal
+            .def("go", &Animal::go);
+
+        py::class_<Dog>(m, "Dog", animal)
+            .def(py::init<>());
+
+        m.def("call_go", &call_go);
+
+        return m.ptr();
+    }
+
+However, these bindings are impossible to extend: ``Animal`` is not
+constructible, and we clearly require some kind of "trampoline" that
+redirects virtual calls back to Python.
+
+Defining a new type of ``Animal`` from within Python is possible but requires a
+helper class that is defined as follows:
+
+.. code-block:: cpp
+
+    class PyAnimal : public Animal {
+    public:
+        /* Inherit the constructors */
+        using Animal::Animal;
+
+        /* Trampoline (need one for each virtual function) */
+        std::string go(int n_times) override {
+            PYBIND11_OVERLOAD_PURE(
+                std::string, /* Return type */
+                Animal,      /* Parent class */
+                go,          /* Name of function */
+                n_times      /* Argument(s) */
+            );
+        }
+    };
+
+The macro :func:`PYBIND11_OVERLOAD_PURE` should be used for pure virtual
+functions, and :func:`PYBIND11_OVERLOAD` should be used for functions which have
+a default implementation.  There are also two alternate macros
+:func:`PYBIND11_OVERLOAD_PURE_NAME` and :func:`PYBIND11_OVERLOAD_NAME` which
+take a string-valued name argument between the *Parent class* and *Name of the
+function* slots. This is useful when the C++ and Python versions of the
+function have different names, e.g.  ``operator()`` vs ``__call__``.
+
+The binding code also needs a few minor adaptations (highlighted):
+
+.. code-block:: cpp
+    :emphasize-lines: 4,6,7
+
+    PYBIND11_PLUGIN(example) {
+        py::module m("example", "pybind11 example plugin");
+
+        py::class_<Animal, PyAnimal /* <--- trampoline*/> animal(m, "Animal");
+        animal
+            .def(py::init<>())
+            .def("go", &Animal::go);
+
+        py::class_<Dog>(m, "Dog", animal)
+            .def(py::init<>());
+
+        m.def("call_go", &call_go);
+
+        return m.ptr();
+    }
+
+Importantly, pybind11 is made aware of the trampoline helper class by
+specifying it as an extra template argument to :class:`class_`.  (This can also
+be combined with other template arguments such as a custom holder type; the
+order of template types does not matter).  Following this, we are able to
+define a constructor as usual.
+
+Note, however, that the above is sufficient for allowing python classes to
+extend ``Animal``, but not ``Dog``: see ref:`virtual_and_inheritance` for the
+necessary steps required to providing proper overload support for inherited
+classes.
+
+The Python session below shows how to override ``Animal::go`` and invoke it via
+a virtual method call.
+
+.. code-block:: pycon
+
+    >>> from example import *
+    >>> d = Dog()
+    >>> call_go(d)
+    u'woof! woof! woof! '
+    >>> class Cat(Animal):
+    ...     def go(self, n_times):
+    ...             return "meow! " * n_times
+    ...
+    >>> c = Cat()
+    >>> call_go(c)
+    u'meow! meow! meow! '
+
+Please take a look at the :ref:`macro_notes` before using this feature.
+
+.. note::
+
+    When the overridden type returns a reference or pointer to a type that
+    pybind11 converts from Python (for example, numeric values, std::string,
+    and other built-in value-converting types), there are some limitations to
+    be aware of:
+
+    - because in these cases there is no C++ variable to reference (the value
+      is stored in the referenced Python variable), pybind11 provides one in
+      the PYBIND11_OVERLOAD macros (when needed) with static storage duration.
+      Note that this means that invoking the overloaded method on *any*
+      instance will change the referenced value stored in *all* instances of
+      that type.
+
+    - Attempts to modify a non-const reference will not have the desired
+      effect: it will change only the static cache variable, but this change
+      will not propagate to underlying Python instance, and the change will be
+      replaced the next time the overload is invoked.
+
+.. seealso::
+
+    The file :file:`tests/test_virtual_functions.cpp` contains a complete
+    example that demonstrates how to override virtual functions using pybind11
+    in more detail.
+
+.. _virtual_and_inheritance:
+
+Combining virtual functions and inheritance
+===========================================
+
+When combining virtual methods with inheritance, you need to be sure to provide
+an override for each method for which you want to allow overrides from derived
+python classes.  For example, suppose we extend the above ``Animal``/``Dog``
+example as follows:
+
+.. code-block:: cpp
+
+    class Animal {
+    public:
+        virtual std::string go(int n_times) = 0;
+        virtual std::string name() { return "unknown"; }
+    };
+    class Dog : public class Animal {
+    public:
+        std::string go(int n_times) override {
+            std::string result;
+            for (int i=0; i<n_times; ++i)
+                result += bark() + " ";
+            return result;
+        }
+        virtual std::string bark() { return "woof!"; }
+    };
+
+then the trampoline class for ``Animal`` must, as described in the previous
+section, override ``go()`` and ``name()``, but in order to allow python code to
+inherit properly from ``Dog``, we also need a trampoline class for ``Dog`` that
+overrides both the added ``bark()`` method *and* the ``go()`` and ``name()``
+methods inherited from ``Animal`` (even though ``Dog`` doesn't directly
+override the ``name()`` method):
+
+.. code-block:: cpp
+
+    class PyAnimal : public Animal {
+    public:
+        using Animal::Animal; // Inherit constructors
+        std::string go(int n_times) override { PYBIND11_OVERLOAD_PURE(std::string, Animal, go, n_times); }
+        std::string name() override { PYBIND11_OVERLOAD(std::string, Animal, name, ); }
+    };
+    class PyDog : public Dog {
+    public:
+        using Dog::Dog; // Inherit constructors
+        std::string go(int n_times) override { PYBIND11_OVERLOAD_PURE(std::string, Dog, go, n_times); }
+        std::string name() override { PYBIND11_OVERLOAD(std::string, Dog, name, ); }
+        std::string bark() override { PYBIND11_OVERLOAD(std::string, Dog, bark, ); }
+    };
+
+A registered class derived from a pybind11-registered class with virtual
+methods requires a similar trampoline class, *even if* it doesn't explicitly
+declare or override any virtual methods itself:
+
+.. code-block:: cpp
+
+    class Husky : public Dog {};
+    class PyHusky : public Husky {
+        using Dog::Dog; // Inherit constructors
+        std::string go(int n_times) override { PYBIND11_OVERLOAD_PURE(std::string, Husky, go, n_times); }
+        std::string name() override { PYBIND11_OVERLOAD(std::string, Husky, name, ); }
+        std::string bark() override { PYBIND11_OVERLOAD(std::string, Husky, bark, ); }
+    };
+
+There is, however, a technique that can be used to avoid this duplication
+(which can be especially helpful for a base class with several virtual
+methods).  The technique involves using template trampoline classes, as
+follows:
+
+.. code-block:: cpp
+
+    template <class AnimalBase = Animal> class PyAnimal : public AnimalBase {
+        using AnimalBase::AnimalBase; // Inherit constructors
+        std::string go(int n_times) override { PYBIND11_OVERLOAD_PURE(std::string, AnimalBase, go, n_times); }
+        std::string name() override { PYBIND11_OVERLOAD(std::string, AnimalBase, name, ); }
+    };
+    template <class DogBase = Dog> class PyDog : public PyAnimal<DogBase> {
+        using PyAnimal<DogBase>::PyAnimal; // Inherit constructors
+        // Override PyAnimal's pure virtual go() with a non-pure one:
+        std::string go(int n_times) override { PYBIND11_OVERLOAD(std::string, DogBase, go, n_times); }
+        std::string bark() override { PYBIND11_OVERLOAD(std::string, DogBase, bark, ); }
+    };
+
+This technique has the advantage of requiring just one trampoline method to be
+declared per virtual method and pure virtual method override.  It does,
+however, require the compiler to generate at least as many methods (and
+possibly more, if both pure virtual and overridden pure virtual methods are
+exposed, as above).
+
+The classes are then registered with pybind11 using:
+
+.. code-block:: cpp
+
+    py::class_<Animal, PyAnimal<>> animal(m, "Animal");
+    py::class_<Dog, PyDog<>> dog(m, "Dog");
+    py::class_<Husky, PyDog<Husky>> husky(m, "Husky");
+    // ... add animal, dog, husky definitions
+
+Note that ``Husky`` did not require a dedicated trampoline template class at
+all, since it neither declares any new virtual methods nor provides any pure
+virtual method implementations.
+
+With either the repeated-virtuals or templated trampoline methods in place, you
+can now create a python class that inherits from ``Dog``:
+
+.. code-block:: python
+
+    class ShihTzu(Dog):
+        def bark(self):
+            return "yip!"
+
+.. seealso::
+
+    See the file :file:`tests/test_virtual_functions.cpp` for complete examples
+    using both the duplication and templated trampoline approaches.
+
+Extended trampoline class functionality
+=======================================
+
+The trampoline classes described in the previous sections are, by default, only
+initialized when needed.  More specifically, they are initialized when a python
+class actually inherits from a registered type (instead of merely creating an
+instance of the registered type), or when a registered constructor is only
+valid for the trampoline class but not the registered class.  This is primarily
+for performance reasons: when the trampoline class is not needed for anything
+except virtual method dispatching, not initializing the trampoline class
+improves performance by avoiding needing to do a run-time check to see if the
+inheriting python instance has an overloaded method.
+
+Sometimes, however, it is useful to always initialize a trampoline class as an
+intermediate class that does more than just handle virtual method dispatching.
+For example, such a class might perform extra class initialization, extra
+destruction operations, and might define new members and methods to enable a
+more python-like interface to a class.
+
+In order to tell pybind11 that it should *always* initialize the trampoline
+class when creating new instances of a type, the class constructors should be
+declared using ``py::init_alias<Args, ...>()`` instead of the usual
+``py::init<Args, ...>()``.  This forces construction via the trampoline class,
+ensuring member initialization and (eventual) destruction.
+
+.. seealso::
+
+    See the file :file:`tests/test_alias_initialization.cpp` for complete examples
+    showing both normal and forced trampoline instantiation.
+
+.. _custom_constructors:
+
+Custom constructors
+===================
+
+The syntax for binding constructors was previously introduced, but it only
+works when a constructor with the given parameters actually exists on the C++
+side. To extend this to more general cases, let's take a look at what actually
+happens under the hood: the following statement
+
+.. code-block:: cpp
+
+    py::class_<Example>(m, "Example")
+        .def(py::init<int>());
+
+is short hand notation for
+
+.. code-block:: cpp
+
+    py::class_<Example>(m, "Example")
+        .def("__init__",
+            [](Example &instance, int arg) {
+                new (&instance) Example(arg);
+            }
+        );
+
+In other words, :func:`init` creates an anonymous function that invokes an
+in-place constructor. Memory allocation etc. is already take care of beforehand
+within pybind11.
+
+.. _classes_with_non_public_destructors:
+
+Non-public destructors
+======================
+
+If a class has a private or protected destructor (as might e.g. be the case in
+a singleton pattern), a compile error will occur when creating bindings via
+pybind11. The underlying issue is that the ``std::unique_ptr`` holder type that
+is responsible for managing the lifetime of instances will reference the
+destructor even if no deallocations ever take place. In order to expose classes
+with private or protected destructors, it is possible to override the holder
+type via a holder type argument to ``class_``. Pybind11 provides a helper class
+``py::nodelete`` that disables any destructor invocations. In this case, it is
+crucial that instances are deallocated on the C++ side to avoid memory leaks.
+
+.. code-block:: cpp
+
+    /* ... definition ... */
+
+    class MyClass {
+    private:
+        ~MyClass() { }
+    };
+
+    /* ... binding code ... */
+
+    py::class_<MyClass, std::unique_ptr<MyClass, py::nodelete>>(m, "MyClass")
+        .def(py::init<>)
+
+Implicit conversions
+====================
+
+Suppose that instances of two types ``A`` and ``B`` are used in a project, and
+that an ``A`` can easily be converted into an instance of type ``B`` (examples of this
+could be a fixed and an arbitrary precision number type).
+
+.. code-block:: cpp
+
+    py::class_<A>(m, "A")
+        /// ... members ...
+
+    py::class_<B>(m, "B")
+        .def(py::init<A>())
+        /// ... members ...
+
+    m.def("func",
+        [](const B &) { /* .... */ }
+    );
+
+To invoke the function ``func`` using a variable ``a`` containing an ``A``
+instance, we'd have to write ``func(B(a))`` in Python. On the other hand, C++
+will automatically apply an implicit type conversion, which makes it possible
+to directly write ``func(a)``.
+
+In this situation (i.e. where ``B`` has a constructor that converts from
+``A``), the following statement enables similar implicit conversions on the
+Python side:
+
+.. code-block:: cpp
+
+    py::implicitly_convertible<A, B>();
+
+.. note::
+
+    Implicit conversions from ``A`` to ``B`` only work when ``B`` is a custom
+    data type that is exposed to Python via pybind11.
+
+.. _static_properties:
+
+Static properties
+=================
+
+The section on :ref:`properties` discussed the creation of instance properties
+that are implemented in terms of C++ getters and setters.
+
+Static properties can also be created in a similar way to expose getters and
+setters of static class attributes. It is important to note that the implicit
+``self`` argument also exists in this case and is used to pass the Python
+``type`` subclass instance. This parameter will often not be needed by the C++
+side, and the following example illustrates how to instantiate a lambda getter
+function that ignores it:
+
+.. code-block:: cpp
+
+    py::class_<Foo>(m, "Foo")
+        .def_property_readonly_static("foo", [](py::object /* self */) { return Foo(); });
+
+Operator overloading
+====================
+
+Suppose that we're given the following ``Vector2`` class with a vector addition
+and scalar multiplication operation, all implemented using overloaded operators
+in C++.
+
+.. code-block:: cpp
+
+    class Vector2 {
+    public:
+        Vector2(float x, float y) : x(x), y(y) { }
+
+        Vector2 operator+(const Vector2 &v) const { return Vector2(x + v.x, y + v.y); }
+        Vector2 operator*(float value) const { return Vector2(x * value, y * value); }
+        Vector2& operator+=(const Vector2 &v) { x += v.x; y += v.y; return *this; }
+        Vector2& operator*=(float v) { x *= v; y *= v; return *this; }
+
+        friend Vector2 operator*(float f, const Vector2 &v) {
+            return Vector2(f * v.x, f * v.y);
+        }
+
+        std::string toString() const {
+            return "[" + std::to_string(x) + ", " + std::to_string(y) + "]";
+        }
+    private:
+        float x, y;
+    };
+
+The following snippet shows how the above operators can be conveniently exposed
+to Python.
+
+.. code-block:: cpp
+
+    #include <pybind11/operators.h>
+
+    PYBIND11_PLUGIN(example) {
+        py::module m("example", "pybind11 example plugin");
+
+        py::class_<Vector2>(m, "Vector2")
+            .def(py::init<float, float>())
+            .def(py::self + py::self)
+            .def(py::self += py::self)
+            .def(py::self *= float())
+            .def(float() * py::self)
+            .def("__repr__", &Vector2::toString);
+
+        return m.ptr();
+    }
+
+Note that a line like
+
+.. code-block:: cpp
+
+            .def(py::self * float())
+
+is really just short hand notation for
+
+.. code-block:: cpp
+
+    .def("__mul__", [](const Vector2 &a, float b) {
+        return a * b;
+    }, py::is_operator())
+
+This can be useful for exposing additional operators that don't exist on the
+C++ side, or to perform other types of customization. The ``py::is_operator``
+flag marker is needed to inform pybind11 that this is an operator, which
+returns ``NotImplemented`` when invoked with incompatible arguments rather than
+throwing a type error.
+
+.. note::
+
+    To use the more convenient ``py::self`` notation, the additional
+    header file :file:`pybind11/operators.h` must be included.
+
+.. seealso::
+
+    The file :file:`tests/test_operator_overloading.cpp` contains a
+    complete example that demonstrates how to work with overloaded operators in
+    more detail.
+
+Pickling support
+================
+
+Python's ``pickle`` module provides a powerful facility to serialize and
+de-serialize a Python object graph into a binary data stream. To pickle and
+unpickle C++ classes using pybind11, two additional functions must be provided.
+Suppose the class in question has the following signature:
+
+.. code-block:: cpp
+
+    class Pickleable {
+    public:
+        Pickleable(const std::string &value) : m_value(value) { }
+        const std::string &value() const { return m_value; }
+
+        void setExtra(int extra) { m_extra = extra; }
+        int extra() const { return m_extra; }
+    private:
+        std::string m_value;
+        int m_extra = 0;
+    };
+
+The binding code including the requisite ``__setstate__`` and ``__getstate__`` methods [#f3]_
+looks as follows:
+
+.. code-block:: cpp
+
+    py::class_<Pickleable>(m, "Pickleable")
+        .def(py::init<std::string>())
+        .def("value", &Pickleable::value)
+        .def("extra", &Pickleable::extra)
+        .def("setExtra", &Pickleable::setExtra)
+        .def("__getstate__", [](const Pickleable &p) {
+            /* Return a tuple that fully encodes the state of the object */
+            return py::make_tuple(p.value(), p.extra());
+        })
+        .def("__setstate__", [](Pickleable &p, py::tuple t) {
+            if (t.size() != 2)
+                throw std::runtime_error("Invalid state!");
+
+            /* Invoke the in-place constructor. Note that this is needed even
+               when the object just has a trivial default constructor */
+            new (&p) Pickleable(t[0].cast<std::string>());
+
+            /* Assign any additional state */
+            p.setExtra(t[1].cast<int>());
+        });
+
+An instance can now be pickled as follows:
+
+.. code-block:: python
+
+    try:
+        import cPickle as pickle  # Use cPickle on Python 2.7
+    except ImportError:
+        import pickle
+
+    p = Pickleable("test_value")
+    p.setExtra(15)
+    data = pickle.dumps(p, 2)
+
+Note that only the cPickle module is supported on Python 2.7. The second
+argument to ``dumps`` is also crucial: it selects the pickle protocol version
+2, since the older version 1 is not supported. Newer versions are also fine—for
+instance, specify ``-1`` to always use the latest available version. Beware:
+failure to follow these instructions will cause important pybind11 memory
+allocation routines to be skipped during unpickling, which will likely lead to
+memory corruption and/or segmentation faults.
+
+.. seealso::
+
+    The file :file:`tests/test_pickling.cpp` contains a complete example
+    that demonstrates how to pickle and unpickle types using pybind11 in more
+    detail.
+
+.. [#f3] http://docs.python.org/3/library/pickle.html#pickling-class-instances
+
+Multiple Inheritance
+====================
+
+pybind11 can create bindings for types that derive from multiple base types
+(aka. *multiple inheritance*). To do so, specify all bases in the template
+arguments of the ``class_`` declaration:
+
+.. code-block:: cpp
+
+    py::class_<MyType, BaseType1, BaseType2, BaseType3>(m, "MyType")
+       ...
+
+The base types can be specified in arbitrary order, and they can even be
+interspersed with alias types and holder types (discussed earlier in this
+document)---pybind11 will automatically find out which is which. The only
+requirement is that the first template argument is the type to be declared.
+
+There are two caveats regarding the implementation of this feature:
+
+1. When only one base type is specified for a C++ type that actually has
+   multiple bases, pybind11 will assume that it does not participate in
+   multiple inheritance, which can lead to undefined behavior. In such cases,
+   add the tag ``multiple_inheritance``:
+
+    .. code-block:: cpp
+
+        py::class_<MyType, BaseType2>(m, "MyType", py::multiple_inheritance());
+
+   The tag is redundant and does not need to be specified when multiple base
+   types are listed.
+
+2. As was previously discussed in the section on :ref:`overriding_virtuals`, it
+   is easy to create Python types that derive from C++ classes. It is even
+   possible to make use of multiple inheritance to declare a Python class which
+   has e.g. a C++ and a Python class as bases. However, any attempt to create a
+   type that has *two or more* C++ classes in its hierarchy of base types will
+   fail with a fatal error message: ``TypeError: multiple bases have instance
+   lay-out conflict``. Core Python types that are implemented in C (e.g.
+   ``dict``, ``list``, ``Exception``, etc.) also fall under this combination
+   and cannot be combined with C++ types bound using pybind11 via multiple
+   inheritance.

docs/advanced/exceptions.rst

@@ -0,0 +1,142 @@
+Exceptions
+##########
+
+Built-in exception translation
+==============================
+
+When C++ code invoked from Python throws an ``std::exception``, it is
+automatically converted into a Python ``Exception``. pybind11 defines multiple
+special exception classes that will map to different types of Python
+exceptions:
+
+.. tabularcolumns:: |p{0.5\textwidth}|p{0.45\textwidth}|
+
++--------------------------------------+------------------------------+
+|  C++ exception type                  |  Python exception type       |
++======================================+==============================+
+| :class:`std::exception`              | ``RuntimeError``             |
++--------------------------------------+------------------------------+
+| :class:`std::bad_alloc`              | ``MemoryError``              |
++--------------------------------------+------------------------------+
+| :class:`std::domain_error`           | ``ValueError``               |
++--------------------------------------+------------------------------+
+| :class:`std::invalid_argument`       | ``ValueError``               |
++--------------------------------------+------------------------------+
+| :class:`std::length_error`           | ``ValueError``               |
++--------------------------------------+------------------------------+
+| :class:`std::out_of_range`           | ``ValueError``               |
++--------------------------------------+------------------------------+
+| :class:`std::range_error`            | ``ValueError``               |
++--------------------------------------+------------------------------+
+| :class:`pybind11::stop_iteration`    | ``StopIteration`` (used to   |
+|                                      | implement custom iterators)  |
++--------------------------------------+------------------------------+
+| :class:`pybind11::index_error`       | ``IndexError`` (used to      |
+|                                      | indicate out of bounds       |
+|                                      | accesses in ``__getitem__``, |
+|                                      | ``__setitem__``, etc.)       |
++--------------------------------------+------------------------------+
+| :class:`pybind11::value_error`       | ``ValueError`` (used to      |
+|                                      | indicate wrong value passed  |
+|                                      | in ``container.remove(...)`` |
++--------------------------------------+------------------------------+
+| :class:`pybind11::key_error`         | ``KeyError`` (used to        |
+|                                      | indicate out of bounds       |
+|                                      | accesses in ``__getitem__``, |
+|                                      | ``__setitem__`` in dict-like |
+|                                      | objects, etc.)               |
++--------------------------------------+------------------------------+
+| :class:`pybind11::error_already_set` | Indicates that the Python    |
+|                                      | exception flag has already   |
+|                                      | been initialized             |
++--------------------------------------+------------------------------+
+
+When a Python function invoked from C++ throws an exception, it is converted
+into a C++ exception of type :class:`error_already_set` whose string payload
+contains a textual summary.
+
+There is also a special exception :class:`cast_error` that is thrown by
+:func:`handle::call` when the input arguments cannot be converted to Python
+objects.
+
+Registering custom translators
+==============================
+
+If the default exception conversion policy described above is insufficient,
+pybind11 also provides support for registering custom exception translators.
+To register a simple exception conversion that translates a C++ exception into
+a new Python exception using the C++ exception's ``what()`` method, a helper
+function is available:
+
+.. code-block:: cpp
+
+    py::register_exception<CppExp>(module, "PyExp");
+
+This call creates a Python exception class with the name ``PyExp`` in the given
+module and automatically converts any encountered exceptions of type ``CppExp``
+into Python exceptions of type ``PyExp``.
+
+When more advanced exception translation is needed, the function
+``py::register_exception_translator(translator)`` can be used to register
+functions that can translate arbitrary exception types (and which may include
+additional logic to do so).  The function takes a stateless callable (e.g.  a
+function pointer or a lambda function without captured variables) with the call
+signature ``void(std::exception_ptr)``.
+
+When a C++ exception is thrown, the registered exception translators are tried
+in reverse order of registration (i.e. the last registered translator gets the
+first shot at handling the exception).
+
+Inside the translator, ``std::rethrow_exception`` should be used within
+a try block to re-throw the exception.  One or more catch clauses to catch
+the appropriate exceptions should then be used with each clause using
+``PyErr_SetString`` to set a Python exception or ``ex(string)`` to set
+the python exception to a custom exception type (see below).
+
+To declare a custom Python exception type, declare a ``py::exception`` variable
+and use this in the associated exception translator (note: it is often useful
+to make this a static declaration when using it inside a lambda expression
+without requiring capturing).
+
+
+The following example demonstrates this for a hypothetical exception classes
+``MyCustomException`` and ``OtherException``: the first is translated to a
+custom python exception ``MyCustomError``, while the second is translated to a
+standard python RuntimeError:
+
+.. code-block:: cpp
+
+    static py::exception<MyCustomException> exc(m, "MyCustomError");
+    py::register_exception_translator([](std::exception_ptr p) {
+        try {
+            if (p) std::rethrow_exception(p);
+        } catch (const MyCustomException &e) {
+            exc(e.what());
+        } catch (const OtherException &e) {
+            PyErr_SetString(PyExc_RuntimeError, e.what());
+        }
+    });
+
+Multiple exceptions can be handled by a single translator, as shown in the
+example above. If the exception is not caught by the current translator, the
+previously registered one gets a chance.
+
+If none of the registered exception translators is able to handle the
+exception, it is handled by the default converter as described in the previous
+section.
+
+.. seealso::
+
+    The file :file:`tests/test_exceptions.cpp` contains examples
+    of various custom exception translators and custom exception types.
+
+.. note::
+
+    You must call either ``PyErr_SetString`` or a custom exception's call
+    operator (``exc(string)``) for every exception caught in a custom exception
+    translator.  Failure to do so will cause Python to crash with ``SystemError:
+    error return without exception set``.
+
+    Exceptions that you do not plan to handle should simply not be caught, or
+    may be explicity (re-)thrown to delegate it to the other,
+    previously-declared existing exception translators.

docs/advanced/functions.rst

@@ -0,0 +1,263 @@
+Functions
+#########
+
+Before proceeding with this section, make sure that you are already familiar
+with the basics of binding functions and classes, as explained in :doc:`/basics`
+and :doc:`/classes`. The following guide is applicable to both free and member
+functions, i.e. *methods* in Python.
+
+Return value policies
+=====================
+
+Python and C++ use wildly different ways of managing the memory and lifetime of
+objects managed by them. This can lead to issues when creating bindings for
+functions that return a non-trivial type. Just by looking at the type
+information, it is not clear whether Python should take charge of the returned
+value and eventually free its resources, or if this is handled on the C++ side.
+For this reason, pybind11 provides a several `return value policy` annotations
+that can be passed to the :func:`module::def` and :func:`class_::def`
+functions. The default policy is :enum:`return_value_policy::automatic`.
+
+Return value policies can also be applied to properties, in which case the
+arguments must be passed through the :class:`cpp_function` constructor:
+
+.. code-block:: cpp
+
+    class_<MyClass>(m, "MyClass")
+        def_property("data"
+            py::cpp_function(&MyClass::getData, py::return_value_policy::copy),
+            py::cpp_function(&MyClass::setData)
+        );
+
+The following table provides an overview of the available return value policies:
+
+.. tabularcolumns:: |p{0.5\textwidth}|p{0.45\textwidth}|
+
++--------------------------------------------------+----------------------------------------------------------------------------+
+| Return value policy                              | Description                                                                |
++==================================================+============================================================================+
+| :enum:`return_value_policy::automatic`           | This is the default return value policy, which falls back to the policy    |
+|                                                  | :enum:`return_value_policy::take_ownership` when the return value is a     |
+|                                                  | pointer. Otherwise, it uses :enum:`return_value::move` or                  |
+|                                                  | :enum:`return_value::copy` for rvalue and lvalue references, respectively. |
+|                                                  | See below for a description of what all of these different policies do.    |
++--------------------------------------------------+----------------------------------------------------------------------------+
+| :enum:`return_value_policy::automatic_reference` | As above, but use policy :enum:`return_value_policy::reference` when the   |
+|                                                  | return value is a pointer. This is the default conversion policy for       |
+|                                                  | function arguments when calling Python functions manually from C++ code    |
+|                                                  | (i.e. via handle::operator()). You probably won't need to use this.        |
++--------------------------------------------------+----------------------------------------------------------------------------+
+| :enum:`return_value_policy::take_ownership`      | Reference an existing object (i.e. do not create a new copy) and take      |
+|                                                  | ownership. Python will call the destructor and delete operator when the    |
+|                                                  | object's reference count reaches zero. Undefined behavior ensues when the  |
+|                                                  | C++ side does the same.                                                    |
++--------------------------------------------------+----------------------------------------------------------------------------+
+| :enum:`return_value_policy::copy`                | Create a new copy of the returned object, which will be owned by Python.   |
+|                                                  | This policy is comparably safe because the lifetimes of the two instances  |
+|                                                  | are decoupled.                                                             |
++--------------------------------------------------+----------------------------------------------------------------------------+
+| :enum:`return_value_policy::move`                | Use ``std::move`` to move the return value contents into a new instance    |
+|                                                  | that will be owned by Python. This policy is comparably safe because the   |
+|                                                  | lifetimes of the two instances (move source and destination) are decoupled.|
++--------------------------------------------------+----------------------------------------------------------------------------+
+| :enum:`return_value_policy::reference`           | Reference an existing object, but do not take ownership. The C++ side is   |
+|                                                  | responsible for managing the object's lifetime and deallocating it when    |
+|                                                  | it is no longer used. Warning: undefined behavior will ensue when the C++  |
+|                                                  | side deletes an object that is still referenced and used by Python.        |
++--------------------------------------------------+----------------------------------------------------------------------------+
+| :enum:`return_value_policy::reference_internal`  | Indicates that the lifetime of the return value is tied to the lifetime    |
+|                                                  | of a parent object, namely the implicit ``this``, or ``self`` argument of  |
+|                                                  | the called method or property. Internally, this policy works just like     |
+|                                                  | :enum:`return_value_policy::reference` but additionally applies a          |
+|                                                  | ``keep_alive<0, 1>`` *call policy* (described in the next section) that    |
+|                                                  | prevents the parent object from being garbage collected as long as the     |
+|                                                  | return value is referenced by Python. This is the default policy for       |
+|                                                  | property getters created via ``def_property``, ``def_readwrite``, etc.     |
++--------------------------------------------------+----------------------------------------------------------------------------+
+
+.. warning::
+
+    Code with invalid return value policies might access unitialized memory or
+    free data structures multiple times, which can lead to hard-to-debug
+    non-determinism and segmentation faults, hence it is worth spending the
+    time to understand all the different options in the table above.
+
+One important aspect of the above policies is that they only apply to instances
+which pybind11 has *not* seen before, in which case the policy clarifies
+essential questions about the return value's lifetime and ownership.  When
+pybind11 knows the instance already (as identified by its type and address in
+memory), it will return the existing Python object wrapper rather than creating
+a new copy.
+
+.. note::
+
+    The next section on :ref:`call_policies` discusses *call policies* that can be
+    specified *in addition* to a return value policy from the list above. Call
+    policies indicate reference relationships that can involve both return values
+    and parameters of functions.
+
+.. note::
+
+   As an alternative to elaborate call policies and lifetime management logic,
+   consider using smart pointers (see the section on :ref:`smart_pointers` for
+   details). Smart pointers can tell whether an object is still referenced from
+   C++ or Python, which generally eliminates the kinds of inconsistencies that
+   can lead to crashes or undefined behavior. For functions returning smart
+   pointers, it is not necessary to specify a return value policy.
+
+.. _call_policies:
+
+Additional call policies
+========================
+
+In addition to the above return value policies, further `call policies` can be
+specified to indicate dependencies between parameters. There is currently just
+one policy named ``keep_alive<Nurse, Patient>``, which indicates that the
+argument with index ``Patient`` should be kept alive at least until the
+argument with index ``Nurse`` is freed by the garbage collector. Argument
+indices start at one, while zero refers to the return value. For methods, index
+``1`` refers to the implicit ``this`` pointer, while regular arguments begin at
+index ``2``. Arbitrarily many call policies can be specified. When a ``Nurse``
+with value ``None`` is detected at runtime, the call policy does nothing.
+
+This feature internally relies on the ability to create a *weak reference* to
+the nurse object, which is permitted by all classes exposed via pybind11. When
+the nurse object does not support weak references, an exception will be thrown.
+
+Consider the following example: here, the binding code for a list append
+operation ties the lifetime of the newly added element to the underlying
+container:
+
+.. code-block:: cpp
+
+    py::class_<List>(m, "List")
+        .def("append", &List::append, py::keep_alive<1, 2>());
+
+.. note::
+
+    ``keep_alive`` is analogous to the ``with_custodian_and_ward`` (if Nurse,
+    Patient != 0) and ``with_custodian_and_ward_postcall`` (if Nurse/Patient ==
+    0) policies from Boost.Python.
+
+.. seealso::
+
+    The file :file:`tests/test_keep_alive.cpp` contains a complete example
+    that demonstrates using :class:`keep_alive` in more detail.
+
+.. _python_objects_as_args:
+
+Python objects as arguments
+===========================
+
+pybind11 exposes all major Python types using thin C++ wrapper classes. These
+wrapper classes can also be used as parameters of functions in bindings, which
+makes it possible to directly work with native Python types on the C++ side.
+For instance, the following statement iterates over a Python ``dict``:
+
+.. code-block:: cpp
+
+    void print_dict(py::dict dict) {
+        /* Easily interact with Python types */
+        for (auto item : dict)
+            std::cout << "key=" << item.first << ", "
+                      << "value=" << item.second << std::endl;
+    }
+
+It can be exported:
+
+.. code-block:: cpp
+
+    m.def("print_dict", &print_dict);
+
+And used in Python as usual:
+
+.. code-block:: pycon
+
+    >>> print_dict({'foo': 123, 'bar': 'hello'})
+    key=foo, value=123
+    key=bar, value=hello
+
+For more information on using Python objects in C++, see :doc:`/advanced/pycpp/index`.
+
+Accepting \*args and \*\*kwargs
+===============================
+
+Python provides a useful mechanism to define functions that accept arbitrary
+numbers of arguments and keyword arguments:
+
+.. code-block:: python
+
+   def generic(*args, **kwargs):
+       ...  # do something with args and kwargs
+
+Such functions can also be created using pybind11:
+
+.. code-block:: cpp
+
+   void generic(py::args args, py::kwargs kwargs) {
+       /// .. do something with args
+       if (kwargs)
+           /// .. do something with kwargs
+   }
+
+   /// Binding code
+   m.def("generic", &generic);
+
+The class ``py::args`` derives from ``py::tuple`` and ``py::kwargs`` derives
+from ``py::dict``. Note that the ``kwargs`` argument is invalid if no keyword
+arguments were actually provided. Please refer to the other examples for
+details on how to iterate over these, and on how to cast their entries into
+C++ objects. A demonstration is also available in
+``tests/test_kwargs_and_defaults.cpp``.
+
+.. warning::
+
+   Unlike Python, pybind11 does not allow combining normal parameters with the
+   ``args`` / ``kwargs`` special parameters.
+
+Default arguments revisited
+===========================
+
+The section on :ref:`default_args` previously discussed basic usage of default
+arguments using pybind11. One noteworthy aspect of their implementation is that
+default arguments are converted to Python objects right at declaration time.
+Consider the following example:
+
+.. code-block:: cpp
+
+    py::class_<MyClass>("MyClass")
+        .def("myFunction", py::arg("arg") = SomeType(123));
+
+In this case, pybind11 must already be set up to deal with values of the type
+``SomeType`` (via a prior instantiation of ``py::class_<SomeType>``), or an
+exception will be thrown.
+
+Another aspect worth highlighting is that the "preview" of the default argument
+in the function signature is generated using the object's ``__repr__`` method.
+If not available, the signature may not be very helpful, e.g.:
+
+.. code-block:: pycon
+
+    FUNCTIONS
+    ...
+    |  myFunction(...)
+    |      Signature : (MyClass, arg : SomeType = <SomeType object at 0x101b7b080>) -> NoneType
+    ...
+
+The first way of addressing this is by defining ``SomeType.__repr__``.
+Alternatively, it is possible to specify the human-readable preview of the
+default argument manually using the ``arg_v`` notation:
+
+.. code-block:: cpp
+
+    py::class_<MyClass>("MyClass")
+        .def("myFunction", py::arg_v("arg", SomeType(123), "SomeType(123)"));
+
+Sometimes it may be necessary to pass a null pointer value as a default
+argument. In this case, remember to cast it to the underlying type in question,
+like so:
+
+.. code-block:: cpp
+
+    py::class_<MyClass>("MyClass")
+        .def("myFunction", py::arg("arg") = (SomeType *) nullptr);

docs/advanced/misc.rst

@@ -0,0 +1,187 @@
+Miscellaneous
+#############
+
+.. _macro_notes:
+
+General notes regarding convenience macros
+==========================================
+
+pybind11 provides a few convenience macros such as
+:func:`PYBIND11_MAKE_OPAQUE` and :func:`PYBIND11_DECLARE_HOLDER_TYPE`, and
+``PYBIND11_OVERLOAD_*``. Since these are "just" macros that are evaluated
+in the preprocessor (which has no concept of types), they *will* get confused
+by commas in a template argument such as ``PYBIND11_OVERLOAD(MyReturnValue<T1,
+T2>, myFunc)``. In this case, the preprocessor assumes that the comma indicates
+the beginning of the next parameter. Use a ``typedef`` to bind the template to
+another name and use it in the macro to avoid this problem.
+
+
+Global Interpreter Lock (GIL)
+=============================
+
+The classes :class:`gil_scoped_release` and :class:`gil_scoped_acquire` can be
+used to acquire and release the global interpreter lock in the body of a C++
+function call. In this way, long-running C++ code can be parallelized using
+multiple Python threads. Taking :ref:`overriding_virtuals` as an example, this
+could be realized as follows (important changes highlighted):
+
+.. code-block:: cpp
+    :emphasize-lines: 8,9,33,34
+
+    class PyAnimal : public Animal {
+    public:
+        /* Inherit the constructors */
+        using Animal::Animal;
+
+        /* Trampoline (need one for each virtual function) */
+        std::string go(int n_times) {
+            /* Acquire GIL before calling Python code */
+            py::gil_scoped_acquire acquire;
+
+            PYBIND11_OVERLOAD_PURE(
+                std::string, /* Return type */
+                Animal,      /* Parent class */
+                go,          /* Name of function */
+                n_times      /* Argument(s) */
+            );
+        }
+    };
+
+    PYBIND11_PLUGIN(example) {
+        py::module m("example", "pybind11 example plugin");
+
+        py::class_<Animal, PyAnimal> animal(m, "Animal");
+        animal
+            .def(py::init<>())
+            .def("go", &Animal::go);
+
+        py::class_<Dog>(m, "Dog", animal)
+            .def(py::init<>());
+
+        m.def("call_go", [](Animal *animal) -> std::string {
+            /* Release GIL before calling into (potentially long-running) C++ code */
+            py::gil_scoped_release release;
+            return call_go(animal);
+        });
+
+        return m.ptr();
+    }
+
+
+Binding sequence data types, iterators, the slicing protocol, etc.
+==================================================================
+
+Please refer to the supplemental example for details.
+
+.. seealso::
+
+    The file :file:`tests/test_sequences_and_iterators.cpp` contains a
+    complete example that shows how to bind a sequence data type, including
+    length queries (``__len__``), iterators (``__iter__``), the slicing
+    protocol and other kinds of useful operations.
+
+
+Partitioning code over multiple extension modules
+=================================================
+
+It's straightforward to split binding code over multiple extension modules,
+while referencing types that are declared elsewhere. Everything "just" works
+without any special precautions. One exception to this rule occurs when
+extending a type declared in another extension module. Recall the basic example
+from Section :ref:`inheritance`.
+
+.. code-block:: cpp
+
+    py::class_<Pet> pet(m, "Pet");
+    pet.def(py::init<const std::string &>())
+       .def_readwrite("name", &Pet::name);
+
+    py::class_<Dog>(m, "Dog", pet /* <- specify parent */)
+        .def(py::init<const std::string &>())
+        .def("bark", &Dog::bark);
+
+Suppose now that ``Pet`` bindings are defined in a module named ``basic``,
+whereas the ``Dog`` bindings are defined somewhere else. The challenge is of
+course that the variable ``pet`` is not available anymore though it is needed
+to indicate the inheritance relationship to the constructor of ``class_<Dog>``.
+However, it can be acquired as follows:
+
+.. code-block:: cpp
+
+    py::object pet = (py::object) py::module::import("basic").attr("Pet");
+
+    py::class_<Dog>(m, "Dog", pet)
+        .def(py::init<const std::string &>())
+        .def("bark", &Dog::bark);
+
+Alternatively, you can specify the base class as a template parameter option to
+``class_``, which performs an automated lookup of the corresponding Python
+type. Like the above code, however, this also requires invoking the ``import``
+function once to ensure that the pybind11 binding code of the module ``basic``
+has been executed:
+
+.. code-block:: cpp
+
+    py::module::import("basic");
+
+    py::class_<Dog, Pet>(m, "Dog")
+        .def(py::init<const std::string &>())
+        .def("bark", &Dog::bark);
+
+Naturally, both methods will fail when there are cyclic dependencies.
+
+Note that compiling code which has its default symbol visibility set to
+*hidden* (e.g. via the command line flag ``-fvisibility=hidden`` on GCC/Clang) can interfere with the
+ability to access types defined in another extension module. Workarounds
+include changing the global symbol visibility (not recommended, because it will
+lead unnecessarily large binaries) or manually exporting types that are
+accessed by multiple extension modules:
+
+.. code-block:: cpp
+
+    #ifdef _WIN32
+    #  define EXPORT_TYPE __declspec(dllexport)
+    #else
+    #  define EXPORT_TYPE __attribute__ ((visibility("default")))
+    #endif
+
+    class EXPORT_TYPE Dog : public Animal {
+        ...
+    };
+
+
+
+Generating documentation using Sphinx
+=====================================
+
+Sphinx [#f4]_ has the ability to inspect the signatures and documentation
+strings in pybind11-based extension modules to automatically generate beautiful
+documentation in a variety formats. The python_example repository [#f5]_ contains a
+simple example repository which uses this approach.
+
+There are two potential gotchas when using this approach: first, make sure that
+the resulting strings do not contain any :kbd:`TAB` characters, which break the
+docstring parsing routines. You may want to use C++11 raw string literals,
+which are convenient for multi-line comments. Conveniently, any excess
+indentation will be automatically be removed by Sphinx. However, for this to
+work, it is important that all lines are indented consistently, i.e.:
+
+.. code-block:: cpp
+
+    // ok
+    m.def("foo", &foo, R"mydelimiter(
+        The foo function
+
+        Parameters
+        ----------
+    )mydelimiter");
+
+    // *not ok*
+    m.def("foo", &foo, R"mydelimiter(The foo function
+
+        Parameters
+        ----------
+    )mydelimiter");
+
+.. [#f4] http://www.sphinx-doc.org
+.. [#f5] http://github.com/pybind/python_example

docs/advanced/pycpp/index.rst

@@ -0,0 +1,13 @@
+Python C++ interface
+####################
+
+pybind11 exposes Python types and functions using thin C++ wrappers, which
+makes it possible to conveniently call Python code from C++ without resorting
+to Python's C API.
+
+.. toctree::
+   :maxdepth: 2
+
+   object
+   numpy
+   utilities

docs/advanced/pycpp/numpy.rst

@@ -0,0 +1,299 @@
+.. _numpy:
+
+NumPy
+#####
+
+Buffer protocol
+===============
+
+Python supports an extremely general and convenient approach for exchanging
+data between plugin libraries. Types can expose a buffer view [#f2]_, which
+provides fast direct access to the raw internal data representation. Suppose we
+want to bind the following simplistic Matrix class:
+
+.. code-block:: cpp
+
+    class Matrix {
+    public:
+        Matrix(size_t rows, size_t cols) : m_rows(rows), m_cols(cols) {
+            m_data = new float[rows*cols];
+        }
+        float *data() { return m_data; }
+        size_t rows() const { return m_rows; }
+        size_t cols() const { return m_cols; }
+    private:
+        size_t m_rows, m_cols;
+        float *m_data;
+    };
+
+The following binding code exposes the ``Matrix`` contents as a buffer object,
+making it possible to cast Matrices into NumPy arrays. It is even possible to
+completely avoid copy operations with Python expressions like
+``np.array(matrix_instance, copy = False)``.
+
+.. code-block:: cpp
+
+    py::class_<Matrix>(m, "Matrix")
+       .def_buffer([](Matrix &m) -> py::buffer_info {
+            return py::buffer_info(
+                m.data(),                               /* Pointer to buffer */
+                sizeof(float),                          /* Size of one scalar */
+                py::format_descriptor<float>::format(), /* Python struct-style format descriptor */
+                2,                                      /* Number of dimensions */
+                { m.rows(), m.cols() },                 /* Buffer dimensions */
+                { sizeof(float) * m.rows(),             /* Strides (in bytes) for each index */
+                  sizeof(float) }
+            );
+        });
+
+The snippet above binds a lambda function, which can create ``py::buffer_info``
+description records on demand describing a given matrix. The contents of
+``py::buffer_info`` mirror the Python buffer protocol specification.
+
+.. code-block:: cpp
+
+    struct buffer_info {
+        void *ptr;
+        size_t itemsize;
+        std::string format;
+        int ndim;
+        std::vector<size_t> shape;
+        std::vector<size_t> strides;
+    };
+
+To create a C++ function that can take a Python buffer object as an argument,
+simply use the type ``py::buffer`` as one of its arguments. Buffers can exist
+in a great variety of configurations, hence some safety checks are usually
+necessary in the function body. Below, you can see an basic example on how to
+define a custom constructor for the Eigen double precision matrix
+(``Eigen::MatrixXd``) type, which supports initialization from compatible
+buffer objects (e.g. a NumPy matrix).
+
+.. code-block:: cpp
+
+    /* Bind MatrixXd (or some other Eigen type) to Python */
+    typedef Eigen::MatrixXd Matrix;
+
+    typedef Matrix::Scalar Scalar;
+    constexpr bool rowMajor = Matrix::Flags & Eigen::RowMajorBit;
+
+    py::class_<Matrix>(m, "Matrix")
+        .def("__init__", [](Matrix &m, py::buffer b) {
+            typedef Eigen::Stride<Eigen::Dynamic, Eigen::Dynamic> Strides;
+
+            /* Request a buffer descriptor from Python */
+            py::buffer_info info = b.request();
+
+            /* Some sanity checks ... */
+            if (info.format != py::format_descriptor<Scalar>::format())
+                throw std::runtime_error("Incompatible format: expected a double array!");
+
+            if (info.ndim != 2)
+                throw std::runtime_error("Incompatible buffer dimension!");
+
+            auto strides = Strides(
+                info.strides[rowMajor ? 0 : 1] / sizeof(Scalar),
+                info.strides[rowMajor ? 1 : 0] / sizeof(Scalar));
+
+            auto map = Eigen::Map<Matrix, 0, Strides>(
+                static_cat<Scalar *>(info.ptr), info.shape[0], info.shape[1], strides);
+
+            new (&m) Matrix(map);
+        });
+
+For reference, the ``def_buffer()`` call for this Eigen data type should look
+as follows:
+
+.. code-block:: cpp
+
+    .def_buffer([](Matrix &m) -> py::buffer_info {
+        return py::buffer_info(
+            m.data(),                /* Pointer to buffer */
+            sizeof(Scalar),          /* Size of one scalar */
+            /* Python struct-style format descriptor */
+            py::format_descriptor<Scalar>::format(),
+            /* Number of dimensions */
+            2,
+            /* Buffer dimensions */
+            { (size_t) m.rows(),
+              (size_t) m.cols() },
+            /* Strides (in bytes) for each index */
+            { sizeof(Scalar) * (rowMajor ? m.cols() : 1),
+              sizeof(Scalar) * (rowMajor ? 1 : m.rows()) }
+        );
+     })
+
+For a much easier approach of binding Eigen types (although with some
+limitations), refer to the section on :doc:`/advanced/cast/eigen`.
+
+.. seealso::
+
+    The file :file:`tests/test_buffers.cpp` contains a complete example
+    that demonstrates using the buffer protocol with pybind11 in more detail.
+
+.. [#f2] http://docs.python.org/3/c-api/buffer.html
+
+Arrays
+======
+
+By exchanging ``py::buffer`` with ``py::array`` in the above snippet, we can
+restrict the function so that it only accepts NumPy arrays (rather than any
+type of Python object satisfying the buffer protocol).
+
+In many situations, we want to define a function which only accepts a NumPy
+array of a certain data type. This is possible via the ``py::array_t<T>``
+template. For instance, the following function requires the argument to be a
+NumPy array containing double precision values.
+
+.. code-block:: cpp
+
+    void f(py::array_t<double> array);
+
+When it is invoked with a different type (e.g. an integer or a list of
+integers), the binding code will attempt to cast the input into a NumPy array
+of the requested type. Note that this feature requires the
+:file:``pybind11/numpy.h`` header to be included.
+
+Data in NumPy arrays is not guaranteed to packed in a dense manner;
+furthermore, entries can be separated by arbitrary column and row strides.
+Sometimes, it can be useful to require a function to only accept dense arrays
+using either the C (row-major) or Fortran (column-major) ordering. This can be
+accomplished via a second template argument with values ``py::array::c_style``
+or ``py::array::f_style``.
+
+.. code-block:: cpp
+
+    void f(py::array_t<double, py::array::c_style | py::array::forcecast> array);
+
+The ``py::array::forcecast`` argument is the default value of the second
+template parameter, and it ensures that non-conforming arguments are converted
+into an array satisfying the specified requirements instead of trying the next
+function overload.
+
+Structured types
+================
+
+In order for ``py::array_t`` to work with structured (record) types, we first need
+to register the memory layout of the type. This can be done via ``PYBIND11_NUMPY_DTYPE``
+macro which expects the type followed by field names:
+
+.. code-block:: cpp
+
+    struct A {
+        int x;
+        double y;
+    };
+
+    struct B {
+        int z;
+        A a;
+    };
+
+    PYBIND11_NUMPY_DTYPE(A, x, y);
+    PYBIND11_NUMPY_DTYPE(B, z, a);
+
+    /* now both A and B can be used as template arguments to py::array_t */
+
+Vectorizing functions
+=====================
+
+Suppose we want to bind a function with the following signature to Python so
+that it can process arbitrary NumPy array arguments (vectors, matrices, general
+N-D arrays) in addition to its normal arguments:
+
+.. code-block:: cpp
+
+    double my_func(int x, float y, double z);
+
+After including the ``pybind11/numpy.h`` header, this is extremely simple:
+
+.. code-block:: cpp
+
+    m.def("vectorized_func", py::vectorize(my_func));
+
+Invoking the function like below causes 4 calls to be made to ``my_func`` with
+each of the array elements. The significant advantage of this compared to
+solutions like ``numpy.vectorize()`` is that the loop over the elements runs
+entirely on the C++ side and can be crunched down into a tight, optimized loop
+by the compiler. The result is returned as a NumPy array of type
+``numpy.dtype.float64``.
+
+.. code-block:: pycon
+
+    >>> x = np.array([[1, 3],[5, 7]])
+    >>> y = np.array([[2, 4],[6, 8]])
+    >>> z = 3
+    >>> result = vectorized_func(x, y, z)
+
+The scalar argument ``z`` is transparently replicated 4 times.  The input
+arrays ``x`` and ``y`` are automatically converted into the right types (they
+are of type  ``numpy.dtype.int64`` but need to be ``numpy.dtype.int32`` and
+``numpy.dtype.float32``, respectively)
+
+Sometimes we might want to explicitly exclude an argument from the vectorization
+because it makes little sense to wrap it in a NumPy array. For instance,
+suppose the function signature was
+
+.. code-block:: cpp
+
+    double my_func(int x, float y, my_custom_type *z);
+
+This can be done with a stateful Lambda closure:
+
+.. code-block:: cpp
+
+    // Vectorize a lambda function with a capture object (e.g. to exclude some arguments from the vectorization)
+    m.def("vectorized_func",
+        [](py::array_t<int> x, py::array_t<float> y, my_custom_type *z) {
+            auto stateful_closure = [z](int x, float y) { return my_func(x, y, z); };
+            return py::vectorize(stateful_closure)(x, y);
+        }
+    );
+
+In cases where the computation is too complicated to be reduced to
+``vectorize``, it will be necessary to create and access the buffer contents
+manually. The following snippet contains a complete example that shows how this
+works (the code is somewhat contrived, since it could have been done more
+simply using ``vectorize``).
+
+.. code-block:: cpp
+
+    #include <pybind11/pybind11.h>
+    #include <pybind11/numpy.h>
+
+    namespace py = pybind11;
+
+    py::array_t<double> add_arrays(py::array_t<double> input1, py::array_t<double> input2) {
+        auto buf1 = input1.request(), buf2 = input2.request();
+
+        if (buf1.ndim != 1 || buf2.ndim != 1)
+            throw std::runtime_error("Number of dimensions must be one");
+
+        if (buf1.size != buf2.size)
+            throw std::runtime_error("Input shapes must match");
+
+        /* No pointer is passed, so NumPy will allocate the buffer */
+        auto result = py::array_t<double>(buf1.size);
+
+        auto buf3 = result.request();
+
+        double *ptr1 = (double *) buf1.ptr,
+               *ptr2 = (double *) buf2.ptr,
+               *ptr3 = (double *) buf3.ptr;
+
+        for (size_t idx = 0; idx < buf1.shape[0]; idx++)
+            ptr3[idx] = ptr1[idx] + ptr2[idx];
+
+        return result;
+    }
+
+    PYBIND11_PLUGIN(test) {
+        py::module m("test");
+        m.def("add_arrays", &add_arrays, "Add two NumPy arrays");
+        return m.ptr();
+    }
+
+.. seealso::
+
+    The file :file:`tests/test_numpy_vectorize.cpp` contains a complete
+    example that demonstrates using :func:`vectorize` in more detail.

docs/advanced/pycpp/object.rst

@@ -0,0 +1,96 @@
+Python types
+############
+
+Available wrappers
+==================
+
+All major Python types are available as thin C++ wrapper classes. These
+can also be used as function parameters -- see :ref:`python_objects_as_args`.
+
+Available types include :class:`handle`, :class:`object`, :class:`bool_`,
+:class:`int_`, :class:`float_`, :class:`str`, :class:`bytes`, :class:`tuple`,
+:class:`list`, :class:`dict`, :class:`slice`, :class:`none`, :class:`capsule`,
+:class:`iterable`, :class:`iterator`, :class:`function`, :class:`buffer`,
+:class:`array`, and :class:`array_t`.
+
+Casting back and forth
+======================
+
+In this kind of mixed code, it is often necessary to convert arbitrary C++
+types to Python, which can be done using :func:`py::cast`:
+
+.. code-block:: cpp
+
+    MyClass *cls = ..;
+    py::object obj = py::cast(cls);
+
+The reverse direction uses the following syntax:
+
+.. code-block:: cpp
+
+    py::object obj = ...;
+    MyClass *cls = obj.cast<MyClass *>();
+
+When conversion fails, both directions throw the exception :class:`cast_error`.
+
+Calling Python functions
+========================
+
+It is also possible to call python functions via ``operator()``.
+
+.. code-block:: cpp
+
+    py::function f = <...>;
+    py::object result_py = f(1234, "hello", some_instance);
+    MyClass &result = result_py.cast<MyClass>();
+
+Keyword arguments are also supported. In Python, there is the usual call syntax:
+
+.. code-block:: python
+
+    def f(number, say, to):
+        ...  # function code
+
+    f(1234, say="hello", to=some_instance)  # keyword call in Python
+
+In C++, the same call can be made using:
+
+.. code-block:: cpp
+
+    using pybind11::literals; // to bring in the `_a` literal
+    f(1234, "say"_a="hello", "to"_a=some_instance); // keyword call in C++
+
+Unpacking of ``*args`` and ``**kwargs`` is also possible and can be mixed with
+other arguments:
+
+.. code-block:: cpp
+
+    // * unpacking
+    py::tuple args = py::make_tuple(1234, "hello", some_instance);
+    f(*args);
+
+    // ** unpacking
+    py::dict kwargs = py::dict("number"_a=1234, "say"_a="hello", "to"_a=some_instance);
+    f(**kwargs);
+
+    // mixed keywords, * and ** unpacking
+    py::tuple args = py::make_tuple(1234);
+    py::dict kwargs = py::dict("to"_a=some_instance);
+    f(*args, "say"_a="hello", **kwargs);
+
+Generalized unpacking according to PEP448_ is also supported:
+
+.. code-block:: cpp
+
+    py::dict kwargs1 = py::dict("number"_a=1234);
+    py::dict kwargs2 = py::dict("to"_a=some_instance);
+    f(**kwargs1, "say"_a="hello", **kwargs2);
+
+.. seealso::
+
+    The file :file:`tests/test_python_types.cpp` contains a complete
+    example that demonstrates passing native Python types in more detail. The
+    file :file:`tests/test_callbacks.cpp` presents a few examples of calling
+    Python functions from C++, including keywords arguments and unpacking.
+
+.. _PEP448: https://www.python.org/dev/peps/pep-0448/

docs/advanced/pycpp/utilities.rst

@@ -0,0 +1,57 @@
+Utilities
+#########
+
+Using Python's print function in C++
+====================================
+
+The usual way to write output in C++ is using ``std::cout`` while in Python one
+would use ``print``. Since these methods use different buffers, mixing them can
+lead to output order issues. To resolve this, pybind11 modules can use the
+:func:`py::print` function which writes to Python's ``sys.stdout`` for consistency.
+
+Python's ``print`` function is replicated in the C++ API including optional
+keyword arguments ``sep``, ``end``, ``file``, ``flush``. Everything works as
+expected in Python:
+
+.. code-block:: cpp
+
+    py::print(1, 2.0, "three"); // 1 2.0 three
+    py::print(1, 2.0, "three", "sep"_a="-"); // 1-2.0-three
+
+    auto args = py::make_tuple("unpacked", true);
+    py::print("->", *args, "end"_a="<-"); // -> unpacked True <-
+
+Evaluating Python expressions from strings and files
+====================================================
+
+pybind11 provides the :func:`eval` and :func:`eval_file` functions to evaluate
+Python expressions and statements. The following example illustrates how they
+can be used.
+
+Both functions accept a template parameter that describes how the argument
+should be interpreted. Possible choices include ``eval_expr`` (isolated
+expression), ``eval_single_statement`` (a single statement, return value is
+always ``none``), and ``eval_statements`` (sequence of statements, return value
+is always ``none``).
+
+.. code-block:: cpp
+
+    // At beginning of file
+    #include <pybind11/eval.h>
+
+    ...
+
+    // Evaluate in scope of main module
+    py::object scope = py::module::import("__main__").attr("__dict__");
+
+    // Evaluate an isolated expression
+    int result = py::eval("my_variable + 10", scope).cast<int>();
+
+    // Evaluate a sequence of statements
+    py::eval<py::eval_statements>(
+        "print('Hello')\n"
+        "print('world!');",
+        scope);
+
+    // Evaluate the statements in an separate Python file on disk
+    py::eval_file("script.py", scope);

docs/advanced/smart_ptrs.rst

@@ -0,0 +1,149 @@
+Smart pointers
+##############
+
+Unique pointers
+===============
+
+Given a class ``Example`` with Python bindings, it's possible to return
+instances wrapped in C++11 unique pointers, like so
+
+.. code-block:: cpp
+
+    std::unique_ptr<Example> create_example() { return std::unique_ptr<Example>(new Example()); }
+
+.. code-block:: cpp
+
+    m.def("create_example", &create_example);
+
+In other words, there is nothing special that needs to be done. While returning
+unique pointers in this way is allowed, it is *illegal* to use them as function
+arguments. For instance, the following function signature cannot be processed
+by pybind11.
+
+.. code-block:: cpp
+
+    void do_something_with_example(std::unique_ptr<Example> ex) { ... }
+
+The above signature would imply that Python needs to give up ownership of an
+object that is passed to this function, which is generally not possible (for
+instance, the object might be referenced elsewhere).
+
+.. _smart_pointers:
+
+Reference-counting pointers
+===========================
+
+This section explains how to pass values that are wrapped in "smart" pointer
+types with internal reference counting. For the simpler C++11 unique pointers,
+refer to the previous section.
+
+The binding generator for classes, :class:`class_`, can be passed a template
+type that denotes a special *holder* type that is used to manage references to
+the object.  If no such holder type template argument is given, the default for
+a type named ``Type`` is ``std::unique_ptr<Type>``, which means that the object
+is deallocated when Python's reference count goes to zero.
+
+It is possible to switch to other types of reference counting wrappers or smart
+pointers, which is useful in codebases that rely on them. For instance, the
+following snippet causes ``std::shared_ptr`` to be used instead.
+
+.. code-block:: cpp
+
+    py::class_<Example, std::shared_ptr<Example> /* <- holder type */> obj(m, "Example");
+
+Note that any particular class can only be associated with a single holder type.
+
+To enable transparent conversions for functions that take shared pointers as an
+argument or that return them, a macro invocation similar to the following must
+be declared at the top level before any binding code:
+
+.. code-block:: cpp
+
+    PYBIND11_DECLARE_HOLDER_TYPE(T, std::shared_ptr<T>);
+
+.. note::
+
+    The first argument of :func:`PYBIND11_DECLARE_HOLDER_TYPE` should be a
+    placeholder name that is used as a template parameter of the second
+    argument. Thus, feel free to use any identifier, but use it consistently on
+    both sides; also, don't use the name of a type that already exists in your
+    codebase.
+
+One potential stumbling block when using holder types is that they need to be
+applied consistently. Can you guess what's broken about the following binding
+code?
+
+.. code-block:: cpp
+
+    PYBIND11_DECLARE_HOLDER_TYPE(T, std::shared_ptr<T>);
+
+    class Child { };
+
+    class Parent {
+    public:
+       Parent() : child(std::make_shared<Child>()) { }
+       Child *get_child() { return child.get(); }  /* Hint: ** DON'T DO THIS ** */
+    private:
+        std::shared_ptr<Child> child;
+    };
+
+    PYBIND11_PLUGIN(example) {
+        py::module m("example");
+
+        py::class_<Child, std::shared_ptr<Child>>(m, "Child");
+
+        py::class_<Parent, std::shared_ptr<Parent>>(m, "Parent")
+           .def(py::init<>())
+           .def("get_child", &Parent::get_child);
+
+        return m.ptr();
+    }
+
+The following Python code will cause undefined behavior (and likely a
+segmentation fault).
+
+.. code-block:: python
+
+   from example import Parent
+   print(Parent().get_child())
+
+The problem is that ``Parent::get_child()`` returns a pointer to an instance of
+``Child``, but the fact that this instance is already managed by
+``std::shared_ptr<...>`` is lost when passing raw pointers. In this case,
+pybind11 will create a second independent ``std::shared_ptr<...>`` that also
+claims ownership of the pointer. In the end, the object will be freed **twice**
+since these shared pointers have no way of knowing about each other.
+
+There are two ways to resolve this issue:
+
+1. For types that are managed by a smart pointer class, never use raw pointers
+   in function arguments or return values. In other words: always consistently
+   wrap pointers into their designated holder types (such as
+   ``std::shared_ptr<...>``). In this case, the signature of ``get_child()``
+   should be modified as follows:
+
+.. code-block:: cpp
+
+    std::shared_ptr<Child> get_child() { return child; }
+
+2. Adjust the definition of ``Child`` by specifying
+   ``std::enable_shared_from_this<T>`` (see cppreference_ for details) as a
+   base class. This adds a small bit of information to ``Child`` that allows
+   pybind11 to realize that there is already an existing
+   ``std::shared_ptr<...>`` and communicate with it. In this case, the
+   declaration of ``Child`` should look as follows:
+
+.. _cppreference: http://en.cppreference.com/w/cpp/memory/enable_shared_from_this
+
+.. code-block:: cpp
+
+    class Child : public std::enable_shared_from_this<Child> { };
+
+
+Please take a look at the :ref:`macro_notes` before using this feature.
+
+.. seealso::
+
+    The file :file:`tests/test_smart_ptr.cpp` contains a complete example
+    that demonstrates how to work with custom reference-counting holder types
+    in more detail.

docs/basics.rst

@@ -60,6 +60,19 @@ command line.
     the tutorial and look at the test cases in the :file:`tests` directory,
     which exercise all features of pybind11.
 
+Header and namespace conventions
+================================
+
+For brevity, all code examples assume that the following two lines are present:
+
+.. code-block:: cpp
+
+    #include <pybind11/pybind11.h>
+
+    namespace py = pybind11;
+
+Some features may require additional headers, but those will be specified as needed.
+
 Creating bindings for a simple function
 =======================================
 
@@ -93,6 +106,9 @@ a file named :file:`example.cpp` with the following contents:
         return m.ptr();
     }
 
+.. [#f1] In practice, implementation and binding code will generally be located
+         in separate files.
+
 The :func:`PYBIND11_PLUGIN` macro creates a function that will be called when an
 ``import`` statement is issued from within Python. The next line creates a
 module named ``example`` (with the supplied docstring). The method
@@ -235,79 +251,37 @@ The shorthand notation is also available for default arguments:
     // shorthand
     m.def("add2", &add, "i"_a=1, "j"_a=2);
 
+Exporting variables
+===================
+
+To expose a value from C++, use the ``attr`` function to register it in a module
+as shown below. Built-in types and general objects (more on that later) can be
+converted using the function ``py::cast``.
+
+.. code-block:: cpp
+
+    PYBIND11_PLUGIN(example) {
+        py::module m("example", "pybind11 example plugin");
+        m.attr("the_answer") = py::cast(42);
+        m.attr("what") = py::cast("World");
+        return m.ptr();
+    }
+
+These are then accessible from Python:
+
+.. code-block:: pycon
+
+    >>> import example
+    >>> example.the_answer
+    42
+    >>> example.what
+    'World'
+
 .. _supported_types:
 
 Supported data types
 ====================
 
-The following basic data types are supported out of the box (some may require
+A large number of data types are supported out of the box and can be used
-an additional extension header to be included). To pass other data structures
+seamlessly as functions arguments, return values or with ``py::cast`` in general.
-as arguments and return values, refer to the section on binding :ref:`classes`.
+For a full overview, see the :doc:`advanced/cast/index` section.
-
-+---------------------------------+--------------------------+-------------------------------+
-|  Data type                      |  Description             | Header file                   |
-+=================================+==========================+===============================+
-| ``int8_t``, ``uint8_t``         | 8-bit integers           | :file:`pybind11/pybind11.h`   |
-+---------------------------------+--------------------------+-------------------------------+
-| ``int16_t``, ``uint16_t``       | 16-bit integers          | :file:`pybind11/pybind11.h`   |
-+---------------------------------+--------------------------+-------------------------------+
-| ``int32_t``, ``uint32_t``       | 32-bit integers          | :file:`pybind11/pybind11.h`   |
-+---------------------------------+--------------------------+-------------------------------+
-| ``int64_t``, ``uint64_t``       | 64-bit integers          | :file:`pybind11/pybind11.h`   |
-+---------------------------------+--------------------------+-------------------------------+
-| ``ssize_t``, ``size_t``         | Platform-dependent size  | :file:`pybind11/pybind11.h`   |
-+---------------------------------+--------------------------+-------------------------------+
-| ``float``, ``double``           | Floating point types     | :file:`pybind11/pybind11.h`   |
-+---------------------------------+--------------------------+-------------------------------+
-| ``bool``                        | Two-state Boolean type   | :file:`pybind11/pybind11.h`   |
-+---------------------------------+--------------------------+-------------------------------+
-| ``char``                        | Character literal        | :file:`pybind11/pybind11.h`   |
-+---------------------------------+--------------------------+-------------------------------+
-| ``wchar_t``                     | Wide character literal   | :file:`pybind11/pybind11.h`   |
-+---------------------------------+--------------------------+-------------------------------+
-| ``const char *``                | UTF-8 string literal     | :file:`pybind11/pybind11.h`   |
-+---------------------------------+--------------------------+-------------------------------+
-| ``const wchar_t *``             | Wide string literal      | :file:`pybind11/pybind11.h`   |
-+---------------------------------+--------------------------+-------------------------------+
-| ``std::string``                 | STL dynamic UTF-8 string | :file:`pybind11/pybind11.h`   |
-+---------------------------------+--------------------------+-------------------------------+
-| ``std::wstring``                | STL dynamic wide string  | :file:`pybind11/pybind11.h`   |
-+---------------------------------+--------------------------+-------------------------------+
-| ``std::pair<T1, T2>``           | Pair of two custom types | :file:`pybind11/pybind11.h`   |
-+---------------------------------+--------------------------+-------------------------------+
-| ``std::tuple<...>``             | Arbitrary tuple of types | :file:`pybind11/pybind11.h`   |
-+---------------------------------+--------------------------+-------------------------------+
-| ``std::reference_wrapper<...>`` | Reference type wrapper   | :file:`pybind11/pybind11.h`   |
-+---------------------------------+--------------------------+-------------------------------+
-| ``std::complex<T>``             | Complex numbers          | :file:`pybind11/complex.h`    |
-+---------------------------------+--------------------------+-------------------------------+
-| ``std::array<T, Size>``         | STL static array         | :file:`pybind11/stl.h`        |
-+---------------------------------+--------------------------+-------------------------------+
-| ``std::vector<T>``              | STL dynamic array        | :file:`pybind11/stl.h`        |
-+---------------------------------+--------------------------+-------------------------------+
-| ``std::list<T>``                | STL linked list          | :file:`pybind11/stl.h`        |
-+---------------------------------+--------------------------+-------------------------------+
-| ``std::map<T1, T2>``            | STL ordered map          | :file:`pybind11/stl.h`        |
-+---------------------------------+--------------------------+-------------------------------+
-| ``std::unordered_map<T1, T2>``  | STL unordered map        | :file:`pybind11/stl.h`        |
-+---------------------------------+--------------------------+-------------------------------+
-| ``std::set<T>``                 | STL ordered set          | :file:`pybind11/stl.h`        |
-+---------------------------------+--------------------------+-------------------------------+
-| ``std::unordered_set<T>``       | STL unordered set        | :file:`pybind11/stl.h`        |
-+---------------------------------+--------------------------+-------------------------------+
-| ``std::function<...>``          | STL polymorphic function | :file:`pybind11/functional.h` |
-+---------------------------------+--------------------------+-------------------------------+
-| ``std::chrono::duration<...>``  | STL time duration        | :file:`pybind11/chrono.h`     |
-+---------------------------------+--------------------------+-------------------------------+
-| ``std::chrono::time_point<...>``| STL date/time            | :file:`pybind11/chrono.h`     |
-+---------------------------------+--------------------------+-------------------------------+
-| ``Eigen::Matrix<...>``          | Eigen: dense matrix      | :file:`pybind11/eigen.h`      |
-+---------------------------------+--------------------------+-------------------------------+
-| ``Eigen::Map<...>``             | Eigen: mapped memory     | :file:`pybind11/eigen.h`      |
-+---------------------------------+--------------------------+-------------------------------+
-| ``Eigen::SparseMatrix<...>``    | Eigen: sparse matrix     | :file:`pybind11/eigen.h`      |
-+---------------------------------+--------------------------+-------------------------------+
-
-
-.. [#f1] In practice, implementation and binding code will generally be located
-         in separate files.

docs/index.rst

@@ -10,15 +10,36 @@ pybind11 --- Seamless operability between C++11 and Python
     Contents:
 
 .. toctree::
-   :maxdepth: 2
+   :maxdepth: 1
 
    intro
+   changelog
+
+.. toctree::
+   :caption: The Basics
+   :maxdepth: 2
+
    basics
    classes
-   advanced
    compiling
+
+.. toctree::
+   :caption: Advanced Topics
+   :maxdepth: 2
+
+   advanced/functions
+   advanced/classes
+   advanced/exceptions
+   advanced/smart_ptrs
+   advanced/cast/index
+   advanced/pycpp/index
+   advanced/misc
+
+.. toctree::
+   :caption: Extra Information
+   :maxdepth: 1
+
+   faq
    benchmark
    limitations
-   faq
    reference
-   changelog

docs/reference.rst

@@ -215,9 +215,9 @@ Passing extra arguments to the def function
 
 .. function:: arg::arg(const char *name)
 
-.. function:: template <typename T> arg_t<T> arg::operator=(const T &value)
+.. function:: template <typename T> arg_v arg::operator=(T &&value)
 
-.. class:: template <typename T> arg_t<T> : public arg
+.. class:: arg_v : public arg
 
     Represents a named argument with a default value