Zengtudor/pybind11
1
0
Fork 0
mirror of https://github.com/pybind/pybind11.git synced 2024-11-21 20:55:11 +00:00
Code Issues Packages Projects Releases Wiki Activity

a bit of work on the new documentation structure

Browse Source
This commit is contained in:
Wenzel Jakob 2016-10-24 23:48:20 +02:00
parent c0d19192d2
commit 6ba98650e2
4 changed files with 235 additions and 164 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/advanced/cast/chrono.rst
View File

@ -39,7 +39,7 @@ depending on the system.
If it is a typedef of the system clock, python will get datetime objects, but if
it is a different clock they will be timedelta objects.
Conversions Provided
Provided conversions
--------------------
.. rubric:: C++ to Python

151
docs/advanced/cast/index.rst
View File

@ -1,148 +1,41 @@
Type conversions
################
There are 3 mechanisms that pybind11 uses to move data between C++ and Python.
We'll take a quick look at each one to get an overview of what's happening.
Apart from enabling cross-language function calls, a fundamental problem
that a binding tool like pybind11 must address is to provide access to
native Python types in C++ and vice versa. There are three fundamentally
different ways to do this—which approach is preferable for a particular type
depends on the situation at hand.
.. rubric:: 1. Native type in C++, wrapper in Python
1. Use a native C++ type everywhere. In this case, the type must be wrapped
using pybind11-generated bindings so that Python can interact with it.
Exposing a custom C++ type using :class:`py::class_` was covered in detail in
the :doc:`/classes` section. There, the underlying data structure is always the
original C++ class while the :class:`py::class_` wrapper provides a Python
interface. Internally, when an object like this is sent from C++ to Python,
pybind11 will just add the outer wrapper layer over the native C++ object.
Getting it back from Python is just a matter of peeling off the wrapper.
2. Use a native Python type everywhere. It will need to be wrapped so that
C++ functions can interact with it.
.. rubric:: 2. Wrapper in C++, native type in Python
3. Use a native C++ type on the C++ side and a native Python type on the
Python side. pybind11 refers to this as a *type conversion*.
This is the exact opposite situation. Now, we have a type which is native to
Python, like a ``tuple`` or a ``list``. One way to get this data into C++ is
with the :class:`py::object` family of wrappers. These are explained in more
detail in the :doc:`/advanced/pycpp/object` section. We'll just give a quick
example here:
Type conversions are the most "natural" option in the sense that native
(non-wrapped) types are used everywhere. The main downside is that a copy
of the data must be made on every Python ↔ C++ transition: this is
needed since the C++ and Python versions of the same type generally won't
have the same memory layout.
.. code-block:: cpp
pybind11 can perform many kinds of conversions automatically. An overview
is provided in the table ":ref:`conversion_table`".
void print_list(py::list my_list) {
for (auto item : my_list)
std::cout << item << " ";
}
.. code-block:: pycon
>>> print_list([1, 2, 3])
1 2 3
The Python ``list`` is not converted in any way -- it's just wrapped in a C++
:class:`py::list` class. At its core it's still a Python object. Copying a
:class:`py::list` will do the usual reference-counting like in Python.
Returning the object to Python will just remove the thin wrapper.
.. rubric:: 3. Converting between native C++ and Python types
In the previous two cases we had a native type in one language and a wrapper in
the other. Now, we have native types on both sides and we convert between them.
.. code-block:: cpp
void print_vector(const std::vector<int> &v) {
for (auto item : v)
std::cout << item << "\n";
}
.. code-block:: pycon
>>> print_vector([1, 2, 3])
1 2 3
In this case, pybind11 will construct a new ``std::vector<int>`` and copy each
element from the Python ``list``. The newly constructed object will be passed
to ``print_vector``. The same thing happens in the other direction: a new
``list`` is made to match the value returned from C++.
Lots of these conversions are supported out of the box, as shown in the table
below. They are very convenient, but keep in mind that these conversions are
fundamentally based on copying data. This is perfectly fine for small immutable
types but it may become quite expensive for large data structures. This can be
avoided by overriding the automatic conversion with a custom wrapper (i.e. the
above-mentioned approach 1). This requires some manual effort and more details
are available in the :ref:`opaque` section.
.. rubric:: Supported automatic conversions
The following subsections discuss the differences between these options in more
detail. The main focus in this section is on type conversions, which represent
the last case of the above list.
.. toctree::
:maxdepth: 1
overview
stl
functional
chrono
eigen
custom
The following basic data types are supported out of the box (some may require
an additional extension header to be included). To pass other data structures
as arguments and return values, refer to the section on binding :ref:`classes`.
+---------------------------------+--------------------------+-------------------------------+
| Data type | Description | Header file |
+=================================+==========================+===============================+
| ``int8_t``, ``uint8_t`` | 8-bit integers | :file:`pybind11/pybind11.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``int16_t``, ``uint16_t`` | 16-bit integers | :file:`pybind11/pybind11.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``int32_t``, ``uint32_t`` | 32-bit integers | :file:`pybind11/pybind11.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``int64_t``, ``uint64_t`` | 64-bit integers | :file:`pybind11/pybind11.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``ssize_t``, ``size_t`` | Platform-dependent size | :file:`pybind11/pybind11.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``float``, ``double`` | Floating point types | :file:`pybind11/pybind11.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``bool`` | Two-state Boolean type | :file:`pybind11/pybind11.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``char`` | Character literal | :file:`pybind11/pybind11.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``wchar_t`` | Wide character literal | :file:`pybind11/pybind11.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``const char *`` | UTF-8 string literal | :file:`pybind11/pybind11.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``const wchar_t *`` | Wide string literal | :file:`pybind11/pybind11.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``std::string`` | STL dynamic UTF-8 string | :file:`pybind11/pybind11.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``std::wstring`` | STL dynamic wide string | :file:`pybind11/pybind11.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``std::pair<T1, T2>`` | Pair of two custom types | :file:`pybind11/pybind11.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``std::tuple<...>`` | Arbitrary tuple of types | :file:`pybind11/pybind11.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``std::reference_wrapper<...>`` | Reference type wrapper | :file:`pybind11/pybind11.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``std::complex<T>`` | Complex numbers | :file:`pybind11/complex.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``std::array<T, Size>`` | STL static array | :file:`pybind11/stl.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``std::vector<T>`` | STL dynamic array | :file:`pybind11/stl.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``std::list<T>`` | STL linked list | :file:`pybind11/stl.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``std::map<T1, T2>`` | STL ordered map | :file:`pybind11/stl.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``std::unordered_map<T1, T2>`` | STL unordered map | :file:`pybind11/stl.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``std::set<T>`` | STL ordered set | :file:`pybind11/stl.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``std::unordered_set<T>`` | STL unordered set | :file:`pybind11/stl.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``std::function<...>`` | STL polymorphic function | :file:`pybind11/functional.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``std::chrono::duration<...>`` | STL time duration | :file:`pybind11/chrono.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``std::chrono::time_point<...>``| STL date/time | :file:`pybind11/chrono.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``Eigen::Matrix<...>`` | Eigen: dense matrix | :file:`pybind11/eigen.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``Eigen::Map<...>`` | Eigen: mapped memory | :file:`pybind11/eigen.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``Eigen::SparseMatrix<...>`` | Eigen: sparse matrix | :file:`pybind11/eigen.h` |
+---------------------------------+--------------------------+-------------------------------+

140
docs/advanced/cast/overview.rst Normal file
View File

@ -0,0 +1,140 @@
Overview
########
.. rubric:: 1. Native type in C++, wrapper in Python
Exposing a custom C++ type using :class:`py::class_` was covered in detail
in the :doc:`/classes` section. There, the underlying data structure is
always the original C++ class while the :class:`py::class_` wrapper provides
a Python interface. Internally, when an object like this is sent from C++ to
Python, pybind11 will just add the outer wrapper layer over the native C++
object. Getting it back from Python is just a matter of peeling off the
wrapper.
.. rubric:: 2. Wrapper in C++, native type in Python
This is the exact opposite situation. Now, we have a type which is native to
Python, like a ``tuple`` or a ``list``. One way to get this data into C++ is
with the :class:`py::object` family of wrappers. These are explained in more
detail in the :doc:`/advanced/pycpp/object` section. We'll just give a quick
example here:
.. code-block:: cpp
void print_list(py::list my_list) {
for (auto item : my_list)
std::cout << item << " ";
}
.. code-block:: pycon
>>> print_list([1, 2, 3])
1 2 3
The Python ``list`` is not converted in any way -- it's just wrapped in a C++
:class:`py::list` class. At its core it's still a Python object. Copying a
:class:`py::list` will do the usual reference-counting like in Python.
Returning the object to Python will just remove the thin wrapper.
.. rubric:: 3. Converting between native C++ and Python types
In the previous two cases we had a native type in one language and a wrapper in
the other. Now, we have native types on both sides and we convert between them.
.. code-block:: cpp
void print_vector(const std::vector<int> &v) {
for (auto item : v)
std::cout << item << "\n";
}
.. code-block:: pycon
>>> print_vector([1, 2, 3])
1 2 3
In this case, pybind11 will construct a new ``std::vector<int>`` and copy each
element from the Python ``list``. The newly constructed object will be passed
to ``print_vector``. The same thing happens in the other direction: a new
``list`` is made to match the value returned from C++.
Lots of these conversions are supported out of the box, as shown in the table
below. They are very convenient, but keep in mind that these conversions are
fundamentally based on copying data. This is perfectly fine for small immutable
types but it may become quite expensive for large data structures. This can be
avoided by overriding the automatic conversion with a custom wrapper (i.e. the
above-mentioned approach 1). This requires some manual effort and more details
are available in the :ref:`opaque` section.
.. _conversion_table:
List of all builtin conversions
-------------------------------
The following basic data types are supported out of the box (some may require
an additional extension header to be included). To pass other data structures
as arguments and return values, refer to the section on binding :ref:`classes`.
+---------------------------------+--------------------------+-------------------------------+
| Data type | Description | Header file |
+=================================+==========================+===============================+
| ``int8_t``, ``uint8_t`` | 8-bit integers | :file:`pybind11/pybind11.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``int16_t``, ``uint16_t`` | 16-bit integers | :file:`pybind11/pybind11.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``int32_t``, ``uint32_t`` | 32-bit integers | :file:`pybind11/pybind11.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``int64_t``, ``uint64_t`` | 64-bit integers | :file:`pybind11/pybind11.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``ssize_t``, ``size_t`` | Platform-dependent size | :file:`pybind11/pybind11.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``float``, ``double`` | Floating point types | :file:`pybind11/pybind11.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``bool`` | Two-state Boolean type | :file:`pybind11/pybind11.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``char`` | Character literal | :file:`pybind11/pybind11.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``wchar_t`` | Wide character literal | :file:`pybind11/pybind11.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``const char *`` | UTF-8 string literal | :file:`pybind11/pybind11.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``const wchar_t *`` | Wide string literal | :file:`pybind11/pybind11.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``std::string`` | STL dynamic UTF-8 string | :file:`pybind11/pybind11.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``std::wstring`` | STL dynamic wide string | :file:`pybind11/pybind11.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``std::pair<T1, T2>`` | Pair of two custom types | :file:`pybind11/pybind11.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``std::tuple<...>`` | Arbitrary tuple of types | :file:`pybind11/pybind11.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``std::reference_wrapper<...>`` | Reference type wrapper | :file:`pybind11/pybind11.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``std::complex<T>`` | Complex numbers | :file:`pybind11/complex.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``std::array<T, Size>`` | STL static array | :file:`pybind11/stl.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``std::vector<T>`` | STL dynamic array | :file:`pybind11/stl.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``std::list<T>`` | STL linked list | :file:`pybind11/stl.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``std::map<T1, T2>`` | STL ordered map | :file:`pybind11/stl.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``std::unordered_map<T1, T2>`` | STL unordered map | :file:`pybind11/stl.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``std::set<T>`` | STL ordered set | :file:`pybind11/stl.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``std::unordered_set<T>`` | STL unordered set | :file:`pybind11/stl.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``std::function<...>`` | STL polymorphic function | :file:`pybind11/functional.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``std::chrono::duration<...>`` | STL time duration | :file:`pybind11/chrono.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``std::chrono::time_point<...>``| STL date/time | :file:`pybind11/chrono.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``Eigen::Matrix<...>`` | Eigen: dense matrix | :file:`pybind11/eigen.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``Eigen::Map<...>`` | Eigen: mapped memory | :file:`pybind11/eigen.h` |
+---------------------------------+--------------------------+-------------------------------+
| ``Eigen::SparseMatrix<...>`` | Eigen: sparse matrix | :file:`pybind11/eigen.h` |
+---------------------------------+--------------------------+-------------------------------+

106
docs/advanced/functions.rst
View File

@ -9,48 +9,62 @@ functions, i.e. *methods* in Python.
Return value policies
=====================
Python and C++ use wildly different ways of managing the memory and lifetime of
objects managed by them. This can lead to issues when creating bindings for
functions that return a non-trivial type. Just by looking at the type
information, it is not clear whether Python should take charge of the returned
value and eventually free its resources, or if this is handled on the C++ side.
For this reason, pybind11 provides a several `return value policy` annotations
that can be passed to the :func:`module::def` and :func:`class_::def`
functions. The default policy is :enum:`return_value_policy::automatic`.
Python and C++ use fundamentally different ways of managing the memory and
lifetime of objects managed by them. This can lead to issues when creating
bindings for functions that return a non-trivial type. Just by looking at the
type information, it is not clear whether Python should take charge of the
returned value and eventually free its resources, or if this is handled on the
C++ side. For this reason, pybind11 provides a several `return value policy`
annotations that can be passed to the :func:`module::def` and
:func:`class_::def` functions. The default policy is
:enum:`return_value_policy::automatic`.
Return value policies can also be applied to properties, in which case the
arguments must be passed through the :class:`cpp_function` constructor:
Return value policies are tricky, and it's very important to get them right.
Just to illustrate what can go wrong, consider the following simple example:
.. code-block:: cpp
class_<MyClass>(m, "MyClass")
def_property("data"
py::cpp_function(&MyClass::getData, py::return_value_policy::copy),
py::cpp_function(&MyClass::setData)
);
/* Function declaration */
Data *get_data() { return _data; /* (pointer to a static data structure) */ }
...
The following table provides an overview of the available return value policies:
/* Binding code */
m.def("get_data", &get_data); // <-- KABOOM, will cause crash when called from Python
What's going on here? When ``get_data()`` is called from Python, the return
value (a native C++ type) must be wrapped to turn it into a usable Python type.
In this case, the default return value policy (:enum:`return_value_policy::automatic`)
causes pybind11 to assume ownership of the static ``_data`` instance.
When Python's garbage collector eventually deletes the Python
wrapper, pybind11 will also attempt to delete the C++ instance (via ``operator
delete()``) due to the implied ownership. At this point, the entire application
will come crashing down, though errors could also be more subtle and involve
silent data corruption.
In the above example, the policy :enum:`return_value_policy::reference` should have
been specified so that the global data instance is only *referenced* without any
implied transfer of ownership, i.e.:
.. code-block:: cpp
m.def("get_data", &get_data, return_value_policy::reference);
On the other hand, this is not the right policy for many other situations,
where ignoring ownership could lead to resource leaks.
As a developer using pybind11, it's important to be familiar with the different
return value policies, including which situation calls for which one of them.
The following table provides an overview of available policies:
.. tabularcolumns:: |p{0.5\textwidth}|p{0.45\textwidth}|
+--------------------------------------------------+----------------------------------------------------------------------------+
| Return value policy | Description |
+==================================================+============================================================================+
| :enum:`return_value_policy::automatic` | This is the default return value policy, which falls back to the policy |
| | :enum:`return_value_policy::take_ownership` when the return value is a |
| | pointer. Otherwise, it uses :enum:`return_value::move` or |
| | :enum:`return_value::copy` for rvalue and lvalue references, respectively. |
| | See below for a description of what all of these different policies do. |
+--------------------------------------------------+----------------------------------------------------------------------------+
| :enum:`return_value_policy::automatic_reference` | As above, but use policy :enum:`return_value_policy::reference` when the |
| | return value is a pointer. This is the default conversion policy for |
| | function arguments when calling Python functions manually from C++ code |
| | (i.e. via handle::operator()). You probably won't need to use this. |
+--------------------------------------------------+----------------------------------------------------------------------------+
| :enum:`return_value_policy::take_ownership` | Reference an existing object (i.e. do not create a new copy) and take |
| | ownership. Python will call the destructor and delete operator when the |
| | object's reference count reaches zero. Undefined behavior ensues when the |
| | C++ side does the same. |
| | C++ side does the same, or when the data was not dynamically allocated. |
+--------------------------------------------------+----------------------------------------------------------------------------+
| :enum:`return_value_policy::copy` | Create a new copy of the returned object, which will be owned by Python. |
| | This policy is comparably safe because the lifetimes of the two instances |
@ -74,6 +88,28 @@ The following table provides an overview of the available return value policies:
| | return value is referenced by Python. This is the default policy for |
| | property getters created via ``def_property``, ``def_readwrite``, etc. |
+--------------------------------------------------+----------------------------------------------------------------------------+
| :enum:`return_value_policy::automatic` | This is the default return value policy, which falls back to the policy |
| | :enum:`return_value_policy::take_ownership` when the return value is a |
| | pointer. Otherwise, it uses :enum:`return_value::move` or |
| | :enum:`return_value::copy` for rvalue and lvalue references, respectively. |
| | See above for a description of what all of these different policies do. |
+--------------------------------------------------+----------------------------------------------------------------------------+
| :enum:`return_value_policy::automatic_reference` | As above, but use policy :enum:`return_value_policy::reference` when the |
| | return value is a pointer. This is the default conversion policy for |
| | function arguments when calling Python functions manually from C++ code |
| | (i.e. via handle::operator()). You probably won't need to use this. |
+--------------------------------------------------+----------------------------------------------------------------------------+
Return value policies can also be applied to properties, in which case the
arguments must be passed through the :class:`cpp_function` constructor:
.. code-block:: cpp
class_<MyClass>(m, "MyClass")
def_property("data"
py::cpp_function(&MyClass::getData, py::return_value_policy::copy),
py::cpp_function(&MyClass::setData)
);
.. warning::
@ -82,12 +118,14 @@ The following table provides an overview of the available return value policies:
non-determinism and segmentation faults, hence it is worth spending the
time to understand all the different options in the table above.
One important aspect of the above policies is that they only apply to instances
which pybind11 has *not* seen before, in which case the policy clarifies
essential questions about the return value's lifetime and ownership. When
pybind11 knows the instance already (as identified by its type and address in
memory), it will return the existing Python object wrapper rather than creating
a new copy.
.. note::
One important aspect of the above policies is that they only apply to
instances which pybind11 has *not* seen before, in which case the policy
clarifies essential questions about the return value's lifetime and
ownership. When pybind11 knows the instance already (as identified by its
type and address in memory), it will return the existing Python object
wrapper rather than creating a new copy.
.. note::