minor doc & style fixes

This commit is contained in:
Wenzel Jakob 2016-09-06 13:02:29 +09:00
parent 07082eecc3
commit fe34241e50
14 changed files with 220 additions and 176 deletions

View File

@ -551,27 +551,150 @@ and the Python ``list``, ``set`` and ``dict`` data structures are automatically
enabled. The types ``std::pair<>`` and ``std::tuple<>`` are already supported
out of the box with just the core :file:`pybind11/pybind11.h` header.
Alternatively it might be desirable to bind STL containers as native C++ classes,
eliminating the need of converting back and forth between C++ representation
and Python one. The downside of this approach in this case users will have to
deal with C++ containers directly instead of using already familiar Python lists
or dicts.
Pybind11 provide set of binder functions to bind various STL containers like vectors,
maps etc. All binder functions are designed to return instances of pybind11::class_
objects so developers can bind extra functions if needed. For complete set of
available functions please see :file:`pybind11/stl_bind.h`. For an example on using
this feature, please see :file:`tests/test_stl_binders.cpp`.
The major downside of these implicit conversions is that containers must be
converted (i.e. copied) on every Python->C++ and C++->Python transition, which
can have implications on the program semantics and performance. Please read the
next sections for more details and alternative approaches that avoid this.
.. note::
Arbitrary nesting of any of these types is supported.
Arbitrary nesting of any of these types is possible.
.. seealso::
The file :file:`tests/test_python_types.cpp` contains a complete
example that demonstrates how to pass STL data types in more detail.
.. _opaque:
Treating STL data structures as opaque objects
==============================================
pybind11 heavily relies on a template matching mechanism to convert parameters
and return values that are constructed from STL data types such as vectors,
linked lists, hash tables, etc. This even works in a recursive manner, for
instance to deal with lists of hash maps of pairs of elementary and custom
types, etc.
However, a fundamental limitation of this approach is that internal conversions
between Python and C++ types involve a copy operation that prevents
pass-by-reference semantics. What does this mean?
Suppose we bind the following function
.. code-block:: cpp
void append_1(std::vector<int> &v) {
v.push_back(1);
}
and call it from Python, the following happens:
.. code-block:: pycon
>>> v = [5, 6]
>>> append_1(v)
>>> print(v)
[5, 6]
As you can see, when passing STL data structures by reference, modifications
are not propagated back the Python side. A similar situation arises when
exposing STL data structures using the ``def_readwrite`` or ``def_readonly``
functions:
.. code-block:: cpp
/* ... definition ... */
class MyClass {
std::vector<int> contents;
};
/* ... binding code ... */
py::class_<MyClass>(m, "MyClass")
.def(py::init<>)
.def_readwrite("contents", &MyClass::contents);
In this case, properties can be read and written in their entirety. However, an
``append`` operaton involving such a list type has no effect:
.. code-block:: pycon
>>> m = MyClass()
>>> m.contents = [5, 6]
>>> print(m.contents)
[5, 6]
>>> m.contents.append(7)
>>> print(m.contents)
[5, 6]
Finally, the involved copy operations can be costly when dealing with very
large lists. To deal with all of the above situations, pybind11 provides a
macro named ``PYBIND11_MAKE_OPAQUE(T)`` that disables the template-based
conversion machinery of types, thus rendering them *opaque*. The contents of
opaque objects are never inspected or extracted, hence they *can* be passed by
reference. For instance, to turn ``std::vector<int>`` into an opaque type, add
the declaration
.. code-block:: cpp
PYBIND11_MAKE_OPAQUE(std::vector<int>);
before any binding code (e.g. invocations to ``class_::def()``, etc.). This
macro must be specified at the top level (and outside of any namespaces), since
it instantiates a partial template overload. If your binding code consists of
multiple compilation units, it must be present in every file preceding any
usage of ``std::vector<int>``. Opaque types must also have a corresponding
``class_`` declaration to associate them with a name in Python, and to define a
set of available operations, e.g.:
.. code-block:: cpp
py::class_<std::vector<int>>(m, "IntVector")
.def(py::init<>())
.def("clear", &std::vector<int>::clear)
.def("pop_back", &std::vector<int>::pop_back)
.def("__len__", [](const std::vector<int> &v) { return v.size(); })
.def("__iter__", [](std::vector<int> &v) {
return py::make_iterator(v.begin(), v.end());
}, py::keep_alive<0, 1>()) /* Keep vector alive while iterator is used */
// ....
The ability to expose STL containers as native Python objects is a fairly
common request, hence pybind11 also provides an optional header file named
:file:`pybind11/stl_bind.h` that does exactly this. The mapped containers try
to match the behavior of their native Python counterparts as much as possible.
The following example showcases usage of :file:`pybind11/stl_bind.h`:
.. code-block:: cpp
// Don't forget this
#include <pybind11/stl_bind.h>
PYBIND11_MAKE_OPAQUE(std::vector<int>);
PYBIND11_MAKE_OPAQUE(std::map<std::string, double>);
// ...
// later in binding code:
py::bind_vector<std::vector<int>>(m, "VectorInt");
py::bind_map<std::map<std::string, double>>(m, "MapStringDouble");
Please take a look at the :ref:`macro_notes` before using the
``PYBIND11_MAKE_OPAQUE`` macro.
.. seealso::
The file :file:`tests/test_opaque_types.cpp` contains a complete
example that demonstrates how to create and expose opaque types using
pybind11 in more detail.
The file :file:`tests/test_stl_binders.cpp` shows how to use the
convenience STL container wrappers.
Binding sequence data types, iterators, the slicing protocol, etc.
==================================================================
@ -1103,108 +1226,6 @@ section.
The ``py::exception`` wrapper for creating custom exceptions cannot (yet)
be used as a ``py::base``.
.. _opaque:
Treating STL data structures as opaque objects
==============================================
pybind11 heavily relies on a template matching mechanism to convert parameters
and return values that are constructed from STL data types such as vectors,
linked lists, hash tables, etc. This even works in a recursive manner, for
instance to deal with lists of hash maps of pairs of elementary and custom
types, etc.
However, a fundamental limitation of this approach is that internal conversions
between Python and C++ types involve a copy operation that prevents
pass-by-reference semantics. What does this mean?
Suppose we bind the following function
.. code-block:: cpp
void append_1(std::vector<int> &v) {
v.push_back(1);
}
and call it from Python, the following happens:
.. code-block:: pycon
>>> v = [5, 6]
>>> append_1(v)
>>> print(v)
[5, 6]
As you can see, when passing STL data structures by reference, modifications
are not propagated back the Python side. A similar situation arises when
exposing STL data structures using the ``def_readwrite`` or ``def_readonly``
functions:
.. code-block:: cpp
/* ... definition ... */
class MyClass {
std::vector<int> contents;
};
/* ... binding code ... */
py::class_<MyClass>(m, "MyClass")
.def(py::init<>)
.def_readwrite("contents", &MyClass::contents);
In this case, properties can be read and written in their entirety. However, an
``append`` operaton involving such a list type has no effect:
.. code-block:: pycon
>>> m = MyClass()
>>> m.contents = [5, 6]
>>> print(m.contents)
[5, 6]
>>> m.contents.append(7)
>>> print(m.contents)
[5, 6]
To deal with both of the above situations, pybind11 provides a macro named
``PYBIND11_MAKE_OPAQUE(T)`` that disables the template-based conversion
machinery of types, thus rendering them *opaque*. The contents of opaque
objects are never inspected or extracted, hence they can be passed by
reference. For instance, to turn ``std::vector<int>`` into an opaque type, add
the declaration
.. code-block:: cpp
PYBIND11_MAKE_OPAQUE(std::vector<int>);
before any binding code (e.g. invocations to ``class_::def()``, etc.). This
macro must be specified at the top level, since instantiates a partial template
overload. If your binding code consists of multiple compilation units, it must
be present in every file preceding any usage of ``std::vector<int>``. Opaque
types must also have a corresponding ``class_`` declaration to associate them
with a name in Python, and to define a set of available operations:
.. code-block:: cpp
py::class_<std::vector<int>>(m, "IntVector")
.def(py::init<>())
.def("clear", &std::vector<int>::clear)
.def("pop_back", &std::vector<int>::pop_back)
.def("__len__", [](const std::vector<int> &v) { return v.size(); })
.def("__iter__", [](std::vector<int> &v) {
return py::make_iterator(v.begin(), v.end());
}, py::keep_alive<0, 1>()) /* Keep vector alive while iterator is used */
// ....
Please take a look at the :ref:`macro_notes` before using this feature.
.. seealso::
The file :file:`tests/test_opaque_types.cpp` contains a complete
example that demonstrates how to create and expose opaque types using
pybind11 in more detail.
.. _eigen:
Transparent conversion of dense and sparse Eigen data types

View File

@ -178,16 +178,16 @@ The keyword names also appear in the function signatures within the documentatio
A shorter notation for named arguments is also available:
.. code-block:: cpp
// regular notation
m.def("add1", &add, py::arg("i"), py::arg("j"));
// shorthand
using namespace pybind11::literals;
m.def("add2", &add, "i"_a, "j"_a);
The :var:`_a` suffix forms a C++11 literal which is equivalent to :class:`arg`.
Note that the literal operator must first be made visible with the directive
``using namespace pybind11::literals``. This does not bring in anything else
The :var:`_a` suffix forms a C++11 literal which is equivalent to :class:`arg`.
Note that the literal operator must first be made visible with the directive
``using namespace pybind11::literals``. This does not bring in anything else
from the ``pybind11`` namespace except for literals.
.. _default_args:
@ -229,7 +229,7 @@ The default values also appear within the documentation.
The shorthand notation is also available for default arguments:
.. code-block:: cpp
// regular notation
m.def("add1", &add, py::arg("i") = 1, py::arg("j") = 2);
// shorthand

View File

@ -26,7 +26,7 @@ Building with cppimport
Building with CMake
===================
For C++ codebases that have an existing CMake-based build system, a Python
For C++ codebases that have an existing CMake-based build system, a Python
extension module can be created with just a few lines of code:
.. code-block:: cmake
@ -37,14 +37,14 @@ extension module can be created with just a few lines of code:
add_subdirectory(pybind11)
pybind11_add_module(example example.cpp)
This assumes that the pybind11 repository is located in a subdirectory named
This assumes that the pybind11 repository is located in a subdirectory named
:file:`pybind11` and that the code is located in a file named :file:`example.cpp`.
The CMake command ``add_subdirectory`` will import a function with the signature
``pybind11_add_module(<name> source1 [source2 ...])``. It will take care of all
the details needed to build a Python extension module on any platform.
The target Python version can be selected by setting the ``PYBIND11_PYTHON_VERSION``
variable before adding the pybind11 subdirectory. Alternatively, an exact Python
The target Python version can be selected by setting the ``PYBIND11_PYTHON_VERSION``
variable before adding the pybind11 subdirectory. Alternatively, an exact Python
installation can be specified by setting ``PYTHON_EXECUTABLE``.
A working sample project, including a way to invoke CMake from :file:`setup.py` for

View File

@ -140,7 +140,7 @@ the included test suite contains the following symbol:
.. code-block:: none
__ZN8pybind1112cpp_functionC1Iv8Example2JRNSt3__16vectorINS3_12basic_stringIwNS3_11char_traitsIwEENS3_9allocatorIwEEEENS8_ISA_EEEEEJNS_4nameENS_7siblingENS_9is_methodEA28_cEEEMT0_FT_DpT1_EDpRKT2_
__ZN8pybind1112cpp_functionC1Iv8Example2JRNSt3__16vectorINS3_12basic_stringIwNS3_11char_traitsIwEENS3_9allocatorIwEEEENS8_ISA_EEEEEJNS_4nameENS_7siblingENS_9is_methodEA28_cEEEMT0_FT_DpT1_EDpRKT2_
.. only:: not html

View File

@ -681,7 +681,7 @@ public:
static PYBIND11_DESCR element_names() {
return detail::concat(type_caster<typename intrinsic_type<Tuple>::type>::name()...);
}
static PYBIND11_DESCR name() {
return type_descr(_("Tuple[") + element_names() + _("]"));
}

View File

@ -1042,7 +1042,7 @@ public:
def("__ne__", [](const Type &value, UnderlyingType value2) { return (UnderlyingType) value != value2; });
}
def("__hash__", [](const Type &value) { return (UnderlyingType) value; });
// Pickling and unpickling -- needed for use with the 'multiprocessing' module
// Pickling and unpickling -- needed for use with the 'multiprocessing' module
def("__getstate__", [](const Type &value) { return pybind11::make_tuple((UnderlyingType) value); });
def("__setstate__", [](Type &p, tuple t) { new (&p) Type((Type) t[0].cast<UnderlyingType>()); });
m_entries = entries;

View File

@ -12,7 +12,6 @@
#include "common.h"
#include "operators.h"
#include <map>
#include <type_traits>
#include <utility>
#include <algorithm>
@ -136,7 +135,7 @@ NAMESPACE_END(detail)
//
template <typename Vector, typename holder_type = std::unique_ptr<Vector>, typename... Args>
pybind11::class_<Vector, holder_type> bind_vector(pybind11::module &m, std::string const &name, Args&&... args) {
using T = typename Vector::value_type;
using T = typename Vector::value_type;
using SizeType = typename Vector::size_type;
using DiffType = typename Vector::difference_type;
using ItType = typename Vector::iterator;
@ -356,7 +355,7 @@ pybind11::class_<Vector, holder_type> bind_vector(pybind11::module &m, std::stri
//
// std::map
// std::map, std::unordered_map
//
NAMESPACE_BEGIN(detail)
@ -373,8 +372,8 @@ template <typename Map, typename Class_, typename... Args> void map_if_copy_assi
auto it = m.find(k);
if (it != m.end()) it->second = v;
else m.emplace(k, v);
});
}
);
}
template<typename Map, typename Class_, typename std::enable_if<!std::is_copy_assignable<typename Map::mapped_type>::value, int>::type = 0>
@ -384,12 +383,15 @@ void map_if_copy_assignable(Class_ &cl) {
cl.def("__setitem__",
[](Map &m, const KeyType &k, const MappedType &v) {
auto r = m.insert( std::make_pair(k, v) ); // We can't use m[k] = v; because value type might not be default constructable
if (!r.second) { // value type might be const so the only way to insert it is to errase it first...
// We can't use m[k] = v; because value type might not be default constructable
auto r = m.insert(std::make_pair(k, v));
if (!r.second) {
// value type might be const so the only way to insert it is to erase it first...
m.erase(r.first);
m.insert( std::make_pair(k, v) );
m.insert(std::make_pair(k, v));
}
});
}
);
}
@ -401,8 +403,9 @@ template <typename Map, typename Class_> auto map_if_insertion_operator(Class_ &
std::ostringstream s;
s << name << '{';
bool f = false;
for (auto const & kv : m) {
if (f) s << ", ";
for (auto const &kv : m) {
if (f)
s << ", ";
s << kv.first << ": " << kv.second;
f = true;
}
@ -428,17 +431,13 @@ pybind11::class_<Map, holder_type> bind_map(module &m, const std::string &name,
detail::map_if_insertion_operator<Map, Class_>(cl, name);
cl.def("__bool__",
[](const Map &m) -> bool {
return !m.empty();
},
[](const Map &m) -> bool { return !m.empty(); },
"Check whether the map is nonempty"
);
cl.def("__iter__",
[](Map &m) {
return pybind11::make_key_iterator(m.begin(), m.end());
},
pybind11::keep_alive<0, 1>() /* Essential: keep list alive while iterator exists */
[](Map &m) { return pybind11::make_key_iterator(m.begin(), m.end()); },
pybind11::keep_alive<0, 1>() /* Essential: keep list alive while iterator exists */
);
cl.def("items",
@ -449,18 +448,22 @@ pybind11::class_<Map, holder_type> bind_map(module &m, const std::string &name,
cl.def("__getitem__",
[](Map &m, const KeyType &k) -> MappedType {
auto it = m.find(k);
if (it != m.end()) return it->second;
else throw pybind11::key_error(); // it is not always possible to convert key to string // pybind11::key_error(k)
});
if (it == m.end())
throw pybind11::key_error();
return it->second;
}
);
detail::map_if_copy_assignable<Map, Class_>(cl);
cl.def("__delitem__",
[](Map &m, const KeyType &k) {
[](Map &m, const KeyType &k) {
auto it = m.find(k);
if (it != m.end()) return m.erase(it);
else throw pybind11::key_error(); // it is not always possible to convert key to string // pybind11::key_error(k)
});
if (it == m.end())
throw pybind11::key_error();
return m.erase(it);
}
);
cl.def("__len__", &Map::size);

View File

@ -82,7 +82,7 @@ public:
/// Move constructor
ref(ref &&r) : m_ptr(r.m_ptr) {
r.m_ptr = nullptr;
r.m_ptr = nullptr;
print_move_created(this, "with pointer", m_ptr); track_move_created((ref_tag*) this);
}

View File

@ -80,7 +80,7 @@ test_initializer callbacks([](py::module &m) {
/* Test cleanup of lambda closure */
m.def("test_cleanup", []() -> std::function<void(void)> {
m.def("test_cleanup", []() -> std::function<void(void)> {
Payload p;
return [p]() {

View File

@ -84,51 +84,51 @@ test_initializer eigen([](py::module &m) {
return m.selfadjointView<Eigen::Upper>();
});
m.def("fixed_r", [mat]() -> FixedMatrixR {
m.def("fixed_r", [mat]() -> FixedMatrixR {
return FixedMatrixR(mat);
});
m.def("fixed_c", [mat]() -> FixedMatrixC {
m.def("fixed_c", [mat]() -> FixedMatrixC {
return FixedMatrixC(mat);
});
m.def("fixed_passthrough_r", [](const FixedMatrixR &m) -> FixedMatrixR {
m.def("fixed_passthrough_r", [](const FixedMatrixR &m) -> FixedMatrixR {
return m;
});
m.def("fixed_passthrough_c", [](const FixedMatrixC &m) -> FixedMatrixC {
m.def("fixed_passthrough_c", [](const FixedMatrixC &m) -> FixedMatrixC {
return m;
});
m.def("dense_r", [mat]() -> DenseMatrixR {
m.def("dense_r", [mat]() -> DenseMatrixR {
return DenseMatrixR(mat);
});
m.def("dense_c", [mat]() -> DenseMatrixC {
m.def("dense_c", [mat]() -> DenseMatrixC {
return DenseMatrixC(mat);
});
m.def("dense_passthrough_r", [](const DenseMatrixR &m) -> DenseMatrixR {
m.def("dense_passthrough_r", [](const DenseMatrixR &m) -> DenseMatrixR {
return m;
});
m.def("dense_passthrough_c", [](const DenseMatrixC &m) -> DenseMatrixC {
m.def("dense_passthrough_c", [](const DenseMatrixC &m) -> DenseMatrixC {
return m;
});
m.def("sparse_r", [mat]() -> SparseMatrixR {
m.def("sparse_r", [mat]() -> SparseMatrixR {
return Eigen::SparseView<Eigen::MatrixXf>(mat);
});
m.def("sparse_c", [mat]() -> SparseMatrixC {
m.def("sparse_c", [mat]() -> SparseMatrixC {
return Eigen::SparseView<Eigen::MatrixXf>(mat);
});
m.def("sparse_passthrough_r", [](const SparseMatrixR &m) -> SparseMatrixR {
m.def("sparse_passthrough_r", [](const SparseMatrixR &m) -> SparseMatrixR {
return m;
});
m.def("sparse_passthrough_c", [](const SparseMatrixC &m) -> SparseMatrixC {
m.def("sparse_passthrough_c", [](const SparseMatrixC &m) -> SparseMatrixC {
return m;
});
});

View File

@ -25,7 +25,7 @@ public:
EFirstMode = 1,
ESecondMode
};
static EMode test_function(EMode mode) {
return mode;
}

View File

@ -59,7 +59,7 @@ test_initializer arg_keywords_and_defaults([](py::module &m) {
using namespace py::literals;
m.def("kw_func_udl", &kw_func, "x"_a, "y"_a=300);
m.def("kw_func_udl_z", &kw_func, "x"_a, "y"_a=0);
py::class_<KWClass>(m, "KWClass")
.def("foo0", &KWClass::foo)
.def("foo1", &KWClass::foo, "x"_a, "y"_a);

View File

@ -10,6 +10,8 @@
#include "pybind11_tests.h"
#include <pybind11/stl_bind.h>
#include <map>
#include <unordered_map>
class El {
public:
@ -28,18 +30,18 @@ test_initializer stl_binder_vector([](py::module &m) {
py::class_<El>(m, "El")
.def(py::init<int>());
py::bind_vector< std::vector<unsigned int> >(m, "VectorInt");
py::bind_vector< std::vector<bool> >(m, "VectorBool");
py::bind_vector<std::vector<unsigned int>>(m, "VectorInt");
py::bind_vector<std::vector<bool>>(m, "VectorBool");
py::bind_vector< std::vector<El> >(m, "VectorEl");
py::bind_vector<std::vector<El>>(m, "VectorEl");
py::bind_vector< std::vector< std::vector<El> > >(m, "VectorVectorEl");
py::bind_vector<std::vector<std::vector<El>>>(m, "VectorVectorEl");
});
test_initializer stl_binder_map([](py::module &m) {
py::bind_map< std::map<std::string, double> >(m, "MapStringDouble");
py::bind_map< std::unordered_map<std::string, double> >(m, "UnorderedMapStringDouble");
py::bind_map<std::map<std::string, double>>(m, "MapStringDouble");
py::bind_map<std::unordered_map<std::string, double>>(m, "UnorderedMapStringDouble");
py::bind_map< std::map<std::string, double const> >(m, "MapStringDoubleConst");
py::bind_map< std::unordered_map<std::string, double const> >(m, "UnorderedMapStringDoubleConst");
py::bind_map<std::map<std::string, double const>>(m, "MapStringDoubleConst");
py::bind_map<std::unordered_map<std::string, double const>>(m, "UnorderedMapStringDoubleConst");
});

View File

@ -1,7 +1,12 @@
#!/bin/bash
#
# Script to check include/test code for common pybind11 code style errors.
# Currently just checks for tabs used instead of spaces.
#
# This script currently checks for
#
# 1. use of tabs instead of spaces
# 2. trailing spaces
# 3. missing space between keyword and parenthesis, e.g.: for(, if(, while(
#
# Invoke as: tools/check-style.sh
#
@ -21,6 +26,19 @@ while read -u 3 f; do
echo " $f"
done
found=
# The mt=41 sets a red background for matched trailing spaces
exec 3< <(GREP_COLORS='mt=41' grep '\s\+$' include/ tests/*.{cpp,py,h} docs/*.rst -rn --color=always)
while read -u 3 f; do
if [ -z "$found" ]; then
echo -e '\e[31m\e[01mError: found trailing spaces in the following files:\e[0m'
found=1
errors=1
fi
echo " $f"
done
found=
exec 3< <(grep '\<\(if\|for\|while\)(\|){' include/ tests/*.{cpp,py,h} -rn --color=always)
while read -u 3 line; do