Add py::module_local() attribute for module-local type bindings

This commit adds a `py::module_local` attribute that lets you confine a
registered type to the module (more technically, the shared object) in
which it is defined, by registering it with:

    py::class_<C>(m, "C", py::module_local())

This will allow the same C++ class `C` to be registered in different
modules with independent sets of class definitions.  On the Python side,
two such types will be completely distinct; on the C++ side, the C++
type resolves to a different Python type in each module.

This applies `py::module_local` automatically to `stl_bind.h` bindings
when the container value type looks like something global: i.e. when it
is a converting type (for example, when binding a `std::vector<int>`),
or when it is a registered type itself bound with `py::module_local`.
This should help resolve potential future conflicts (e.g. if two
completely unrelated modules both try to bind a `std::vector<int>`.
Users can override the automatic selection by adding a
`py::module_local()` or `py::module_local(false)`.

Note that this does mildly break backwards compatibility: bound stl
containers of basic types like `std::vector<int>` cannot be bound in one
module and returned in a different module.  (This can be re-enabled with
`py::module_local(false)` as described above, but with the potential for
eventual load conflicts).
This commit is contained in:
Jason Rhinelander 2017-07-28 22:03:44 -04:00
parent d598172993
commit 7437c69500
13 changed files with 491 additions and 32 deletions

View File

@ -150,10 +150,10 @@ the declaration
before any binding code (e.g. invocations to ``class_::def()``, etc.). This before any binding code (e.g. invocations to ``class_::def()``, etc.). This
macro must be specified at the top level (and outside of any namespaces), since macro must be specified at the top level (and outside of any namespaces), since
it instantiates a partial template overload. If your binding code consists of it instantiates a partial template overload. If your binding code consists of
multiple compilation units, it must be present in every file preceding any multiple compilation units, it must be present in every file (typically via a
usage of ``std::vector<int>``. Opaque types must also have a corresponding common header) preceding any usage of ``std::vector<int>``. Opaque types must
``class_`` declaration to associate them with a name in Python, and to define a also have a corresponding ``class_`` declaration to associate them with a name
set of available operations, e.g.: in Python, and to define a set of available operations, e.g.:
.. code-block:: cpp .. code-block:: cpp
@ -167,6 +167,20 @@ set of available operations, e.g.:
}, py::keep_alive<0, 1>()) /* Keep vector alive while iterator is used */ }, py::keep_alive<0, 1>()) /* Keep vector alive while iterator is used */
// .... // ....
Please take a look at the :ref:`macro_notes` before using the
``PYBIND11_MAKE_OPAQUE`` macro.
.. seealso::
The file :file:`tests/test_opaque_types.cpp` contains a complete
example that demonstrates how to create and expose opaque types using
pybind11 in more detail.
.. _stl_bind:
Binding STL containers
======================
The ability to expose STL containers as native Python objects is a fairly The ability to expose STL containers as native Python objects is a fairly
common request, hence pybind11 also provides an optional header file named common request, hence pybind11 also provides an optional header file named
:file:`pybind11/stl_bind.h` that does exactly this. The mapped containers try :file:`pybind11/stl_bind.h` that does exactly this. The mapped containers try
@ -188,14 +202,34 @@ The following example showcases usage of :file:`pybind11/stl_bind.h`:
py::bind_vector<std::vector<int>>(m, "VectorInt"); py::bind_vector<std::vector<int>>(m, "VectorInt");
py::bind_map<std::map<std::string, double>>(m, "MapStringDouble"); py::bind_map<std::map<std::string, double>>(m, "MapStringDouble");
Please take a look at the :ref:`macro_notes` before using the When binding STL containers pybind11 considers the types of the container's
``PYBIND11_MAKE_OPAQUE`` macro. elements to decide whether the container should be confined to the local module
(via the :ref:`module_local` feature). If the container element types are
anything other than already-bound custom types bound without
``py::module_local()`` the container binding will have ``py::module_local()``
applied. This includes converting types such as numeric types, strings, Eigen
types; and types that have not yet been bound at the time of the stl container
binding. This module-local binding is designed to avoid potential conflicts
between module bindings (for example, from two separate modules each attempting
to bind ``std::vector<int>`` as a python type).
It is possible to override this behavior to force a definition to be either
module-local or global. To do so, you can pass the attributes
``py::module_local()`` (to make the binding module-local) or
``py::module_local(false)`` (to make the binding global) into the
``py::bind_vector`` or ``py::bind_map`` arguments:
.. code-block:: cpp
py::bind_vector<std::vector<int>>(m, "VectorInt", py::module_local(false));
Note, however, that such a global binding would make it impossible to load this
module at the same time as any other pybind module that also attempts to bind
the same container type (``std::vector<int>`` in the above example).
See :ref:`module_local` for more details on module-local bindings.
.. seealso:: .. seealso::
The file :file:`tests/test_opaque_types.cpp` contains a complete
example that demonstrates how to create and expose opaque types using
pybind11 in more detail.
The file :file:`tests/test_stl_binders.cpp` shows how to use the The file :file:`tests/test_stl_binders.cpp` shows how to use the
convenience STL container wrappers. convenience STL container wrappers.

View File

@ -635,3 +635,139 @@ inheritance, which can lead to undefined behavior. In such cases, add the tag
The tag is redundant and does not need to be specified when multiple base types The tag is redundant and does not need to be specified when multiple base types
are listed. are listed.
.. _module_local:
Module-local class bindings
===========================
When creating a binding for a class, pybind by default makes that binding
"global" across modules. What this means is that a type defined in one module
can be passed to functions of other modules that expect the same C++ type. For
example, this allows the following:
.. code-block:: cpp
// In the module1.cpp binding code for module1:
py::class_<Pet>(m, "Pet")
.def(py::init<std::string>());
.. code-block:: cpp
// In the module2.cpp binding code for module2:
m.def("pet_name", [](Pet &p) { return p.name(); });
.. code-block:: pycon
>>> from module1 import Pet
>>> from module2 import pet_name
>>> mypet = Pet("Kitty")
>>> pet_name(mypet)
'Kitty'
When writing binding code for a library, this is usually desirable: this
allows, for example, splitting up a complex library into multiple Python
modules.
In some cases, however, this can cause conflicts. For example, suppose two
unrelated modules make use of an external C++ library and each provide custom
bindings for one of that library's classes. This will result in an error when
a Python program attempts to import both modules (directly or indirectly)
because of conflicting definitions on the external type:
.. code-block:: cpp
// dogs.cpp
// Binding for external library class:
py::class<pets::Pet>(m, "Pet")
.def("name", &pets::Pet::name);
// Binding for local extension class:
py::class<Dog, pets::Pet>(m, "Dog")
.def(py::init<std::string>());
.. code-block:: cpp
// cats.cpp, in a completely separate project from the above dogs.cpp.
// Binding for external library class:
py::class<pets::Pet>(m, "Pet")
.def("get_name", &pets::Pet::name);
// Binding for local extending class:
py::class<Cat, pets::Pet>(m, "Cat")
.def(py::init<std::string>());
.. code-block:: pycon
>>> import cats
>>> import dogs
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: generic_type: type "Pet" is already registered!
To get around this, you can tell pybind11 to keep the external class binding
localized to the module by passing the ``py::module_local()`` attribute into
the ``py::class_`` constructor:
.. code-block:: cpp
// Pet binding in dogs.cpp:
py::class<pets::Pet>(m, "Pet", py::module_local())
.def("name", &pets::Pet::name);
.. code-block:: cpp
// Pet binding in cats.cpp:
py::class<pets::Pet>(m, "Pet", py::module_local())
.def("get_name", &pets::Pet::name);
This makes the Python-side ``dogs.Pet`` and ``cats.Pet`` into distinct classes
that can only be accepted as ``Pet`` arguments within those classes. This
avoids the conflict and allows both modules to be loaded.
One limitation of this approach is that because ``py::module_local`` types are
distinct on the Python side, it is not possible to pass such a module-local
type as a C++ ``Pet``-taking function outside that module. For example, if the
above ``cats`` and ``dogs`` module are each extended with a function:
.. code-block:: cpp
m.def("petname", [](pets::Pet &p) { return p.name(); });
you will only be able to call the function with the local module's class:
.. code-block:: pycon
>>> import cats, dogs # No error because of the added py::module_local()
>>> mycat, mydog = cats.Cat("Fluffy"), dogs.Dog("Rover")
>>> (cats.petname(mycat), dogs.petname(mydog))
('Fluffy', 'Rover')
>>> cats.petname(mydog)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: petname(): incompatible function arguments. The following argument types are supported:
1. (arg0: cats.Pet) -> str
Invoked with: <dogs.Dog object at 0x123>
.. note::
STL bindings (as provided via the optional :file:`pybind11/stl_bind.h`
header) apply ``py::module_local`` by default when the bound type might
conflict with other modules; see :ref:`stl_bind` for details.
.. note::
The localization of the bound types is actually tied to the shared object
or binary generated by the compiler/linker. For typical modules created
with ``PYBIND11_MODULE()``, this distinction is not significant. It is
possible, however, when :ref:`embedding` to embed multiple modules in the
same binary (see :ref:`embedding_modules`). In such a case, the
localization will apply across all embedded modules within the same binary.
.. seealso::
The file :file:`tests/test_local_bindings.cpp` contains additional examples
that demonstrate how ``py::module_local()`` works.

View File

@ -1,3 +1,5 @@
.. _embedding:
Embedding the interpreter Embedding the interpreter
######################### #########################
@ -131,6 +133,7 @@ embedding the interpreter. This makes it easy to import local Python files:
int n = result.cast<int>(); int n = result.cast<int>();
assert(n == 3); assert(n == 3);
.. _embedding_modules:
Adding embedded modules Adding embedded modules
======================= =======================

View File

@ -64,6 +64,9 @@ struct metaclass {
explicit metaclass(handle value) : value(value) { } explicit metaclass(handle value) : value(value) { }
}; };
/// Annotation that marks a class as local to the module:
struct module_local { const bool value; constexpr module_local(bool v = true) : value(v) { } };
/// Annotation to mark enums as an arithmetic type /// Annotation to mark enums as an arithmetic type
struct arithmetic { }; struct arithmetic { };
@ -196,7 +199,7 @@ struct function_record {
/// Special data structure which (temporarily) holds metadata about a bound class /// Special data structure which (temporarily) holds metadata about a bound class
struct type_record { struct type_record {
PYBIND11_NOINLINE type_record() PYBIND11_NOINLINE type_record()
: multiple_inheritance(false), dynamic_attr(false), buffer_protocol(false) { } : multiple_inheritance(false), dynamic_attr(false), buffer_protocol(false), module_local(false) { }
/// Handle to the parent scope /// Handle to the parent scope
handle scope; handle scope;
@ -243,6 +246,9 @@ struct type_record {
/// Is the default (unique_ptr) holder type used? /// Is the default (unique_ptr) holder type used?
bool default_holder : 1; bool default_holder : 1;
/// Is the class definition local to the module shared object?
bool module_local : 1;
PYBIND11_NOINLINE void add_base(const std::type_info &base, void *(*caster)(void *)) { PYBIND11_NOINLINE void add_base(const std::type_info &base, void *(*caster)(void *)) {
auto base_info = detail::get_type_info(base, false); auto base_info = detail::get_type_info(base, false);
if (!base_info) { if (!base_info) {
@ -408,6 +414,10 @@ struct process_attribute<metaclass> : process_attribute_default<metaclass> {
static void init(const metaclass &m, type_record *r) { r->metaclass = m.value; } static void init(const metaclass &m, type_record *r) { r->metaclass = m.value; }
}; };
template <>
struct process_attribute<module_local> : process_attribute_default<module_local> {
static void init(const module_local &l, type_record *r) { r->module_local = l.value; }
};
/// Process an 'arithmetic' attribute for enums (does nothing here) /// Process an 'arithmetic' attribute for enums (does nothing here)
template <> template <>

View File

@ -58,6 +58,8 @@ struct type_info {
bool simple_ancestors : 1; bool simple_ancestors : 1;
/* for base vs derived holder_type checks */ /* for base vs derived holder_type checks */
bool default_holder : 1; bool default_holder : 1;
/* true if this is a type registered with py::module_local */
bool module_local : 1;
}; };
PYBIND11_UNSHARED_STATIC_LOCALS PYBIND11_NOINLINE inline internals *&get_internals_ptr() { PYBIND11_UNSHARED_STATIC_LOCALS PYBIND11_NOINLINE inline internals *&get_internals_ptr() {
@ -126,6 +128,12 @@ PYBIND11_NOINLINE inline internals &get_internals() {
return *internals_ptr; return *internals_ptr;
} }
// Works like internals.registered_types_cpp, but for module-local registered types:
PYBIND11_NOINLINE PYBIND11_UNSHARED_STATIC_LOCALS inline type_map<void *> &registered_local_types_cpp() {
static type_map<void *> locals{};
return locals;
}
/// A life support system for temporary objects created by `type_caster::load()`. /// A life support system for temporary objects created by `type_caster::load()`.
/// Adding a patient will keep it alive up until the enclosing function returns. /// Adding a patient will keep it alive up until the enclosing function returns.
class loader_life_support { class loader_life_support {
@ -213,7 +221,7 @@ PYBIND11_NOINLINE inline void all_type_info_populate(PyTypeObject *t, std::vecto
// registered types // registered types
if (i + 1 == check.size()) { if (i + 1 == check.size()) {
// When we're at the end, we can pop off the current element to avoid growing // When we're at the end, we can pop off the current element to avoid growing
// `check` when adding just one base (which is typical--.e. when there is no // `check` when adding just one base (which is typical--i.e. when there is no
// multiple inheritance) // multiple inheritance)
check.pop_back(); check.pop_back();
i--; i--;
@ -257,13 +265,18 @@ PYBIND11_NOINLINE inline detail::type_info* get_type_info(PyTypeObject *type) {
return bases.front(); return bases.front();
} }
PYBIND11_NOINLINE inline detail::type_info *get_type_info(const std::type_info &tp, /// Return the type info for a given C++ type; on lookup failure can either throw or return nullptr.
PYBIND11_NOINLINE inline detail::type_info *get_type_info(const std::type_index &tp,
bool throw_if_missing = false) { bool throw_if_missing = false) {
std::type_index type_idx(tp);
auto &types = get_internals().registered_types_cpp; auto &types = get_internals().registered_types_cpp;
auto it = types.find(type_idx);
auto it = types.find(std::type_index(tp));
if (it != types.end()) if (it != types.end())
return (detail::type_info *) it->second; return (detail::type_info *) it->second;
auto &locals = registered_local_types_cpp();
it = locals.find(type_idx);
if (it != locals.end())
return (detail::type_info *) it->second;
if (throw_if_missing) { if (throw_if_missing) {
std::string tname = tp.name(); std::string tname = tp.name();
detail::clean_type_id(tname); detail::clean_type_id(tname);
@ -731,10 +744,8 @@ protected:
// with .second = nullptr. (p.first = nullptr is not an error: it becomes None). // with .second = nullptr. (p.first = nullptr is not an error: it becomes None).
PYBIND11_NOINLINE static std::pair<const void *, const type_info *> src_and_type( PYBIND11_NOINLINE static std::pair<const void *, const type_info *> src_and_type(
const void *src, const std::type_info &cast_type, const std::type_info *rtti_type = nullptr) { const void *src, const std::type_info &cast_type, const std::type_info *rtti_type = nullptr) {
auto &internals = get_internals(); if (auto *tpi = get_type_info(cast_type))
auto it = internals.registered_types_cpp.find(std::type_index(cast_type)); return {src, const_cast<const type_info *>(tpi)};
if (it != internals.registered_types_cpp.end())
return {src, (const type_info *) it->second};
// Not found, set error: // Not found, set error:
std::string tname = rtti_type ? rtti_type->name() : cast_type.name(); std::string tname = rtti_type ? rtti_type->name() : cast_type.name();
@ -819,7 +830,6 @@ public:
template <typename T = itype, enable_if_t<std::is_polymorphic<T>::value, int> = 0> template <typename T = itype, enable_if_t<std::is_polymorphic<T>::value, int> = 0>
static std::pair<const void *, const type_info *> src_and_type(const itype *src) { static std::pair<const void *, const type_info *> src_and_type(const itype *src) {
const void *vsrc = src; const void *vsrc = src;
auto &internals = get_internals();
auto &cast_type = typeid(itype); auto &cast_type = typeid(itype);
const std::type_info *instance_type = nullptr; const std::type_info *instance_type = nullptr;
if (vsrc) { if (vsrc) {
@ -828,9 +838,8 @@ public:
// This is a base pointer to a derived type; if it is a pybind11-registered type, we // This is a base pointer to a derived type; if it is a pybind11-registered type, we
// can get the correct derived pointer (which may be != base pointer) by a // can get the correct derived pointer (which may be != base pointer) by a
// dynamic_cast to most derived type: // dynamic_cast to most derived type:
auto it = internals.registered_types_cpp.find(std::type_index(*instance_type)); if (auto *tpi = get_type_info(*instance_type))
if (it != internals.registered_types_cpp.end()) return {dynamic_cast<const void *>(src), const_cast<const type_info *>(tpi)};
return {dynamic_cast<const void *>(src), (const type_info *) it->second};
} }
} }
// Otherwise we have either a nullptr, an `itype` pointer, or an unknown derived pointer, so // Otherwise we have either a nullptr, an `itype` pointer, or an unknown derived pointer, so

View File

@ -839,12 +839,16 @@ protected:
tinfo->dealloc = rec.dealloc; tinfo->dealloc = rec.dealloc;
tinfo->simple_type = true; tinfo->simple_type = true;
tinfo->simple_ancestors = true; tinfo->simple_ancestors = true;
tinfo->default_holder = rec.default_holder;
tinfo->module_local = rec.module_local;
auto &internals = get_internals(); auto &internals = get_internals();
auto tindex = std::type_index(*rec.type); auto tindex = std::type_index(*rec.type);
tinfo->direct_conversions = &internals.direct_conversions[tindex]; tinfo->direct_conversions = &internals.direct_conversions[tindex];
tinfo->default_holder = rec.default_holder; if (rec.module_local)
internals.registered_types_cpp[tindex] = tinfo; registered_local_types_cpp()[tindex] = tinfo;
else
internals.registered_types_cpp[tindex] = tinfo;
internals.registered_types_py[(PyTypeObject *) m_ptr] = { tinfo }; internals.registered_types_py[(PyTypeObject *) m_ptr] = { tinfo };
if (rec.bases.size() > 1 || rec.multiple_inheritance) { if (rec.bases.size() > 1 || rec.multiple_inheritance) {
@ -986,7 +990,7 @@ public:
generic_type::initialize(record); generic_type::initialize(record);
if (has_alias) { if (has_alias) {
auto &instances = get_internals().registered_types_cpp; auto &instances = record.module_local ? registered_local_types_cpp() : get_internals().registered_types_cpp;
instances[std::type_index(typeid(type_alias))] = instances[std::type_index(typeid(type))]; instances[std::type_index(typeid(type_alias))] = instances[std::type_index(typeid(type))];
} }
} }
@ -1442,7 +1446,7 @@ iterator make_iterator(Iterator first, Sentinel last, Extra &&... extra) {
typedef detail::iterator_state<Iterator, Sentinel, false, Policy> state; typedef detail::iterator_state<Iterator, Sentinel, false, Policy> state;
if (!detail::get_type_info(typeid(state), false)) { if (!detail::get_type_info(typeid(state), false)) {
class_<state>(handle(), "iterator") class_<state>(handle(), "iterator", pybind11::module_local())
.def("__iter__", [](state &s) -> state& { return s; }) .def("__iter__", [](state &s) -> state& { return s; })
.def("__next__", [](state &s) -> ValueType { .def("__next__", [](state &s) -> ValueType {
if (!s.first_or_done) if (!s.first_or_done)
@ -1471,7 +1475,7 @@ iterator make_key_iterator(Iterator first, Sentinel last, Extra &&... extra) {
typedef detail::iterator_state<Iterator, Sentinel, true, Policy> state; typedef detail::iterator_state<Iterator, Sentinel, true, Policy> state;
if (!detail::get_type_info(typeid(state), false)) { if (!detail::get_type_info(typeid(state), false)) {
class_<state>(handle(), "iterator") class_<state>(handle(), "iterator", pybind11::module_local())
.def("__iter__", [](state &s) -> state& { return s; }) .def("__iter__", [](state &s) -> state& { return s; })
.def("__next__", [](state &s) -> KeyType { .def("__next__", [](state &s) -> KeyType {
if (!s.first_or_done) if (!s.first_or_done)

View File

@ -373,10 +373,16 @@ NAMESPACE_END(detail)
// std::vector // std::vector
// //
template <typename Vector, typename holder_type = std::unique_ptr<Vector>, typename... Args> template <typename Vector, typename holder_type = std::unique_ptr<Vector>, typename... Args>
class_<Vector, holder_type> bind_vector(module &m, std::string const &name, Args&&... args) { class_<Vector, holder_type> bind_vector(handle scope, std::string const &name, Args&&... args) {
using Class_ = class_<Vector, holder_type>; using Class_ = class_<Vector, holder_type>;
Class_ cl(m, name.c_str(), std::forward<Args>(args)...); // If the value_type is unregistered (e.g. a converting type) or is itself registered
// module-local then make the vector binding module-local as well:
using vtype = typename Vector::value_type;
auto vtype_info = detail::get_type_info(typeid(vtype));
bool local = !vtype_info || vtype_info->module_local;
Class_ cl(scope, name.c_str(), pybind11::module_local(local), std::forward<Args>(args)...);
// Declare the buffer interface if a buffer_protocol() is passed in // Declare the buffer interface if a buffer_protocol() is passed in
detail::vector_buffer<Vector, Class_, Args...>(cl); detail::vector_buffer<Vector, Class_, Args...>(cl);
@ -528,12 +534,22 @@ template <typename Map, typename Class_> auto map_if_insertion_operator(Class_ &
NAMESPACE_END(detail) NAMESPACE_END(detail)
template <typename Map, typename holder_type = std::unique_ptr<Map>, typename... Args> template <typename Map, typename holder_type = std::unique_ptr<Map>, typename... Args>
class_<Map, holder_type> bind_map(module &m, const std::string &name, Args&&... args) { class_<Map, holder_type> bind_map(handle scope, const std::string &name, Args&&... args) {
using KeyType = typename Map::key_type; using KeyType = typename Map::key_type;
using MappedType = typename Map::mapped_type; using MappedType = typename Map::mapped_type;
using Class_ = class_<Map, holder_type>; using Class_ = class_<Map, holder_type>;
Class_ cl(m, name.c_str(), std::forward<Args>(args)...); // If either type is a non-module-local bound type then make the map binding non-local as well;
// otherwise (e.g. both types are either module-local or converting) the map will be
// module-local.
auto tinfo = detail::get_type_info(typeid(MappedType));
bool local = !tinfo || tinfo->module_local;
if (local) {
tinfo = detail::get_type_info(typeid(KeyType));
local = !tinfo || tinfo->module_local;
}
Class_ cl(scope, name.c_str(), pybind11::module_local(local), std::forward<Args>(args)...);
cl.def(init<>()); cl.def(init<>());

View File

@ -40,6 +40,7 @@ set(PYBIND11_TEST_FILES
test_eval.cpp test_eval.cpp
test_exceptions.cpp test_exceptions.cpp
test_kwargs_and_defaults.cpp test_kwargs_and_defaults.cpp
test_local_bindings.cpp
test_methods_and_attributes.cpp test_methods_and_attributes.cpp
test_modules.cpp test_modules.cpp
test_multiple_inheritance.cpp test_multiple_inheritance.cpp
@ -72,6 +73,7 @@ string(REPLACE ".cpp" ".py" PYBIND11_PYTEST_FILES "${PYBIND11_TEST_FILES}")
# doesn't include them) the second module doesn't get built. # doesn't include them) the second module doesn't get built.
set(PYBIND11_CROSS_MODULE_TESTS set(PYBIND11_CROSS_MODULE_TESTS
test_exceptions.py test_exceptions.py
test_local_bindings.py
) )
# Check if Eigen is available; if not, remove from PYBIND11_TEST_FILES (but # Check if Eigen is available; if not, remove from PYBIND11_TEST_FILES (but

26
tests/local_bindings.h Normal file
View File

@ -0,0 +1,26 @@
#pragma once
#include "pybind11_tests.h"
/// Simple class used to test py::local:
template <int> class LocalBase {
public:
LocalBase(int i) : i(i) { }
int i = -1;
};
/// Registered with py::local in both main and secondary modules:
using LocalType = LocalBase<0>;
/// Registered without py::local in both modules:
using NonLocalType = LocalBase<1>;
/// A second non-local type (for stl_bind tests):
using NonLocal2 = LocalBase<2>;
/// Tests within-module, different-compilation-unit local definition conflict:
using LocalExternal = LocalBase<3>;
// Simple bindings (used with the above):
template <typename T, int Adjust, typename... Args>
py::class_<T> bind_local(Args && ...args) {
return py::class_<T>(std::forward<Args>(args)...)
.def(py::init<int>())
.def("get", [](T &i) { return i.i + Adjust; });
};

View File

@ -8,6 +8,8 @@
*/ */
#include "pybind11_tests.h" #include "pybind11_tests.h"
#include "local_bindings.h"
#include <pybind11/stl_bind.h>
PYBIND11_MODULE(pybind11_cross_module_tests, m) { PYBIND11_MODULE(pybind11_cross_module_tests, m) {
m.doc() = "pybind11 cross-module test module"; m.doc() = "pybind11 cross-module test module";
@ -24,4 +26,45 @@ PYBIND11_MODULE(pybind11_cross_module_tests, m) {
m.def("throw_pybind_type_error", []() { throw py::type_error("pybind11 type error"); }); m.def("throw_pybind_type_error", []() { throw py::type_error("pybind11 type error"); });
m.def("throw_stop_iteration", []() { throw py::stop_iteration(); }); m.def("throw_stop_iteration", []() { throw py::stop_iteration(); });
// test_local_bindings.py
// Local to both:
bind_local<LocalType, 1>(m, "LocalType", py::module_local())
.def("get2", [](LocalType &t) { return t.i + 2; })
;
// Can only be called with our python type:
m.def("local_value", [](LocalType &l) { return l.i; });
// test_nonlocal_failure
// This registration will fail (global registration when LocalFail is already registered
// globally in the main test module):
m.def("register_nonlocal", [m]() {
bind_local<NonLocalType, 0>(m, "NonLocalType");
});
// test_stl_bind_local
// stl_bind.h binders defaults to py::module_local if the types are local or converting:
py::bind_vector<std::vector<LocalType>>(m, "LocalVec");
py::bind_map<std::unordered_map<std::string, LocalType>>(m, "LocalMap");
// and global if the type (or one of the types, for the map) is global (so these will fail,
// assuming pybind11_tests is already loaded):
m.def("register_nonlocal_vec", [m]() {
py::bind_vector<std::vector<NonLocalType>>(m, "NonLocalVec");
});
m.def("register_nonlocal_map", [m]() {
py::bind_map<std::unordered_map<std::string, NonLocalType>>(m, "NonLocalMap");
});
// test_stl_bind_global
// The default can, however, be overridden to global using `py::module_local()` or
// `py::module_local(false)`.
// Explicitly made local:
py::bind_vector<std::vector<NonLocal2>>(m, "NonLocalVec2", py::module_local());
// Explicitly made global (and so will fail to bind):
m.def("register_nonlocal_map2", [m]() {
py::bind_map<std::unordered_map<std::string, uint8_t>>(m, "NonLocalMap2", py::module_local(false));
});
// test_internal_locals_differ
m.def("local_cpp_types_addr", []() { return (uintptr_t) &py::detail::registered_local_types_cpp(); });
} }

View File

@ -9,6 +9,7 @@
#include "pybind11_tests.h" #include "pybind11_tests.h"
#include "constructor_stats.h" #include "constructor_stats.h"
#include "local_bindings.h"
TEST_SUBMODULE(class_, m) { TEST_SUBMODULE(class_, m) {
// test_instance // test_instance
@ -224,6 +225,10 @@ TEST_SUBMODULE(class_, m) {
aliased.def(py::init<>()); aliased.def(py::init<>());
aliased.attr("size_noalias") = py::int_(sizeof(AliasedHasOpNewDelSize)); aliased.attr("size_noalias") = py::int_(sizeof(AliasedHasOpNewDelSize));
aliased.attr("size_alias") = py::int_(sizeof(PyAliasedHasOpNewDelSize)); aliased.attr("size_alias") = py::int_(sizeof(PyAliasedHasOpNewDelSize));
// This test is actually part of test_local_bindings (test_duplicate_local), but we need a
// definition in a different compilation unit within the same module:
bind_local<LocalExternal, 17>(m, "LocalExternal", py::module_local());
} }
template <int N> class BreaksBase {}; template <int N> class BreaksBase {};

View File

@ -0,0 +1,62 @@
/*
tests/test_local_bindings.cpp -- tests the py::module_local class feature which makes a class
binding local to the module in which it is defined.
Copyright (c) 2017 Jason Rhinelander <jason@imaginary.ca>
All rights reserved. Use of this source code is governed by a
BSD-style license that can be found in the LICENSE file.
*/
#include "pybind11_tests.h"
#include "local_bindings.h"
#include <pybind11/stl_bind.h>
TEST_SUBMODULE(local_bindings, m) {
// test_local_bindings
// Register a class with py::module_local:
bind_local<LocalType, -1>(m, "LocalType", py::module_local())
.def("get3", [](LocalType &t) { return t.i + 3; })
;
m.def("local_value", [](LocalType &l) { return l.i; });
// test_nonlocal_failure
// The main pybind11 test module is loaded first, so this registration will succeed (the second
// one, in pybind11_cross_module_tests.cpp, is designed to fail):
bind_local<NonLocalType, 0>(m, "NonLocalType")
.def(py::init<int>())
.def("get", [](LocalType &i) { return i.i; })
;
// test_duplicate_local
// py::module_local declarations should be visible across compilation units that get linked together;
// this tries to register a duplicate local. It depends on a definition in test_class.cpp and
// should raise a runtime error from the duplicate definition attempt. If test_class isn't
// available it *also* throws a runtime error (with "test_class not enabled" as value).
m.def("register_local_external", [m]() {
auto main = py::module::import("pybind11_tests");
if (py::hasattr(main, "class_")) {
bind_local<LocalExternal, 7>(m, "LocalExternal", py::module_local());
}
else throw std::runtime_error("test_class not enabled");
});
// test_stl_bind_local
// stl_bind.h binders defaults to py::module_local if the types are local or converting:
py::bind_vector<std::vector<LocalType>>(m, "LocalVec");
py::bind_map<std::unordered_map<std::string, LocalType>>(m, "LocalMap");
// and global if the type (or one of the types, for the map) is global:
py::bind_vector<std::vector<NonLocalType>>(m, "NonLocalVec");
py::bind_map<std::unordered_map<std::string, NonLocalType>>(m, "NonLocalMap");
// test_stl_bind_global
// They can, however, be overridden to global using `py::module_local(false)`:
bind_local<NonLocal2, 10>(m, "NonLocal2");
py::bind_vector<std::vector<NonLocal2>>(m, "LocalVec2", py::module_local());
py::bind_map<std::unordered_map<std::string, uint8_t>>(m, "NonLocalMap2", py::module_local(false));
// test_internal_locals_differ
m.def("local_cpp_types_addr", []() { return (uintptr_t) &py::detail::registered_local_types_cpp(); });
}

View File

@ -0,0 +1,109 @@
import pytest
from pybind11_tests import local_bindings as m
def test_local_bindings():
"""Tests that duplicate py::local class bindings work across modules"""
# Make sure we can load the second module with the conflicting (but local) definition:
import pybind11_cross_module_tests as cm
i1 = m.LocalType(5)
assert i1.get() == 4
assert i1.get3() == 8
i2 = cm.LocalType(10)
assert i2.get() == 11
assert i2.get2() == 12
assert not hasattr(i1, 'get2')
assert not hasattr(i2, 'get3')
assert m.local_value(i1) == 5
assert cm.local_value(i2) == 10
with pytest.raises(TypeError) as excinfo:
m.local_value(i2)
assert "incompatible function arguments" in str(excinfo.value)
with pytest.raises(TypeError) as excinfo:
cm.local_value(i1)
assert "incompatible function arguments" in str(excinfo.value)
def test_nonlocal_failure():
"""Tests that attempting to register a non-local type in multiple modules fails"""
import pybind11_cross_module_tests as cm
with pytest.raises(RuntimeError) as excinfo:
cm.register_nonlocal()
assert str(excinfo.value) == 'generic_type: type "NonLocalType" is already registered!'
def test_duplicate_local():
"""Tests expected failure when registering a class twice with py::local in the same module"""
with pytest.raises(RuntimeError) as excinfo:
m.register_local_external()
import pybind11_tests
assert str(excinfo.value) == (
'generic_type: type "LocalExternal" is already registered!'
if hasattr(pybind11_tests, 'class_') else 'test_class not enabled')
def test_stl_bind_local():
import pybind11_cross_module_tests as cm
v1, v2 = m.LocalVec(), cm.LocalVec()
v1.append(m.LocalType(1))
v1.append(m.LocalType(2))
v2.append(cm.LocalType(1))
v2.append(cm.LocalType(2))
with pytest.raises(TypeError):
v1.append(cm.LocalType(3))
with pytest.raises(TypeError):
v2.append(m.LocalType(3))
assert [i.get() for i in v1] == [0, 1]
assert [i.get() for i in v2] == [2, 3]
v3, v4 = m.NonLocalVec(), cm.NonLocalVec2()
v3.append(m.NonLocalType(1))
v3.append(m.NonLocalType(2))
v4.append(m.NonLocal2(3))
v4.append(m.NonLocal2(4))
assert [i.get() for i in v3] == [1, 2]
assert [i.get() for i in v4] == [13, 14]
d1, d2 = m.LocalMap(), cm.LocalMap()
d1["a"] = v1[0]
d1["b"] = v1[1]
d2["c"] = v2[0]
d2["d"] = v2[1]
assert {i: d1[i].get() for i in d1} == {'a': 0, 'b': 1}
assert {i: d2[i].get() for i in d2} == {'c': 2, 'd': 3}
def test_stl_bind_global():
import pybind11_cross_module_tests as cm
with pytest.raises(RuntimeError) as excinfo:
cm.register_nonlocal_map()
assert str(excinfo.value) == 'generic_type: type "NonLocalMap" is already registered!'
with pytest.raises(RuntimeError) as excinfo:
cm.register_nonlocal_vec()
assert str(excinfo.value) == 'generic_type: type "NonLocalVec" is already registered!'
with pytest.raises(RuntimeError) as excinfo:
cm.register_nonlocal_map2()
assert str(excinfo.value) == 'generic_type: type "NonLocalMap2" is already registered!'
def test_internal_locals_differ():
"""Makes sure the internal local type map differs across the two modules"""
import pybind11_cross_module_tests as cm
assert m.local_cpp_types_addr() != cm.local_cpp_types_addr()