stl casters were using a value cast to (Value) or (Key), but that isn't
always appropriate. This changes it to use the appropriate value
converter's cast_op_type.
A flake8 configuration is included in setup.cfg and the checks are
executed automatically on Travis:
* Ensures a consistent PEP8 code style
* Does basic linting to prevent possible bugs
Fixes#509.
The move policy was already set for rvalues in PR #473, but this only
applied to directly cast user-defined types. The problem is that STL
containers cast values indirectly and the rvalue information is lost.
Therefore the move policy was not set correctly. This commit fixes it.
This also makes an additional adjustment to remove the `copy` policy
exception: rvalues now always use the `move` policy. This is also safe
for copy-only rvalues because the `move` policy has an internal fallback
to copying.
Following commit 90d278, the object code generated by the python
bindings of nanogui (github.com/wjakob/nanogui) went up by a whopping
12%. It turns out that that project has quite a few enums where we don't
really care about arithmetic operators.
This commit thus partially reverts the effects of #503 by introducing
an additional attribute py::arithmetic() that must be specified if the
arithmetic operators are desired.
* `array_t(const object &)` now throws on error
* `array_t::ensure()` is intended for casters —- old constructor is
deprecated
* `array` and `array_t` get default constructors (empty array)
* `array` gets a converting constructor
* `py::isinstance<array_T<T>>()` checks the type (but not flags)
There is only one special thing which must remain: `array_t` gets
its own `type_caster` specialization which uses `ensure` instead
of a simple check.
* Deprecate the `py::object::str()` member function since `py::str(obj)`
is now equivalent and preferred
* Make `py::repr()` a free function
* Make sure obj.cast<T>() works as expected when T is a Python type
`obj.cast<T>()` should be the same as `T(obj)`, i.e. it should convert
the given object to a different Python type. However, `obj.cast<T>()`
usually calls `type_caster::load()` which only checks the type without
doing any actual conversion. That causes a very unexpected `cast_error`.
This commit makes it so that `obj.cast<T>()` and `T(obj)` are the same
when T is a Python type.
* Simplify pytypes converting constructor implementation
It's not necessary to maintain a full set of converting constructors
and assignment operators + const& and &&. A single converting const&
constructor will work and there is no impact on binary size. On the
other hand, the conversion functions can be significantly simplified.
Allows checking the Python types before creating an object instead of
after. For example:
```c++
auto l = list(ptr, true);
if (l.check())
// ...
```
The above is replaced with:
```c++
if (isinstance<list>(ptr)) {
auto l = reinterpret_borrow(ptr);
// ...
}
```
This deprecates `py::object::check()`. `py::isinstance()` covers the
same use case, but it can also check for user-defined types:
```c++
class Pet { ... };
py::class_<Pet>(...);
m.def("is_pet", [](py::object obj) {
return py::isinstance<Pet>(obj); // works as expected
});
```
This commit includes the following changes:
* Don't provide make_copy_constructor for non-copyable container
make_copy_constructor currently fails for various stl containers (e.g.
std::vector, std::unordered_map, std::deque, etc.) when the container's
value type (e.g. the "T" or the std::pair<K,T> for a map) is
non-copyable. This adds an override that, for types that look like
containers, also requires that the value_type be copyable.
* stl_bind.h: make bind_{vector,map} work for non-copy-constructible types
Most stl_bind modifiers require copying, so if the type isn't copy
constructible, we provide a read-only interface instead.
In practice, this means that if the type is non-copyable, it will be,
for all intents and purposes, read-only from the Python side (but
currently it simply fails to compile with such a container).
It is still possible for the caller to provide an interface manually
(by defining methods on the returned class_ object), but this isn't
something stl_bind can handle because the C++ code to construct values
is going to be highly dependent on the container value_type.
* stl_bind: copy only for arithmetic value types
For non-primitive types, we may well be copying some complex type, when
returning by reference is more appropriate. This commit returns by
internal reference for all but basic arithmetic types.
* Return by reference whenever possible
Only if we definitely can't--i.e. std::vector<bool>--because v[i]
returns something that isn't a T& do we copy; for everything else, we
return by reference.
For the map case, we can always return by reference (at least for the
default stl map/unordered_map).
When working on some particular feature, it's nice to be able to disable
all the tests except for the one I'm working on; this is currently
possible by editing tests/CMakeLists.txt, and commenting out the tests
you don't want.
This commit goes a step further by letting you give a list of tests you
do want when invoking cmake, e.g.:
cmake -DPYBIND11_TEST_OVERRIDE="test_issues.cpp;test_pickling.cpp" ..
changes the build to build just those two tests (and changes the `pytest`
target to invoke just the two associated tests).
This persists in the build directory until you disable it again by
running cmake with `-DPYBIND11_TEST_OVERRIDE=`. It also adds a message
after the pytest output to remind you that it is in effect:
Note: not all tests run: -DPYBIND11_TEST_OVERRIDE is in effect
If we need to initialize a holder around an unowned instance, and the
holder type is non-copyable (i.e. a unique_ptr), we currently construct
the holder type around the value pointer, but then never actually
destruct the holder: the holder destructor is called only for the
instance that actually has `inst->owned = true` set.
This seems no pointer, however, in creating such a holder around an
unowned instance: we never actually intend to use anything that the
unique_ptr gives us: and, in fact, do not want the unique_ptr (because
if it ever actually got destroyed, it would cause destruction of the
wrapped pointer, despite the fact that that wrapped pointer isn't
owned).
This commit changes the logic to only create a unique_ptr holder if we
actually own the instance, and to destruct via the constructed holder
whenever we have a constructed holder--which will now only be the case
for owned-unique-holder or shared-holder types.
Other changes include:
* Added test for non-movable holder constructor/destructor counts
The three alive assertions now pass, before #478 they fail with counts
of 2/2/1 respectively, because of the unique_ptr that we don't want and
don't destroy (because we don't *want* its destructor to run).
* Return cstats reference; fix ConstructStats doc
Small cleanup to the #478 test code, and fix to the ConstructStats
documentation (the static method definition should use `reference` not
`reference_internal`).
* Rename inst->constructed to inst->holder_constructed
This makes it clearer exactly what it's referring to.
* Add debugging info about so size to build output
This adds a small python script to tools that captures before-and-after
.so sizes between builds and outputs this in the build output via a
string such as:
------ pybind11_tests.cpython-35m-x86_64-linux-gnu.so file size: 924696 (decrease of 73680 bytes = 7.38%)
------ pybind11_tests.cpython-35m-x86_64-linux-gnu.so file size: 998376 (increase of 73680 bytes = 7.97%)
------ pybind11_tests.cpython-35m-x86_64-linux-gnu.so file size: 998376 (no change)
Or, if there was no .so during the build, just the .so size by itself:
------ pybind11_tests.cpython-35m-x86_64-linux-gnu.so file size: 998376
This allows you to, for example, build, checkout a different branch,
rebuild, and easily see exactly the change in the pybind11_tests.so
size.
It also allows looking at the travis and appveyor build logs to get an
idea of .so/.dll sizes across different build systems.
* Minor libsize.py script changes
- Use RAII open
- Remove unused libsize=-1
- Report change as [+-]xyz bytes = [+-]a.bc%
* Add type caster for std::experimental::optional
* Add tests for std::experimental::optional
* Support both <optional> / <experimental/optional>
* Mention std{::experimental,}::optional in the docs
* Make reference(_internal) the default return value policy for properties
Before this, all `def_property*` functions used `automatic` as their
default return value policy. This commit makes it so that:
* Non-static properties use `reference_interal` by default, thus
matching `def_readonly` and `def_readwrite`.
* Static properties use `reference` by default, thus matching
`def_readonly_static` and `def_readwrite_static`.
In case `cpp_function` is passed to any `def_property*`, its policy will
be used instead of any defaults. User-defined arguments in `extras`
still have top priority and will override both the default policies and
the ones from `cpp_function`.
Resolves#436.
* Almost always use return_value_policy::move for rvalues
For functions which return rvalues or rvalue references, the only viable
return value policies are `copy` and `move`. `reference(_internal)` and
`take_ownership` would take the address of a temporary which is always
an error.
This commit prevents possible user errors by overriding the bad rvalue
policies with `move`. Besides `move`, only `copy` is allowed, and only
if it's explicitly selected by the user.
This is also a necessary safety feature to support the new default
return value policies for properties: `reference(_internal)`.
The current integer caster was unnecessarily strict and rejected
various kinds of NumPy integer types when calling C++ functions
expecting normal integers. This relaxes the current behavior.
Currently pybind11 doesn't check when you define a new object (e.g. a
class, function, or exception) that overwrites an existing one. If the
thing being overwritten is a class, this leads to a segfault (because
pybind still thinks the type is defined, even though Python no longer
has the type). In other cases this is harmless (e.g. replacing a
function with an exception), but even in that case it's most likely a
bug.
This code doesn't prevent you from actively doing something harmful,
like deliberately overwriting a previous definition, but detects
overwriting with a run-time error if it occurs in the standard
class/function/exception/def registration interfaces.
All of the additions are in non-template code; the result is actually a
tiny decrease in .so size compared to master without the new test code
(977304 to 977272 bytes), and about 4K higher with the new tests.
With this there is no more need for manual user declarations like
`PYBIND11_DECLARE_HOLDER_TYPE(T, std::shared_ptr<T>)`. Existing ones
will still compile without error -- they will just be ignored silently.
Resolves#446.
This patch adds an extra base handle parameter to most ``py::array`` and
``py::array_t<>`` constructors. If specified along with a pointer to
data, the base object will be registered within NumPy, which increases
the base's reference count. This feature is useful to create shallow
copies of C++ or Python arrays while ensuring that the owners of the
underlying can't be garbage collected while referenced by NumPy.
The commit also adds a simple test function involving a ``wrap()``
function that creates shallow copies of various N-D arrays.
`auto var = l[0]` has a strange quirk: `var` is actually an accessor and
not an object, so any later assignment of `var = ...` would modify l[0]
instead of `var`. This is surprising compared to the non-auto assignment
`py::object var = l[0]; var = ...`.
By overloading `operator=` on lvalue/rvalue, the expected behavior is
restored even for `auto` variables.
This also adds the `hasattr` and `getattr` functions which are needed
with the new attribute behavior. The new functions behave exactly like
their Python counterparts.
Similarly `object` gets a `contains` method which calls `__contains__`,
i.e. it's the same as the `in` keyword in Python.
The custom exception handling added in PR #273 is robust, but is overly
complex for declaring the most common simple C++ -> Python exception
mapping that needs only to copy `what()`. This add a simpler
`py::register_exception<CppExp>(module, "PyExp");` function that greatly
simplifies the common basic case of translation of a simple CppException
into a simple PythonException, while not removing the more advanced
capabilities of defining custom exception handlers.
The current inheritance testing isn't sufficient to detect a cache
failure; the test added here breaks PR #390, which caches the
run-time-determined return type the first time a function is called,
then reuses that cached type even though the run-time type could be
different for a future call.
This adds a static local variable (in dead code unless actually needed)
in the overload code that is used for storage if the overload is for
some convert-by-value type (such as numeric values or std::string).
This has limitations (as written up in the advanced doc), but is better
than simply not being able to overload reference or pointer methods.
This clears the Python error at the error_already_set throw site, thus
allowing Python calls to be made in destructors which are triggered by
the exception. This is preferable to the alternative, which would be
guarding every Python API call with an error_scope.
This effectively flips the behavior of error_already_set. Previously,
it was assumed that the error stays in Python, so handling the exception
in C++ would require explicitly calling PyErr_Clear(), but nothing was
needed to propagate the error to Python. With this change, handling the
error in C++ does not require a PyErr_Clear() call, but propagating the
error to Python requires an explicit error_already_set::restore().
The change does not break old code which explicitly calls PyErr_Clear()
for cleanup, which should be the majority of user code. The need for an
explicit restore() call does break old code, but this should be mostly
confined to the library and not user code.
This commit adds support for forcing alias type initialization by
defining constructors with `py::init_alias<arg1, arg2>()` instead of
`py::init<arg1, arg2>()`. Currently py::init<> only results in Alias
initialization if the type is extended in python, or the given
arguments can't be used to construct the base type, but can be used to
construct the alias. py::init_alias<>, in contrast, always invokes the
constructor of the alias type.
It looks like this was already the intention of
`py::detail::init_alias`, which was forward-declared in
86d825f330, but was apparently never
finished: despite the existance of a .def method accepting it, the
`detail::init_alias` class isn't actually defined anywhere.
This commit completes the feature (or possibly repurposes it), allowing
declaration of classes that will always initialize the trampoline which
is (as I argued in #397) sometimes useful.
Type alias for alias classes with members didn't work properly: space
was only allocated for sizeof(type), but if we want to be able to put a
type_alias instance there, we need sizeof(type_alias), but
sizeof(type_alias) > sizeof(type) whenever type_alias has members.
The previous commit to address #392 triggers a compiler warning about
returning a reference to a local variable, which is *not* a false alarm:
the following:
py::cast<int &>(o)
(which happens internally in an overload declaration) really is
returning a reference to a local, because the cast operators for the
type_caster for numeric types returns a reference to its own member.
This commit adds a static_assert to make that a compilation failure
rather than returning a reference into about-to-be-freed memory.
Incidentally, this is also a fix for #219, which is exactly the same
issue: we can't reference numeric primitives that are cast from
wrappers around python numeric types.
This allows a slightly cleaner base type specification of:
py::class_<Type, Base>("Type")
as an alternative to
py::class_<Type>("Type", py::base<Base>())
As with the other template parameters, the order relative to the holder
or trampoline types doesn't matter.
This also includes a compile-time assertion failure if attempting to
specify more than one base class (but is easily extendible to support
multiple inheritance, someday, by updating the class_selector::set_bases
function to set multiple bases).
With this change both C++ and Python write to sys.stdout which resolves
the capture issues noted in #351. Therefore, the related workarounds are
removed.
The current pybind11::class_<Type, Holder, Trampoline> fixed template
ordering results in a requirement to repeat the Holder with its default
value (std::unique_ptr<Type>) argument, which is a little bit annoying:
it needs to be specified not because we want to override the default,
but rather because we need to specify the third argument.
This commit removes this limitation by making the class_ template take
the type name plus a parameter pack of options. It then extracts the
first valid holder type and the first subclass type for holder_type and
trampoline type_alias, respectively. (If unfound, both fall back to
their current defaults, `std::unique_ptr<type>` and `type`,
respectively). If any unmatched template arguments are provided, a
static assertion fails.
What this means is that you can specify or omit the arguments in any
order:
py::class_<A, PyA> c1(m, "A");
py::class_<B, PyB, std::shared_ptr<B>> c2(m, "B");
py::class_<C, std::shared_ptr<C>, PyB> c3(m, "C");
It also allows future class attributes (such as base types in the next
commit) to be passed as class template types rather than needing to use
a py::base<> wrapper.
The variadic handle::operator() offers the same functionality as well
as mixed positional, keyword, * and ** arguments. The tests are also
superseded by the ones in `test_callbacks`.
A Python function can be called with the syntax:
```python
foo(a1, a2, *args, ka=1, kb=2, **kwargs)
```
This commit adds support for the equivalent syntax in C++:
```c++
foo(a1, a2, *args, "ka"_a=1, "kb"_a=2, **kwargs)
```
In addition, generalized unpacking is implemented, as per PEP 448,
which allows calls with multiple * and ** unpacking:
```python
bar(*args1, 99, *args2, 101, **kwargs1, kz=200, **kwargs2)
```
and
```c++
bar(*args1, 99, *args2, 101, **kwargs1, "kz"_a=200, **kwargs2)
```
Currently pybind11 only supports std::unique_ptr<T> holders by default
(other holders can, of course, be declared using the macro). PR #368
added a `py::nodelete` that is intended to be used as:
py::class_<Type, std::unique_ptr<Type, py::nodelete>> c("Type");
but this doesn't work out of the box. (You could add an explicit
holder type declaration, but this doesn't appear to have been the
intention of the commit).
This commit fixes it by generalizing the unique_ptr type_caster to take
both the type and deleter as template arguments, so that *any*
unique_ptr instances are now automatically handled by pybind. It also
adds a test to test_smart_ptr, testing both that py::nodelete (now)
works, and that the object is indeed not deleted as intended.
Adding or removing tests is a little bit cumbersome currently: the test
needs to be added to CMakeLists.txt, the init function needs to be
predeclared in pybind11_tests.cpp, then called in the plugin
initialization. While this isn't a big deal for tests that are being
committed, it's more of a hassle when working on some new feature or
test code for which I temporarily only care about building and linking
the test being worked on rather than the entire test suite.
This commit changes tests to self-register their initialization by
having each test initialize a local object (which stores the
initialization function in a static variable). This makes changing the
set of tests being build easy: one only needs to add or comment out
test names in tests/CMakeLists.txt.
A couple other minor changes that go along with this:
- test_eigen.cpp is now included in the test list, then removed if eigen
isn't available. This lets you disable the eigen tests by commenting
it out, just like all the other tests, but keeps the build working
without eigen eigen isn't available. (Also, if it's commented out, we
don't even bother looking for and reporting the building with/without
eigen status message).
- pytest is now invoked with all the built test names (with .cpp changed
to .py) so that it doesn't try to run tests that weren't built.
Problem
=======
The template trampoline pattern documented in PR #322 has a problem with
virtual method overloads in intermediate classes in the inheritance
chain between the trampoline class and the base class.
For example, consider the following inheritance structure, where `B` is
the actual class, `PyB<B>` is the trampoline class, and `PyA<B>` is an
intermediate class adding A's methods into the trampoline:
PyB<B> -> PyA<B> -> B -> A
Suppose PyA<B> has a method `some_method()` with a PYBIND11_OVERLOAD in
it to overload the virtual `A::some_method()`. If a Python class `C` is
defined that inherits from the pybind11-registered `B` and tries to
provide an overriding `some_method()`, the PYBIND11_OVERLOADs declared
in PyA<B> fails to find this overloaded method, and thus never invoke it
(or, if pure virtual and not overridden in PyB<B>, raises an exception).
This happens because the base (internal) `PYBIND11_OVERLOAD_INT` macro
simply calls `get_overload(this, name)`; `get_overload()` then uses the
inferred type of `this` to do a type lookup in `registered_types_cpp`.
This is where it fails: `this` will be a `PyA<B> *`, but `PyA<B>` is
neither the base type (`B`) nor the trampoline type (`PyB<B>`). As a
result, the overload fails and we get a failed overload lookup.
The fix
=======
The fix is relatively simple: we can cast `this` passed to
`get_overload()` to a `const B *`, which lets get_overload look up the
correct class. Since trampoline classes should be derived from `B`
classes anyway, this cast should be perfectly safe.
This does require adding the class name as an argument to the
PYBIND11_OVERLOAD_INT macro, but leaves the public macro signatures
unchanged.
Installing something outside the project directory from a cmake
invocation is overly intrusive; this changes tests/CMakeLists.txt to
just fail with an informative message instead, and changes the
travis-ci builds to install pytest via pip or apt-get.
- ICPC can't handle the NCVirt trampoline which returns a non-copyable
type, which is likely due to a constexpr/SFINAE issue. This disables
the test on that compiler so that at least the rest can be tested.
The C++ part of the test code is modified to achieve this. As a result,
this kind of test:
```python
with capture:
kw_func1(5, y=10)
assert capture == "kw_func(x=5, y=10)"
```
can be replaced with a simple:
`assert kw_func1(5, y=10) == "x=5, y=10"`
Use simple asserts and pytest's powerful introspection to make testing
simpler. This merges the old .py/.ref file pairs into simple .py files
where the expected values are right next to the code being tested.
This commit does not touch the C++ part of the code and replicates the
Python tests exactly like the old .ref-file-based approach.
