We currently fail at runtime when trying to call a method that is
overloaded with both static and non-static methods. This is something
python won't allow: the object is either a function or an instance, and
can't be both.
py::arg() doesn't only specify named arguments anymore, so the error
message was misleading (e.g. when using `py::arg().noconvert()` and
forgetting `py::arg()` for a second positional argument).
This adds brief API documentation for make_iterator/make_key_iterator,
specifically mentioning that it requires InputIterators.
Closes#734.
[skip ci] (no code change here)
We can't support this for classes from imported modules (which is the
primary purpose of a ctor argument base class) because we *have* to
have both parent and derived to properly extract a multiple-inheritance
base class pointer from a derived class pointer.
We could support this for actual `class_<Base, ...> instances, but since
in that case the `Base` is already present in the code, it seems more
consistent to simply always require MI to go via template options.
The `decltype(...)` in the template parameter that gives us SFINAE
matching for a lambda makes MSVC 2017 ICE; this works around if by
changing the test to an explicit not-a-function-or-pointer test, which
seems to work everywhere.
When using pybind::options to disable function signatures, user-defined
docstrings only get appended if they exist, but newlines were getting
appended unconditionally, so the docstring could end up with blank lines
(depending on which overloads, in particular, provided docstrings).
This commit suppresses the empty lines by only adding newlines for
overloads when needed.
Added in 6fb48490ef
The second constructor can't be doing anything--the signatures are
exactly the same, and so the first is always going to be the one
invoked by the dispatcher.
* The definition of `PySequence_Fast` is more restrictive on PyPy, so
use the slow path instead.
* `PyDict_Next` has been fixed in PyPy -> remove workaround.
Eigen::Ref objects, when returned, are almost always returned as
rvalues; what's important is the data they reference, not the outer
shell, and so we want to be able to use `::copy`,
`::reference_internal`, etc. to refer to the data the Eigen::Ref
references (in the following commits), rather than the Eigen::Ref
instance itself.
This moves the policy override into a struct so that code that wants to
avoid it (or wants to provide some other Return-type-conditional
override) can create a specialization of
return_value_policy_override<Return> in order to override the override.
This lets an Eigen::Ref-returning function be bound with `rvp::copy`,
for example, to specify that the data should be copied into a new numpy
array rather than referenced, or `rvp::reference_internal` to indicate
that it should be referenced, but a keep-alive used (actually, we used
the array's `base` rather than a py::keep_alive in such a case, but it
accomplishes the same thing).
With the previous commit, output can be very confusing because you only
see positional arguments in the "invoked with" line, but you can have a
failure from kwargs as well (in particular, when a value is invalidly
specified via both via positional and kwargs). This commits adds
kwargs to the output, and updates the associated tests to match.
Now that only one shared metaclass is ever allocated, it's extremely
cheap to enable it for all pybind11 types.
* Deprecate the default py::metaclass() since it's not needed anymore.
* Allow users to specify a custom metaclass via py::metaclass(handle).
In order to fully satisfy Python's inheritance type layout requirements,
all types should have a common 'solid' base. A solid base is one which
has the same instance size as the derived type (not counting the space
required for the optional `dict_ptr` and `weakrefs_ptr`). Thus, `object`
does not qualify as a solid base for pybind11 types and this can lead to
issues with multiple inheritance.
To get around this, new base types are created: one per unique instance
size. There is going to be very few of these bases. They ensure Python's
MRO checks will pass when multiple bases are involved.
Instead of creating a new unique metaclass for each type, the builtin
`property` type is subclassed to support static properties. The new
setter/getters always pass types instead of instances in their `self`
argument. A metaclass is still required to support this behavior, but
it doesn't store any data anymore, so a new one doesn't need to be
created for each class. There is now only one common metaclass which
is shared by all pybind11 types.
noexcept deduction, added in PR #555, doesn't work with clang's
-std=c++1z; and while it works with g++, it isn't entirely clear to me
that it is required to work in C++17.
What should work, however, is that C++17 allows implicit conversion of a
`noexcept(true)` function pointer to a `noexcept(false)` (i.e. default,
noexcept-not-specified) function pointer. That was breaking in pybind11
because the cpp_function template used for lambdas provided a better
match (i.e. without requiring an implicit conversion), but it then
failed.
This commit takes a different approach of using SFINAE on the lambda
function to prevent it from matching a non-lambda object, which then
gets implicit conversion from a `noexcept` function pointer to a
`noexcept(false)` function pointer. This much nicer solution also gets
rid of the C++17 NOEXCEPT macros, and works in both clang and g++.
* Avoid C-style const casts
Replace C-style casts that discard `const` with `const_cast` (and, where
necessary, `reinterpret_cast` as well).
* Warn about C-style const-discarding casts
Change pybind11_enable_warnings to also enable `-Wcast-qual` (warn if a
C-style cast discards `const`) by default. The previous commit should
have gotten rid of all of these (at least, all the ones that tripped in
my build, which included the tests), and this should discourage more
from newly appearing.
Fixes#656.
Before this commit, the problematic sequence was:
1. `catch (const std::exception &e)` gets a Python exception,
i.e. `error_already_set`.
2. `PyErr_SetString(PyExc_ImportError, e.what())` sets an `ImportError`.
3. `~error_already_set()` now runs, but `gil_scoped_acquire` fails due
to an unhandled `ImportError` (which was just set in step 2).
This commit adds a separate catch block for Python exceptions which just
clears the Python error state a little earlier and replaces it with an
`ImportError`, thus making sure that there is only a single Python
exception in flight at a time. (After step 2 in the sequence above,
there were effectively two Python expections set.)
This changes the function dispatching code for overloaded functions into
a two-pass procedure where we first try all overloads with
`convert=false` for all arguments. If no function calls succeeds in the
first pass, we then try a second pass where we allow arguments to have
`convert=true` (unless, of course, the argument was explicitly specified
with `py::arg().noconvert()`).
For non-overloaded methods, the two-pass procedure is skipped (we just
make the overload-allowed call). The second pass is also skipped if it
would result in the same thing (i.e. where all arguments are
`.noconvert()` arguments).
This adds support for controlling the `convert` flag of arguments
through the py::arg annotation. This then allows arguments to be
flagged as non-converting, which the type_caster is able to use to
request different behaviour.
Currently, AFAICS `convert` is only used for type converters of regular
pybind11-registered types; all of the other core type_casters ignore it.
We can, however, repurpose it to control internal conversion of
converters like Eigen and `array`: most usefully to give callers a way
to disable the conversion that would otherwise occur when a
`Eigen::Ref<const Eigen::Matrix>` argument is passed a numpy array that
requires conversion (either because it has an incompatible stride or the
wrong dtype).
Specifying a noconvert looks like one of these:
m.def("f1", &f, "a"_a.noconvert() = "default"); // Named, default, noconvert
m.def("f2", &f, "a"_a.noconvert()); // Named, no default, no converting
m.def("f3", &f, py::arg().noconvert()); // Unnamed, no default, no converting
(The last part--being able to declare a py::arg without a name--is new:
previous py::arg() only accepted named keyword arguments).
Such an non-convert argument is then passed `convert = false` by the
type caster when loading the argument. Whether this has an effect is up
to the type caster itself, but as mentioned above, this would be
extremely helpful for the Eigen support to give a nicer way to specify
a "no-copy" mode than the custom wrapper in the current PR, and
moreover isn't an Eigen-specific hack.
This cleans up the previous commit slightly by further reducing the
function call arguments to a single struct (containing the
function_record, arguments vector, and parent).
Although this doesn't currently change anything, it does allow for
future functionality to have a place for precalls to store temporary
objects that need to be destroyed after a function call (whether or not
the call succeeds).
As a concrete example, with this change #625 could be easily implemented
(I think) by adding a std::unique_ptr<gil_scoped_release> member to the
`function_call` struct with a precall that actually constructs it.
Without this, the precall can't do that: the postcall won't be invoked
if the call throws an exception.
This doesn't seems to affect the .so size noticeably (either way).
This commit rewrites the function dispatcher code to support mixing
regular arguments with py::args/py::kwargs arguments. It also
simplifies the argument loader noticeably as it no longer has to worry
about args/kwargs: all of that is now sorted out in the dispatcher,
which now simply appends a tuple/dict if the function takes
py::args/py::kwargs, then passes all the arguments in a vector.
When the argument loader hit a py::args or py::kwargs, it doesn't do
anything special: it just calls the appropriate type_caster just like it
does for any other argument (thus removing the previous special cases
for args/kwargs).
Switching to passing arguments in a single std::vector instead of a pair
of tuples also makes things simpler, both in the dispatch and the
argument_loader: since this argument list is strictly pybind-internal
(i.e. it never goes to Python) we have no particular reason to use a
Python tuple here.
Some (intentional) restrictions:
- you may not bind a function that has args/kwargs somewhere other than
the end (this somewhat matches Python, and keeps the dispatch code a
little cleaner by being able to not worry about where to inject the
args/kwargs in the argument list).
- If you specify an argument both positionally and via a keyword
argument, you get a TypeError alerting you to this (as you do in
Python).
* Abstract away some holder functionality (resolve#585)
Custom holder types which don't have `.get()` can select the correct
function to call by specializing `holder_traits`.
* Add support for move-only holders (fix#605)
* Make 'any' the default markup role for Sphinx docs
* Automate generation of reference docs with doxygen and breathe
* Improve reference docs coverage
* Fixed a regression that was introduced in the PyPy patch: use ht_qualname_meta instead of ht_qualname to fix PyHeapTypeObject->ht_qualname field.
* Added a qualname/repr test that works in both Python 3.3+ and previous versions
This commit includes modifications that are needed to get pybind11 to work with PyPy. The full test suite compiles and runs except for a last few functions that are commented out (due to problems in PyPy that were reported on the PyPy bugtracker).
Two somewhat intrusive changes were needed to make it possible: two new tags ``py::buffer_protocol()`` and ``py::metaclass()`` must now be specified to the ``class_`` constructor if the class uses the buffer protocol and/or requires a metaclass (e.g. for static properties).
Note that this is only for the PyPy version based on Python 2.7 for now. When the PyPy 3.x has caught up in terms of cpyext compliance, a PyPy 3.x patch will follow.
This replaces the current `all_of_t<Pred, Ts...>` with `all_of<Ts...>`,
with previous use of `all_of_t<Pred, Ts...>` becoming
`all_of<Pred<Ts>...>` (and similarly for `any_of_t`). It also adds a
`none_of<Ts...>`, a shortcut for `negation<any_of<Ts...>>`.
This allows `all_of` and `any_of` to be used a bit more flexible, e.g.
in cases where several predicates need to be tested for the same type
instead of the same predicate for multiple types.
This commit replaces the implementation with a more efficient version
for non-MSVC. For MSVC, this changes the workaround to use the
built-in, recursive std::conjunction/std::disjunction instead.
This also removes the `count_t` since `any_of_t` and `all_of_t` were the
only things using it.
This commit also rearranges some of the future std imports to use actual
`std` implementations for C++14/17 features when under the appropriate
compiler mode, as we were already doing for a few things (like
index_sequence). Most of these aren't saving much (the implementation
for enable_if_t, for example, is trivial), but I think it makes the
intention of the code instantly clear. It also enables MSVC's native
std::index_sequence support.
When compiling in C++17 mode the noexcept specifier is part of the
function type. This causes a failure in pybind11 because, by omitting
a noexcept specifier when deducing function return and argument types,
we are implicitly making `noexcept(false)` part of the type.
This means that functions with `noexcept` fail to match the function
templates in cpp_function (and other places), and we get compilation
failure (we end up trying to fit it into the lambda function version,
which fails since a function pointer has no `operator()`).
We can, however, deduce the true/false `B` in noexcept(B), so we don't
need to add a whole other set of overloads, but need to deduce the extra
argument when under C++17. That will *not* work under pre-C++17,
however.
This commit adds two macros to fix the problem: under C++17 (with the
appropriate feature macro set) they provide an extra `bool NoExceptions`
template argument and provide the `noexcept(NoExceptions)` deduced
specifier. Under pre-C++17 they expand to nothing.
This is needed to compile pybind11 with gcc7 under -std=c++17.
Since the argument loader split off from the tuple converter, it is
never called with a `convert` argument set to anything but true. This
removes the argument entirely, passing a literal `true` from within
`argument_loader` to the individual value casters.
This is needed in order to allow the tuple caster to accept any sequence
while keeping the argument loader fast. There is also very little overlap
between the two classes which makes the separation clean. It’s also good
practice not to have completely new functionality in a specialization.
Fixes#509.
The move policy was already set for rvalues in PR #473, but this only
applied to directly cast user-defined types. The problem is that STL
containers cast values indirectly and the rvalue information is lost.
Therefore the move policy was not set correctly. This commit fixes it.
This also makes an additional adjustment to remove the `copy` policy
exception: rvalues now always use the `move` policy. This is also safe
for copy-only rvalues because the `move` policy has an internal fallback
to copying.
Following commit 90d278, the object code generated by the python
bindings of nanogui (github.com/wjakob/nanogui) went up by a whopping
12%. It turns out that that project has quite a few enums where we don't
really care about arithmetic operators.
This commit thus partially reverts the effects of #503 by introducing
an additional attribute py::arithmetic() that must be specified if the
arithmetic operators are desired.
The pytype converting constructors are convenient and safe for user
code, but for library internals the additional type checks and possible
conversions are sometimes not desired. `reinterpret_borrow<T>()` and
`reinterpret_steal<T>()` serve as the low-level unsafe counterparts
of `cast<T>()`.
This deprecates the `object(handle, bool)` constructor.
Renamed `borrowed` parameter to `is_borrowed` to avoid shadowing
warnings on MSVC.
* Deprecate the `py::object::str()` member function since `py::str(obj)`
is now equivalent and preferred
* Make `py::repr()` a free function
* Make sure obj.cast<T>() works as expected when T is a Python type
`obj.cast<T>()` should be the same as `T(obj)`, i.e. it should convert
the given object to a different Python type. However, `obj.cast<T>()`
usually calls `type_caster::load()` which only checks the type without
doing any actual conversion. That causes a very unexpected `cast_error`.
This commit makes it so that `obj.cast<T>()` and `T(obj)` are the same
when T is a Python type.
* Simplify pytypes converting constructor implementation
It's not necessary to maintain a full set of converting constructors
and assignment operators + const& and &&. A single converting const&
constructor will work and there is no impact on binary size. On the
other hand, the conversion functions can be significantly simplified.
If we need to initialize a holder around an unowned instance, and the
holder type is non-copyable (i.e. a unique_ptr), we currently construct
the holder type around the value pointer, but then never actually
destruct the holder: the holder destructor is called only for the
instance that actually has `inst->owned = true` set.
This seems no pointer, however, in creating such a holder around an
unowned instance: we never actually intend to use anything that the
unique_ptr gives us: and, in fact, do not want the unique_ptr (because
if it ever actually got destroyed, it would cause destruction of the
wrapped pointer, despite the fact that that wrapped pointer isn't
owned).
This commit changes the logic to only create a unique_ptr holder if we
actually own the instance, and to destruct via the constructed holder
whenever we have a constructed holder--which will now only be the case
for owned-unique-holder or shared-holder types.
Other changes include:
* Added test for non-movable holder constructor/destructor counts
The three alive assertions now pass, before #478 they fail with counts
of 2/2/1 respectively, because of the unique_ptr that we don't want and
don't destroy (because we don't *want* its destructor to run).
* Return cstats reference; fix ConstructStats doc
Small cleanup to the #478 test code, and fix to the ConstructStats
documentation (the static method definition should use `reference` not
`reference_internal`).
* Rename inst->constructed to inst->holder_constructed
This makes it clearer exactly what it's referring to.
* Make reference(_internal) the default return value policy for properties
Before this, all `def_property*` functions used `automatic` as their
default return value policy. This commit makes it so that:
* Non-static properties use `reference_interal` by default, thus
matching `def_readonly` and `def_readwrite`.
* Static properties use `reference` by default, thus matching
`def_readonly_static` and `def_readwrite_static`.
In case `cpp_function` is passed to any `def_property*`, its policy will
be used instead of any defaults. User-defined arguments in `extras`
still have top priority and will override both the default policies and
the ones from `cpp_function`.
Resolves#436.
* Almost always use return_value_policy::move for rvalues
For functions which return rvalues or rvalue references, the only viable
return value policies are `copy` and `move`. `reference(_internal)` and
`take_ownership` would take the address of a temporary which is always
an error.
This commit prevents possible user errors by overriding the bad rvalue
policies with `move`. Besides `move`, only `copy` is allowed, and only
if it's explicitly selected by the user.
This is also a necessary safety feature to support the new default
return value policies for properties: `reference(_internal)`.
Currently pybind11 doesn't check when you define a new object (e.g. a
class, function, or exception) that overwrites an existing one. If the
thing being overwritten is a class, this leads to a segfault (because
pybind still thinks the type is defined, even though Python no longer
has the type). In other cases this is harmless (e.g. replacing a
function with an exception), but even in that case it's most likely a
bug.
This code doesn't prevent you from actively doing something harmful,
like deliberately overwriting a previous definition, but detects
overwriting with a run-time error if it occurs in the standard
class/function/exception/def registration interfaces.
All of the additions are in non-template code; the result is actually a
tiny decrease in .so size compared to master without the new test code
(977304 to 977272 bytes), and about 4K higher with the new tests.
We have various classes that have non-explicit constructors that accept
a single argument, which is implicitly making them implicitly
convertible from the argument. In a few cases, this is desirable (e.g.
implicit conversion of std::string to py::str, or conversion of double
to py::float_); in many others, however, it is unintended (e.g. implicit
conversion of size_t to some pre-declared py::array_t<T> type).
This disables most of the unwanted implicit conversions by marking them
`explicit`, and comments the ones that are deliberately left implicit.
`auto var = l[0]` has a strange quirk: `var` is actually an accessor and
not an object, so any later assignment of `var = ...` would modify l[0]
instead of `var`. This is surprising compared to the non-auto assignment
`py::object var = l[0]; var = ...`.
By overloading `operator=` on lvalue/rvalue, the expected behavior is
restored even for `auto` variables.
This also adds the `hasattr` and `getattr` functions which are needed
with the new attribute behavior. The new functions behave exactly like
their Python counterparts.
Similarly `object` gets a `contains` method which calls `__contains__`,
i.e. it's the same as the `in` keyword in Python.
The custom exception handling added in PR #273 is robust, but is overly
complex for declaring the most common simple C++ -> Python exception
mapping that needs only to copy `what()`. This add a simpler
`py::register_exception<CppExp>(module, "PyExp");` function that greatly
simplifies the common basic case of translation of a simple CppException
into a simple PythonException, while not removing the more advanced
capabilities of defining custom exception handlers.
This adds a static local variable (in dead code unless actually needed)
in the overload code that is used for storage if the overload is for
some convert-by-value type (such as numeric values or std::string).
This has limitations (as written up in the advanced doc), but is better
than simply not being able to overload reference or pointer methods.
This clears the Python error at the error_already_set throw site, thus
allowing Python calls to be made in destructors which are triggered by
the exception. This is preferable to the alternative, which would be
guarding every Python API call with an error_scope.
This effectively flips the behavior of error_already_set. Previously,
it was assumed that the error stays in Python, so handling the exception
in C++ would require explicitly calling PyErr_Clear(), but nothing was
needed to propagate the error to Python. With this change, handling the
error in C++ does not require a PyErr_Clear() call, but propagating the
error to Python requires an explicit error_already_set::restore().
The change does not break old code which explicitly calls PyErr_Clear()
for cleanup, which should be the majority of user code. The need for an
explicit restore() call does break old code, but this should be mostly
confined to the library and not user code.
This commit adds support for forcing alias type initialization by
defining constructors with `py::init_alias<arg1, arg2>()` instead of
`py::init<arg1, arg2>()`. Currently py::init<> only results in Alias
initialization if the type is extended in python, or the given
arguments can't be used to construct the base type, but can be used to
construct the alias. py::init_alias<>, in contrast, always invokes the
constructor of the alias type.
It looks like this was already the intention of
`py::detail::init_alias`, which was forward-declared in
86d825f330, but was apparently never
finished: despite the existance of a .def method accepting it, the
`detail::init_alias` class isn't actually defined anywhere.
This commit completes the feature (or possibly repurposes it), allowing
declaration of classes that will always initialize the trampoline which
is (as I argued in #397) sometimes useful.
Type alias for alias classes with members didn't work properly: space
was only allocated for sizeof(type), but if we want to be able to put a
type_alias instance there, we need sizeof(type_alias), but
sizeof(type_alias) > sizeof(type) whenever type_alias has members.
This allows a slightly cleaner base type specification of:
py::class_<Type, Base>("Type")
as an alternative to
py::class_<Type>("Type", py::base<Base>())
As with the other template parameters, the order relative to the holder
or trampoline types doesn't matter.
This also includes a compile-time assertion failure if attempting to
specify more than one base class (but is easily extendible to support
multiple inheritance, someday, by updating the class_selector::set_bases
function to set multiple bases).
The current pybind11::class_<Type, Holder, Trampoline> fixed template
ordering results in a requirement to repeat the Holder with its default
value (std::unique_ptr<Type>) argument, which is a little bit annoying:
it needs to be specified not because we want to override the default,
but rather because we need to specify the third argument.
This commit removes this limitation by making the class_ template take
the type name plus a parameter pack of options. It then extracts the
first valid holder type and the first subclass type for holder_type and
trampoline type_alias, respectively. (If unfound, both fall back to
their current defaults, `std::unique_ptr<type>` and `type`,
respectively). If any unmatched template arguments are provided, a
static assertion fails.
What this means is that you can specify or omit the arguments in any
order:
py::class_<A, PyA> c1(m, "A");
py::class_<B, PyB, std::shared_ptr<B>> c2(m, "B");
py::class_<C, std::shared_ptr<C>, PyB> c3(m, "C");
It also allows future class attributes (such as base types in the next
commit) to be passed as class template types rather than needing to use
a py::base<> wrapper.
Problem
=======
The template trampoline pattern documented in PR #322 has a problem with
virtual method overloads in intermediate classes in the inheritance
chain between the trampoline class and the base class.
For example, consider the following inheritance structure, where `B` is
the actual class, `PyB<B>` is the trampoline class, and `PyA<B>` is an
intermediate class adding A's methods into the trampoline:
PyB<B> -> PyA<B> -> B -> A
Suppose PyA<B> has a method `some_method()` with a PYBIND11_OVERLOAD in
it to overload the virtual `A::some_method()`. If a Python class `C` is
defined that inherits from the pybind11-registered `B` and tries to
provide an overriding `some_method()`, the PYBIND11_OVERLOADs declared
in PyA<B> fails to find this overloaded method, and thus never invoke it
(or, if pure virtual and not overridden in PyB<B>, raises an exception).
This happens because the base (internal) `PYBIND11_OVERLOAD_INT` macro
simply calls `get_overload(this, name)`; `get_overload()` then uses the
inferred type of `this` to do a type lookup in `registered_types_cpp`.
This is where it fails: `this` will be a `PyA<B> *`, but `PyA<B>` is
neither the base type (`B`) nor the trampoline type (`PyB<B>`). As a
result, the overload fails and we get a failed overload lookup.
The fix
=======
The fix is relatively simple: we can cast `this` passed to
`get_overload()` to a `const B *`, which lets get_overload look up the
correct class. Since trampoline classes should be derived from `B`
classes anyway, this cast should be perfectly safe.
This does require adding the class name as an argument to the
PYBIND11_OVERLOAD_INT macro, but leaves the public macro signatures
unchanged.
- ICPC can't handle the NCVirt trampoline which returns a non-copyable
type, which is likely due to a constexpr/SFINAE issue. This disables
the test on that compiler so that at least the rest can be tested.
