This adds a PYBIND11_NAMESPACE macro that expands to the `pybind11`
namespace with hidden visibility under gcc-type compilers, and otherwise
to the plain `pybind11`. This then forces hidden visibility on
everything in pybind, solving the visibility issues discussed at end
end of #949.
The current `py::overload_cast` is hitting some ICEs under both MSVC
2015 and clang 3.8 on debian with the rewritten test suites; adding an
empty constexpr constructor to the `overload_cast_impl` class seems to
avoid the ICE.
Types need `tp_name` set to a C-style string, but the current `strdup`
ends up with a leak (issue #977). This avoids the strdup by storing
the `std::string` in internals so that during interpreter shutdown it
will be properly destroyed.
This commit adds a PYBIND11_UNSHARED_STATIC_LOCALS macro that forces a
function to have hidden visibility under gcc and gcc-compatible
compilers. gcc, in particular, needs this to to avoid sharing static
local variables across modules (which happens even under a RTLD_LOCAL
dlopen()!). clang doesn't appear to have this issue, but the forced
visibility on internal pybind functions certainly won't hurt it and icc.
This updates the workaround from #862 to use this rather than the
version-specific template.
The fix for #960 could result a type being registered multiple times if
its `__init__` is called multiple times. This can happen perfectly
ordinarily when python-side multiple inheritance is involved: for
example, with a diamond inheritance pattern with each intermediate
classes invoking the parent constructor.
With the change in #960, the multiple `__init__` calls meant
`register_instance` was called multiple times, but the deletion only
deleted it once. Thus, if a future instance of the same type was
allocated at the same location, pybind would pick it up as a registered
type.
This fixes the issue by tracking whether a value pointer has been
registered to avoid both double-registering it. (There's also a slight
optimization of not needing to do a registered_instances lookup when the
type is known not registered, but this is secondary).
`error_already_set` is more complicated than it needs to be, partly
because it manages reference counts itself rather than using
`py::object`, and partly because it tries to do more exception clearing
than is needed. This commit greatly simplifies it, and fixes#927.
Using `py::object` instead of `PyObject *` means we can rely on
implicit copy/move constructors.
The current logic did both a `PyErr_Clear` on deletion *and* a
`PyErr_Fetch` on creation. I can't see how the `PyErr_Clear` on
deletion is ever useful: the `Fetch` on creation itself clears the
error, so the only way doing a `PyErr_Clear` on deletion could do
anything if is some *other* exception was raised while the
`error_already_set` object was alive--but in that case, clearing some
other exception seems wrong. (Code that is worried about an exception
handler raising another exception would already catch a second
`error_already_set` from exception code).
The destructor itself called `clear()`, but `clear()` was a little bit
more paranoid that needed: it called `restore()` to restore the
currently captured error, but then immediately cleared it, using the
`PyErr_Restore` to release the references. That's unnecessary: it's
valid for us to release the references manually. This updates the code
to simply release the references on the three objects (preserving the
gil acquire).
`clear()`, however, also had the side effect of clearing the current
error, even if the current `error_already_set` didn't have a current
error (e.g. because of a previous `restore()` or `clear()` call). I
don't really see how clearing the error here can ever actually be
useful: the only way the current error could be set is if you called
`restore()` (in which case the current stored error-related members have
already been released), or if some *other* code raised the error, in
which case `clear()` on *this* object is clearing an error for which it
shouldn't be responsible.
Neither of those seem like intentional or desirable features, and
manually requesting deletion of the stored references similarly seems
pointless, so I've just made `clear()` an empty method and marked it
deprecated.
This also fixes a minor potential issue with the destruction: it is
technically possible for `value` to be null (though this seems likely to
be rare in practice); this updates the check to look at `type` which
will always be non-null for a `Fetch`ed exception.
This also adds error_already_set round-trip throw tests to the test
suite.
This adds support for implicit conversions to bool from Python types
with `__bool__` (Python 3) or `__nonzero__` (Python 2) attributes, and
adds direct (i.e. non-converting) support for numpy bools.
When defining method from a member function pointer (e.g. `.def("f",
&Derived::f)`) we run into a problem if `&Derived::f` is actually
implemented in some base class `Base` when `Base` isn't
pybind-registered.
This happens because the class type is deduced from the member function
pointer, which then becomes a lambda with first argument this deduced
type. For a base class implementation, the deduced type is `Base`, not
`Derived`, and so we generate and registered an overload which takes a
`Base *` as first argument. Trying to call this fails if `Base` isn't
registered (e.g. because it's an implementation detail class that isn't
intended to be exposed to Python) because the type caster for an
unregistered type always fails.
This commit adds a `method_adaptor` function that rebinds a member
function to a derived type member function and otherwise (i.e. regular
functions/lambda) leaves the argument as-is. This is now used for class
definitions so that they are bound with type being registered rather
than a potential base type.
A closely related fix in this commit is to similarly update the lambdas
used for `def_readwrite` (and related) to bind to the class type being
registered rather than the deduced type so that registering a property
that resolves to a base class member similarly generates a usable
function.
Fixes#854, #910.
Co-Authored-By: Dean Moldovan <dean0x7d@gmail.com>
When casting to an unsigned type from a python 2 `int`, we currently
cast using `(unsigned long long) PyLong_AsUnsignedLong(src.ptr())`.
If the Python cast fails, it returns (unsigned long) -1, but then we
cast this to `unsigned long long`, which means we get 4294967295, but
because that isn't equal to `(unsigned long long) -1`, we don't detect
the failure.
This commit moves the unsigned casting into a `detail::as_unsigned`
function which, upon error, casts -1 to the final type, and otherwise
casts the return value to the final type to avoid the problematic double
cast when an error occurs.
The error most commonly shows up wherever `long` is 32-bits (e.g. under
both 32- and 64-bit Windows, and under 32-bit linux) when passing a
negative value to a bound function taking an `unsigned long`.
Fixes#929.
The added tests also trigger a latent segfault under PyPy: when casting
to an integer smaller than `long` (e.g. casting to a `uint32_t` on a
64-bit `long` architecture) we check both for a Python error and also
that the resulting intermediate value will fit in the final type. If
there is no conversion error, but we get a value that would overflow, we
end up calling `PyErr_ExceptionMatches()` illegally: that call is only
allowed when there is a current exception. Under PyPy, this segfaults
the test suite. It doesn't appear to segfault under CPython, but the
documentation suggests that it *could* do so. The fix is to only check
for the exception match if we actually got an error.
This fixes#856. Instead of the weakref trick, the internals structure
holds an unordered_map from PyObject* to a vector of references. To
avoid the cost of the unordered_map lookup for objects that don't have
any keep_alive patients, a flag is added to each instance to indicate
whether there is anything to do.
Using `std::type_info::operator==` fails under libc++ because the .so
is loaded with RTLD_LOCAL. libc++ considers types under such .sos
distinct, and so comparing typeid() values directly isn't going to work.
This adds a custom hasher and equality class for the type lookup maps
when not under stdlibc++, and adds a `detail::same_type` function to
perform the equality test. It also converts a few pointer arguments to
const lvalue references, particularly since doing the pointer
comparison wasn't technically valid to being with (though in practice,
appeared to work everywhere).
This fixes#912.
This commit allows multiple inheritance of pybind11 classes from
Python, e.g.
class MyType(Base1, Base2):
def __init__(self):
Base1.__init__(self)
Base2.__init__(self)
where Base1 and Base2 are pybind11-exported classes.
This requires collapsing the various builtin base objects
(pybind11_object_56, ...) introduced in 2.1 into a single
pybind11_object of a fixed size; this fixed size object allocates enough
space to contain either a simple object (one base class & small* holder
instance), or a pointer to a new allocation that can contain an
arbitrary number of base classes and holders, with holder size
unrestricted.
* "small" here means having a sizeof() of at most 2 pointers, which is
enough to fit unique_ptr (sizeof is 1 ptr) and shared_ptr (sizeof is 2
ptrs).
To minimize the performance impact, this repurposes
`internals::registered_types_py` to store a vector of pybind-registered
base types. For direct-use pybind types (e.g. the `PyA` for a C++ `A`)
this is simply storing the same thing as before, but now in a vector;
for Python-side inherited types, the map lets us avoid having to do a
base class traversal as long as we've seen the class before. The
change to vector is needed for multiple inheritance: Python types
inheriting from multiple registered bases have one entry per base.
This commit also adds `doc()` to `object_api` as a shortcut for the
`attr("__doc__")` accessor.
The module macro changes from:
```c++
PYBIND11_PLUGIN(example) {
pybind11::module m("example", "pybind11 example plugin");
m.def("add", [](int a, int b) { return a + b; });
return m.ptr();
}
```
to:
```c++
PYBIND11_MODULE(example, m) {
m.doc() = "pybind11 example plugin";
m.def("add", [](int a, int b) { return a + b; });
}
```
Using the old macro results in a deprecation warning. The warning
actually points to the `pybind11_init` function (since attributes
don't bind to macros), but the message should be quite clear:
"PYBIND11_PLUGIN is deprecated, use PYBIND11_MODULE".
This extends py::vectorize to automatically pass through
non-vectorizable arguments. This removes the need for the documented
"explicitly exclude an argument" workaround.
Vectorization now applies to arithmetic, std::complex, and POD types,
passed as plain value or by const lvalue reference (previously only
pass-by-value types were supported). Non-const lvalue references and
any other types are passed through as-is.
Functions with rvalue reference arguments (whether vectorizable or not)
are explicitly prohibited: an rvalue reference is inherently not
something that can be passed multiple times and is thus unsuitable to
being in a vectorized function.
The vectorize returned value is also now more sensitive to inputs:
previously it would return by value when all inputs are of size 1; this
is now amended to having all inputs of size 1 *and* 0 dimensions. Thus
if you pass in, for example, [[1]], you get back a 1x1, 2D array, while
previously you got back just the resulting single value.
Vectorization of member function specializations is now also supported
via `py::vectorize(&Class::method)`; this required passthrough support
for the initial object pointer on the wrapping function pointer.
This allows calling of functions (typically void) over a parameter
pack, replacing usage such as:
bool unused[] = { (voidfunc(param_pack_arg), false)..., false };
(void) unused;
with a much cleaner:
PYBIND11_EXPAND_SIDE_EFFECTS(voidfunc(param_pack_arg));
This exposed a few underlying issues:
1. is_pod_struct was too strict to allow this. I've relaxed it to
require only trivially copyable and standard layout, rather than POD
(which additionally requires a trivial constructor, which std::complex
violates).
2. format_descriptor<std::complex<T>>::format() returned numpy format
strings instead of PEP3118 format strings, but register_dtype
feeds format codes of its fields to _dtype_from_pep3118. I've changed it
to return PEP3118 format codes. format_descriptor is a public type, so
this may be considered an incompatible change.
3. register_structured_dtype tried to be smart about whether to mark
fields as unaligned (with ^). However, it's examining the C++ alignment,
rather than what numpy (or possibly PEP3118) thinks the alignment should
be. For complex values those are different. I've made it mark all fields
as ^ unconditionally, which should always be safe even if they are
aligned, because we explicitly mark the padding.
The PYBIND11_CPP14 macro started out as a guard for the compile-time
path code in `descr.h`, but has since come to mean other things. This
means that while the `descr.h` check has just checked the
`PYBIND11_CPP14` macro, various other places now check `PYBIND11_CPP14
|| _MSC_VER`. This reverses that by now setting the CPP14 macro when
MSVC is trying to support C++14, but disabling the `descr.h` C++14 code
(which still fails under MSVC 2017).
The CPP17 macro also gets enabled when MSVC 2017 is compiling with
/std:c++latest (the default is /std:c++14), which enables
`std::optional` and `std::variant` support under MSVC.
GCC supports `deprecated(msg)` since v4.5 and VS supports the standard
[[deprecated(msg)]] since 2015 RTM.
The deprecated constructor change from `= default` to `{}` is
a workaround for a VS2015 bug.
This removes the convert-from-arithemtic-scalar constructor of
any_container as it can result in ambiguous calls, as in:
py::array_t<float>({ 1, 2 })
which could be intepreted as either of:
py::array_t<float>(py::array_t<float>(1, 2))
py::array_t<float>(py::detail::any_container({ 1, 2 }))
Removing the convert-from-arithmetic constructor reduces the number of
implicit conversions, avoiding the ambiguity for array and array_t.
This also re-adds the array/array_t constructors taking a scalar
argument for backwards compatibility.
Python 3's `PyInstanceMethod_Type` hides itself via its `tp_descr_get`,
which prevents aliasing methods via `cls.attr("m2") = cls.attr("m1")`:
instead the `tp_descr_get` returns a plain function, when called on a
class, or a `PyMethod`, when called on an instance. Override that
behaviour for pybind11 types with a special bypass for
`PyInstanceMethod_Types`.
I got some unexpected errors from code using `overload_cast` until I
realized that I'd configured the build with -std=c++11.
This commit adds a fake `overload_cast` class in C++11 mode that
triggers a static_assert failure indicating that C++14 is needed.
This further reduces the constructors required in buffer_info/numpy by
removing the need for the constructors that take a single size_t and
just forward it on via an initializer_list to the container-accepting
constructor.
Unfortunately, in `array` one of the constructors runs into an ambiguity
problem with the deprecated `array(handle, bool)` constructor (because
both the bool constructor and the any_container constructor involve an
implicit conversion, so neither has precedence), so a forwarding
constructor is kept there (until the deprecated constructor is
eventually removed).
This adds support for constructing `buffer_info` and `array`s using
arbitrary containers or iterator pairs instead of requiring a vector.
This is primarily needed by PR #782 (which makes strides signed to
properly support negative strides, and will likely also make shape and
itemsize to avoid mixed integer issues), but also needs to preserve
backwards compatibility with 2.1 and earlier which accepts the strides
parameter as a vector of size_t's.
Rather than adding nearly duplicate constructors for each stride-taking
constructor, it seems nicer to simply allow any type of container (or
iterator pairs). This works by replacing the existing vector arguments
with a new `detail::any_container` class that handles implicit
conversion of arbitrary containers into a vector of the desired type.
It can also be explicitly instantiated with a pair of iterators (e.g.
by passing {begin, end} instead of the container).
Upcoming changes to buffer_info make it need some things declared in
common.h; it also feels a bit misplaced in common.h (which is arguably
too large already), so move it out. (Separating this and the subsequent
changes into separate commits to make the changes easier to distinguish
from the move.)
This commit adds `error_already_set::matches()` convenience method to
check if the exception trapped by `error_already_set` matches a given
Python exception type. This will address #700 by providing a less
verbose way to check exceptions.
We now require (and enforce at compile time):
- GCC 4.8+
- clang 3.3+ (5.0+ for Apple's renumbered clang)
- MSVC 2015u3+
- ICC 15+
This also updates the versions listed in the README, and removes a
now-redundant MSVC version check.
This puts the fold expressions behind the feature macro instead of a
general C++17 macro.
It also adds a fold expression optimization to constexpr_sum (guarded
by the same feature macro).
`is_template_base_of<T>` fails when `T` is `const` (because its
implementation relies on being able to convert a `T*` to a `Base<U>*`,
which won't work when `T` is const).
(This also agrees with std::is_base_of, which ignores cv qualification.)
In order to fully satisfy Python's inheritance type layout requirements,
all types should have a common 'solid' base. A solid base is one which
has the same instance size as the derived type (not counting the space
required for the optional `dict_ptr` and `weakrefs_ptr`). Thus, `object`
does not qualify as a solid base for pybind11 types and this can lead to
issues with multiple inheritance.
To get around this, new base types are created: one per unique instance
size. There is going to be very few of these bases. They ensure Python's
MRO checks will pass when multiple bases are involved.
Instead of creating a new unique metaclass for each type, the builtin
`property` type is subclassed to support static properties. The new
setter/getters always pass types instead of instances in their `self`
argument. A metaclass is still required to support this behavior, but
it doesn't store any data anymore, so a new one doesn't need to be
created for each class. There is now only one common metaclass which
is shared by all pybind11 types.
noexcept deduction, added in PR #555, doesn't work with clang's
-std=c++1z; and while it works with g++, it isn't entirely clear to me
that it is required to work in C++17.
What should work, however, is that C++17 allows implicit conversion of a
`noexcept(true)` function pointer to a `noexcept(false)` (i.e. default,
noexcept-not-specified) function pointer. That was breaking in pybind11
because the cpp_function template used for lambdas provided a better
match (i.e. without requiring an implicit conversion), but it then
failed.
This commit takes a different approach of using SFINAE on the lambda
function to prevent it from matching a non-lambda object, which then
gets implicit conversion from a `noexcept` function pointer to a
`noexcept(false)` function pointer. This much nicer solution also gets
rid of the C++17 NOEXCEPT macros, and works in both clang and g++.
* Propagate unicode conversion failure
If returning a std::string with invalid utf-8 data, we currently fail
with an uninformative TypeError instead of propagating the
UnicodeDecodeError that Python sets on failure.
* Add support for u16/u32strings and literals
This adds support for wchar{16,32}_t character literals and the
associated std::u{16,32}string types. It also folds the
character/string conversion into a single type_caster template, since
the type casters for string and wstring were mostly the same anyway.
* Added too-long and too-big character conversion errors
With this commit, when casting to a single character, as opposed to a
C-style string, we make sure the input wasn't a multi-character string
or a single character with codepoint too large for the character type.
This also changes the character cast op to CharT instead of CharT& (we
need to be able to return a temporary decoded char value, but also
because there's little gained by bothering with an lvalue return here).
Finally it changes the char caster to 'has-a-string-caster' instead of
'is-a-string-caster' because, with the cast_op change above, there's
nothing at all gained from inheritance. This also lets us remove the
`success` from the string caster (which was only there for the char
caster) into the char caster itself. (I also renamed it to 'none' and
inverted its value to better reflect its purpose). The None -> nullptr
loading also now takes place only under a `convert = true` load pass.
Although it's unlikely that a function taking a char also has overloads
that can take a None, it seems marginally more correct to treat it as a
conversion.
This commit simplifies the size assumptions about character sizes with
static_asserts to back them up.
Fixes#656.
Before this commit, the problematic sequence was:
1. `catch (const std::exception &e)` gets a Python exception,
i.e. `error_already_set`.
2. `PyErr_SetString(PyExc_ImportError, e.what())` sets an `ImportError`.
3. `~error_already_set()` now runs, but `gil_scoped_acquire` fails due
to an unhandled `ImportError` (which was just set in step 2).
This commit adds a separate catch block for Python exceptions which just
clears the Python error state a little earlier and replaces it with an
`ImportError`, thus making sure that there is only a single Python
exception in flight at a time. (After step 2 in the sequence above,
there were effectively two Python expections set.)
* Clarify PYBIND11_NUMPY_DTYPE documentation
The current documentation and example reads as though
PYBIND11_NUMPY_DTYPE is a declarative macro along the same lines as
PYBIND11_DECLARE_HOLDER_TYPE, but it isn't. The changes the
documentation and docs example to make it clear that you need to "call"
the macro.
* Add satisfies_{all,any,none}_of<T, Preds>
`satisfies_all_of<T, Pred1, Pred2, Pred3>` is a nice legibility-enhanced
shortcut for `is_all<Pred1<T>, Pred2<T>, Pred3<T>>`.
* Give better error message for non-POD dtype attempts
If you try to use a non-POD data type, you get difficult-to-interpret
compilation errors (about ::name() not being a member of an internal
pybind11 struct, among others), for which isn't at all obvious what the
problem is.
This adds a static_assert for such cases.
It also changes the base case from an empty struct to the is_pod_struct
case by no longer using `enable_if<is_pod_struct>` but instead using a
static_assert: thus specializations avoid the base class, POD types
work, and non-POD types (and unimplemented POD types like std::array)
get a more informative static_assert failure.
* Prefix macros with PYBIND11_
numpy.h uses unprefixed macros, which seems undesirable. This prefixes
them with PYBIND11_ to match all the other macros in numpy.h (and
elsewhere).
* Add long double support
This adds long double and std::complex<long double> support for numpy
arrays.
This allows some simplification of the code used to generate format
descriptors; the new code uses fewer macros, instead putting the code as
different templated options; the template conditions end up simpler with
this because we are now supporting all basic C++ arithmetic types (and
so can use is_arithmetic instead of is_integral + multiple
different specializations).
In addition to testing that it is indeed working in the test script, it
also adds various offset and size calculations there, which
fixes the test failures under x86 compilations.
* Make 'any' the default markup role for Sphinx docs
* Automate generation of reference docs with doxygen and breathe
* Improve reference docs coverage