* Avoid thread termination in scoped_released
Do not call `PyEval_RestoreThread()` from `~gil_scoped_release()` if python runtime is finalizing, as it will result in thread termination in Python runtime newer than 3.6, as documented in https://docs.python.org/3/c-api/init.html#c.PyEval_RestoreThread
Similarly do not call `PyThreadState_DeleteCurrent` from `~gil_scoped_acquire()` if runtime is finalizing.
Discovered while debugging PyTorch crash using Python-3.9 described in https://github.com/pytorch/pytorch/issues/47776
* Simplify _Py_IsFinalizing() availability check
* Fix typo
* Add version agnostic `detail::finalization_guard()`
* Move `finalization_guard` to detail/common.h
And rename it to `is_finalizing`
* Move `is_finalizing()` back to pybind11.h
* Simplify `is_finalizing()` check
One should follow documentation rather than make any assumptions
* feat: disarm
* docs: fix comment
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
* Allow type_caster of std::reference_wrapper<T> to be the same as a native reference.
Before, both std::reference_wrapper<T> and std::reference_wrapper<const T> would
invoke cast_op<type>. This doesn't allow the type_caster<> specialization for T
to distinguish reference_wrapper types from value types.
After, the type_caster<> specialization invokes cast_op<type&>, which allows
reference_wrapper to behave in the same way as a native reference type.
* Add tests/examples for std::reference_wrapper<const T>
* Add tests which use mutable/immutable variants
This test is a chimera; it blends the pybind11 casters with a custom
pytype implementation that supports immutable and mutable calls.
In order to detect the immutable/mutable state, the cast_op needs
to propagate it, even through e.g. std::reference<const T>
Note: This is still a work in progress; some things are crashing,
which likely means that I have a refcounting bug or something else
missing.
* Add/finish tests that distinguish const& from &
Fixes the bugs in my custom python type implementation,
demonstrate test that requires const& and reference_wrapper<const T>
being treated differently from Non-const.
* Add passing a const to non-const method.
* Demonstrate non-const conversion of reference_wrapper in tests.
Apply formatting presubmit check.
* Fix build errors from presubmit checks.
* Try and fix a few more CI errors
* More CI fixes.
* More CI fixups.
* Try and get PyPy to work.
* Additional minor fixups. Getting close to CI green.
* More ci fixes?
* fix clang-tidy warnings from presubmit
* fix more clang-tidy warnings
* minor comment and consistency cleanups
* PyDECREF -> Py_DECREF
* copy/move constructors
* Resolve codereview comments
* more review comment fixes
* review comments: remove spurious &
* Make the test fail even when the static_assert is commented out.
This expands the test_freezable_type_caster a bit by:
1/ adding accessors .is_immutable and .addr to compare identity
from python.
2/ Changing the default cast_op of the type_caster<> specialization
to return a non-const value. In normal codepaths this is a reasonable
default.
3/ adding roundtrip variants to exercise the by reference, by pointer
and by reference_wrapper in all call paths. In conjunction with 2/, this
demonstrates the failure case of the existing std::reference_wrpper conversion,
which now loses const in a similar way that happens when using the default cast_op_type<>.
* apply presubmit formatting
* Revert inclusion of test_freezable_type_caster
There's some concern that this test is a bit unwieldly because of the use
of the raw <Python.h> functions. Removing for now.
* Add a test that validates const references propagation.
This test verifies that cast_op may be used to correctly detect
const reference types when used with std::reference_wrapper.
* mend
* Review comments based changes.
1. std::add_lvalue_reference<type> -> type&
2. Simplify the test a little more; we're never returning the ConstRefCaster
type so the class_ definition can be removed.
* formatted files again.
* Move const_ref_caster test to builtin_casters
* Review comments: use cast_op and adjust some comments.
* Simplify ConstRefCasted test
I like this version better as it moves the assertion that matters
back into python.
This line had two bugs:
1. It declares `py::detail::class_` as `friend`.
2. After fixing that, we would have to change it to
`template <typename, typename...>`
The first one was introduced ~5 years ago, when a large refactoring was
made, probably as an intermediate step during refactoring.
The second was made when `generic_type` was made to be agnostic with
respect to the order of `py::class_` template parameters.
We're removing the declaration altogether, because it was never relied
on. This is what makes me think that it was an intermediate step in
refactoring that shouldn't have ended up in commit history.
* demo kwarg with none(false)
* Reorder and extend tests for arg::none(false) in test_methods_and_attributes.py::test_accepts_none
* Fix arg::none() for keyword arguments
* Add changelog note
* Fix names of no_none_kw test functions
Co-authored-by: Yannick Jadoul <yannick.jadoul@belgacom.net>
* Demonstrate test_factory_constructors.py failure without functional changes from #2335
* Revert "Demonstrate test_factory_constructors.py failure without functional changes from #2335"
This reverts commit ca33a8021fc2a3617c3356b188796528f4594419.
* Fix test crash where registered Python type gets garbage collected
* Clean up some more internal structures when class objects go out of scope
* Reduce length of std::erase_if-in-C++20 comment
* Clean up code for cleaning up type internals
* Move cleaning up of type info in internals to tp_dealloc on pybind11_metaclass
* add support for LLVM compiler
* Update include/pybind11/pybind11.h
__forced_unwind libc++ vs libstdc++
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
* Deprecated public constructors of module
* Turn documentation comment of module_::add_object into valid doxygen documentation
* Move definition of PYBIND11_DETAIL_MODULE_STATIC_DEF and PYBIND11_DETAIL_MODULE_CREATE macros up
* Move detail::create_top_level_module to module_::create_extension_module, and unify Python 2 and 3 signature again
* Throw error_already_set if module creation fails in module_::create_extension_module
* Mention module_::create_extension_module in deprecation warning message of module_::module_
* Check scope's __dict__ instead of using hasattr when registering classes and exceptions, to allow registering the same name in a derived class scope
* Extend test_base_and_derived_nested_scope test
* Add tests on error being thrown registering duplicate classes
* Circumvent bug with combination of test_class.py::test_register_duplicate_class and test_factory_constructors.py::test_init_factory_alias
* fix: PYBIND11_OBJECT could only be used inside the pybind11 namespace (regression)
* docs: add changelog for conversion protection change
* ci: update to Python 3.9
* Fail on passing py::object with wrong Python type to py::object subclass using PYBIND11_OBJECT macro
* Split off test_non_converting_constructors from test_constructors
* Fix test_as_type, as py::type constructor now throws an error itself if the argument is not a type
* Replace tp_name access by pybind11::detail::get_fully_qualified_tp_name
* Move forward-declaration of get_fully_qualified_tp_name to detail/common.h
* Don't add the builtins module name in get_fully_qualified_tp_name for PyPy
* Add PYBIND11_BUILTINS_MODULE macro, and use it in get_fully_qualified_tp_name
* Remove code inside 'PYPY_VERSION_NUM < 0x06000000' preprocessor if branch
* fix: more cleanup
* Remove more references to PyPy 5.7 and 5.9 in the docs
* Update comment on PyUnicode_UTF* in PyPy
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
* tests: New test for ctypes buffers (pybind#2502)
* fix: fix buffer_info segfault on views with no stride (pybind11#2502)
* Explicit conversions in buffer_info to make clang happy (pybind#2502)
* Another explicit cast in buffer_info constructor for clang (pybind#2502)
* Simpler implementation of buffer_info constructor from Py_buffer.
* Move test_ctypes_buffer into test_buffers
* Comment on why view->strides may be NULL (and fix some whitespace)
* Use c_strides() instead of zero when view->strides is NULL.
c_strides and f_strides are moved from numpy.h (py::array)
to buffer_info.h (py::detail) so they can be used from the
buffer_info Py_buffer constructor.
* Increase ctypes buffer test coverage in test_buffers.
* Split ctypes tests and skip one which is broken in PyPy2.
* Allow function/functor passed to py::vectorize to return void
* Stealing @sizmailov's test and fixing unused argument warning
* Add missing std::move()
RVO doesn't work here because function return type is different from
actual returned type
* remove extra EOL
* docs: add a few details
* chore: pre-commit autoupdate
* Remove array_iterator, array_begin, and array_end (in detail namespace)
Co-authored-by: Sergei Izmailov <sergei.a.izmailov@gmail.com>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
* Add tests demonstrating the problem with deregistering pybind11 instances
* Fix deregistering of different pybind11 instance from internals
Co-authored-by: Yannick Jadoul <yannick.jadoul@belgacom.net>
Co-authored-by: Blistic <wots_wot@hotmail.com>
* Initializing PyModuleDef object with PyModuleDef_HEAD_INIT.
Python 3.8 documentation: m_base - Always initialize this member to PyModuleDef_HEAD_INIT.
Long-standing (since first github commit in 2015), inconsequential bug.
Also removing inconsequential Py_INCREF(def): PyModule_Create() resets the reference count to 1.
* git rebase master
* moving static PyModuleDef declaration to global scope, as requested by @wjakob
* renaming the two new macros, to start with PYBIND11_DETAIL_MODULE
This changes enum reprs to look like `<Enum.name: value>` similarly to
the Python enum module.
This keeps the str of enums as `Enum.name`, like the Python enum module.
Support C++20. For backwards compatibility, we provide an alias for the old name.
This change is necessary to easily avoid errors when a compiler thinks
`module` is used as a keyword.
* feat: setup.py redesign and helpers
* refactor: simpler design with two outputs
* refactor: helper file update and Windows support
* fix: review points from @YannickJadoul
* refactor: fixes to naming and more docs
* feat: more customization points
* feat: add entry point pybind11-config
* refactor: Try Extension-focused method
* refactor: rename alt/inplace to global
* fix: allow usage with git modules, better docs
* feat: global as an extra (@YannickJadoul's suggestion)
* feat: single version location
* fix: remove the requirement that setuptools must be imported first
* fix: some review points from @wjacob
* fix: use .in, add procedure to docs
* refactor: avoid monkeypatch copy
* docs: minor typos corrected
* fix: minor points from @YannickJadoul
* fix: typo on Windows C++ mode
* fix: MSVC 15 update 3+ have c++14 flag
See <https://docs.microsoft.com/en-us/cpp/build/reference/std-specify-language-standard-version?view=vs-2019>
* docs: discuss making SDists by hand
* ci: use pep517.build instead of manual setup.py
* refactor: more comments from @YannickJadoul
* docs: updates from @ktbarrett
* fix: change to newly recommended tool instead of pep517.build
This was intended as a proof of concept; build seems to be the correct replacement.
See https://github.com/pypa/pep517/pull/83
* docs: updates from @wjakob
* refactor: dual version locations
* docs: typo spotted by @wjakob
* Change base parameter type in register_exception and excepion constructor from PyObject* to handle
* Fix compilation error passing `handle` to `PyObject*`
* Wrap PYBIND11_OVERLOAD_NAME and PYBIND11_OVERLOAD_PURE_NAME in do { ... } while (false), and resolve trailing semicolon
* Deprecate PYBIND11_OVERLOAD_* and get_overload in favor of PYBIND11_OVERRIDE_* and get_override
* Correct erroneous usage of 'overload' instead of 'override' in the implementation and internals
* Fix tests to use non-deprecated PYBIND11_OVERRIDE_* macros
* Update docs to use override instead of overload where appropriate, and add warning about deprecated aliases
* Add semicolons to deprecated PYBIND11_OVERLOAD macros to match original behavior
* Remove deprecation of PYBIND11_OVERLOAD_* macros and get_overload
* Add note to changelog and upgrade guide
* feat: type<T>()
* refactor: using py::type as class
* refactor: py::object as base
* wip: tigher api
* refactor: fix conversion and limit API further
* docs: some added notes from @EricCousineau-TRI
* refactor: use py::type::of
* Added guards to the includes
Added new CI config
Added new trigger
Changed CI workflow name
Debug CI
Debug CI
Debug CI
Debug CI
Added flags fro PGI
Disable Eigen
Removed tests that fail
Uncomment lines
* fix: missing include
fix: minor style cleanup
tests: support skipping
ci: remove and tighten a bit
fix: try msvc workaround for pgic
* tests: split up prealoc tests
* fix: PGI compiler fix
* fix: PGI void_t only
* fix: try to appease nvcc
* ci: better ordering for slow tests
* ci: minor improvements to testing
* ci: Add NumPy to testing
* ci: Eigen generates CUDA warnings / PGI errors
* Added CentOS7 back for a moment
* Fix YAML
* ci: runs-on missing
* centos7 is missing pytest
* ci: use C++11 on CentOS 7
* ci: test something else
* Try just adding flags on CentOS 7
* fix: CentOS 7
* refactor: move include to shared location
* Added verbose flag
* Try to use system cmake3 on CI
* Try to use system cmake3 on CI, attempt2
* Try to use system cmake3 on CI, attempt3
* tests: not finding pytest should be a warning, not a fatal error
* tests: cleanup
* Weird issue?
* fix: final polish
Co-authored-by: Andrii Verbytskyi <andrii.verbytskyi@mpp.mpg.de>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Andrii Verbytskyi <averbyts@cern.ch>
* Add check if `str(handle)` correctly converted the object, and throw py::error_already_set if not
* Fix tests on Python 3
* Apply @rwgk's fixes to cherry-picked commits from #2392
The variables PYBIND11_HAS_OPTIONAL, PYBIND11_HAS_EXP_OPTIONAL, PYBIND11_HAS_VARIANT,
__clang__, __APPLE__ were not checked for defined in a minortity of instances.
If the project using pybind11 sets -Wundef, the warnings will show.
The test build is also modified to catch the problem.
* fix: support nvcc and test
* fixup! fix: support nvcc and test
* docs: mention what compilers fail
* fix: much simpler logic
* refactor: slightly faster / clearer
Adding missing `bytes` type to `test_constructors()`, to exercise the code change.
The changes in the PR were cherry-picked from PR #2409 (with a very minor
modification in test_pytypes.py related to flake8). Via PR #2409, these
changes were extensively tested in the Google environment, as summarized here:
https://docs.google.com/document/d/1TPL-J__mph_yHa1quDvsO12E_F5OZnvBaZlW9IIrz8M/
The changes in this PR did not cause an issues at all.
Note that `test_constructors()` before this PR passes for Python 2 only
because `pybind11::str` can hold `PyUnicodeObject` or `PyBytesObject`. As a
side-effect of this PR, `test_constructors()` no longer relies on this
permissive `pybind11::str` behavior. However, the permissive behavior is still
exercised/exposed via the existing `test_pybind11_str_raw_str()`.
The test code change is designed to enable easy removal later, when Python 2
support is dropped.
For completeness: confusingly, the non-test code changes travelled through PR
Example `ambiguous conversion` error fixed by this PR:
```
pybind11/tests/test_pytypes.cpp:214:23: error: ambiguous conversion for functional-style cast from 'pybind11::detail::item_accessor' (aka 'accessor<accessor_policies::generic_item>') to 'py::bytes'
"bytes"_a=py::bytes(d["bytes"]),
^~~~~~~~~~~~~~~~~~~~
pybind11/include/pybind11/detail/../pytypes.h:957:21: note: candidate constructor
PYBIND11_OBJECT(bytes, object, PYBIND11_BYTES_CHECK)
^
pybind11/include/pybind11/detail/../pytypes.h:957:21: note: candidate constructor
pybind11/include/pybind11/detail/../pytypes.h:987:15: note: candidate constructor
inline bytes::bytes(const pybind11::str &s) {
^
1 error generated.
```
To deal with exceptions that hit destructors or other noexcept functions.
Includes fixes to support Python 2.7 and extends documentation on
error handling.
@virtuald and @YannickJadoul both contributed to this PR.
* Fix warning C26817: Potentially expensive copy of variable 'vh' in range-for loop. Consider making it a const reference (es.71).
* Replace another instance of `for (auto vh : values_and_holders(...))` with `auto vh &` (found by @bstaletic)
Co-authored-by: Michael Goulding <Michael.Goulding@microsoft.com>
Co-authored-by: Yannick Jadoul <yannick.jadoul@belgacom.net>
* Modified Vector STL bind initialization from a buffer type with optimization for simple arrays
* Add subtests to demonstrate processing Python buffer protocol objects with step > 1
* Fixed memoryview step test to only run on Python 3+
* Modified Vector constructor from buffer to return by value for readability
If the default argument value is a class, and not an instance of a
class, `a.value.attr("__repr__")` raises a `ValueError`. Switching to
`repr(a.value)` makes this use case work.
Fixes#2028
* Fix undefined memoryview format
* Add missing <algorithm> header
* Add workaround for py27 array compatibility
* Workaround py27 memoryview behavior
* Fix memoryview constructor from buffer_info
* Workaround PyMemoryView_FromMemory availability in py27
* Fix up memoryview tests
* Update memoryview test from buffer to check signedness
* Use static factory method to create memoryview
* Remove ndim arg from memoryview::frombuffer and add tests
* Allow ndim=0 memoryview and documentation fixup
* Use void* to align to frombuffer method signature
* Add const variants of frombuffer and frommemory
* Add memory view section in doc
* Fix docs
* Add test for null buffer
* Workaround py27 nullptr behavior in test
* Rename frombuffer to from_buffer
This avoids a potential conflict with names in the same scope of the
same name as the embedded module, like namespaces or other global
variables.
Fixes#2172
* Change NAMESPACE_BEGIN and NAMESPACE_END macros into PYBIND11_NAMESPACE_BEGIN and PYBIND11_NAMESPACE_END
* Fix sudden HomeBrew 'python not installed' error
* Sweep difference in 'Class.__init__() must be called when overriding __init__' error message between CPython and PyPy under the rug
* Homebrew updated to 3.8 yesterday.
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
* Error out eval_file
* Enable dynamic attribute support for Pypy >= 6
* Add a test for dynamic attribute support
* Skip test for eval_file on pypy
* Workaround for __qualname__ on PyPy3
* Add a PyPy3.6 7.3.0 build
* Only disable in PyPy3
* Fix travis testing
* No numpy and scipy for pypy
* Enable test on pypy2
* Fix logic in eval_file
* Skip a few tests due to bugs in PyPy
* scipy wheels are broken. make pypy2 a failrue
Co-authored-by: Andreas Kloeckner <inform@tiker.net>
The PyEval_InitThreads() and PyEval_ThreadsInitialized() functions are
now deprecated and will be removed in Python 3.11. Calling
PyEval_InitThreads() now does nothing. The GIL is initialized by
Py_Initialize() since Python 3.7.
For rationale, see #2241, eeb1044818af5b70761deae602c49eba439164dc;
there is a second entry point function defined by the PYBIND11_MODULE
macro that also needs to be annotated as unused.
This change defines a new, portable macro PYBIND11_MAYBE_UNUSED to
mark declarations as unused, and annotates the PYBIND11_MODULE entry
point with this attribute.
The purpose of this annotation is to facilitate dead code detection,
which might otherwise consider the module entry point function dead,
since it isn't otherwise used. (It is only used via FFI.)
This adds support for a `py::args_kw_only()` annotation that can be
specified between `py::arg` annotations to indicate that any following
arguments are keyword-only. This allows you to write:
m.def("f", [](int a, int b) { /* ... */ },
py::arg("a"), py::args_kw_only(), py::arg("b"));
and have it work like Python 3's:
def f(a, *, b):
# ...
with respect to how `a` and `b` arguments are accepted (that is, `a` can
be positional or by keyword; `b` can only be specified by keyword).
Currently user specializations of the form
template <typename itype> struct polymorphic_type_hook<itype, std::enable_if_t<...>> { ... };
will fail if itype is also polymorphic, because the existing specialization will also
be enabled, which leads to 2 equally viable candidates. With this change, user provided
specializations have higher priority than the built in specialization for polymorphic types.
When binding code immediately throws an exception of type
py::error_already_set (e.g. via py::module::import that fails), the
catch block sets an import error as expected. Unfortunately, following
this, the deconstructor of py::error_already_set decides to call
py::detail::get_internals() and set up various internal data structures
of pybind11, which fails given that the error flag is active. The call
stack of this looks as follows:
Py_init_mymodule() -> __cxa_decrement_exception_refcount ->
error_already_set::~error_already_set() ->
gil_scoped_acquire::gil_scoped_acquire() -> detail::get_internals() ->
... -> pybind11::detail::simple_collector() -> uh oh..
The solution is simple: we call detail::get_internals() once before
running any binding code to make sure that the internal data structures
are ready.
This commit introduces the use of C++17-style fold expressions when
casting tuples & the argument lists of functions.
This change can improve performance of the resulting bindings: because
fold expressions have short-circuiting semantics, pybind11 e.g. won't
try to cast the second argument of a function if the first one failed.
This is particularly effective when working with functions that have
many overloads with long argument lists.
* test pair-copyability on C++17 upwards
The stdlib falsely detects containers like M=std::map<T, U>
as copyable, even when one of T and U is not copyable.
Therefore we cannot rely on the stdlib dismissing std::pair<T, M>
by itself, even on C++17.
* fix is_copy_assignable
bind_map used std::is_copy_assignable which suffers from the same problems
as std::is_copy_constructible, therefore the same fix has been applied.
* created tests for copyability
Don't assume that just because the language version is C++17 that the
standard library offers all C++17 features, too. When using clang-6.0
and --std=c++17 on Ubuntu 18.04 with libstdc++, __cpp_sized_deallocation
is false.
When building with `-Werror,-Wmissing-prototypes`, `clang` complains about missing prototypes for functions defined through macro expansions. This PR adds the missing prototypes.
```
error: no previous prototype for function 'pybind11_init_impl_embedded' [
-Werror,-Wmissing-prototypes]
PYBIND11_EMBEDDED_MODULE(embedded, mod) {
^
external/pybind11/include/pybind11/embed.h:61:5: note: expanded from macro 'PYBIND11_EMBEDDED_MODULE'
PYBIND11_EMBEDDED_MODULE_IMPL(name) \
^
external/pybind11/include/pybind11/embed.h:26:23: note: expanded from macro 'PYBIND11_EMBEDDED_MODULE_IMPL'
extern "C" void pybind11_init_impl_##name() { \
^
<scratch space>:380:1: note: expanded from here
pybind11_init_impl_embedded
^
1 error generated.
```
* Adapt to python3.8 C API change
Do `Py_DECREF(type)` on all python objects on deallocation
fix#1946
* Add bare python3.8 build to CI matrix
While numpy/scipy wheels are available, run python3.8 test without them
* fix: Avoid conversion to `int_` rhs argument of enum eq/ne
* test: compare unscoped enum with strings
* suppress comparison to None warning
* test unscoped enum arithmetic and comparision with unsupported type
* Make `overload_cast_impl` available in C++11 mode.
Narrow the scope of the `#if defined(PYBIND11_CPP14)` block around overload_cast to only
cover the parts where C++14 is stricly required. Thus, the implementation in
`pybind11::details::overload_cast_impl` is still available in C++11 mode.
* PR #1581: Modify test to use overload_cast_impl, update docs and change log
The -Wmissing-prototypes Clang warning (or -Wmissing-declarations on
GCC) is very useful to avoid accidents where a function definition in a
source file doesn't match the corresponding declaration in a header
file, as it would warn already during compilation and not much later
during link time.
Unfortunately this means that exported functions defined only in the
source file (usually the ones annotated with `extern "C"`) will cause
this warning to be emitted too (on Clang, GCC has a slightly different
behavior with -Wmissing-declarations and doesn't warn here). This fixes
the warning by providing a declaration right before the definition.
Clang has a bug [1] in x86 Windows that is exposed by the use of lambdas with "unforwardable" prototypes. The error is "error: cannot compile this forwarded non-trivially copyable parameter yet", and the message was introduced in [2] (used to be an assertion).
[1] https://llvm.org/bugs/show_bug.cgi?id=28299
[2] feb1567e07
This is only necessary if `get_internals` is called for the first time in a given module when the running thread is in a GIL-released state.
Fixes#1364
* Test dtype field order in numpy dtype tests
When running tests with NumPy 1.14 or later this test exposes the
"invalid buffer descriptor" error reported in #1274.
* Create dtype_ptr with ordered fields
* Fix casting of time points with non-system-clock duration on Windows
Add explicit `time_point_cast` to time point with duration of system
clock. Fixes Visual Studio compile error.
* Add test case for custom time points casting
