* For PyPy only, re-enable old behavior (likely to mask bugs), to avoid segfault with unknown root cause.
Change prompted by https://github.com/pybind/pybind11/issues/4075
* Undo the change in tests/test_exceptions.py
I turns out (I forgot) that PyPy segfaults in `test_flaky_exception_failure_point_init` already before the `MISMATCH` code path is reached:
https://github.com/pybind/pybind11/runs/7383663596
```
RPython traceback:
test_exceptions.py .......X.........Error in cpyext, CPython compatibility layer:
File "pypy_module_cpyext.c", line 14052, in wrapper_second_level__star_3_1
File "pypy_module_cpyext_1.c", line 35750, in not_supposed_to_fail
Fatal Python error: Segmentation fault
Stack (most recent call first, approximate line numbers):
File "/home/runner/work/pybind11/pybind11/tests/test_exceptions.py", line 306 in test_flaky_exception_failure_point_init
The function PyErr_NormalizeException was not supposed to fail
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/python.py", line 185 in pytest_pyfunc_call
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_callers.py", line 9 in _multicall
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_manager.py", line 77 in _hookexec
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_hooks.py", line 244 in __call__
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/python.py", line 1716 in runtest
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/runner.py", line 159 in pytest_runtest_call
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_callers.py", line 9 in _multicall
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_manager.py", line 77 in _hookexec
Fatal error in cpyext, CPython compatibility layer, calling PyErr_NormalizeException
Either report a bug or consider not using this particular extension
<SystemError object at 0x7fcc8cea6868>
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_hooks.py", line 244 in __call__
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/runner.py", line 261 in <lambda>
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/runner.py", line 317 in from_call
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/runner.py", line 246 in call_runtest_hook
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/runner.py", line 218 in call_and_report
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/runner.py", line 118 in runtestprotocol
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/runner.py", line 110 in pytest_runtest_protocol
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_callers.py", line 9 in _multicall
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_manager.py", line 77 in _hookexec
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_hooks.py", line 244 in __call__
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/main.py", line 335 in pytest_runtestloop
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_callers.py", line 9 in _multicall
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_manager.py", line 77 in _hookexec
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_hooks.py", line 244 in __call__
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/main.py", line 318 in _main
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/main.py", line 255 in wrap_session
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/main.py", line 314 in pytest_cmdline_main
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_callers.py", line 9 in _multicall
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_manager.py", line 77 in _hookexec
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_hooks.py", line 244 in __call__
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/config/__init__.py", line 133 in main
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/config/__init__.py", line 181 in console_main
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pytest/__main__.py", line 1 in <module>
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/lib-python/3/runpy.py", line 62 in _run_code
File "/opt/hostedtoolcache/PyPy/3.7.13/x64/lib-python/3/runpy.py", line 170 in _run_module_as_main
File "<builtin>/app_main.py", line 109 in run_toplevel
File "<builtin>/app_main.py", line 652 in run_command_line
File "<builtin>/app_main.py", line 996 in entry_point
Segmentation fault (core dumped)
```
* Add test_pypy_oserror_normalization
* Disable new `PYPY_VERSION` `#if`, to verify that the new test actually fails.
* Restore PYPY_VERSION workaround and update comment to reflect what was learned.
* [ci skip] Fix trivial oversight in comment.
* Disable implicit conversion from `0` to `pybind11::handle`.
* Reverse or-ed condition in an attempt to resolve GCC 8.3.0 errors (i386/debian:buster).
* Trying the simpler `std::is_same<T, PyObject *>`
* Add implicit_conversion_from_pytorch_THPObjectPtr_to_handle test.
* Accommodate types with implicit conversions to `PyObject *`, other than `handle` & `handle` subclasses, or integral types.
* Fix copy-paste mishap (picked wrong name).
* Revamp SFINAE construct to actually fix the pytorch issue (already validated against pytorch proper).
The first version of the reduced pytorch code was critically missing the move ctor. The first version of the accompanying test was meaningless.
Note: It turns out the `!std::is_arithmetic<T>` condition is not needed: `int` is not in general implicitly convertible to `PyObject *`, only the literal `0` is.
* Use `NOLINT(performance-noexcept-move-constructor)` for reduced code from the wild (rather than changing the code).
* Use any_of, all_of, negation. It turns out to clang-format nicer.
* Clean up comments for changed code.
* Reduce pytorch situation further, add test for operator ... const.
* Use `none_of` as suggested by @skylion007
* Add `pure_compile_tests_for_handle_from_PyObject_pointers()`
* Fix inconsequential oversight (retested).
* Factor our `is_pyobj_ptr_or_nullptr_t` to make the SFINAE conditions more readable.
* Remove stray line (oversight).
* Make the `pure_compile_tests_for_handle_from_PyObject_pointers()` "rhs-const-complete", too.
* Remove the temporary PYBIND11_UNDO_PR4008 `#ifdef`.
* Placeholder commit for 3.11 testing
* Does this fix it?
* Try suggestion
* Placeholder commit for 3.11 testing
* Does this fix it?
* Try suggestion
* fix: try using modern init for embedded interp
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* fix: error message changed in 3.11
* fix: apply logic in Python manually
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* fix autodetect dynamic attrs in 3.11
* fix: include error message if possible in error
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* ci: enable standard Python 3.11 testing
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* Make dynamic attrs condtiion exclusive to ver.
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
When converting an array to an Eigen matrix, ignore the strides if any
dimension size is 0. If the array is empty, the strides aren't relevant,
and especially numpy ≥ 1.23 explicitly sets the strides to 0 in this
case. (See numpy commit dd5ab7b11520.)
Update tests to verify that this works, and continues to work.
* error_already_set::what() is now constructed lazily
Prior to this commit throwing error_already_set was expensive due to the
eager construction of the error string (which required traversing the
Python stack). See #1853 for more context and an alternative take on the
issue.
Note that error_already_set no longer inherits from std::runtime_error
because the latter has no default constructor.
* Do not attempt to normalize if no exception occurred
This is not supported on PyPy-2.7 5.8.0.
* Extract exception name via tp_name
This is faster than dynamically looking up __name__ via GetAttrString.
Note though that the runtime of the code throwing an error_already_set
will be dominated by stack unwinding so the improvement will not be
noticeable.
Before:
396 ns ± 0.913 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
After:
277 ns ± 0.549 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
Benchmark:
const std::string foo() {
PyErr_SetString(PyExc_KeyError, "");
const std::string &s = py::detail::error_string();
PyErr_Clear();
return s;
}
PYBIND11_MODULE(foo, m) {
m.def("foo", &::foo);
}
* Reverted error_already_set to subclass std::runtime_error
* Revert "Extract exception name via tp_name"
The implementation of __name__ is slightly more complex than that.
It handles the module name prefix, and heap-allocated types. We could
port it to pybind11 later on but for now it seems like an overkill.
This reverts commit f1435c7e6b.
* Cosmit following @YannickJadoul's comments
Note that detail::error_string() no longer calls PyException_SetTraceback
as it is unncessary for pretty-printing the exception.
* Fixed PyPy build
* Moved normalization to error_already_set ctor
* Fix merge bugs
* Fix more merge errors
* Improve formatting
* Improve error message in rare case
* Revert back if statements
* Fix clang-tidy
* Try removing mutable
* Does build_mode release fix it
* Set to Debug to expose segfault
* Fix remove set error string
* Do not run error_string() more than once
* Trying setting the tracebackk to the value
* guard if m_type is null
* Try to debug PGI
* One last try for PGI
* Does reverting this fix PyPy
* Reviewer suggestions
* Remove unnecessary initialization
* Add noexcept move and explicit fail throw
* Optimize error_string creation
* Fix typo
* Revert noexcept
* Fix merge conflict error
* Abuse assignment operator
* Revert operator abuse
* See if we still need debug
* Remove unnecessary mutable
* Report "FATAL failure building pybind11::error_already_set error_string" and terminate process.
* Try specifying noexcept again
* Try explicit ctor
* default ctor is noexcept too
* Apply reviewer suggestions, simplify code, and make helper method private
* Remove unnecessary include
* Clang-Tidy fix
* detail::obj_class_name(), fprintf with [STDERR], [STDOUT] tags, polish comments
* consistently check m_lazy_what.empty() also in production builds
* Make a comment slightly less ambiguous.
* Bug fix: Remove `what();` from `restore()`.
It sure would need to be guarded by `if (m_type)`, otherwise `what()` fails and masks that no error was set (see update unit test). But since `error_already_set` is copyable, there is no point in releasing m_type, m_value, m_trace, therefore we can just as well avoid the runtime overhead of force-building `m_lazy_what`, it may never be used.
* Replace extremely opaque (unhelpful) error message with a truthful reflection of what we know.
* Fix clang-tidy error [performance-move-constructor-init].
* Make expected error message less specific.
* Various changes.
* bug fix: error_string(PyObject **, ...)
* Putting back the two eager PyErr_NormalizeException() calls.
* Change error_already_set() to call pybind11_fail() if the Python error indicator not set. The net result is that a std::runtime_error is thrown instead of error_already_set, but all tests pass as is.
* Remove mutable (fixes oversight in the previous commit).
* Normalize the exception only locally in error_string(). Python 3.6 & 3.7 test failures expected. This is meant for benchmarking, to determine if it is worth the trouble looking into the failures.
* clang-tidy: use auto
* Use `gil_scoped_acquire_local` in `error_already_set` destructor. See long comment.
* For Python < 3.8: `PyErr_NormalizeException` before `PyErr_WriteUnraisable`
* Go back to replacing the held Python exception with then normalized exception, if & when needed. Consistently document the side-effect.
* Slightly rewording comment. (There were also other failures.)
* Add 1-line comment for obj_class_name()
* Benchmark code, with results in this commit message.
function #calls test time [s] μs / call
master pure_unwind 729540 1.061 14.539876
err_set_unwind_err_clear 681476 1.040 15.260282
err_set_error_already_set 508038 1.049 20.640525
error_already_set_restore 555578 1.052 18.933288
pr1895_original_foo 244113 1.050 43.018168
PR / master
PR #1895 pure_unwind 736981 1.054 14.295685 98.32%
err_set_unwind_err_clear 685820 1.045 15.237399 99.85%
err_set_error_already_set 661374 1.046 15.811879 76.61%
error_already_set_restore 669881 1.048 15.645176 82.63%
pr1895_original_foo 318243 1.059 33.290806 77.39%
master @ commit ad146b2a18
Running tests in directory "/usr/local/google/home/rwgk/forked/pybind11/tests":
============================= test session starts ==============================
platform linux -- Python 3.9.10, pytest-6.2.3, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /usr/local/google/home/rwgk/forked/pybind11/tests, configfile: pytest.ini
collecting ... collected 5 items
test_perf_error_already_set.py::test_perf[pure_unwind]
PERF pure_unwind,729540,1.061,14.539876
PASSED
test_perf_error_already_set.py::test_perf[err_set_unwind_err_clear]
PERF err_set_unwind_err_clear,681476,1.040,15.260282
PASSED
test_perf_error_already_set.py::test_perf[err_set_error_already_set]
PERF err_set_error_already_set,508038,1.049,20.640525
PASSED
test_perf_error_already_set.py::test_perf[error_already_set_restore]
PERF error_already_set_restore,555578,1.052,18.933288
PASSED
test_perf_error_already_set.py::test_perf[pr1895_original_foo]
PERF pr1895_original_foo,244113,1.050,43.018168
PASSED
============================== 5 passed in 12.38s ==============================
pr1895 @ commit 8dff51d12e
Running tests in directory "/usr/local/google/home/rwgk/forked/pybind11/tests":
============================= test session starts ==============================
platform linux -- Python 3.9.10, pytest-6.2.3, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /usr/local/google/home/rwgk/forked/pybind11/tests, configfile: pytest.ini
collecting ... collected 5 items
test_perf_error_already_set.py::test_perf[pure_unwind]
PERF pure_unwind,736981,1.054,14.295685
PASSED
test_perf_error_already_set.py::test_perf[err_set_unwind_err_clear]
PERF err_set_unwind_err_clear,685820,1.045,15.237399
PASSED
test_perf_error_already_set.py::test_perf[err_set_error_already_set]
PERF err_set_error_already_set,661374,1.046,15.811879
PASSED
test_perf_error_already_set.py::test_perf[error_already_set_restore]
PERF error_already_set_restore,669881,1.048,15.645176
PASSED
test_perf_error_already_set.py::test_perf[pr1895_original_foo]
PERF pr1895_original_foo,318243,1.059,33.290806
PASSED
============================== 5 passed in 12.40s ==============================
clang++ -o pybind11/tests/test_perf_error_already_set.os -c -std=c++17 -fPIC -fvisibility=hidden -Os -flto -Wall -Wextra -Wconversion -Wcast-qual -Wdeprecated -Wnon-virtual-dtor -Wunused-result -isystem /usr/include/python3.9 -isystem /usr/include/eigen3 -DPYBIND11_STRICT_ASSERTS_CLASS_HOLDER_VS_TYPE_CASTER_MIX -DPYBIND11_TEST_BOOST -Ipybind11/include -I/usr/local/google/home/rwgk/forked/pybind11/include -I/usr/local/google/home/rwgk/clone/pybind11/include /usr/local/google/home/rwgk/forked/pybind11/tests/test_perf_error_already_set.cpp
clang++ -o lib/pybind11_tests.so -shared -fPIC -Os -flto -shared ...
Debian clang version 13.0.1-3+build2
Target: x86_64-pc-linux-gnu
Thread model: posix
* Changing call_repetitions_target_elapsed_secs to 0.1 for regular unit testing.
* Adding in `recursion_depth`
* Optimized ctor
* Fix silly bug in recurse_first_then_call()
* Add tests that have equivalent PyErr_Fetch(), PyErr_Restore() but no try-catch.
* Add call_error_string to tests. Sample only recursion_depth 0, 100.
* Show lazy-what speed-up in percent.
* Include real_work in benchmarks.
* Replace all PyErr_SetString() with generate_python_exception_with_traceback()
* Better organization of test loops.
* Add test_error_already_set_copy_move
* Fix bug in newly added test (discovered by clang-tidy): actually use move ctor
* MSVC detects the unreachable return
* change test_perf_error_already_set.py back to quick mode
* Inherit from std::exception (instead of std::runtime_error, which does not make sense anymore with the lazy what)
* Special handling under Windows.
* print with leading newline
* Removing test_perf_error_already_set (copies are under 7765113fbb).
* Avoid gil and scope overhead if there is nothing to release.
* Restore default move ctor. "member function" instead of "function" (note that "method" is Python terminology).
* Delete error_already_set copy ctor.
* Make restore() non-const again to resolve clang-tidy failure (still experimenting).
* Bring back error_already_set copy ctor, to see if that resolves the 4 MSVC test failures.
* Add noexcept to error_already_set copy & move ctors (as suggested by @skylion007 IIUC).
* Trying one-by-one noexcept copy ctor for old compilers.
* Add back test covering copy ctor. Add another simple test that exercises the copy ctor.
* Exclude more older compilers from using the noexcept = default ctors. (The tests in the previous commit exposed that those are broken.)
* Factor out & reuse gil_scoped_acquire_local as gil_scoped_acquire_simple
* Guard gil_scoped_acquire_simple by _Py_IsFinalizing() check.
* what() GIL safety
* clang-tidy & Python 3.6 fixes
* Use `gil_scoped_acquire` in dtor, copy ctor, `what()`. Remove `_Py_IsFinalizing()` checks (they are racy: https://github.com/python/cpython/pull/28525).
* Remove error_scope from copy ctor.
* Add `error_scope` to `get_internals()`, to cover the situation that `get_internals()` is called from the `error_already_set` dtor while a new Python error is in flight already. Also backing out `gil_scoped_acquire_simple` change.
* Add `FlakyException` tests with failure triggers in `__init__` and `__str__`
THIS IS STILL A WORK IN PROGRESS. This commit is only an important resting point.
This commit is a first attempt at addressing the observation that `PyErr_NormalizeException()` completely replaces the original exception if `__init__` fails. This can be very confusing even in small applications, and extremely confusing in large ones.
* Tweaks to resolve Py 3.6 and PyPy CI failures.
* Normalize Python exception immediately in error_already_set ctor.
For background see: https://github.com/pybind/pybind11/pull/1895#issuecomment-1135304081
* Fix oversights based on CI failures (copy & move ctor initialization).
* Move @pytest.mark.xfail("env.PYPY") after @pytest.mark.parametrize(...)
* Use @pytest.mark.skipif (xfail does not work for segfaults, of course).
* Remove unused obj_class_name_or() function (it was added only under this PR).
* Remove already obsolete C++ comments and code that were added only under this PR.
* Slightly better (newly added) comments.
* Factor out detail::error_fetch_and_normalize. Preparation for producing identical results from error_already_set::what() and detail::error_string(). Note that this is a very conservative refactoring. It would be much better to first move detail::error_string into detail/error_string.h
* Copy most of error_string() code to new error_fetch_and_normalize::complete_lazy_error_string()
* Remove all error_string() code from detail/type_caster_base.h. Note that this commit includes a subtle bug fix: previously error_string() restored the Python error, which will upset pybind11_fail(). This never was a problem in practice because the two PyType_Ready() calls in detail/class.h do not usually fail.
* Return const std::string& instead of const char * and move error_string() to pytypes.h
* Remove gil_scope_acquire from error_fetch_and_normalize, add back to error_already_set
* Better handling of FlakyException __str__ failure.
* Move error_fetch_and_normalize::complete_lazy_error_string() implementation from pybind11.h to pytypes.h
* Add error_fetch_and_normalize::release_py_object_references() and use from error_already_set dtor.
* Use shared_ptr for m_fetched_error => 1. non-racy, copy ctor that does not need the GIL; 2. enables guard against duplicate restore() calls.
* Add comments.
* Trivial renaming of a newly introduced member function.
* Workaround for PyPy
* Bug fix (oversight). Only valgrind got this one.
* Use shared_ptr custom deleter for m_fetched_error in error_already_set. This enables removing the dtor, copy ctor, move ctor completely.
* Further small simplification. With the GIL held, simply deleting the raw_ptr takes care of everything.
* IWYU cleanup
```
iwyu version: include-what-you-use 0.17 based on Debian clang version 13.0.1-3+build2
```
Command used:
```
iwyu -c -std=c++17 -DPYBIND11_TEST_BOOST -Iinclude/pybind11 -I/usr/include/python3.9 -I/usr/include/eigen3 include/pybind11/pytypes.cpp
```
pytypes.cpp is a temporary file: `#include "pytypes.h"`
The raw output is very long and noisy.
I decided to use `#include <cstddef>` instead of `#include <cstdio>` for `std::size_t` (iwyu sticks to the manual choice).
I ignored all iwyu suggestions that are indirectly covered by `#include <Python.h>`.
I manually verified that all added includes are actually needed.
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
* Add test_perf_accessors (to be merged into test_pytypes).
* Python < 3.8 f-string compatibility
* Use thread_local in inc_ref_counter()
* Intentional breakage, brute-force way to quickly find out how many platforms reach the PYBIND11_HANDLE_REF_DEBUG code, with and without threads.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove Intentional breakage
* Drop perf test, move inc_refs tests to test_pytypes
* Fold in PR #3970 with `#ifdef`s
* Complete test coverage for all newly added code.
* Condense new unit tests via a simple local helper macro.
* Remove PYBIND11_PR3970 define. See https://github.com/pybind/pybind11/pull/3977#issuecomment-1142526417
* Move static keyword first (fixes silly oversight).
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add missing error handling to module_::def_submodule
* Add test_def_submodule_failures
* PyPy only: Skip test with trigger for PyModule_GetName() failure.
* Reapply minor fix that accidentally got lost in transfer from PR #3964
* Add error_already_set_what what tests, asserting the status quo.
* Move PyErr_NormalizeException() up a few lines.
* @pytest.mark.skipif("env.PYPY") from PR #1895 is required even for this much simpler PR
* Move PyException_SetTraceback() with PyErr_NormalizeException() as suggested by @skylion007
* Insert a std::move() as suggested by @skylion007
* Don't return pointers to static objects with return_value_policy::take_ownership.
This fixes -Wfree-nonheap-object warnings produced by GCC.
* Use return value policy fix instead
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
* Fix py::cast from pytype rvalue to pytype
Previously, py::cast blindly assumed that the destination type was a C++
type rather than a python type when the source type was an rvalue.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add frozenset, and allow it cast to std::set
For the reverse direction, std::set still casts to set. This is in concordance with the behavior for sequence containers, where e.g. tuple casts to std::vector but std::vector casts to list.
Extracted from #3886.
* Rename set_base to any_set to match Python C API
since this will be part of pybind11 public API
* PR: static_cast, anyset
* Add tests for frozenset
and rename anyset methods
* Remove frozenset default ctor, add tests
Making frozenset non-default constructible means that we need to adjust pyobject_caster to not require that its value is default constructible, by initializing value to a nil handle. This also allows writing C++ functions taking anyset, and is arguably a performance improvement, since there is no need to allocate an object that will just be replaced by load.
Add some more tests, including anyset::empty, anyset::size, set::add and set::clear.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add rationale to `pyobject_caster` default ctor
* Remove ineffectual protected: access control
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Test pickling a simple callable (does not work).
Currently only documents that it does not work. Starting point for future fix.
* Use re.search to accommodate variations of the TypeError message.
* PyPy: exercise full dumps/loads cycle.
* Adding explicit "broken" comment.
* Update cast.h
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Move definition to detail/common, change name, apply everywhere
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Rename debug_enabled in tests to detailed_error_messages_enabled
* Added constructor based on typenum, based on PyArray_DescrFromType
Added accessors for typenum, alignment, byteorder and flags fields of
PyArray_Descr struct.
* Added tests for new py::dtype constructor, and for accessors
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fixed the comment for alignment method
* Update include/pybind11/numpy.h
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
* Adds set_name method of pybind11::capsule class
This calls PyCapsule_SetName on the underlying capsule object.
modified destructors to query capsules's Name
[pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Handle possible exception thrown by PyCapsule_GetName
Also removed accidentally reintroduced use of `const char *&`.
[pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Fixed function name
* Introduced private static function to reuse get_name_or_throw
* added tests for capsule renaming
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* handle python error in flight
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Initialized PyObject * variables to nullptr
* use write-unraisable if PyCapsule_GetName raises
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* get_name_or_throw->get_name_no_throw
If PyCapsule_GetName raises an error we should write as unraisable
to consume it and notify user, and then restore the error in flight if any.
This way this method called from destructor would not modify interpreter
error state.
* used error_scope struct
* Renamed get_name_no_throw->get_name_in_error_scope
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add type_caster<std::monostate> for std::variant
Add type_caster<std::monostate>, allowing std::variant<std::monostate, ...>
* Add variant<std::monostate, ...> test methods
* Add std::monostate tests
* Update test_stl.py
Remove erroneous extra tests
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update test fn name
* And update the doc() test
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* PYBIND11_OBJECT_CVT should use namespace for error_already_set()
This change makes the macro usable outside of pybind11 namespace.
* added test for use of PYBIND11_OBJECT_CVT for classes in external to pybind11 namespaces
* Extended test_pytypes.cpp and test_pytest.py
The added test defines a dummy function that takes a custom-defined class external::float_
that uses PYBIND11_OBJECT_CVT
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fixed issues pointed out by CI
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fixed memory leak in default constructor
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add bytearray to string cast, testcase and rename load_bytes to load_raw
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* New bytearray test case and convert failure to pybind11_fail
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix merge comments
* Actually fix merge comments
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Assert early if AsString fails
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Porras Huang <pohuang@jackx-vm-1.nvidia.com>
* Clear local internals after finalizing interpreter
* Add descriptive comments
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Clear Python error state if pybind11::weakref() fails.
The weakref() constructor calls pybind11_fail() without clearing any
Python interpreter error state. If a client catches the C++ exception
thrown by pybind11_fail(), the Python interpreter will be left in an
error state.
* Add test case for failing to create weakref
* Add Debug asserts for pybind11 fail
* Make error handling more pythonic
* Does this fix PyPy?
* Adapt test to PyPy differences
* Simplify test to remove redundancy
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
* chore: drop Python 3.5 support
* chore: more fstrings with flynt's help
* ci: drop Python 3.5
* chore: bump dependency versions
* docs: touch up py::args
* tests: remove deprecation warning
* Ban smartquotes
* Very minor tweaks (by-product of reviewing PR #3719).
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
* `#error BYE_BYE_GOLDEN_SNAKE`
* Removing everything related to 2.7 from ci.yml
* Commenting-out Centos7
* Removing `PYTHON: 27` from .appveyor.yml
* "PY2" removal, mainly from tests. C++ code is not touched.
* Systematic removal of `u` prefix from `u"..."` and `u'...'` literals. Collateral cleanup of a couple minor other things.
* Cleaning up around case-insensitive hits for `[^a-z]py.*2` in tests/.
* Removing obsolete Python 2 mention in compiling.rst
* Proper `#error` for Python 2.
* Using PY_VERSION_HEX to guard `#error "PYTHON 2 IS NO LONGER SUPPORTED.`
* chore: bump pre-commit
* style: run pre-commit for pyupgrade 3+
* tests: use sys.version_info, not PY
* chore: more Python 2 removal
* Uncommenting Centos7 block (PR #3691 showed that it is working again).
* Update pre-commit hooks
* Fix pre-commit hook
* refactor: remove Python 2 from CMake
* refactor: remove Python 2 from setup code
* refactor: simplify, better static typing
* feat: fail with nice messages
* refactor: drop Python 2 C++ code
* docs: cleanup for Python 3
* revert: intree
revert: intree
* docs: minor touchup to py2 statement
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
* Manual line breaks to pre-empt undesired `clang-format`ing.
Informed by work under https://github.com/pybind/pybind11/pull/3683:
60b7eb410f59572e6559
* Manual curation of clang-format diffs involving source code comments.
Very labor-intensive and dull.
* Pulling .clang-format change from @henryiii's 9057962d40
* Adding commonly used .clang-format `CommentPragmas:`
* Ensure short lambdas are allowed
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
* Adding readability-qualified-auto to .clang-tidy
Ported from @henryiii's 287527f705
* fix: support Python < 3.6
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
* Download catch for MinGw
* Fix rest of MinGW
* fix: update catch to 2.13.5 to fix glibc 2.34 failures
Update the downloaded Catch version to 2.13.5, in order to fix build
failure on glibc 2.34:
```
In file included from /usr/include/signal.h:328,
from /tmp/pybind11/.nox/tests-3-9/tmp/tests/catch/catch.hpp:8030,
from /tmp/pybind11/tests/test_embed/catch.cpp:13:
/tmp/pybind11/.nox/tests-3-9/tmp/tests/catch/catch.hpp:10818:58: error: call to non-‘constexpr’ function ‘long int sysconf(int)’
10818 | static constexpr std::size_t sigStackSize = 32768 >= MINSIGSTKSZ ? 32768 : MINSIGSTKSZ;
| ^~~~~~~~~~~
In file included from /usr/include/python3.9/Python.h:36,
from /tmp/pybind11/include/pybind11/detail/common.h:215,
from /tmp/pybind11/include/pybind11/pytypes.h:12,
from /tmp/pybind11/include/pybind11/cast.h:13,
from /tmp/pybind11/include/pybind11/attr.h:13,
from /tmp/pybind11/include/pybind11/pybind11.h:13,
from /tmp/pybind11/include/pybind11/embed.h:12,
from /tmp/pybind11/tests/test_embed/catch.cpp:4:
/usr/include/unistd.h:640:17: note: ‘long int sysconf(int)’ declared here
640 | extern long int sysconf (int __name) __THROW;
| ^~~~~~~
In file included from /tmp/pybind11/tests/test_embed/catch.cpp:13:
/tmp/pybind11/.nox/tests-3-9/tmp/tests/catch/catch.hpp:10877:45: error: size of array ‘altStackMem’ is not an integral constant-expression
10877 | char FatalConditionHandler::altStackMem[sigStackSize] = {};
| ^~~~~~~~~~~~
```
The newest Catch version cannot be used yet because of regression:
https://github.com/catchorg/Catch2/pull/2364
* fix: add option for _ check, only define on pybind11
* Revert "fix: add option for _ check, only define on pybind11"
This reverts commit 86817db488.
* fix: only undef _ for catch cpp creation.
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
* test: Strip whitespace when comparing numpy dtypes for 1.22 compat
Strip whitespace when comparing numpy dtype str() in order to preserve
test compatibility with both numpy 1.22 and older versions whose output
differ by whitespace.
Fixes#3680
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add tests demonstrating smart_holder issues with multiple inheritance
* Propagate C++ multiple inheritance markers to all children
- Makes py::multiple_inheritance only needed in base classes hidden from pybind11
* Add support for nested C++11 exceptions
* Remove wrong include
* Fix if directive
* Fix missing skipif
* Simplify code and try to work around MSVC bug
* Clarify comment
* Further simplify code
* Remove the last extra throw statement
* Qualify auto
* Fix typo
* Add missing return for consistency
* Fix clang-tidy complaint
* Fix python2 stub
* Make clang-tidy happy
* Fix compile error
* Fix python2 function signature
* Extract C++20 utility and backport
* Cleanup code a bit more
* Improve test case
* Consolidate code and fix signature
* Fix typo
* A way to register additional test targets.
* Support specifying tests with extension.
* Ensure TEST_OVERRIDE is backwards compatible.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Ensure regex is non greedy.
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Replacing ICC C++14 with C++17
* ICPC: -diag-disable:conversion
Try to suppress the `-Werror-all` promotion in `#include <variant>`
* Trying `-DPYBIND11_WERROR=OFF`
* Trying `-Wno-conversion` (todo: make specific to C++17 if it works)
* Inserting -Wno-conversion only for C++17
Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja>
* Expand string_view support to str, bytes, memoryview
1. Allows constructing a str or bytes implicitly from a string_view;
this is essentially a small shortcut allowing a caller to write
`py::bytes{sv}` rather than `py::bytes{sv.data(), sv.size()}`.
2. Allows implicit conversion *to* string_view from py::bytes -- this
saves a fair bit more as currently there is no simple way to get such
a view of the bytes without copying it (or resorting to Python API
calls).
(This is not done for `str` because when the str contains unicode we
have to allocate to a temporary and so there might not be some string
data we can properly view without owning.)
3. Allows `memoryview::from_memory` to accept a string_view. As with
the other from_memory calls, it's entirely your responsibility to
keep it alive.
This also required moving the string_view availability detection into
detail/common.h because this PR needs it in pytypes.h, which is higher
up the include chain than cast.h where it was being detected currently.
* Move string_view include to pytypes.h
* CI-testing a fix for the "ambiguous conversion" issue.
This change is known to fix the `tensorflow::tstring` issue reported under https://github.com/pybind/pybind11/pull/3521#issuecomment-985100965
TODO: Minimal reproducer for the `tensorflow::tstring` issue.
* Make clang-tidy happy (hopefully).
* Adding minimal reproducer for the `tensorflow::tstring` issue.
Error without the enable_if trick:
```
/usr/local/google/home/rwgk/forked/pybind11/tests/test_builtin_casters.cpp:169:16: error: ambiguous conversion for functional-style cast from 'TypeWithBothOperatorStringAndStringView' to 'py::bytes'
return py::bytes(TypeWithBothOperatorStringAndStringView());
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/google/home/rwgk/forked/pybind11/include/pybind11/detail/../pytypes.h:1174:5: note: candidate constructor
bytes(const std::string &s) : bytes(s.data(), s.size()) { }
^
/usr/local/google/home/rwgk/forked/pybind11/include/pybind11/detail/../pytypes.h:1191:5: note: candidate constructor
bytes(std::string_view s) : bytes(s.data(), s.size()) { }
^
```
* Adding missing NOLINTNEXTLINE
* Also apply ambiguous conversion workaround to str()
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
* Fix py::kw_only when used before the first arg of a method
The implicit space for the `self` argument isn't added until we hit the
first argument, but this wasn't being done for kw_only or pos_only, and
so a kw_only before the first argument would break.
This fixes it by properly checking whether we need to add the self arg.
(The pos_only issue here was extremely mild -- you didn't get the `/` in
the docstring, but AFAICT it has no other effect since there are no
meaningful arguments before it anyway).
* Style changes
- rename check_have_self_arg -> append_self_arg_if_needed
- move the argument name inline comments before the args instead of
after
* ci: support development releases of Python
* fix: better PyPy support
* fix: patch over a few more pypy issues
* Try to patch
* Properly follow pep667
* Fix typo
* Whoops, 667 not in yet
* For testing
* More testing
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Try to backport
* Try to simplify fix
* Nail down the fix
* Try pypy workaround
* Typo
* one last typo
* Replacing 0x03110000 with 0x030B0000
* Add TODO. Drop PyPy
* Fix typo
* Revert catch upgrade
* fix: minor cleanup, try pypy again
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
* override: Fix wrong caching of the overrides
There was a problem when the python type, which was stored in override
cache for C++ functions, was destroyed and the record wasn't removed from the
override cache. Therefor, dangling pointer was stored there. Then when the
memory was reused and new type was allocated at the given address and the
method with the same name (as previously stored in the cache) was actually
overridden in python, it would wrongly find it in the override cache for C++
functions and therefor override from python wouldn't be called.
The fix is to erase the type from the override cache when the type is destroyed.
* test: Pass by const ref instead of by value (clang-tidy)
* test: Rename classes and move to different files
Rename the classes and files so they're no too generic. Also, better place to
test the stuff is in test_virtual_functions.cpp/.py as we're basically testing
the virtual functions/trampolines.
* Add TODO for erasure code
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
On Unix, setuptools prepends $CFLAGS and $CPPFLAGS to the compiler flags
(they always come before extra_compile_args and anything else; see
distutils.sysconfig.customize_compiler). In practice, the environment
variables are useful e.g. to quickly generate a debug build (e.g. by
setting CFLAGS=-g), but Pybind11Extension currently unconditionally
overwrites this with -g0.
Instead, check the environment variables and only insert -g0 if not
overridden by them.
* fix: add missing std::forward calls
Two of the four cpp_function overloads are missing std::forward calls, which seems like a simple oversight.
* add test for https://github.com/pybind/pybind11/pull/3443
* add py tests
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix test
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Simply has_kw_only_args handling
This simplifies tracking the number of kw-only args by instead tracking
the number of positional arguments (which is really what we care about
everywhere this is used).
* Allow keyword-only arguments to follow py::args
This removes the constraint that py::args has to be last (or
second-last, with py::kwargs) and instead makes py::args imply
py::kw_only for any remaining arguments, allowing you to bind a function
that works the same way as a Python function such as:
def f(a, *args, b):
return a * b + sum(args)
f(10, 1, 2, 3, b=20) # == 206
With this change, you can bind such a function using:
m.def("f", [](int a, py::args args, int b) { /* ... */ },
"a"_a, "b"_a);
Or, to be more explicit about the keyword-only arguments:
m.def("g", [](int a, py::args args, int b) { /* ... */ },
"a"_a, py::kw_only{}, "b"_a);
(The only difference between the two is that the latter will fail at
binding time if the `kw_only{}` doesn't match the `py::args` position).
This doesn't affect backwards compatibility at all because, currently,
you can't have a py::args anywhere except the end/2nd-last.
* Take args/kwargs by const lvalue ref
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
* ci: support Python 3.11-dev
Also update 3.10 to final, better PyPy usage
* fix: use PyFrame_GetCode on Python 3.9+
* ci: some bitiness of pypy not supported on win
* chore: update CMake support to 3.22rc1 to quiet warning
* fix: use dev version of py to fix Py 3.11
* tests: print proper Eigen version
* ci: include pypy2, not sure why
* ci: avoid running on Python 3.11 for now
* ci: fix runs
* ci: simpler PyPy usage, drop unmaintained scipy + pypy index
* ci: only binary numpy, wait on pypy 3.8
* refactor: address review
* fix: the types for return_value_policy_override in optional_caster
`return_value_policy_override` was not being applied correctly in
`optional_caster` in two ways:
- The `is_lvalue_reference` condition referenced `T`, which was the
`optional<T>` type parameter from the class, when it should have used `T_`,
which was the parameter to the `cast` function. `T_` can potentially be a
reference type, but `T` will never be.
- The type parameter passed to `return_value_policy_override` should be
`T::value_type`, not `T`. This matches the way that the other STL container
type casters work.
The result of these issues was that a method/property definition which used a
`reference` or `reference_internal` return value policy would create a Python
value that's bound by reference to a temporary C++ object, resulting in
undefined behavior. For reasons that I was not able to figure out fully, it
seems like this causes problems when using old versions of `boost::optional`,
but not with recent versions of `boost::optional` or the `libstdc++`
implementation of `std::optional`. The issue (that the override to
`return_value_policy::move` is never being applied) is present for all
implementations, it just seems like that somehow doesn't result in problems for
the some implementation of `optional`. This change includes a regression type
with a custom optional-like type which was able to reproduce the issue.
Part of the issue with using the wrong types may have stemmed from the type
variables `T` and `T_` having very similar names. This also changes the type
variables in `optional_caster` to use slightly more descriptive names, which
also more closely follow the naming convention used by the other STL casters.
Fixes#3330
* Fix clang-tidy complaints
* Add missing NOLINT
* Apply a couple more fixes
* fix: support GCC 4.8
* tests: avoid warning about unknown compiler for compilers missing C++17
* Remove unneeded test module attribute
* Change test enum to have more unique int values
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
* Adding MSVC C4127 suppression around Eigen includes.
* For MSVC 2015 only: also adding the C4127 suppression to test_eigen.cpp
* Copying original change from PR #3343, with extra line breaks to not run past 99 columns (our desired but currently not enforced limit).
* Add a test showing a flaw in make_key_iterator/make_value_iterator
If the iterator dereference operator returns a value rather than a
reference (and that pair also does not *contain* references),
make_key_iterator and make_value_iterator will return a reference to a
temporary, causing a segfault.
* Fix make_key_iterator/make_value_iterator for prvalue iterators
If an iterator returns a pair<T1, T2> rather than a reference to a pair
or a pair of references, make_key_iterator and make_value_iterator would
return a reference to a temporary, typically leading to a segfault. This
is because the value category of member access to a prvalue is an
xvalue, not a prvalue, so decltype produces an rvalue reference type.
Fix the type calculation to handle this case.
I also removed some decltype parentheses that weren't needed, either
because the expression isn't one of the special cases for decltype or
because decltype was only used for SFINAE. Hopefully that makes the code
a bit more readable.
Closes#3347
* Attempt a workaround for nvcc
* Add `.keys` and `.values` to bind_map
Both of these implement views (rather than just iterators), and `.items`
is also upgraded to a view. In practical terms, this allows a view to be
iterated multiple times and have its size taken, neither of which works
with an iterator.
The views implement `__len__`, `__iter__`, and the keys view implements
`__contains__`. Testing membership also works in item and value views
because Python falls back to iteration. This won't be optimal
for item values since it's linear rather than O(log n) or O(1), but I
didn't fancy trying to get all the corner cases to match Python
behaviour (tuple of wrong types, wrong length tuple, not a tuple etc).
Missing relative to Python dictionary views is `__reversed__` (only
added to Python in 3.8). Implementing that could break code that binds
custom map classes which don't provide `rbegin`/`rend` (at least without
doing clever things with SFINAE), so I've not tried.
The size increase on my system is 131072 bytes, which is rather large
(5%) but also suspiciously round (2^17) and makes me suspect some
quantisation effect.
* bind_map: support any object in __contains__
Add extra overload of `__contains__` (for both the map itself and
KeysView) which takes an arbitrary object and returns false.
* Take py::object by const reference in __contains__
To keep clang-tidy happy.
* Removing stray `py::` (detected via interactive testing in Google environment).
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
* Fix `pybind11::object::operator=` to be safe if `*this` is accessible from Python
* Add `custom_type_setup` attribute
This allows for custom modifications to the PyHeapTypeObject prior to
calling `PyType_Ready`. This may be used, for example, to define
`tp_traverse` and `tp_clear` functions.
* Add make_value_iterator (#3271)
* Add make_value_iterator
This is the counterpart to make_key_iterator, and will allow
implementing a `value` method in `bind_map` (although doing so is left
for a subsequent PR).
I made a few design changes to reduce copy-and-paste boilerplate.
Previously detail::iterator_state had a boolean template parameter to
indicate whether it was being used for make_iterator or
make_key_iterator. I replaced the boolean with a class that determines
how to dereference the iterator. This allows for a generic
implementation of `__next__`.
I also added the ValueType and Extra... parameters to the iterator_state
template args, because I think it was a bug that they were missing: if
make_iterator is called twice with different values of these, only the
first set has effect (because the state class is only registered once).
There is still a potential issue in that the *values* of the extra
arguments are latched on the first call, but since most policies are
empty classes this should be even less common.
* Add some remove_cv_t to appease clang-tidy
* Make iterator_access and friends take reference
For some reason I'd accidentally made it take a const value, which
caused some issues with third-party packages.
* Another attempt to remove remove_cv_t from iterators
Some of the return types were const (non-reference) types because of the
pecularities of decltype: `decltype((*it).first)` is the *declared* type
of the member of the pair, rather than the type of the expression. So if
the reference type of the iterator is `pair<const int, int> &`, then the
decltype is `const int`. Wrapping an extra set of parentheses to form
`decltype(((*it).first))` would instead give `const int &`.
This means that the existing make_key_iterator actually returns by value
from `__next__`, rather than by reference. Since for mapping types, keys
are always const, this probably hasn't been noticed, but it will affect
make_value_iterator if the Python code tries to mutate the returned
objects. I've changed things to use double parentheses so that
make_iterator, make_key_iterator and make_value_iterator should now all
return the reference type of the iterator. I'll still need to add a test
for that; for now I'm just checking whether I can keep Clang-Tidy happy.
* Add back some NOLINTNEXTLINE to appease Clang-Tidy
This is favoured over using remove_cv_t because in some cases a const
value return type is deliberate (particularly for Eigen).
* Add a unit test for iterator referencing
Ensure that make_iterator, make_key_iterator and make_value_iterator
return references to the container elements, rather than copies. The
test for make_key_iterator fails to compile on master, which gives me
confidence that this branch has fixed it.
* Make the iterator_access etc operator() const
I'm actually a little surprised it compiled at all given that the
operator() is called on a temporary, but I don't claim to fully
understand all the different value types in C++11.
* Attempt to work around compiler bugs
https://godbolt.org/ shows an example where ICC gets the wrong result
for a decltype used as the default for a template argument, and CI also
showed problems with PGI. This is a shot in the dark to see if it fixes
things.
* Make a test constructor explicit (Clang-Tidy)
* Fix unit test on GCC 4.8.5
It seems to require the arguments to the std::pair constructor to be
implicitly convertible to the types in the pair, rather than just
requiring is_constructible.
* Remove DOXYGEN_SHOULD_SKIP_THIS guards
Now that a complex decltype expression has been replaced by a simpler
nested type, I'm hoping Doxygen will be able to build it without issues.
* Add comment to explain iterator_state template params
* fix: regression in #3271
Co-authored-by: Bruce Merry <1963944+bmerry@users.noreply.github.com>
* Adding nullptr slices
Using example from #1095
Some fixes from @wjakob's review
Stop clang-tidy from complaining
New proposal for py::slice constructor
Eric's suggested changes: simplify testing; shift def's
* chore: drop MSVC pragma (hopefully unneeded)
* Apply suggestions from code review
* Add make_value_iterator
This is the counterpart to make_key_iterator, and will allow
implementing a `value` method in `bind_map` (although doing so is left
for a subsequent PR).
I made a few design changes to reduce copy-and-paste boilerplate.
Previously detail::iterator_state had a boolean template parameter to
indicate whether it was being used for make_iterator or
make_key_iterator. I replaced the boolean with a class that determines
how to dereference the iterator. This allows for a generic
implementation of `__next__`.
I also added the ValueType and Extra... parameters to the iterator_state
template args, because I think it was a bug that they were missing: if
make_iterator is called twice with different values of these, only the
first set has effect (because the state class is only registered once).
There is still a potential issue in that the *values* of the extra
arguments are latched on the first call, but since most policies are
empty classes this should be even less common.
* Add some remove_cv_t to appease clang-tidy
* Make iterator_access and friends take reference
For some reason I'd accidentally made it take a const value, which
caused some issues with third-party packages.
* Another attempt to remove remove_cv_t from iterators
Some of the return types were const (non-reference) types because of the
pecularities of decltype: `decltype((*it).first)` is the *declared* type
of the member of the pair, rather than the type of the expression. So if
the reference type of the iterator is `pair<const int, int> &`, then the
decltype is `const int`. Wrapping an extra set of parentheses to form
`decltype(((*it).first))` would instead give `const int &`.
This means that the existing make_key_iterator actually returns by value
from `__next__`, rather than by reference. Since for mapping types, keys
are always const, this probably hasn't been noticed, but it will affect
make_value_iterator if the Python code tries to mutate the returned
objects. I've changed things to use double parentheses so that
make_iterator, make_key_iterator and make_value_iterator should now all
return the reference type of the iterator. I'll still need to add a test
for that; for now I'm just checking whether I can keep Clang-Tidy happy.
* Add back some NOLINTNEXTLINE to appease Clang-Tidy
This is favoured over using remove_cv_t because in some cases a const
value return type is deliberate (particularly for Eigen).
* Add a unit test for iterator referencing
Ensure that make_iterator, make_key_iterator and make_value_iterator
return references to the container elements, rather than copies. The
test for make_key_iterator fails to compile on master, which gives me
confidence that this branch has fixed it.
* Make the iterator_access etc operator() const
I'm actually a little surprised it compiled at all given that the
operator() is called on a temporary, but I don't claim to fully
understand all the different value types in C++11.
* Attempt to work around compiler bugs
https://godbolt.org/ shows an example where ICC gets the wrong result
for a decltype used as the default for a template argument, and CI also
showed problems with PGI. This is a shot in the dark to see if it fixes
things.
* Make a test constructor explicit (Clang-Tidy)
* Fix unit test on GCC 4.8.5
It seems to require the arguments to the std::pair constructor to be
implicitly convertible to the types in the pair, rather than just
requiring is_constructible.
* Remove DOXYGEN_SHOULD_SKIP_THIS guards
Now that a complex decltype expression has been replaced by a simpler
nested type, I'm hoping Doxygen will be able to build it without issues.
* Add comment to explain iterator_state template params
* Enable defining custom __new__
* See if xfail needed
* Qualify auto self
* Unconditionally defining PYBIND11_DISABLE_NEW_STYLE_INIT_WARNING. Returning pointer from "__init__" instead of reference.
* Use new style __init__
* Simplify __new__ creation
* Reviewer suggestions
* Match indentation
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
* Eliminate duplicate TLS keys for loader_life_support stack
This revises the existing fix for
https://github.com/pybind/pybind11/issues/2765 in
https://github.com/pybind/pybind11/pull/3237 to reduce the amount of
TLS storage used.
The shared TLS key is stored in two different ways, depending on
`PYBIND11_INTERNALS_VERSION`. If `PYBIND11_INTERNALS_VERSION ==
4` (as is currently set), the TLS key is stored in the
`internal::shared_data` map to avoid breaking ABI compatibility. If
`PYBIND11_INTERNALS_VERSION > 4`, the TLS key is stored directly in
the `internals` struct.
* Fix test_pytypes.py::test_issue2361 failure on PyPy3.7
* Add github actions tests for unstable ABI
* Restoring `const` removed from pytypes.h in PR #3254, adding tests reflective of user code that breaks when those `const` are removed.
* clang-tidy NOLINTs (and one collateral fix).
* Inserting PYBIND11_CONST_FOR_STRICT_PLATFORMS
* Trying `defined(__APPLE__)`
* Trying again: `auto it` for strict platforms.
* Adding NOLINTNEXTLINE(bugprone-macro-parentheses), expanding comments.
* Labeling all changes with `PR #3263`, for easy reference, and to make it easy to undo these changes if we decide to do so in the future.
* Fix thread safety for pybind11 loader_life_support
Fixes issue: https://github.com/pybind/pybind11/issues/2765
This converts the vector of PyObjects to either a single void* or
a per-thread void* depending on the WITH_THREAD define.
The new field is used by each thread to construct a stack
of loader_life_support frames that can extend the life of python
objects.
The pointer is updated when the loader_life_support object is allocated
(which happens before a call) as well as on release.
Each loader_life_support maintains a set of PyObject references
that need to be lifetime extended; this is done by storing them
in a c++ std::unordered_set and clearing the references when the
method completes.
* Also update the internals version as the internal struct is no longer compatible
* Add test demonstrating threading works correctly.
It may be appropriate to run this under msan/tsan/etc.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update test to use lifetime-extended references rather than
std::string_view, as that's a C++ 17 feature.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Make loader_life_support members private
* Update version to dev2
* Update test to use python threading rather than concurrent.futures
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove unnecessary env in test
* Remove unnecessary pytest in test
* Use native C++ thread_local in place of python per-thread data structures to retain compatability
* clang-format test_thread.cpp
* Add a note about debugging the py::cast() error
* thread_test.py now propagates exceptions on join() calls.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* remove unused sys / merge
* Update include order in test_thread.cpp
* Remove spurious whitespace
* Update comment / whitespace.
* Address review comments
* lint cleanup
* Fix test IntStruct constructor.
* Add explicit to constructor
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
* Set __file__ constant when using eval_file
* Use const ref
* Use a move instead
* Revert
* Improve test
* Guard test with Python version
* Fix tests
* Dont support Python2 API
* Drop Python2 eval __file__ support
* Hack
* Semisupport Python2
* Take2
* Remove Python2 support
* Minor tweaks.
* Restoring tests/pybind11_tests.h version from master, removing just the comment and empty line that was added in PR #3087; those were made obsolete by the pragma cleanup that concluded with PR #3186.
* More-to-the-point test for Python 3.
* Minor tweaks.
* Restoring tests/pybind11_tests.h version from master, removing just the comment and empty line that was added in PR #3087; those were made obsolete by the pragma cleanup that concluded with PR #3186.
* [ci skip] Restoring tests/test_enum.py from master.
* fix: memory leak in cpp_function (#3228)
* add a test case to check objects are deconstructed in cpp_function
* update the test case about cpp_function
* fix the test case about cpp_function: remove "noexcept"
* Actually calling func. CHECK(stat.alive() == 2); Manually verified that the new tests fails without the change in pybind11.h
* Moving new test to test_callbacks.cpp,py, with small enhancements.
* Removing new test from test_interpreter.cpp (after it was moved to test_callbacks.cpp,py). This restores test_interpreter.cpp to the current state on master.
* Using py::detail::silence_unused_warnings(py_func); to make the intent clear.
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
* Trivial change to avoid (ssize_t) cast.
* Demo for safe_ssize_t idea.
* Removing safe_ssize_t.cpp (proof-of-concept code) to not upset the GHA Format workflow.
* Completing changes in pytypes.h
* New ssize_t_cast (better replacement for safe_ssize_t).
* clang-format-diff (no manual changes).
* bytes_ssize_t -Wnarrowing reproducer (see PR #2692).
* Backing out tuple(), list() ssize_t support, for compatibility with older compilers (to resolve link failures).
* Bug fix: missing `py::` for `py::ssize_t`
* Restoring tuple(), list() ssize_t support, but passing `size` by value, for compatibility with older compilers (to resolve link failures).
* Full test coverage of all functions with modified signatures.
* Use equivalent_integer for enum's Scalar decision
* Add test for char underlying enum
* Support translating bool type in enum's Scalar
* Add test for bool underlying enum
* Fix comment in test
* Switch from `PYBIND11_CPP20` macro to `PYBIND11_HAS_U8STRING`
* Refine tests
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
* Call PySys_SetArgv when initializing interpreter.
* Document argc/argv parameters in initialize_interpreter.
* Remove manual memory management from set_interpreter_argv in favor of smart pointers.
* Use size_t for indexers in set_interpreter_argv.
* Minimize macros for flow control in set_interpreter_argv.
* Fix 'unused variable' warning on Py2
* whitespace
* Define wide_char_arg_deleter outside set_interpreter_argv.
* Do sys.path workaround in C++ rather than eval.
* Factor out wchar conversion to a separate function.
* Restore widened_argv variable declaration.
* Fix undeclared widened_arg variable on some paths.
* Use delete[] to match new wchar_t[].
* Fix compiler errors
* Use PY_VERSION_HEX for a cleaner CVE-2008-5983 mode check.
* Fix typo
* Use explicit type for deleter so delete[] works cross-compiler.
* Always use PySys_SetArgvEx because pybind11 doesn't support pythons that don't include it.
* Remove pointless ternary operator.
* Use unique_ptr.reset instead of a second initialization.
* Rename add_program_dir_to_path parameter to clarify intent.
* Add defined() check before evaluating HAVE_BROKEN_MBSTOWCS.
* Apply clang-tidy fixes
* Pre-commit
* refactor: use const for set_interpreter_argv
* Try to fix const issue and allocate vector properly
* fix: copy strings on Python 2
* Applying clang-format-diff relative to master.
The only manual change is an added empty line between pybind11 and system `#include`s.
```
git diff -U0 --no-color master | python3 $HOME/clone/llvm-project/clang/tools/clang-format/clang-format-diff.py -p1 -style=file -i
```
Co-authored-by: Boris Staletic <boris.staletic@gmail.com>
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
* Add py::raise_from to enable chaining exceptions on Python 3.3+
* Use 'raise from' in initialization
* Documenting the exact base version of _PyErr_FormatVFromCause, adding back `assert`s.
Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com>
* extend msys2 CI
- add 32-bit job
- add c++11/17 c++/interface tests copied from standard ci
- add numpy/scipy
* account for padding of PartialStruct in numpy dtypes test with mingw32
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* msys2 ci: add c++14 tests
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Removing all MSVC C4127 warning suppression pragmas.
* Removing MSVC /WX (WERROR). To get a full list of all warnings.
* Inserting PYBIND11_SILENCE_MSVC_C4127. Changing one runtime if to #if.
* Changing PYBIND11_SILENCE_MSVC_C4127 macro to use absolute namespace (for use outside pybind11 include directory).
* Restoring MSVC /WX (WERROR).
* Removing globally-scoped suppression for clang -Wunsequenced. Based on an experiment under PR #3202 it is obsolete and can simply be removed.
* Apply isort
* Tweak isort config
* Add env.py as a known_first_party
* Add one missing known first party
* Make config compat with older isort versions
* Add another comment
* Revert pyproject setting
* test_eval: Show example of working closure
* Extend test_eval_closure with weirder examples of closures for py::eval
Co-authored-by: Yannick Jadoul <yannick.jadoul@belgacom.net>
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
* mingw64 platform string is like mingw_xxx not "mingw"
See https://github.com/msys2/MINGW-packages/blob/master/mingw-w64-python/0099-Change-the-get_platform-method-in-sysconfig-and-dist.patch
* Mingw: Do not dllexport exceptions
This is a fix for errors like:
D:/a/pybind11/pybind11/include/pybind11/detail/common.h:735:23: error: 'dllexport' implies default visibility, but 'class pybind11::builtin_exception' has already been declared with a different visibility
735 | class PYBIND11_EXPORT builtin_exception : public std::runtime_error {
| ^~~~~~~~~~~~~~~~~
* GHA: Test Mingw64 build
* fix: avoid thin binaries on mingw
* fix: drop lto on MinGW
* Mingw64: disable PYBIND11_DEPRECATED
It trigger many warnings for unknown reasons
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
* Fixing `pragma warning pop` for `__INTEL_COMPILER`.
* Adding push/pop to 3 tests. Removing #878 from top of pybind11.h (it was/is only needed for 1 test).
* Trying again after CI failure, moving the push to the top of 2 tests.
* Trying more after CI failure, adding push/pop to pybind11_tests.h, constructor_stats.h.
* Moving ICC #2196 suppression to CMakeLists.txt
* Fixing condition for `pragma GCC diagnostic push` in pybind11.h
* Moving `pragma warning disable 2196` to common.h
* Revising #ifdef to be more conservative.
* Undoing insertion of notes that will hopefully soon be completely obsolete anyway.
* maint(clang-tidy) Improve code readability
* Fix minor typos
* Revert optimization that removed test case
* Fix comment formatting
* Revert another optimization to repro an issue
* Remove make_unique since it C++14 and newer only
* eformat comments
* Fix unsignedness of comparison
* Update comment
* Allow python builtins to be used as callbacks
* Try to fix pypy segfault
* Add expected fail for PyPy
* Fix typo
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add more info to xfail
* Add env
* Try returning false
* Try removing the move for pypy
* Fix bugs
* Try removing move
* Just keep ignoring for PyPy
* Add back xfail
* Fix ctors
* Revert change of std::move
* Change to skip
* Fix bug and edit comments
* Remove clang-tidy bugprone fix skip bug
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Create a module_internals struct
Since we now have two things that are going to be module local, it felt
correct to add a struct to manage them.
* Add local exception translators
These are added via the register_local_exception_translator function
and are then applied before the global translators
* Add unit tests to show the local exception translator works
* Fix a bug in the unit test with the string value of KeyError
* Fix a formatting issue
* Rename registered_local_types_cpp()
Rename it to get_registered_local_types_cpp() to disambiguate from the
new member of module_internals
* Add additional comments to new local exception code path
* Add a register_local_exception function
* Add additional unit tests for register_local_exception
* Use get_local_internals like get_internals
* Update documentation for new local exception feature
* Add back a missing space
* Clean-up some issues in the docs
* Remove the code duplication when translating exceptions
Separated out the exception processing into a standalone function in the
details namespace.
Clean-up some comments as per PR notes as well
* Remove the code duplication in register_exception
* Cleanup some formatting things caught by clang-format
* Remove the templates from exception translators
But I added a using declaration to alias the type.
* Remove the extra local from local_internals variable names
* Add an extra explanatory comment to local_internals
* Fix a typo in the code
* Raise codec errors when casting to std::string
Allow the codec's exception to be raised instead of RuntimeError when
casting from py::str to std::string.
PY2 allows ucs surrogates in UTF-8 conversion
Signed-off-by: Shane Loretz <sloretz@openrobotics.org>
Signed-off-by: Shane Loretz <sloretz@osrfoundation.org>
* Attempt to fix py2 error
* Revert all unicode literals
* Fixed
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
* Manually moving `// NOLINT` comments so that clang-format does not move them to the wrong places.
* Manually reformatting comments related to `static_assert`s so that clang-format does not need two passes.
* Empty lines between #includes, to prevent clang-format from shuffling the order and thereby confusing MSVC 2015.
* git diff -U0 --no-color HEAD^ | python3 $HOME/clone/llvm-project/clang/tools/clang-format/clang-format-diff.py -p1 -style=file -i
For single-file extensions, a convenient pattern offered by cython
is to place the source files directly in the python source tree
(`foo/__init__.py`, `foo/ext.pyx`), deriving the package names from
their filesystem location. Adapt this pattern for pybind11, using an
`intree_extensions` helper, which should be thought of as the moral
equivalent to `cythonize`.
Differences with cythonize: I chose not to include globbing support
(`intree_extensions(glob.glob("**/*.cpp"))` seems sufficient), nor to
provide extension-customization kwargs (directly setting the attributes
on the resulting Pybind11Extension objects seems sufficient).
We could choose to have `intree_extension` (singular instead) and make
users write `[*map(intree_extension, glob.glob("**/*.cpp"))]`; no strong
opinion here.
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
* Adding test_return_vector_bool_raw_ptr to test_stl.py.
* First attempt to make the documentation more accurate, but not trying to be comprehensive, to not bloat the reference table with too many details.
* Fixing minor oversights.
* Applying reviewer suggestion.
* Adding iostream.h thread-safety documentation.
* Restoring `TestThread` code with added `std::lock_guard<std::mutex>`.
* Updating new comments to reflect new information.
* Fixing up `git rebase -X theirs` accidents.
* Splitting out pybind11/stl/filesystem.h.
To solve breakages like: https://github.com/deepmind/open_spiel/runs/2999582108
Mostly following the suggestion here: https://github.com/pybind/pybind11/pull/2730#issuecomment-750507575
Except using pybind11/stl/filesystem.h instead of pybind11/stlfs.h, as decided via chat.
stl.h restored to the exact state before merging PR #2730 via:
```
git checkout 733f8de24f stl.h
```
* Properly including new stl subdirectory in pip wheel config.
This now passes interactively:
```
pytest tests/extra_python_package/
```
* iwyu cleanup.
iwyuh.py -c -std=c++17 -DPYBIND11_TEST_BOOST -Ipybind11/include -I/usr/include/python3.9 -I/usr/include/eigen3 include/pybind11/stl/filesystem.h
* Adding PYBIND11_HAS_FILESYSTEM_IS_OPTIONAL.
* Eliminating else after return.
* Unify cast_error message thrown by [simple|unpacking]_collector
simple_collector and unpacking_collector throw different error messages
when the casting of an argument failed: While the former mentions make_tuple(),
the latter emphasises the call argument (and its name/position).
* Consolidating "Unable to convert call argument" error reporting code to guarantee uniformity.
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
* pickle setstate: setattr __dict__ only if not empty, to not force use of py::dynamic_attr() unnecessarily.
* Adding unit test.
* Clang 3.6 & 3.7 compatibility.
* PyPy compatibility.
* Minor iwyu fix, additional comment.
* Addressing reviewer requests.
* Applying clang-tidy suggested fixes.
* Adding check_dynamic_cast_SimpleCppDerived, related to issue #3062.
* Add const T to docstring generation.
* Change order.
* See if existing test triggers for a const type.
* Add tests.
* Fix test.
* Remove experiment.
* Reformat.
* More tests, checks run.
* Adding `test_fmt_desc_` prefix to new test functions.
* Using pytest.mark.parametrize to 1. condense test; 2. exercise all functions even if one fails; 3. be less platform-specific (e.g. C++ float is not necessarily float32).
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
* Crash when printing Unicode to redirected cout
Add failing tests
* Fix Unicode crashes redirected cout
* pythonbuf::utf8_remainder check end iterator
* Remove trailing whitespace and formatting iostream
* Avoid buffer overflow if ostream redirect races
This doesn't solve the actual race, but at least it now has a much lower
probability of reading past the end of the buffer even when data races
do occur.
* Adding PyGILState_Check() in object_api<>::operator().
* Enabling PyGILState_Check() for Python >= 3.6 only.
Possibly, this explains why PyGILState_Check() cannot safely be used with Python 3.4 and 3.5:
https://github.com/python/cpython/pull/10267#issuecomment-434881587
* Adding simple micro benchmark.
* Reducing test time to minimum (purely for coverage, not for accurate results).
* Fixing silly oversight.
* Minor code organization improvement in test.
* Adding example runtimes.
* Removing capsys (just run with `-k test_callback_num_times -s` and using `.format()`.
* Use correct duration representation when casting from datetime.timdelta to std::chrono::duration
* When asserting datetime/timedelta/date/time we can equality-compare whole objects
* [dtype]: add type() method to access type attribute of PyArray_Descr (eq. to dtype.char in Python)
* [dtype] change type() name method to char_() to be compliant with Python numpy interface
* [dtype] fix by pre-commit
* [dtype] Change comments and solutions format for test
* Clarify documentation and move note about dtype.char vs PyArray_Descr::type to a plain, non-doxygen comment
* Fix and extend tests
* Fix the supposedly fixed tests
* Fix the fixed tests again
Co-authored-by: Bertrand MICHEL <bertrand.michel@onera.fr>
Co-authored-by: Yannick Jadoul <yannick.jadoul@belgacom.net>
* Make sure all warnings in pytest get turned into errors
* Suppress DeprecationWarnings in test_int_convert and test_numpy_int_convert
* PyLong_AsLong only shouts "Deprecated!" on Python>=3.8
* Fix remaining warnings on PyPy and CPython 3.10-dev
* Demonstrate issue with weakref constructor overloads
* Fix weakref constructor to convert on being passed a non-weakref object
* Improve on nonlocal-scoped variable in test_weakref
* Keep backwards-compatibility by introducing PYBIND11_OBJECT_CVT_DEFAULT macro
* Simplify test_weakref
* Always call PyNumber_Index when casting from Python to a C++ integral type, also pre-3.8
* Fixed on PyPy
* Simplify use of PyNumber_Index, following @rwgk's idea, and ignore warnings in >=3.8
* Reproduce mismatch between pre-3.8 and post-3.8 behavior on __index__ throwing TypeError
* Fix tests on 3.6 <= Python < 3.8
* No, I don't have an uninitialized variable
* Fix use of __index__ on Python 2
* Make types in test_int_convert more ~boring~ descriptive
* Force the builtin module key to be the correct type.
Previously it was always going to be a std::string which converted into
unicode. Python 2 appears to want module keys to be normal str types, so
this was breaking code that expected plain string types in the
builtins.keys() data structure
* Add a simple unit test to ensure all built-in keys are str
* Update the unit test so it will also run on pypy
* Run pre-commit.
Co-authored-by: Jesse Clemens <jesse.clemens@sony.com>
* CI: Intel icc/icpc via oneAPI
Add testing for Intel icc/icpc via the oneAPI images.
Intel oneAPI is in a late beta stage, currently shipping
oneAPI beta09 with ICC 20.2.
CI: Skip Interpreter Tests for Intel
Cannot find how to add this, neiter the package `libc6-dev` nor
`intel-oneapi-mkl-devel` help when installed to solve this:
```
-- Looking for C++ include pthread.h
-- Looking for C++ include pthread.h - not found
CMake Error at /__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:165 (message):
Could NOT find Threads (missing: Threads_FOUND)
Call Stack (most recent call first):
/__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:458 (_FPHSA_FAILURE_MESSAGE)
/__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindThreads.cmake:234 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
tests/test_embed/CMakeLists.txt:17 (find_package)
```
CI: libc6-dev from GCC for ICC
CI: Run bare metal for oneAPI
CI: Ubuntu 18.04 for oneAPI
CI: Intel +Catch -Eigen
CI: CMake from Apt (ICC tests)
CI: Replace Intel Py with GCC Py
CI: Intel w/o GCC's Eigen
CI: ICC with verbose make
[Debug] Find core dump
tests: use arg{} instead of arg() for Intel
tests: adding a few more missing {}
fix: sync with @tobiasleibner's branch
fix: try ubuntu 20-04
fix: drop exit 1
docs: Apply suggestions from code review
Co-authored-by: Tobias Leibner <tobias.leibner@googlemail.com>
Workaround for ICC enable_if issues
Another workaround for ICC's enable_if issues
fix error in previous commit
Disable one test for the Intel compiler in C++17 mode
Add back one instance of py::arg().noconvert()
Add NOLINT to fix clang-tidy check
Work around for ICC internal error in PYBIND11_EXPAND_SIDE_EFFECTS in C++17 mode
CI: Intel ICC with C++17
docs: pybind11/numpy.h does not require numpy at build time. (#2720)
This is nice enough to be mentioned explicitly in the docs.
docs: Update warning about Python 3.9.0 UB, now that 3.9.1 has been released (#2719)
Adjusting `type_caster<std::reference_wrapper<T>>` to support const/non-const propagation in `cast_op`. (#2705)
* Allow type_caster of std::reference_wrapper<T> to be the same as a native reference.
Before, both std::reference_wrapper<T> and std::reference_wrapper<const T> would
invoke cast_op<type>. This doesn't allow the type_caster<> specialization for T
to distinguish reference_wrapper types from value types.
After, the type_caster<> specialization invokes cast_op<type&>, which allows
reference_wrapper to behave in the same way as a native reference type.
* Add tests/examples for std::reference_wrapper<const T>
* Add tests which use mutable/immutable variants
This test is a chimera; it blends the pybind11 casters with a custom
pytype implementation that supports immutable and mutable calls.
In order to detect the immutable/mutable state, the cast_op needs
to propagate it, even through e.g. std::reference<const T>
Note: This is still a work in progress; some things are crashing,
which likely means that I have a refcounting bug or something else
missing.
* Add/finish tests that distinguish const& from &
Fixes the bugs in my custom python type implementation,
demonstrate test that requires const& and reference_wrapper<const T>
being treated differently from Non-const.
* Add passing a const to non-const method.
* Demonstrate non-const conversion of reference_wrapper in tests.
Apply formatting presubmit check.
* Fix build errors from presubmit checks.
* Try and fix a few more CI errors
* More CI fixes.
* More CI fixups.
* Try and get PyPy to work.
* Additional minor fixups. Getting close to CI green.
* More ci fixes?
* fix clang-tidy warnings from presubmit
* fix more clang-tidy warnings
* minor comment and consistency cleanups
* PyDECREF -> Py_DECREF
* copy/move constructors
* Resolve codereview comments
* more review comment fixes
* review comments: remove spurious &
* Make the test fail even when the static_assert is commented out.
This expands the test_freezable_type_caster a bit by:
1/ adding accessors .is_immutable and .addr to compare identity
from python.
2/ Changing the default cast_op of the type_caster<> specialization
to return a non-const value. In normal codepaths this is a reasonable
default.
3/ adding roundtrip variants to exercise the by reference, by pointer
and by reference_wrapper in all call paths. In conjunction with 2/, this
demonstrates the failure case of the existing std::reference_wrpper conversion,
which now loses const in a similar way that happens when using the default cast_op_type<>.
* apply presubmit formatting
* Revert inclusion of test_freezable_type_caster
There's some concern that this test is a bit unwieldly because of the use
of the raw <Python.h> functions. Removing for now.
* Add a test that validates const references propagation.
This test verifies that cast_op may be used to correctly detect
const reference types when used with std::reference_wrapper.
* mend
* Review comments based changes.
1. std::add_lvalue_reference<type> -> type&
2. Simplify the test a little more; we're never returning the ConstRefCaster
type so the class_ definition can be removed.
* formatted files again.
* Move const_ref_caster test to builtin_casters
* Review comments: use cast_op and adjust some comments.
* Simplify ConstRefCasted test
I like this version better as it moves the assertion that matters
back into python.
ci: drop pypy2 linux, PGI 20.7, add Python 10 dev (#2724)
* ci: drop pypy2 linux, add Python 10 dev
* ci: fix mistake
* ci: commented-out PGI 20.11, drop 20.7
fix: regression with installed pybind11 overriding local one (#2716)
* fix: regression with installed pybind11 overriding discovered one
Closes#2709
* docs: wording incorrect
style: remove redundant instance->owned = true (#2723)
which was just before set to True in instance->allocate_layout()
fix: also throw in the move-constructor added by the PYBIND11_OBJECT macro, after the argument has been moved-out (if necessary) (#2701)
Make args_are_all_* ICC workarounds unconditional
Disable test_aligned on Intel ICC
Fix test_aligned on Intel ICC
Skip test_python_alreadyset_in_destructor on Intel ICC
Fix test_aligned again
ICC CI: Downgrade pytest
pytest 6 does not capture the `discard_as_unraisable` stderr and
just writes a warning with its content instead.
* refactor: simpler Intel workaround, suggested by @laramiel
* fix: try version with impl to see if it is easier to compile
* docs: update README for ICC
Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
* Only allow integer type_caster to call __int__ or __index__ method when conversion is allowed
* Remove tests for __index__ as this seems to only be used to convert to int in 3.8+
* Take both `int` and `long` types into account for Python 2
* Add test_numpy_int_convert to assert tests currently fail, even though np.intc has an __index__ method
* Also consider __index__ as noconvert to a C++ integer
* New-style classes for Python 2.7; sigh
* Add some tests on types with custom __index__ method
* Ignore some tests in Python <3.8
* Update comment about conversion from np.float32 to C++ int
* Workaround difference between CPython and PyPy's different PyIndex_Check (unnoticed because we currently don't have PyPy >= 3.8)
* Avoid ICC segfault with py::arg()
* CI: Intel icc/icpc via oneAPI
Add testing for Intel icc/icpc via the oneAPI images.
Intel oneAPI is in a late beta stage, currently shipping
oneAPI beta09 with ICC 20.2.
* CI: Skip Interpreter Tests for Intel
Cannot find how to add this, neiter the package `libc6-dev` nor
`intel-oneapi-mkl-devel` help when installed to solve this:
```
-- Looking for C++ include pthread.h
-- Looking for C++ include pthread.h - not found
CMake Error at /__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:165 (message):
Could NOT find Threads (missing: Threads_FOUND)
Call Stack (most recent call first):
/__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:458 (_FPHSA_FAILURE_MESSAGE)
/__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindThreads.cmake:234 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
tests/test_embed/CMakeLists.txt:17 (find_package)
```
* CI: libc6-dev from GCC for ICC
* CI: Run bare metal for oneAPI
* CI: Ubuntu 18.04 for oneAPI
* CI: Intel +Catch -Eigen
* CI: CMake from Apt (ICC tests)
* CI: Replace Intel Py with GCC Py
* CI: Intel w/o GCC's Eigen
* CI: ICC with verbose make
* [Debug] Find core dump
* tests: use arg{} instead of arg() for Intel
* tests: adding a few more missing {}
* fix: sync with @tobiasleibner's branch
* fix: try ubuntu 20-04
* fix: drop exit 1
* style: clang tidy fix
* style: fix missing NOLINT
* ICC: Update Compiler Name
Changed upstream with the last oneAPI release.
* ICC CI: Downgrade pytest
pytest 6 does not capture the `discard_as_unraisable` stderr and
just writes a warning with its content instead.
* Use new test pinning requirements.txt
* tests: add notes about intel, cleanup
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
* Adding a valgrind build on debug Python 3.9
Co-authored-by: Boris Staletic <boris.staletic@gmail.com>
* Add Valgrind suppression files
- Introduce suppression file, populate it with a first suppression taken from CPython, and fix one leak in the tests
- Suppress leak in NumPy
- More clean tests!
- Tests with names a-e passing (except for test_buffer)
- Suppress multiprocessing errors
- Merge multiprocessing suppressions into other suppression files
- Numpy seems to be spelled with a big P
- Append single entry from valgrind-misc.supp to valgrind-python.supp, and make clear valgrind-python.supp is only CPython
Co-authored-by: Boris Staletic <boris.staletic@gmail.com>
* Enable test_virtual_functions with a workaround
* Add a memcheck cmake target
- Add a memcheck cmake target
- Reformat cmake
- Appease the formatting overlords - they are angry
- Format CMake valgrind target decently
* Update CI config to new action versions
* fix: separate memcheck from pytest
* ci: cleanup
* Merge Valgrind and other deadsnakes builds
Co-authored-by: Boris Staletic <boris.staletic@gmail.com>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
* Plug leaking function_record objects when exceptions are thrown
* Plug leak of strdup'ed strings in function_record
* Some extra comments about the function_record ownership dance
* Clean up the function_record better, in case of exceptions
* Demonstrate some extra function_record leaks
* Change DeleteStrings template argument to free_strings runtime argument in destruct(function_record *)
* Zero-state unique_function_record deleter object
* Clarify rvalue reference to unique_ptr parameter in initialize_generic
* Use push_back with const char * instead of emplace_back
* Ignore old-style __init__ warnings
* Simplify ignoreOldStyleInitWarnings with py::exec
* Only wrap single class_::defs to ignore DeprecationWarnings about old-style __init__
* Fix leak in the test_copy_move::test_move_fallback
* Fix leaking PyMethodDef in test_class::test_implicit_conversion_life_support
* Plumb leak in test_buffer, occuring when a mutable buffer is requested for a read-only object, and enable test_buffer.py
* Fix weird return_value_policy::reference in test_stl_binders, and enable those tests
* Cleanup nodelete holder objects in test_smart_ptr, and enable those tests
* Update pytest to 6.2.1 in tests/requirements.txt
* Pin pytest to last supported version for 3.5
* Suppress PytestUnraisableExceptionWarning and use sys.__unraisablehook__ instead of sys.unraisablehook
* Fix filterwarnings mark on old pytest and old Python versions
* Cleanup ignore_pytest_unraisable_warning decorator
* minor cleanup: fixing or silencing flake8 errors
* ci: lock CMake to non-Universal version
* Update .github/workflows/ci.yml
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
* Allow type_caster of std::reference_wrapper<T> to be the same as a native reference.
Before, both std::reference_wrapper<T> and std::reference_wrapper<const T> would
invoke cast_op<type>. This doesn't allow the type_caster<> specialization for T
to distinguish reference_wrapper types from value types.
After, the type_caster<> specialization invokes cast_op<type&>, which allows
reference_wrapper to behave in the same way as a native reference type.
* Add tests/examples for std::reference_wrapper<const T>
* Add tests which use mutable/immutable variants
This test is a chimera; it blends the pybind11 casters with a custom
pytype implementation that supports immutable and mutable calls.
In order to detect the immutable/mutable state, the cast_op needs
to propagate it, even through e.g. std::reference<const T>
Note: This is still a work in progress; some things are crashing,
which likely means that I have a refcounting bug or something else
missing.
* Add/finish tests that distinguish const& from &
Fixes the bugs in my custom python type implementation,
demonstrate test that requires const& and reference_wrapper<const T>
being treated differently from Non-const.
* Add passing a const to non-const method.
* Demonstrate non-const conversion of reference_wrapper in tests.
Apply formatting presubmit check.
* Fix build errors from presubmit checks.
* Try and fix a few more CI errors
* More CI fixes.
* More CI fixups.
* Try and get PyPy to work.
* Additional minor fixups. Getting close to CI green.
* More ci fixes?
* fix clang-tidy warnings from presubmit
* fix more clang-tidy warnings
* minor comment and consistency cleanups
* PyDECREF -> Py_DECREF
* copy/move constructors
* Resolve codereview comments
* more review comment fixes
* review comments: remove spurious &
* Make the test fail even when the static_assert is commented out.
This expands the test_freezable_type_caster a bit by:
1/ adding accessors .is_immutable and .addr to compare identity
from python.
2/ Changing the default cast_op of the type_caster<> specialization
to return a non-const value. In normal codepaths this is a reasonable
default.
3/ adding roundtrip variants to exercise the by reference, by pointer
and by reference_wrapper in all call paths. In conjunction with 2/, this
demonstrates the failure case of the existing std::reference_wrpper conversion,
which now loses const in a similar way that happens when using the default cast_op_type<>.
* apply presubmit formatting
* Revert inclusion of test_freezable_type_caster
There's some concern that this test is a bit unwieldly because of the use
of the raw <Python.h> functions. Removing for now.
* Add a test that validates const references propagation.
This test verifies that cast_op may be used to correctly detect
const reference types when used with std::reference_wrapper.
* mend
* Review comments based changes.
1. std::add_lvalue_reference<type> -> type&
2. Simplify the test a little more; we're never returning the ConstRefCaster
type so the class_ definition can be removed.
* formatted files again.
* Move const_ref_caster test to builtin_casters
* Review comments: use cast_op and adjust some comments.
* Simplify ConstRefCasted test
I like this version better as it moves the assertion that matters
back into python.
* Adding missing virtual destructors, to silence clang -Wnon-virtual-dtor warnings.
Tested with clang version 9.0.1-12 under an Ubuntu-like OS.
Originally discovered in the Google-internal environment.
* adding -Wnon-virtual-dtor for GNU|Intel|Clang
* demo kwarg with none(false)
* Reorder and extend tests for arg::none(false) in test_methods_and_attributes.py::test_accepts_none
* Fix arg::none() for keyword arguments
* Add changelog note
* Fix names of no_none_kw test functions
Co-authored-by: Yannick Jadoul <yannick.jadoul@belgacom.net>
* Demonstrate test_factory_constructors.py failure without functional changes from #2335
* Revert "Demonstrate test_factory_constructors.py failure without functional changes from #2335"
This reverts commit ca33a8021fc2a3617c3356b188796528f4594419.
* Fix test crash where registered Python type gets garbage collected
* Clean up some more internal structures when class objects go out of scope
* Reduce length of std::erase_if-in-C++20 comment
* Clean up code for cleaning up type internals
* Move cleaning up of type info in internals to tp_dealloc on pybind11_metaclass
* Deprecated public constructors of module
* Turn documentation comment of module_::add_object into valid doxygen documentation
* Move definition of PYBIND11_DETAIL_MODULE_STATIC_DEF and PYBIND11_DETAIL_MODULE_CREATE macros up
* Move detail::create_top_level_module to module_::create_extension_module, and unify Python 2 and 3 signature again
* Throw error_already_set if module creation fails in module_::create_extension_module
* Mention module_::create_extension_module in deprecation warning message of module_::module_
* Check scope's __dict__ instead of using hasattr when registering classes and exceptions, to allow registering the same name in a derived class scope
* Extend test_base_and_derived_nested_scope test
* Add tests on error being thrown registering duplicate classes
* Circumvent bug with combination of test_class.py::test_register_duplicate_class and test_factory_constructors.py::test_init_factory_alias
* tests: Don't run tests that often segfault
* tests: drop all cross module gil tests
* tests: try skipping all macOS Python 3.9 tests
* tests: drop macOS Python 3.9
* Fail on passing py::object with wrong Python type to py::object subclass using PYBIND11_OBJECT macro
* Split off test_non_converting_constructors from test_constructors
* Fix test_as_type, as py::type constructor now throws an error itself if the argument is not a type
* Replace tp_name access by pybind11::detail::get_fully_qualified_tp_name
* Move forward-declaration of get_fully_qualified_tp_name to detail/common.h
* Don't add the builtins module name in get_fully_qualified_tp_name for PyPy
* Add PYBIND11_BUILTINS_MODULE macro, and use it in get_fully_qualified_tp_name
* Remove code inside 'PYPY_VERSION_NUM < 0x06000000' preprocessor if branch
* fix: more cleanup
* Remove more references to PyPy 5.7 and 5.9 in the docs
* Update comment on PyUnicode_UTF* in PyPy
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
* tests: New test for ctypes buffers (pybind#2502)
* fix: fix buffer_info segfault on views with no stride (pybind11#2502)
* Explicit conversions in buffer_info to make clang happy (pybind#2502)
* Another explicit cast in buffer_info constructor for clang (pybind#2502)
* Simpler implementation of buffer_info constructor from Py_buffer.
* Move test_ctypes_buffer into test_buffers
* Comment on why view->strides may be NULL (and fix some whitespace)
* Use c_strides() instead of zero when view->strides is NULL.
c_strides and f_strides are moved from numpy.h (py::array)
to buffer_info.h (py::detail) so they can be used from the
buffer_info Py_buffer constructor.
* Increase ctypes buffer test coverage in test_buffers.
* Split ctypes tests and skip one which is broken in PyPy2.
* Allow function/functor passed to py::vectorize to return void
* Stealing @sizmailov's test and fixing unused argument warning
* Add missing std::move()
RVO doesn't work here because function return type is different from
actual returned type
* remove extra EOL
* docs: add a few details
* chore: pre-commit autoupdate
* Remove array_iterator, array_begin, and array_end (in detail namespace)
Co-authored-by: Sergei Izmailov <sergei.a.izmailov@gmail.com>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
* Add tests demonstrating the problem with deregistering pybind11 instances
* Fix deregistering of different pybind11 instance from internals
Co-authored-by: Yannick Jadoul <yannick.jadoul@belgacom.net>
Co-authored-by: Blistic <wots_wot@hotmail.com>
