Commit Graph

2551 Commits

Author SHA1 Message Date
Ralf W. Grosse-Kunstleve
9a2963734d
More systematic gcc & clang coverage (#4083)
* More systematic gcc coverage, based on https://github.com/pybind/pybind11/pull/4074#issuecomment-1188385580

* Fix complete fail.

* Resolve GCC 11 & 12 "redundant move in return statement" warnings.

* Also add clang 11, 12, 13 (to gather info for warning suppressions).

* Add & use `PYBIND11_DETECTED_CLANG_WITH_MISLEADING_CALL_STD_MOVE_EXPLICITLY_WARNING`
2022-07-21 06:40:34 -07:00
Ralf W. Grosse-Kunstleve
cb35a3c143
For PyPy only, re-enable old behavior (runs the risk of masking bugs) (#4079)
* For PyPy only, re-enable old behavior (likely to mask bugs), to avoid segfault with unknown root cause.

Change prompted by https://github.com/pybind/pybind11/issues/4075

* Undo the change in tests/test_exceptions.py

I turns out (I forgot) that PyPy segfaults in `test_flaky_exception_failure_point_init` already before the `MISMATCH` code path is reached:

https://github.com/pybind/pybind11/runs/7383663596

```
RPython traceback:
test_exceptions.py .......X.........Error in cpyext, CPython compatibility layer:
  File "pypy_module_cpyext.c", line 14052, in wrapper_second_level__star_3_1
  File "pypy_module_cpyext_1.c", line 35750, in not_supposed_to_fail
Fatal Python error: Segmentation fault
Stack (most recent call first, approximate line numbers):
  File "/home/runner/work/pybind11/pybind11/tests/test_exceptions.py", line 306 in test_flaky_exception_failure_point_init
The function PyErr_NormalizeException was not supposed to fail
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/python.py", line 185 in pytest_pyfunc_call
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_callers.py", line 9 in _multicall
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_manager.py", line 77 in _hookexec
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_hooks.py", line 244 in __call__
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/python.py", line 1716 in runtest
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/runner.py", line 159 in pytest_runtest_call
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_callers.py", line 9 in _multicall
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_manager.py", line 77 in _hookexec
Fatal error in cpyext, CPython compatibility layer, calling PyErr_NormalizeException
Either report a bug or consider not using this particular extension
<SystemError object at 0x7fcc8cea6868>
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_hooks.py", line 244 in __call__
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/runner.py", line 261 in <lambda>
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/runner.py", line 317 in from_call
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/runner.py", line 246 in call_runtest_hook
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/runner.py", line 218 in call_and_report
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/runner.py", line 118 in runtestprotocol
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/runner.py", line 110 in pytest_runtest_protocol
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_callers.py", line 9 in _multicall
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_manager.py", line 77 in _hookexec
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_hooks.py", line 244 in __call__
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/main.py", line 335 in pytest_runtestloop
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_callers.py", line 9 in _multicall
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_manager.py", line 77 in _hookexec
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_hooks.py", line 244 in __call__
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/main.py", line 318 in _main
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/main.py", line 255 in wrap_session
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/main.py", line 314 in pytest_cmdline_main
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_callers.py", line 9 in _multicall
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_manager.py", line 77 in _hookexec
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_hooks.py", line 244 in __call__
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/config/__init__.py", line 133 in main
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/config/__init__.py", line 181 in console_main
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pytest/__main__.py", line 1 in <module>
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/lib-python/3/runpy.py", line 62 in _run_code
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/lib-python/3/runpy.py", line 170 in _run_module_as_main
  File "<builtin>/app_main.py", line 109 in run_toplevel
  File "<builtin>/app_main.py", line 652 in run_command_line
  File "<builtin>/app_main.py", line 996 in entry_point
Segmentation fault (core dumped)
```

* Add test_pypy_oserror_normalization

* Disable new `PYPY_VERSION` `#if`, to verify that the new test actually fails.

* Restore PYPY_VERSION workaround and update comment to reflect what was learned.

* [ci skip] Fix trivial oversight in comment.
2022-07-21 06:38:00 -07:00
Aaron Gokaslan
42b54507ea
chore: use explicit defaulting in pyobject macros (#4017)
* Use equals default in pyobject macros

* Remove extra semicolon

* Update clang-tidy equals-default rule to not ignore macros

* Fix formatting

* One last formatting change
2022-07-20 12:02:20 -04:00
Aaron Gokaslan
f47f1edfe8
Fix #3812 and fix const of inplace assignments (#4065)
* Fix #3812 and fix const of inplace assignments

* Fix missing tests

* Revert operator overloading changes

* calculate answer first for tests

* Simplify tests

* Add more tests

* Add a couple more tests

* Add test_inplace_lshift, test_inplace_rshift for completeness.

* Update tests

* Shortcircuit on self assigment and address reviewer comment

* broaden skip for self assignment

* One more reviewer comment

* Document opt behavior and make consistent

* Revert unnecessary change

* Clarify comment

Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
2022-07-20 11:42:24 -04:00
pre-commit-ci[bot]
ef7d971e03
[pre-commit.ci] pre-commit autoupdate (#4082)
updates:
- [github.com/hadialqattan/pycln: v2.0.1 → v2.0.4](https://github.com/hadialqattan/pycln/compare/v2.0.1...v2.0.4)
- [github.com/PyCQA/pylint: v2.14.4 → v2.14.5](https://github.com/PyCQA/pylint/compare/v2.14.4...v2.14.5)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2022-07-18 19:39:10 -05:00
Henry Schreiner
59f03ee389 tests: include pypy3.9 in nox if present
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
2022-07-15 22:22:51 -05:00
Henry Schreiner
5116a629e9 fix(spelling): PYTHON_VERSIONS 2022-07-15 22:22:51 -05:00
Henry Schreiner
8d82f29887 chore: back to work
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
2022-07-15 22:22:51 -05:00
Henry Schreiner
aa304c9c7d
chore: preapre for 2.10.0 release (#4068)
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
2022-07-15 15:47:47 -05:00
Henry Schreiner
87f64c4321
docs: update changelog for 2.10.0 (#4066) 2022-07-15 13:45:16 -05:00
Ralf W. Grosse-Kunstleve
1d81191077
Disable implicit conversion of 0 to pybind11::handle. (#4008)
* Disable implicit conversion from `0` to `pybind11::handle`.

* Reverse or-ed condition in an attempt to resolve GCC 8.3.0 errors (i386/debian:buster).

* Trying the simpler `std::is_same<T, PyObject *>`

* Add implicit_conversion_from_pytorch_THPObjectPtr_to_handle test.

* Accommodate types with implicit conversions to `PyObject *`, other than `handle` & `handle` subclasses, or integral types.

* Fix copy-paste mishap (picked wrong name).

* Revamp SFINAE construct to actually fix the pytorch issue (already validated against pytorch proper).

The first version of the reduced pytorch code was critically missing the move ctor. The first version of the accompanying test was meaningless.

Note: It turns out the `!std::is_arithmetic<T>` condition is not needed: `int` is not in general implicitly convertible to `PyObject *`, only the literal `0` is.

* Use `NOLINT(performance-noexcept-move-constructor)` for reduced code from the wild (rather than changing the code).

* Use any_of, all_of, negation. It turns out to clang-format nicer.

* Clean up comments for changed code.

* Reduce pytorch situation further, add test for operator ... const.

* Use `none_of` as suggested by @skylion007

* Add `pure_compile_tests_for_handle_from_PyObject_pointers()`

* Fix inconsequential oversight (retested).

* Factor our `is_pyobj_ptr_or_nullptr_t` to make the SFINAE conditions more readable.

* Remove stray line (oversight).

* Make the `pure_compile_tests_for_handle_from_PyObject_pointers()` "rhs-const-complete", too.

* Remove the temporary PYBIND11_UNDO_PR4008 `#ifdef`.
2022-07-14 09:53:39 -07:00
Aaron Gokaslan
bc9315fe38
chore: optimize sparse matrix casting with python tuple (#4064)
* optimize casting of sparse Eigen arrays

* move array

* Revert for safety
2022-07-13 19:27:00 -04:00
Henry Schreiner
6b60d97d30
docs: use Furo (#3109)
* docs: try using Furo

* docs: darker output

* docs: improve logo for dark background

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
2022-07-13 10:13:35 -05:00
Henry Schreiner
7c3a031766
chore: bump clang-tidy to 13 (#3997)
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
2022-07-13 09:06:32 -05:00
Ralf W. Grosse-Kunstleve
023b3f32c8
Undo accidental one-line change under PR #3913 (#4060) 2022-07-12 12:02:20 -07:00
pre-commit-ci[bot]
790241bce2
[pre-commit.ci] pre-commit autoupdate (#4058)
updates:
- [github.com/asottile/pyupgrade: v2.34.0 → v2.37.1](https://github.com/asottile/pyupgrade/compare/v2.34.0...v2.37.1)
- [github.com/hadialqattan/pycln: v1.3.5 → v2.0.1](https://github.com/hadialqattan/pycln/compare/v1.3.5...v2.0.1)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2022-07-11 22:31:54 -04:00
Aaron Gokaslan
f9f00495a3
Properly visit self in >=3.9 traverse (#4051)
* Properly visit self in >=3.9 traverse

* Add comment about 3.9 behavior
2022-07-09 21:43:53 -07:00
Ralf W. Grosse-Kunstleve
432bc5cfb5
Add std::string clean_type_id(const char *typeid_name) overload (in namespace detail). (#4049)
Very minor refactoring to ease development and debugging.

Having to declare a local `std::string` has bugged me many times. Nice to get this little nuisance out of the way.

Extracted from PR #4022, where it is used like this:

```
    std::fprintf(stdout,
                 "\nTYPE_CASTER_ODR_GUARD_IMPL %s %s\n",
                 clean_type_id(intrinsic_type_info.name()).c_str(),
                 source_file_line_from_sloc.c_str());
```
2022-07-07 19:08:10 -07:00
Ralf W. Grosse-Kunstleve
85bc088441
Report C++ Info: via pytest_report_header() (#4046)
* Report `C++ Info:` from `pytest_configure()`

* Use pytest_report_header() as suggested by @skylion007
2022-07-07 17:51:44 -07:00
Ralf W. Grosse-Kunstleve
cd08869df1
PYBIND11_NAMESPACE consistency fixes. (#4043) 2022-07-06 14:29:20 -07:00
Henry Schreiner
0ab1fcfb1c
docs: update changelog (#4042)
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
2022-07-06 16:36:05 -04:00
Aaron Gokaslan
2af163d9c7
Fix: 3.11 beta support (#3923)
* Placeholder commit for 3.11 testing

* Does this fix it?

* Try suggestion

* Placeholder commit for 3.11 testing

* Does this fix it?

* Try suggestion

* fix: try using modern init for embedded interp

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* fix: error message changed in 3.11

* fix: apply logic in Python manually

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* fix autodetect dynamic attrs in 3.11

* fix: include error message if possible in error

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* ci: enable standard Python 3.11 testing

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* Make dynamic attrs condtiion exclusive to ver.

Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
2022-07-06 16:35:12 -04:00
pre-commit-ci[bot]
c42e3ab793
[pre-commit.ci] pre-commit autoupdate (#4041)
* [pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/psf/black: 22.3.0 → 22.6.0](https://github.com/psf/black/compare/22.3.0...22.6.0)
- [github.com/Lucas-C/pre-commit-hooks: v1.2.0 → v1.3.0](https://github.com/Lucas-C/pre-commit-hooks/compare/v1.2.0...v1.3.0)
- [github.com/PyCQA/pylint: v2.14.3 → v2.14.4](https://github.com/PyCQA/pylint/compare/v2.14.3...v2.14.4)
- [github.com/pre-commit/mirrors-clang-format: v14.0.5 → v14.0.6](https://github.com/pre-commit/mirrors-clang-format/compare/v14.0.5...v14.0.6)

* Update blacken-docs

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
2022-07-04 22:21:59 -04:00
dependabot[bot]
bc1f9f9ba6
chore(deps): bump actions/setup-python from 3 to 4 (#3999)
* chore(deps): bump actions/setup-python from 3 to 4

Bumps [actions/setup-python](https://github.com/actions/setup-python) from 3 to 4.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Apply suggestions from code review

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
2022-07-04 10:24:55 -04:00
Henry Schreiner
5a3a1e3415
chore: simpler dependabot (#4035)
Ignores no longer needed after April 2022. Dependabot keeps the same style pinning now.
2022-07-04 10:24:34 -04:00
Matthew Woehlke
479e9a50f3
Fix arrays with zero-size dimensions (#4038)
When converting an array to an Eigen matrix, ignore the strides if any
dimension size is 0. If the array is empty, the strides aren't relevant,
and especially numpy ≥ 1.23 explicitly sets the strides to 0 in this
case. (See numpy commit dd5ab7b11520.)

Update tests to verify that this works, and continues to work.
2022-06-29 11:37:16 -07:00
Masaki Kozuki
374a5b000a
[docs] Fix "Enumerations and internal types" example (#4034)
* Fix binding of `Pet::Attributes`

* omit `attributes` as it's not needed

Signed-off-by: Masaki Kozuki <mkozuki@nvidia.com>
2022-06-28 12:38:37 -07:00
pre-commit-ci[bot]
dd3bf7fd12
[pre-commit.ci] pre-commit autoupdate (#4030)
updates:
- [github.com/hadialqattan/pycln: v1.3.3 → v1.3.5](https://github.com/hadialqattan/pycln/compare/v1.3.3...v1.3.5)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2022-06-27 19:39:19 -04:00
Henry Schreiner
c3e9173f0f
ci: use almalinux instead of centos, add 9 (#4020)
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
2022-06-21 14:55:49 -04:00
pre-commit-ci[bot]
2ad974c945
[pre-commit.ci] pre-commit autoupdate (#4021)
updates:
- [github.com/PyCQA/pylint: v2.14.1 → v2.14.3](https://github.com/PyCQA/pylint/compare/v2.14.1...v2.14.3)
- [github.com/pre-commit/mirrors-clang-format: v14.0.4-1 → v14.0.5](https://github.com/pre-commit/mirrors-clang-format/compare/v14.0.4-1...v14.0.5)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2022-06-21 14:36:49 -04:00
Aaron Gokaslan
0964a9093a
Add a missing std::move in numpy.h (#4005) 2022-06-14 14:20:26 -04:00
pre-commit-ci[bot]
21f0e72b0f
[pre-commit.ci] pre-commit autoupdate (#4003) 2022-06-13 21:02:25 -04:00
dependabot[bot]
0e956a2e4f
chore(deps): bump pre-commit/action from 2.0.3 to 3.0.0 (#3992)
Bumps [pre-commit/action](https://github.com/pre-commit/action) from 2.0.3 to 3.0.0.
- [Release notes](https://github.com/pre-commit/action/releases)
- [Commits](https://github.com/pre-commit/action/compare/v2.0.3...v3.0.0)

---
updated-dependencies:
- dependency-name: pre-commit/action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-08 09:22:57 -04:00
Aaron Gokaslan
e2dcd95407
chore: optimize dictionary access in strip_padding numpy (#3994)
* emplace field descriptors

* reserve sufficient capacity

* remove std::move

* properly iterate through dict

* make handle casting more explicit

* Revert to old dict api
2022-06-06 22:33:28 -04:00
Maarten Baert
918892b97e
Make dtype::num() return type consistent with other functions (#3995) 2022-06-06 19:41:38 -04:00
pre-commit-ci[bot]
c5fa3436c6
[pre-commit.ci] pre-commit autoupdate (#3951)
updates:
- [github.com/asottile/pyupgrade: v2.32.1 → v2.33.0](https://github.com/asottile/pyupgrade/compare/v2.32.1...v2.33.0)
- [github.com/Lucas-C/pre-commit-hooks: v1.1.13 → v1.2.0](https://github.com/Lucas-C/pre-commit-hooks/compare/v1.1.13...v1.2.0)
- [github.com/hadialqattan/pycln: v1.3.2 → v1.3.3](https://github.com/hadialqattan/pycln/compare/v1.3.2...v1.3.3)
- [github.com/PyCQA/pylint: v2.13.8 → v2.14.1](https://github.com/PyCQA/pylint/compare/v2.13.8...v2.14.1)
- [github.com/pre-commit/mirrors-mypy: v0.950 → v0.960](https://github.com/pre-commit/mirrors-mypy/compare/v0.950...v0.960)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2022-06-06 17:56:18 -04:00
Aaron Gokaslan
554c045377
enable two new clang-tidy checks (#3988)
* enable two new clang-tidy checks

* Use better loop var for char
2022-06-06 12:15:45 -04:00
Aaron Gokaslan
42a41bf3e7
remove useless ctor (#3989) 2022-06-03 15:18:06 -04:00
Sergei Lebedev
a05bc3d235
error_already_set::what() is now constructed lazily (#1895)
* error_already_set::what() is now constructed lazily

Prior to this commit throwing error_already_set was expensive due to the
eager construction of the error string (which required traversing the
Python stack). See #1853 for more context and an alternative take on the
issue.

Note that error_already_set no longer inherits from std::runtime_error
because the latter has no default constructor.

* Do not attempt to normalize if no exception occurred

This is not supported on PyPy-2.7 5.8.0.

* Extract exception name via tp_name

This is faster than dynamically looking up __name__ via GetAttrString.
Note though that the runtime of the code throwing an error_already_set
will be dominated by stack unwinding so the improvement will not be
noticeable.

Before:

396 ns ± 0.913 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

After:

277 ns ± 0.549 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

Benchmark:

const std::string foo() {
    PyErr_SetString(PyExc_KeyError, "");
    const std::string &s = py::detail::error_string();
    PyErr_Clear();
    return s;
}

PYBIND11_MODULE(foo, m) {
    m.def("foo", &::foo);
}

* Reverted error_already_set to subclass std::runtime_error

* Revert "Extract exception name via tp_name"

The implementation of __name__ is slightly more complex than that.
It handles the module name prefix, and heap-allocated types. We could
port it to pybind11 later on but for now it seems like an overkill.

This reverts commit f1435c7e6b.

* Cosmit following @YannickJadoul's comments

Note that detail::error_string() no longer calls PyException_SetTraceback
as it is unncessary for pretty-printing the exception.

* Fixed PyPy build

* Moved normalization to error_already_set ctor

* Fix merge bugs

* Fix more merge errors

* Improve formatting

* Improve error message in rare case

* Revert back if statements

* Fix clang-tidy

* Try removing mutable

* Does build_mode release fix it

* Set to Debug to expose segfault

* Fix remove set error string

* Do not run error_string() more than once

* Trying setting the tracebackk to the value

* guard if m_type is null

* Try to debug PGI

* One last try for PGI

* Does reverting this fix PyPy

* Reviewer suggestions

* Remove unnecessary initialization

* Add noexcept move and explicit fail throw

* Optimize error_string creation

* Fix typo

* Revert noexcept

* Fix merge conflict error

* Abuse assignment operator

* Revert operator abuse

* See if we still need debug

* Remove unnecessary mutable

* Report "FATAL failure building pybind11::error_already_set error_string" and terminate process.

* Try specifying noexcept again

* Try explicit ctor

* default ctor is noexcept too

* Apply reviewer suggestions, simplify code, and make helper method private

* Remove unnecessary include

* Clang-Tidy fix

* detail::obj_class_name(), fprintf with [STDERR], [STDOUT] tags, polish comments

* consistently check m_lazy_what.empty() also in production builds

* Make a comment slightly less ambiguous.

* Bug fix: Remove `what();` from `restore()`.

It sure would need to be guarded by `if (m_type)`, otherwise `what()` fails and masks that no error was set (see update unit test). But since `error_already_set` is copyable, there is no point in releasing m_type, m_value, m_trace, therefore we can just as well avoid the runtime overhead of force-building `m_lazy_what`, it may never be used.

* Replace extremely opaque (unhelpful) error message with a truthful reflection of what we know.

* Fix clang-tidy error [performance-move-constructor-init].

* Make expected error message less specific.

* Various changes.

* bug fix: error_string(PyObject **, ...)

* Putting back the two eager PyErr_NormalizeException() calls.

* Change error_already_set() to call pybind11_fail() if the Python error indicator not set. The net result is that a std::runtime_error is thrown instead of error_already_set, but all tests pass as is.

* Remove mutable (fixes oversight in the previous commit).

* Normalize the exception only locally in error_string(). Python 3.6 & 3.7 test failures expected. This is meant for benchmarking, to determine if it is worth the trouble looking into the failures.

* clang-tidy: use auto

* Use `gil_scoped_acquire_local` in `error_already_set` destructor. See long comment.

* For Python < 3.8: `PyErr_NormalizeException` before `PyErr_WriteUnraisable`

* Go back to replacing the held Python exception with then normalized exception, if & when needed. Consistently document the side-effect.

* Slightly rewording comment. (There were also other failures.)

* Add 1-line comment for obj_class_name()

* Benchmark code, with results in this commit message.

          function                   #calls  test time [s]  μs / call
master    pure_unwind                729540      1.061      14.539876
          err_set_unwind_err_clear   681476      1.040      15.260282
          err_set_error_already_set  508038      1.049      20.640525
          error_already_set_restore  555578      1.052      18.933288
          pr1895_original_foo        244113      1.050      43.018168
                                                                       PR / master
PR #1895  pure_unwind                736981      1.054      14.295685       98.32%
          err_set_unwind_err_clear   685820      1.045      15.237399       99.85%
          err_set_error_already_set  661374      1.046      15.811879       76.61%
          error_already_set_restore  669881      1.048      15.645176       82.63%
          pr1895_original_foo        318243      1.059      33.290806       77.39%

master @ commit ad146b2a18

Running tests in directory "/usr/local/google/home/rwgk/forked/pybind11/tests":
============================= test session starts ==============================
platform linux -- Python 3.9.10, pytest-6.2.3, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /usr/local/google/home/rwgk/forked/pybind11/tests, configfile: pytest.ini
collecting ... collected 5 items

test_perf_error_already_set.py::test_perf[pure_unwind]
PERF pure_unwind,729540,1.061,14.539876
PASSED
test_perf_error_already_set.py::test_perf[err_set_unwind_err_clear]
PERF err_set_unwind_err_clear,681476,1.040,15.260282
PASSED
test_perf_error_already_set.py::test_perf[err_set_error_already_set]
PERF err_set_error_already_set,508038,1.049,20.640525
PASSED
test_perf_error_already_set.py::test_perf[error_already_set_restore]
PERF error_already_set_restore,555578,1.052,18.933288
PASSED
test_perf_error_already_set.py::test_perf[pr1895_original_foo]
PERF pr1895_original_foo,244113,1.050,43.018168
PASSED

============================== 5 passed in 12.38s ==============================

pr1895 @ commit 8dff51d12e

Running tests in directory "/usr/local/google/home/rwgk/forked/pybind11/tests":
============================= test session starts ==============================
platform linux -- Python 3.9.10, pytest-6.2.3, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /usr/local/google/home/rwgk/forked/pybind11/tests, configfile: pytest.ini
collecting ... collected 5 items

test_perf_error_already_set.py::test_perf[pure_unwind]
PERF pure_unwind,736981,1.054,14.295685
PASSED
test_perf_error_already_set.py::test_perf[err_set_unwind_err_clear]
PERF err_set_unwind_err_clear,685820,1.045,15.237399
PASSED
test_perf_error_already_set.py::test_perf[err_set_error_already_set]
PERF err_set_error_already_set,661374,1.046,15.811879
PASSED
test_perf_error_already_set.py::test_perf[error_already_set_restore]
PERF error_already_set_restore,669881,1.048,15.645176
PASSED
test_perf_error_already_set.py::test_perf[pr1895_original_foo]
PERF pr1895_original_foo,318243,1.059,33.290806
PASSED

============================== 5 passed in 12.40s ==============================

clang++ -o pybind11/tests/test_perf_error_already_set.os -c -std=c++17 -fPIC -fvisibility=hidden -Os -flto -Wall -Wextra -Wconversion -Wcast-qual -Wdeprecated -Wnon-virtual-dtor -Wunused-result -isystem /usr/include/python3.9 -isystem /usr/include/eigen3 -DPYBIND11_STRICT_ASSERTS_CLASS_HOLDER_VS_TYPE_CASTER_MIX -DPYBIND11_TEST_BOOST -Ipybind11/include -I/usr/local/google/home/rwgk/forked/pybind11/include -I/usr/local/google/home/rwgk/clone/pybind11/include /usr/local/google/home/rwgk/forked/pybind11/tests/test_perf_error_already_set.cpp

clang++ -o lib/pybind11_tests.so -shared -fPIC -Os -flto -shared ...

Debian clang version 13.0.1-3+build2
Target: x86_64-pc-linux-gnu
Thread model: posix

* Changing call_repetitions_target_elapsed_secs to 0.1 for regular unit testing.

* Adding in `recursion_depth`

* Optimized ctor

* Fix silly bug in recurse_first_then_call()

* Add tests that have equivalent PyErr_Fetch(), PyErr_Restore() but no try-catch.

* Add call_error_string to tests. Sample only recursion_depth 0, 100.

* Show lazy-what speed-up in percent.

* Include real_work in benchmarks.

* Replace all PyErr_SetString() with generate_python_exception_with_traceback()

* Better organization of test loops.

* Add test_error_already_set_copy_move

* Fix bug in newly added test (discovered by clang-tidy): actually use move ctor

* MSVC detects the unreachable return

* change test_perf_error_already_set.py back to quick mode

* Inherit from std::exception (instead of std::runtime_error, which does not make sense anymore with the lazy what)

* Special handling under Windows.

* print with leading newline

* Removing test_perf_error_already_set (copies are under 7765113fbb).

* Avoid gil and scope overhead if there is nothing to release.

* Restore default move ctor. "member function" instead of "function" (note that "method" is Python terminology).

* Delete error_already_set copy ctor.

* Make restore() non-const again to resolve clang-tidy failure (still experimenting).

* Bring back error_already_set copy ctor, to see if that resolves the 4 MSVC test failures.

* Add noexcept to error_already_set copy & move ctors (as suggested by @skylion007 IIUC).

* Trying one-by-one noexcept copy ctor for old compilers.

* Add back test covering copy ctor. Add another simple test that exercises the copy ctor.

* Exclude more older compilers from using the noexcept = default ctors. (The tests in the previous commit exposed that those are broken.)

* Factor out & reuse gil_scoped_acquire_local as gil_scoped_acquire_simple

* Guard gil_scoped_acquire_simple by _Py_IsFinalizing() check.

* what() GIL safety

* clang-tidy & Python 3.6 fixes

* Use `gil_scoped_acquire` in dtor, copy ctor, `what()`. Remove `_Py_IsFinalizing()` checks (they are racy: https://github.com/python/cpython/pull/28525).

* Remove error_scope from copy ctor.

* Add `error_scope` to `get_internals()`, to cover the situation that `get_internals()` is called from the `error_already_set` dtor while a new Python error is in flight already. Also backing out `gil_scoped_acquire_simple` change.

* Add `FlakyException` tests with failure triggers in `__init__` and `__str__`

THIS IS STILL A WORK IN PROGRESS. This commit is only an important resting point.

This commit is a first attempt at addressing the observation that `PyErr_NormalizeException()` completely replaces the original exception if `__init__` fails. This can be very confusing even in small applications, and extremely confusing in large ones.

* Tweaks to resolve Py 3.6 and PyPy CI failures.

* Normalize Python exception immediately in error_already_set ctor.

For background see: https://github.com/pybind/pybind11/pull/1895#issuecomment-1135304081

* Fix oversights based on CI failures (copy & move ctor initialization).

* Move @pytest.mark.xfail("env.PYPY") after @pytest.mark.parametrize(...)

* Use @pytest.mark.skipif (xfail does not work for segfaults, of course).

* Remove unused obj_class_name_or() function (it was added only under this PR).

* Remove already obsolete C++ comments and code that were added only under this PR.

* Slightly better (newly added) comments.

* Factor out detail::error_fetch_and_normalize. Preparation for producing identical results from error_already_set::what() and detail::error_string(). Note that this is a very conservative refactoring. It would be much better to first move detail::error_string into detail/error_string.h

* Copy most of error_string() code to new error_fetch_and_normalize::complete_lazy_error_string()

* Remove all error_string() code from detail/type_caster_base.h. Note that this commit includes a subtle bug fix: previously error_string() restored the Python error, which will upset pybind11_fail(). This never was a problem in practice because the two PyType_Ready() calls in detail/class.h do not usually fail.

* Return const std::string& instead of const char * and move error_string() to pytypes.h

* Remove gil_scope_acquire from error_fetch_and_normalize, add back to error_already_set

* Better handling of FlakyException __str__ failure.

* Move error_fetch_and_normalize::complete_lazy_error_string() implementation from pybind11.h to pytypes.h

* Add error_fetch_and_normalize::release_py_object_references() and use from error_already_set dtor.

* Use shared_ptr for m_fetched_error => 1. non-racy, copy ctor that does not need the GIL; 2. enables guard against duplicate restore() calls.

* Add comments.

* Trivial renaming of a newly introduced member function.

* Workaround for PyPy

* Bug fix (oversight). Only valgrind got this one.

* Use shared_ptr custom deleter for m_fetched_error in error_already_set. This enables removing the dtor, copy ctor, move ctor completely.

* Further small simplification. With the GIL held, simply deleting the raw_ptr takes care of everything.

* IWYU cleanup

```
iwyu version: include-what-you-use 0.17 based on Debian clang version 13.0.1-3+build2
```

Command used:

```
iwyu -c -std=c++17 -DPYBIND11_TEST_BOOST -Iinclude/pybind11 -I/usr/include/python3.9 -I/usr/include/eigen3 include/pybind11/pytypes.cpp
```

pytypes.cpp is a temporary file: `#include "pytypes.h"`

The raw output is very long and noisy.

I decided to use `#include <cstddef>` instead of `#include <cstdio>` for `std::size_t` (iwyu sticks to the manual choice).

I ignored all iwyu suggestions that are indirectly covered by `#include <Python.h>`.

I manually verified that all added includes are actually needed.

Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
2022-06-02 16:17:38 -07:00
Aaron Gokaslan
58802de41b
perf: Add object rvalue overload for accessors. Enables reference stealing (#3970)
* Add object rvalue overload for accessors. Enables reference stealing

* Fix comments

* Fix more comment typos

* Fix bug

* reorder declarations for clarity

* fix another perf bug

* should be static

* future proof operator overloads

* Fix perfect forwarding

* Add a couple of tests

* Remove errant include

* Improve test documentation

* Add dict test

* add object attr tests

* Optimize STL map caster and cleanup enum

* Reorder to match declarations

* adjust increfs

* Remove comment

* revert value change

* add missing move
2022-06-01 15:19:13 -04:00
Ralf W. Grosse-Kunstleve
9f7b3f735a
addl unit tests for PR #3970 (#3977)
* Add test_perf_accessors (to be merged into test_pytypes).

* Python < 3.8 f-string compatibility

* Use thread_local in inc_ref_counter()

* Intentional breakage, brute-force way to quickly find out how many platforms reach the PYBIND11_HANDLE_REF_DEBUG code, with and without threads.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Remove Intentional breakage

* Drop perf test, move inc_refs tests to test_pytypes

* Fold in PR #3970 with `#ifdef`s

* Complete test coverage for all newly added code.

* Condense new unit tests via a simple local helper macro.

* Remove PYBIND11_PR3970 define. See https://github.com/pybind/pybind11/pull/3977#issuecomment-1142526417

* Move static keyword first (fixes silly oversight).

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2022-05-31 12:59:19 -07:00
Ralf W. Grosse-Kunstleve
b24c5ed204
Replace "Unknown internal error occurred" with a more helpful message. (#3982) 2022-05-31 11:54:33 -07:00
Ralf W. Grosse-Kunstleve
de4ba92c9f
Add error_scope to detail::get_internals() (#3981)
* Add `error_scope` to `detail::get_internals()`

* Adjust test to tolerate macOS PyPy behavior.
2022-05-31 11:51:13 -07:00
Aaron Gokaslan
8da58da539
chore: perfectly forward all make_iterator args (#3980)
* Perfectly forward all make_iterator args

* Try emplace back
2022-05-28 19:58:15 -04:00
Ralf W. Grosse-Kunstleve
748ae2270b
Add missing error handling to module_::def_submodule (#3973)
* Add missing error handling to module_::def_submodule

* Add test_def_submodule_failures

* PyPy only: Skip test with trigger for PyModule_GetName() failure.

* Reapply minor fix that accidentally got lost in transfer from PR #3964
2022-05-28 16:40:57 -07:00
Aaron Gokaslan
68f8010500
chore: add err guard to capsule destructor and add a move to iostream (#3958)
* Add err guard to capsule destructor

* only uses ostream currently

* can these be noexcept

* Add back header

* fix for older compilers

* This should at least be noexcept

* Add missing move

* Apparently not noexcept for old llvm
2022-05-27 14:32:57 -04:00
Ralf W. Grosse-Kunstleve
8d14e666e3
fix: avoid catch (...) for expected import numpy failures (#3974)
* Replace import numpy catch (...) with catch (error_already_set)

* Add missing const (not sure how those got lost).
2022-05-26 11:07:40 -04:00
Ralf W. Grosse-Kunstleve
2c549eb7aa
Move PyErr_NormalizeException() up a few lines (#3971)
* Add error_already_set_what what tests, asserting the status quo.

* Move PyErr_NormalizeException() up a few lines.

* @pytest.mark.skipif("env.PYPY") from PR #1895 is required even for this much simpler PR

* Move PyException_SetTraceback() with PyErr_NormalizeException() as suggested by @skylion007

* Insert a std::move() as suggested by @skylion007
2022-05-25 21:44:55 -07:00
Aaron Gokaslan
2d4a20c8cb
chore: add missing moves for buffer_func and staticmethod in pybind11.h (#3969)
* Use move converting ctor when making class staticmethod

* Add missing caster move in buffer func

* fix use after move

* add back move to staticmethod

* avoid shadowing with varname
2022-05-25 12:14:07 -04:00
Maarten Baert
4624e8e164
Don't return pointers to static objects with return_value_policy::take_ownership. (#3946)
* Don't return pointers to static objects with return_value_policy::take_ownership.

This fixes -Wfree-nonheap-object warnings produced by GCC.

* Use return value policy fix instead

Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
2022-05-24 13:46:31 -04:00