Commit Graph

838 Commits

Author SHA1 Message Date
Antoine Prouvost
a67d786571
fix(stl_bind): Enable bind_map with using declarations. (#4952)
* Enable `bind_map` with `using` declarations.

* style: pre-commit fixes

* Enable directives in bind_vector

* Add tests for bind_ and using directives

* style: pre-commit fixes

* Remove C++17 functions

* Fix test comment

* Add minimal user like map py test

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-11-29 12:56:25 -08:00
Henry Schreiner
6831666f5c
ci: add more versions of numpy/scipy/pypy (#4714)
* ci: add more versions of numpy/scipy/pypy

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* Apply suggestions from code review

* style: pre-commit fixes

* Update requirements.txt

* Update requirements.txt

* Apply suggestions from code review

---------

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-11-16 00:52:05 -05:00
Ralf W. Grosse-Kunstleve
e250155afa
Fix a long-standing bug in the handling of Python multiple inheritance (#4762)
* Equivalent of 5718e4d080

* Resolve clang-tidy errors.

* Moving test_PPCCInit() first changes the behavior!

* Resolve new Clang dev C++11 errors:

```
The CXX compiler identification is Clang 17.0.0
```

```
pytypes.h:1615:23: error: identifier '_s' preceded by whitespace in a literal operator declaration is deprecated [-Werror,-Wdeprecated-literal-operator]
```

```
cast.h:1380:26: error: identifier '_a' preceded by whitespace in a literal operator declaration is deprecated [-Werror,-Wdeprecated-literal-operator]
```

* Resolve gcc 4.8.5 error:

```
pytypes.h:1615:12: error: missing space between '""' and suffix identifier
```

* Specifically exclude `__clang__`

* Snapshot of debugging code (does NOT pass pre-commit checks).

* Revert "Snapshot of debugging code (does NOT pass pre-commit checks)."

This reverts commit 1d4f9ff263.

* [ci skip] Order Dependence Demo

* Revert "[ci skip] Order Dependence Demo"

This reverts commit d37b5409d4.

* One way to deal with the order dependency issue. This is not the best way, more like a proof of concept.

* Move test_PC() first again.

* Add `all_type_info_add_base_most_derived_first()`, use in `all_type_info_populate()`

* Revert "One way to deal with the order dependency issue. This is not the best way, more like a proof of concept."

This reverts commit eb09c6c1b9.

* clang-tidy fixes (automatic)

* Add `is_redundant_value_and_holder()` and use to avoid forcing `__init__` overrides when they are not needed.

* Streamline implementation and avoid unsafe `reinterpret_cast<instance *>()` introduced with PR #2152

The `reinterpret_cast<instance *>(self)` is unsafe if `__new__` is mocked,
which was actually found in the wild: the mock returned `None` for `self`.
This was inconsequential because `inst` is currently cast straight back to
`PyObject *` to compute `all_type_info()`, which is empty if `self` is not
a pybind11 `instance`, and then `inst` is never dereferenced. However, the
unsafe detour through `instance *` is easily avoided and the updated
implementation is less prone to accidents while debugging or refactoring.

* Fix actual undefined behavior exposed by previous changes.

It turns out the previous commit message is incorrect, the `inst` pointer is actually dereferenced, in the `value_and_holder` ctor here:

f3e0602802/include/pybind11/detail/type_caster_base.h (L262-L263)

```
259     // Main constructor for a found value/holder:
260     value_and_holder(instance *i, const detail::type_info *type, size_t vpos, size_t index)
261         : inst{i}, index{index}, type{type},
262           vh{inst->simple_layout ? inst->simple_value_holder
263                                  : &inst->nonsimple.values_and_holders[vpos]} {}
```

* Add test_mock_new()

* Experiment: specify indirect bases

* Revert "Experiment: specify indirect bases"

This reverts commit 4f90d85f9f.

* Add `all_type_info_check_for_divergence()` and some tests.

* Call `all_type_info_check_for_divergence()` also from `type_caster_generic::load_impl<>`

* Resolve clang-tidy error:

```
include/pybind11/detail/type_caster_base.h:795:21: error: the 'empty' method should be used to check for emptiness instead of 'size' [readability-container-size-empty,-warnings-as-errors]
                if (matching_bases.size() != 0) {
                    ^~~~~~~~~~~~~~~~~~~~~~~~~~
                    !matching_bases.empty()
```

* Revert "Resolve clang-tidy error:"

This reverts commit df27188dc6.

* Revert "Call `all_type_info_check_for_divergence()` also from `type_caster_generic::load_impl<>`"

This reverts commit 5f5fd6a68e.

* Revert "Add `all_type_info_check_for_divergence()` and some tests."

This reverts commit 0a9599f775.
2023-11-08 12:44:04 -08:00
Henry Schreiner
c758b81f3b
chore: move to ruff-format (#4912)
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
2023-11-07 23:42:54 -08:00
Henry Schreiner
3aece819fd
chore: update hooks and Ruff config (#4904)
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
2023-10-27 01:26:28 -04:00
Sergei Izmailov
74439a64a2
feature: Use typed iterators in make_*iterator (#4876) 2023-10-17 12:04:46 -07:00
Pablo Speciale
0cbd92bab8
Update pytest to version 7.2.0 (which removes their dependency on py) (#4880)
* Update pytest (which removes their dependency on py)

The py library through 1.11.0 for Python allows remote attackers to conduct a ReDoS (Regular expression Denial of Service) attack via a Subversion repository with crafted info data, because the InfoSvnCommand argument is mishandled.

The particular codepath in question is the regular expression at py._path.svnurl.InfoSvnCommand.lspattern and is only relevant when dealing with subversion (svn) projects. Notably the codepath is not used in the popular pytest project. The developers of the pytest package have released version 7.2.0 which removes their dependency on py. Users of pytest seeing alerts relating to this advisory may update to version 7.2.0 of pytest to resolve this issue. See https://github.com/pytest-dev/py/issues/287#issuecomment-1290407715 for additional context.

* Added conditions so that we keep using 7.0.0 on python 3.6
2023-10-16 07:42:30 -07:00
Ralf W. Grosse-Kunstleve
0e2c3e5db4
Add pybind11/gil_safe_call_once.h (to fix deadlocks in pybind11/numpy.h) (#4877)
* LazyInitializeAtLeastOnceDestroyNever v1

* Go back to using `union` as originally suggested by jbms@. The trick (also suggested by jbms@) is to add empty ctor + dtor.

* Revert "Go back to using `union` as originally suggested by jbms@. The trick (also suggested by jbms@) is to add empty ctor + dtor."

This reverts commit e7b8c4f0fc.

* Remove `#include <stdalign.h>`

* `include\pybind11/numpy.h(24,10): fatal error C1083: Cannot open include file: 'stdalign.h': No such file or directory`

* @tkoeppe wrote: this is a C interop header (and we're not writing C)

* Suppress gcc 4.8.5 (CentOS 7) warning.

```
include/pybind11/eigen/../numpy.h:63:53: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing]
         return *reinterpret_cast<T *>(value_storage_);
                                                     ^
```

* Replace comments:

Document PRECONDITION.

Adopt comment suggested by @tkoeppe: https://github.com/pybind/pybind11/pull/4877#discussion_r1350356093

* Adopt suggestion by @tkoeppe:

* https://github.com/pybind/pybind11/pull/4877#issuecomment-1752969127

* https://godbolt.org/z/Wa79nKz6e

* Add `PYBIND11_CONSTINIT`, but it does not work for the current use cases:

```
g++ -o pybind11/tests/test_numpy_array.os -c -std=c++20 -fPIC -fvisibility=hidden -O0 -g -Wall -Wextra -Wconversion -Wcast-qual -Wdeprecated -Wundef -Wnon-virtual-dtor -Wunused-result -Werror -isystem /usr/include/python3.11 -isystem /usr/include/eigen3 -DPYBIND11_STRICT_ASSERTS_CLASS_HOLDER_VS_TYPE_CASTER_MIX -DPYBIND11_ENABLE_TYPE_CASTER_ODR_GUARD_IF_AVAILABLE -DPYBIND11_TEST_BOOST -Ipybind11/include -I/usr/local/google/home/rwgk/forked/pybind11/include -I/usr/local/google/home/rwgk/clone/pybind11/include /usr/local/google/home/rwgk/forked/pybind11/tests/test_numpy_array.cpp
```

```
In file included from /usr/local/google/home/rwgk/forked/pybind11/tests/test_numpy_array.cpp:10:
/usr/local/google/home/rwgk/forked/pybind11/include/pybind11/numpy.h: In static member function ‘static pybind11::detail::npy_api& pybind11::detail::npy_api::get()’:
/usr/local/google/home/rwgk/forked/pybind11/include/pybind11/numpy.h:258:82: error: ‘constinit’ variable ‘api_init’ does not have a constant initializer
  258 |         PYBIND11_CONSTINIT static LazyInitializeAtLeastOnceDestroyNever<npy_api> api_init;
      |                                                                                  ^~~~~~~~
```

```
In file included from /usr/local/google/home/rwgk/forked/pybind11/tests/test_numpy_array.cpp:10:
/usr/local/google/home/rwgk/forked/pybind11/include/pybind11/numpy.h: In static member function ‘static pybind11::object& pybind11::dtype::_dtype_from_pep3118()’:
/usr/local/google/home/rwgk/forked/pybind11/include/pybind11/numpy.h:697:13: error: ‘constinit’ variable ‘imported_obj’ does not have a constant initializer
  697 |             imported_obj;
      |             ^~~~~~~~~~~~
```

* Revert "Add `PYBIND11_CONSTINIT`, but it does not work for the current use cases:"

This reverts commit f07b28bda9.

* Reapply "Add `PYBIND11_CONSTINIT`, but it does not work for the current use cases:"

This reverts commit 36be645758.

* Add Default Member Initializer on `value_storage_` as suggested by @tkoeppe:

https://github.com/pybind/pybind11/pull/4877#issuecomment-1753201342

This fixes the errors reported under commit f07b28bda9.

* Fix copy-paste-missed-a-change mishap in commit 88cec1152a.

* Semi-paranoid placement new (based on https://github.com/pybind/pybind11/pull/4877#discussion_r1350573114).

* Move PYBIND11_CONSTINIT to detail/common.h

* Move code to the right places, rename new class and some variables.

* Fix oversight: update tests/extra_python_package/test_files.py

* Get the name right first.

* Use `std::call_once`, `std::atomic`, following a pattern developed by @tkoeppe

* Make the API more self-documenting (and possibly more easily reusable).

* google-clang-tidy IWYU fixes

* Rewrite comment as suggested by @tkoeppe

* Update test_exceptions.cpp and exceptions.rst

* Fix oversight in previous commit: add `PYBIND11_CONSTINIT`

* Make `get_stored()` non-const for simplicity.

As suggested by @tkoeppe: not seeing any reasonable use in which `get_stored` has to be const.

* Add comment regarding `KeyboardInterrupt` behavior, based heavily on information provided by @jbms.

* Add `assert(PyGILState_Check())` in `gil_scoped_release` ctor (simple & non-simple implementation) as suggested by @EthanSteinberg.

* Fix oversight in previous commit (missing include cassert).

* Remove use of std::atomic, leaving comments with rationale, why it is not needed.

* Rewrite comment re `std:optional` based on deeper reflection (aka 2nd thoughts).

* Additional comment with the conclusion of a discussion under PR #4877.

* https://github.com/pybind/pybind11/pull/4877#issuecomment-1757363179

* Small comment changes suggested by @tkoeppe.
2023-10-11 21:05:31 -07:00
Mateusz Sokół
dd64df73c3
MAINT: Remove np.int_ (#4867) 2023-10-03 09:12:58 -07:00
Ralf W. Grosse-Kunstleve
d06d53694a
Fix small bug introduced with PR #4735 (#4845)
* Bug fix: `result[0]` called if `result.empty()`

* Add unit test that fails without the fix.
2023-09-14 09:47:34 -07:00
Sergei Izmailov
8c7b8dd0ae
fix: Missing typed variants of iterator and iterable (#4832) 2023-09-12 12:48:27 -07:00
Sergei Izmailov
b4573674bc
Update render for buffer sequence and handle (#4831)
* fix: Add capitalize render name of `py::buffer` and `py::sequence`

* fix: Render `py::handle` same way as `py::object`

* tests: Fix tests `handle` -> `object`

* tests: Test capitaliation of `py::sequence` and `py::buffer`

* style: pre-commit fixes

* fix: Render `py::object` as `Any`

* Revert "fix: Render `py::object` as `Any`"

This reverts commit 7861dcfabb.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
2023-09-12 12:47:39 -07:00
Sergei Izmailov
c9149d995c
fix: Use lowercase builtin collection names (#4833) 2023-09-12 12:46:58 -07:00
Sergei Izmailov
c83605936b
feature: Support move-only iterators in py::make_*iterator (#4834)
* feature: Support move-only iterators in `py::make_*iterator`

* fix: Missing static assertion message

* fixup: Missing `explicit` in single argument constructors

* fix: Simplify tests: make existing iterator move-only

* fix: Missing `noexcept`
2023-09-07 05:57:39 -07:00
pre-commit-ci[bot]
467fe27bd9
chore(deps): update pre-commit hooks (#4838)
* chore(deps): update pre-commit hooks

updates:
- [github.com/astral-sh/ruff-pre-commit: v0.0.281 → v0.0.287](https://github.com/astral-sh/ruff-pre-commit/compare/v0.0.281...v0.0.287)
- [github.com/pre-commit/mirrors-mypy: v1.4.1 → v1.5.1](https://github.com/pre-commit/mirrors-mypy/compare/v1.4.1...v1.5.1)
- [github.com/asottile/blacken-docs: 1.15.0 → 1.16.0](https://github.com/asottile/blacken-docs/compare/1.15.0...1.16.0)
- [github.com/Lucas-C/pre-commit-hooks: v1.5.1 → v1.5.4](https://github.com/Lucas-C/pre-commit-hooks/compare/v1.5.1...v1.5.4)
- [github.com/PyCQA/pylint: v3.0.0a6 → v3.0.0a7](https://github.com/PyCQA/pylint/compare/v3.0.0a6...v3.0.0a7)

* style: pre-commit fixes

* Update pyproject.toml

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
2023-09-06 06:04:27 -07:00
Sergei Izmailov
db412e6e86
fix: Render py::function as Callable (#4829)
* fix: Render `py::function` as `Callable`

* style: pre-commit fixes

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-08-30 22:43:01 -07:00
Sergei Izmailov
e705fb5f27
Fix enum's __str__ docstring (#4827)
* fix: Enum __str__ function name

* tests: Test enum.__str__.__doc__
2023-08-30 14:20:46 -07:00
Jean Elsner
b9359ceadb
Remove newlines from docstring signature (#4735)
* Remove newlines from docstring signature

* Jean/dev (#1)

Replace newlines in arg values with spaces

* style: pre-commit fixes

* Don't use std::find_if for C++ 11 compatibility

* Avoid implicit char to bool conversion

* Test default arguments for line breaks

* style: pre-commit fixes

* Separate Eigen tests

* style: pre-commit fixes

* Fix merge

* Try importing numpy

* Avoid unreferenced variable in catch block

* style: pre-commit fixes

* Update squash function

* Reduce try block

* Additional test cases

* style: pre-commit fixes

* Put statement inside braces

* Move string into function body

* Rename repr for better readability. Make constr explicit.

* Add multiline string default argument test case

* style: pre-commit fixes

* Add std namespace, do not modify string repr

* Test for all space chars, test str repr not modified

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-08-15 07:48:59 -07:00
Ralf W. Grosse-Kunstleve
690a115d84
Add py::set_error(), use in updated py::exception<> documentation (#4772)
* Copy clang 17 compatibility fixes from PR #4762 to a separate PR.

* static py::exception<> -> static py::handle

* Add `py::set_error()` but also try the suggestion of @malfet (https://github.com/pytorch/pytorch/pull/106401#pullrequestreview-1559961407).

* clang 17 compatibility fixes (#4767)

* Copy clang 17 compatibility fixes from PR #4762 to a separate PR.

* Add gcc:13 C++20

* Add silkeh/clang:16-bullseye C++20

* chore(deps): update pre-commit hooks (#4770)

updates:
- [github.com/psf/black: 23.3.0 → 23.7.0](https://github.com/psf/black/compare/23.3.0...23.7.0)
- [github.com/astral-sh/ruff-pre-commit: v0.0.276 → v0.0.281](https://github.com/astral-sh/ruff-pre-commit/compare/v0.0.276...v0.0.281)
- [github.com/asottile/blacken-docs: 1.14.0 → 1.15.0](https://github.com/asottile/blacken-docs/compare/1.14.0...1.15.0)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* docs: Remove upper bound on pybind11 in example pyproject.toml for setuptools (#4774)

* docs: Remove upper bound on pybind11 in example pyproject.toml for setuptools

* Update docs/compiling.rst

---------

Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>

* Provide better type hints for a variety of generic types (#4259)

* Provide better type hints for a variety of generic types

* Makes better documentation
* tuple, dict, list, set, function

* Move to py::typing

* style: pre-commit fixes

* Update copyright line with correct year and actual author. The author information was copy-pasted from the git log output.

---------

Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Use `py::set_error()` everywhere possible (only one special case, in common.h).
Overload `py::set_error(py::handle, py::handle)`.
Change back to `static py::handle exc = ... .release();`
Deprecate `py::exception<>::operator()`

* Add `PYBIND11_WARNING_DISABLE` for INTEL and MSVC (and sort alphabetically).

* `PYBIND11_WARNING_DISABLE_INTEL(10441)` does not work.

For ICC only, falling back to the recommended `py::set_error()` to keep the testing simple.

It is troublesome to add `--diag-disable=10441` specifically for test_exceptions.cpp, even that is non-ideal because it covers the entire file, not just the one line we need it for, and the value of exercising the trivial deprecated `operator()` on this one extra platform is practically zero.

* Fix silly oversight.

* NVHPC 23.5.0 generates deprecation warnings. They are currently not treated as errors, but falling back to using `py::set_error()` to not have to deal with that distraction.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Keto D. Zhang <keto.zhang@gmail.com>
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com>
2023-08-07 20:48:20 -07:00
Dustin Spicuzza
f8703154ec
Provide better type hints for a variety of generic types (#4259)
* Provide better type hints for a variety of generic types

* Makes better documentation
* tuple, dict, list, set, function

* Move to py::typing

* style: pre-commit fixes

* Update copyright line with correct year and actual author. The author information was copy-pasted from the git log output.

---------

Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-08-03 22:48:57 -07:00
Aaron Gokaslan
b33d06f615
bugfix: fixes a test suite bug in the __new__ example (#4698)
* bugfix: fixes a test suite bug in the __new__ example

* See https://github.com/pybind/pybind11/pull/4698#discussion_r1227107682

---------

Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
2023-07-12 13:20:08 -07:00
Henry Schreiner
2e5f5c4cf8
fix: support CMake 3.27, drop 3.4 (#4719)
* fix: support CMake 3.27, drop 3.4

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* Update upgrade.rst

* Update upgrade.rst

* Update upgrade.rst

---------

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
2023-07-12 13:10:24 -05:00
dependabot[bot]
d462dd91a5
chore(deps): bump scipy from 1.8.0 to 1.10.0 in /tests (#4731)
Bumps [scipy](https://github.com/scipy/scipy) from 1.8.0 to 1.10.0.
- [Release notes](https://github.com/scipy/scipy/releases)
- [Commits](https://github.com/scipy/scipy/compare/v1.8.0...v1.10.0)

---
updated-dependencies:
- dependency-name: scipy
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
2023-07-10 10:43:20 -07:00
Ralf W. Grosse-Kunstleve
2fb3d7cbde
Trivial refactoring to make the capsule API more user friendly. (#4720)
* Trivial refactoring to make the capsule API more user friendly.

* Use new API in production code. Thanks @Lalaland for pointing this out.
2023-06-27 15:08:32 -07:00
pre-commit-ci[bot]
3617f3554a
chore(deps): update pre-commit hooks (#4689)
* chore(deps): update pre-commit hooks

updates:
- [github.com/charliermarsh/ruff-pre-commit: v0.0.263 → v0.0.270](https://github.com/charliermarsh/ruff-pre-commit/compare/v0.0.263...v0.0.270)
- [github.com/pre-commit/mirrors-mypy: v1.2.0 → v1.3.0](https://github.com/pre-commit/mirrors-mypy/compare/v1.2.0...v1.3.0)
- [github.com/shellcheck-py/shellcheck-py: v0.9.0.2 → v0.9.0.5](https://github.com/shellcheck-py/shellcheck-py/compare/v0.9.0.2...v0.9.0.5)
- [github.com/pre-commit/mirrors-clang-format: v16.0.2 → v16.0.4](https://github.com/pre-commit/mirrors-clang-format/compare/v16.0.2...v16.0.4)

* Resolve new ruff error:

```
tests/test_pytypes.py:478:20: PLW0130 Duplicate value `3` in set
```

The reduction from `{3, 3}` to `{3}` never happened in pybind11 code, therefore this change is essentially a no-op for the purpose of unit-testing pybind11.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
2023-06-07 12:57:38 -07:00
T.Yamada
d0232b119f
Use annotated for array (#4679)
* use Annotated for std::array docstring

* add tests

* fix test

* style: pre-commit fixes

* fix valarray annotation

* style: pre-commit fixes

* refix test

* add FixedSize

* style: pre-commit fixes

* Update test_stl.py

* style: pre-commit fixes

---------

Co-authored-by: Taiju Yamada <taiju.yamada@mujin.co.jp>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-05-24 21:39:36 -07:00
Ralf W. Grosse-Kunstleve
8e1f9d5c40
Add format_descriptor<> & npy_format_descriptor<> PyObject * specializations. (#4674)
* Add `npy_format_descriptor<PyObject *>` to enable `py::array_t<PyObject *>` to/from-python conversions.

* resolve clang-tidy warning

* Use existing constructor instead of adding a static method. Thanks @Skylion007 for pointing out.

* Add `format_descriptor<PyObject *>`

Trivial addition, but still in search for a meaningful test.

* Add test_format_descriptor_format

* Ensure the Eigen `type_caster`s do not segfault when loading arrays with dtype=object

* Use `static_assert()` `!std::is_pointer<>` to replace runtime guards.

* Add comments to explain how to check for ref-count bugs. (NO code changes.)

* Make the "Pointer types ... are not supported" message Eigen-specific, as suggested by @Lalaland. Move to new pybind11/eigen/common.h header.

* Change "format_descriptor_format" implementation as suggested by @Lalaland. Additional tests meant to ensure consistency between py::format_descriptor<>, np.array, np.format_parser turn out to be useful only to highlight long-standing inconsistencies.

* resolve clang-tidy warning

* Account for np.float128, np.complex256 not being available on Windows, in a future-proof way.

* Fully address i|q|l ambiguity (hopefully).

* Remove the new `np.format_parser()`-based test, it's much more distracting than useful.

* Use bi.itemsize to disambiguate "l" or "L"

* Use `py::detail::compare_buffer_info<T>::compare()` to validate the `format_descriptor<T>::format()` strings.

* Add `buffer_info::compare<T>` to make `detail::compare_buffer_info<T>::compare` more visible & accessible.

* silence clang-tidy warning

* pytest-compatible access to np.float128, np.complex256

* Revert "pytest-compatible access to np.float128, np.complex256"

This reverts commit e9a289c50f.

* Use `sizeof(long double) == sizeof(double)` instead of `std::is_same<>`

* Report skipped `long double` tests.

* Change the name of the new `buffer_info` member function to `item_type_is_equivalent_to`. Add comment defining "equivalent" by example.

* Change `item_type_is_equivalent_to<>()` from `static` function to member function, as suggested by @Lalaland
2023-05-23 10:49:32 -07:00
Joyce
6e6bcca5b2
Create s Security Policy (#4671)
* Create SECURITY.md

* Update test_files.py to include SECURITY.md file

* Update MANIFEST.in to include SECURITY.md file
2023-05-23 10:05:25 -07:00
Ralf W. Grosse-Kunstleve
ce9bbc0a21
Python 3.11+: Add __notes__ to error_already_set::what() output. (#4678)
* First version adding `__notes__` to `error_already_set::what()` output.

* Fix trivial oversight (missing adjustment in existing test).

* Minor enhancements of new code.

* Re-enable `cmake --target cpptest -j 2`

* Revert "Re-enable `cmake --target cpptest -j 2`"

This reverts commit 60816285e9.

* Add general comment explaining why the `error_fetch_and_normalize` code is so unusual.
2023-05-23 10:03:33 -07:00
Ralf W. Grosse-Kunstleve
e9b961d9b9
Elide to-python conversion of setter return values (#4621)
* Reproducer for property setter with return type that is not wrapped.

* Use `py::class_<OptionsBase>()` to work around the return value conversion issue.

* WIP drop_return_value

* Remove struct drop_return_value

* Introduce `return_value_policy::return_none` for use by setters.

* Add `is_setter` to attr.h and use from `.def_property()`

* Merge the new test into test_methods_and_attributes

* Remove return_none return_value_policy again.

* Fix oversight (NOLINTNEXTLINE placement).

* Simplification (for the better) found while searching for a way to resolve GCC build failures.

Example of failure resolved by this change:

g++ (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0

```
cd /build/tests && /usr/bin/c++ -DPYBIND11_TEST_EIGEN -Dpybind11_tests_EXPORTS -I/mounted_pybind11/include -isystem /usr/include/python3.8 -isystem /build/_deps/eigen-src -g -std=c++17 -fPIC -fvisibility=hidden -Wall -Wextra -Wconversion -Wcast-qual -Wdeprecated -Wundef -Wnon-virtual-dtor -MD -MT tests/CMakeFiles/pybind11_tests.dir/test_buffers.cpp.o -MF CMakeFiles/pybind11_tests.dir/test_buffers.cpp.o.d -o CMakeFiles/pybind11_tests.dir/test_buffers.cpp.o -c /mounted_pybind11/tests/test_buffers.cpp
In file included from /mounted_pybind11/include/pybind11/stl.h:12,
                 from /mounted_pybind11/tests/test_buffers.cpp:10:
/mounted_pybind11/include/pybind11/pybind11.h: In instantiation of ‘pybind11::class_<type_, options>& pybind11::class_<type_, options>::def_property(const char*, const Getter&, const Setter&, const Extra& ...) [with Getter = pybind11::cpp_function; Setter = std::nullptr_t; Extra = {pybind11::return_value_policy}; type_ = pybind11::buffer_info; options = {}]’:
/mounted_pybind11/include/pybind11/pybind11.h:1716:58:   required from ‘pybind11::class_<type_, options>& pybind11::class_<type_, options>::def_property_readonly(const char*, const pybind11::cpp_function&, const Extra& ...) [with Extra = {pybind11::return_value_policy}; type_ = pybind11::buffer_info; options = {}]’
/mounted_pybind11/include/pybind11/pybind11.h:1684:9:   required from ‘pybind11::class_<type_, options>& pybind11::class_<type_, options>::def_readonly(const char*, const D C::*, const Extra& ...) [with C = pybind11::buffer_info; D = long int; Extra = {}; type_ = pybind11::buffer_info; options = {}]’
/mounted_pybind11/tests/test_buffers.cpp:209:61:   required from here
/mounted_pybind11/include/pybind11/pybind11.h:1740:25: error: call of overloaded ‘cpp_function(std::nullptr_t&, pybind11::is_setter)’ is ambiguous
 1740 |             name, fget, cpp_function(method_adaptor<type>(fset), is_setter()), extra...);
      |                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/mounted_pybind11/include/pybind11/pybind11.h:101:5: note: candidate: ‘pybind11::cpp_function::cpp_function(Func&&, const Extra& ...) [with Func = std::nullptr_t&; Extra = {pybind11::is_setter}; <template-parameter-1-3> = void]’
  101 |     cpp_function(Func &&f, const Extra &...extra) {
      |     ^~~~~~~~~~~~
In file included from /mounted_pybind11/include/pybind11/stl.h:12,
                 from /mounted_pybind11/tests/test_buffers.cpp:10:
/mounted_pybind11/include/pybind11/pybind11.h:87:5: note: candidate: ‘pybind11::cpp_function::cpp_function(std::nullptr_t, const Extra& ...) [with Extra = {pybind11::is_setter}; std::nullptr_t = std::nullptr_t]’
   87 |     cpp_function(std::nullptr_t, const Extra &...) {}
      |     ^~~~~~~~~~~~
```

* Bug fix: obvious in hindsight. I thought the original version was incrementing the reference count for None, but no.

Discovered via many failing tests in the wild (10s of thousands).

It is very tricky to construct a meaningful unit test for this bug specifically. It's unlikely to come back, because 10s of thousands of tests will fail again.
2023-05-08 10:13:54 -07:00
Ralf W. Grosse-Kunstleve
90312a6ee8
Add type_caster<PyObject> (#4601)
* Add `type_caster<PyObject>` (tests are still incomplete).

* Fix oversight (`const PyObject *`).

* Ensure `type_caster<PyObject>` only works for `PyObject *`

* Move `is_same_ignoring_cvref` into `detail` namespace.

* Add test_cast_nullptr

* Change is_same_ignoring_cvref from variable template to using.

```
test_type_caster_pyobject_ptr.cpp:8:23: error: variable templates only available with ‘-std=c++14’ or ‘-std=gnu++14’ [-Werror]
    8 | static constexpr bool is_same_ignoring_cvref = std::is_same<detail::remove_cvref_t<T>, U>::value;
      |                       ^~~~~~~~~~~~~~~~~~~~~~
```

* Remove `return_value_policy::reference_internal` `keep_alive` feature (because of doubts about it actually being useful).

* Add missing test, fix bug (missing `throw error_already_set();`), various cosmetic changes.

* Move `type_caster<PyObject>` from test to new include (pybind11/type_caster_pyobject_ptr.h)

* Add new header file to CMakeLists.txt and tests/extra_python_package/test_files.py

* Backport changes from https://github.com/google/pywrapcc/pull/30021 to https://github.com/pybind/pybind11/pull/4601

* Fix oversight in test (to resolve a valgrind leak detection error) and add a related comment in cast.h.

No production code changes.

Make tests more sensitive by using `ValueHolder` instead of empty tuples and dicts.

Manual leak checks with `while True:` & top command repeated for all tests.

* Add tests for interop with stl.h `list_caster`

(No production code changes.)

* Bug fix in test. Minor comment enhancements.

* Change `type_caster<PyObject>::name` to `object`, as suggested by @Skylion007

* Expand comment for the new `T cast(const handle &handle)` [`T` = `PyObject *`]

* Add `T cast(object &&obj)` overload as suggested by @Skylion007

The original suggestion leads to `error: call to 'cast' is ambiguous` (full error message below), therefore SFINAE guarding is needed.

```
clang++ -o pybind11/tests/test_type_caster_pyobject_ptr.os -c -std=c++17 -fPIC -fvisibility=hidden -O0 -g -Wall -Wextra -Wconversion -Wcast-qual -Wdeprecated -Wundef -Wnon-virtual-dtor -Wunused-result -Werror -isystem /usr/include/python3.10 -isystem /usr/include/eigen3 -DPYBIND11_STRICT_ASSERTS_CLASS_HOLDER_VS_TYPE_CASTER_MIX -DPYBIND11_ENABLE_TYPE_CASTER_ODR_GUARD_IF_AVAILABLE -DPYBIND11_TEST_BOOST -Ipybind11/include -I/usr/local/google/home/rwgk/forked/pybind11/include -I/usr/local/google/home/rwgk/clone/pybind11/include /usr/local/google/home/rwgk/forked/pybind11/tests/test_type_caster_pyobject_ptr.cpp
In file included from /usr/local/google/home/rwgk/forked/pybind11/tests/test_type_caster_pyobject_ptr.cpp:1:
In file included from /usr/local/google/home/rwgk/forked/pybind11/include/pybind11/functional.h:12:
In file included from /usr/local/google/home/rwgk/forked/pybind11/include/pybind11/pybind11.h:13:
In file included from /usr/local/google/home/rwgk/forked/pybind11/include/pybind11/detail/class.h:12:
In file included from /usr/local/google/home/rwgk/forked/pybind11/include/pybind11/attr.h:14:
/usr/local/google/home/rwgk/forked/pybind11/include/pybind11/cast.h:1165:12: error: call to 'cast' is ambiguous
    return pybind11::cast<T>(std::move(*this));
           ^~~~~~~~~~~~~~~~~
/usr/local/google/home/rwgk/forked/pybind11/include/pybind11/functional.h:109:70: note: in instantiation of function template specialization 'pybind11::object::cast<_object *>' requested here
                return hfunc.f(std::forward<Args>(args)...).template cast<Return>();
                                                                     ^
/usr/local/google/home/rwgk/forked/pybind11/include/pybind11/functional.h:103:16: note: in instantiation of member function 'pybind11::detail::type_caster<std::function<_object *(int)>>::load(pybind11::handle, bool)::func_wrapper::operator()' requested here
        struct func_wrapper {
               ^
/usr/local/google/home/rwgk/forked/pybind11/include/pybind11/cast.h:1456:47: note: in instantiation of member function 'pybind11::detail::type_caster<std::function<_object *(int)>>::load' requested here
        if ((... || !std::get<Is>(argcasters).load(call.args[Is], call.args_convert[Is]))) {
                                              ^
/usr/local/google/home/rwgk/forked/pybind11/include/pybind11/cast.h:1434:50: note: in instantiation of function template specialization 'pybind11::detail::argument_loader<const std::function<_object *(int)> &, int>::load_impl_sequence<0UL, 1UL>' requested here
    bool load_args(function_call &call) { return load_impl_sequence(call, indices{}); }
                                                 ^
/usr/local/google/home/rwgk/forked/pybind11/include/pybind11/pybind11.h:227:33: note: in instantiation of member function 'pybind11::detail::argument_loader<const std::function<_object *(int)> &, int>::load_args' requested here
            if (!args_converter.load_args(call)) {
                                ^
/usr/local/google/home/rwgk/forked/pybind11/include/pybind11/pybind11.h:101:9: note: in instantiation of function template specialization 'pybind11::cpp_function::initialize<(lambda at /usr/local/google/home/rwgk/forked/pybind11/tests/test_type_caster_pyobject_ptr.cpp:50:9), _object *, const std::function<_object *(int)> &, int, pybind11::name, pybind11::scope, pybind11::sibling, pybind11::return_value_policy>' requested here
        initialize(
        ^
/usr/local/google/home/rwgk/forked/pybind11/include/pybind11/pybind11.h:1163:22: note: in instantiation of function template specialization 'pybind11::cpp_function::cpp_function<(lambda at /usr/local/google/home/rwgk/forked/pybind11/tests/test_type_caster_pyobject_ptr.cpp:50:9), pybind11::name, pybind11::scope, pybind11::sibling, pybind11::return_value_policy, void>' requested here
        cpp_function func(std::forward<Func>(f),
                     ^
/usr/local/google/home/rwgk/forked/pybind11/tests/test_type_caster_pyobject_ptr.cpp:48:7: note: in instantiation of function template specialization 'pybind11::module_::def<(lambda at /usr/local/google/home/rwgk/forked/pybind11/tests/test_type_caster_pyobject_ptr.cpp:50:9), pybind11::return_value_policy>' requested here
    m.def(
      ^
/usr/local/google/home/rwgk/forked/pybind11/include/pybind11/cast.h:1077:3: note: candidate function [with T = _object *, $1 = 0]
T cast(object &&obj) {
  ^
/usr/local/google/home/rwgk/forked/pybind11/include/pybind11/cast.h:1149:1: note: candidate function [with T = _object *]
cast(object &&object) {
^
1 error generated.
```
2023-05-07 10:15:53 -07:00
Franz Pöschel
f701654633
Introduce recursive_container_traits (#4623)
* Testing

* Similar fix for std::vector

* Fix infinite recursion check:

1) Apply to is_copy_assignable additionally
2) Check infinite recursion for map-like types

* style: pre-commit fixes

* Optional commit that demonstrates the limitations of this PR

* Fix positioning of container bindings

The bindings were previously in a block that was only activated if numpy
was available.

* Suggestions from code review: API side

* Suggestions from code review: Test side

* Suggestions from code review

1) Renaming: is_recursive_container and
   MutuallyRecursiveContainerPair(MV|VM)
2) Avoid ambiguous specializations of is_recursive_container

* Some little fixes

* Reordering of structs

* Add recursive checks for is_move_constructible

* Static testing for pybind11 type traits

* More precise checking of recursive types

Instead of a trait `is_recursive_container`, use a trait
`recursive_container_traits` with dependent type
`recursive_container_traits::type_to_check_recursively`.
So, instead of just checking if a type is recursive and then trying to
somehow deal with it, recursively-defined traits such as
is_move_constructible can now directly ask this trait where the
recursion should proceed.

* Review suggestions

1. Use std::conditional
2. Fix typo

* Remove leftover include from test

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-05-04 22:39:05 -07:00
pre-commit-ci[bot]
5e946c2fa5
chore(deps): update pre-commit hooks (#4605)
* chore(deps): update pre-commit hooks

updates:
- [github.com/psf/black: 23.1.0 → 23.3.0](https://github.com/psf/black/compare/23.1.0...23.3.0)
- [github.com/Lucas-C/pre-commit-hooks: v1.4.2 → v1.5.1](https://github.com/Lucas-C/pre-commit-hooks/compare/v1.4.2...v1.5.1)
- [github.com/charliermarsh/ruff-pre-commit: v0.0.254 → v0.0.260](https://github.com/charliermarsh/ruff-pre-commit/compare/v0.0.254...v0.0.260)
- [github.com/PyCQA/pylint: v2.16.4 → v3.0.0a6](https://github.com/PyCQA/pylint/compare/v2.16.4...v3.0.0a6)
- [github.com/pre-commit/mirrors-mypy: v1.0.1 → v1.1.1](https://github.com/pre-commit/mirrors-mypy/compare/v1.0.1...v1.1.1)
- [github.com/codespell-project/codespell: v2.2.2 → v2.2.4](https://github.com/codespell-project/codespell/compare/v2.2.2...v2.2.4)
- [github.com/pre-commit/mirrors-clang-format: v15.0.7 → v16.0.0](https://github.com/pre-commit/mirrors-clang-format/compare/v15.0.7...v16.0.0)

* style: pre-commit fixes

* style: fix issues

* Update tests/test_call_policies.py

* Update tests/test_call_policies.py

* fix: ignore code in file

* style: pre-commit fixes

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
2023-04-28 14:32:32 -04:00
Ralf W. Grosse-Kunstleve
3f366ff888
Remove stray comment. (Oversight in PR #4631. Noticed by chance.) (#4641) 2023-04-27 07:24:48 -07:00
Ralf W. Grosse-Kunstleve
6de6191a0c
Use std::hash<std::type_index>, std::equal_to<std::type_index> everywhere **except when libc++ is in use** (#4319)
* Try using `std::hash<std::type_index>`, `std::equal_to<std::type_index>` everywhere.

From PR #4316 we know that types in the unnamed namespace in different translation units do not compare equal, as desired.

But do types in named namespaces compare equal, as desired?

* Revert "Try using `std::hash<std::type_index>`, `std::equal_to<std::type_index>` everywhere."

This reverts commit a06949a926.

* Use "our own name-based hash and equality functions" for `std::type_index` only under macOS, based on results shown under https://github.com/pybind/pybind11/pull/4316#issuecomment-1305097879

* Patch in PR #4313: Minimal reproducer for clash when binding types defined in the unnamed namespace.

* test_unnamed_namespace_b xfail for clang

* `PYBIND11_INTERNALS_VERSION 5`

* Add a note to docs/classes.rst

* For compatibility with Google-internal testing, test_unnamed_namespace_a & test_unnamed_namespace_b need to work when imported in any order.

* Trying "__GLIBCXX__ or Windows", based on observations from Google-internal testing.

* Try _LIBCPP_VERSION

* Account for libc++ behavior in tests and documentation.

* Adjust expectations for Windows Clang (and make code less redundant).

* Add WindowsClang to ci.yml

Added block transferred from PR #4321

* Add clang-latest to name that appears in the GitHub Actions web view.

* Tweak the note in classes.rst again.

* Add `pip install --upgrade pip`, Show env, cosmetic changes

Already tested under PR #4321

* Add macos_brew_install_llvm to ci.yml

Added block transferred from PR #4324

* `test_cross_module_exception_translator` xfail 'Homebrew Clang'

* Revert back to base version of .github/workflows/ci.yml (the ci.yml changes were merged under #4323 and #4326)

* Fixes for ruff

* Make updated condition in internals.h dependent on ABI version.

* Remove PYBIND11_TEST_OVERRIDE when testing with PYBIND11_INTERNALS_VERSION=10000000

* Selectively exercise cmake `-DPYBIND11_TEST_OVERRIDE`: ubuntu, macos, windows

Extra work added to quick jobs, based on timings below, to not increase the GHA start-to-last-job-finished time.

```
Duration
^              Number of pytest runs
^              ^ Job identifier
^              ^ ^
0:03:48.024227 1 1___3___Clang_3.6___C++11___x64.txt
0:03:58.992814 1 2___3___Clang_3.7___C++11___x64.txt
0:04:25.758942 1 1___3.7___Debian___x86____Install.txt
0:04:50.148276 1 4___3___Clang_7___C++11___x64.txt
0:04:55.784558 1 13___3___Clang_15___C++20___x64.txt
0:04:57.048754 1 6___3___Clang_dev___C++11___x64.txt
0:05:00.485181 1 7___3___Clang_5___C++14___x64.txt
0:05:03.744964 1 2___3___almalinux8___x64.txt
0:05:06.222752 1 5___3___Clang_9___C++11___x64.txt
0:05:11.767022 1 2___3___GCC_7___C++17__x64.txt
0:05:18.634930 1 2___3.11__deadsnakes____x64.txt
0:05:22.810995 1 1___3___GCC_7___C++11__x64.txt
0:05:25.275317 1 12___3___Clang_14___C++20___x64.txt
0:05:32.058174 1 5___3___GCC_10___C++17__x64.txt
0:05:39.381351 1 7___3___GCC_12___C++20__x64.txt
0:05:40.502252 1 8___3___Clang_10___C++17___x64.txt
0:05:59.344905 1 3___3___Clang_3.9___C++11___x64.txt
0:06:10.825147 1 6___3___GCC_11___C++20__x64.txt
0:06:20.655443 1 3___3___almalinux9___x64.txt
0:06:22.472061 1 3___3___GCC_8___C++14__x64.txt
0:06:42.647406 1 11___3___Clang_13___C++20___x64.txt
0:06:53.352720 1 1___3.10___CUDA_11.7___Ubuntu_22.04.txt
0:07:07.357801 1 2___3.7___MSVC_2019___x86_-DCMAKE_CXX_STANDARD=14.txt
0:07:09.057603 1 1___3___centos7___x64.txt
0:07:15.546282 1 1___3.8___MSVC_2019__Debug____x86_-DCMAKE_CXX_STANDARD=17.txt
0:07:22.566022 1 4___3___GCC_8___C++17__x64.txt
0:08:13.592674 1 2___3.9___MSVC_2019__Debug____x86_-DCMAKE_CXX_STANDARD=20.txt
0:08:16.422768 1 9___3___Clang_11___C++20___x64.txt
0:08:21.168457 1 3___3.8___MSVC_2019___x86_-DCMAKE_CXX_STANDARD=17.txt
0:08:27.129468 1 10___3___Clang_12___C++20___x64.txt
0:09:35.045470 1 1___3.10___windows-latest___clang-latest.txt
0:09:57.361843 1 1___3.9___MSVC_2022_C++20___x64.txt
0:10:35.187767 1 1___3.6___MSVC_2019___x86.txt
0:11:14.691200 4 2___3.9___ubuntu-20.04___x64.txt
0:11:37.701167 1 1_macos-latest___brew_install_llvm.txt
0:11:38.688299 4 4___3.11___ubuntu-20.04___x64.txt
0:11:52.720216 1 4___3.9___MSVC_2019___x86_-DCMAKE_CXX_STANDARD=20.txt
0:13:23.456591 4 6___pypy-3.8___ubuntu-20.04___x64_-DPYBIND11_FINDPYTHON=ON.txt
0:13:25.863592 2 1___3___ICC_latest___x64.txt
0:13:32.411758 3 9___3.9___windows-2022___x64.txt
0:13:45.473377 4 3___3.10___ubuntu-20.04___x64.txt
0:13:55.366447 4 5___pypy-3.7___ubuntu-20.04___x64.txt
0:13:57.969502 3 10___3.10___windows-2022___x64.txt
0:14:19.837475 3 11___3.11___windows-2022___x64.txt
0:14:33.316770 4 1___3.6___ubuntu-20.04___x64_-DPYBIND11_FINDPYTHON=ON_-DCMA.txt
0:15:34.449278 4 22___3.6___windows-2019___x64_-DPYBIND11_FINDPYTHON=ON.txt
0:16:25.189055 2 1___3.9-dbg__deadsnakes____Valgrind___x64.txt
0:17:20.956667 4 15___3.6___macos-latest___x64.txt
0:17:27.513891 4 23___3.9___windows-2019___x64.txt
0:17:58.783286 3 8___3.6___windows-2022___x64.txt
0:18:25.917828 4 7___pypy-3.9___ubuntu-20.04___x64.txt
0:19:17.399820 3 13___pypy-3.8___windows-2022___x64.txt
0:19:45.002122 3 12___pypy-3.7___windows-2022___x64.txt
0:20:03.201926 4 16___3.9___macos-latest___x64.txt
0:20:15.415178 4 17___3.10___macos-latest___x64.txt
0:20:20.263216 4 20___pypy-3.8___macos-latest___x64.txt
0:20:31.998226 3 1___3___windows-latest___mingw64.txt
0:20:40.812286 4 18___3.11___macos-latest___x64.txt
0:22:47.714749 4 19___pypy-3.7___macos-latest___x64.txt
0:23:04.435859 3 2___3___windows-latest___mingw32.txt
0:25:48.719597 3 14___pypy-3.9___windows-2022___x64.txt
0:26:01.211688 4 21___pypy-3.9___macos-latest___x64.txt
0:28:19.971015 1 1___3___CentOS7__PGI_22.9___x64.txt
```

* Update skipif for Python 3.12a7 (the WIP needs to be handled in a separate PR).
2023-04-25 14:03:24 -07:00
Ralf W. Grosse-Kunstleve
07725c28c0
Introduce pybind11::detail::is_move_constructible (#4631)
To support the use case captured in the new test_vector_unique_ptr_member.cpp
2023-04-24 00:19:21 -07:00
Henry Schreiner
4ce05175d5
ci: Python 3.12 optional test job (#4575)
* ci: Python 3.12 optional testing

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* Skip test_flaky_exception_failure_point_init() for Python 3.12.0a6 (similar to af5c6536ab (diff-f46006e3f43ffb1dd5d6862005427f6620f4dcfb1fa2f883d8482550069eeecc)).

* Disable tests/test_embed/test_interpreter.cpp (broken with Python 3.12.0alpha6)

```
free(): invalid pointer

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
test_embed is a Catch v2.13.9 host application.
Run with -? for options

-------------------------------------------------------------------------------
Custom PyConfig
-------------------------------------------------------------------------------
/home/runner/work/pybind11/pybind11/tests/test_embed/test_interpreter.cpp:175
...............................................................................

/home/runner/work/pybind11/pybind11/tests/test_embed/test_interpreter.cpp:179: FAILED:
  {Unknown expression after the reported line}
due to a fatal error condition:
  SIGABRT - Abort (abnormal termination) signal

===============================================================================
test cases:    6 |    5 passed | 1 failed
assertions: 1518 | 1517 passed | 1 failed
```

---------

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
2023-03-30 10:29:25 -04:00
Ralf W. Grosse-Kunstleve
654fe92652
Introduce get_python_state_dict() for Python 3.12 compatibility. (#4570)
* Introduce `get_python_state_dict()`

* Conditional version bump for Python 3.12+

* Shuffle subexpressions to make the condition easier to understand (no change to logic).

* Make pybind11 ABI version 5 the minimum for Python 3.12+ (as suggested by @Lalaland)

* Add back condition for PYPY_VERSION, but keep it open for future PyPy versions.

* Fall back to simple `|| defined(PYPY_VERSION)`. `PY_VERSION_HEX` does not appear to be meaningful with PyPy.
2023-03-27 17:52:57 -07:00
Aaron Gokaslan
1e8b52a9ac
bugfix: allow noexcept lambdas in C++17. Fix #4565 (#4593)
* bugfix: allow noexcept lambdas in CPP17. Fix #4565

* Remove unused code from test case

* Fix clang-tidy error

* Address reviewer comment
2023-03-27 20:21:06 -04:00
Ralf W. Grosse-Kunstleve
cf7d2e6f15
Change always_forkserver_on_unix() to use_multiprocessing_forkserver_on_linux() (#4577) 2023-03-16 14:33:34 -07:00
Ralf W. Grosse-Kunstleve
442261da58
Remove test code that does not exercise anything in pybind11, but breaks between Python 3.12alpha3 (still working) and 3.12alpha6 (broken): (#4559)
```
str(OrderedDict([(1, "a"), (2, "b")]))
```

Old:
```
OrderedDict([(1, 'a'), (2, 'b')])
```

New:
```
OrderedDict({1: 'a', 2: 'b'})
```

See also: https://github.com/python/cpython/issues/101446
2023-03-09 13:02:34 -08:00
Henry Schreiner
438034c5b8
chore: move to Ruff and add rules (#4483) 2023-02-22 06:18:55 -08:00
Dustin Spicuzza
8dcced29ae
Always display python type information in cast errors (#4463)
* Always display python type information in cast errors

* Address comments

* Update comment
2023-02-09 21:21:17 -08:00
Mike Essenmacher
531144dddc
Replace "whitelist" with "allowlist" (#4506) 2023-02-09 10:11:01 -08:00
Henry Schreiner
3efe9d4cb5
chore: update to black 23 (#4482)
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
2023-02-01 14:23:37 -05:00
Daniel Jacobs
44e9368222
Use PyConfig_InitPythonConfig instead of PyConfig_InitIsolatedConfig (#4473)
* Use PyConfig_InitPythonConfig instead of PyConfig_InitIsolatedConfig

* add unit test for default python configuration

---------

Co-authored-by: Daniel Jacobs <daniel.jacobs@is4s.com>
2023-01-31 23:42:05 -08:00
Ralf W. Grosse-Kunstleve
a500f439d0
Resolve new flake8 error (#4462)
* Resolve flake8 error by replacing `pytest.raises(Exception)` with `SystemError`

* Also remove the obsolete comment.

* Tweak comment instead of removing it.
2023-01-19 10:48:46 -08:00
Ralf W. Grosse-Kunstleve
e53d58af6c
Ensure import pybind11_tests traceback is shown. (#4455) 2023-01-14 16:47:56 -05:00
Ralf W. Grosse-Kunstleve
f12e098f1d
Fix detail::obj_class_name() to work correctly for meta classes. (#4436)
* Fix `detail::obj_class_name()` to work correctly for meta classes.

* Adjust expected name for PyPy
2023-01-02 03:46:17 -08:00
Ethan Steinberg
ee4b9f5d7b
Fix ODR violations in our Eigen Tensor tests (#4412)
* First

* Fix centos 7 again :(

* Fix minor nits
2022-12-20 05:34:00 -08:00
aimir
9db988013c
Correct class names for KeysView, ValuesView and ItemsView in bind_map (#4353)
* Create templated abstract classes KeysView, ValuesView and ItemsView, and implement them on-the-fly when wrapping any specific map type

* We don't want to wrap different ValuesView objects for double values and const double, for example, as both wrappers will be named ValuesView[float]

* Fallback to C++ names if key or values types are not wrapped

* Added a test for .keys(), .values() and .items() returning the same types for similarly-typed maps

* Fixed wrong use of auto in a declarator list: the two descriptions might have different types

* Fixes for clang-tidy issues: explicit single-argument constructor, using the 'override' keyword when overriding functions

* Bugfix for old versions of clang++, which seem to have trouble with the struct being defined inside a module, which was also needlessly ugly anyway

* Bugfix for clang++, which doesn't have some of the names in runtime uness they are specified to be static

* A fix for clang-tidy performance-inefficient-string-concatenation issues - I personally think this looks uglier, but it's probably worth it for clang-tidy to be happy

* Possible fix for clang++ linking issues - make the descriptions static constexpr to make sure they are known before linking

* Correct names for previously-wrapped types as keys/values of maps

* Bugfix - typo in type info names which caused things to segfault

* Apply suggestions from code review

Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>

* Use detail::remove_cvref_t instead of doing remove_cv and remove_reference separately

* Avoid names with double underscore, as they are reserved

* Improved testing for KeysView, ValuesView and ItemsView: check type names + stricter asserts

* Moved description logic to helper function in type_caster_base.h

* style: pre-commit fixes

* Fix a clang-tidy issue: do not use 'else' after 'return'

* Apply suggestion by @Skylion007, with additional trivial simplification.

Co-authored-by: Amir <aimir@local>
Co-authored-by: aimir <aimir@localhost>
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
2022-12-08 23:15:11 -08:00
Frank
00126859a5
Add option for enable/disable enum members in docstring. (#2768)
* Add option for enable/disable enum members in docstring

* Add tests for disable enum members docstring option

* Add docstring options to documentation

* style: pre-commit fixes

* Fix typos in documentation

* Improve documentation wording

* Apply suggestions by @Skylion007

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
2022-12-08 23:10:10 -08:00
pre-commit-ci[bot]
4768a6f8f5
chore(deps): update pre-commit hooks (#4386)
* chore(deps): update pre-commit hooks

updates:
- [github.com/pre-commit/pre-commit-hooks: v4.3.0 → v4.4.0](https://github.com/pre-commit/pre-commit-hooks/compare/v4.3.0...v4.4.0)
- [github.com/asottile/pyupgrade: v3.2.0 → v3.3.0](https://github.com/asottile/pyupgrade/compare/v3.2.0...v3.3.0)
- [github.com/hadialqattan/pycln: v2.1.1 → v2.1.2](https://github.com/hadialqattan/pycln/compare/v2.1.1...v2.1.2)
- [github.com/PyCQA/flake8: 5.0.4 → 6.0.0](https://github.com/PyCQA/flake8/compare/5.0.4...6.0.0)
- [github.com/PyCQA/pylint: v2.15.5 → v2.15.8](https://github.com/PyCQA/pylint/compare/v2.15.5...v2.15.8)
- [github.com/pre-commit/mirrors-mypy: v0.982 → v0.991](https://github.com/pre-commit/mirrors-mypy/compare/v0.982...v0.991)
- [github.com/mgedmin/check-manifest: 0.48 → 0.49](https://github.com/mgedmin/check-manifest/compare/0.48...0.49)
- [github.com/pre-commit/mirrors-clang-format: v14.0.6 → v15.0.4](https://github.com/pre-commit/mirrors-clang-format/compare/v14.0.6...v15.0.4)

* style: pre-commit fixes

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2022-12-06 10:10:48 -08:00
Aaron Gokaslan
e133c33d5c
chore: Convert direct multiprocessing.set_start_method("forkserver") call to a pytest fixture. (#4377)
* chore: convert multiprocessing set_spawn to fixture in pytest

* Switch to early return
2022-12-01 15:15:47 -05:00
Ralf W. Grosse-Kunstleve
358ba459d2
Fix test added with PR #4330 (#4372) 2022-12-01 09:25:30 -08:00
Arman
8869984926
scoped_interpreter. overloaded constructor: PyConfig param (#4330)
* scoped_interpreter overloaded ctor: PyConfig param

* style: pre-commit fixes

* refact: some logics extracted into funcs (precheck_interpreter, _initialize_interpreter); config_guard

* style: pre-commit fixes

* refact: scoped_config, some funcs hidden in detail ns

* refact: macro PYBIND11_PYCONFIG_SUPPORT_PY_VERSION + undef

* feat: PYBIND11_PYCONFIG_SUPPORT_PY_VERSION set to 3.8

* tests: Custom PyConfig

* ci: python 3.6 -> 3.8

* ci: reverted py 38 back  to 36; refact: initialize_interpreter overloads

* style: pre-commit fixes

* fix: readability-implicit-bool-conversion

* refact: each initialize_interpreter overloads in pybind11 ns

* Move `initialize_interpreter_pre_pyconfig()` into the `detail` namespace.

Move the `PYBIND11_PYCONFIG_SUPPORT_PY_VERSION_HEX` define down to where it is used for the first time, and check if it is defined already, so that it is possible to customize from the compilation command line, just in case there is some unforeseen issue for Python 3.8, 3.9, 3.10.

* tests: Add program dir to path, Custom PyConfig with argv

* refact: clang-formatted

* tests: Add-program-dir-to-path covers both scoped_interpreter overloads

* tests: Add-program-dir-to-path fixed

* tests: Add-program-dir-to-path py_version dependant validation

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
2022-11-30 21:17:59 -08:00
Ethan Steinberg
06003e82b3
Introduce a new style of warning suppression based on push/pop (#4285)
* Introduce a new warning suppression system

* Switch to better name

* Nits
2022-11-28 07:39:38 -08:00
Ralf W. Grosse-Kunstleve
9c18a74e37
Use multiprocessing start_method "forkserver" (#4306)
* Use `multiprocessing` `start_method` `"forkserver"`

Alternative to PR #4305

* Add link to comment under PR #4105

* Unconditionally `pytest.skip("DEADLOCK")` for PyPy Windows

* Remove `SKIP_IF_DEADLOCK` entirely, for simplicity. Hopefully this PR will resolve the deadlocks for good.

* Add "In a nutshell" comment, in response to request by @EricCousineau-TRI
2022-11-22 17:17:02 -08:00
Ralf W. Grosse-Kunstleve
296615ad34
Add macos_brew_install_llvm to ci.yml (#4326)
* Add macos_brew_install_llvm to ci.yml

Added block transferred from PR #4324

* `test_cross_module_exception_translator` xfail 'Homebrew Clang'

* Add `pip install numpy scipy` (tested already under PR #4324).
2022-11-12 12:24:19 -08:00
Ethan Steinberg
ee2b522629
Fix functional.h bug + introduce test to verify that it is fixed (#4254)
* Illustrate bug in functional.h

* style: pre-commit fixes

* Make functional casting more robust / add workaround

* Make function_record* casting even more robust

* See if this fixes PyPy issue

* It still fails on PyPy sadly

* Do not make new CTOR just yet

* Fix test

* Add name to ensure correctness

* style: pre-commit fixes

* Clean up tests + remove ifdef guards

* Add comments

* Improve comments, error handling, and safety

* Fix compile error

* Fix magic logic

* Extract helper function

* Fix func signature

* move to local internals

* style: pre-commit fixes

* Switch to simpler design

* style: pre-commit fixes

* Move to function_record

* style: pre-commit fixes

* Switch to internals, update tests and docs

* Fix lint

* Oops, forgot to resolve last comment

* Fix typo

* Update in response to comments

* Implement suggestion to improve test

* Update comment

* Simple fixes

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com>
2022-11-02 11:32:53 -07:00
Ralf W. Grosse-Kunstleve
3a2c96bd6f
fix: unicode surrogate character in Python exception message. (#4297)
* Fix & test for issue #4288 (unicode surrogate character in Python exception message).

* DRY `message_unavailable_exc`

* fix: add a constexpr

Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>

* style: pre-commit fixes

Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2022-10-31 12:18:05 -04:00
Ralf W. Grosse-Kunstleve
5bc0943ed9
Ensure config, build, toolchain, spelling, etc. issues are not masked. (#4255) 2022-10-30 13:24:41 -07:00
Chekov2k
b07d08f600
Add PYBIND11_SIMPLE_GIL_MANAGEMENT option (cmake, C++ define) (#4216)
* Add option to force the use of the PYPY GIL scoped acquire/release logic to support nested gil access, see https://github.com/pybind/pybind11/issues/1276 and https://github.com/pytorch/pytorch/issues/83101

* Apply suggestions from code review

* Update CMakeLists.txt

* docs: update upgrade guide

* Update docs/upgrade.rst

* All bells & whistles.

* Add Reminder to common.h, so that we will not forget to purge `!WITH_THREAD` branches when dropping Python 3.6

* New sentence instead of semicolon.

* Temporarily pull in snapshot of PR #4246

* Add `test_release_acquire`

* Add more unit tests for nested gil locking

* Add test_report_builtins_internals_keys

* Very minor enhancement: sort list only after filtering.

* Revert change in docs/upgrade.rst

* Add test_multi_acquire_release_cross_module, while also forcing unique PYBIND11_INTERNALS_VERSION for cross_module_gil_utils.cpp

* Hopefully fix apparently new ICC error.

```
2022-10-28T07:57:54.5187728Z -- The CXX compiler identification is Intel 2021.7.0.20220726
...
2022-10-28T07:58:53.6758994Z icpc: remark #10441: The Intel(R) C++ Compiler Classic (ICC) is deprecated and will be removed from product release in the second half of 2023. The Intel(R) oneAPI DPC++/C++ Compiler (ICX) is the recommended compiler moving forward. Please transition to use this compiler. Use '-diag-disable=10441' to disable this message.
2022-10-28T07:58:54.5801597Z In file included from /home/runner/work/pybind11/pybind11/include/pybind11/detail/../detail/type_caster_base.h(15),
2022-10-28T07:58:54.5803794Z                  from /home/runner/work/pybind11/pybind11/include/pybind11/detail/../cast.h(15),
2022-10-28T07:58:54.5805740Z                  from /home/runner/work/pybind11/pybind11/include/pybind11/detail/../attr.h(14),
2022-10-28T07:58:54.5809556Z                  from /home/runner/work/pybind11/pybind11/include/pybind11/detail/class.h(12),
2022-10-28T07:58:54.5812154Z                  from /home/runner/work/pybind11/pybind11/include/pybind11/pybind11.h(13),
2022-10-28T07:58:54.5948523Z                  from /home/runner/work/pybind11/pybind11/tests/cross_module_gil_utils.cpp(13):
2022-10-28T07:58:54.5949009Z /home/runner/work/pybind11/pybind11/include/pybind11/detail/../detail/internals.h(177): error #2282: unrecognized GCC pragma
2022-10-28T07:58:54.5949374Z       PYBIND11_TLS_KEY_INIT(tstate)
2022-10-28T07:58:54.5949579Z       ^
2022-10-28T07:58:54.5949695Z
```

* clang-tidy fixes

* Workaround for PYPY WIN exitcode None

* Revert "Temporarily pull in snapshot of PR #4246"

This reverts commit 23ac16e859150f27fda25ca865cabcb4444e0770.

* Another workaround for PYPY WIN exitcode None

* Clean up how the tests are run "run in process" Part 1: uniformity

* Clean up how the tests are run "run in process" Part 2: use `@pytest.mark.parametrize` and clean up the naming.

* Skip some tests `#if defined(THREAD_SANITIZER)` (tested with TSAN using the Google-internal toolchain).

* Run all tests again but ignore ThreadSanitizer exitcode 66 (this is less likely to mask unrelated ThreadSanitizer issues in the future).

* bug fix: missing common.h include before using `PYBIND11_SIMPLE_GIL_MANAGEMENT`

For the tests in the github CI this does not matter, because
`PYBIND11_SIMPLE_GIL_MANAGEMENT` is always defined from the command line,
but when monkey-patching common.h locally, it matters.

* if process.exitcode is None: assert t_delta > 9.9

* More sophisiticated `_run_in_process()` implementation, clearly reporting `DEADLOCK`, additionally exercised via added `intentional_deadlock()`

* Wrap m.intentional_deadlock in a Python function, for `ForkingPickler` compatibility.

```
>       ForkingPickler(file, protocol).dump(obj)
E       TypeError: cannot pickle 'PyCapsule' object
```

Observed with all Windows builds including mingw but not PyPy, and macos-latest with Python 3.9, 3.10, 3.11 but not 3.6.

* Add link to potential solution for WOULD-BE-NICE-TO-HAVE feature.

* Add `SKIP_IF_DEADLOCK = True` option, to not pollute the CI results with expected `DEADLOCK` failures while we figure out what to do about them.

* Add COPY-PASTE-THIS: gdb ... command (to be used for debugging the detected deadlock)

* style: pre-commit fixes

* Do better than automatic pre-commit fixes.

* Add `PYBIND11_SIMPLE_GIL_MANAGEMENT` to `pytest_report_header()` (so that we can easily know when harvesting deadlock information from the CI logs).

Co-authored-by: Arnim Balzer <arnim@seechange.ai>
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2022-10-30 08:57:23 -07:00
Aaron Gokaslan
b07223fa69
fix: improve bytes to str decoding error handling (#4294)
* (bugfix): Improve bytes to str decoding error handling

* regroup test

* Further broaden tests

* Add another decode error test

* Fix bug in tests

* Reviewer suggestions
2022-10-29 11:12:24 -04:00
Lalaland
8ea75ab4d7
Fix casts to void* (#4275)
* Fix casts to void*

* Improve tests

* style: pre-commit fixes

* remove c style cast

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com>
2022-10-22 16:52:35 -07:00
Aaron Gokaslan
17c1e27b3d
fix: Revert pfect args make iterator (#4234)
* Revert "chore: perfectly forward all make_iterator args (#3980)"

This reverts commit 8da58da539.

* Redo unrelated optimization in commit

* Add tests for ambiguous overloads
2022-10-21 18:04:01 -04:00
Henry Schreiner
91cfb77022
Revert "feat: add entrypoint for cmake modules dir" (#4270)
* Revert "feat: add entrypoint for cmake modules dir (#4258)"

This reverts commit 1d4a65e2f1.

* docs: revert changelog mention too
2022-10-21 17:25:53 -04:00
Henry Schreiner
1d4a65e2f1
feat: add entrypoint for cmake modules dir (#4258)
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
2022-10-20 10:35:18 -04:00
Lalaland
fab1eebe2c
First draft of Eigen::Tensor support (#4201)
* First draft of Eigen::Tensor support

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix build errors

* Weird allocator stuff?

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Remove unused + additional allocator junk

* Disable warning

* Use constexpr

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* clang tidy fixes

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Resolve comments

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Remove auto constexpr function

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Try again for older C++

* Oops forgot constexpr

* Move to new files as suggested

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix weird tests

* Fix nits

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Oops, forgot import

* Fix clang 3.6 bug

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* More comprehensive test suite

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Refactor allocators to make things more clear

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Switch to std::copy

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Switch to DSizes instead of array

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Address feedback

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix python + dummy c++ change to trigger build

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Alignment

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add include guard

* Forgot inline

* Fix compiler warning

* Remove bad test

* Better type signatures

* Add guards to make compiler requirements more explicit

* style: pre-commit fixes

* Force rerun of tests due to flake

* style: pre-commit fixes

* Keep pragmas & all related comments together, add PLEASE KEEP IN SYNC

* Move headers out of detail

* style: pre-commit fixes

* Fix cmake

* Improve casting

* style: pre-commit fixes

* Add a ton more tests + refactor

* Improve names

* style: pre-commit fixes

* Update include/pybind11/eigen/tensor.h

Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>

* Fix tests

* style: pre-commit fixes

* Update

* Add a test to verify that strange numpy arrays work

* Fix dumb compiler warning

* Better tests

* Better tests

* Fix tests

* style: pre-commit fixes

* More test fixes

* style: pre-commit fixes

* A ton more test coverage

* Fix tests

* style: pre-commit fixes

* style: pre-commit fixes

* Add back constexpr

* Another test

* style: pre-commit fixes

* Improve tests

* Whoops

* Less magic numbers

* Update tests/test_eigen_tensor.py

Co-authored-by: Sergiu Deitsch <sergiud@users.noreply.github.com>

* Update tests/test_eigen_tensor.py

Co-authored-by: Sergiu Deitsch <sergiud@users.noreply.github.com>

* style: pre-commit fixes

* Fix tests

* style: pre-commit fixes

* Fix memory leak

* style: pre-commit fixes

* Fix order

* style: pre-commit fixes

* Add test to make sure unsafe casts fail

* Minor bug fix to work on 32 bit machines

* Implement convert flag

* style: pre-commit fixes

* Switch to correct TensorMap const use

* style: pre-commit fixes

* Support older versions of eigen

* Weird c++ compilers

* Fix Eigen bug

* Fix another eigen bug

* Yet another eigen bug

* Potential flakes?

* style: pre-commit fixes

* Rerun tests with dummy exception to find out what is going on

* Another dummy test run

* Ablate more

* Found the broken test?

* One step closer

* one step further

* Double check

* one thing at a time

* Give up and disable the test

* Clang lies about being gcc

* Oops, fix matrix test

* style: pre-commit fixes

* Add tests to verify scalar conversions

* style: pre-commit fixes

* Fix nits

* Support no_array

* Fix tests

* style: pre-commit fixes

* Silence compiler warning

* Improve build system for ancient compilers

* Make clang happy

* Make gcc happy

* Implement Skylion's suggestions

* Fix warning

* Inline const pointer check

* Implement suggestions

* style: pre-commit fixes

* Improve tests

* Typo

* style: pre-commit fixes

* Support Google's build environment

* style: pre-commit fixes

* Update include/pybind11/eigen/tensor.h

Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>

* style: pre-commit fixes

* Test cleanup per Skylion

* Switch to remvove_cv_t

* Cleaner test

* style: pre-commit fixes

* Remove tensor from eigen.h, update tests

* style: pre-commit fixes

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com>
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
Co-authored-by: Sergiu Deitsch <sergiud@users.noreply.github.com>
2022-10-18 16:54:16 -07:00
Aaron Gokaslan
b926396bdf
bugfix: py contains raises errors when appropiate (#4209)
* bugfix: contains now throws an exception if the key is not hashable

* Fix tests and improve robustness

* Remove todo

* Workaround PyPy corner case

* PyPy xfail

* Fix typo

* fix xfail

* Make clang-tidy happy

* Remove redundant exc checking
2022-10-17 16:15:08 -07:00
Ralf W. Grosse-Kunstleve
da104a9efd
Reproducer and fix for issue encountered in smart_holder update. (#4228)
* Reproducer for issue encountered in smart_holder update.

* clang-tidy compatibility (untested).

* Add `enable_if_t` to workaround.

* Bug fix: Move `PYBIND11_USING_WORKAROUND_FOR_CUDA_11_4_THROUGH_8` determination to detail/common.h

So that it actually is defined in pybind11.h

* Try using the workaround (which is nicer than the original code) universally.

* Reduce reproducer for CUDA 11.7 issue encountered in smart_holder update.

This commit tested in isolation on top of current master + first version of reproducer (62311eb431).

Succeeds with Debian Clang 14.0.6 C++17 (and probably all other compilers).

Fails for CUDA 11.7:

```
cd /build/tests && /usr/local/cuda/bin/nvcc -forward-unknown-to-host-compiler -Dpybind11_tests_EXPORTS -I/mounted_pybind11/include -isystem=/usr/include/python3.10 -g --generate-code=arch=compute_52,code=[compute_52,sm_52] -Xcompiler=-fPIC -Xcompiler=-fvisibility=hidden -Werror all-warnings -std=c++17 -MD -MT tests/CMakeFiles/pybind11_tests.dir/test_class.cpp.o -MF CMakeFiles/pybind11_tests.dir/test_class.cpp.o.d -x cu -c /mounted_pybind11/tests/test_class.cpp -o CMakeFiles/pybind11_tests.dir/test_class.cpp.o
/mounted_pybind11/tests/test_class.cpp(53): error: more than one instance of overloaded function "pybind11::class_<type_, options...>::def [with type_=test_class::pr4220_tripped_over_this::Empty0, options=<>]" matches the argument list:
            function template "pybind11::class_<test_class::pr4220_tripped_over_this::Empty0> &pybind11::class_<type_, options...>::def(const char *, Func &&, const Extra &...) [with type_=test_class::pr4220_tripped_over_this::Empty0, options=<>]"
/mounted_pybind11/include/pybind11/pybind11.h(1557): here
            function template "pybind11::class_<test_class::pr4220_tripped_over_this::Empty0> &pybind11::class_<type_, options...>::def(const T &, const Extra &...) [with type_=test_class::pr4220_tripped_over_this::Empty0, options=<>]"
/mounted_pybind11/include/pybind11/pybind11.h(1586): here
            argument types are: (const char [8], <unknown-type>)
            object type is: pybind11::class_<test_class::pr4220_tripped_over_this::Empty0>

1 error detected in the compilation of "/mounted_pybind11/tests/test_class.cpp".
```
2022-10-09 21:50:35 -07:00
Daniel Galvez
7c6f2f80a7
fix: PyCapsule_GetDestructor is allowed to return a nullptr destructor (#4221)
* fix: PyCapsule_GetDestructor is allowed to return a nullptr destructor

Previously, this code would error out if the destructor happened to be
a nullptr. This is incorrect. nullptrs are allowed for capsule
destructors.

"It is legal for a capsule to have a NULL destructor. This makes a
NULL return code somewhat ambiguous; use PyCapsule_IsValid() or
PyErr_Occurred() to disambiguate."

See:

https://docs.python.org/3/c-api/capsule.html#c.PyCapsule_GetDestructor

I noticed this while working on a type caster related to #3858 DLPack
happens to allow the destructor not to be defined on a capsule, and I
encountered such a case. See:

e2bdd3bee8/include/dlpack/dlpack.h (L219)

* Add test for the fix.

* Update tests/test_pytypes.cpp

I tried this locally and it works!
I never knew that there are cases where `reinterpret_cast` does not work but `static_cast` does. Let's see if all compilers are happy with this.

Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>

* style: pre-commit fixes

Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgkio@gmail.com>
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2022-10-07 12:27:54 -07:00
Ralf W. Grosse-Kunstleve
4a42156209
test_eigen.py test_nonunit_stride_to_python bug fix (ASAN failure) (#4217)
* Disable test triggering ASAN failure (to pin-point where the problem is).

* Fix unsafe "block" implementation in test_eigen.cpp

* Undo changes (i.e. revert back to master).

* Detect "type_caster for Eigen::Ref made a copy."

This is achieved without
* reaching into internals,
* making test_eigen.cpp depend on pybind11/numpy.h.

* Add comment pointing to PR, for easy reference.
2022-10-07 09:20:38 -07:00
Axel Huebl
0b4c1bc286
test: ConstructorStats newline (PyPy) (#4167)
This looks like it lacks a newline.
2022-08-29 23:25:01 -04:00
Brad Messer
a48ec3e882
Words matter updates (#4155)
* Remove sanity check from code base.

* Use main over master.

* Better alternative that doesn't collide with language keywords/frequent usage words.
2022-08-24 07:34:31 -07:00
Ralf W. Grosse-Kunstleve
68e6fdaa90
embed.h Python 3.11 config.use_environment=1 + PYTHONPATH test (#4119)
* Add debug fprintf to test_interpreter.cpp

* Update `sys.path` from `PYTHONPATH` in Python >= 3.11 branch of `initialize_interpreter()`

* Use `config.isolated = 0; config.use_environment = 1;`

As suggsted by @vstinner here: https://github.com/pybind/pybind11/pull/4119#issuecomment-1219442853

* Add `TEST_CASE("PYTHONPATH is used to update sys.path")`

* Fix clang-tidy error.

* Use `_putenv_s()` under Windows.

* Fix clang-tidy error: argument name ... in comment does not match parameter name

* Remove slash from PYTHONPATH addition, to work around Windows slash-vs-backslash issue.

* Use `py::str(...)` instead of `.attr("__str__")` as suggested by @skylion007

Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>

Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
2022-08-21 09:44:01 -07:00
Aaron Gokaslan
b884b9dc6b
chore: Add pytests for constructing pytypes from iterable (#4138)
* Add some additional pytests

* Reorder tests

* Further reorder tests

* remove stray lines

* remove unused fixtures
2022-08-11 16:14:17 -04:00
Eli Schwartz
5bdd3d59be
feat(cmake): add installation support for pkg-config dependency detection (#4077)
* add installation support for pkg-config dependency detection

pkg-config is a buildsystem-agnostic alternative to
`pybind11Config.cmake` that can be used from build systems other than
cmake.

Fixes #230

* tests: add test for pkg config

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
2022-08-09 00:02:45 -04:00
pre-commit-ci[bot]
ba5ccd845a
[pre-commit.ci] pre-commit autoupdate (#4104)
* [pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/asottile/pyupgrade: v2.37.2 → v2.37.3](https://github.com/asottile/pyupgrade/compare/v2.37.2...v2.37.3)
- [github.com/hadialqattan/pycln: v2.0.4 → v2.1.1](https://github.com/hadialqattan/pycln/compare/v2.0.4...v2.1.1)
- [github.com/PyCQA/flake8: 4.0.1 → 5.0.2](https://github.com/PyCQA/flake8/compare/4.0.1...5.0.2)

* fix: minor touchups for flake8

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
2022-08-03 11:38:07 -04:00
Thomas Eding
f8e8403b85
Open pybind11 namespace with consistent visility. (#4098) 2022-08-01 11:31:31 -07:00
Ralf W. Grosse-Kunstleve
3665530264
Add -DPYBIND11_WERROR=ON to mingw cmake commands (#4073)
* Add `-DPYBIND11_WERROR=ON` to mingw cmake commands (and `-DCMAKE_VERBOSE_MAKEFILE:BOOL=ON`).

* Using no-destructor idiom to side-step overzealous MINGW warning.

* Add __MINGW32__ pragma GCC diagnostic ignored in eigen.h

* Add another no-destructor workaround.

* Temporarily add -k (keep-going) flags to hopefully speed up finding all warnings.

* Revert "Temporarily add -k (keep-going) flags to hopefully speed up finding all warnings."

This reverts commit f36b0af8f9.

* Very minor shuffle to avoid MSVC warnings.

* Remove all `:BOOL` as suggested by @henryiii
2022-08-01 06:18:48 -07:00
Ralf W. Grosse-Kunstleve
9a2963734d
More systematic gcc & clang coverage (#4083)
* More systematic gcc coverage, based on https://github.com/pybind/pybind11/pull/4074#issuecomment-1188385580

* Fix complete fail.

* Resolve GCC 11 & 12 "redundant move in return statement" warnings.

* Also add clang 11, 12, 13 (to gather info for warning suppressions).

* Add & use `PYBIND11_DETECTED_CLANG_WITH_MISLEADING_CALL_STD_MOVE_EXPLICITLY_WARNING`
2022-07-21 06:40:34 -07:00
Ralf W. Grosse-Kunstleve
cb35a3c143
For PyPy only, re-enable old behavior (runs the risk of masking bugs) (#4079)
* For PyPy only, re-enable old behavior (likely to mask bugs), to avoid segfault with unknown root cause.

Change prompted by https://github.com/pybind/pybind11/issues/4075

* Undo the change in tests/test_exceptions.py

I turns out (I forgot) that PyPy segfaults in `test_flaky_exception_failure_point_init` already before the `MISMATCH` code path is reached:

https://github.com/pybind/pybind11/runs/7383663596

```
RPython traceback:
test_exceptions.py .......X.........Error in cpyext, CPython compatibility layer:
  File "pypy_module_cpyext.c", line 14052, in wrapper_second_level__star_3_1
  File "pypy_module_cpyext_1.c", line 35750, in not_supposed_to_fail
Fatal Python error: Segmentation fault
Stack (most recent call first, approximate line numbers):
  File "/home/runner/work/pybind11/pybind11/tests/test_exceptions.py", line 306 in test_flaky_exception_failure_point_init
The function PyErr_NormalizeException was not supposed to fail
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/python.py", line 185 in pytest_pyfunc_call
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_callers.py", line 9 in _multicall
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_manager.py", line 77 in _hookexec
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_hooks.py", line 244 in __call__
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/python.py", line 1716 in runtest
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/runner.py", line 159 in pytest_runtest_call
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_callers.py", line 9 in _multicall
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_manager.py", line 77 in _hookexec
Fatal error in cpyext, CPython compatibility layer, calling PyErr_NormalizeException
Either report a bug or consider not using this particular extension
<SystemError object at 0x7fcc8cea6868>
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_hooks.py", line 244 in __call__
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/runner.py", line 261 in <lambda>
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/runner.py", line 317 in from_call
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/runner.py", line 246 in call_runtest_hook
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/runner.py", line 218 in call_and_report
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/runner.py", line 118 in runtestprotocol
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/runner.py", line 110 in pytest_runtest_protocol
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_callers.py", line 9 in _multicall
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_manager.py", line 77 in _hookexec
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_hooks.py", line 244 in __call__
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/main.py", line 335 in pytest_runtestloop
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_callers.py", line 9 in _multicall
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_manager.py", line 77 in _hookexec
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_hooks.py", line 244 in __call__
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/main.py", line 318 in _main
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/main.py", line 255 in wrap_session
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/main.py", line 314 in pytest_cmdline_main
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_callers.py", line 9 in _multicall
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_manager.py", line 77 in _hookexec
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pluggy/_hooks.py", line 244 in __call__
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/config/__init__.py", line 133 in main
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/_pytest/config/__init__.py", line 181 in console_main
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/site-packages/pytest/__main__.py", line 1 in <module>
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/lib-python/3/runpy.py", line 62 in _run_code
  File "/opt/hostedtoolcache/PyPy/3.7.13/x64/lib-python/3/runpy.py", line 170 in _run_module_as_main
  File "<builtin>/app_main.py", line 109 in run_toplevel
  File "<builtin>/app_main.py", line 652 in run_command_line
  File "<builtin>/app_main.py", line 996 in entry_point
Segmentation fault (core dumped)
```

* Add test_pypy_oserror_normalization

* Disable new `PYPY_VERSION` `#if`, to verify that the new test actually fails.

* Restore PYPY_VERSION workaround and update comment to reflect what was learned.

* [ci skip] Fix trivial oversight in comment.
2022-07-21 06:38:00 -07:00
Aaron Gokaslan
f47f1edfe8
Fix #3812 and fix const of inplace assignments (#4065)
* Fix #3812 and fix const of inplace assignments

* Fix missing tests

* Revert operator overloading changes

* calculate answer first for tests

* Simplify tests

* Add more tests

* Add a couple more tests

* Add test_inplace_lshift, test_inplace_rshift for completeness.

* Update tests

* Shortcircuit on self assigment and address reviewer comment

* broaden skip for self assignment

* One more reviewer comment

* Document opt behavior and make consistent

* Revert unnecessary change

* Clarify comment

Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
2022-07-20 11:42:24 -04:00
Ralf W. Grosse-Kunstleve
1d81191077
Disable implicit conversion of 0 to pybind11::handle. (#4008)
* Disable implicit conversion from `0` to `pybind11::handle`.

* Reverse or-ed condition in an attempt to resolve GCC 8.3.0 errors (i386/debian:buster).

* Trying the simpler `std::is_same<T, PyObject *>`

* Add implicit_conversion_from_pytorch_THPObjectPtr_to_handle test.

* Accommodate types with implicit conversions to `PyObject *`, other than `handle` & `handle` subclasses, or integral types.

* Fix copy-paste mishap (picked wrong name).

* Revamp SFINAE construct to actually fix the pytorch issue (already validated against pytorch proper).

The first version of the reduced pytorch code was critically missing the move ctor. The first version of the accompanying test was meaningless.

Note: It turns out the `!std::is_arithmetic<T>` condition is not needed: `int` is not in general implicitly convertible to `PyObject *`, only the literal `0` is.

* Use `NOLINT(performance-noexcept-move-constructor)` for reduced code from the wild (rather than changing the code).

* Use any_of, all_of, negation. It turns out to clang-format nicer.

* Clean up comments for changed code.

* Reduce pytorch situation further, add test for operator ... const.

* Use `none_of` as suggested by @skylion007

* Add `pure_compile_tests_for_handle_from_PyObject_pointers()`

* Fix inconsequential oversight (retested).

* Factor our `is_pyobj_ptr_or_nullptr_t` to make the SFINAE conditions more readable.

* Remove stray line (oversight).

* Make the `pure_compile_tests_for_handle_from_PyObject_pointers()` "rhs-const-complete", too.

* Remove the temporary PYBIND11_UNDO_PR4008 `#ifdef`.
2022-07-14 09:53:39 -07:00
Henry Schreiner
7c3a031766
chore: bump clang-tidy to 13 (#3997)
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
2022-07-13 09:06:32 -05:00
Ralf W. Grosse-Kunstleve
85bc088441
Report C++ Info: via pytest_report_header() (#4046)
* Report `C++ Info:` from `pytest_configure()`

* Use pytest_report_header() as suggested by @skylion007
2022-07-07 17:51:44 -07:00
Aaron Gokaslan
2af163d9c7
Fix: 3.11 beta support (#3923)
* Placeholder commit for 3.11 testing

* Does this fix it?

* Try suggestion

* Placeholder commit for 3.11 testing

* Does this fix it?

* Try suggestion

* fix: try using modern init for embedded interp

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* fix: error message changed in 3.11

* fix: apply logic in Python manually

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* fix autodetect dynamic attrs in 3.11

* fix: include error message if possible in error

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* ci: enable standard Python 3.11 testing

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* Make dynamic attrs condtiion exclusive to ver.

Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
2022-07-06 16:35:12 -04:00
Matthew Woehlke
479e9a50f3
Fix arrays with zero-size dimensions (#4038)
When converting an array to an Eigen matrix, ignore the strides if any
dimension size is 0. If the array is empty, the strides aren't relevant,
and especially numpy ≥ 1.23 explicitly sets the strides to 0 in this
case. (See numpy commit dd5ab7b11520.)

Update tests to verify that this works, and continues to work.
2022-06-29 11:37:16 -07:00
Aaron Gokaslan
554c045377
enable two new clang-tidy checks (#3988)
* enable two new clang-tidy checks

* Use better loop var for char
2022-06-06 12:15:45 -04:00
Sergei Lebedev
a05bc3d235
error_already_set::what() is now constructed lazily (#1895)
* error_already_set::what() is now constructed lazily

Prior to this commit throwing error_already_set was expensive due to the
eager construction of the error string (which required traversing the
Python stack). See #1853 for more context and an alternative take on the
issue.

Note that error_already_set no longer inherits from std::runtime_error
because the latter has no default constructor.

* Do not attempt to normalize if no exception occurred

This is not supported on PyPy-2.7 5.8.0.

* Extract exception name via tp_name

This is faster than dynamically looking up __name__ via GetAttrString.
Note though that the runtime of the code throwing an error_already_set
will be dominated by stack unwinding so the improvement will not be
noticeable.

Before:

396 ns ± 0.913 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

After:

277 ns ± 0.549 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

Benchmark:

const std::string foo() {
    PyErr_SetString(PyExc_KeyError, "");
    const std::string &s = py::detail::error_string();
    PyErr_Clear();
    return s;
}

PYBIND11_MODULE(foo, m) {
    m.def("foo", &::foo);
}

* Reverted error_already_set to subclass std::runtime_error

* Revert "Extract exception name via tp_name"

The implementation of __name__ is slightly more complex than that.
It handles the module name prefix, and heap-allocated types. We could
port it to pybind11 later on but for now it seems like an overkill.

This reverts commit f1435c7e6b.

* Cosmit following @YannickJadoul's comments

Note that detail::error_string() no longer calls PyException_SetTraceback
as it is unncessary for pretty-printing the exception.

* Fixed PyPy build

* Moved normalization to error_already_set ctor

* Fix merge bugs

* Fix more merge errors

* Improve formatting

* Improve error message in rare case

* Revert back if statements

* Fix clang-tidy

* Try removing mutable

* Does build_mode release fix it

* Set to Debug to expose segfault

* Fix remove set error string

* Do not run error_string() more than once

* Trying setting the tracebackk to the value

* guard if m_type is null

* Try to debug PGI

* One last try for PGI

* Does reverting this fix PyPy

* Reviewer suggestions

* Remove unnecessary initialization

* Add noexcept move and explicit fail throw

* Optimize error_string creation

* Fix typo

* Revert noexcept

* Fix merge conflict error

* Abuse assignment operator

* Revert operator abuse

* See if we still need debug

* Remove unnecessary mutable

* Report "FATAL failure building pybind11::error_already_set error_string" and terminate process.

* Try specifying noexcept again

* Try explicit ctor

* default ctor is noexcept too

* Apply reviewer suggestions, simplify code, and make helper method private

* Remove unnecessary include

* Clang-Tidy fix

* detail::obj_class_name(), fprintf with [STDERR], [STDOUT] tags, polish comments

* consistently check m_lazy_what.empty() also in production builds

* Make a comment slightly less ambiguous.

* Bug fix: Remove `what();` from `restore()`.

It sure would need to be guarded by `if (m_type)`, otherwise `what()` fails and masks that no error was set (see update unit test). But since `error_already_set` is copyable, there is no point in releasing m_type, m_value, m_trace, therefore we can just as well avoid the runtime overhead of force-building `m_lazy_what`, it may never be used.

* Replace extremely opaque (unhelpful) error message with a truthful reflection of what we know.

* Fix clang-tidy error [performance-move-constructor-init].

* Make expected error message less specific.

* Various changes.

* bug fix: error_string(PyObject **, ...)

* Putting back the two eager PyErr_NormalizeException() calls.

* Change error_already_set() to call pybind11_fail() if the Python error indicator not set. The net result is that a std::runtime_error is thrown instead of error_already_set, but all tests pass as is.

* Remove mutable (fixes oversight in the previous commit).

* Normalize the exception only locally in error_string(). Python 3.6 & 3.7 test failures expected. This is meant for benchmarking, to determine if it is worth the trouble looking into the failures.

* clang-tidy: use auto

* Use `gil_scoped_acquire_local` in `error_already_set` destructor. See long comment.

* For Python < 3.8: `PyErr_NormalizeException` before `PyErr_WriteUnraisable`

* Go back to replacing the held Python exception with then normalized exception, if & when needed. Consistently document the side-effect.

* Slightly rewording comment. (There were also other failures.)

* Add 1-line comment for obj_class_name()

* Benchmark code, with results in this commit message.

          function                   #calls  test time [s]  μs / call
master    pure_unwind                729540      1.061      14.539876
          err_set_unwind_err_clear   681476      1.040      15.260282
          err_set_error_already_set  508038      1.049      20.640525
          error_already_set_restore  555578      1.052      18.933288
          pr1895_original_foo        244113      1.050      43.018168
                                                                       PR / master
PR #1895  pure_unwind                736981      1.054      14.295685       98.32%
          err_set_unwind_err_clear   685820      1.045      15.237399       99.85%
          err_set_error_already_set  661374      1.046      15.811879       76.61%
          error_already_set_restore  669881      1.048      15.645176       82.63%
          pr1895_original_foo        318243      1.059      33.290806       77.39%

master @ commit ad146b2a18

Running tests in directory "/usr/local/google/home/rwgk/forked/pybind11/tests":
============================= test session starts ==============================
platform linux -- Python 3.9.10, pytest-6.2.3, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /usr/local/google/home/rwgk/forked/pybind11/tests, configfile: pytest.ini
collecting ... collected 5 items

test_perf_error_already_set.py::test_perf[pure_unwind]
PERF pure_unwind,729540,1.061,14.539876
PASSED
test_perf_error_already_set.py::test_perf[err_set_unwind_err_clear]
PERF err_set_unwind_err_clear,681476,1.040,15.260282
PASSED
test_perf_error_already_set.py::test_perf[err_set_error_already_set]
PERF err_set_error_already_set,508038,1.049,20.640525
PASSED
test_perf_error_already_set.py::test_perf[error_already_set_restore]
PERF error_already_set_restore,555578,1.052,18.933288
PASSED
test_perf_error_already_set.py::test_perf[pr1895_original_foo]
PERF pr1895_original_foo,244113,1.050,43.018168
PASSED

============================== 5 passed in 12.38s ==============================

pr1895 @ commit 8dff51d12e

Running tests in directory "/usr/local/google/home/rwgk/forked/pybind11/tests":
============================= test session starts ==============================
platform linux -- Python 3.9.10, pytest-6.2.3, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /usr/local/google/home/rwgk/forked/pybind11/tests, configfile: pytest.ini
collecting ... collected 5 items

test_perf_error_already_set.py::test_perf[pure_unwind]
PERF pure_unwind,736981,1.054,14.295685
PASSED
test_perf_error_already_set.py::test_perf[err_set_unwind_err_clear]
PERF err_set_unwind_err_clear,685820,1.045,15.237399
PASSED
test_perf_error_already_set.py::test_perf[err_set_error_already_set]
PERF err_set_error_already_set,661374,1.046,15.811879
PASSED
test_perf_error_already_set.py::test_perf[error_already_set_restore]
PERF error_already_set_restore,669881,1.048,15.645176
PASSED
test_perf_error_already_set.py::test_perf[pr1895_original_foo]
PERF pr1895_original_foo,318243,1.059,33.290806
PASSED

============================== 5 passed in 12.40s ==============================

clang++ -o pybind11/tests/test_perf_error_already_set.os -c -std=c++17 -fPIC -fvisibility=hidden -Os -flto -Wall -Wextra -Wconversion -Wcast-qual -Wdeprecated -Wnon-virtual-dtor -Wunused-result -isystem /usr/include/python3.9 -isystem /usr/include/eigen3 -DPYBIND11_STRICT_ASSERTS_CLASS_HOLDER_VS_TYPE_CASTER_MIX -DPYBIND11_TEST_BOOST -Ipybind11/include -I/usr/local/google/home/rwgk/forked/pybind11/include -I/usr/local/google/home/rwgk/clone/pybind11/include /usr/local/google/home/rwgk/forked/pybind11/tests/test_perf_error_already_set.cpp

clang++ -o lib/pybind11_tests.so -shared -fPIC -Os -flto -shared ...

Debian clang version 13.0.1-3+build2
Target: x86_64-pc-linux-gnu
Thread model: posix

* Changing call_repetitions_target_elapsed_secs to 0.1 for regular unit testing.

* Adding in `recursion_depth`

* Optimized ctor

* Fix silly bug in recurse_first_then_call()

* Add tests that have equivalent PyErr_Fetch(), PyErr_Restore() but no try-catch.

* Add call_error_string to tests. Sample only recursion_depth 0, 100.

* Show lazy-what speed-up in percent.

* Include real_work in benchmarks.

* Replace all PyErr_SetString() with generate_python_exception_with_traceback()

* Better organization of test loops.

* Add test_error_already_set_copy_move

* Fix bug in newly added test (discovered by clang-tidy): actually use move ctor

* MSVC detects the unreachable return

* change test_perf_error_already_set.py back to quick mode

* Inherit from std::exception (instead of std::runtime_error, which does not make sense anymore with the lazy what)

* Special handling under Windows.

* print with leading newline

* Removing test_perf_error_already_set (copies are under 7765113fbb).

* Avoid gil and scope overhead if there is nothing to release.

* Restore default move ctor. "member function" instead of "function" (note that "method" is Python terminology).

* Delete error_already_set copy ctor.

* Make restore() non-const again to resolve clang-tidy failure (still experimenting).

* Bring back error_already_set copy ctor, to see if that resolves the 4 MSVC test failures.

* Add noexcept to error_already_set copy & move ctors (as suggested by @skylion007 IIUC).

* Trying one-by-one noexcept copy ctor for old compilers.

* Add back test covering copy ctor. Add another simple test that exercises the copy ctor.

* Exclude more older compilers from using the noexcept = default ctors. (The tests in the previous commit exposed that those are broken.)

* Factor out & reuse gil_scoped_acquire_local as gil_scoped_acquire_simple

* Guard gil_scoped_acquire_simple by _Py_IsFinalizing() check.

* what() GIL safety

* clang-tidy & Python 3.6 fixes

* Use `gil_scoped_acquire` in dtor, copy ctor, `what()`. Remove `_Py_IsFinalizing()` checks (they are racy: https://github.com/python/cpython/pull/28525).

* Remove error_scope from copy ctor.

* Add `error_scope` to `get_internals()`, to cover the situation that `get_internals()` is called from the `error_already_set` dtor while a new Python error is in flight already. Also backing out `gil_scoped_acquire_simple` change.

* Add `FlakyException` tests with failure triggers in `__init__` and `__str__`

THIS IS STILL A WORK IN PROGRESS. This commit is only an important resting point.

This commit is a first attempt at addressing the observation that `PyErr_NormalizeException()` completely replaces the original exception if `__init__` fails. This can be very confusing even in small applications, and extremely confusing in large ones.

* Tweaks to resolve Py 3.6 and PyPy CI failures.

* Normalize Python exception immediately in error_already_set ctor.

For background see: https://github.com/pybind/pybind11/pull/1895#issuecomment-1135304081

* Fix oversights based on CI failures (copy & move ctor initialization).

* Move @pytest.mark.xfail("env.PYPY") after @pytest.mark.parametrize(...)

* Use @pytest.mark.skipif (xfail does not work for segfaults, of course).

* Remove unused obj_class_name_or() function (it was added only under this PR).

* Remove already obsolete C++ comments and code that were added only under this PR.

* Slightly better (newly added) comments.

* Factor out detail::error_fetch_and_normalize. Preparation for producing identical results from error_already_set::what() and detail::error_string(). Note that this is a very conservative refactoring. It would be much better to first move detail::error_string into detail/error_string.h

* Copy most of error_string() code to new error_fetch_and_normalize::complete_lazy_error_string()

* Remove all error_string() code from detail/type_caster_base.h. Note that this commit includes a subtle bug fix: previously error_string() restored the Python error, which will upset pybind11_fail(). This never was a problem in practice because the two PyType_Ready() calls in detail/class.h do not usually fail.

* Return const std::string& instead of const char * and move error_string() to pytypes.h

* Remove gil_scope_acquire from error_fetch_and_normalize, add back to error_already_set

* Better handling of FlakyException __str__ failure.

* Move error_fetch_and_normalize::complete_lazy_error_string() implementation from pybind11.h to pytypes.h

* Add error_fetch_and_normalize::release_py_object_references() and use from error_already_set dtor.

* Use shared_ptr for m_fetched_error => 1. non-racy, copy ctor that does not need the GIL; 2. enables guard against duplicate restore() calls.

* Add comments.

* Trivial renaming of a newly introduced member function.

* Workaround for PyPy

* Bug fix (oversight). Only valgrind got this one.

* Use shared_ptr custom deleter for m_fetched_error in error_already_set. This enables removing the dtor, copy ctor, move ctor completely.

* Further small simplification. With the GIL held, simply deleting the raw_ptr takes care of everything.

* IWYU cleanup

```
iwyu version: include-what-you-use 0.17 based on Debian clang version 13.0.1-3+build2
```

Command used:

```
iwyu -c -std=c++17 -DPYBIND11_TEST_BOOST -Iinclude/pybind11 -I/usr/include/python3.9 -I/usr/include/eigen3 include/pybind11/pytypes.cpp
```

pytypes.cpp is a temporary file: `#include "pytypes.h"`

The raw output is very long and noisy.

I decided to use `#include <cstddef>` instead of `#include <cstdio>` for `std::size_t` (iwyu sticks to the manual choice).

I ignored all iwyu suggestions that are indirectly covered by `#include <Python.h>`.

I manually verified that all added includes are actually needed.

Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
2022-06-02 16:17:38 -07:00
Aaron Gokaslan
58802de41b
perf: Add object rvalue overload for accessors. Enables reference stealing (#3970)
* Add object rvalue overload for accessors. Enables reference stealing

* Fix comments

* Fix more comment typos

* Fix bug

* reorder declarations for clarity

* fix another perf bug

* should be static

* future proof operator overloads

* Fix perfect forwarding

* Add a couple of tests

* Remove errant include

* Improve test documentation

* Add dict test

* add object attr tests

* Optimize STL map caster and cleanup enum

* Reorder to match declarations

* adjust increfs

* Remove comment

* revert value change

* add missing move
2022-06-01 15:19:13 -04:00
Ralf W. Grosse-Kunstleve
9f7b3f735a
addl unit tests for PR #3970 (#3977)
* Add test_perf_accessors (to be merged into test_pytypes).

* Python < 3.8 f-string compatibility

* Use thread_local in inc_ref_counter()

* Intentional breakage, brute-force way to quickly find out how many platforms reach the PYBIND11_HANDLE_REF_DEBUG code, with and without threads.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Remove Intentional breakage

* Drop perf test, move inc_refs tests to test_pytypes

* Fold in PR #3970 with `#ifdef`s

* Complete test coverage for all newly added code.

* Condense new unit tests via a simple local helper macro.

* Remove PYBIND11_PR3970 define. See https://github.com/pybind/pybind11/pull/3977#issuecomment-1142526417

* Move static keyword first (fixes silly oversight).

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2022-05-31 12:59:19 -07:00
Ralf W. Grosse-Kunstleve
b24c5ed204
Replace "Unknown internal error occurred" with a more helpful message. (#3982) 2022-05-31 11:54:33 -07:00
Ralf W. Grosse-Kunstleve
de4ba92c9f
Add error_scope to detail::get_internals() (#3981)
* Add `error_scope` to `detail::get_internals()`

* Adjust test to tolerate macOS PyPy behavior.
2022-05-31 11:51:13 -07:00
Ralf W. Grosse-Kunstleve
748ae2270b
Add missing error handling to module_::def_submodule (#3973)
* Add missing error handling to module_::def_submodule

* Add test_def_submodule_failures

* PyPy only: Skip test with trigger for PyModule_GetName() failure.

* Reapply minor fix that accidentally got lost in transfer from PR #3964
2022-05-28 16:40:57 -07:00
Ralf W. Grosse-Kunstleve
8d14e666e3
fix: avoid catch (...) for expected import numpy failures (#3974)
* Replace import numpy catch (...) with catch (error_already_set)

* Add missing const (not sure how those got lost).
2022-05-26 11:07:40 -04:00
Ralf W. Grosse-Kunstleve
2c549eb7aa
Move PyErr_NormalizeException() up a few lines (#3971)
* Add error_already_set_what what tests, asserting the status quo.

* Move PyErr_NormalizeException() up a few lines.

* @pytest.mark.skipif("env.PYPY") from PR #1895 is required even for this much simpler PR

* Move PyException_SetTraceback() with PyErr_NormalizeException() as suggested by @skylion007

* Insert a std::move() as suggested by @skylion007
2022-05-25 21:44:55 -07:00
Maarten Baert
4624e8e164
Don't return pointers to static objects with return_value_policy::take_ownership. (#3946)
* Don't return pointers to static objects with return_value_policy::take_ownership.

This fixes -Wfree-nonheap-object warnings produced by GCC.

* Use return value policy fix instead

Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
2022-05-24 13:46:31 -04:00
Henry Schreiner
1a7b12983e
ci: fix cuda issue & MSVC spurious warning (#3950)
* ci: fix cuda issue

* ci: cuda 11.3-11.4 produce warnings -> errors

* tests: ignore unused warning for MSVC

* Update tests/CMakeLists.txt
2022-05-16 17:27:19 -04:00