* Add option to force the use of the PYPY GIL scoped acquire/release logic to support nested gil access, see https://github.com/pybind/pybind11/issues/1276 and https://github.com/pytorch/pytorch/issues/83101
* Apply suggestions from code review
* Update CMakeLists.txt
* docs: update upgrade guide
* Update docs/upgrade.rst
* All bells & whistles.
* Add Reminder to common.h, so that we will not forget to purge `!WITH_THREAD` branches when dropping Python 3.6
* New sentence instead of semicolon.
* Temporarily pull in snapshot of PR #4246
* Add `test_release_acquire`
* Add more unit tests for nested gil locking
* Add test_report_builtins_internals_keys
* Very minor enhancement: sort list only after filtering.
* Revert change in docs/upgrade.rst
* Add test_multi_acquire_release_cross_module, while also forcing unique PYBIND11_INTERNALS_VERSION for cross_module_gil_utils.cpp
* Hopefully fix apparently new ICC error.
```
2022-10-28T07:57:54.5187728Z -- The CXX compiler identification is Intel 2021.7.0.20220726
...
2022-10-28T07:58:53.6758994Z icpc: remark #10441: The Intel(R) C++ Compiler Classic (ICC) is deprecated and will be removed from product release in the second half of 2023. The Intel(R) oneAPI DPC++/C++ Compiler (ICX) is the recommended compiler moving forward. Please transition to use this compiler. Use '-diag-disable=10441' to disable this message.
2022-10-28T07:58:54.5801597Z In file included from /home/runner/work/pybind11/pybind11/include/pybind11/detail/../detail/type_caster_base.h(15),
2022-10-28T07:58:54.5803794Z from /home/runner/work/pybind11/pybind11/include/pybind11/detail/../cast.h(15),
2022-10-28T07:58:54.5805740Z from /home/runner/work/pybind11/pybind11/include/pybind11/detail/../attr.h(14),
2022-10-28T07:58:54.5809556Z from /home/runner/work/pybind11/pybind11/include/pybind11/detail/class.h(12),
2022-10-28T07:58:54.5812154Z from /home/runner/work/pybind11/pybind11/include/pybind11/pybind11.h(13),
2022-10-28T07:58:54.5948523Z from /home/runner/work/pybind11/pybind11/tests/cross_module_gil_utils.cpp(13):
2022-10-28T07:58:54.5949009Z /home/runner/work/pybind11/pybind11/include/pybind11/detail/../detail/internals.h(177): error #2282: unrecognized GCC pragma
2022-10-28T07:58:54.5949374Z PYBIND11_TLS_KEY_INIT(tstate)
2022-10-28T07:58:54.5949579Z ^
2022-10-28T07:58:54.5949695Z
```
* clang-tidy fixes
* Workaround for PYPY WIN exitcode None
* Revert "Temporarily pull in snapshot of PR #4246"
This reverts commit 23ac16e859150f27fda25ca865cabcb4444e0770.
* Another workaround for PYPY WIN exitcode None
* Clean up how the tests are run "run in process" Part 1: uniformity
* Clean up how the tests are run "run in process" Part 2: use `@pytest.mark.parametrize` and clean up the naming.
* Skip some tests `#if defined(THREAD_SANITIZER)` (tested with TSAN using the Google-internal toolchain).
* Run all tests again but ignore ThreadSanitizer exitcode 66 (this is less likely to mask unrelated ThreadSanitizer issues in the future).
* bug fix: missing common.h include before using `PYBIND11_SIMPLE_GIL_MANAGEMENT`
For the tests in the github CI this does not matter, because
`PYBIND11_SIMPLE_GIL_MANAGEMENT` is always defined from the command line,
but when monkey-patching common.h locally, it matters.
* if process.exitcode is None: assert t_delta > 9.9
* More sophisiticated `_run_in_process()` implementation, clearly reporting `DEADLOCK`, additionally exercised via added `intentional_deadlock()`
* Wrap m.intentional_deadlock in a Python function, for `ForkingPickler` compatibility.
```
> ForkingPickler(file, protocol).dump(obj)
E TypeError: cannot pickle 'PyCapsule' object
```
Observed with all Windows builds including mingw but not PyPy, and macos-latest with Python 3.9, 3.10, 3.11 but not 3.6.
* Add link to potential solution for WOULD-BE-NICE-TO-HAVE feature.
* Add `SKIP_IF_DEADLOCK = True` option, to not pollute the CI results with expected `DEADLOCK` failures while we figure out what to do about them.
* Add COPY-PASTE-THIS: gdb ... command (to be used for debugging the detected deadlock)
* style: pre-commit fixes
* Do better than automatic pre-commit fixes.
* Add `PYBIND11_SIMPLE_GIL_MANAGEMENT` to `pytest_report_header()` (so that we can easily know when harvesting deadlock information from the CI logs).
Co-authored-by: Arnim Balzer <arnim@seechange.ai>
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* add installation support for pkg-config dependency detection
pkg-config is a buildsystem-agnostic alternative to
`pybind11Config.cmake` that can be used from build systems other than
cmake.
Fixes#230
* tests: add test for pkg config
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
* ci: support Python 3.11-dev
Also update 3.10 to final, better PyPy usage
* fix: use PyFrame_GetCode on Python 3.9+
* ci: some bitiness of pypy not supported on win
* chore: update CMake support to 3.22rc1 to quiet warning
* fix: use dev version of py to fix Py 3.11
* tests: print proper Eigen version
* ci: include pypy2, not sure why
* ci: avoid running on Python 3.11 for now
* ci: fix runs
* ci: simpler PyPy usage, drop unmaintained scipy + pypy index
* ci: only binary numpy, wait on pypy 3.8
* refactor: address review
* Eliminate duplicate TLS keys for loader_life_support stack
This revises the existing fix for
https://github.com/pybind/pybind11/issues/2765 in
https://github.com/pybind/pybind11/pull/3237 to reduce the amount of
TLS storage used.
The shared TLS key is stored in two different ways, depending on
`PYBIND11_INTERNALS_VERSION`. If `PYBIND11_INTERNALS_VERSION ==
4` (as is currently set), the TLS key is stored in the
`internal::shared_data` map to avoid breaking ABI compatibility. If
`PYBIND11_INTERNALS_VERSION > 4`, the TLS key is stored directly in
the `internals` struct.
* Fix test_pytypes.py::test_issue2361 failure on PyPy3.7
* Add github actions tests for unstable ABI
* Splitting out pybind11/stl/filesystem.h.
To solve breakages like: https://github.com/deepmind/open_spiel/runs/2999582108
Mostly following the suggestion here: https://github.com/pybind/pybind11/pull/2730#issuecomment-750507575
Except using pybind11/stl/filesystem.h instead of pybind11/stlfs.h, as decided via chat.
stl.h restored to the exact state before merging PR #2730 via:
```
git checkout 733f8de24f stl.h
```
* Properly including new stl subdirectory in pip wheel config.
This now passes interactively:
```
pytest tests/extra_python_package/
```
* iwyu cleanup.
iwyuh.py -c -std=c++17 -DPYBIND11_TEST_BOOST -Ipybind11/include -I/usr/include/python3.9 -I/usr/include/eigen3 include/pybind11/stl/filesystem.h
* Adding PYBIND11_HAS_FILESYSTEM_IS_OPTIONAL.
* Eliminating else after return.
* feat: setup.py redesign and helpers
* refactor: simpler design with two outputs
* refactor: helper file update and Windows support
* fix: review points from @YannickJadoul
* refactor: fixes to naming and more docs
* feat: more customization points
* feat: add entry point pybind11-config
* refactor: Try Extension-focused method
* refactor: rename alt/inplace to global
* fix: allow usage with git modules, better docs
* feat: global as an extra (@YannickJadoul's suggestion)
* feat: single version location
* fix: remove the requirement that setuptools must be imported first
* fix: some review points from @wjacob
* fix: use .in, add procedure to docs
* refactor: avoid monkeypatch copy
* docs: minor typos corrected
* fix: minor points from @YannickJadoul
* fix: typo on Windows C++ mode
* fix: MSVC 15 update 3+ have c++14 flag
See <https://docs.microsoft.com/en-us/cpp/build/reference/std-specify-language-standard-version?view=vs-2019>
* docs: discuss making SDists by hand
* ci: use pep517.build instead of manual setup.py
* refactor: more comments from @YannickJadoul
* docs: updates from @ktbarrett
* fix: change to newly recommended tool instead of pep517.build
This was intended as a proof of concept; build seems to be the correct replacement.
See https://github.com/pypa/pep517/pull/83
* docs: updates from @wjakob
* refactor: dual version locations
* docs: typo spotted by @wjakob
* tests: keep source dir clean
* ci: make first build inplace
* ci: drop dev setting (wasn't doing anything)
* tests: warn if source directory is dirty
* add uninstall target for cmake
* only add target when built as master project
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
It is useful not only to remember the python libs and includes but
also the interpreter version in cache.
If users call pybind11 throught `add_subdirectories` they will
otherwise have no access to the selected interpreter version.
The interpreter version is useful for downstream projects, e.g.
to select default `lib/pythonX.Y/site-packages/` install paths.
This allows you to use:
cls.def(py::init(&factory_function));
where `factory_function` returns a pointer, holder, or value of the
class type (or a derived type). Various compile-time checks
(static_asserts) are performed to ensure the function is valid, and
various run-time type checks where necessary.
Some other details of this feature:
- The `py::init` name doesn't conflict with the templated no-argument
`py::init<...>()`, but keeps the naming consistent: the existing
templated, no-argument one wraps constructors, the no-template,
function-argument one wraps factory functions.
- If returning a CppClass (whether by value or pointer) when an CppAlias
is required (i.e. python-side inheritance and a declared alias), a
dynamic_cast to the alias is attempted (for the pointer version); if
it fails, or if returned by value, an Alias(Class &&) constructor
is invoked. If this constructor doesn't exist, a runtime error occurs.
- for holder returns when an alias is required, we try a dynamic_cast of
the wrapped pointer to the alias to see if it is already an alias
instance; if it isn't, we raise an error.
- `py::init(class_factory, alias_factory)` is also available that takes
two factories: the first is called when an alias is not needed, the
second when it is.
- Reimplement factory instance clearing. The previous implementation
failed under python-side multiple inheritance: *each* inherited
type's factory init would clear the instance instead of only setting
its own type value. The new implementation here clears just the
relevant value pointer.
- dealloc is updated to explicitly set the leftover value pointer to
nullptr and the `holder_constructed` flag to false so that it can be
used to clear preallocated value without needing to rebuild the
instance internals data.
- Added various tests to test out new allocation/deallocation code.
- With preallocation now done lazily, init factory holders can
completely avoid the extra overhead of needing an extra
allocation/deallocation.
- Updated documentation to make factory constructors the default
advanced constructor style.
- If an `__init__` is called a second time, we have two choices: we can
throw away the first instance, replacing it with the second; or we can
ignore the second call. The latter is slightly easier, so do that.
Embedding may well be used in places where hidden visibility isn't
desired. It should be relatively safe to allow it there; any potential
conflict would come in if modules are loaded into that embedded
interpreter, but as long as the modules are compiled with hidden
visibility they shouldn't conflict.
There could still be warnings if the embedded code attempts to export
classes with internal (hidden) pybind members, but that seems a
legitimate warning (and already has a FAQ entry).
This updates the compilation to always apply hidden visibility to
resolve the issues with default visibility causing problems under debug
compilations. Moreover using the cmake property makes it easier for a
caller to override if absolutely needed for some reason.
For `pybind11_add_module` we use cmake to set the property; for the
targets, we append to compilation option to non-MSVC compilers.
When Pybind11 is used via `add_subdirectory`, when targets are installed
from the parent project, CMake wants all of the dependencies built by
the project in the same export set. Projects may now set
`PYBIND11_EXPORT_NAME` to have Pybind11 put it targets into the
project's export set. If so, do not install Pybind11's export file.
All targets provided by pybind11:
* pybind11::module - the existing target for creating extension modules
* pybind11::embed - new target for embedding the interpreter
* pybind11::pybind11 - common "base" target (headers only)
Upcoming changes to buffer_info make it need some things declared in
common.h; it also feels a bit misplaced in common.h (which is arguably
too large already), so move it out. (Separating this and the subsequent
changes into separate commits to make the changes easier to distinguish
from the move.)
* Arch-indep CMake packaging
Since pybind11 is a header-only library, the CMake packaging does not have to carry any architecture specific checks. Without this patch, the detection of pybind11 will fail on 32-bit architectures if the project was built on a 64-bit machine and vice-versa. This fix is similar to what is applied to `Eigen` and other header-only C++ libraries.
* Make tests buildable independently
This makes "tests" buildable as a separate project that uses
find_package(pybind11 CONFIG) when invoked independently.
This also moves the WERROR option into tests/CMakeLists.txt, as that's
the only place it is used.
* Use Eigen 3.3.1's cmake target, if available
This changes the eigen finding code to attempt to use Eigen's
system-installed Eigen3Config first. In Eigen 3.3.1, it exports a cmake
Eigen3::Eigen target to get dependencies from (rather than setting the
include path directly).
If it fails, we fall back to the trying to load allowing modules (i.e.
allowing our tools/FindEigen3.cmake). If we either fallback, or the
eigen version is older than 3.3.1 (or , we still set the include
directory manually; otherwise, for CONFIG + new Eigen, we get it via
the target.
This is also needed to allow 'tests' to be built independently, when
the find_package(Eigen3) is going to find via the system-installed
Eigen3Config.cmake.
* Add a install-then-build test, using clang on linux
This tests that `make install` to the actual system, followed by a build
of the tests (without the main pybind11 repository available) works as
expected.
To also expand the testing variety a bit, it also builds using
clang-3.9 instead of gcc.
* Don't try loading Eigen3Config in cmake < 3.0
It could FATAL_ERROR as the newer cmake includes a cmake 3.0 required
line.
If doing an independent, out-of-tree "tests" build, the regular
find_package(Eigen3) is likely to fail with the same error, but I think
we can just let that be: if you want a recent Eigen with proper cmake
loading support *and* want to do an independent tests build, you'll
need at least cmake 3.0.
Instead of creating a new unique metaclass for each type, the builtin
`property` type is subclassed to support static properties. The new
setter/getters always pass types instead of instances in their `self`
argument. A metaclass is still required to support this behavior, but
it doesn't store any data anymore, so a new one doesn't need to be
created for each class. There is now only one common metaclass which
is shared by all pybind11 types.
* Avoid C-style const casts
Replace C-style casts that discard `const` with `const_cast` (and, where
necessary, `reinterpret_cast` as well).
* Warn about C-style const-discarding casts
Change pybind11_enable_warnings to also enable `-Wcast-qual` (warn if a
C-style cast discards `const`) by default. The previous commit should
have gotten rid of all of these (at least, all the ones that tripped in
my build, which included the tests), and this should discourage more
from newly appearing.
Fixes#567.
If pybind's CMakeLists gets loaded via an include_directory from another
CMakeLists with a higher minimum version (e.g. 3.0), the project()
command without a version produces a CMP0048 warning.
This commit explicitly requests the new behaviour if the policy exists,
as it won't cause problems (we set VERSION later).
