Commit Graph

1453 Commits

Author SHA1 Message Date
Aaron Gokaslan
6cbabc4b8c
maint(clang-tidy): Enable cpp-coreguideline slicing checks (#3210)
* maint(clang-tidy): add a clang-tidy slicing check

* Add self + touch up readme

* Fix typo
2021-08-23 18:42:19 -04:00
Ralf W. Grosse-Kunstleve
b176057abc Merge branch 'master' into smart_holder 2021-08-19 11:40:24 -07:00
Ralf W. Grosse-Kunstleve
998d45e431
Cleanup of file-scoped and globally-scoped warning suppression pragmas across pybind11 header files. (#3201)
* Removing all MSVC C4127 warning suppression pragmas.

* Removing MSVC /WX (WERROR). To get a full list of all warnings.

* Inserting PYBIND11_SILENCE_MSVC_C4127. Changing one runtime if to #if.

* Changing PYBIND11_SILENCE_MSVC_C4127 macro to use absolute namespace (for use outside pybind11 include directory).

* Restoring MSVC /WX (WERROR).

* Removing globally-scoped suppression for clang -Wunsequenced. Based on an experiment under PR #3202 it is obsolete and can simply be removed.
2021-08-19 11:37:04 -07:00
Ralf W. Grosse-Kunstleve
774b5ff90b
Removing obsolete eigen.h warning suppression pragmas. (#3198)
* Removing all pragma from eigen.h

* Removing -Werror or equivalent from tests/CMakeLists.txt

* Restoring tests/CMakeLists.txt from master.

* Adding 4 PYBIND11_SILENCE_MSVC_C4127.

* Compatibility with -Wconversion, -Wdeprecated

* Introducing PYBIND11_COMPATIBILITY_WDEPRECATED_COPY

* Systematically using --verbose for compilations where possible (cmake 3.14 or newer).

Also changing all `cmake -t` to `--target`, `-v` to `--verbose`, `check` to `pytest`, for consistency (to make it easier to pin-point all commands of a certain type).

Also removing one `-j 2` for `pytest` in hopes of reducing flakes due to races in test_iostream and in prints from destructors.

* Commenting out pragmas as an experiment to reproduce previous observation.

* Removing all (newly added, but already commented-out) pragma code, adding HINT use -isystem (as cmake does).

* Restoring ci.yml from master. Those changes are better handled separately. BTW: in the last CI run there was still a test_iostream flake, even without the -j 2 for running the tests (verfied by inspecting the log).
2021-08-17 16:49:39 -07:00
Ralf W. Grosse-Kunstleve
a0a30c4532 Merge branch 'master' into sh_merge_master 2021-08-14 12:46:18 -07:00
Ralf W. Grosse-Kunstleve
e41fb99e7e clang-tidy fixes (mostly manual) related to PR #3166 2021-08-14 12:45:50 -07:00
Aaron Gokaslan
617cb653ec
[Bugfix] Fix errant const methods (#3194)
* Fix errant const methods

* Remove NOLINT since clang-tidy is pretty conservative

* Missed one

* Fix a few more errors

* Add reviewer suggested comments

* Run clang-format
2021-08-14 12:25:54 -04:00
Ralf W. Grosse-Kunstleve
e3e1d29fdb Removing obsolete detail/pragma_warning_block.h 2021-08-14 08:42:38 -07:00
Ralf W. Grosse-Kunstleve
8865d28317 Automatic part of merging branch 'master' into sh_merge_master.
The only manual intervention was to remove the merge conflict markers in pybind11.h.

TODO in separate commit: complete removal of the obsolete detail/pragma_warning_block.h.
2021-08-14 07:54:40 -07:00
Ralf W. Grosse-Kunstleve
1bcd94c481
Removing last remnants of pragma block at the top of pybind11.h (#3186)
* Removing last remnants of pragma block at the top of pybind11.h, defaulting CUDA, GCC7, GCC8 to PYBIND11_NOINLINE_DISABLED, with the option to define PYBIND11_NOINLINE_FORCED.

* Unique SOSIZE prefix to make it easier to extract the sosizes from the GitHub logs.

* Commenting out PYBIND11_WERROR block, for noinline testing.

* Undoing accidental change.

* `#define PYBIND11_NOINLINE_FORCED`

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* `#define PYBIND11_NOINLINE_DISABLED`

* Going back to default (removing `#define PYBIND11_NOINLINE_DISABLED`).

* `#define PYBIND11_NOINLINE_FORCED`

* Undoing all changes releated to measuring sosizes.

* Rollback of PR #3030 (Working around Centos 8 failure).

* Disabling -Werror for GNU (experiment).

* Commenting out the entire `if(PYBIND11_WERROR)` again (although that is not expected to make a difference, but who knows what I am overlooking).

* Adding `-DCMAKE_BUILD_TYPE=Release`

* Undoing change to tests/CMakeLists.txt (uncommenting `if(PYBIND11_WERROR)` block).

* post `git rebase master -X theirs` fixups.

* Adding measurements to comment for `PYBIND11_NOINLINE_FORCED`.

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-08-14 07:41:27 -07:00
Ralf W. Grosse-Kunstleve
4c7e509fa4
PYBIND11_NOINLINE-related cleanup. (#3179)
* Removing pragma for GCC -Wattributes, fixing forward declarations.

* Introducing PYBIND11_NOINLINE_FWD to deal with CUDA, GCC7, GCC8.

* Updating PYBIND11_NOINLINE_DCL in Doxyfile.

* Trying noinline, noinline for {CUDA, GCC7, GCC8}

* Trying noinline, inline for {CUDA, GCC7, GCC8}

* Adding GCC -Wattributes `pragma` in 3 header files.

* Introducing PYBIND11_NOINLINE_GCC_PRAGMA_ATTRIBUTES_NEEDED, used in 9 header files.

* Removing ICC pragma 2196, to see if it is still needed.

* Trying noinline, noinline for ICC

* Trying noinline, inline for ICC

* Restoring ICC pragma 2196, introducing PYBIND11_NOINLINE_FORCED, defined for testing.

* Removing code accidentally left in (was for experimentation only).

* Removing one-time-test define.

* Removing PYBIND11_NOINLINE_FWD macro (after learning that it makes no sense).

* Testing with PYBIND11_NOINLINE_DISABLED. Minor non-functional enhancements.

* Removing #define PYBIND11_NOINLINE_DISABLED (test was successful).

* Removing PYBIND11_NOINLINE_FORCED and enhancing comments for PYBIND11_NOINLINE.

* WIP stripping back

* Making -Wattributes pragma in pybind11 specific to GCC7, GCC8, CUDA.
2021-08-09 10:10:38 -07:00
Aaron Gokaslan
ff590c1258
maint(perf): Optimize Numpy constructor to remove copies by value. (#3183)
* maint(perf): Optimize Numpy Constructor with additional std::move

* Add more moves
2021-08-09 12:48:27 -04:00
Ralf W. Grosse-Kunstleve
af7007331a
Removing GCC -Wunused-but-set-parameter from pragma block at the top of pybind11.h (#3164)
* Cleanup triggered by work on pragma for GCC -Wunused-but-set-parameter.

* Backing out changes to eigen.h (to be worked on later).

* Adding PYBIND11_WORKAROUND_INCORRECT_GCC_UNUSED_BUT_SET_PARAMETER in type_caster_base.h (apparently needed only for older GCCs).

* Apparently older compilers need a simpler overload for silence_unused_warnings().

* clang C++11 compatibility: removing constexpr

* Special case for MSVC 2017: `constexpr void` return

* Trying again without the silence_unused_warnings(const int *) overload.

* Separate macros for ALL_GCC, OLD_GCC_UNUSED_BUT_SET_PARAMETER

* Changing to __GNUC__ <= 2 (turning off)

* Refined condition for PYBIND11_WORKAROUND_INCORRECT_OLD_GCC_UNUSED_BUT_SET_PARAMETER.

* Quick experiment trying out suggestion by @henryiii

* Introducing macro: PYBIND11_INT_ARRAY_WORKING_AROUND_MSVC_CLANG_ISSUES

* Trying henryiii@ (void) expander idea.

* fix: apply simpler expression with fewer workarounds

* Purging new-but-already-obsoleted  macro, made possible by @henryiii's commit.

* Renaming `ALL_GCC` macro back to just `GCC` (because there is no `OLD` anymore, luckily).

* [actions skip] Adding "All GCC versions" to comment, to be clear about it.

Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
2021-08-06 12:27:11 -07:00
Aaron Gokaslan
3893f37bce
maint(clang-tidy): Bugprone enable checks (#3166)
* Enable bugprone checks

* Reset delta and massage config

* Start to apply bugprone fixes

* try to fix minor bug

* Fix later

* Fix perfect forwarding bugprone

* Remove nolint

* undo constructor delete

* Fix bugprone-perfect-forwarding again

* Remove TODO

* Add another nolint for bugprone-exception-escape in scoped interpreter

* Fix remaining bugprone errors

* Properly apply bugprone-macro-parantheses

* Redo formatting and remove bugprone nolint

* Add coment and revert more whitespace changes

* Fix typo

* Fix parsing bug

* Add back comma

* Fix clang-tidy issue

* Apply remaining clang-tidy fixes
2021-08-06 14:30:28 -04:00
Henry Schreiner
089328f779 Revert "fix: apply simpler expression with fewer workarounds"
This reverts commit 1fafd1b447.
2021-08-06 13:09:48 -04:00
Henry Schreiner
1fafd1b447
fix: apply simpler expression with fewer workarounds 2021-08-06 13:03:26 -04:00
Henry Schreiner
5f4d725918
fix: version number hex 2021-08-03 17:03:11 -04:00
Henry Schreiner
c30f57d2ed
chore: start development for 2.8.0 2021-08-03 16:11:07 -04:00
Henry Schreiner
82adacb31d
fix: include hex version in bump 2021-08-03 15:20:23 -04:00
Henry Schreiner
5f34c42d70 chore: bump to version 2.7.1 2021-08-03 15:06:57 -04:00
Aaron Gokaslan
c0756ccd93
fix: func_handle for rule of two (#3169)
* Fix func_handle for rule of two

* Apply reviewer suggestion
2021-08-03 13:15:48 -04:00
Ralf W. Grosse-Kunstleve
aff3cf7b65 Merge branch 'master' into sh_merge_master 2021-07-30 12:25:43 -07:00
Ralf W. Grosse-Kunstleve
dcbda8d7ff
Removing MSVC C4127 from pragma block at the top of pybind11.h (#3152)
* Removing pragma for 4127 (to see what is still broken with the latest code).

* Using new constexpr_bool() to suppress warning C4127.

* One missed case, Python 2 only.

* PYBIND11_SILENCE_MSVC_C4127 (more similar to the approach for C4100).
2021-07-30 11:25:29 -07:00
Ralf W. Grosse-Kunstleve
e2573dc961
Moving pragma for MSVC warning C4505 from pybind11.h to existing list in detail/common.h (#3160)
* Moving pragma for C4505 from pybind11.h to existing list in detail/common.h.

* Removing 4 existing suppressions to 1. see what is still needed and 2. capture the MSVC messages.

* It turns out none of the 4 pragmas are needed anymore.
2021-07-30 10:51:50 -07:00
Jerome Robert
9e8a741baa
fix: Mingw64 corrected and add a CI job to test it (#3132)
* mingw64 platform string is like mingw_xxx not "mingw"

See https://github.com/msys2/MINGW-packages/blob/master/mingw-w64-python/0099-Change-the-get_platform-method-in-sysconfig-and-dist.patch

* Mingw: Do not dllexport exceptions

This is a fix for errors like:

D:/a/pybind11/pybind11/include/pybind11/detail/common.h:735:23: error: 'dllexport' implies default visibility, but 'class pybind11::builtin_exception' has already been declared with a different visibility
  735 | class PYBIND11_EXPORT builtin_exception : public std::runtime_error {
      |                       ^~~~~~~~~~~~~~~~~

* GHA: Test Mingw64 build

* fix: avoid thin binaries on mingw

* fix: drop lto on MinGW

* Mingw64: disable PYBIND11_DEPRECATED

It trigger many warnings for unknown reasons

Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
2021-07-30 13:48:41 -04:00
Ralf W. Grosse-Kunstleve
b42597291f
Limiting pragma for ignoring GCC 7 -Wnoexcept-type to the scope of pybind11.h. (#3161)
* Moving pragma for ignoring -Wnoexcept-type to the one location where it is needed.

* Trying a second location.

* The previous commit worked (GitHub Actions green), but see the added comment about the dicy nature of -Wnoexcept-type ("if and only if").

* Applying reviewer suggestion.
2021-07-30 07:09:55 -07:00
Ralf W. Grosse-Kunstleve
624cd3e0af
Adding PYBIND11_WORKAROUND_INCORRECT_MSVC_C4100 (follow-on to PR #3150). (#3154) 2021-07-28 22:49:00 -07:00
Ralf W. Grosse-Kunstleve
3754d6436b Merge branch 'master' into smart_holder 2021-07-28 17:03:25 -07:00
Ralf W. Grosse-Kunstleve
b72ca7d1bd
Removing MSVC C4100 from pragma block at the top of pybind11.h (#3150)
* Removing pragma for 4100 (to see what is still broken with the latest code).

* Adding --keep-going

* Revert "Adding --keep-going"

This reverts commit 1c844c6ffd07a6111b644811e7e3b0a50b9d44bb.

* Introducing PYBIND11_WORKAROUND_INCORRECT_MSVC_C4100.

* _MSC_VER <= 1916

* Replacing simple variadic function with variadic template (attempt to resolve MSVC 2017 failures).

* Preserving existing comment (moved from pybind11.h to detail/common.h).

* Adding blank lines for readability.
2021-07-28 17:01:21 -07:00
Ralf W. Grosse-Kunstleve
dcdadd73f5 Merge branch 'master' into smart_holder
[actions skip]
2021-07-28 10:24:02 -07:00
Ralf W. Grosse-Kunstleve
f4721a7b44
Accommodating environments that define __STDC_WANT_LIB_EXT1__ even if __STDC_LIB_EXT1__ is not defined by the implementation. (#3151)
Follow-on to PR #3129.
2021-07-28 08:58:36 -07:00
Ralf W. Grosse-Kunstleve
45ef484366 Merge branch 'master' into smart_holder 2021-07-27 15:38:09 -07:00
Ralf W. Grosse-Kunstleve
2164c2e0e7
Removing __INTEL_COMPILER section from pragma block at the top of pybind11.h (#3135)
* Fixing `pragma warning pop` for `__INTEL_COMPILER`.

* Adding push/pop to 3 tests. Removing #878 from top of pybind11.h (it was/is only needed for 1 test).

* Trying again after CI failure, moving the push to the top of 2 tests.

* Trying more after CI failure, adding push/pop to pybind11_tests.h, constructor_stats.h.

* Moving ICC #2196 suppression to CMakeLists.txt

* Fixing condition for `pragma GCC diagnostic push` in pybind11.h

* Moving `pragma warning disable 2196` to common.h

* Revising #ifdef to be more conservative.

* Undoing insertion of notes that will hopefully soon be completely obsolete anyway.
2021-07-27 15:33:31 -07:00
Aaron Gokaslan
9beaa925db
maint(clang-tidy): Improve code readability with explicit boolean casts (#3148)
* maint(clang-tidy) Improve code readability

* Fix minor typos

* Revert optimization that removed test case

* Fix comment formatting

* Revert another optimization to repro an issue

* Remove make_unique since it C++14 and newer only

* eformat comments

* Fix unsignedness of comparison

* Update comment
2021-07-27 15:32:26 -07:00
Ralf W. Grosse-Kunstleve
1cd7179f1f Merge branch 'master' into smart_holder 2021-07-27 11:30:23 -07:00
David Hewitt
a0b975965f
Allow python builtins to be used as callbacks (#1413)
* Allow python builtins to be used as callbacks

* Try to fix pypy segfault

* Add expected fail for PyPy

* Fix typo

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add more info to xfail

* Add env

* Try returning false

* Try removing the move for pypy

* Fix bugs

* Try removing move

* Just keep ignoring for PyPy

* Add back xfail

* Fix ctors

* Revert change of std::move

* Change to skip

* Fix bug and edit comments

* Remove clang-tidy bugprone fix skip bug

Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-07-27 14:16:28 -04:00
Ralf W. Grosse-Kunstleve
8ed9e93d57 Merge branch 'master' into smart_holder 2021-07-26 13:43:29 -07:00
Ralf W. Grosse-Kunstleve
a0f862d428
Removing MSVC C4800 from pragma block at the top of pybind11.h (#3141)
* Adding PYBIND11_COMPAT_BOOL_CAST to appease MSVC 2015 warning C4800.

* Replacing PYBIND11_COMPAT_BOOL_CAST with simpler != 0

* Extra parentheses (almost all compilers failed without these).
2021-07-26 13:26:36 -07:00
Ralf W. Grosse-Kunstleve
7904ba1a5c
Adding pragma warning(disable: 4522) for MSVC <= 2017. (#3142) 2021-07-26 12:02:50 -07:00
Ralf W. Grosse-Kunstleve
ff97f101d9
Removing MSVC C4996 from pragma block at the top of pybind11.h (#3129)
* Removing MSVC C4996 from pragma block at the top of pybind11.h

* localtime_thread_safe, PYBIND11_COMPAT_STRDUP

* Adding #include <ctime> (attempt to fix MSVC 2015, 2017 errors).
2021-07-26 11:28:36 -07:00
Ralf W. Grosse-Kunstleve
7cc433ca7f Merge branch 'master' into smart_holder 2021-07-21 06:56:03 -07:00
jesse-sony
d65edfb024
Feature/local exception translator (#2650)
* Create a module_internals struct

Since we now have two things that are going to be module local, it felt
correct to add a struct to manage them.

* Add local exception translators

These are added via the  register_local_exception_translator function
and are then applied before the global translators

* Add unit tests to show the local exception translator works

* Fix a bug in the unit test with the string value of KeyError

* Fix a formatting issue

* Rename registered_local_types_cpp()

Rename it to get_registered_local_types_cpp() to disambiguate from the
new member of module_internals

* Add additional comments to new local exception code path

* Add a register_local_exception function

* Add additional unit tests for register_local_exception

* Use get_local_internals like get_internals

* Update documentation for new local exception feature

* Add back a missing space

* Clean-up some issues in the docs

* Remove the code duplication when translating exceptions

Separated out the exception processing into a standalone function in the
details namespace.

Clean-up some comments as per PR notes as well

* Remove the code duplication in register_exception

* Cleanup some formatting things caught by clang-format

* Remove the templates from exception translators

But I added a using declaration to alias the type.

* Remove the extra local from local_internals variable names

* Add an extra explanatory comment to local_internals

* Fix a typo in the code
2021-07-21 05:22:18 -07:00
blacktea
6d5d4e738c
Move object in pop method of List. (#3116)
* Move item instead of copy.

* Make Clang 3.6 happy.

Co-authored-by: c99 <email@dummy.com>
2021-07-20 11:48:09 -04:00
Ralf W. Grosse-Kunstleve
a1918dd6d9 Merge branch 'master' into smart_holder 2021-07-17 10:32:03 -07:00
Ralf W. Grosse-Kunstleve
34f587dd23
Removing all warning pragmas that have not effect. (#3127) 2021-07-17 08:54:31 -07:00
Henry Schreiner
74935f8d67
chore: post-release (#3128) 2021-07-17 11:50:42 -04:00
Ralf W. Grosse-Kunstleve
02dc01b536 Merge branch 'master' into smart_holder 2021-07-16 08:14:43 -07:00
Henry Schreiner
65e95ea867
chore: bump to 2.7.0 (#3123) 2021-07-16 09:27:47 -04:00
Ralf W. Grosse-Kunstleve
1f04229733 Merge branch 'master' into smart_holder 2021-07-15 15:05:40 -07:00
Ralf W. Grosse-Kunstleve
4359e00b97
Introducing PYBIND11_VERSION_HEX (#3120)
* Introducing PYBIND11_VERSION_HEX (better late than never!)

* PYBIND11_VERSION_HEX consistency check in setup.py
2021-07-15 15:00:57 -07:00
Ralf W. Grosse-Kunstleve
6d1026763a Merge branch 'master' into smart_holder 2021-07-15 10:13:07 -07:00
Boris Staletic
5cd3750757
Enable -Wstrict-aliasing warning (#2816)
* Enable -Wstrict-aliasing warning

* Narrow down the scope of -Wstrict-aliasing

* Go home, MSVC, you're drunk

* Make sure "pragma GCC" is not executed on ICC

Co-authored-by: Yannick Jadoul <yannick.jadoul@belgacom.net>
2021-07-15 12:38:52 -04:00
Ralf W. Grosse-Kunstleve
279c93654e Merge branch 'master' into smart_holder 2021-07-13 21:24:02 -07:00
Shane Loretz
7331d381af
Raise codec errors when casting to std::string (#2903)
* Raise codec errors when casting to std::string

Allow the codec's exception to be raised instead of RuntimeError when
casting from py::str to std::string.

PY2 allows ucs surrogates in UTF-8 conversion

Signed-off-by: Shane Loretz <sloretz@openrobotics.org>
Signed-off-by: Shane Loretz <sloretz@osrfoundation.org>

* Attempt to fix py2 error

* Revert all unicode literals

* Fixed

Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
2021-07-13 21:21:55 -07:00
Ralf W. Grosse-Kunstleve
aca6c3ba37
* Removing stray semicolons (discovered by running clang-format v12 followed by tools/check-style.sh). (#3087)
* Manually moving `// NOLINT` comments so that clang-format does not move them to the wrong places.

* Manually reformatting comments related to `static_assert`s so that clang-format does not need two passes.

* Empty lines between #includes, to prevent clang-format from shuffling the order and thereby confusing MSVC 2015.

* git diff -U0 --no-color HEAD^ | python3 $HOME/clone/llvm-project/clang/tools/clang-format/clang-format-diff.py -p1 -style=file -i
2021-07-13 18:14:58 -07:00
Ralf W. Grosse-Kunstleve
42c123f59d Merge branch 'master' into smart_holder 2021-07-13 13:27:31 -07:00
jbarlow83
2b7985e548
Improve documentation of discard_as_unraisable() API (#2697)
* Improve documentation of discard_as_unraisable() API

* Update pytypes.h

Remove "the above"

* Update pytypes.h

Fix precommit error

Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
2021-07-13 15:32:56 -04:00
Ralf W. Grosse-Kunstleve
6133fc7b0c Merge branch 'master' into smart_holder 2021-07-13 10:29:36 -07:00
Aaron Gokaslan
25e470c57d
fix(clang-tidy): Add cppcoreguidelines-init-vars,slicing, and throw-by-value-catch-by-reference checks (#3094)
* clang-tidy: guard against more UB behavior

* Remove slicing check for now
2021-07-13 09:54:32 -04:00
Ralf W. Grosse-Kunstleve
932a4cf409 Merge branch 'master' into smart_holder 2021-07-12 14:07:26 -07:00
Henry Schreiner
f0a65c899c
docs(fix): spelling mistake in recent commit 2021-07-12 16:57:28 -04:00
Ralf W. Grosse-Kunstleve
7472d37a93
Adding iostream.h thread-safety documentation. (#2995)
* Adding iostream.h thread-safety documentation.

* Restoring `TestThread` code with added `std::lock_guard<std::mutex>`.

* Updating new comments to reflect new information.

* Fixing up `git rebase -X theirs` accidents.
2021-07-12 13:39:06 -07:00
Ralf W. Grosse-Kunstleve
4a3444ad2f clang-tidy fixes related to the just-merged PR #3080. Also fixing a minor clang-format mishap. 2021-07-09 07:11:07 -07:00
Ralf W. Grosse-Kunstleve
e7d146bdbd Merge branch 'master' into smart_holder 2021-07-09 06:47:46 -07:00
Aaron Gokaslan
b5357d1fa8
fix(clang-tidy): Enable clang-tidy else-after-return and redundant void checks (#3080)
* Enable clang-tidy else-after-return and redundant void checks

* Fix remaining else-after

* Address reviewer comments

* Fix indentation

* Rerun clang-tidy post merge
2021-07-09 06:45:53 -07:00
Ralf W. Grosse-Kunstleve
c03061fcff Merge branch 'master' into smart_holder 2021-07-08 09:48:42 -07:00
Ralf W. Grosse-Kunstleve
6d1b197b46
Splitting out pybind11/stl/filesystem.h. (#3077)
* Splitting out pybind11/stl/filesystem.h.

To solve breakages like: https://github.com/deepmind/open_spiel/runs/2999582108

Mostly following the suggestion here: https://github.com/pybind/pybind11/pull/2730#issuecomment-750507575

Except using pybind11/stl/filesystem.h instead of pybind11/stlfs.h, as decided via chat.

stl.h restored to the exact state before merging PR #2730 via:
```
git checkout 733f8de24f stl.h
```

* Properly including new stl subdirectory in pip wheel config.

This now passes interactively:
```
pytest tests/extra_python_package/
```

* iwyu cleanup.

iwyuh.py -c -std=c++17 -DPYBIND11_TEST_BOOST -Ipybind11/include -I/usr/include/python3.9 -I/usr/include/eigen3 include/pybind11/stl/filesystem.h

* Adding PYBIND11_HAS_FILESYSTEM_IS_OPTIONAL.

* Eliminating else after return.
2021-07-08 09:02:48 -07:00
Robert Haschke
c090c8c409
Unify cast_error message thrown by [simple|unpacking]_collector (#3013)
* Unify cast_error message thrown by [simple|unpacking]_collector

simple_collector and unpacking_collector throw different error messages
when the casting of an argument failed: While the former mentions make_tuple(),
the latter emphasises the call argument (and its name/position).

* Consolidating "Unable to convert call argument" error reporting code to guarantee uniformity.

Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
2021-07-06 15:13:13 -07:00
Ralf W. Grosse-Kunstleve
00b0365ca3 codespell fixes and adding .codespell-ignorelines 2021-07-06 14:31:18 -07:00
Ralf W. Grosse-Kunstleve
704daac879 Merge branch 'master' into smart_holder 2021-07-06 14:12:08 -07:00
luzpaz
8bee61b645
docs: fix various typos (#3075)
Found via `codespell -q 3 -L nd,ot,thist`
2021-07-04 19:58:35 -04:00
Ralf W. Grosse-Kunstleve
52c3f4cc30
This was meant to be PR #3065: pure clang-format changes. NO manual changes. (#3073) 2021-07-02 16:51:24 -07:00
Ralf W. Grosse-Kunstleve
84cbec0bc1 Merge branch 'master' into smart_holder 2021-07-02 07:07:49 -07:00
Dustin Spicuzza
6d4409466b
Check dict item accesses where it isn't already checked (#2863)
* Convert PyDict_GetXXX to internal error checking variant

* Check unlikely error return from PyDict_DelItemString
2021-07-02 07:02:33 -07:00
Antony Lee
5bcaaa0423
Add a std::filesystem::path <-> os.PathLike caster. (#2730) 2021-07-02 07:00:50 -07:00
Ralf W. Grosse-Kunstleve
9ae7f87fb6 Merge branch 'master' into smart_holder 2021-06-30 23:39:32 -07:00
cyy
f067deb563
avoid unnecessary strlen (#3058) 2021-06-30 23:35:25 -07:00
jonathan-conder-sm
733f8de24f
Avoid string copy if possible when passing a Python object to std::ostream (#3042) 2021-06-30 22:19:14 -07:00
Ralf W. Grosse-Kunstleve
c2111fd353 Merge branch 'master' into smart_holder 2021-06-30 12:36:27 -07:00
Ralf W. Grosse-Kunstleve
fbae8f313b
pickle setstate: setattr __dict__ only if not empty (#2972)
* pickle setstate: setattr __dict__ only if not empty, to not force use of py::dynamic_attr() unnecessarily.

* Adding unit test.

* Clang 3.6 & 3.7 compatibility.

* PyPy compatibility.

* Minor iwyu fix, additional comment.

* Addressing reviewer requests.

* Applying clang-tidy suggested fixes.

* Adding check_dynamic_cast_SimpleCppDerived, related to issue #3062.
2021-06-30 12:34:32 -07:00
Ralf W. Grosse-Kunstleve
93169cc907 Small reduction in code complexity. 2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
f128f1b6db Converting C assert to pybind11_fail (to play safe). 2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
a021e045a9 Adding test_multiple_registered_instances_for_same_pointee_leak. Subtle changes in smart_holder_type_caster_load, trying to work on weak_ptr for shared_ptr_trampoline_self_life_support, but that didn't work out. Manually fully leak-checked again. 2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
e2108546e1 Adding unit tests: 2 x test_std_make_shared_factory
Completes unit test coverage for the changed code in smart_holder_type_casters.h.
2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
1b752f56c1 Adding unit test: test_multiple_registered_instances_for_same_pointee 2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
cb548bb3cc This change fixes the 6 unit tests in
bbfb0259b5/open_spiel/python/mfg/algorithms
    best_response_value_test
    distribution_test
    fictitious_play_test
    greedy_policy_test
    nash_conv_test
    policy_value_test

CAVEAT: The fix is NOT covered by pybind11 unit tests.
2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
9fce2956bc A couple more enhancements based on feedback by @laramiel. 2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
64716cc14c Adding a few comments after review by @laramiel. 2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
ab3c5134ad Cleaning out debug code. 2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
146a925e4e Resolving TODOs for from_unique_ptr void_cast_raw_ptr. Adding test to exercise the path through cast. The path through init.h is missing a test that would fail if the flag is incorrect. The plan is to systematically cover all situations (there are many that are not exercised for shared_from_this). 2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
a0e19bdc46 Use casting to void * to evade to shared_from_this mechanism only if pointee_depends_on_holder_owner, but currently only for from_raw_ptr_take_ownership. 2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
1b9f85a09e Preparation for being smarter about casting to void * to evade to shared_from_this mechanism. 2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
48eb78ae77 Simpler and safe implementation based on new requirement: C++ (outside of the Python object) needs to hold a shared_ptr before shared_from_this is called. This is more similar to the existing implementation on current smart_holder HEAD, but still uses a weak_ptr to always return the same shared_ptr as long as C++ does not let it expire. 2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
3dd89f14df smart_holder_poc.h: type-erase raw_ptr before passing to shared_ptr::reset, to not trigger populating the shared_from_this weak_ptr. This way the only way the weak_ptr is populated is through loaded_as_shared_ptr. Breaks all but one test in test_class_sh_trampoline_shared_from_this.py, now marked skip WIP to test everything else with the GitHub CI. 2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
adfd3e1700 Partial cleanup of tests. WIP. The cleanup uncovered a major problem. 2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
bfd4c4893e Leveraging new noop_deleter_acting_as_weak_ptr_owner to retrieve released vptr.
The placeholder `vptr` is never exposed anymore, therefore the externally visible `use_count`s are more intuitive.
2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
4b872febcc Adding test_pass_released_shared_ptr_as_unique_ptr, exercising new guard in smart_holder_type_casters.h. 2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
223283295f Two minor platform-specific fixes. Using static_cast instead of reinterpret_cast. 2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
993613d9bf Trying again, to get around unused parameter warnings-as-erros. 2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
20ef96c87d Disabling debugging call of weak_from_this for C++ < 17. 2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
bf8d6a2900 First fully successful attempt to make shared_from_this and trampolines play nicely.
Now also passes the open_spiel iterated_prisoners_dilemma_test ASAN clean, in addition to all pybind11 and PyCLIF unit tests.

The problem was that calling `std::shared_ptr<void>::reset()` with a `void` pointer cannot possibly update the `shared_from_this` `weak_ptr`.

The solution is to store a `shd_ptr_reset` function pointer in `guarded_deleter` (similar in idea to the stored function pointer for calling `delete`).

This commit still includes all debugging code, i.e. is "dirty". The code will be cleaned up after the GitHub CI is fully successful.
2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
dcf8d0762f First attempt to make shared_from_this and trampolines play nicely. Passes all pybind11 and PyCLIF unit tests ASAN clean, but not the open_spiel iterated_prisoners_dilemma_test. 2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
bc21158082 Fixing git rebase -X theirs accident. 2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
4ab4f36a92 Fixing silly oversight (discovered while creating PR #3041). 2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
cd5fbc56db Replacing virtual guarded_operator_call with non-virtual guarded_delete. 2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
1e6cc9dd69 Adding back explicit but default copy constructor, to keep some older compilers happy. 2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
275aaf977a Inserting const_cast for std::get_deleter return, to keep Ubuntu 20 GCC 6.3.0 happy. 2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
b04a70b5cc Fixing oversight (clang-tidy error). 2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
362e64e1ca WIP snapshot: replacing std::shared_ptr<bool> flag_ptr with simple bool armed_flag. 2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
459301d9a8 WIP snapshot: std::get_deleter experiment. 2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
e80a1f0617 WIP snapshot. 2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
eae174b491 Revert "Experiment: effectively undoing all shared_from_this modifications."
This reverts commit d72d54ebb0b7784f5616edc02910dbd9cce0b2a4.
2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
a6abb7cff1 Revert "Experiment: undoing even more."
This reverts commit 59bc2e183e7aef5e45c06aed6965de0ac52d7b06.
2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
5f92b4ff1c Revert "Experiment: restoring original smart_holder_type_casters.h from smart_holder branch."
This reverts commit 19c5a3613fee71878ba2af0339eed325a5916089.
2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
2ae3c2ceb7 Experiment: restoring original smart_holder_type_casters.h from smart_holder branch. 2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
9da1e81590 Experiment: undoing even more. 2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
715ceb4bf3 Experiment: effectively undoing all shared_from_this modifications. 2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
05bd93543b Fully emulating type_caster_base-related behavior: trying shared_from_this also for unowned pointees. 2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
775873d0b6 Adding from_raw_pointer_take_ownership_or_shared_from_this(). 2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
2a265860a7 enable_shared_from_this_from_raw_ptr_take_ownership_guard: better static_assert that also triggers for derived classes. 2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
24c223ad8e static_assert in from_raw_ptr_take_ownership, to be tested. 2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
7a74bfede7 Restoring init_holder overload for std::enable_shared_from_this and original tests/test_smart_ptr.py. 2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
94523e8a02 Experiments: 1. disabling enable_shared_from_this, 2. using smart_holder in test_class_sh_shared_from_this.cpp 2021-06-30 07:04:31 -07:00
Ralf W. Grosse-Kunstleve
bcad852f6b Pure clang-format fixes (minor accident in PR #3039). 2021-06-29 16:16:00 -07:00
Jakob Lykke Andersen
7312e624b2 SH, improve error message from shared_ptr cast policy check 2021-06-29 05:57:32 -07:00
Jakob Lykke Andersen
be60fc52a4 Allow move policy in smart holder caster for shared_ptr 2021-06-29 05:57:32 -07:00
Jakob Lykke Andersen
c807807c55 Allow copy policy in smart holder caster for shared_ptr 2021-06-29 05:57:32 -07:00
Ralf W. Grosse-Kunstleve
68b6f8c612 Tracking change in type_caster_base.h on master (PR #3059). 2021-06-25 21:57:45 -07:00
Cris Luengo
57a36633c4 fix: enable py::implicitly_convertible<py::none, ...> for py::class_-wrapped types (#3059)
* Allow casting from None to a custom object, closes #2778

* ci.yml patch from the smart_holder branch for full CI coverage.
2021-06-25 21:57:45 -07:00
Cris Luengo
93e69191c1
fix: enable py::implicitly_convertible<py::none, ...> for py::class_-wrapped types (#3059)
* Allow casting from None to a custom object, closes #2778

* ci.yml patch from the smart_holder branch for full CI coverage.
2021-06-25 17:56:17 -07:00
Ralf W. Grosse-Kunstleve
898d5b301c Manually fixing merge conflict. 2021-06-22 12:19:40 -07:00
Aaron Gokaslan
dac74ebdf5
fix(clang-tidy): performance fixes applied in tests and CI (#3051)
* Initial fixes

* Whoops

* Finish clang-tidy manual fixes

* Add two missing fixes

* Revert

* Update clang-tidy

* Try to fix unreachable code error

* Move nolint comment

* Apply missing fix

* Don't override clang-tidy config

* Does this fix clang-tidy?

* Make all clang-tidy errors visible

* Add comments about NOLINTs and remove a few

* Fix typo
2021-06-22 12:11:54 -04:00
Ralf W. Grosse-Kunstleve
8d1e0b3903
[smart_holder] clang-tidy fixes (related to recent clang-tidy changes on master). (#3053)
* clang-tidy fixes (related to recent clang-tidy changes on master).

* More clang-tidy fixes.
2021-06-21 12:40:10 -07:00
Ralf W. Grosse-Kunstleve
274b014578 Merge branch 'master' into smart_holder 2021-06-21 07:51:19 -07:00
Aaron Gokaslan
3b30b0a51e
fix(clang-tidy): clang-tidy readability and misc fixes, like adding const (#3052)
* Enable and apply clang-tidy readability and misc fixes.

* Revert deprecated tester

* add space to tests/test_constants_and_functions.cpp
2021-06-21 10:37:48 -04:00
Ralf W. Grosse-Kunstleve
dca304f29e Merge branch 'master' into smart_holder 2021-06-19 10:55:20 -07:00
Aaron Gokaslan
af6218ff78
fix(clang-tidy): Apply performance fixes from clang-tidy (#3046)
* Apply performance fixes from clang-tidy

* 2nd Round of Perf Optimizations

* 3rd round of fixes & handle false-positive

* Apply missing fix and clang-format

* Apply reviewer comment
2021-06-19 10:53:27 -07:00
Aaron Gokaslan
c44b41e7c6
[SmartHolder] fix(clang-tidy): apply clang-tidy performance fixes (#3048)
* Apply clang-tidy performance fixes

* Fix bug introduced by double insert

* Revert all non-smart-holder changes
2021-06-18 12:43:31 -07:00
Ralf W. Grosse-Kunstleve
840eb84aff
Removing std::get_deleter const_cast to retest because the code has changed significantly. (#3045) 2021-06-17 13:13:21 -07:00
Ralf W. Grosse-Kunstleve
4f61912646
py::smart_holder std::shared_ptr deleter simplification & optimization. (#3041) 2021-06-16 17:47:22 -07:00
Ralf W. Grosse-Kunstleve
a6b2aadf45 Merge branch 'smart_holder' of https://github.com/pybind/pybind11 into smart_holder 2021-06-08 12:15:34 -07:00
Ralf W. Grosse-Kunstleve
48c7a3a68f Merge branch 'master' into smart_holder 2021-06-08 12:05:19 -07:00
Jack S. Hale
4c7697dbe9
Add const T to docstring generation. (#3020)
* Add const T to docstring generation.

* Change order.

* See if existing test triggers for a const type.

* Add tests.

* Fix test.

* Remove experiment.

* Reformat.

* More tests, checks run.

* Adding `test_fmt_desc_` prefix to new test functions.

* Using pytest.mark.parametrize to 1. condense test; 2. exercise all functions even if one fails; 3. be less platform-specific (e.g. C++ float is not necessarily float32).

Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
2021-06-08 11:56:45 -07:00
Robert Haschke
91f97ca401
smart_holder fixups (#3012)
* Drop constraints on casting of std::shared_ptr

std::shared_ptrs can be shared across python and C++ by design.

* Correctly report casting error

It is important to return an empty handle.
Simply returning None, would skip the error handling in
simple_collector / unpacking_collector, although a python exception is set.
A function call would then be processed with a (wrong) None argument!

* Return None for nullptr

* Revert "Drop constraints on casting of std::shared_ptr"

This reverts commit 7cf53ae8b4.
2021-05-28 06:35:50 -07:00
Ralf W. Grosse-Kunstleve
2cfc017e2d Merge branch 'master' into smart_holder 2021-05-27 11:30:03 -07:00
Yichen
3ac690b88b
Explicitly export exception types. (#2999)
* Set visibility of exceptions to default.

Co-authored-by: XZiar <czktc2007@gmail.com>

* add test

* update docs

* Skip failed test.
2021-05-27 08:00:18 -07:00
Ralf W. Grosse-Kunstleve
6cca66b276 Merge branch 'master' into smart_holder 2021-05-03 22:15:44 -07:00
Ralf W. Grosse-Kunstleve
bc3cd8a8a8
Splitting out include/pybind11/detail/pragma_warning_block.h (#2988)
* Splitting out include/pybind11/detail/pragma_warning_block.h

* Always including pragma_warning_block.h before common.h (resolved 8 CI failures).
2021-05-03 22:08:04 -07:00
Pieter P
0c93a0f3fc
Fix Unicode support for ostream redirects (#2982)
* Crash when printing Unicode to redirected cout
Add failing tests

* Fix Unicode crashes redirected cout

* pythonbuf::utf8_remainder check end iterator

* Remove trailing whitespace and formatting iostream

* Avoid buffer overflow if ostream redirect races
This doesn't solve the actual race, but at least it now has a much lower
probability of reading past the end of the buffer even when data races
do occur.
2021-05-03 22:04:38 -07:00
Ralf W. Grosse-Kunstleve
7c7c336d6d Merge branch 'master' into smart_holder 2021-04-19 11:32:47 -07:00
Ralf W. Grosse-Kunstleve
99de498b26
Bug fix: adding back !is_alias<Class>(ptr) that were accidentally omitted. (#2958)
* Bug fix: adding back `!is_alias<Class>(ptr)` that were accidentally omitted.

* Introducing PYBIND11_SH_AVL, PYBIND11_SH_DEF macros. Applying PYBIND11_SH_DEF to test_factory_constructors.py to complete test coverage.

* Using PYBIND11_SH_DEF in test_methods_and_attributes.cpp, for more complete test coverage.

* Using PYBIND11_SH_DEF in test_multiple_inheritance.cpp, for more complete test coverage.

* Cleaning up test_classh_mock.cpp.

* Better explanations for PYBIND11_SH_AVL, PYBIND11_SH_DEF.

* Disabling 3.10-dev builds.
2021-04-19 10:54:37 -07:00
mvoelkle-cern
e08a58111d
Fix compilation with gcc < 5 (#2956)
When the user defines _GLIBCXX_USE_CXX11_ABI=0 to force the pre-c++11 ABI, numpy.h assumes that is_trivially_copyable is available.
It is not necessarily the case. This patch uses clang's feature detection instead.
The workaround is for certain libstdc++ versions, so the test should target these particular versions.
2021-04-19 13:53:57 -04:00
Ralf W. Grosse-Kunstleve
d368b72881
Connecting PYBIND11_INTERNALS_VERSION to PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. (#2939)
* Connecting PYBIND11_INTERNALS_VERSION to PYBIND11_USE_SMART_HOLDER_AS_DEFAULT.

* Adding section: Classic / Conservative / Progressive cross-module compatibility
2021-04-16 07:15:23 -07:00
Ralf W. Grosse-Kunstleve
1c8795a205 Changing PYBIND11_SMART_HOLDER_TYPE_CASTERS to use __VA_ARGS__. 2021-04-15 10:33:44 -07:00
Ralf W. Grosse-Kunstleve
5f050c4a7e Merge branch 'master' into smart_holder 2021-04-14 12:21:49 -07:00
Philipp Bucher
62976cfcb8
fix: using -Werror-all for Intel (#2948)
* correcting Werror for Intel

* adding ward for Intel

* adding wards for intel

* another ward for Intel

* missed one intel ward

* exact match for intel compiler

* removing inline limits

* disable warnings about inline limits

* formatter suggestion

* more indent

* hopefully make formatter happy

* addressed review

* fix &&

* Update tests/CMakeLists.txt

Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>

Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
2021-04-14 14:01:27 -04:00
Ralf W. Grosse-Kunstleve
e9ae11f9ef Merge branch 'master' into smart_holder 2021-04-13 17:08:30 -07:00
Tamaki Nishino
6709abba93
Allow function pointer extraction from overloaded functions (#2944)
* Add a failure test for overloaded functions

* Allow function pointer extraction from overloaded functions
2021-04-13 16:53:56 -07:00
Ralf W. Grosse-Kunstleve
8efd5e3820
Bug fix: trampoline_self_life_support CpCtor, MvCtor. (#2947) 2021-04-13 05:34:46 -07:00
Ralf W. Grosse-Kunstleve
6c922614ed
Adding reclaim_disowned logic & miscellaneous naming and documentation improvements. (#2943)
* Using new smart_holder::reclaim_disowned in smart_holder_type_caster for unique_ptr.

* Systematically renaming was_disowned to is_disowned (because disowning is now reversible: reclaim_disowned).

* Systematically renaming virtual_overrider_self_life_support to trampoline_self_life_support (to reuse existing terminology instead of introducing new one).

* Systematically renaming test_class_sh_with_alias to test_class_sh_trampoline_basic.

* Adding a Trampolines and std::unique_ptr section to README_smart_holder.rst.

* MSVC compatibility.
2021-04-09 23:08:44 -07:00
Ralf W. Grosse-Kunstleve
2b4fbbd521
Bug fix for virtual_overrider_self_life_support ASAN heap-use-after-free failure. (#2942)
* Porting subset of absltest code from reproducer provided by @elkhrt. Baseline for debugging ASAN heap-use-after-free.

* Moving Py_DECREF to resolve ASAN heap-use-after-free failure.

* Fixing trivial formatting issue.

* Workaround for clang 3.6 and 3.7.
2021-04-08 22:56:46 -07:00
Ralf W. Grosse-Kunstleve
2f624af1ac Merge branch 'master' into smart_holder 2021-04-02 18:20:27 -07:00
Ralf W. Grosse-Kunstleve
ad6bf5cd39
Adding PyGILState_Check() in object_api<>::operator(). (#2919)
* Adding PyGILState_Check() in object_api<>::operator().

* Enabling PyGILState_Check() for Python >= 3.6 only.

Possibly, this explains why PyGILState_Check() cannot safely be used with Python 3.4 and 3.5:

https://github.com/python/cpython/pull/10267#issuecomment-434881587

* Adding simple micro benchmark.

* Reducing test time to minimum (purely for coverage, not for accurate results).

* Fixing silly oversight.

* Minor code organization improvement in test.

* Adding example runtimes.

* Removing capsys (just run with `-k test_callback_num_times -s` and using `.format()`.
2021-04-02 18:17:12 -07:00
Ralf W. Grosse-Kunstleve
488014076b Merge branch 'master' into smart_holder 2021-04-02 11:36:35 -07:00
Robert Haschke
c2db53da56
fix: catch missing self argument in overloads constructor (#2914) 2021-04-02 13:13:44 -04:00
Ralf W. Grosse-Kunstleve
5319ca3817
Using dynamic_cast<AliasType> to determine pointee_depends_on_holder_owner. (#2910)
* Adaption of PyCLIF virtual_py_cpp_mix test.

* Removing ValueError: Ownership of instance with virtual overrides in Python cannot be transferred to C++. TODO: static_assert alias class needs to inherit from virtual_overrider_self_life_support.

* Bringing back ValueError: "... instance cannot safely be transferred to C++.", but based on dynamic_cast<AliasType>.

* Fixing oversight: adding test_class_sh_virtual_py_cpp_mix.cpp to cmake file.

* clang <= 3.6 compatibility.

* Fixing oversight: dynamic_raw_ptr_cast_if_possible needs special handling for To = void. Adding corresponding missing test in test_class_sh_virtual_py_cpp_mix. Moving dynamic_raw_ptr_cast_if_possible to separate header.

* Changing py::detail::virtual_overrider_self_life_support to py::virtual_overrider_self_life_support.
2021-03-19 12:18:39 -07:00
Ralf W. Grosse-Kunstleve
62a8d35831 Fixing oversight: clang-format pybind11.h (affecting smart_holder code only). 2021-03-18 11:23:14 -07:00
Ralf W. Grosse-Kunstleve
245d31cb03
Renaming PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS to PYBIND11_TYPE_CASTER_BASE_HOLDER. (#2907) 2021-03-17 04:56:11 -07:00
Ralf W. Grosse-Kunstleve
2ada792085 Pure clang-format cleanup (after #2904), NO other changes. 2021-03-17 03:52:43 -07:00
Robert Haschke
784092dfd2
Missing cast from const unique_ptr& (#2904)
* Add roundtrip tests for unique_ptr

* Implementation for casting from const std::unique_ptr&

... forwarding to smart_holder_type_caster<T>::cast(T*)
2021-03-16 18:10:12 -07:00
Ralf W. Grosse-Kunstleve
469792032a
Adding virtual_overrider_self_life_support. (#2902)
* Initial version of virtual_overrider_self_life_support (enables safely passing unique_ptr to C++).

* Clang 3.6, 3.7 compatibility.

* Adding missing default constructor.

* Restoring test for exception for the case that virtual_overrider_self_life_support is not used.

* Fixing oversight: Adding missing holder().ensure_was_not_disowned().

* Adding unit tests for new `struct smart_holder` member functions.

* Moving virtual_overrider_self_life_support to separate include file, with iwyu cleanup.
2021-03-16 06:31:24 -07:00
Ralf W. Grosse-Kunstleve
d6cf6dfed3 Merge branch 'master' into smart_holder 2021-03-09 15:11:35 -08:00
Boris Staletic
f110889dde
Use correct duration representation when casting from datetime.timdelta to std::chrono::duration (#2870)
* Use correct duration representation when casting from datetime.timdelta to std::chrono::duration

* When asserting datetime/timedelta/date/time we can equality-compare whole objects
2021-03-07 23:42:01 -08:00
Ralf W. Grosse-Kunstleve
97a7fb722a
Porting/adapting Dustin's PR #2839 to smart_holder branch (#2886)
* WIP: test setup complete, AddInCppUniquePtr failing (reproduces PyCLIF smart_ptrs_test failure).

* Fully tested locally.

* Adding new tests to cmake file.
2021-03-03 17:58:42 -08:00
Ralf W. Grosse-Kunstleve
6a7e9f42fe
Changing all but one std::runtime_error to std::invalid_argument, which appears as ValueError in the Python interpreter. Adding test_cannot_disown_use_count_ne_1. (#2883) 2021-03-03 05:08:47 -08:00
Ralf W. Grosse-Kunstleve
3a336a2047
shared_ptr<bool> vptr_deleter_armed_flag_ptr (instead of unique_ptr) (#2882)
* shared_ptr<bool> vptr_deleter_armed_flag_ptr (instead of unique_ptr), to fix heap-use-after-free bug.

* Fixing  generated by some compilers in the pybind11 CI suite.
2021-03-02 17:43:25 -08:00
Ralf W. Grosse-Kunstleve
666fc0d8e8
Changing cast_error("... Python instance was disowned.") to value_error (which changes RuntimeError to ValueError). (#2880) 2021-02-26 23:19:23 -08:00
Ralf W. Grosse-Kunstleve
01e0045547
Enabling use of smart_holder for types with non-public destructors. (#2878)
* Enabling use of smart_holder for types with non-public destructors.

* Resolving clang-tidy error (GitHub CI).
2021-02-26 17:51:50 -08:00
Ralf W. Grosse-Kunstleve
1bafd5db5f
Adding py::smart_holder (for smart-pointer interoperability). (#2672)
* Adding test_unique_ptr_member (for desired PyCLIF behavior).

See also: https://github.com/pybind/pybind11/issues/2583

Does not build with upstream master or
https://github.com/pybind/pybind11/pull/2047, but builds with
https://github.com/RobotLocomotion/pybind11 and almost runs:

```
Running tests in directory "/usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests":
================================================================================= test session starts =================================================================================
platform linux -- Python 3.8.5, pytest-5.4.3, py-1.9.0, pluggy-0.13.1
rootdir: /usr/local/google/home/rwgk/forked/EricCousineau-TRI/pybind11/tests, inifile: pytest.ini
collected 2 items

test_unique_ptr_member.py .F                                                                                                                                                    [100%]

====================================================================================== FAILURES =======================================================================================
_____________________________________________________________________________ test_pointee_and_ptr_owner ______________________________________________________________________________

    def test_pointee_and_ptr_owner():
        obj = m.pointee()
        assert obj.get_int() == 213
        m.ptr_owner(obj)
        with pytest.raises(ValueError) as exc_info:
>           obj.get_int()
E           Failed: DID NOT RAISE <class 'ValueError'>

test_unique_ptr_member.py:17: Failed
============================================================================= 1 failed, 1 passed in 0.06s =============================================================================
```

* unique_ptr or shared_ptr return

* new test_variant_unique_shared with vptr_holder prototype

* moving prototype code to pybind11/vptr_holder.h, adding type_caster specialization to make the bindings involving unique_ptr passing compile, but load and cast implementations are missing

* disabling GitHub Actions on pull_request (for this PR)

* disabling AppVeyor (for this PR)

* TRIGGER_SEGSEV macro, annotations for GET_STACK (vptr::get), GET_INT_STACK (pointee)

* adding test_promotion_of_disowned_to_shared

* Copying tests as-is from xxx_value_ptr_xxx_holder branch.

https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder

Systematically exercising returning and passing unique_ptr<T>, shared_ptr<T>
with unique_ptr, shared_ptr holder.

Observations:

test_holder_unique_ptr:
  make_unique_pointee  OK
  pass_unique_pointee  BUILD_FAIL (as documented)
  make_shared_pointee  Abort free(): double free detected
  pass_shared_pointee  RuntimeError: Unable to load a custom holder type from a default-holder instance

test_holder_shared_ptr:
  make_unique_pointee  Segmentation fault (#1138)
  pass_unique_pointee  BUILD_FAIL (as documented)
  make_shared_pointee  OK
  pass_shared_pointee  OK

* Copying tests as-is from xxx_value_ptr_xxx_holder branch.

https://github.com/rwgk/pybind11/tree/xxx_value_ptr_xxx_holder

Systematically exercising casting between shared_ptr<base>, shared_ptr<derived>.

* Demonstration of Undefined Behavior in handling of shared_ptr holder.

Based on https://godbolt.org/z/4fdjaW by jorgbrown@ (thanks Jorg!).

* Additional demonstration of Undefined Behavior in handling of shared_ptr holder.

* fixing up-down mixup in comment

* Demonstration of Undefined Behavior in handling of polymorphic pointers.

(This demo does NOT involve smart pointers at all, unlike the otherwise similar test_smart_ptr_private_first_base.)

* minor test_private_first_base.cpp simplification (after discovering that this can be wrapped with Boost.Python, using boost::noncopyable)

* pybind11 equivalent of Boost.Python test similar to reproducer under #1333

* Snapshot of WIP, TODO: shared_ptr deleter with on/off switch

* Adding vptr_deleter.

* Adding from/as unique_ptr<T> and unique_ptr<T, D>.

* Adding from_shared_ptr. Some polishing.

* New tests/core/smart_holder_poc_test.cpp, using Catch2.

* Adding in vptr_deleter_guard_flag.

* Improved labeling of TEST_CASEs.

* Shuffling existing TEST_CASEs into systematic matrix.

* Implementing all [S]uccess tests.

* Implementing all [E]xception tests.

* Testing of exceptions not covered by the from-as matrix.

* Adding top-level comment.

* Converting from methods to factory functions (no functional change).

* Removing obsolete and very incomplete test (replaced by Catch2-based test).

* Removing stray file.

* Adding type_caster_bare_interface_demo.

* Adding shared_ptr<mpty>, shared_ptr<mpty const> casters.

* Adding unique_ptr<mpty>, unique_ptr<mpty const> casters.

* Pure copy of `class class_` implementation in pybind11.h (master commit 98f1bbb800).

* classh.h: renaming of class_ to classh + namespace; forking test_classh_wip from test_type_caster_bare_interface_demo.

* Hard-coding smart_holder into classh.

* Adding mpty::mtxt string member.

* Adding isinstance<mpty> in type_caster::load functions.

* Adding rvalue_ref, renaming const_value_ref to lvalue_ref & removing const.

* Retrieving smart_holder pointer in type_caster<mpty>::load, and using it cast_op operators.

* Factoring out smart_holder_type_caster_load.

* Retrieving smart_holder pointer in type_caster<std::shared_ptr<mpty[ const]>>::load, and using it cast_op operators.

* Improved error messaging: Cannot disown nullptr (as_unique_ptr).

* Retrieving smart_holder pointer in type_caster<std::unique_ptr<mpty[ const]>>::load, and using it cast_op operators.

* Pure `clang-format --style=file -i` change.

* Pure `clang-format --style=file -i` change, with two `clang-format off` directives.

* Fixing oversight (discovered by flake8).

* flake8 cleanup

* Systematically setting mtxt for all rtrn_mpty_* functions (preparation, the values are not actually used yet).

* static cast handle for rtrn_cptr works by simply dropping in code from type_caster_base (marked with comments).

* static cast handle for rtrn_cref works by simply dropping in code from type_caster_base (marked with comments). rtrn_mref and rtrn_mptr work via const_cast (to add const).

* static cast handle for rtrn_valu works by simply dropping in code from type_caster_base (marked with comments). rtrn_rref raises a RuntimeError, to be investigated.

* Copying type_caster_generic::cast into type_caster<mpty> as-is (preparation for handling smart pointers).

* Pure clang-format change (applied to original type_caster_generic::cast).

* Adding comment re potential use_count data race.

* static handle cast implementations for rtrn_shmp, rtrn_shcp.

* Adding MISSING comments in operator std::unique_ptr<mpty[ const]>.

* static handle cast implementations for rtrn_uqmp, rtrn_uqcp.

* Bug fix: vptr_deleter_armed_flag_ptr has to live on the heap.

See new bullet point in comment section near the top.

The variable was also renamed to reflect its function more accurately.

* Fixing bugs discovered by ASAN. The code is now ASAN, MSAN, UBSAN clean.

* Making test_type_caster_bare_interface_demo.cpp slightly more realistic, ASAN, MSAN, UBSAN clean.

* Calling deregister_instance after disowning via unique_ptr.

* Removing enable_shared_from_this stub, simplifying existing code, clang-format.

Open question, with respect to the original code:
76a160070b/include/pybind11/pybind11.h (L1510)
To me it looks like the exact situation marked as `std::shared_ptr<Good> gp1 = not_so_good.getptr();` here: https://en.cppreference.com/w/cpp/memory/enable_shared_from_this
The comment there is: `// undefined behavior (until C++17) and std::bad_weak_ptr thrown (since C++17)`
Does the existing code have UB pre C++17?

I'll leave handling of enable_shared_from_this for later, as the need arises.

* Cosmetical change around helper functions.

* Using type_caster_base<mpty>::src_and_type directly, removing copy. Also renaming one cast to cast_const_raw_ptr, for clarity.

* Fixing clang-format oversight.

* Using factored-out make_constructor (PR #2798), removing duplicate code.

* Inserting additional assert to ensure a returned unique_ptr is always a new Python instance.

* Adding minor comment (change to internals needed to distinguish uninitialized/disowned in error message).

* Factoring out find_existing_python_instance().

* Moving factored-out make_constructor to test_classh_wip.cpp, restoring previous version of cast.h. This is currently the most practical approach. See PR #2798 for background.

* Copying classh type_casters from test_classh_wip.cpp UNMODIFIED, as a baseline for generalizing the code.

* Using pybind11/detail/classh_type_casters.h from test_classh_wip.cpp.

* Adding & using PYBIND11_CLASSH_TYPE_CASTERS define.

* Adding test_classh_inheritance, currently failing (passes with class_).

* Removing .clang-format before git rebase master (where the file was added).

* Bringing back .clang-format, the previous rm was a bad idea.

* Folding in modified_type_caster_generic_load_impl, just enough to pass test_class_wip. test_classh_inheritance is still failing, but with a different error: [RuntimeError: Incompatible type (as_raw_ptr_unowned).]

* Minimal changes needed to pass test_classh_inheritance.

* First pass adjusting try_implicit_casts and try_load_foreign_module_local to capture loaded_v_h, but untested and guarded with pybind11_failure("Untested"). This was done mainly to determine general feasibility. Note the TODO in pybind11.h, where type_caster_generic::local_load is currently hard-coded. test_classh_wip and test_classh_inheritance still pass, as before.

* Decoupling generic_type from type_caster_generic.

* Changes and tests covering classh_type_casters try_implicit_casts.

* Minimal test covering classh_type_casters load_impl Case 2b.

* Removing stray isinstance<T>(src): it interferes with the py::module_local feature. Adding missing #includes.

* Tests for classh py::module_local() feature.

* Pure renaming of function names in test_classh_inheritance, similar to the systematic approach used in test_class_wip. NO functional changes.

* Pure renaming of function and variable names, for better generalization when convoluting with inheritance. NO functional changes.

* Adopting systematic naming scheme from test_classh_wip. NO functional changes.

* Moving const after type name, for functions that cover a systematic scheme. NO functional changes.

* Adding smart_holder_type_caster_load::loaded_as_shared_ptr, currently bypassing smart_holder shared_ptr tracking completely, but the tests pass and are sanitizer clean.

* Removing rtti_held from smart_holder. See updated comment.

* Cleaning up loaded_as_raw_ptr_unowned, loaded_as_shared_ptr.

* Factoring out convert_type and folding into loaded_as_unique_ptr.

* Folding convert_type into lvalue_ref and rvalue_ref paths. Some smart_holder_type_caster_load cleanup.

* Using unique_ptr in local_load to replace static variable. Also adding local_load_safety_guard.

* Converting test_unique_ptr_member to using classh: fully working, ASAN, MSAN, UBSAN clean.

* Removing debugging comments (GET_STACK, GET_INT_STACK). cast.h is identical to current master again, pybind11.h only has the generic_type::initialize(..., &type_caster_generic::local_load) change.

* Purging obsolete pybind11/vptr_holder.h and associated test.

* Moving several tests to github.com/rwgk/rwgk_tbx/tree/main/pybind11_tests

a2c2f88174

These tests are from experimenting, and for demonstrating UB in pybind11 multiple inheritance handling ("first_base"), to be fixed later.

* Adding py::smart_holder support to py::class_, purging py::classh completely.

* Renaming files in include directory, creating pybind11/smart_holder.h.

* Renaming all "classh" to "smart_holder" in pybind11/detail/smart_holder_type_casters.h.

The user-facing macro is now PYBIND11_SMART_HOLDER_TYPE_CASTERS.

* Systematically renaming tests to use "class_sh" in the name.

* Renaming test_type_caster_bare_interface_demo to test_type_caster_bare_interface.

* Renaming new tests/core subdirectory to tests/pure_cpp.

* Adding new tests to CMake config, resetting CI config.

* Changing CMake file so that test_class_sh_module_local.py actually runs.

* clang-tidy fixes.

* 32-bit compatibility.

* Reusing type_caster_base make_copy_constructor, make_move_constructor with a trick.

* CMake COMPARE NATURAL is not available with older versions.

* Adding copyright notices to new header files.

* Explicitly define copy/move constructors/assignments.

* Adding new header files to tests/extra_python_package/test_files.py.

* Adding tests/pure_cpp/CMakeLists.txt.

* Making use of the new find_existing_python_instance() function factored out with PR #2822.

* Moving define PYBIND11_SMART_HOLDER_TYPE_CASTERS(T) down in the file. NO functional changes. Preparation for follow-up work (to keep that diff smaller).

* Reintroducing py::classh, this time as a simple alias for py::class_<U, py::smart_holder>.

* Replacing detail::is_smart_holder<H> in cast.h with detail::is_smart_holder_type_caster<T>.
Moving get_local_load_function_ptr, init_instance_for_type to smart_holder_type_caster_class_hooks.
Expanding static_assert in py::type::handle_of<> to accommodate smart_holder_type_casters.

* Fixing oversight.

* Adding classu alias for class_<U, std::unique_ptr<U>>.

* Giving up on idea to use legacy init_instance only if is_base_of<type_caster_generic, type_caster<T>. There are use cases in the wild that define both a custom type_caster and class_.

* Removing test_type_caster_bare_interface, which was moved to the separate PR #2834.

* Moving up is_smart_holder_type_caster, to also use in cast_is_temporary_value_reference.

* Adding smart_holder_type_casters for unique_ptr with custom deleter. SEVERE CODE DUPLICATION. This commit is to establish a baseline for consolidating the unique_ptr code.

* Unification of unique_ptr, unique_ptr_with_deleter code in smart_holder_poc.h. Leads to more fitting error messages. Enables use of unique_ptr<T, D> smart_holder_type_casters also for unique_ptr<T>.

* Copying files as-is from branch test_unique_ptr_member (PR #2672).

* Adding comment, simplifying naming, cmake addition.

* Introducing PYBIND11_USE_SMART_HOLDER_AS_DEFAULT macro (tested only undefined; there are many errors with the macro defined).

* Removing test_type_caster_bare_interface, which was moved to the separate PR #2834.

* Fixing oversight introduced with commit 95425f13d6.

* Setting record.default_holder correctly for PYBIND11_USE_SMART_HOLDER_AS_DEFAULT.

With this test_class.cpp builds and even mostly runs, except
`test_multiple_instances_with_same_pointer`, which segfaults because it is
using a `unique_ptr` holder but `smart_holder` `type_caster`.

Also adding `static_assert`s to generate build errors for such situations,
but guarding with `#if 0` to first pivot to test_factory_constructors.cpp.

* Fixing up cast.h and smart_holder.h after rebase.

* Removing detail/smart_holder_type_casters.h in separate commit.

* Commenting out const in def_buffer(... const). With this, test_buffers builds and runs with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT. Explanation why the const needs to be removed, or fix elsewhere, is still needed, but left for later.

* Adding test_class_sh_factory_constructors, reproducing test_factory_constructors failure. Using py::class_ in this commit, to be changed to py::classh for debugging.

* Removing include/pybind11/detail/smart_holder_type_casters.h from CMakeLists.txt, test_files.py (since it does not exist in this branch).

* Adding // DANGER ZONE reminders.

* Converting as many py::class_ to py::classh as possible, not breaking tests.

* Adding initimpl::construct() overloads, resulting in test_class_sh_factory_constructors feature parity for py::class_ and py::classh.

* Adding enable_if !is_smart_holder_type_caster to existing initimpl::construct(). With this test_factory_constructors.cpp builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT.

* Disabling shared_ptr&, shared_ptr* tests when building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT for now, pending work on smart_holder_type_caster<shared_ptr>.

* Factoring out struct and class definitions into anonymous namespace. Preparation for building with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT.

* Simplifying from_unique_ptr(): typename D = std::default_delete<T> is not needed. Factoring out is_std_default_delete<T>() for consistentcy between ensure_compatible_rtti_uqp_del() and from_unique_ptr().

* Introducing PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. Using it in test_smart_ptr.cpp. With this test_smart_ptr builds with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT and all but one test run successfully.

* Introducing 1. type_caster_for_class_, used in PYBIND11_MAKE_OPAQUE, and 2. default_holder_type, used in stl_bind.h.

* Using __VA_ARGS__ in PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS.

* Replacing condense_for_macro with much simpler approach.

* Softening static_assert, to only check specifically that smart_holder is not mixed with type_caster_base, and unique_ptr/shared_ptr holders are not mixed with smart_holder_type_casters.

* Adding PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS in test_class.cpp (with this all but one test succeed with PYBIND11_USE_SMART_HOLDER_AS_DEFAULT).

* Adding remaining PYBIND11_SMART_POINTER_HOLDER_TYPE_CASTERS. static_assert for "necessary conditions" for both types of default holder, static_assert for "strict conditions" guarded by new PYBIND11_STRICT_ASSERTS_CLASS_HOLDER_VS_TYPE_CASTER_MIX. All tests build & run as before with unique_ptr as the default holder, all tests build for smart_holder as the default holder, even with the strict static_assert.

* Introducing check_is_smart_holder_type_caster() function for runtime check, and reinterpreting record.default_holder as "uses_unique_ptr_holder". With this test_smart_ptr succeeds. (All 42 tests build, 35 tests succeed, 5 run but have some failures, 2 segfault.)

* Bug fix: Adding have_value() to smart_holder_type_caster_load. With this test_builtin_casters succeeds. (All 42 tests build, 36 tests succeed, 5 run but have some failures, 1 segfault.)

* Adding unowned_void_ptr_from_direct_conversion to modified_type_caster_generic_load_impl. This fixes the last remaining segfault (test_numpy_dtypes). New stats for all tests combined: 12 failed, 458 passed.

* Adding "Lazy allocation for unallocated values" (for old-style __init__) into load_value_and_holder. Deferring destruction of disowned holder until clear_instance, to remain inspectable for "uninitialized" or "disowned" detection. New stats for all tests combined: 5 failed, 465 passed.

* Changing std::shared_ptr pointer/reference to const pointer/reference. New stats for all tests combined: 4 failed, 466 passed.

* Adding return_value_policy::move to permissible policies for unique_ptr returns. New stats for all tests combined: 3 failed, 467 passed.

* Overlooked flake8 fixes.

* Manipulating failing ConstructorStats test to pass, to be able to run all tests with ASAN.

This version of the code is ASAN clean with unique_ptr or smart_holder as the default.

This change needs to be reverted after adopting the existing move-only-if-refcount-is-1
logic used by type_caster_base.

* Adding copy constructor and move constructor tracking to atyp. Preparation for a follow-up change in smart_holder_type_caster, to make this test sensitive to the changing behavior.

[skip ci]

* Removing `operator T&&() &&` from smart_holder_type_caster, for compatibility with the behavior of type_caster_base. Enables reverting 2 of 3 test manipulations applied under commit 249df7cbdb. The manipulation in test_factory_constructors.py is NOT reverted in this commit.

[skip ci]

* Fixing unfortunate editing mishap. This reverts the last remaining test manipulation in commit 249df7cbdb and makes all existing unit tests pass with smart_holder as default holder.

* GitHub CI clang-tidy fixes.

* Adding messages to terse `static_assert`s, for pre-C++17 compatibility.

* Using @pytest.mark.parametrize to run each assert separately (to see all errors, not just the first).

* Systematically removing _atyp from function names, to make the test code simpler.

* Using re.match to accommodate variable number of intermediate MvCtor.

* Also removing `operator T()` from smart_holder_type_caster, to fix gcc compilation errors. The only loss is pass_rref in test_class_sh_basic.

* Systematically replacing `detail::enable_if_t<...smart_holder...>` with `typename std::enable_if<...smart_holder...>::type`. Attempt to work around MSVC 2015 issues, to be tested via GitHub CI. The idea for this change originates from this comment: https://github.com/pybind/pybind11/issues/1616#issuecomment-444536813

* Importing re before pytest after observing a PyPy CI flake when importing pytest first.

* Copying MSVC 2015 compatibility change from branch pr2672_use_smart_holder_as_default.

* Introducing is_smart_holder_type_caster_base_tag, to keep smart_holder code more disconnected.

* Working around MSVC 2015 bug.

* Expanding comment for MSVC 2015 workaround.

* Systematically changing std::enable_if back to detail::enable_if_t, effectively reverting commit 5d4b6890a3.

* Removing unused smart_holder_type_caster_load::loaded_as_rvalue_ref (it was an oversight that it was not removed with commit 23036a45eb).

* Removing py::classu, because it does not seem useful enough.

* Reverting commit 6349531306 by un-commenting `const` in `def_buffer(...)`. To make this possible, `operator T const&` and `operator T const*` in `smart_holder_type_caster` need to be marked as `const` member functions.

* Adding construct() overloads for constructing smart_holder from alias unique_ptr, shared_ptr returns.

* Adding test_class_sh_factory_constructors.cpp to tests/CMakeLists.txt (fixes oversight, this should have been added long before).

* Compatibility with old clang versions (clang 3.6, 3.7 C++11).

* Cleaning up changes to existing unit tests.

* Systematically adding SMART_HOLDER_WIP tag. Removing minor UNTESTED tags (only the throw are not actually exercised, investing time there has a high cost but very little benefit).

* Splitting out smart_holder_type_casters again, into new detail/smart_holder_type_casters_inline_include.h.

* Splitting out smart_holder_init_inline_include.h.

* Adding additional new include files to CMakeLists.txt, tests/extra_python_package/test_files.py.

* clang-format cleanup of most smart_holder code.

* Adding source code comments in response to review.

* Simple micro-benchmark ("ubench") comparing runtime performance for several holders.

Tested using github.com/rwgk/pybind11_scons and Google-internal build system.
Sorry, no cmake support at the moment.

First results: https://docs.google.com/spreadsheets/d/1InapCYws2Gt-stmFf_Bwl33eOMo3aLE_gc9adveY7RU/edit#gid=0

* Breaking out number_bucket.h, adding hook for also collecting performance data for PyCLIF.

* Accounting for ubench in MANIFEST.in (simply prune, for now).

* Smarter determination of call_repetitions.

[skip ci]

* Also scaling performance data to PyCLIF.

[skip ci]

* Adding ubench/python/number_bucket.clif here for general visibility.

* Fix after rebase

* Merging detail/smart_holder_init_inline_include.h into detail/init.h.

* Renaming detail/is_smart_holder_type_caster.h -> detail/smart_holder_sfinae_hooks_only.h.

* Renaming is_smart_holder_type_caster -> type_uses_smart_holder_type_caster for clarity.

* Renaming type_caster_type_is_smart_holder_type_caster -> wrapped_type_uses_smart_holder_type_caster for clarity.

* Renaming is_smart_holder_type_caster_base_tag -> smart_holder_type_caster_base_tag for simplicity.

* Adding copyright notices and minor colateral cleanup.

* iwyu cleanup (comprehensive only for cast.h and smart_holder*.h files).

* Fixing `git rebase master` accident.

* Moving large `pragma warning` block from pybind11.h to detail/common.h.

* Fixing another `git rebase master` accident.
2021-02-23 21:50:42 -08:00
Bertrand MICHEL
74a767d429
Dtype kind vs char (#2864)
* [dtype]: add type() method to access type attribute of PyArray_Descr (eq. to dtype.char in Python)

* [dtype] change type() name method to char_() to be compliant with Python numpy interface

* [dtype] fix by pre-commit

* [dtype] Change comments and solutions format for test

* Clarify documentation and move note about dtype.char vs PyArray_Descr::type to a plain, non-doxygen comment

* Fix and extend tests

* Fix the supposedly fixed tests

* Fix the fixed tests again

Co-authored-by: Bertrand MICHEL <bertrand.michel@onera.fr>
Co-authored-by: Yannick Jadoul <yannick.jadoul@belgacom.net>
2021-02-23 10:57:28 +01:00
Dustin Spicuzza
c0fbb02c9f
Extract gil management functions to separate header (#2845) 2021-02-22 19:15:40 -08:00
Ralf W. Grosse-Kunstleve
0c42250a4e
Splitting out detail/type_caster_base.h from cast.h, with iwyu cleanup. (#2841)
* Splitting out detail/type_caster_base.h from cast.h.
* Include cleanup guided by include-what-you-use 0.12 based on clang version 9.0.1-11.
2021-02-22 18:38:18 -08:00
Vikram Pal
417067eeb8
Add pybind11::bytearray (#2799)
* Add initial implementation

* Add few more methods

* Add tests

* Fix a typo

* Use std::string constructor which takes size

* Fix implicit sign conversion error

* Add size method and test

* Remove implicit conversion

* Fix bytearray constructors and operator std::string()

* Make implicit bytearray constructor explicit

* Rerun tests

* Add null check

* Rerun tests

* Rerun tests - 2

* Remove NULL check
2021-02-14 15:51:13 +01:00
Karthik Nishanth
e791ec4e27
fix: add null pointer check with std::localtime (#2846) 2021-02-08 15:38:22 -05:00
Yannick Jadoul
6cf6bf203e
Fix confusing weakref constructor overload (#2832)
* Demonstrate issue with weakref constructor overloads

* Fix weakref constructor to convert on being passed a non-weakref object

* Improve on nonlocal-scoped variable in test_weakref

* Keep backwards-compatibility by introducing PYBIND11_OBJECT_CVT_DEFAULT macro

* Simplify test_weakref
2021-01-31 23:13:31 +01:00
Ralf W. Grosse-Kunstleve
932769b038
Adding holder_caster typename SFINAE = void hooks to help work around the current lack of smart-pointer interoperability (#2833)
* Adding move_only_holder_caster `typename SFINAE = void` to enable external specializations.

* Adding SFINAE hook also to copyable_holder_caster, for uniformity, with comment to explain the purpose.
2021-01-30 12:02:24 -08:00
Edward Lockhart
23c3edcf21
When determining if a shared_ptr already exists, use a test on the we… (#2819)
* When determining if a shared_ptr already exists, use a test on the weak_ptr instead of a try/catch block.

* When determining if a shared_ptr already exists, use a test on the weak_ptr instead of a try/catch block.

* weak_from_this is only available in C++17 and later

* Switch to use feature flag instead of C++ version flag.

* Add Microsoft-specific check.

* Avoid undefined preprocessor macro warning treated as error.

* Simplify shared_from_this in init_holder

* Include <version> in detail/common.h (~stolen~ borrowed from @bstaletic's #2816)

* Move class_::get_shared_from_this to detail::try_get_shared_from_this

* Simplify try_get_shared_from_this by using weak_ptr::lock()

Co-authored-by: Yannick Jadoul <yannick.jadoul@belgacom.net>
2021-01-30 20:05:13 +01:00
Ralf W. Grosse-Kunstleve
0432ae7c52
Changing pybind11::str to exclusively hold PyUnicodeObject (#2409)
* Changing pybind11::str to exclusively hold PyUnicodeObject
2021-01-29 09:41:42 -08:00
Yannick Jadoul
587d5f840a
Update breathe to 4.26.1, add make_tuple, make_iterator, and make_key_iterator (#2828) 2021-01-28 12:28:16 +01:00
Ralf W. Grosse-Kunstleve
9b7bfef833
Factoring out find_registered_python_instance() from type_caster_generic::cast. (#2822)
Factoring out find_registered_python_instance() from type_caster_generic::cast.
2021-01-26 21:08:46 -08:00
Henry Fredrick Schreiner
87954e7a54 fix: corrected dev versioning 2021-01-26 22:42:14 -05:00
Henry Fredrick Schreiner
4a5b81b1b7 chore: get back to work 2021-01-26 22:28:09 -05:00
Henry Schreiner
8de7772cc7
chore: prepare for the 2.6.2 release (#2821) 2021-01-26 21:26:45 -05:00
Henry Fredrick Schreiner
732bf88d19 fix: avoid changing class outside of GIL 2021-01-26 20:59:27 -05:00
Yannick Jadoul
0bb8ca2639
Always call PyNumber_Index when casting from Python to a C++ integral type, also pre-3.8 (#2801)
* Always call PyNumber_Index when casting from Python to a C++ integral type, also pre-3.8

* Fixed on PyPy

* Simplify use of PyNumber_Index, following @rwgk's idea, and ignore warnings in >=3.8

* Reproduce mismatch between pre-3.8 and post-3.8 behavior on __index__ throwing TypeError

* Fix tests on 3.6 <= Python < 3.8

* No, I don't have an uninitialized variable

* Fix use of __index__ on Python 2

* Make types in test_int_convert more ~boring~ descriptive
2021-01-25 21:05:17 +01:00
crimsoncor
9ea39dc356
Force the builtin module key to be the correct type. (#2814)
* Force the builtin module key to be the correct type.

Previously it was always going to be a std::string which converted into
unicode. Python 2 appears to want module keys to be normal str types, so
this was breaking code that expected plain string types in the
builtins.keys() data structure

* Add a simple unit test to ensure all built-in keys are str

* Update the unit test so it will also run on pypy

* Run pre-commit.

Co-authored-by: Jesse Clemens <jesse.clemens@sony.com>
2021-01-24 09:17:28 -08:00
Henry Fredrick Schreiner
5b43ac42a0 docs: fix missing line from #2310 2021-01-19 19:56:22 -05:00
Henry Schreiner
2db0264aca
style: add clang-format file (#2310)
* style: adding clang-format as manual hook

* docs: adding a Clang-Format section. (#2803)

Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
2021-01-19 19:10:26 -05:00
Michael Kuron
48534089f7
fix: Intel ICC C++17 compatibility (#2729)
* CI: Intel icc/icpc via oneAPI

Add testing for Intel icc/icpc via the oneAPI images.
Intel oneAPI is in a late beta stage, currently shipping
oneAPI beta09 with ICC 20.2.

CI: Skip Interpreter Tests for Intel

Cannot find how to add this, neiter the package `libc6-dev` nor
`intel-oneapi-mkl-devel` help when installed to solve this:
```
-- Looking for C++ include pthread.h
-- Looking for C++ include pthread.h - not found
CMake Error at /__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:165 (message):
  Could NOT find Threads (missing: Threads_FOUND)
Call Stack (most recent call first):
  /__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:458 (_FPHSA_FAILURE_MESSAGE)
  /__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindThreads.cmake:234 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
  tests/test_embed/CMakeLists.txt:17 (find_package)
```

CI: libc6-dev from GCC for ICC

CI: Run bare metal for oneAPI

CI: Ubuntu 18.04 for oneAPI

CI: Intel +Catch -Eigen

CI: CMake from Apt (ICC tests)

CI: Replace Intel Py with GCC Py

CI: Intel w/o GCC's Eigen

CI: ICC with verbose make

[Debug] Find core dump

tests: use arg{} instead of arg() for Intel

tests: adding a few more missing {}

fix: sync with @tobiasleibner's branch

fix: try ubuntu 20-04

fix: drop exit 1

docs: Apply suggestions from code review

Co-authored-by: Tobias Leibner <tobias.leibner@googlemail.com>

Workaround for ICC enable_if issues

Another workaround for ICC's enable_if issues

fix error in previous commit

Disable one test for the Intel compiler in C++17 mode

Add back one instance of py::arg().noconvert()

Add NOLINT to fix clang-tidy check

Work around for ICC internal error in PYBIND11_EXPAND_SIDE_EFFECTS in C++17 mode

CI: Intel ICC with C++17

docs: pybind11/numpy.h does not require numpy at build time. (#2720)

This is nice enough to be mentioned explicitly in the docs.

docs: Update warning about Python 3.9.0 UB, now that 3.9.1 has been released (#2719)

Adjusting `type_caster<std::reference_wrapper<T>>` to support const/non-const propagation in `cast_op`. (#2705)

* Allow type_caster of std::reference_wrapper<T> to be the same as a native reference.

Before, both std::reference_wrapper<T> and std::reference_wrapper<const T> would
invoke cast_op<type>. This doesn't allow the type_caster<> specialization for T
to distinguish reference_wrapper types from value types.

After, the type_caster<> specialization invokes cast_op<type&>, which allows
reference_wrapper to behave in the same way as a native reference type.

* Add tests/examples for std::reference_wrapper<const T>

* Add tests which use mutable/immutable variants

This test is a chimera; it blends the pybind11 casters with a custom
pytype implementation that supports immutable and mutable calls.

In order to detect the immutable/mutable state, the cast_op needs
to propagate it, even through e.g. std::reference<const T>

Note: This is still a work in progress; some things are crashing,
which likely means that I have a refcounting bug or something else
missing.

* Add/finish tests that distinguish const& from &

Fixes the bugs in my custom python type implementation,
demonstrate test that requires const& and reference_wrapper<const T>
being treated differently from Non-const.

* Add passing a const to non-const method.

* Demonstrate non-const conversion of reference_wrapper in tests.

Apply formatting presubmit check.

* Fix build errors from presubmit checks.

* Try and fix a few more CI errors

* More CI fixes.

* More CI fixups.

* Try and get PyPy to work.

* Additional minor fixups. Getting close to CI green.

* More ci fixes?

* fix clang-tidy warnings from presubmit

* fix more clang-tidy warnings

* minor comment and consistency cleanups

* PyDECREF -> Py_DECREF

* copy/move constructors

* Resolve codereview comments

* more review comment fixes

* review comments: remove spurious &

* Make the test fail even when the static_assert is commented out.

This expands the test_freezable_type_caster a bit by:
1/ adding accessors .is_immutable and .addr to compare identity
from python.
2/ Changing the default cast_op of the type_caster<> specialization
to return a non-const value. In normal codepaths this is a reasonable
default.
3/ adding roundtrip variants to exercise the by reference, by pointer
and by reference_wrapper in all call paths.  In conjunction with 2/, this
demonstrates the failure case of the existing std::reference_wrpper conversion,
which now loses const in a similar way that happens when using the default cast_op_type<>.

* apply presubmit formatting

* Revert inclusion of test_freezable_type_caster

There's some concern that this test is a bit unwieldly because of the use
of the raw <Python.h> functions. Removing for now.

* Add a test that validates const references propagation.

This test verifies that cast_op may be used to correctly detect
const reference types when used with std::reference_wrapper.

* mend

* Review comments based changes.

1. std::add_lvalue_reference<type> -> type&
2. Simplify the test a little more; we're never returning the ConstRefCaster
type so the class_ definition can be removed.

* formatted files again.

* Move const_ref_caster test to builtin_casters

* Review comments: use cast_op and adjust some comments.

* Simplify ConstRefCasted test

I like this version better as it moves the assertion that matters
back into python.

ci: drop pypy2 linux, PGI 20.7, add Python 10 dev (#2724)

* ci: drop pypy2 linux, add Python 10 dev

* ci: fix mistake

* ci: commented-out PGI 20.11, drop 20.7

fix: regression with installed pybind11 overriding local one (#2716)

* fix: regression with installed pybind11 overriding discovered one

Closes #2709

* docs: wording incorrect

style: remove redundant instance->owned = true (#2723)

which was just before set to True in instance->allocate_layout()

fix: also throw in the move-constructor added by the PYBIND11_OBJECT macro, after the argument has been moved-out (if necessary) (#2701)

Make args_are_all_* ICC workarounds unconditional

Disable test_aligned on Intel ICC

Fix test_aligned on Intel ICC

Skip test_python_alreadyset_in_destructor on Intel ICC

Fix test_aligned again

ICC CI: Downgrade pytest

pytest 6 does not capture the `discard_as_unraisable` stderr and
just writes a warning with its content instead.

* refactor: simpler Intel workaround, suggested by @laramiel

* fix: try version with impl to see if it is easier to compile

* docs: update README for ICC

Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
2021-01-17 19:53:07 -05:00
Yannick Jadoul
8449a8089c
fix: only allow integer type_caster to call __int__ method when conversion is allowed; always call __index__ (#2698)
* Only allow integer type_caster to call __int__ or __index__ method when conversion is allowed

* Remove tests for __index__ as this seems to only be used to convert to int in 3.8+

* Take both `int` and `long` types into account for Python 2

* Add test_numpy_int_convert to assert tests currently fail, even though np.intc has an __index__ method

* Also consider __index__ as noconvert to a C++ integer

* New-style classes for Python 2.7; sigh

* Add some tests on types with custom __index__ method

* Ignore some tests in Python <3.8

* Update comment about conversion from np.float32 to C++ int

* Workaround difference between CPython and PyPy's different PyIndex_Check (unnoticed because we currently don't have PyPy >= 3.8)

* Avoid ICC segfault with py::arg()
2021-01-16 20:52:14 -05:00