Commit Graph

669 Commits

Author SHA1 Message Date
Chris Ohk
1a432b426f
docs: Correct minor typos (#3721) 2022-02-11 10:42:48 -05:00
Ralf W. Grosse-Kunstleve
6493f496e3
Python 2 removal part 1: tests (C++ code is intentionally ~untouched) (#3688)
* `#error BYE_BYE_GOLDEN_SNAKE`

* Removing everything related to 2.7 from ci.yml

* Commenting-out Centos7

* Removing `PYTHON: 27` from .appveyor.yml

* "PY2" removal, mainly from tests. C++ code is not touched.

* Systematic removal of `u` prefix from `u"..."` and `u'...'` literals. Collateral cleanup of a couple minor other things.

* Cleaning up around case-insensitive hits for `[^a-z]py.*2` in tests/.

* Removing obsolete Python 2 mention in compiling.rst

* Proper `#error` for Python 2.

* Using PY_VERSION_HEX to guard `#error "PYTHON 2 IS NO LONGER SUPPORTED.`

* chore: bump pre-commit

* style: run pre-commit for pyupgrade 3+

* tests: use sys.version_info, not PY

* chore: more Python 2 removal

* Uncommenting Centos7 block (PR #3691 showed that it is working again).

* Update pre-commit hooks

* Fix pre-commit hook

* refactor: remove Python 2 from CMake

* refactor: remove Python 2 from setup code

* refactor: simplify, better static typing

* feat: fail with nice messages

* refactor: drop Python 2 C++ code

* docs: cleanup for Python 3

* revert: intree

revert: intree

* docs: minor touchup to py2 statement

Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
2022-02-10 18:28:08 -08:00
Ralf W. Grosse-Kunstleve
ec24786eab
Fully-automatic clang-format with include reordering (#3713)
* chore: add clang-format

* Removing check-style (Classic check-style)

Ported from @henryiii's 53056b1b0e

* Automatic clang-format changes (NO manual changes).

Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
2022-02-10 12:17:07 -08:00
Ralf W. Grosse-Kunstleve
e96221beff
Final manual curation in preparation for global clang-formating (#3712)
* Manual line breaks to pre-empt undesired `clang-format`ing.

Informed by work under https://github.com/pybind/pybind11/pull/3683:

60b7eb410f

59572e6559

* Manual curation of clang-format diffs involving source code comments.

Very labor-intensive and dull.

* Pulling .clang-format change from @henryiii's 9057962d40

* Adding commonly used .clang-format `CommentPragmas:`

* Ensure short lambdas are allowed

Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
2022-02-10 11:42:03 -08:00
Aaron Gokaslan
d6c66d25bb
chore(clang-tidy): Add clang-tidy rules: prefer-member-initializer and optin.performance.Padding (#3716)
* Add clang-tidy prefer-member-initializer

* Fix clang-tdy config

* Fix incorrect change

* Fix sorting of .clang-tidy
2022-02-10 09:45:46 -08:00
Aaron Gokaslan
dc9803cef2
Add missing clang-tidy fixes (#3715) 2022-02-10 09:23:15 -08:00
Ralf W. Grosse-Kunstleve
abc38690dc
Manually applying two clang-format changes that need fix-ups for clang-tidy. (#3705) 2022-02-09 06:32:41 -08:00
Ralf W. Grosse-Kunstleve
7769e7719c
clang-tidy readability-qualified-auto (#3702)
* Adding readability-qualified-auto to .clang-tidy

Ported from @henryiii's 287527f705

* fix: support Python < 3.6

Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
2022-02-09 06:24:57 -08:00
Ralf W. Grosse-Kunstleve
ddbc74c674 Adding .clang-tidy readability-braces-around-statements option.
clang-tidy automatic changes. NO manual changes.
2022-02-08 13:02:20 -08:00
Michał Górny
96b943be1d
tests: update catch to 2.13.5 to fix glibc 2.34 failures (#3679)
* Download catch for MinGw

* Fix rest of MinGW

* fix: update catch to 2.13.5 to fix glibc 2.34 failures

Update the downloaded Catch version to 2.13.5, in order to fix build
failure on glibc 2.34:

```
In file included from /usr/include/signal.h:328,
                 from /tmp/pybind11/.nox/tests-3-9/tmp/tests/catch/catch.hpp:8030,
                 from /tmp/pybind11/tests/test_embed/catch.cpp:13:
/tmp/pybind11/.nox/tests-3-9/tmp/tests/catch/catch.hpp:10818:58: error: call to non-‘constexpr’ function ‘long int sysconf(int)’
10818 |     static constexpr std::size_t sigStackSize = 32768 >= MINSIGSTKSZ ? 32768 : MINSIGSTKSZ;
      |                                                          ^~~~~~~~~~~
In file included from /usr/include/python3.9/Python.h:36,
                 from /tmp/pybind11/include/pybind11/detail/common.h:215,
                 from /tmp/pybind11/include/pybind11/pytypes.h:12,
                 from /tmp/pybind11/include/pybind11/cast.h:13,
                 from /tmp/pybind11/include/pybind11/attr.h:13,
                 from /tmp/pybind11/include/pybind11/pybind11.h:13,
                 from /tmp/pybind11/include/pybind11/embed.h:12,
                 from /tmp/pybind11/tests/test_embed/catch.cpp:4:
/usr/include/unistd.h:640:17: note: ‘long int sysconf(int)’ declared here
  640 | extern long int sysconf (int __name) __THROW;
      |                 ^~~~~~~
In file included from /tmp/pybind11/tests/test_embed/catch.cpp:13:
/tmp/pybind11/.nox/tests-3-9/tmp/tests/catch/catch.hpp:10877:45: error: size of array ‘altStackMem’ is not an integral constant-expression
10877 |     char FatalConditionHandler::altStackMem[sigStackSize] = {};
      |                                             ^~~~~~~~~~~~
```

The newest Catch version cannot be used yet because of regression:
https://github.com/catchorg/Catch2/pull/2364

* fix: add option for _ check, only define on pybind11

* Revert "fix: add option for _ check, only define on pybind11"

This reverts commit 86817db488.

* fix: only undef _ for catch cpp creation.

Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
2022-02-05 23:53:02 -05:00
Michał Górny
1d3b04e805
test: Strip whitespace when comparing numpy dtypes for 1.22 compat (#3682)
* test: Strip whitespace when comparing numpy dtypes for 1.22 compat

Strip whitespace when comparing numpy dtype str() in order to preserve
test compatibility with both numpy 1.22 and older versions whose output
differ by whitespace.

Fixes #3680

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2022-02-03 12:29:46 -05:00
Aaron Gokaslan
3a8d92308d
Fix caster optimization regression introduced in #3650 (#3659)
* Fix optimization bug introduced in #3650

* Add simple Python extension test for MVF

* Improve comments

* Clarify comment

* Clarify another comment

* Add test docstring

* Fix typo
2022-01-31 12:19:48 -05:00
Dustin Spicuzza
ec81e8e778
Propagate py::multiple_inheritance to all children (#3650)
* Add tests demonstrating smart_holder issues with multiple inheritance

* Propagate C++ multiple inheritance markers to all children

- Makes py::multiple_inheritance only needed in base classes hidden from pybind11
2022-01-26 17:03:52 -08:00
Sergei Izmailov
5194855900
Render py::bool_ and py::float_ without _ in docstrings (#3622)
* Render `py::bool_` as `bool` in docstrings

* Render `py::float_` as `float` in docstrings
2022-01-16 07:05:46 -08:00
Aaron Gokaslan
d2ec836712
Add support for nested C++11 exceptions (#3608)
* Add support for nested C++11 exceptions

* Remove wrong include

* Fix if directive

* Fix missing skipif

* Simplify code and try to work around MSVC bug

* Clarify comment

* Further simplify code

* Remove the last extra throw statement

* Qualify auto

* Fix typo

* Add missing return for consistency

* Fix clang-tidy complaint

* Fix python2 stub

* Make clang-tidy happy

* Fix compile error

* Fix python2 function signature

* Extract C++20 utility and backport

* Cleanup code a bit more

* Improve test case

* Consolidate code and fix signature

* Fix typo
2022-01-14 14:22:47 -05:00
Aaron Gokaslan
f8d4aa47b6
Add clang-tidy readability checks for sus args (#3611) 2022-01-11 17:34:16 -08:00
Ivor Wanders
21911e126f
A way to register additional test targets and support .py only tests. (#3590)
* A way to register additional test targets.

* Support specifying tests with extension.

* Ensure TEST_OVERRIDE is backwards compatible.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Ensure regex is non greedy.

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2022-01-11 17:31:19 -08:00
Ralf W. Grosse-Kunstleve
f588810871
Replacing ICC C++14 with C++17 (#3570)
* Replacing ICC C++14 with C++17

* ICPC: -diag-disable:conversion

Try to suppress the `-Werror-all` promotion in `#include <variant>`

* Trying `-DPYBIND11_WERROR=OFF`

* Trying `-Wno-conversion` (todo: make specific to C++17 if it works)

* Inserting -Wno-conversion only for C++17

Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja>
2022-01-11 12:13:02 -08:00
Aaron Gokaslan
ef070f7750
Add additional info to TypeError when C++->Python casting fails (#3605)
* Add additional info to TypeInfo when C++->Python casting fails

* Fix typo

* Address reviewer comments
2022-01-10 21:18:00 -05:00
Ralf W. Grosse-Kunstleve
1bbaeb3462
Adding dedicated test_const_name. (#3578)
* Adding dedicated test_const_name.

Also exercises pybind11::detail::_ backward compatibility.

See also: PR #3423

* Backing out tests involving int_to_str (requires C++17 or higher).

* Suppressing clang-tidy errors.

* Disabling test_const_name for MSVC 2015 due to bizarre failures.

* Stacking @pytest.mark.parametrize (thanks to @skylion007 for pointing out).
2021-12-29 12:54:25 -08:00
Henry Schreiner
39fbc7992b
fix: avoiding usage of _ if already defined (#3423)
* fix: avoid usage of _

* ci: test _ defined

* docs: include change in docs

* fix: add a test and comment

* refactor: const_str -> const_name
2021-12-21 14:24:21 -05:00
Jason Rhinelander
b4939fcbfb
Expand std::string_view support to str, bytes, memoryview (#3521)
* Expand string_view support to str, bytes, memoryview

1. Allows constructing a str or bytes implicitly from a string_view;
   this is essentially a small shortcut allowing a caller to write
   `py::bytes{sv}` rather than `py::bytes{sv.data(), sv.size()}`.

2. Allows implicit conversion *to* string_view from py::bytes -- this
   saves a fair bit more as currently there is no simple way to get such
   a view of the bytes without copying it (or resorting to Python API
   calls).

   (This is not done for `str` because when the str contains unicode we
   have to allocate to a temporary and so there might not be some string
   data we can properly view without owning.)

3. Allows `memoryview::from_memory` to accept a string_view.  As with
   the other from_memory calls, it's entirely your responsibility to
   keep it alive.

This also required moving the string_view availability detection into
detail/common.h because this PR needs it in pytypes.h, which is higher
up the include chain than cast.h where it was being detected currently.

* Move string_view include to pytypes.h

* CI-testing a fix for the "ambiguous conversion" issue.

This change is known to fix the `tensorflow::tstring` issue reported under https://github.com/pybind/pybind11/pull/3521#issuecomment-985100965

TODO: Minimal reproducer for the `tensorflow::tstring` issue.

* Make clang-tidy happy (hopefully).

* Adding minimal reproducer for the `tensorflow::tstring` issue.

Error without the enable_if trick:

```
/usr/local/google/home/rwgk/forked/pybind11/tests/test_builtin_casters.cpp:169:16: error: ambiguous conversion for functional-style cast from 'TypeWithBothOperatorStringAndStringView' to 'py::bytes'
        return py::bytes(TypeWithBothOperatorStringAndStringView());
               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/google/home/rwgk/forked/pybind11/include/pybind11/detail/../pytypes.h:1174:5: note: candidate constructor
    bytes(const std::string &s) : bytes(s.data(), s.size()) { }
    ^
/usr/local/google/home/rwgk/forked/pybind11/include/pybind11/detail/../pytypes.h:1191:5: note: candidate constructor
    bytes(std::string_view s) : bytes(s.data(), s.size()) { }
    ^
```

* Adding missing NOLINTNEXTLINE

* Also apply ambiguous conversion workaround to str()

Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
2021-12-03 13:20:32 -04:00
Ralf W. Grosse-Kunstleve
9281faf429
Fixing stict vs strict typo. (#3493) 2021-11-22 01:33:03 -08:00
Jason Rhinelander
673b4be3d7
Fix py::kw_only when used before the first arg of a method (#3488)
* Fix py::kw_only when used before the first arg of a method

The implicit space for the `self` argument isn't added until we hit the
first argument, but this wasn't being done for kw_only or pos_only, and
so a kw_only before the first argument would break.

This fixes it by properly checking whether we need to add the self arg.

(The pos_only issue here was extremely mild -- you didn't get the `/` in
the docstring, but AFAICT it has no other effect since there are no
meaningful arguments before it anyway).

* Style changes

- rename check_have_self_arg -> append_self_arg_if_needed

- move the argument name inline comments before the args instead of
  after
2021-11-20 16:01:57 -08:00
Henry Schreiner
15f8d7c12e
fix(build): cleaner CMake printouts & IDE folders (#3479) 2021-11-18 10:01:24 -05:00
Henry Schreiner
72282f75a1
ci: support development releases of Python (#3419)
* ci: support development releases of Python

* fix: better PyPy support

* fix: patch over a few more pypy issues

* Try to patch

* Properly follow pep667

* Fix typo

* Whoops, 667 not in yet

* For testing

* More testing

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Try to backport

* Try to simplify fix

* Nail down the fix

* Try pypy workaround

* Typo

* one last typo

* Replacing 0x03110000 with 0x030B0000

* Add TODO. Drop PyPy

* Fix typo

* Revert catch upgrade

* fix: minor cleanup, try pypy again

Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
2021-11-17 09:44:19 -05:00
Trigve
afdc09deda
[master] Wrong caching of overrides (#3465)
* override: Fix wrong caching of the overrides

There was a problem when the python type, which was stored in override
cache for C++ functions, was destroyed and  the record wasn't removed from the
override cache. Therefor, dangling pointer was stored there. Then when the
memory was reused and new type was allocated at the given address and the
method with the same name (as previously stored in the cache) was actually
overridden in python, it would wrongly find it in the override cache for C++
functions and therefor override from python wouldn't be called.
The fix is to erase the type from the override cache when the type is destroyed.

* test: Pass by const ref instead of by value (clang-tidy)

* test: Rename classes and move to different files

Rename the classes and files so they're no too generic. Also, better place to
test the stuff is in test_virtual_functions.cpp/.py as we're basically testing
the virtual functions/trampolines.

* Add TODO for erasure code

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-11-15 13:36:41 -05:00
Antony Lee
b11ff912a6
fix(setup =_helpers): don't add -g0 CFLAGS sets -g (#3436)
On Unix, setuptools prepends $CFLAGS and $CPPFLAGS to the compiler flags
(they always come before extra_compile_args and anything else; see
distutils.sysconfig.customize_compiler).  In practice, the environment
variables are useful e.g. to quickly generate a debug build (e.g. by
setting CFLAGS=-g), but Pybind11Extension currently unconditionally
overwrites this with -g0.

Instead, check the environment variables and only insert -g0 if not
overridden by them.
2021-11-08 16:27:32 -05:00
Boris Rasin
01f938e799
fix: add missing std::forward calls (#3443)
* fix: add missing std::forward calls

Two of the four cpp_function overloads are missing std::forward calls, which seems like a simple oversight.

* add test for https://github.com/pybind/pybind11/pull/3443

* add py tests

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix test

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-11-07 18:35:25 -05:00
Jason Rhinelander
e7c9753f1d
feat: allow kw-only args after a py::args (#3402)
* Simply has_kw_only_args handling

This simplifies tracking the number of kw-only args by instead tracking
the number of positional arguments (which is really what we care about
everywhere this is used).

* Allow keyword-only arguments to follow py::args

This removes the constraint that py::args has to be last (or
second-last, with py::kwargs) and instead makes py::args imply
py::kw_only for any remaining arguments, allowing you to bind a function
that works the same way as a Python function such as:

    def f(a, *args, b):
        return a * b + sum(args)

    f(10, 1, 2, 3, b=20)  # == 206

With this change, you can bind such a function using:

    m.def("f", [](int a, py::args args, int b) { /* ... */ },
        "a"_a, "b"_a);

Or, to be more explicit about the keyword-only arguments:

    m.def("g", [](int a, py::args args, int b) { /* ... */ },
        "a"_a, py::kw_only{}, "b"_a);

(The only difference between the two is that the latter will fail at
binding time if the `kw_only{}` doesn't match the `py::args` position).

This doesn't affect backwards compatibility at all because, currently,
you can't have a py::args anywhere except the end/2nd-last.

* Take args/kwargs by const lvalue ref

Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>

Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
2021-10-28 23:16:55 -04:00
Henry Schreiner
e7e2c79f3f
fix: improve support for Python 3.11-dev (#3368)
* ci: support Python 3.11-dev

Also update 3.10 to final, better PyPy usage

* fix: use PyFrame_GetCode on Python 3.9+

* ci: some bitiness of pypy not supported on win

* chore: update CMake support to 3.22rc1 to quiet warning

* fix: use dev version of py to fix Py 3.11

* tests: print proper Eigen version

* ci: include pypy2, not sure why

* ci: avoid running on Python 3.11 for now

* ci: fix runs

* ci: simpler PyPy usage, drop unmaintained scipy + pypy index

* ci: only binary numpy, wait on pypy 3.8

* refactor: address review
2021-10-26 14:50:34 -04:00
Ryan Cahoon
c2d3e220bd
fix: the types for return_value_policy_override in optional_caster (#3376)
* fix: the types for return_value_policy_override in optional_caster

`return_value_policy_override` was not being applied correctly in
`optional_caster` in two ways:
- The `is_lvalue_reference` condition referenced `T`, which was the
`optional<T>` type parameter from the class, when it should have used `T_`,
which was the parameter to the `cast` function. `T_` can potentially be a
reference type, but `T` will never be.
- The type parameter passed to `return_value_policy_override` should be
`T::value_type`, not `T`. This matches the way that the other STL container
type casters work.

The result of these issues was that a method/property definition which used a
`reference` or `reference_internal` return value policy would create a Python
value that's bound by reference to a temporary C++ object, resulting in
undefined behavior. For reasons that I was not able to figure out fully, it
seems like this causes problems when using old versions of `boost::optional`,
but not with recent versions of `boost::optional` or the `libstdc++`
implementation of `std::optional`. The issue (that the override to
`return_value_policy::move` is never being applied) is present for all
implementations, it just seems like that somehow doesn't result in problems for
the some implementation of `optional`. This change includes a regression type
with a custom optional-like type which was able to reproduce the issue.

Part of the issue with using the wrong types may have stemmed from the type
variables `T` and `T_` having very similar names. This also changes the type
variables in `optional_caster` to use slightly more descriptive names, which
also more closely follow the naming convention used by the other STL casters.

Fixes #3330

* Fix clang-tidy complaints

* Add missing NOLINT

* Apply a couple more fixes

* fix: support GCC 4.8

* tests: avoid warning about unknown compiler for compilers missing C++17

* Remove unneeded test module attribute

* Change test enum to have more unique int values

Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
2021-10-25 19:04:45 -07:00
Dmitry Yershov
076c89fc54
tests: test recursive dispatch using visitor pattern (#3365) 2021-10-22 17:09:15 -04:00
Henry Schreiner
f791dc8648
fix: deprecate make_simple_namespace, fix Python 3.11 (#3374)
* fix: deprecate make_simple_namespace, fix Python 3.11

* docs: update links
2021-10-19 14:39:29 -04:00
Ralf W. Grosse-Kunstleve
7c580586f8
Correct options on Eigen::MappedSparseMatrix & adding MSVC C4127 suppression around Eigen includes. (#3352)
* Adding MSVC C4127 suppression around Eigen includes.

* For MSVC 2015 only: also adding the C4127 suppression to test_eigen.cpp

* Copying original change from PR #3343, with extra line breaks to not run past 99 columns (our desired but currently not enforced limit).
2021-10-11 13:13:01 -07:00
Bruce Merry
8a7c266d26
Fix make_key_iterator/make_value_iterator for prvalue iterators (#3348)
* Add a test showing a flaw in make_key_iterator/make_value_iterator

If the iterator dereference operator returns a value rather than a
reference (and that pair also does not *contain* references),
make_key_iterator and make_value_iterator will return a reference to a
temporary, causing a segfault.

* Fix make_key_iterator/make_value_iterator for prvalue iterators

If an iterator returns a pair<T1, T2> rather than a reference to a pair
or a pair of references, make_key_iterator and make_value_iterator would
return a reference to a temporary, typically leading to a segfault. This
is because the value category of member access to a prvalue is an
xvalue, not a prvalue, so decltype produces an rvalue reference type.
Fix the type calculation to handle this case.

I also removed some decltype parentheses that weren't needed, either
because the expression isn't one of the special cases for decltype or
because decltype was only used for SFINAE. Hopefully that makes the code
a bit more readable.

Closes #3347

* Attempt a workaround for nvcc
2021-10-11 08:35:39 -07:00
Aaron Gokaslan
f4c81e0877
maint: Add additional linter-related pre-commit hooks (#3337)
* Add additional pygrep pre-commit hooks

* Remove useless noqas with hook

* Fix all single rst backticks

* Simplify mypy pre-commit hook with upstream fixes

* Add back missing comment

* Add one last pygrep hook
2021-10-08 08:38:04 -04:00
Henry Schreiner
d7a7edc12b tests: support Eigen configuration 2021-10-04 10:06:12 -07:00
Aaron Gokaslan
ad96655605
fix: replace free() with std::free() (#3321)
* Disambiguate free() to use std::free()

* Add cstdlib include
2021-10-03 20:15:37 -04:00
Bruce Merry
b3573ac961
feat: add .keys and .values to bind_map (#3310)
* Add `.keys` and `.values` to bind_map

Both of these implement views (rather than just iterators), and `.items`
is also upgraded to a view. In practical terms, this allows a view to be
iterated multiple times and have its size taken, neither of which works
with an iterator.

The views implement `__len__`, `__iter__`, and the keys view implements
`__contains__`. Testing membership also works in item and value views
because Python falls back to iteration. This won't be optimal
for item values since it's linear rather than O(log n) or O(1), but I
didn't fancy trying to get all the corner cases to match Python
behaviour (tuple of wrong types, wrong length tuple, not a tuple etc).

Missing relative to Python dictionary views is `__reversed__` (only
added to Python in 3.8). Implementing that could break code that binds
custom map classes which don't provide `rbegin`/`rend` (at least without
doing clever things with SFINAE), so I've not tried.

The size increase on my system is 131072 bytes, which is rather large
(5%) but also suspiciously round (2^17) and makes me suspect some
quantisation effect.

* bind_map: support any object in __contains__

Add extra overload of `__contains__` (for both the map itself and
KeysView) which takes an arbitrary object and returns false.

* Take py::object by const reference in __contains__

To keep clang-tidy happy.

* Removing stray `py::` (detected via interactive testing in Google environment).

Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
2021-10-01 09:24:36 -04:00
Jeremy Maitin-Shepard
62c4909cce
Add custom_type_setup attribute (#3287)
* Fix `pybind11::object::operator=` to be safe if `*this` is accessible from Python

* Add `custom_type_setup` attribute

This allows for custom modifications to the PyHeapTypeObject prior to
calling `PyType_Ready`.  This may be used, for example, to define
`tp_traverse` and `tp_clear` functions.
2021-09-24 12:08:22 -07:00
Henry Schreiner
21282e645a
feat: reapply fixed version of #3271 (#3293)
* Add make_value_iterator (#3271)

* Add make_value_iterator

This is the counterpart to make_key_iterator, and will allow
implementing a `value` method in `bind_map` (although doing so is left
for a subsequent PR).

I made a few design changes to reduce copy-and-paste boilerplate.
Previously detail::iterator_state had a boolean template parameter to
indicate whether it was being used for make_iterator or
make_key_iterator. I replaced the boolean with a class that determines
how to dereference the iterator. This allows for a generic
implementation of `__next__`.

I also added the ValueType and Extra... parameters to the iterator_state
template args, because I think it was a bug that they were missing: if
make_iterator is called twice with different values of these, only the
first set has effect (because the state class is only registered once).
There is still a potential issue in that the *values* of the extra
arguments are latched on the first call, but since most policies are
empty classes this should be even less common.

* Add some remove_cv_t to appease clang-tidy

* Make iterator_access and friends take reference

For some reason I'd accidentally made it take a const value, which
caused some issues with third-party packages.

* Another attempt to remove remove_cv_t from iterators

Some of the return types were const (non-reference) types because of the
pecularities of decltype: `decltype((*it).first)` is the *declared* type
of the member of the pair, rather than the type of the expression. So if
the reference type of the iterator is `pair<const int, int> &`, then the
decltype is `const int`. Wrapping an extra set of parentheses to form
`decltype(((*it).first))` would instead give `const int &`.

This means that the existing make_key_iterator actually returns by value
from `__next__`, rather than by reference. Since for mapping types, keys
are always const, this probably hasn't been noticed, but it will affect
make_value_iterator if the Python code tries to mutate the returned
objects. I've changed things to use double parentheses so that
make_iterator, make_key_iterator and make_value_iterator should now all
return the reference type of the iterator. I'll still need to add a test
for that; for now I'm just checking whether I can keep Clang-Tidy happy.

* Add back some NOLINTNEXTLINE to appease Clang-Tidy

This is favoured over using remove_cv_t because in some cases a const
value return type is deliberate (particularly for Eigen).

* Add a unit test for iterator referencing

Ensure that make_iterator, make_key_iterator and make_value_iterator
return references to the container elements, rather than copies. The
test for make_key_iterator fails to compile on master, which gives me
confidence that this branch has fixed it.

* Make the iterator_access etc operator() const

I'm actually a little surprised it compiled at all given that the
operator() is called on a temporary, but I don't claim to fully
understand all the different value types in C++11.

* Attempt to work around compiler bugs

https://godbolt.org/ shows an example where ICC gets the wrong result
for a decltype used as the default for a template argument, and CI also
showed problems with PGI. This is a shot in the dark to see if it fixes
things.

* Make a test constructor explicit (Clang-Tidy)

* Fix unit test on GCC 4.8.5

It seems to require the arguments to the std::pair constructor to be
implicitly convertible to the types in the pair, rather than just
requiring is_constructible.

* Remove DOXYGEN_SHOULD_SKIP_THIS guards

Now that a complex decltype expression has been replaced by a simpler
nested type, I'm hoping Doxygen will be able to build it without issues.

* Add comment to explain iterator_state template params

* fix: regression in #3271

Co-authored-by: Bruce Merry <1963944+bmerry@users.noreply.github.com>
2021-09-23 15:06:07 -04:00
Henry Schreiner
5f46e47da8
tests: check simple iteration of pairs (#3296) 2021-09-23 08:01:06 -04:00
Henry Schreiner
2fa3fcfda5 Revert "Add make_value_iterator (#3271)"
This reverts commit ee0c5ee405.
2021-09-22 23:10:03 -04:00
Henry Schreiner
b06a6f4f62
feat: Slice allowing None with py::object or std::optional (#1101)
* Adding nullptr slices

Using example from #1095

Some fixes from @wjakob's review

Stop clang-tidy from complaining

New proposal for py::slice constructor

Eric's suggested changes: simplify testing; shift def's

* chore: drop MSVC pragma (hopefully unneeded)

* Apply suggestions from code review
2021-09-22 17:41:56 -04:00
Bruce Merry
ee0c5ee405
Add make_value_iterator (#3271)
* Add make_value_iterator

This is the counterpart to make_key_iterator, and will allow
implementing a `value` method in `bind_map` (although doing so is left
for a subsequent PR).

I made a few design changes to reduce copy-and-paste boilerplate.
Previously detail::iterator_state had a boolean template parameter to
indicate whether it was being used for make_iterator or
make_key_iterator. I replaced the boolean with a class that determines
how to dereference the iterator. This allows for a generic
implementation of `__next__`.

I also added the ValueType and Extra... parameters to the iterator_state
template args, because I think it was a bug that they were missing: if
make_iterator is called twice with different values of these, only the
first set has effect (because the state class is only registered once).
There is still a potential issue in that the *values* of the extra
arguments are latched on the first call, but since most policies are
empty classes this should be even less common.

* Add some remove_cv_t to appease clang-tidy

* Make iterator_access and friends take reference

For some reason I'd accidentally made it take a const value, which
caused some issues with third-party packages.

* Another attempt to remove remove_cv_t from iterators

Some of the return types were const (non-reference) types because of the
pecularities of decltype: `decltype((*it).first)` is the *declared* type
of the member of the pair, rather than the type of the expression. So if
the reference type of the iterator is `pair<const int, int> &`, then the
decltype is `const int`. Wrapping an extra set of parentheses to form
`decltype(((*it).first))` would instead give `const int &`.

This means that the existing make_key_iterator actually returns by value
from `__next__`, rather than by reference. Since for mapping types, keys
are always const, this probably hasn't been noticed, but it will affect
make_value_iterator if the Python code tries to mutate the returned
objects. I've changed things to use double parentheses so that
make_iterator, make_key_iterator and make_value_iterator should now all
return the reference type of the iterator. I'll still need to add a test
for that; for now I'm just checking whether I can keep Clang-Tidy happy.

* Add back some NOLINTNEXTLINE to appease Clang-Tidy

This is favoured over using remove_cv_t because in some cases a const
value return type is deliberate (particularly for Eigen).

* Add a unit test for iterator referencing

Ensure that make_iterator, make_key_iterator and make_value_iterator
return references to the container elements, rather than copies. The
test for make_key_iterator fails to compile on master, which gives me
confidence that this branch has fixed it.

* Make the iterator_access etc operator() const

I'm actually a little surprised it compiled at all given that the
operator() is called on a temporary, but I don't claim to fully
understand all the different value types in C++11.

* Attempt to work around compiler bugs

https://godbolt.org/ shows an example where ICC gets the wrong result
for a decltype used as the default for a template argument, and CI also
showed problems with PGI. This is a shot in the dark to see if it fixes
things.

* Make a test constructor explicit (Clang-Tidy)

* Fix unit test on GCC 4.8.5

It seems to require the arguments to the std::pair constructor to be
implicitly convertible to the types in the pair, rather than just
requiring is_constructible.

* Remove DOXYGEN_SHOULD_SKIP_THIS guards

Now that a complex decltype expression has been replaced by a simpler
nested type, I'm hoping Doxygen will be able to build it without issues.

* Add comment to explain iterator_state template params
2021-09-21 13:37:19 -04:00
Aaron Gokaslan
6e6975e217
Fix test case with __new__ (#3285) 2021-09-20 16:03:21 -04:00
Aaron Gokaslan
d0f3c51f01
Enable defining custom __new__ (#3265)
* Enable defining custom __new__

* See if xfail needed

* Qualify auto self

* Unconditionally defining PYBIND11_DISABLE_NEW_STYLE_INIT_WARNING. Returning pointer from "__init__" instead of reference.

* Use new style __init__

* Simplify __new__ creation

* Reviewer suggestions

* Match indentation

Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
2021-09-20 10:42:14 -04:00
Jeremy Maitin-Shepard
14976c853b
Eliminate duplicate TLS keys for loader_life_support stack (#3275)
* Eliminate duplicate TLS keys for loader_life_support stack

This revises the existing fix for
https://github.com/pybind/pybind11/issues/2765 in
https://github.com/pybind/pybind11/pull/3237 to reduce the amount of
TLS storage used.

The shared TLS key is stored in two different ways, depending on
`PYBIND11_INTERNALS_VERSION`.  If `PYBIND11_INTERNALS_VERSION ==
4` (as is currently set), the TLS key is stored in the
`internal::shared_data` map to avoid breaking ABI compatibility.  If
`PYBIND11_INTERNALS_VERSION > 4`, the TLS key is stored directly in
the `internals` struct.

* Fix test_pytypes.py::test_issue2361 failure on PyPy3.7

* Add github actions tests for unstable ABI
2021-09-20 04:57:38 -07:00
Ralf W. Grosse-Kunstleve
6c65ab5950
Follow-on to PR #3254, to address user code breakages. (#3263)
* Restoring `const` removed from pytypes.h in PR #3254, adding tests reflective of user code that breaks when those `const` are removed.

* clang-tidy NOLINTs (and one collateral fix).

* Inserting PYBIND11_CONST_FOR_STRICT_PLATFORMS

* Trying `defined(__APPLE__)`

* Trying again: `auto it` for strict platforms.

* Adding NOLINTNEXTLINE(bugprone-macro-parentheses), expanding comments.

* Labeling all changes with `PR #3263`, for easy reference, and to make it easy to undo these changes if we decide to do so in the future.
2021-09-12 19:53:26 -07:00