Commit Graph

1479 Commits

Author SHA1 Message Date
Ralf W. Grosse-Kunstleve 1d81191077
Disable implicit conversion of `0` to `pybind11::handle`. (#4008)
* Disable implicit conversion from `0` to `pybind11::handle`.

* Reverse or-ed condition in an attempt to resolve GCC 8.3.0 errors (i386/debian:buster).

* Trying the simpler `std::is_same<T, PyObject *>`

* Add implicit_conversion_from_pytorch_THPObjectPtr_to_handle test.

* Accommodate types with implicit conversions to `PyObject *`, other than `handle` & `handle` subclasses, or integral types.

* Fix copy-paste mishap (picked wrong name).

* Revamp SFINAE construct to actually fix the pytorch issue (already validated against pytorch proper).

The first version of the reduced pytorch code was critically missing the move ctor. The first version of the accompanying test was meaningless.

Note: It turns out the `!std::is_arithmetic<T>` condition is not needed: `int` is not in general implicitly convertible to `PyObject *`, only the literal `0` is.

* Use `NOLINT(performance-noexcept-move-constructor)` for reduced code from the wild (rather than changing the code).

* Use any_of, all_of, negation. It turns out to clang-format nicer.

* Clean up comments for changed code.

* Reduce pytorch situation further, add test for operator ... const.

* Use `none_of` as suggested by @skylion007

* Add `pure_compile_tests_for_handle_from_PyObject_pointers()`

* Fix inconsequential oversight (retested).

* Factor our `is_pyobj_ptr_or_nullptr_t` to make the SFINAE conditions more readable.

* Remove stray line (oversight).

* Make the `pure_compile_tests_for_handle_from_PyObject_pointers()` "rhs-const-complete", too.

* Remove the temporary PYBIND11_UNDO_PR4008 `#ifdef`.
2022-07-14 09:53:39 -07:00
Aaron Gokaslan bc9315fe38
chore: optimize sparse matrix casting with python tuple (#4064)
* optimize casting of sparse Eigen arrays

* move array

* Revert for safety
2022-07-13 19:27:00 -04:00
Ralf W. Grosse-Kunstleve 023b3f32c8
Undo accidental one-line change under PR #3913 (#4060) 2022-07-12 12:02:20 -07:00
Aaron Gokaslan f9f00495a3
Properly visit self in >=3.9 traverse (#4051)
* Properly visit self in >=3.9 traverse

* Add comment about 3.9 behavior
2022-07-09 21:43:53 -07:00
Ralf W. Grosse-Kunstleve 432bc5cfb5
Add `std::string clean_type_id(const char *typeid_name)` overload (in namespace detail). (#4049)
Very minor refactoring to ease development and debugging.

Having to declare a local `std::string` has bugged me many times. Nice to get this little nuisance out of the way.

Extracted from PR #4022, where it is used like this:

```
    std::fprintf(stdout,
                 "\nTYPE_CASTER_ODR_GUARD_IMPL %s %s\n",
                 clean_type_id(intrinsic_type_info.name()).c_str(),
                 source_file_line_from_sloc.c_str());
```
2022-07-07 19:08:10 -07:00
Ralf W. Grosse-Kunstleve 85bc088441
Report `C++ Info:` via `pytest_report_header()` (#4046)
* Report `C++ Info:` from `pytest_configure()`

* Use pytest_report_header() as suggested by @skylion007
2022-07-07 17:51:44 -07:00
Ralf W. Grosse-Kunstleve cd08869df1
PYBIND11_NAMESPACE consistency fixes. (#4043) 2022-07-06 14:29:20 -07:00
Aaron Gokaslan 2af163d9c7
Fix: 3.11 beta support (#3923)
* Placeholder commit for 3.11 testing

* Does this fix it?

* Try suggestion

* Placeholder commit for 3.11 testing

* Does this fix it?

* Try suggestion

* fix: try using modern init for embedded interp

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* fix: error message changed in 3.11

* fix: apply logic in Python manually

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* fix autodetect dynamic attrs in 3.11

* fix: include error message if possible in error

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* ci: enable standard Python 3.11 testing

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* Make dynamic attrs condtiion exclusive to ver.

Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
2022-07-06 16:35:12 -04:00
Matthew Woehlke 479e9a50f3
Fix arrays with zero-size dimensions (#4038)
When converting an array to an Eigen matrix, ignore the strides if any
dimension size is 0. If the array is empty, the strides aren't relevant,
and especially numpy ≥ 1.23 explicitly sets the strides to 0 in this
case. (See numpy commit dd5ab7b11520.)

Update tests to verify that this works, and continues to work.
2022-06-29 11:37:16 -07:00
Aaron Gokaslan 0964a9093a
Add a missing std::move in numpy.h (#4005) 2022-06-14 14:20:26 -04:00
Aaron Gokaslan e2dcd95407
chore: optimize dictionary access in strip_padding numpy (#3994)
* emplace field descriptors

* reserve sufficient capacity

* remove std::move

* properly iterate through dict

* make handle casting more explicit

* Revert to old dict api
2022-06-06 22:33:28 -04:00
Maarten Baert 918892b97e
Make dtype::num() return type consistent with other functions (#3995) 2022-06-06 19:41:38 -04:00
Aaron Gokaslan 42a41bf3e7
remove useless ctor (#3989) 2022-06-03 15:18:06 -04:00
Sergei Lebedev a05bc3d235
error_already_set::what() is now constructed lazily (#1895)
* error_already_set::what() is now constructed lazily

Prior to this commit throwing error_already_set was expensive due to the
eager construction of the error string (which required traversing the
Python stack). See #1853 for more context and an alternative take on the
issue.

Note that error_already_set no longer inherits from std::runtime_error
because the latter has no default constructor.

* Do not attempt to normalize if no exception occurred

This is not supported on PyPy-2.7 5.8.0.

* Extract exception name via tp_name

This is faster than dynamically looking up __name__ via GetAttrString.
Note though that the runtime of the code throwing an error_already_set
will be dominated by stack unwinding so the improvement will not be
noticeable.

Before:

396 ns ± 0.913 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

After:

277 ns ± 0.549 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

Benchmark:

const std::string foo() {
    PyErr_SetString(PyExc_KeyError, "");
    const std::string &s = py::detail::error_string();
    PyErr_Clear();
    return s;
}

PYBIND11_MODULE(foo, m) {
    m.def("foo", &::foo);
}

* Reverted error_already_set to subclass std::runtime_error

* Revert "Extract exception name via tp_name"

The implementation of __name__ is slightly more complex than that.
It handles the module name prefix, and heap-allocated types. We could
port it to pybind11 later on but for now it seems like an overkill.

This reverts commit f1435c7e6b.

* Cosmit following @YannickJadoul's comments

Note that detail::error_string() no longer calls PyException_SetTraceback
as it is unncessary for pretty-printing the exception.

* Fixed PyPy build

* Moved normalization to error_already_set ctor

* Fix merge bugs

* Fix more merge errors

* Improve formatting

* Improve error message in rare case

* Revert back if statements

* Fix clang-tidy

* Try removing mutable

* Does build_mode release fix it

* Set to Debug to expose segfault

* Fix remove set error string

* Do not run error_string() more than once

* Trying setting the tracebackk to the value

* guard if m_type is null

* Try to debug PGI

* One last try for PGI

* Does reverting this fix PyPy

* Reviewer suggestions

* Remove unnecessary initialization

* Add noexcept move and explicit fail throw

* Optimize error_string creation

* Fix typo

* Revert noexcept

* Fix merge conflict error

* Abuse assignment operator

* Revert operator abuse

* See if we still need debug

* Remove unnecessary mutable

* Report "FATAL failure building pybind11::error_already_set error_string" and terminate process.

* Try specifying noexcept again

* Try explicit ctor

* default ctor is noexcept too

* Apply reviewer suggestions, simplify code, and make helper method private

* Remove unnecessary include

* Clang-Tidy fix

* detail::obj_class_name(), fprintf with [STDERR], [STDOUT] tags, polish comments

* consistently check m_lazy_what.empty() also in production builds

* Make a comment slightly less ambiguous.

* Bug fix: Remove `what();` from `restore()`.

It sure would need to be guarded by `if (m_type)`, otherwise `what()` fails and masks that no error was set (see update unit test). But since `error_already_set` is copyable, there is no point in releasing m_type, m_value, m_trace, therefore we can just as well avoid the runtime overhead of force-building `m_lazy_what`, it may never be used.

* Replace extremely opaque (unhelpful) error message with a truthful reflection of what we know.

* Fix clang-tidy error [performance-move-constructor-init].

* Make expected error message less specific.

* Various changes.

* bug fix: error_string(PyObject **, ...)

* Putting back the two eager PyErr_NormalizeException() calls.

* Change error_already_set() to call pybind11_fail() if the Python error indicator not set. The net result is that a std::runtime_error is thrown instead of error_already_set, but all tests pass as is.

* Remove mutable (fixes oversight in the previous commit).

* Normalize the exception only locally in error_string(). Python 3.6 & 3.7 test failures expected. This is meant for benchmarking, to determine if it is worth the trouble looking into the failures.

* clang-tidy: use auto

* Use `gil_scoped_acquire_local` in `error_already_set` destructor. See long comment.

* For Python < 3.8: `PyErr_NormalizeException` before `PyErr_WriteUnraisable`

* Go back to replacing the held Python exception with then normalized exception, if & when needed. Consistently document the side-effect.

* Slightly rewording comment. (There were also other failures.)

* Add 1-line comment for obj_class_name()

* Benchmark code, with results in this commit message.

          function                   #calls  test time [s]  μs / call
master    pure_unwind                729540      1.061      14.539876
          err_set_unwind_err_clear   681476      1.040      15.260282
          err_set_error_already_set  508038      1.049      20.640525
          error_already_set_restore  555578      1.052      18.933288
          pr1895_original_foo        244113      1.050      43.018168
                                                                       PR / master
PR #1895  pure_unwind                736981      1.054      14.295685       98.32%
          err_set_unwind_err_clear   685820      1.045      15.237399       99.85%
          err_set_error_already_set  661374      1.046      15.811879       76.61%
          error_already_set_restore  669881      1.048      15.645176       82.63%
          pr1895_original_foo        318243      1.059      33.290806       77.39%

master @ commit ad146b2a18

Running tests in directory "/usr/local/google/home/rwgk/forked/pybind11/tests":
============================= test session starts ==============================
platform linux -- Python 3.9.10, pytest-6.2.3, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /usr/local/google/home/rwgk/forked/pybind11/tests, configfile: pytest.ini
collecting ... collected 5 items

test_perf_error_already_set.py::test_perf[pure_unwind]
PERF pure_unwind,729540,1.061,14.539876
PASSED
test_perf_error_already_set.py::test_perf[err_set_unwind_err_clear]
PERF err_set_unwind_err_clear,681476,1.040,15.260282
PASSED
test_perf_error_already_set.py::test_perf[err_set_error_already_set]
PERF err_set_error_already_set,508038,1.049,20.640525
PASSED
test_perf_error_already_set.py::test_perf[error_already_set_restore]
PERF error_already_set_restore,555578,1.052,18.933288
PASSED
test_perf_error_already_set.py::test_perf[pr1895_original_foo]
PERF pr1895_original_foo,244113,1.050,43.018168
PASSED

============================== 5 passed in 12.38s ==============================

pr1895 @ commit 8dff51d12e

Running tests in directory "/usr/local/google/home/rwgk/forked/pybind11/tests":
============================= test session starts ==============================
platform linux -- Python 3.9.10, pytest-6.2.3, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /usr/local/google/home/rwgk/forked/pybind11/tests, configfile: pytest.ini
collecting ... collected 5 items

test_perf_error_already_set.py::test_perf[pure_unwind]
PERF pure_unwind,736981,1.054,14.295685
PASSED
test_perf_error_already_set.py::test_perf[err_set_unwind_err_clear]
PERF err_set_unwind_err_clear,685820,1.045,15.237399
PASSED
test_perf_error_already_set.py::test_perf[err_set_error_already_set]
PERF err_set_error_already_set,661374,1.046,15.811879
PASSED
test_perf_error_already_set.py::test_perf[error_already_set_restore]
PERF error_already_set_restore,669881,1.048,15.645176
PASSED
test_perf_error_already_set.py::test_perf[pr1895_original_foo]
PERF pr1895_original_foo,318243,1.059,33.290806
PASSED

============================== 5 passed in 12.40s ==============================

clang++ -o pybind11/tests/test_perf_error_already_set.os -c -std=c++17 -fPIC -fvisibility=hidden -Os -flto -Wall -Wextra -Wconversion -Wcast-qual -Wdeprecated -Wnon-virtual-dtor -Wunused-result -isystem /usr/include/python3.9 -isystem /usr/include/eigen3 -DPYBIND11_STRICT_ASSERTS_CLASS_HOLDER_VS_TYPE_CASTER_MIX -DPYBIND11_TEST_BOOST -Ipybind11/include -I/usr/local/google/home/rwgk/forked/pybind11/include -I/usr/local/google/home/rwgk/clone/pybind11/include /usr/local/google/home/rwgk/forked/pybind11/tests/test_perf_error_already_set.cpp

clang++ -o lib/pybind11_tests.so -shared -fPIC -Os -flto -shared ...

Debian clang version 13.0.1-3+build2
Target: x86_64-pc-linux-gnu
Thread model: posix

* Changing call_repetitions_target_elapsed_secs to 0.1 for regular unit testing.

* Adding in `recursion_depth`

* Optimized ctor

* Fix silly bug in recurse_first_then_call()

* Add tests that have equivalent PyErr_Fetch(), PyErr_Restore() but no try-catch.

* Add call_error_string to tests. Sample only recursion_depth 0, 100.

* Show lazy-what speed-up in percent.

* Include real_work in benchmarks.

* Replace all PyErr_SetString() with generate_python_exception_with_traceback()

* Better organization of test loops.

* Add test_error_already_set_copy_move

* Fix bug in newly added test (discovered by clang-tidy): actually use move ctor

* MSVC detects the unreachable return

* change test_perf_error_already_set.py back to quick mode

* Inherit from std::exception (instead of std::runtime_error, which does not make sense anymore with the lazy what)

* Special handling under Windows.

* print with leading newline

* Removing test_perf_error_already_set (copies are under 7765113fbb).

* Avoid gil and scope overhead if there is nothing to release.

* Restore default move ctor. "member function" instead of "function" (note that "method" is Python terminology).

* Delete error_already_set copy ctor.

* Make restore() non-const again to resolve clang-tidy failure (still experimenting).

* Bring back error_already_set copy ctor, to see if that resolves the 4 MSVC test failures.

* Add noexcept to error_already_set copy & move ctors (as suggested by @skylion007 IIUC).

* Trying one-by-one noexcept copy ctor for old compilers.

* Add back test covering copy ctor. Add another simple test that exercises the copy ctor.

* Exclude more older compilers from using the noexcept = default ctors. (The tests in the previous commit exposed that those are broken.)

* Factor out & reuse gil_scoped_acquire_local as gil_scoped_acquire_simple

* Guard gil_scoped_acquire_simple by _Py_IsFinalizing() check.

* what() GIL safety

* clang-tidy & Python 3.6 fixes

* Use `gil_scoped_acquire` in dtor, copy ctor, `what()`. Remove `_Py_IsFinalizing()` checks (they are racy: https://github.com/python/cpython/pull/28525).

* Remove error_scope from copy ctor.

* Add `error_scope` to `get_internals()`, to cover the situation that `get_internals()` is called from the `error_already_set` dtor while a new Python error is in flight already. Also backing out `gil_scoped_acquire_simple` change.

* Add `FlakyException` tests with failure triggers in `__init__` and `__str__`

THIS IS STILL A WORK IN PROGRESS. This commit is only an important resting point.

This commit is a first attempt at addressing the observation that `PyErr_NormalizeException()` completely replaces the original exception if `__init__` fails. This can be very confusing even in small applications, and extremely confusing in large ones.

* Tweaks to resolve Py 3.6 and PyPy CI failures.

* Normalize Python exception immediately in error_already_set ctor.

For background see: https://github.com/pybind/pybind11/pull/1895#issuecomment-1135304081

* Fix oversights based on CI failures (copy & move ctor initialization).

* Move @pytest.mark.xfail("env.PYPY") after @pytest.mark.parametrize(...)

* Use @pytest.mark.skipif (xfail does not work for segfaults, of course).

* Remove unused obj_class_name_or() function (it was added only under this PR).

* Remove already obsolete C++ comments and code that were added only under this PR.

* Slightly better (newly added) comments.

* Factor out detail::error_fetch_and_normalize. Preparation for producing identical results from error_already_set::what() and detail::error_string(). Note that this is a very conservative refactoring. It would be much better to first move detail::error_string into detail/error_string.h

* Copy most of error_string() code to new error_fetch_and_normalize::complete_lazy_error_string()

* Remove all error_string() code from detail/type_caster_base.h. Note that this commit includes a subtle bug fix: previously error_string() restored the Python error, which will upset pybind11_fail(). This never was a problem in practice because the two PyType_Ready() calls in detail/class.h do not usually fail.

* Return const std::string& instead of const char * and move error_string() to pytypes.h

* Remove gil_scope_acquire from error_fetch_and_normalize, add back to error_already_set

* Better handling of FlakyException __str__ failure.

* Move error_fetch_and_normalize::complete_lazy_error_string() implementation from pybind11.h to pytypes.h

* Add error_fetch_and_normalize::release_py_object_references() and use from error_already_set dtor.

* Use shared_ptr for m_fetched_error => 1. non-racy, copy ctor that does not need the GIL; 2. enables guard against duplicate restore() calls.

* Add comments.

* Trivial renaming of a newly introduced member function.

* Workaround for PyPy

* Bug fix (oversight). Only valgrind got this one.

* Use shared_ptr custom deleter for m_fetched_error in error_already_set. This enables removing the dtor, copy ctor, move ctor completely.

* Further small simplification. With the GIL held, simply deleting the raw_ptr takes care of everything.

* IWYU cleanup

```
iwyu version: include-what-you-use 0.17 based on Debian clang version 13.0.1-3+build2
```

Command used:

```
iwyu -c -std=c++17 -DPYBIND11_TEST_BOOST -Iinclude/pybind11 -I/usr/include/python3.9 -I/usr/include/eigen3 include/pybind11/pytypes.cpp
```

pytypes.cpp is a temporary file: `#include "pytypes.h"`

The raw output is very long and noisy.

I decided to use `#include <cstddef>` instead of `#include <cstdio>` for `std::size_t` (iwyu sticks to the manual choice).

I ignored all iwyu suggestions that are indirectly covered by `#include <Python.h>`.

I manually verified that all added includes are actually needed.

Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
2022-06-02 16:17:38 -07:00
Aaron Gokaslan 58802de41b
perf: Add object rvalue overload for accessors. Enables reference stealing (#3970)
* Add object rvalue overload for accessors. Enables reference stealing

* Fix comments

* Fix more comment typos

* Fix bug

* reorder declarations for clarity

* fix another perf bug

* should be static

* future proof operator overloads

* Fix perfect forwarding

* Add a couple of tests

* Remove errant include

* Improve test documentation

* Add dict test

* add object attr tests

* Optimize STL map caster and cleanup enum

* Reorder to match declarations

* adjust increfs

* Remove comment

* revert value change

* add missing move
2022-06-01 15:19:13 -04:00
Ralf W. Grosse-Kunstleve 9f7b3f735a
addl unit tests for PR #3970 (#3977)
* Add test_perf_accessors (to be merged into test_pytypes).

* Python < 3.8 f-string compatibility

* Use thread_local in inc_ref_counter()

* Intentional breakage, brute-force way to quickly find out how many platforms reach the PYBIND11_HANDLE_REF_DEBUG code, with and without threads.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Remove Intentional breakage

* Drop perf test, move inc_refs tests to test_pytypes

* Fold in PR #3970 with `#ifdef`s

* Complete test coverage for all newly added code.

* Condense new unit tests via a simple local helper macro.

* Remove PYBIND11_PR3970 define. See https://github.com/pybind/pybind11/pull/3977#issuecomment-1142526417

* Move static keyword first (fixes silly oversight).

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2022-05-31 12:59:19 -07:00
Ralf W. Grosse-Kunstleve b24c5ed204
Replace "Unknown internal error occurred" with a more helpful message. (#3982) 2022-05-31 11:54:33 -07:00
Ralf W. Grosse-Kunstleve de4ba92c9f
Add `error_scope` to `detail::get_internals()` (#3981)
* Add `error_scope` to `detail::get_internals()`

* Adjust test to tolerate macOS PyPy behavior.
2022-05-31 11:51:13 -07:00
Aaron Gokaslan 8da58da539
chore: perfectly forward all make_iterator args (#3980)
* Perfectly forward all make_iterator args

* Try emplace back
2022-05-28 19:58:15 -04:00
Ralf W. Grosse-Kunstleve 748ae2270b
Add missing error handling to `module_::def_submodule` (#3973)
* Add missing error handling to module_::def_submodule

* Add test_def_submodule_failures

* PyPy only: Skip test with trigger for PyModule_GetName() failure.

* Reapply minor fix that accidentally got lost in transfer from PR #3964
2022-05-28 16:40:57 -07:00
Aaron Gokaslan 68f8010500
chore: add err guard to capsule destructor and add a move to iostream (#3958)
* Add err guard to capsule destructor

* only uses ostream currently

* can these be noexcept

* Add back header

* fix for older compilers

* This should at least be noexcept

* Add missing move

* Apparently not noexcept for old llvm
2022-05-27 14:32:57 -04:00
Ralf W. Grosse-Kunstleve 2c549eb7aa
Move `PyErr_NormalizeException()` up a few lines (#3971)
* Add error_already_set_what what tests, asserting the status quo.

* Move PyErr_NormalizeException() up a few lines.

* @pytest.mark.skipif("env.PYPY") from PR #1895 is required even for this much simpler PR

* Move PyException_SetTraceback() with PyErr_NormalizeException() as suggested by @skylion007

* Insert a std::move() as suggested by @skylion007
2022-05-25 21:44:55 -07:00
Aaron Gokaslan 2d4a20c8cb
chore: add missing moves for buffer_func and staticmethod in pybind11.h (#3969)
* Use move converting ctor when making class staticmethod

* Add missing caster move in buffer func

* fix use after move

* add back move to staticmethod

* avoid shadowing with varname
2022-05-25 12:14:07 -04:00
Aaron Gokaslan c42414db86
(perf): use a rvalue cast in func_wrapper (#3966)
* (perf): use an rvalue cast in func_wrapper

* Try to clarify comment

* Fix comment typo
2022-05-23 12:26:53 -04:00
Aaron Gokaslan a8b3ff30f9
chore: add a couple of moves in pybind11.h (#3941)
* Add missing std::moves in enum_base methods

* Remove useless move

* Add one more std::move in print

* Further optimize print

* Some more misc optimizations
2022-05-17 11:29:49 -04:00
Maarten Baert 72eea20afd
Fix py::cast from pytype rvalue to pytype (#3949)
* Fix py::cast from pytype rvalue to pytype

Previously, py::cast blindly assumed that the destination type was a C++
type rather than a python type when the source type was an rvalue.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2022-05-16 13:51:01 -07:00
Ralf W. Grosse-Kunstleve 48c7be4a56 Undoing previous accidental commit. Sorry I forgot to git branch. 2022-05-16 04:31:01 -07:00
Ralf W. Grosse-Kunstleve 5621ab853a Do we have a unit test for the traceback code in error_string()? 2022-05-16 04:26:35 -07:00
Aaron Gokaslan 2e331308d3
chore: remove unused include from stl.h (#3928)
* change iostream include to ostream include
2022-05-06 16:57:39 -04:00
Ed Catmur 68a0b2dfd8
Add anyset & frozenset, enable copying (cast) to std::set (#3901)
* Add frozenset, and allow it cast to std::set

For the reverse direction, std::set still casts to set. This is in concordance with the behavior for sequence containers, where e.g. tuple casts to std::vector but std::vector casts to list.

Extracted from #3886.

* Rename set_base to any_set to match Python C API

since this will be part of pybind11 public API

* PR: static_cast, anyset

* Add tests for frozenset

and rename anyset methods

* Remove frozenset default ctor, add tests

Making frozenset non-default constructible means that we need to adjust pyobject_caster to not require that its value is default constructible, by initializing value to a nil handle.  This also allows writing C++ functions taking anyset, and is arguably a performance improvement, since there is no need to allocate an object that will just be replaced by load.

Add some more tests, including anyset::empty, anyset::size, set::add and set::clear.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add rationale to `pyobject_caster` default ctor

* Remove ineffectual protected: access control

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2022-05-05 12:09:56 -07:00
Michael Voznesensky f0b9f755e4
Replace error printing code gated by NDEBUG with a new flag: PYBIND11_DETAILED_ERROR_MESSAGES (#3913)
* Update cast.h

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Move definition to detail/common, change name, apply everywhere

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Rename debug_enabled in tests to detailed_error_messages_enabled
2022-05-02 15:30:19 -04:00
Aaron Gokaslan 75007dda72
chore: rule of 3 for strdup guard (#3905) 2022-04-26 15:49:24 -04:00
Aaron Gokaslan 82455a41fd
Minor opt to cache tuple casting (#3894) 2022-04-24 14:46:39 -04:00
Aaron Gokaslan 1c636f4dce
chore: Change numpy dtype from_args call sig to const ref (#3878)
* Change numpy from_args call signature to avoid copy

* Reorder ctors

* Rename arg

* Fix unnecessary move

* Fix clang-tidy and Add a few missing moves to memory_view pytype
2022-04-18 11:11:24 -04:00
Aaron Gokaslan fbcde3f0af
chore: enable clang-tidy check modernize-use-nullptr (#3881)
* Enable clang-tidy check modernize-use-nullptr

* Sort clang-tidy

* Sorted again
2022-04-18 11:09:45 -04:00
Ralf W. Grosse-Kunstleve 30716c67a1
Also add error_scope assignment operator to complete the rule-of-3 (follow-on to PR #3870). (#3872) 2022-04-15 14:44:48 -04:00
Aaron Gokaslan 3829b7624c
chore: simplify numpy dtype ctor (#3869)
* Simplify numpy dtype ctor

* Simplify c_str ctor

* Remove compat macro for str in numpy
2022-04-14 14:12:44 -07:00
Oleksandr Pavlyk 45164c1fcb
Added deleted copy constructor for error_scope to comply with rule of 3. (#3870)
* Added deleted copy constructor for error_scope to comply with rule of 3.
2022-04-14 12:16:58 -04:00
Oleksandr Pavlyk ba7a0fac73
Expand dtype accessors (#3868)
* Added constructor based on typenum, based on PyArray_DescrFromType

Added accessors for typenum, alignment, byteorder and flags fields of
PyArray_Descr struct.

* Added tests for new py::dtype constructor, and for accessors

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fixed the comment for alignment method

* Update include/pybind11/numpy.h

Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
2022-04-14 10:53:16 -04:00
Oleksandr Pavlyk fa98804a07
Adds set_name method of pybind11::capsule class (#3866)
* Adds set_name method of pybind11::capsule class

This calls PyCapsule_SetName on the underlying capsule object.

modified destructors to query capsules's Name

[pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Handle possible exception thrown by PyCapsule_GetName

Also removed accidentally reintroduced use of `const char *&`.

[pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Fixed function name

* Introduced private static function to reuse get_name_or_throw

* added tests for capsule renaming

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* handle python error in flight

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Initialized PyObject * variables to nullptr

* use write-unraisable if PyCapsule_GetName raises

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* get_name_or_throw->get_name_no_throw

If PyCapsule_GetName raises an error we should write as unraisable
to consume it and notify user, and then restore the error in flight if any.
This way this method called from destructor would not modify interpreter
error state.

* used error_scope struct

* Renamed get_name_no_throw->get_name_in_error_scope

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2022-04-14 10:51:27 -04:00
Laramie Leavitt 088ad4f298
Cleanup cast_safe<void> specialization (#3861)
* Cleanup cast_safe<void> specialization

Replace explicit specialization of cast_safe<void> with SFINAE.
It's better for SFINAE cases to cover all type-sets rather than mixing SFINAE and explicit specialization.

Extracted from #3674

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update cast.h

Use detail::none_of<> as suggested

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update cast.h

Reorder:
If TEMP_REF
If VOID
if (!VOID && !TEMP_REF)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2022-04-11 11:57:05 -07:00
Aaron Gokaslan e3aa215b02
Add perfect forwarding to make_iterator calls (#3860) 2022-04-11 14:03:43 -04:00
Aaron Gokaslan 1b27b744c1
chore: Make stl_bind take slice as const_ref (#3852)
* Make stl_bind take slice as const_ref

* Change eval order if

* Silence MSVC warning
2022-04-11 13:53:30 -04:00
Laramie Leavitt ab59f45d2e
Prefer make_caster<T> to type_caster<T> (#3859) 2022-04-11 10:36:24 -07:00
Aaron Gokaslan c4e295287b
perf: Add more moves and optimize (#3845)
* Make slice constructor consistent

* Add more missing std::move for ref steals

* Add missing perfect forwarding for arg_v ctor

* Add missing move in arg_v constructor

* Revert "Add missing move in arg_v constructor"

This reverts commit 126fc7c524.

* Add another missing move in cast.h

* Optimize object move ctor

* Don't do useless move

* Make move ctor same as nb

* Make obj move ctor same as nb

* Revert changes which break MSVC
2022-04-05 14:36:39 -04:00
Lonnie L. Souder II b3ebd11d98
feature: support compilers that use std::experimental::filesystem (#3840)
* feature: support compilers that use std::experimental::filesystem such as gcc7

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* only use this feature if cpp17

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* more specific namespace alias + style

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* move to pybind11 namespace

* no namespace alias

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: Lonnie Souder II <lonnie.souder@g6labs.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2022-03-31 22:16:10 -07:00
Aaron Gokaslan 3a183d4b58
fix: improve str exceptions and consistency with python (#3826)
* Improve str exceptions

* Revert macro change just in case

* Make clang-tidy happy

* Fix one more clang-tidy issue

* Refactor duplicate method
2022-03-25 16:01:34 -04:00
Aaron Gokaslan 146695a904
fix: better exception and error handling for capsules (#3825)
* Make capsule errors better match python
2022-03-25 10:55:13 -04:00
Aaron Gokaslan 47079b9e7b
(perf): Add missing move in sp matrix caster and microopt char concats (#3823)
* Add missing move in sp matrix caster and microopt char concat

* Remove useless move

* Add a couple more std::move

* Missed one char

* Improve error_string

* Ensure no temp reallocs in errorString concat

* Remove useless move
2022-03-24 12:57:37 -04:00
Laramie Leavitt b22ee64c73
Add type_caster<std::monostate> (#3818)
* Add type_caster<std::monostate> for std::variant

Add type_caster<std::monostate>, allowing std::variant<std::monostate, ...>

* Add  variant<std::monostate, ...> test methods

* Add std::monostate tests

* Update test_stl.py

Remove erroneous extra tests

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update test fn name

* And update the doc() test

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2022-03-21 22:58:04 -07:00
Laramie Leavitt b3a43d137c
Use rvalue reference for std::variant cast_op<T> (#3811)
Nearly every call site of cast_op<T> uses an r-value reference.

Except stl.h variant_caster::load_alternative for handling std::variant.

Fix that.
2022-03-18 11:10:31 -07:00
Oleksandr Pavlyk 91a6e129d9
PYBIND11_OBJECT_CVT should use namespace for error_already_set() (#3797)
* PYBIND11_OBJECT_CVT should use namespace for error_already_set()

This change makes the macro usable outside of pybind11 namespace.

* added test for use of PYBIND11_OBJECT_CVT for classes in external to pybind11 namespaces

* Extended test_pytypes.cpp and test_pytest.py

The added test defines a dummy function that takes a custom-defined class external::float_
that uses PYBIND11_OBJECT_CVT

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fixed issues pointed out by CI

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fixed memory leak in default constructor

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2022-03-11 12:18:25 -08:00
Aaron Gokaslan 2dd5254494
fix: missing move in eval.h (#3775) 2022-03-02 15:25:43 -05:00
Aaron Gokaslan af08a95b56
fix: potential memory leak in pypy (#3774) 2022-03-02 14:14:52 -05:00
Aaron Gokaslan 42a8e31253
Improve Python 3.11 support (#3694)
* Test out Python 3.11 migration

* Clean up a bit

* Remove todo

* Test workaround

* Fix potential bug uncovered in 3.11

* Try to fix it more

* last ditch fix

* Revert. Tp-traverse isn't the problem

* Test workaround

* Try this hack

* Revert MRO changes

* Use f_back properly

* Qualify auto

* Update include/pybind11/pybind11.h

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Simplify code slightly

* Ensure co_varnames decref if dict_getitem throws

* Eager decref f_code

Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2022-03-02 13:17:52 -05:00
Henry Schreiner 5f9b090a91
ci: fix PyPy (#3768)
* chore: minor fixes for newer PyPys

* ci: fix issue with PyPy
2022-03-01 12:42:52 -05:00
Eric Cousineau f495dfc433
cast: Qualify symbol usage in PYBIND11_TYPE_CASTER (#3758)
* cast: Qualify symbol usage in PYBIND11_TYPE_CASTER

Permits using macro outside of pybind11::detail

* fixup! review
2022-02-25 13:25:23 -08:00
kururu002 da15bb206c
Cast bytearray to string (#3707)
* Add bytearray to string cast, testcase and rename load_bytes to load_raw

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* New bytearray test case and convert failure to pybind11_fail

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix merge comments

* Actually fix merge comments

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Assert early if AsString fails

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Porras Huang <pohuang@jackx-vm-1.nvidia.com>
2022-02-23 18:21:03 -05:00
StarQTius 9aa676d38d
fix: clear local internals after finalizing interpreter #2101 (#3744)
* Clear local internals after finalizing interpreter

* Add descriptive comments

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2022-02-20 08:00:29 -08:00
Peter Hawkins 44596bc4ee
Fix exception handling when pybind11::weakref() fails. (#3739)
* Clear Python error state if pybind11::weakref() fails.

The weakref() constructor calls pybind11_fail() without clearing any
Python interpreter error state. If a client catches the C++ exception
thrown by pybind11_fail(), the Python interpreter will be left in an
error state.

* Add test case for failing to create weakref

* Add Debug asserts for pybind11 fail

* Make error handling more pythonic

* Does this fix PyPy?

* Adapt test to PyPy differences

* Simplify test to remove redundancy

Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
2022-02-18 14:12:00 -05:00
Ralf W. Grosse-Kunstleve 009ffc3362
MSVC C++20 test_eigen (#3741)
* Removing C++20 condition for MSVC is_template_base_of decltype workaround.

* `-DDOWNLOAD_EIGEN=ON` for MSVC 2022 C++20

* `-DDOWNLOAD_EIGEN=ON` for MSVC 2019 C++20

* `-DPYBIND11_WERROR=OFF` for MSVC C++20 (2019, 2020)

* Restoring `defined(PYBIND11_CPP20)` in common.h

* pragma warning(disable : 5054) in eigen.h

* Reverting `-DPYBIND11_WERROR=OFF` changes.
2022-02-16 07:07:53 -08:00
Ralf W. Grosse-Kunstleve a97e9d8cac
Dropping MSVC 2015 (#3722)
* Changing `_MSC_VER` guard to `< 1910` (dropping MSVC 2015).

* Removing MSVC 2015 from ci.yml, and .appveyor.yml entirely.

* Bringing back .appveyor.yml from master.

* appveyor Visual Studio 2017

* 1st manual pass, builds & tests with unix_clang, before pre-commit.

* After clang-format (via pre-commit).

* Manual pass looking for "2015", builds & tests with unix_clang, before pre-commit.

* Backtracking for include/pybind11 changes in previous commit.

git checkout d07865846c include/pybind11/attr.h include/pybind11/detail/common.h include/pybind11/functional.h

--------------------

CI #4160 errors observed:

2a26873727
https://github.com/pybind/pybind11/runs/5168332130?check_suite_focus=true

$ grep ' error C' *.txt | sed 's/2022-02-12[^ ]*//' | sed 's/^[0-9][0-9]*//' | sed 's/^.*\.txt: //' | sort | uniqD:\a\pybind11\pybind11\include\pybind11\cast.h(1364,1): error C2752: 'pybind11::detail::type_caster<Eigen::Ref<Eigen::Vector3f,0,pybind11::EigenDStride>,void>': more than one partial specialization matches the template argument list [D:\a\pybind11\pybind11\build\tests\pybind11_tests.vcxproj]

d:\a\pybind11\pybind11\include\pybind11\detail/common.h(1023): error C2737: 'pybind11::overload_cast': 'constexpr' object must be initialized [D:\a\pybind11\pybind11\build\tests\cross_module_gil_utils.vcxproj]
d:\a\pybind11\pybind11\include\pybind11\detail/common.h(1023): error C2737: 'pybind11::overload_cast': 'constexpr' object must be initialized [D:\a\pybind11\pybind11\build\tests\pybind11_cross_module_tests.vcxproj]
d:\a\pybind11\pybind11\include\pybind11\detail/common.h(1023): error C2737: 'pybind11::overload_cast': 'constexpr' object must be initialized [D:\a\pybind11\pybind11\build\tests\pybind11_tests.vcxproj]
d:\a\pybind11\pybind11\include\pybind11\detail/common.h(1023): error C2737: 'pybind11::overload_cast': 'constexpr' object must be initialized [D:\a\pybind11\pybind11\build\tests\test_embed\external_module.vcxproj]
D:\a\pybind11\pybind11\include\pybind11\detail/common.h(624): fatal error C1001: Internal compiler error. [D:\a\pybind11\pybind11\build\tests\pybind11_tests.vcxproj]
D:\a\pybind11\pybind11\include\pybind11\detail/common.h(624): fatal error C1001: Internal compiler error. [D:\a\pybind11\pybind11\tests\pybind11_tests.vcxproj]

$ grep ': error C2737' *.txt | sed 's/^.*MSVC//' | sed 's/___.*//' | sort | uniq

_2017

$ grep ': error C2752' *.txt

3______3.8_____MSVC_2019_____x86_-DCMAKE_CXX_STANDARD=17.txt:2022-02-12T16:12:45.9921122Z D:\a\pybind11\pybind11\include\pybind11\cast.h(1364,1): error C2752: 'pybind11::detail::type_caster<Eigen::Ref<Eigen::Vector3f,0,pybind11::EigenDStride>,void>': more than one partial specialization matches the template argument list [D:\a\pybind11\pybind11\build\tests\pybind11_tests.vcxproj]

$ grep ': fatal error C1001:' *.txt

10______pypy-3.8-v7.3.7_____windows-2022_____x64.txt:2022-02-12T16:12:56.3163683Z D:\a\pybind11\pybind11\include\pybind11\detail/common.h(624): fatal error C1001: Internal compiler error. [D:\a\pybind11\pybind11\tests\pybind11_tests.vcxproj]
1______3.6_____MSVC_2019_____x86.txt:2022-02-12T16:12:47.6774625Z D:\a\pybind11\pybind11\include\pybind11\detail/common.h(624): fatal error C1001: Internal compiler error. [D:\a\pybind11\pybind11\build\tests\pybind11_tests.vcxproj]
16______3.6_____windows-latest_____x64_-DPYBIND11_FINDPYTHON=ON.txt:2022-02-12T16:12:27.0556151Z D:\a\pybind11\pybind11\include\pybind11\detail/common.h(624): fatal error C1001: Internal compiler error. [D:\a\pybind11\pybind11\tests\pybind11_tests.vcxproj]
17______3.9_____windows-2019_____x64.txt:2022-02-12T16:12:30.3822566Z D:\a\pybind11\pybind11\include\pybind11\detail/common.h(624): fatal error C1001: Internal compiler error. [D:\a\pybind11\pybind11\tests\pybind11_tests.vcxproj]
2______3.7_____MSVC_2019_____x86.txt:2022-02-12T16:12:38.7018911Z D:\a\pybind11\pybind11\include\pybind11\detail/common.h(624): fatal error C1001: Internal compiler error. [D:\a\pybind11\pybind11\build\tests\pybind11_tests.vcxproj]
6______3.6_____windows-2022_____x64.txt:2022-02-12T16:12:00.4513642Z D:\a\pybind11\pybind11\include\pybind11\detail/common.h(624): fatal error C1001: Internal compiler error. [D:\a\pybind11\pybind11\tests\pybind11_tests.vcxproj]
7______3.9_____windows-2022_____x64.txt:2022-02-12T16:11:43.6306160Z D:\a\pybind11\pybind11\include\pybind11\detail/common.h(624): fatal error C1001: Internal compiler error. [D:\a\pybind11\pybind11\tests\pybind11_tests.vcxproj]
8______3.10_____windows-2022_____x64.txt:2022-02-12T16:11:49.9589644Z D:\a\pybind11\pybind11\include\pybind11\detail/common.h(624): fatal error C1001: Internal compiler error. [D:\a\pybind11\pybind11\tests\pybind11_tests.vcxproj]
9______pypy-3.7-v7.3.7_____windows-2022_____x64.txt:2022-02-12T16:11:53.7912112Z D:\a\pybind11\pybind11\include\pybind11\detail/common.h(624): fatal error C1001: Internal compiler error. [D:\a\pybind11\pybind11\tests\pybind11_tests.vcxproj]

* common.h: is_template_base_of

* Re-applying 4 changes from 2a26873727 that work universally.

* `overload_cast = {};` only for MSVC 2017 and Clang 5

* Refining condition for using is_template_base_of workaround.

* Undoing MSVC 2015 workaround in test_constants_and_functions.cpp

* CentOS7: silence_unused_warnings

* Tweaks in response to reviews.

* Adding windows-2022 C++20

* Trying another way of adding windows-2022 C++20
2022-02-14 11:36:22 -08:00
Henry Schreiner 522c59ceb2
chore: drop Python 3.5 (#3719)
* chore: drop Python 3.5 support

* chore: more fstrings with flynt's help

* ci: drop Python 3.5

* chore: bump dependency versions

* docs: touch up py::args

* tests: remove deprecation warning

* Ban smartquotes

* Very minor tweaks (by-product of reviewing PR #3719).

Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
2022-02-11 19:06:16 -05:00
Ralf W. Grosse-Kunstleve 6493f496e3
Python 2 removal part 1: tests (C++ code is intentionally ~untouched) (#3688)
* `#error BYE_BYE_GOLDEN_SNAKE`

* Removing everything related to 2.7 from ci.yml

* Commenting-out Centos7

* Removing `PYTHON: 27` from .appveyor.yml

* "PY2" removal, mainly from tests. C++ code is not touched.

* Systematic removal of `u` prefix from `u"..."` and `u'...'` literals. Collateral cleanup of a couple minor other things.

* Cleaning up around case-insensitive hits for `[^a-z]py.*2` in tests/.

* Removing obsolete Python 2 mention in compiling.rst

* Proper `#error` for Python 2.

* Using PY_VERSION_HEX to guard `#error "PYTHON 2 IS NO LONGER SUPPORTED.`

* chore: bump pre-commit

* style: run pre-commit for pyupgrade 3+

* tests: use sys.version_info, not PY

* chore: more Python 2 removal

* Uncommenting Centos7 block (PR #3691 showed that it is working again).

* Update pre-commit hooks

* Fix pre-commit hook

* refactor: remove Python 2 from CMake

* refactor: remove Python 2 from setup code

* refactor: simplify, better static typing

* feat: fail with nice messages

* refactor: drop Python 2 C++ code

* docs: cleanup for Python 3

* revert: intree

revert: intree

* docs: minor touchup to py2 statement

Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
2022-02-10 18:28:08 -08:00
Ralf W. Grosse-Kunstleve ec24786eab
Fully-automatic clang-format with include reordering (#3713)
* chore: add clang-format

* Removing check-style (Classic check-style)

Ported from @henryiii's 53056b1b0e

* Automatic clang-format changes (NO manual changes).

Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
2022-02-10 12:17:07 -08:00
Ralf W. Grosse-Kunstleve e96221beff
Final manual curation in preparation for global `clang-format`ing (#3712)
* Manual line breaks to pre-empt undesired `clang-format`ing.

Informed by work under https://github.com/pybind/pybind11/pull/3683:

60b7eb410f

59572e6559

* Manual curation of clang-format diffs involving source code comments.

Very labor-intensive and dull.

* Pulling .clang-format change from @henryiii's 9057962d40

* Adding commonly used .clang-format `CommentPragmas:`

* Ensure short lambdas are allowed

Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
2022-02-10 11:42:03 -08:00
Aaron Gokaslan d6c66d25bb
chore(clang-tidy): Add clang-tidy rules: prefer-member-initializer and optin.performance.Padding (#3716)
* Add clang-tidy prefer-member-initializer

* Fix clang-tdy config

* Fix incorrect change

* Fix sorting of .clang-tidy
2022-02-10 09:45:46 -08:00
Aaron Gokaslan dc9803cef2
Add missing clang-tidy fixes (#3715) 2022-02-10 09:23:15 -08:00
Ralf W. Grosse-Kunstleve abc38690dc
Manually applying two clang-format changes that need fix-ups for clang-tidy. (#3705) 2022-02-09 06:32:41 -08:00
Ralf W. Grosse-Kunstleve 7769e7719c
clang-tidy readability-qualified-auto (#3702)
* Adding readability-qualified-auto to .clang-tidy

Ported from @henryiii's 287527f705

* fix: support Python < 3.6

Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
2022-02-09 06:24:57 -08:00
Ralf W. Grosse-Kunstleve b4f5350d0d
chore: use member initializer (#3704)
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
2022-02-08 22:08:31 -05:00
Ralf W. Grosse-Kunstleve ddbc74c674 Adding .clang-tidy readability-braces-around-statements option.
clang-tidy automatic changes. NO manual changes.
2022-02-08 13:02:20 -08:00
Ralf W. Grosse-Kunstleve 8581584e60 Manual fix-ups in preparation for clang-tidy readability-braces-around-statements.
Informed by experiments under PR #3698.
2022-02-08 13:02:20 -08:00
Henry Schreiner af056b65d3
fix: __index__ on Enum should always be present. (#3700)
* chore: minor odd py version cleanup

* Update include/pybind11/pybind11.h

* fix: always make __index__ available
2022-02-08 11:47:30 -05:00
Aaron Gokaslan 1b84188330
Minor change to improve readability (#3695) 2022-02-07 15:29:25 -05:00
Henry Schreiner 36813cfa12
chore: back to work 2022-02-03 12:44:10 -05:00
Henry Schreiner ffa346860b
chore: bump to 2.9.1 2022-02-02 17:34:15 -05:00
Ralf W. Grosse-Kunstleve 3899dc65b9
Documenting missing unit test coverage. (#3673) 2022-02-02 13:16:44 -08:00
Aaron Gokaslan ce18721d83
Ensure TypeError use raise_from for C++->Python overload res. (#3671) 2022-01-31 15:13:05 -05:00
Aaron Gokaslan 978617f6b5
fix issue 3668 by removing bool casts in numpy.h (#3669) 2022-01-31 12:57:32 -05:00
Aaron Gokaslan 3a8d92308d
Fix caster optimization regression introduced in #3650 (#3659)
* Fix optimization bug introduced in #3650

* Add simple Python extension test for MVF

* Improve comments

* Clarify comment

* Clarify another comment

* Add test docstring

* Fix typo
2022-01-31 12:19:48 -05:00
Mattia Basaglia 07103d6570
Remove extra semicolon (#3666) 2022-01-29 14:44:48 -08:00
Dustin Spicuzza ec81e8e778
Propagate py::multiple_inheritance to all children (#3650)
* Add tests demonstrating smart_holder issues with multiple inheritance

* Propagate C++ multiple inheritance markers to all children

- Makes py::multiple_inheritance only needed in base classes hidden from pybind11
2022-01-26 17:03:52 -08:00
Sergei Izmailov 5194855900
Render `py::bool_` and `py::float_` without `_` in docstrings (#3622)
* Render `py::bool_` as `bool` in docstrings

* Render `py::float_` as `float` in docstrings
2022-01-16 07:05:46 -08:00
Aaron Gokaslan d2ec836712
Add support for nested C++11 exceptions (#3608)
* Add support for nested C++11 exceptions

* Remove wrong include

* Fix if directive

* Fix missing skipif

* Simplify code and try to work around MSVC bug

* Clarify comment

* Further simplify code

* Remove the last extra throw statement

* Qualify auto

* Fix typo

* Add missing return for consistency

* Fix clang-tidy complaint

* Fix python2 stub

* Make clang-tidy happy

* Fix compile error

* Fix python2 function signature

* Extract C++20 utility and backport

* Cleanup code a bit more

* Improve test case

* Consolidate code and fix signature

* Fix typo
2022-01-14 14:22:47 -05:00
Aaron Gokaslan d434b5f31e
(chore): Remove deprecated c-headers (#3610)
* Remove deprecated c-headers

* Update calls to old cfunctions

* Add missing one

* Add another missing one
2022-01-11 17:57:59 -05:00
Aaron Gokaslan ef070f7750
Add additional info to TypeError when C++->Python casting fails (#3605)
* Add additional info to TypeInfo when C++->Python casting fails

* Fix typo

* Address reviewer comments
2022-01-10 21:18:00 -05:00
Ralf W. Grosse-Kunstleve 1bbaeb3462
Adding dedicated test_const_name. (#3578)
* Adding dedicated test_const_name.

Also exercises pybind11::detail::_ backward compatibility.

See also: PR #3423

* Backing out tests involving int_to_str (requires C++17 or higher).

* Suppressing clang-tidy errors.

* Disabling test_const_name for MSVC 2015 due to bizarre failures.

* Stacking @pytest.mark.parametrize (thanks to @skylion007 for pointing out).
2021-12-29 12:54:25 -08:00
Henry Schreiner 45f792efdd
chore: prepare for 2.9 2021-12-28 10:47:21 -05:00
Henry Schreiner cb302305a3
fix: restore full range of _ functions (#3571) 2021-12-23 14:50:10 -05:00
Henry Schreiner 39fbc7992b
fix: avoiding usage of _ if already defined (#3423)
* fix: avoid usage of _

* ci: test _ defined

* docs: include change in docs

* fix: add a test and comment

* refactor: const_str -> const_name
2021-12-21 14:24:21 -05:00
Boris Rasin a224d0cca5
fix: vs2022 compilation, issue #3477 (#3497)
* fix: vs2022 compilation, issue #3477

* silence warning for python 2.7

* disable warning around mbstowcs call

* move disable warning code closer to call site

* turn on vs2022 ci test

* ci: don't run helpers on Windows 2022 & Python 3.5

* limit workaround for stdlib shipped with vs2022 or later

* fix for: limit workaround for stdlib shipped with vs2022 or later

* fix 2 for: limit workaround for stdlib shipped with vs2022 or later

* comment

* ci: add a Windows 2019 run

* ci: add Python 2.7 check too

Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
2021-12-03 11:10:36 -08:00
Jason Rhinelander b4939fcbfb
Expand std::string_view support to str, bytes, memoryview (#3521)
* Expand string_view support to str, bytes, memoryview

1. Allows constructing a str or bytes implicitly from a string_view;
   this is essentially a small shortcut allowing a caller to write
   `py::bytes{sv}` rather than `py::bytes{sv.data(), sv.size()}`.

2. Allows implicit conversion *to* string_view from py::bytes -- this
   saves a fair bit more as currently there is no simple way to get such
   a view of the bytes without copying it (or resorting to Python API
   calls).

   (This is not done for `str` because when the str contains unicode we
   have to allocate to a temporary and so there might not be some string
   data we can properly view without owning.)

3. Allows `memoryview::from_memory` to accept a string_view.  As with
   the other from_memory calls, it's entirely your responsibility to
   keep it alive.

This also required moving the string_view availability detection into
detail/common.h because this PR needs it in pytypes.h, which is higher
up the include chain than cast.h where it was being detected currently.

* Move string_view include to pytypes.h

* CI-testing a fix for the "ambiguous conversion" issue.

This change is known to fix the `tensorflow::tstring` issue reported under https://github.com/pybind/pybind11/pull/3521#issuecomment-985100965

TODO: Minimal reproducer for the `tensorflow::tstring` issue.

* Make clang-tidy happy (hopefully).

* Adding minimal reproducer for the `tensorflow::tstring` issue.

Error without the enable_if trick:

```
/usr/local/google/home/rwgk/forked/pybind11/tests/test_builtin_casters.cpp:169:16: error: ambiguous conversion for functional-style cast from 'TypeWithBothOperatorStringAndStringView' to 'py::bytes'
        return py::bytes(TypeWithBothOperatorStringAndStringView());
               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/google/home/rwgk/forked/pybind11/include/pybind11/detail/../pytypes.h:1174:5: note: candidate constructor
    bytes(const std::string &s) : bytes(s.data(), s.size()) { }
    ^
/usr/local/google/home/rwgk/forked/pybind11/include/pybind11/detail/../pytypes.h:1191:5: note: candidate constructor
    bytes(std::string_view s) : bytes(s.data(), s.size()) { }
    ^
```

* Adding missing NOLINTNEXTLINE

* Also apply ambiguous conversion workaround to str()

Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
2021-12-03 13:20:32 -04:00
Rasmus Munk Larsen 70a58c577e
Replace usage of deprecated Eigen class MappedSparseMatrix. (#3499)
* Replace usage of deprecated Eigen class

Eigen::MappedSparseMatrix has been deprecated since Eigen 3.3 from 2016. Use the equivalent modern syntax Eigen::Map<Eigen::SparseMatrix<...>>.

* Update eigen.h

* Update eigen.h
2021-11-22 17:01:35 -08:00
Lishen1 5d067e870a
fix: remove redundant copy operation to fix warning (#3486)
* fix compiler warning: deprecated implicit copy constructor

* take care of the bug http://eigen.tuxfamily.org/bz/show_bug.cgi?id=747

* add parenthesis for better reading

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update include/pybind11/eigen.h

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
2021-11-22 15:27:00 -05:00
Jason Rhinelander 673b4be3d7
Fix py::kw_only when used before the first arg of a method (#3488)
* Fix py::kw_only when used before the first arg of a method

The implicit space for the `self` argument isn't added until we hit the
first argument, but this wasn't being done for kw_only or pos_only, and
so a kw_only before the first argument would break.

This fixes it by properly checking whether we need to add the self arg.

(The pos_only issue here was extremely mild -- you didn't get the `/` in
the docstring, but AFAICT it has no other effect since there are no
meaningful arguments before it anyway).

* Style changes

- rename check_have_self_arg -> append_self_arg_if_needed

- move the argument name inline comments before the args instead of
  after
2021-11-20 16:01:57 -08:00
Henry Schreiner 72282f75a1
ci: support development releases of Python (#3419)
* ci: support development releases of Python

* fix: better PyPy support

* fix: patch over a few more pypy issues

* Try to patch

* Properly follow pep667

* Fix typo

* Whoops, 667 not in yet

* For testing

* More testing

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Try to backport

* Try to simplify fix

* Nail down the fix

* Try pypy workaround

* Typo

* one last typo

* Replacing 0x03110000 with 0x030B0000

* Add TODO. Drop PyPy

* Fix typo

* Revert catch upgrade

* fix: minor cleanup, try pypy again

Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
2021-11-17 09:44:19 -05:00
Trigve afdc09deda
[master] Wrong caching of overrides (#3465)
* override: Fix wrong caching of the overrides

There was a problem when the python type, which was stored in override
cache for C++ functions, was destroyed and  the record wasn't removed from the
override cache. Therefor, dangling pointer was stored there. Then when the
memory was reused and new type was allocated at the given address and the
method with the same name (as previously stored in the cache) was actually
overridden in python, it would wrongly find it in the override cache for C++
functions and therefor override from python wouldn't be called.
The fix is to erase the type from the override cache when the type is destroyed.

* test: Pass by const ref instead of by value (clang-tidy)

* test: Rename classes and move to different files

Rename the classes and files so they're no too generic. Also, better place to
test the stuff is in test_virtual_functions.cpp/.py as we're basically testing
the virtual functions/trampolines.

* Add TODO for erasure code

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-11-15 13:36:41 -05:00
Henry Schreiner aebd21b53c
docs: rework CI a bit, more modern skipping (#3424)
* docs: rework CI a bit, more modern skipping

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-11-10 12:13:10 -05:00
Boris Rasin 01f938e799
fix: add missing std::forward calls (#3443)
* fix: add missing std::forward calls

Two of the four cpp_function overloads are missing std::forward calls, which seems like a simple oversight.

* add test for https://github.com/pybind/pybind11/pull/3443

* add py tests

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix test

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-11-07 18:35:25 -05:00
Jason Rhinelander e7c9753f1d
feat: allow kw-only args after a py::args (#3402)
* Simply has_kw_only_args handling

This simplifies tracking the number of kw-only args by instead tracking
the number of positional arguments (which is really what we care about
everywhere this is used).

* Allow keyword-only arguments to follow py::args

This removes the constraint that py::args has to be last (or
second-last, with py::kwargs) and instead makes py::args imply
py::kw_only for any remaining arguments, allowing you to bind a function
that works the same way as a Python function such as:

    def f(a, *args, b):
        return a * b + sum(args)

    f(10, 1, 2, 3, b=20)  # == 206

With this change, you can bind such a function using:

    m.def("f", [](int a, py::args args, int b) { /* ... */ },
        "a"_a, "b"_a);

Or, to be more explicit about the keyword-only arguments:

    m.def("g", [](int a, py::args args, int b) { /* ... */ },
        "a"_a, py::kw_only{}, "b"_a);

(The only difference between the two is that the latter will fail at
binding time if the `kw_only{}` doesn't match the `py::args` position).

This doesn't affect backwards compatibility at all because, currently,
you can't have a py::args anywhere except the end/2nd-last.

* Take args/kwargs by const lvalue ref

Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>

Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
2021-10-28 23:16:55 -04:00
Ralf W. Grosse-Kunstleve a80b22374a chore: get back to work after 2.8.1
[skip ci]
2021-10-27 15:15:11 -07:00
Ralf W. Grosse-Kunstleve f7b499615e
[skip ci] Tweaks in preparation for the 2.8.1 release. (#3421) 2021-10-27 14:35:10 -07:00
Henry Schreiner e7e2c79f3f
fix: improve support for Python 3.11-dev (#3368)
* ci: support Python 3.11-dev

Also update 3.10 to final, better PyPy usage

* fix: use PyFrame_GetCode on Python 3.9+

* ci: some bitiness of pypy not supported on win

* chore: update CMake support to 3.22rc1 to quiet warning

* fix: use dev version of py to fix Py 3.11

* tests: print proper Eigen version

* ci: include pypy2, not sure why

* ci: avoid running on Python 3.11 for now

* ci: fix runs

* ci: simpler PyPy usage, drop unmaintained scipy + pypy index

* ci: only binary numpy, wait on pypy 3.8

* refactor: address review
2021-10-26 14:50:34 -04:00
Ryan Cahoon c2d3e220bd
fix: the types for return_value_policy_override in optional_caster (#3376)
* fix: the types for return_value_policy_override in optional_caster

`return_value_policy_override` was not being applied correctly in
`optional_caster` in two ways:
- The `is_lvalue_reference` condition referenced `T`, which was the
`optional<T>` type parameter from the class, when it should have used `T_`,
which was the parameter to the `cast` function. `T_` can potentially be a
reference type, but `T` will never be.
- The type parameter passed to `return_value_policy_override` should be
`T::value_type`, not `T`. This matches the way that the other STL container
type casters work.

The result of these issues was that a method/property definition which used a
`reference` or `reference_internal` return value policy would create a Python
value that's bound by reference to a temporary C++ object, resulting in
undefined behavior. For reasons that I was not able to figure out fully, it
seems like this causes problems when using old versions of `boost::optional`,
but not with recent versions of `boost::optional` or the `libstdc++`
implementation of `std::optional`. The issue (that the override to
`return_value_policy::move` is never being applied) is present for all
implementations, it just seems like that somehow doesn't result in problems for
the some implementation of `optional`. This change includes a regression type
with a custom optional-like type which was able to reproduce the issue.

Part of the issue with using the wrong types may have stemmed from the type
variables `T` and `T_` having very similar names. This also changes the type
variables in `optional_caster` to use slightly more descriptive names, which
also more closely follow the naming convention used by the other STL casters.

Fixes #3330

* Fix clang-tidy complaints

* Add missing NOLINT

* Apply a couple more fixes

* fix: support GCC 4.8

* tests: avoid warning about unknown compiler for compilers missing C++17

* Remove unneeded test module attribute

* Change test enum to have more unique int values

Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
2021-10-25 19:04:45 -07:00
Henry Schreiner 9379b399d9
fix: MSVC 2017 C++17 on Python 3 regression (#3407)
* fix: MSVC 2017 C++17 on Python 3 regression

* ci: add 3.7 job on CI
2021-10-25 16:01:19 -04:00
Aaron Gokaslan 78ee782bd4
feat: Add C++ binding to throw AttributeError (#3387)
* Add C++ bindings to throw AttributeError

* Fix formatting bug
2021-10-23 00:07:22 -04:00
Geoffrey Gunter 2d6014e417
docs: fix minor typo (#3390) 2021-10-21 10:37:54 -04:00
Henry Schreiner f791dc8648
fix: deprecate make_simple_namespace, fix Python 3.11 (#3374)
* fix: deprecate make_simple_namespace, fix Python 3.11

* docs: update links
2021-10-19 14:39:29 -04:00
Ralf W. Grosse-Kunstleve 7c580586f8
Correct options on Eigen::MappedSparseMatrix & adding MSVC C4127 suppression around Eigen includes. (#3352)
* Adding MSVC C4127 suppression around Eigen includes.

* For MSVC 2015 only: also adding the C4127 suppression to test_eigen.cpp

* Copying original change from PR #3343, with extra line breaks to not run past 99 columns (our desired but currently not enforced limit).
2021-10-11 13:13:01 -07:00
Bruce Merry 8a7c266d26
Fix make_key_iterator/make_value_iterator for prvalue iterators (#3348)
* Add a test showing a flaw in make_key_iterator/make_value_iterator

If the iterator dereference operator returns a value rather than a
reference (and that pair also does not *contain* references),
make_key_iterator and make_value_iterator will return a reference to a
temporary, causing a segfault.

* Fix make_key_iterator/make_value_iterator for prvalue iterators

If an iterator returns a pair<T1, T2> rather than a reference to a pair
or a pair of references, make_key_iterator and make_value_iterator would
return a reference to a temporary, typically leading to a segfault. This
is because the value category of member access to a prvalue is an
xvalue, not a prvalue, so decltype produces an rvalue reference type.
Fix the type calculation to handle this case.

I also removed some decltype parentheses that weren't needed, either
because the expression isn't one of the special cases for decltype or
because decltype was only used for SFINAE. Hopefully that makes the code
a bit more readable.

Closes #3347

* Attempt a workaround for nvcc
2021-10-11 08:35:39 -07:00
NaDDu 750e38dcfd
Update eval.h (#3344)
typo correction

pybind11/exec.h → pybind11/eval.h
2021-10-09 11:38:34 -04:00
Henry Schreiner ba9f919b85
chore: get back to work after 2.8.0 2021-10-04 17:37:27 -04:00
Henry Schreiner c9a319c607
chore: version 2.8.0 final 2021-10-04 15:33:58 -04:00
Aaron Gokaslan ad96655605
fix: replace free() with std::free() (#3321)
* Disambiguate free() to use std::free()

* Add cstdlib include
2021-10-03 20:15:37 -04:00
Bruce Merry b3573ac961
feat: add `.keys` and `.values` to bind_map (#3310)
* Add `.keys` and `.values` to bind_map

Both of these implement views (rather than just iterators), and `.items`
is also upgraded to a view. In practical terms, this allows a view to be
iterated multiple times and have its size taken, neither of which works
with an iterator.

The views implement `__len__`, `__iter__`, and the keys view implements
`__contains__`. Testing membership also works in item and value views
because Python falls back to iteration. This won't be optimal
for item values since it's linear rather than O(log n) or O(1), but I
didn't fancy trying to get all the corner cases to match Python
behaviour (tuple of wrong types, wrong length tuple, not a tuple etc).

Missing relative to Python dictionary views is `__reversed__` (only
added to Python in 3.8). Implementing that could break code that binds
custom map classes which don't provide `rbegin`/`rend` (at least without
doing clever things with SFINAE), so I've not tried.

The size increase on my system is 131072 bytes, which is rather large
(5%) but also suspiciously round (2^17) and makes me suspect some
quantisation effect.

* bind_map: support any object in __contains__

Add extra overload of `__contains__` (for both the map itself and
KeysView) which takes an arbitrary object and returns false.

* Take py::object by const reference in __contains__

To keep clang-tidy happy.

* Removing stray `py::` (detected via interactive testing in Google environment).

Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
2021-10-01 09:24:36 -04:00
Jeremy Maitin-Shepard 62c4909cce
Add `custom_type_setup` attribute (#3287)
* Fix `pybind11::object::operator=` to be safe if `*this` is accessible from Python

* Add `custom_type_setup` attribute

This allows for custom modifications to the PyHeapTypeObject prior to
calling `PyType_Ready`.  This may be used, for example, to define
`tp_traverse` and `tp_clear` functions.
2021-09-24 12:08:22 -07:00
Henry Schreiner 21282e645a
feat: reapply fixed version of #3271 (#3293)
* Add make_value_iterator (#3271)

* Add make_value_iterator

This is the counterpart to make_key_iterator, and will allow
implementing a `value` method in `bind_map` (although doing so is left
for a subsequent PR).

I made a few design changes to reduce copy-and-paste boilerplate.
Previously detail::iterator_state had a boolean template parameter to
indicate whether it was being used for make_iterator or
make_key_iterator. I replaced the boolean with a class that determines
how to dereference the iterator. This allows for a generic
implementation of `__next__`.

I also added the ValueType and Extra... parameters to the iterator_state
template args, because I think it was a bug that they were missing: if
make_iterator is called twice with different values of these, only the
first set has effect (because the state class is only registered once).
There is still a potential issue in that the *values* of the extra
arguments are latched on the first call, but since most policies are
empty classes this should be even less common.

* Add some remove_cv_t to appease clang-tidy

* Make iterator_access and friends take reference

For some reason I'd accidentally made it take a const value, which
caused some issues with third-party packages.

* Another attempt to remove remove_cv_t from iterators

Some of the return types were const (non-reference) types because of the
pecularities of decltype: `decltype((*it).first)` is the *declared* type
of the member of the pair, rather than the type of the expression. So if
the reference type of the iterator is `pair<const int, int> &`, then the
decltype is `const int`. Wrapping an extra set of parentheses to form
`decltype(((*it).first))` would instead give `const int &`.

This means that the existing make_key_iterator actually returns by value
from `__next__`, rather than by reference. Since for mapping types, keys
are always const, this probably hasn't been noticed, but it will affect
make_value_iterator if the Python code tries to mutate the returned
objects. I've changed things to use double parentheses so that
make_iterator, make_key_iterator and make_value_iterator should now all
return the reference type of the iterator. I'll still need to add a test
for that; for now I'm just checking whether I can keep Clang-Tidy happy.

* Add back some NOLINTNEXTLINE to appease Clang-Tidy

This is favoured over using remove_cv_t because in some cases a const
value return type is deliberate (particularly for Eigen).

* Add a unit test for iterator referencing

Ensure that make_iterator, make_key_iterator and make_value_iterator
return references to the container elements, rather than copies. The
test for make_key_iterator fails to compile on master, which gives me
confidence that this branch has fixed it.

* Make the iterator_access etc operator() const

I'm actually a little surprised it compiled at all given that the
operator() is called on a temporary, but I don't claim to fully
understand all the different value types in C++11.

* Attempt to work around compiler bugs

https://godbolt.org/ shows an example where ICC gets the wrong result
for a decltype used as the default for a template argument, and CI also
showed problems with PGI. This is a shot in the dark to see if it fixes
things.

* Make a test constructor explicit (Clang-Tidy)

* Fix unit test on GCC 4.8.5

It seems to require the arguments to the std::pair constructor to be
implicitly convertible to the types in the pair, rather than just
requiring is_constructible.

* Remove DOXYGEN_SHOULD_SKIP_THIS guards

Now that a complex decltype expression has been replaced by a simpler
nested type, I'm hoping Doxygen will be able to build it without issues.

* Add comment to explain iterator_state template params

* fix: regression in #3271

Co-authored-by: Bruce Merry <1963944+bmerry@users.noreply.github.com>
2021-09-23 15:06:07 -04:00
Jeremy Maitin-Shepard 2a78abffd8
Ensure PYBIND11_TLS_REPLACE_VALUE evaluates its arguments only once (#3290) 2021-09-23 13:36:25 -04:00
Henry Schreiner 2fa3fcfda5 Revert "Add make_value_iterator (#3271)"
This reverts commit ee0c5ee405.
2021-09-22 23:10:03 -04:00
Henry Schreiner b06a6f4f62
feat: Slice allowing None with py::object or std::optional (#1101)
* Adding nullptr slices

Using example from #1095

Some fixes from @wjakob's review

Stop clang-tidy from complaining

New proposal for py::slice constructor

Eric's suggested changes: simplify testing; shift def's

* chore: drop MSVC pragma (hopefully unneeded)

* Apply suggestions from code review
2021-09-22 17:41:56 -04:00
Bruce Merry ee0c5ee405
Add make_value_iterator (#3271)
* Add make_value_iterator

This is the counterpart to make_key_iterator, and will allow
implementing a `value` method in `bind_map` (although doing so is left
for a subsequent PR).

I made a few design changes to reduce copy-and-paste boilerplate.
Previously detail::iterator_state had a boolean template parameter to
indicate whether it was being used for make_iterator or
make_key_iterator. I replaced the boolean with a class that determines
how to dereference the iterator. This allows for a generic
implementation of `__next__`.

I also added the ValueType and Extra... parameters to the iterator_state
template args, because I think it was a bug that they were missing: if
make_iterator is called twice with different values of these, only the
first set has effect (because the state class is only registered once).
There is still a potential issue in that the *values* of the extra
arguments are latched on the first call, but since most policies are
empty classes this should be even less common.

* Add some remove_cv_t to appease clang-tidy

* Make iterator_access and friends take reference

For some reason I'd accidentally made it take a const value, which
caused some issues with third-party packages.

* Another attempt to remove remove_cv_t from iterators

Some of the return types were const (non-reference) types because of the
pecularities of decltype: `decltype((*it).first)` is the *declared* type
of the member of the pair, rather than the type of the expression. So if
the reference type of the iterator is `pair<const int, int> &`, then the
decltype is `const int`. Wrapping an extra set of parentheses to form
`decltype(((*it).first))` would instead give `const int &`.

This means that the existing make_key_iterator actually returns by value
from `__next__`, rather than by reference. Since for mapping types, keys
are always const, this probably hasn't been noticed, but it will affect
make_value_iterator if the Python code tries to mutate the returned
objects. I've changed things to use double parentheses so that
make_iterator, make_key_iterator and make_value_iterator should now all
return the reference type of the iterator. I'll still need to add a test
for that; for now I'm just checking whether I can keep Clang-Tidy happy.

* Add back some NOLINTNEXTLINE to appease Clang-Tidy

This is favoured over using remove_cv_t because in some cases a const
value return type is deliberate (particularly for Eigen).

* Add a unit test for iterator referencing

Ensure that make_iterator, make_key_iterator and make_value_iterator
return references to the container elements, rather than copies. The
test for make_key_iterator fails to compile on master, which gives me
confidence that this branch has fixed it.

* Make the iterator_access etc operator() const

I'm actually a little surprised it compiled at all given that the
operator() is called on a temporary, but I don't claim to fully
understand all the different value types in C++11.

* Attempt to work around compiler bugs

https://godbolt.org/ shows an example where ICC gets the wrong result
for a decltype used as the default for a template argument, and CI also
showed problems with PGI. This is a shot in the dark to see if it fixes
things.

* Make a test constructor explicit (Clang-Tidy)

* Fix unit test on GCC 4.8.5

It seems to require the arguments to the std::pair constructor to be
implicitly convertible to the types in the pair, rather than just
requiring is_constructible.

* Remove DOXYGEN_SHOULD_SKIP_THIS guards

Now that a complex decltype expression has been replaced by a simpler
nested type, I'm hoping Doxygen will be able to build it without issues.

* Add comment to explain iterator_state template params
2021-09-21 13:37:19 -04:00
Aaron Gokaslan d0f3c51f01
Enable defining custom __new__ (#3265)
* Enable defining custom __new__

* See if xfail needed

* Qualify auto self

* Unconditionally defining PYBIND11_DISABLE_NEW_STYLE_INIT_WARNING. Returning pointer from "__init__" instead of reference.

* Use new style __init__

* Simplify __new__ creation

* Reviewer suggestions

* Match indentation

Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
2021-09-20 10:42:14 -04:00
Jeremy Maitin-Shepard 14976c853b
Eliminate duplicate TLS keys for loader_life_support stack (#3275)
* Eliminate duplicate TLS keys for loader_life_support stack

This revises the existing fix for
https://github.com/pybind/pybind11/issues/2765 in
https://github.com/pybind/pybind11/pull/3237 to reduce the amount of
TLS storage used.

The shared TLS key is stored in two different ways, depending on
`PYBIND11_INTERNALS_VERSION`.  If `PYBIND11_INTERNALS_VERSION ==
4` (as is currently set), the TLS key is stored in the
`internal::shared_data` map to avoid breaking ABI compatibility.  If
`PYBIND11_INTERNALS_VERSION > 4`, the TLS key is stored directly in
the `internals` struct.

* Fix test_pytypes.py::test_issue2361 failure on PyPy3.7

* Add github actions tests for unstable ABI
2021-09-20 04:57:38 -07:00
Matthias Köppe e0031bfceb include/pybind11/numpy.h: gcc 4.8.4 does not have is_trivially_copyable (#3270) 2021-09-15 14:01:24 -07:00
Ralf W. Grosse-Kunstleve 6c65ab5950
Follow-on to PR #3254, to address user code breakages. (#3263)
* Restoring `const` removed from pytypes.h in PR #3254, adding tests reflective of user code that breaks when those `const` are removed.

* clang-tidy NOLINTs (and one collateral fix).

* Inserting PYBIND11_CONST_FOR_STRICT_PLATFORMS

* Trying `defined(__APPLE__)`

* Trying again: `auto it` for strict platforms.

* Adding NOLINTNEXTLINE(bugprone-macro-parentheses), expanding comments.

* Labeling all changes with `PR #3263`, for easy reference, and to make it easy to undo these changes if we decide to do so in the future.
2021-09-12 19:53:26 -07:00
Aaron Gokaslan 9978ed588b
Fix capsule bug (#3261)
Thanks Aaron for jumping in fixing this!
2021-09-10 11:23:32 -07:00
Laramie Leavitt 0e599589fe
Fix thread safety for pybind11 loader_life_support (#3237)
* Fix thread safety for pybind11 loader_life_support

Fixes issue: https://github.com/pybind/pybind11/issues/2765

This converts the vector of PyObjects to either a single void* or
a per-thread void* depending on the WITH_THREAD define.

The new field is used by each thread to construct a stack
of loader_life_support frames that can extend the life of python
objects.

The pointer is updated when the loader_life_support object is allocated
(which happens before a call) as well as on release.

Each loader_life_support maintains a set of PyObject references
that need to be lifetime extended; this is done by storing them
in a c++ std::unordered_set and clearing the references when the
method completes.

* Also update the internals version as the internal struct is no longer compatible

* Add test demonstrating threading works correctly.

It may be appropriate to run this under msan/tsan/etc.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update test to use lifetime-extended references rather than
std::string_view, as that's a C++ 17 feature.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Make loader_life_support members private

* Update version to dev2

* Update test to use python threading rather than concurrent.futures

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Remove unnecessary env in test

* Remove unnecessary pytest in test

* Use native C++ thread_local in place of python per-thread data structures to retain compatability

* clang-format test_thread.cpp

* Add a note about debugging the py::cast() error

* thread_test.py now propagates exceptions on join() calls.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* remove unused sys / merge

* Update include order in test_thread.cpp

* Remove spurious whitespace

* Update comment / whitespace.

* Address review comments

* lint cleanup

* Fix test IntStruct constructor.

* Add explicit to constructor

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
2021-09-10 12:29:21 -04:00
Ralf W. Grosse-Kunstleve 121b91f99c
Fixing NOLINT mishap (#3260)
* Removing NOLINT pointed out by Aaron.

* Removing another NOLINT.
2021-09-10 10:16:09 -04:00
Aaron Gokaslan ae07d4c6c6
maint(Clang-Tidy): readability-const-return (#3254)
* Enable clang-tidy readability-const-return

* PyTest functional

* Fix regression

* Fix actual regression

* Remove one more NOLINT

* Update comment
2021-09-09 21:27:36 -07:00
Jeremy Maitin-Shepard 4d5ad03e1f
Avoid use of temporary `bytes` object in string_caster for UTF-8 (#3257)
Fixes #3252
2021-09-09 15:56:10 -04:00
Aaron Gokaslan 4c6bee3514
fix: Set __file__ constant when using eval_file (#1300) (#3233)
* Set __file__ constant when using eval_file

* Use const ref

* Use a move instead

* Revert

* Improve test

* Guard test with Python version

* Fix tests

* Dont support Python2 API

* Drop Python2 eval __file__ support

* Hack

* Semisupport Python2

* Take2

* Remove Python2 support
2021-09-09 14:06:33 -04:00
Ralf W. Grosse-Kunstleve 6abf2baa62
CodeHealth: Enabling clang-tidy google-explicit-constructor (#3250)
* Adding google-explicit-constructor to .clang-tidy

* clang-tidy explicit attr.h (all automatic)

* clang-tidy explicit cast.h (all automatic)

* clang-tidy detail/init.h (1 NOLINT)

* clang-tidy detail/type_caster_base.h (2 NOLINT)

* clang-tidy pybind11.h (7 NOLINT)

* clang-tidy detail/common.h (3 NOLINT)

* clang-tidy detail/descr.h (2 NOLINT)

* clang-tidy pytypes.h (23 NOLINT, only 1 explicit)

* clang-tidy eigen.h (7 NOLINT, 0 explicit)

* Adding 2 explicit in functional.h

* Adding 4 explicit in iostream.h

* clang-tidy numpy.h (1 NOLINT, 1 explicit)

* clang-tidy embed.h (0 NOLINT, 1 explicit)

* clang-tidy tests/local_bindings.h (0 NOLINT, 4 explicit)

* clang-tidy tests/pybind11_cross_module_tests.cpp (0 NOLINT, 1 explicit)

* clang-tidy tests/pybind11_tests.h (0 NOLINT, 2 explicit)

* clang-tidy tests/test_buffers.cpp (0 NOLINT, 2 explicit)

* clang-tidy tests/test_builtin_casters.cpp (0 NOLINT, 4 explicit)

* clang-tidy tests/test_class.cpp (0 NOLINT, 6 explicit)

* clang-tidy tests/test_copy_move.cpp (0 NOLINT, 7 explicit)

* clang-tidy tests/test_embed/external_module.cpp (0 NOLINT, 1 explicit)

* clang-tidy tests/test_embed/test_interpreter.cpp (0 NOLINT, 1 explicit)

* clang-tidy tests/object.h (0 NOLINT, 2 explicit)

* clang-tidy batch of fully automatic fixes.

* Workaround for MSVC 19.16.27045.0 C++17 Python 2 C++ syntax error.
2021-09-08 18:53:38 -07:00
Aaron Gokaslan d71ba0cb73
(perf): Add a missing noexcept to a pytype constructor (#3236)
* Add a missing noexcept to pytypes constructor.

Adds a few missing noexcept to PyType constructors for perf reasons.

* Revert exception ctor
2021-09-02 15:18:42 -04:00
Tailing Yuan d6474ed7d2
fix: memory leak in cpp_function (#3228) (#3229)
* fix: memory leak in cpp_function (#3228)

* add a test case to check objects are deconstructed in cpp_function

* update the test case about cpp_function

* fix the test case about cpp_function: remove "noexcept"

* Actually calling func. CHECK(stat.alive() == 2); Manually verified that the new tests fails without the change in pybind11.h

* Moving new test to test_callbacks.cpp,py, with small enhancements.

* Removing new test from test_interpreter.cpp (after it was moved to test_callbacks.cpp,py). This restores test_interpreter.cpp to the current state on master.

* Using py::detail::silence_unused_warnings(py_func); to make the intent clear.

Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
2021-08-30 21:48:33 -04:00
Ralf W. Grosse-Kunstleve 777352fcd1
Adding `ssize_t_cast` to support passing `size_t` or `ssize_t` values where `ssize_t` is needed. (#3219)
* Trivial change to avoid (ssize_t) cast.

* Demo for safe_ssize_t idea.

* Removing safe_ssize_t.cpp (proof-of-concept code) to not upset the GHA Format workflow.

* Completing changes in pytypes.h

* New ssize_t_cast (better replacement for safe_ssize_t).

* clang-format-diff (no manual changes).

* bytes_ssize_t -Wnarrowing reproducer (see PR #2692).

* Backing out tuple(), list() ssize_t support, for compatibility with older compilers (to resolve link failures).

* Bug fix: missing `py::` for `py::ssize_t`

* Restoring tuple(), list() ssize_t support, but passing `size` by value, for compatibility with older compilers (to resolve link failures).

* Full test coverage of all functions with modified signatures.
2021-08-28 16:40:46 -07:00
Ye Zhihao cb60ed49e4
Fix enum value's __int__ returning non-int when underlying type is bool or of char type (#1334)
* Use equivalent_integer for enum's Scalar decision

* Add test for char underlying enum

* Support translating bool type in enum's Scalar

* Add test for bool underlying enum

* Fix comment in test

* Switch from `PYBIND11_CPP20` macro to `PYBIND11_HAS_U8STRING`

* Refine tests

Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
2021-08-26 14:34:24 -07:00
Dan 930bb16c79
Call PySys_SetArgv when initializing interpreter. (#2341)
* Call PySys_SetArgv when initializing interpreter.

* Document argc/argv parameters in initialize_interpreter.

* Remove manual memory management from set_interpreter_argv in favor of smart pointers.

* Use size_t for indexers in set_interpreter_argv.

* Minimize macros for flow control in set_interpreter_argv.

* Fix 'unused variable' warning on Py2

* whitespace

* Define wide_char_arg_deleter outside set_interpreter_argv.

* Do sys.path workaround in C++ rather than eval.

* Factor out wchar conversion to a separate function.

* Restore widened_argv variable declaration.

* Fix undeclared widened_arg variable on some paths.

* Use delete[] to match new wchar_t[].

* Fix compiler errors

* Use PY_VERSION_HEX for a cleaner CVE-2008-5983 mode check.

* Fix typo

* Use explicit type for deleter so delete[] works cross-compiler.

* Always use PySys_SetArgvEx because pybind11 doesn't support pythons that don't include it.

* Remove pointless ternary operator.

* Use unique_ptr.reset instead of a second initialization.

* Rename add_program_dir_to_path parameter to clarify intent.

* Add defined() check before evaluating HAVE_BROKEN_MBSTOWCS.

* Apply clang-tidy fixes

* Pre-commit

* refactor: use const for set_interpreter_argv

* Try to fix const issue and allocate vector properly

* fix: copy strings on Python 2

* Applying clang-format-diff relative to master.

The only manual change is an added empty line between pybind11 and system `#include`s.

```
git diff -U0 --no-color master | python3 $HOME/clone/llvm-project/clang/tools/clang-format/clang-format-diff.py -p1 -style=file -i
```

Co-authored-by: Boris Staletic <boris.staletic@gmail.com>
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
2021-08-26 14:12:54 -07:00
Nick Cullen 503ff2a6fb
view for numpy arrays (#987)
* reshape

* more tests

* Update numpy.h

* Update test_numpy_array.py

* array view

* test

* Update test_numpy_array.cpp

* Update numpy.h

* Update numpy.h

* Update test_numpy_array.cpp

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix merge bug

* Make clang-tidy happy

* Add xfail for PyPy

* Fix casting issue

* Fix formatting

* Apply clang-tidy

* Address reviews on additional tests

* Fix ordering

* Do a little more reordering

* Fix typo

* Try improving tests

* Fix error in reshape

* Add one more reshape test

* Fix bugs and add test

* Relax test

* streamlining new tests; removing a few stray msg

* Fix style revert

* Fix clang-tidy

* Misc tweaks:
* Comment: matching style in file (///), responsibility sentence, consistent punctuation.
* Replacing `unsigned char` with `uint8_t` for max consistency.
* Removing `1` from `array_view1` because there is only one.

* Partial clang-format-diff.

Co-authored-by: ncullen93 <ncullen.th@dartmouth.edu>
Co-authored-by: NC Cullen <nicholas.c.cullen.th@dartmouth.edu>
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ralf Grosse-Kunstleve <rwgk@google.com>
2021-08-26 14:11:01 -07:00
Nick Cullen 59ad1e7d05
reshape for numpy arrays (#984)
* reshape

* more tests

* Update numpy.h

* Update test_numpy_array.py

* Update numpy.h

* Update numpy.h

* Update test_numpy_array.cpp

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix merge bug

* Make clang-tidy happy

* Add xfail for PyPy

* Fix casting issue

* Address reviews on additional tests

* Fix ordering

* Do a little more reordering

* Fix typo

* Try improving tests

* Fix error in reshape

* Add one more reshape test

* streamlining new tests; removing a few stray msg

Co-authored-by: ncullen93 <ncullen.th@dartmouth.edu>
Co-authored-by: NC Cullen <nicholas.c.cullen.th@dartmouth.edu>
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ralf Grosse-Kunstleve <rwgk@google.com>
2021-08-26 08:12:35 -07:00
Jouke Witteveen 031a700dfd
Add make_simple_namespace function and tests (#2840)
Co-authored-by: Jouke Witteveen <j.witteveen@cosine.nl>
2021-08-26 08:04:22 -07:00
Ralf W. Grosse-Kunstleve c8ce4b8df8
Clone of @virtuald's PR #2112 with minor enhancements. (#3215)
* Add py::raise_from to enable chaining exceptions on Python 3.3+

* Use 'raise from' in initialization

* Documenting the exact base version of _PyErr_FormatVFromCause, adding back `assert`s.

Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com>
2021-08-23 17:30:01 -07:00
Aaron Gokaslan 6cbabc4b8c
maint(clang-tidy): Enable cpp-coreguideline slicing checks (#3210)
* maint(clang-tidy): add a clang-tidy slicing check

* Add self + touch up readme

* Fix typo
2021-08-23 18:42:19 -04:00
Ralf W. Grosse-Kunstleve 998d45e431
Cleanup of file-scoped and globally-scoped warning suppression pragmas across pybind11 header files. (#3201)
* Removing all MSVC C4127 warning suppression pragmas.

* Removing MSVC /WX (WERROR). To get a full list of all warnings.

* Inserting PYBIND11_SILENCE_MSVC_C4127. Changing one runtime if to #if.

* Changing PYBIND11_SILENCE_MSVC_C4127 macro to use absolute namespace (for use outside pybind11 include directory).

* Restoring MSVC /WX (WERROR).

* Removing globally-scoped suppression for clang -Wunsequenced. Based on an experiment under PR #3202 it is obsolete and can simply be removed.
2021-08-19 11:37:04 -07:00
Ralf W. Grosse-Kunstleve 774b5ff90b
Removing obsolete eigen.h warning suppression pragmas. (#3198)
* Removing all pragma from eigen.h

* Removing -Werror or equivalent from tests/CMakeLists.txt

* Restoring tests/CMakeLists.txt from master.

* Adding 4 PYBIND11_SILENCE_MSVC_C4127.

* Compatibility with -Wconversion, -Wdeprecated

* Introducing PYBIND11_COMPATIBILITY_WDEPRECATED_COPY

* Systematically using --verbose for compilations where possible (cmake 3.14 or newer).

Also changing all `cmake -t` to `--target`, `-v` to `--verbose`, `check` to `pytest`, for consistency (to make it easier to pin-point all commands of a certain type).

Also removing one `-j 2` for `pytest` in hopes of reducing flakes due to races in test_iostream and in prints from destructors.

* Commenting out pragmas as an experiment to reproduce previous observation.

* Removing all (newly added, but already commented-out) pragma code, adding HINT use -isystem (as cmake does).

* Restoring ci.yml from master. Those changes are better handled separately. BTW: in the last CI run there was still a test_iostream flake, even without the -j 2 for running the tests (verfied by inspecting the log).
2021-08-17 16:49:39 -07:00
Aaron Gokaslan 617cb653ec
[Bugfix] Fix errant const methods (#3194)
* Fix errant const methods

* Remove NOLINT since clang-tidy is pretty conservative

* Missed one

* Fix a few more errors

* Add reviewer suggested comments

* Run clang-format
2021-08-14 12:25:54 -04:00
Ralf W. Grosse-Kunstleve 1bcd94c481
Removing last remnants of pragma block at the top of pybind11.h (#3186)
* Removing last remnants of pragma block at the top of pybind11.h, defaulting CUDA, GCC7, GCC8 to PYBIND11_NOINLINE_DISABLED, with the option to define PYBIND11_NOINLINE_FORCED.

* Unique SOSIZE prefix to make it easier to extract the sosizes from the GitHub logs.

* Commenting out PYBIND11_WERROR block, for noinline testing.

* Undoing accidental change.

* `#define PYBIND11_NOINLINE_FORCED`

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* `#define PYBIND11_NOINLINE_DISABLED`

* Going back to default (removing `#define PYBIND11_NOINLINE_DISABLED`).

* `#define PYBIND11_NOINLINE_FORCED`

* Undoing all changes releated to measuring sosizes.

* Rollback of PR #3030 (Working around Centos 8 failure).

* Disabling -Werror for GNU (experiment).

* Commenting out the entire `if(PYBIND11_WERROR)` again (although that is not expected to make a difference, but who knows what I am overlooking).

* Adding `-DCMAKE_BUILD_TYPE=Release`

* Undoing change to tests/CMakeLists.txt (uncommenting `if(PYBIND11_WERROR)` block).

* post `git rebase master -X theirs` fixups.

* Adding measurements to comment for `PYBIND11_NOINLINE_FORCED`.

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-08-14 07:41:27 -07:00
Ralf W. Grosse-Kunstleve 4c7e509fa4
PYBIND11_NOINLINE-related cleanup. (#3179)
* Removing pragma for GCC -Wattributes, fixing forward declarations.

* Introducing PYBIND11_NOINLINE_FWD to deal with CUDA, GCC7, GCC8.

* Updating PYBIND11_NOINLINE_DCL in Doxyfile.

* Trying noinline, noinline for {CUDA, GCC7, GCC8}

* Trying noinline, inline for {CUDA, GCC7, GCC8}

* Adding GCC -Wattributes `pragma` in 3 header files.

* Introducing PYBIND11_NOINLINE_GCC_PRAGMA_ATTRIBUTES_NEEDED, used in 9 header files.

* Removing ICC pragma 2196, to see if it is still needed.

* Trying noinline, noinline for ICC

* Trying noinline, inline for ICC

* Restoring ICC pragma 2196, introducing PYBIND11_NOINLINE_FORCED, defined for testing.

* Removing code accidentally left in (was for experimentation only).

* Removing one-time-test define.

* Removing PYBIND11_NOINLINE_FWD macro (after learning that it makes no sense).

* Testing with PYBIND11_NOINLINE_DISABLED. Minor non-functional enhancements.

* Removing #define PYBIND11_NOINLINE_DISABLED (test was successful).

* Removing PYBIND11_NOINLINE_FORCED and enhancing comments for PYBIND11_NOINLINE.

* WIP stripping back

* Making -Wattributes pragma in pybind11 specific to GCC7, GCC8, CUDA.
2021-08-09 10:10:38 -07:00
Aaron Gokaslan ff590c1258
maint(perf): Optimize Numpy constructor to remove copies by value. (#3183)
* maint(perf): Optimize Numpy Constructor with additional std::move

* Add more moves
2021-08-09 12:48:27 -04:00
Ralf W. Grosse-Kunstleve af7007331a
Removing GCC -Wunused-but-set-parameter from pragma block at the top of pybind11.h (#3164)
* Cleanup triggered by work on pragma for GCC -Wunused-but-set-parameter.

* Backing out changes to eigen.h (to be worked on later).

* Adding PYBIND11_WORKAROUND_INCORRECT_GCC_UNUSED_BUT_SET_PARAMETER in type_caster_base.h (apparently needed only for older GCCs).

* Apparently older compilers need a simpler overload for silence_unused_warnings().

* clang C++11 compatibility: removing constexpr

* Special case for MSVC 2017: `constexpr void` return

* Trying again without the silence_unused_warnings(const int *) overload.

* Separate macros for ALL_GCC, OLD_GCC_UNUSED_BUT_SET_PARAMETER

* Changing to __GNUC__ <= 2 (turning off)

* Refined condition for PYBIND11_WORKAROUND_INCORRECT_OLD_GCC_UNUSED_BUT_SET_PARAMETER.

* Quick experiment trying out suggestion by @henryiii

* Introducing macro: PYBIND11_INT_ARRAY_WORKING_AROUND_MSVC_CLANG_ISSUES

* Trying henryiii@ (void) expander idea.

* fix: apply simpler expression with fewer workarounds

* Purging new-but-already-obsoleted  macro, made possible by @henryiii's commit.

* Renaming `ALL_GCC` macro back to just `GCC` (because there is no `OLD` anymore, luckily).

* [actions skip] Adding "All GCC versions" to comment, to be clear about it.

Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
2021-08-06 12:27:11 -07:00
Aaron Gokaslan 3893f37bce
maint(clang-tidy): Bugprone enable checks (#3166)
* Enable bugprone checks

* Reset delta and massage config

* Start to apply bugprone fixes

* try to fix minor bug

* Fix later

* Fix perfect forwarding bugprone

* Remove nolint

* undo constructor delete

* Fix bugprone-perfect-forwarding again

* Remove TODO

* Add another nolint for bugprone-exception-escape in scoped interpreter

* Fix remaining bugprone errors

* Properly apply bugprone-macro-parantheses

* Redo formatting and remove bugprone nolint

* Add coment and revert more whitespace changes

* Fix typo

* Fix parsing bug

* Add back comma

* Fix clang-tidy issue

* Apply remaining clang-tidy fixes
2021-08-06 14:30:28 -04:00
Henry Schreiner 089328f779 Revert "fix: apply simpler expression with fewer workarounds"
This reverts commit 1fafd1b447.
2021-08-06 13:09:48 -04:00
Henry Schreiner 1fafd1b447
fix: apply simpler expression with fewer workarounds 2021-08-06 13:03:26 -04:00
Henry Schreiner 5f4d725918
fix: version number hex 2021-08-03 17:03:11 -04:00
Henry Schreiner c30f57d2ed
chore: start development for 2.8.0 2021-08-03 16:11:07 -04:00
Henry Schreiner 82adacb31d
fix: include hex version in bump 2021-08-03 15:20:23 -04:00
Henry Schreiner 5f34c42d70 chore: bump to version 2.7.1 2021-08-03 15:06:57 -04:00
Aaron Gokaslan c0756ccd93
fix: func_handle for rule of two (#3169)
* Fix func_handle for rule of two

* Apply reviewer suggestion
2021-08-03 13:15:48 -04:00
Ralf W. Grosse-Kunstleve dcbda8d7ff
Removing MSVC C4127 from pragma block at the top of pybind11.h (#3152)
* Removing pragma for 4127 (to see what is still broken with the latest code).

* Using new constexpr_bool() to suppress warning C4127.

* One missed case, Python 2 only.

* PYBIND11_SILENCE_MSVC_C4127 (more similar to the approach for C4100).
2021-07-30 11:25:29 -07:00
Ralf W. Grosse-Kunstleve e2573dc961
Moving pragma for MSVC warning C4505 from pybind11.h to existing list in detail/common.h (#3160)
* Moving pragma for C4505 from pybind11.h to existing list in detail/common.h.

* Removing 4 existing suppressions to 1. see what is still needed and 2. capture the MSVC messages.

* It turns out none of the 4 pragmas are needed anymore.
2021-07-30 10:51:50 -07:00
Jerome Robert 9e8a741baa
fix: Mingw64 corrected and add a CI job to test it (#3132)
* mingw64 platform string is like mingw_xxx not "mingw"

See https://github.com/msys2/MINGW-packages/blob/master/mingw-w64-python/0099-Change-the-get_platform-method-in-sysconfig-and-dist.patch

* Mingw: Do not dllexport exceptions

This is a fix for errors like:

D:/a/pybind11/pybind11/include/pybind11/detail/common.h:735:23: error: 'dllexport' implies default visibility, but 'class pybind11::builtin_exception' has already been declared with a different visibility
  735 | class PYBIND11_EXPORT builtin_exception : public std::runtime_error {
      |                       ^~~~~~~~~~~~~~~~~

* GHA: Test Mingw64 build

* fix: avoid thin binaries on mingw

* fix: drop lto on MinGW

* Mingw64: disable PYBIND11_DEPRECATED

It trigger many warnings for unknown reasons

Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
2021-07-30 13:48:41 -04:00
Ralf W. Grosse-Kunstleve b42597291f
Limiting pragma for ignoring GCC 7 -Wnoexcept-type to the scope of pybind11.h. (#3161)
* Moving pragma for ignoring -Wnoexcept-type to the one location where it is needed.

* Trying a second location.

* The previous commit worked (GitHub Actions green), but see the added comment about the dicy nature of -Wnoexcept-type ("if and only if").

* Applying reviewer suggestion.
2021-07-30 07:09:55 -07:00
Ralf W. Grosse-Kunstleve b72ca7d1bd
Removing MSVC C4100 from pragma block at the top of pybind11.h (#3150)
* Removing pragma for 4100 (to see what is still broken with the latest code).

* Adding --keep-going

* Revert "Adding --keep-going"

This reverts commit 1c844c6ffd07a6111b644811e7e3b0a50b9d44bb.

* Introducing PYBIND11_WORKAROUND_INCORRECT_MSVC_C4100.

* _MSC_VER <= 1916

* Replacing simple variadic function with variadic template (attempt to resolve MSVC 2017 failures).

* Preserving existing comment (moved from pybind11.h to detail/common.h).

* Adding blank lines for readability.
2021-07-28 17:01:21 -07:00
Ralf W. Grosse-Kunstleve f4721a7b44
Accommodating environments that define __STDC_WANT_LIB_EXT1__ even if __STDC_LIB_EXT1__ is not defined by the implementation. (#3151)
Follow-on to PR #3129.
2021-07-28 08:58:36 -07:00
Ralf W. Grosse-Kunstleve 2164c2e0e7
Removing __INTEL_COMPILER section from pragma block at the top of pybind11.h (#3135)
* Fixing `pragma warning pop` for `__INTEL_COMPILER`.

* Adding push/pop to 3 tests. Removing #878 from top of pybind11.h (it was/is only needed for 1 test).

* Trying again after CI failure, moving the push to the top of 2 tests.

* Trying more after CI failure, adding push/pop to pybind11_tests.h, constructor_stats.h.

* Moving ICC #2196 suppression to CMakeLists.txt

* Fixing condition for `pragma GCC diagnostic push` in pybind11.h

* Moving `pragma warning disable 2196` to common.h

* Revising #ifdef to be more conservative.

* Undoing insertion of notes that will hopefully soon be completely obsolete anyway.
2021-07-27 15:33:31 -07:00
Aaron Gokaslan 9beaa925db
maint(clang-tidy): Improve code readability with explicit boolean casts (#3148)
* maint(clang-tidy) Improve code readability

* Fix minor typos

* Revert optimization that removed test case

* Fix comment formatting

* Revert another optimization to repro an issue

* Remove make_unique since it C++14 and newer only

* eformat comments

* Fix unsignedness of comparison

* Update comment
2021-07-27 15:32:26 -07:00
David Hewitt a0b975965f
Allow python builtins to be used as callbacks (#1413)
* Allow python builtins to be used as callbacks

* Try to fix pypy segfault

* Add expected fail for PyPy

* Fix typo

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add more info to xfail

* Add env

* Try returning false

* Try removing the move for pypy

* Fix bugs

* Try removing move

* Just keep ignoring for PyPy

* Add back xfail

* Fix ctors

* Revert change of std::move

* Change to skip

* Fix bug and edit comments

* Remove clang-tidy bugprone fix skip bug

Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-07-27 14:16:28 -04:00
Ralf W. Grosse-Kunstleve a0f862d428
Removing MSVC C4800 from pragma block at the top of pybind11.h (#3141)
* Adding PYBIND11_COMPAT_BOOL_CAST to appease MSVC 2015 warning C4800.

* Replacing PYBIND11_COMPAT_BOOL_CAST with simpler != 0

* Extra parentheses (almost all compilers failed without these).
2021-07-26 13:26:36 -07:00
Ralf W. Grosse-Kunstleve 7904ba1a5c
Adding pragma warning(disable: 4522) for MSVC <= 2017. (#3142) 2021-07-26 12:02:50 -07:00
Ralf W. Grosse-Kunstleve ff97f101d9
Removing MSVC C4996 from pragma block at the top of pybind11.h (#3129)
* Removing MSVC C4996 from pragma block at the top of pybind11.h

* localtime_thread_safe, PYBIND11_COMPAT_STRDUP

* Adding #include <ctime> (attempt to fix MSVC 2015, 2017 errors).
2021-07-26 11:28:36 -07:00
jesse-sony d65edfb024
Feature/local exception translator (#2650)
* Create a module_internals struct

Since we now have two things that are going to be module local, it felt
correct to add a struct to manage them.

* Add local exception translators

These are added via the  register_local_exception_translator function
and are then applied before the global translators

* Add unit tests to show the local exception translator works

* Fix a bug in the unit test with the string value of KeyError

* Fix a formatting issue

* Rename registered_local_types_cpp()

Rename it to get_registered_local_types_cpp() to disambiguate from the
new member of module_internals

* Add additional comments to new local exception code path

* Add a register_local_exception function

* Add additional unit tests for register_local_exception

* Use get_local_internals like get_internals

* Update documentation for new local exception feature

* Add back a missing space

* Clean-up some issues in the docs

* Remove the code duplication when translating exceptions

Separated out the exception processing into a standalone function in the
details namespace.

Clean-up some comments as per PR notes as well

* Remove the code duplication in register_exception

* Cleanup some formatting things caught by clang-format

* Remove the templates from exception translators

But I added a using declaration to alias the type.

* Remove the extra local from local_internals variable names

* Add an extra explanatory comment to local_internals

* Fix a typo in the code
2021-07-21 05:22:18 -07:00
blacktea 6d5d4e738c
Move object in pop method of List. (#3116)
* Move item instead of copy.

* Make Clang 3.6 happy.

Co-authored-by: c99 <email@dummy.com>
2021-07-20 11:48:09 -04:00
Ralf W. Grosse-Kunstleve 34f587dd23
Removing all warning pragmas that have not effect. (#3127) 2021-07-17 08:54:31 -07:00
Henry Schreiner 74935f8d67
chore: post-release (#3128) 2021-07-17 11:50:42 -04:00
Henry Schreiner 65e95ea867
chore: bump to 2.7.0 (#3123) 2021-07-16 09:27:47 -04:00
Ralf W. Grosse-Kunstleve 4359e00b97
Introducing PYBIND11_VERSION_HEX (#3120)
* Introducing PYBIND11_VERSION_HEX (better late than never!)

* PYBIND11_VERSION_HEX consistency check in setup.py
2021-07-15 15:00:57 -07:00
Boris Staletic 5cd3750757
Enable -Wstrict-aliasing warning (#2816)
* Enable -Wstrict-aliasing warning

* Narrow down the scope of -Wstrict-aliasing

* Go home, MSVC, you're drunk

* Make sure "pragma GCC" is not executed on ICC

Co-authored-by: Yannick Jadoul <yannick.jadoul@belgacom.net>
2021-07-15 12:38:52 -04:00
Shane Loretz 7331d381af
Raise codec errors when casting to std::string (#2903)
* Raise codec errors when casting to std::string

Allow the codec's exception to be raised instead of RuntimeError when
casting from py::str to std::string.

PY2 allows ucs surrogates in UTF-8 conversion

Signed-off-by: Shane Loretz <sloretz@openrobotics.org>
Signed-off-by: Shane Loretz <sloretz@osrfoundation.org>

* Attempt to fix py2 error

* Revert all unicode literals

* Fixed

Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
2021-07-13 21:21:55 -07:00
Ralf W. Grosse-Kunstleve aca6c3ba37
* Removing stray semicolons (discovered by running clang-format v12 followed by tools/check-style.sh). (#3087)
* Manually moving `// NOLINT` comments so that clang-format does not move them to the wrong places.

* Manually reformatting comments related to `static_assert`s so that clang-format does not need two passes.

* Empty lines between #includes, to prevent clang-format from shuffling the order and thereby confusing MSVC 2015.

* git diff -U0 --no-color HEAD^ | python3 $HOME/clone/llvm-project/clang/tools/clang-format/clang-format-diff.py -p1 -style=file -i
2021-07-13 18:14:58 -07:00
jbarlow83 2b7985e548
Improve documentation of discard_as_unraisable() API (#2697)
* Improve documentation of discard_as_unraisable() API

* Update pytypes.h

Remove "the above"

* Update pytypes.h

Fix precommit error

Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
2021-07-13 15:32:56 -04:00
Aaron Gokaslan 25e470c57d
fix(clang-tidy): Add cppcoreguidelines-init-vars,slicing, and throw-by-value-catch-by-reference checks (#3094)
* clang-tidy: guard against more UB behavior

* Remove slicing check for now
2021-07-13 09:54:32 -04:00
Henry Schreiner f0a65c899c
docs(fix): spelling mistake in recent commit 2021-07-12 16:57:28 -04:00
Ralf W. Grosse-Kunstleve 7472d37a93
Adding iostream.h thread-safety documentation. (#2995)
* Adding iostream.h thread-safety documentation.

* Restoring `TestThread` code with added `std::lock_guard<std::mutex>`.

* Updating new comments to reflect new information.

* Fixing up `git rebase -X theirs` accidents.
2021-07-12 13:39:06 -07:00
Aaron Gokaslan b5357d1fa8
fix(clang-tidy): Enable clang-tidy else-after-return and redundant void checks (#3080)
* Enable clang-tidy else-after-return and redundant void checks

* Fix remaining else-after

* Address reviewer comments

* Fix indentation

* Rerun clang-tidy post merge
2021-07-09 06:45:53 -07:00
Ralf W. Grosse-Kunstleve 6d1b197b46
Splitting out pybind11/stl/filesystem.h. (#3077)
* Splitting out pybind11/stl/filesystem.h.

To solve breakages like: https://github.com/deepmind/open_spiel/runs/2999582108

Mostly following the suggestion here: https://github.com/pybind/pybind11/pull/2730#issuecomment-750507575

Except using pybind11/stl/filesystem.h instead of pybind11/stlfs.h, as decided via chat.

stl.h restored to the exact state before merging PR #2730 via:
```
git checkout 733f8de24f stl.h
```

* Properly including new stl subdirectory in pip wheel config.

This now passes interactively:
```
pytest tests/extra_python_package/
```

* iwyu cleanup.

iwyuh.py -c -std=c++17 -DPYBIND11_TEST_BOOST -Ipybind11/include -I/usr/include/python3.9 -I/usr/include/eigen3 include/pybind11/stl/filesystem.h

* Adding PYBIND11_HAS_FILESYSTEM_IS_OPTIONAL.

* Eliminating else after return.
2021-07-08 09:02:48 -07:00
Robert Haschke c090c8c409
Unify cast_error message thrown by [simple|unpacking]_collector (#3013)
* Unify cast_error message thrown by [simple|unpacking]_collector

simple_collector and unpacking_collector throw different error messages
when the casting of an argument failed: While the former mentions make_tuple(),
the latter emphasises the call argument (and its name/position).

* Consolidating "Unable to convert call argument" error reporting code to guarantee uniformity.

Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
2021-07-06 15:13:13 -07:00
luzpaz 8bee61b645
docs: fix various typos (#3075)
Found via `codespell -q 3 -L nd,ot,thist`
2021-07-04 19:58:35 -04:00
Dustin Spicuzza 6d4409466b
Check dict item accesses where it isn't already checked (#2863)
* Convert PyDict_GetXXX to internal error checking variant

* Check unlikely error return from PyDict_DelItemString
2021-07-02 07:02:33 -07:00
Antony Lee 5bcaaa0423
Add a std::filesystem::path <-> os.PathLike caster. (#2730) 2021-07-02 07:00:50 -07:00
cyy f067deb563
avoid unnecessary strlen (#3058) 2021-06-30 23:35:25 -07:00
jonathan-conder-sm 733f8de24f
Avoid string copy if possible when passing a Python object to std::ostream (#3042) 2021-06-30 22:19:14 -07:00
Ralf W. Grosse-Kunstleve fbae8f313b
pickle setstate: setattr __dict__ only if not empty (#2972)
* pickle setstate: setattr __dict__ only if not empty, to not force use of py::dynamic_attr() unnecessarily.

* Adding unit test.

* Clang 3.6 & 3.7 compatibility.

* PyPy compatibility.

* Minor iwyu fix, additional comment.

* Addressing reviewer requests.

* Applying clang-tidy suggested fixes.

* Adding check_dynamic_cast_SimpleCppDerived, related to issue #3062.
2021-06-30 12:34:32 -07:00
Cris Luengo 93e69191c1
fix: enable py::implicitly_convertible<py::none, ...> for py::class_-wrapped types (#3059)
* Allow casting from None to a custom object, closes #2778

* ci.yml patch from the smart_holder branch for full CI coverage.
2021-06-25 17:56:17 -07:00
Aaron Gokaslan dac74ebdf5
fix(clang-tidy): performance fixes applied in tests and CI (#3051)
* Initial fixes

* Whoops

* Finish clang-tidy manual fixes

* Add two missing fixes

* Revert

* Update clang-tidy

* Try to fix unreachable code error

* Move nolint comment

* Apply missing fix

* Don't override clang-tidy config

* Does this fix clang-tidy?

* Make all clang-tidy errors visible

* Add comments about NOLINTs and remove a few

* Fix typo
2021-06-22 12:11:54 -04:00
Aaron Gokaslan 3b30b0a51e
fix(clang-tidy): clang-tidy readability and misc fixes, like adding const (#3052)
* Enable and apply clang-tidy readability and misc fixes.

* Revert deprecated tester

* add space to tests/test_constants_and_functions.cpp
2021-06-21 10:37:48 -04:00
Aaron Gokaslan af6218ff78
fix(clang-tidy): Apply performance fixes from clang-tidy (#3046)
* Apply performance fixes from clang-tidy

* 2nd Round of Perf Optimizations

* 3rd round of fixes & handle false-positive

* Apply missing fix and clang-format

* Apply reviewer comment
2021-06-19 10:53:27 -07:00
Jack S. Hale 4c7697dbe9
Add const T to docstring generation. (#3020)
* Add const T to docstring generation.

* Change order.

* See if existing test triggers for a const type.

* Add tests.

* Fix test.

* Remove experiment.

* Reformat.

* More tests, checks run.

* Adding `test_fmt_desc_` prefix to new test functions.

* Using pytest.mark.parametrize to 1. condense test; 2. exercise all functions even if one fails; 3. be less platform-specific (e.g. C++ float is not necessarily float32).

Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
2021-06-08 11:56:45 -07:00
Yichen 3ac690b88b
Explicitly export exception types. (#2999)
* Set visibility of exceptions to default.

Co-authored-by: XZiar <czktc2007@gmail.com>

* add test

* update docs

* Skip failed test.
2021-05-27 08:00:18 -07:00
Pieter P 0c93a0f3fc
Fix Unicode support for ostream redirects (#2982)
* Crash when printing Unicode to redirected cout
Add failing tests

* Fix Unicode crashes redirected cout

* pythonbuf::utf8_remainder check end iterator

* Remove trailing whitespace and formatting iostream

* Avoid buffer overflow if ostream redirect races
This doesn't solve the actual race, but at least it now has a much lower
probability of reading past the end of the buffer even when data races
do occur.
2021-05-03 22:04:38 -07:00
mvoelkle-cern e08a58111d
Fix compilation with gcc < 5 (#2956)
When the user defines _GLIBCXX_USE_CXX11_ABI=0 to force the pre-c++11 ABI, numpy.h assumes that is_trivially_copyable is available.
It is not necessarily the case. This patch uses clang's feature detection instead.
The workaround is for certain libstdc++ versions, so the test should target these particular versions.
2021-04-19 13:53:57 -04:00