* Demonstrate issue with weakref constructor overloads
* Fix weakref constructor to convert on being passed a non-weakref object
* Improve on nonlocal-scoped variable in test_weakref
* Keep backwards-compatibility by introducing PYBIND11_OBJECT_CVT_DEFAULT macro
* Simplify test_weakref
* Always call PyNumber_Index when casting from Python to a C++ integral type, also pre-3.8
* Fixed on PyPy
* Simplify use of PyNumber_Index, following @rwgk's idea, and ignore warnings in >=3.8
* Reproduce mismatch between pre-3.8 and post-3.8 behavior on __index__ throwing TypeError
* Fix tests on 3.6 <= Python < 3.8
* No, I don't have an uninitialized variable
* Fix use of __index__ on Python 2
* Make types in test_int_convert more ~boring~ descriptive
* Force the builtin module key to be the correct type.
Previously it was always going to be a std::string which converted into
unicode. Python 2 appears to want module keys to be normal str types, so
this was breaking code that expected plain string types in the
builtins.keys() data structure
* Add a simple unit test to ensure all built-in keys are str
* Update the unit test so it will also run on pypy
* Run pre-commit.
Co-authored-by: Jesse Clemens <jesse.clemens@sony.com>
* CI: Intel icc/icpc via oneAPI
Add testing for Intel icc/icpc via the oneAPI images.
Intel oneAPI is in a late beta stage, currently shipping
oneAPI beta09 with ICC 20.2.
CI: Skip Interpreter Tests for Intel
Cannot find how to add this, neiter the package `libc6-dev` nor
`intel-oneapi-mkl-devel` help when installed to solve this:
```
-- Looking for C++ include pthread.h
-- Looking for C++ include pthread.h - not found
CMake Error at /__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:165 (message):
Could NOT find Threads (missing: Threads_FOUND)
Call Stack (most recent call first):
/__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:458 (_FPHSA_FAILURE_MESSAGE)
/__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindThreads.cmake:234 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
tests/test_embed/CMakeLists.txt:17 (find_package)
```
CI: libc6-dev from GCC for ICC
CI: Run bare metal for oneAPI
CI: Ubuntu 18.04 for oneAPI
CI: Intel +Catch -Eigen
CI: CMake from Apt (ICC tests)
CI: Replace Intel Py with GCC Py
CI: Intel w/o GCC's Eigen
CI: ICC with verbose make
[Debug] Find core dump
tests: use arg{} instead of arg() for Intel
tests: adding a few more missing {}
fix: sync with @tobiasleibner's branch
fix: try ubuntu 20-04
fix: drop exit 1
docs: Apply suggestions from code review
Co-authored-by: Tobias Leibner <tobias.leibner@googlemail.com>
Workaround for ICC enable_if issues
Another workaround for ICC's enable_if issues
fix error in previous commit
Disable one test for the Intel compiler in C++17 mode
Add back one instance of py::arg().noconvert()
Add NOLINT to fix clang-tidy check
Work around for ICC internal error in PYBIND11_EXPAND_SIDE_EFFECTS in C++17 mode
CI: Intel ICC with C++17
docs: pybind11/numpy.h does not require numpy at build time. (#2720)
This is nice enough to be mentioned explicitly in the docs.
docs: Update warning about Python 3.9.0 UB, now that 3.9.1 has been released (#2719)
Adjusting `type_caster<std::reference_wrapper<T>>` to support const/non-const propagation in `cast_op`. (#2705)
* Allow type_caster of std::reference_wrapper<T> to be the same as a native reference.
Before, both std::reference_wrapper<T> and std::reference_wrapper<const T> would
invoke cast_op<type>. This doesn't allow the type_caster<> specialization for T
to distinguish reference_wrapper types from value types.
After, the type_caster<> specialization invokes cast_op<type&>, which allows
reference_wrapper to behave in the same way as a native reference type.
* Add tests/examples for std::reference_wrapper<const T>
* Add tests which use mutable/immutable variants
This test is a chimera; it blends the pybind11 casters with a custom
pytype implementation that supports immutable and mutable calls.
In order to detect the immutable/mutable state, the cast_op needs
to propagate it, even through e.g. std::reference<const T>
Note: This is still a work in progress; some things are crashing,
which likely means that I have a refcounting bug or something else
missing.
* Add/finish tests that distinguish const& from &
Fixes the bugs in my custom python type implementation,
demonstrate test that requires const& and reference_wrapper<const T>
being treated differently from Non-const.
* Add passing a const to non-const method.
* Demonstrate non-const conversion of reference_wrapper in tests.
Apply formatting presubmit check.
* Fix build errors from presubmit checks.
* Try and fix a few more CI errors
* More CI fixes.
* More CI fixups.
* Try and get PyPy to work.
* Additional minor fixups. Getting close to CI green.
* More ci fixes?
* fix clang-tidy warnings from presubmit
* fix more clang-tidy warnings
* minor comment and consistency cleanups
* PyDECREF -> Py_DECREF
* copy/move constructors
* Resolve codereview comments
* more review comment fixes
* review comments: remove spurious &
* Make the test fail even when the static_assert is commented out.
This expands the test_freezable_type_caster a bit by:
1/ adding accessors .is_immutable and .addr to compare identity
from python.
2/ Changing the default cast_op of the type_caster<> specialization
to return a non-const value. In normal codepaths this is a reasonable
default.
3/ adding roundtrip variants to exercise the by reference, by pointer
and by reference_wrapper in all call paths. In conjunction with 2/, this
demonstrates the failure case of the existing std::reference_wrpper conversion,
which now loses const in a similar way that happens when using the default cast_op_type<>.
* apply presubmit formatting
* Revert inclusion of test_freezable_type_caster
There's some concern that this test is a bit unwieldly because of the use
of the raw <Python.h> functions. Removing for now.
* Add a test that validates const references propagation.
This test verifies that cast_op may be used to correctly detect
const reference types when used with std::reference_wrapper.
* mend
* Review comments based changes.
1. std::add_lvalue_reference<type> -> type&
2. Simplify the test a little more; we're never returning the ConstRefCaster
type so the class_ definition can be removed.
* formatted files again.
* Move const_ref_caster test to builtin_casters
* Review comments: use cast_op and adjust some comments.
* Simplify ConstRefCasted test
I like this version better as it moves the assertion that matters
back into python.
ci: drop pypy2 linux, PGI 20.7, add Python 10 dev (#2724)
* ci: drop pypy2 linux, add Python 10 dev
* ci: fix mistake
* ci: commented-out PGI 20.11, drop 20.7
fix: regression with installed pybind11 overriding local one (#2716)
* fix: regression with installed pybind11 overriding discovered one
Closes#2709
* docs: wording incorrect
style: remove redundant instance->owned = true (#2723)
which was just before set to True in instance->allocate_layout()
fix: also throw in the move-constructor added by the PYBIND11_OBJECT macro, after the argument has been moved-out (if necessary) (#2701)
Make args_are_all_* ICC workarounds unconditional
Disable test_aligned on Intel ICC
Fix test_aligned on Intel ICC
Skip test_python_alreadyset_in_destructor on Intel ICC
Fix test_aligned again
ICC CI: Downgrade pytest
pytest 6 does not capture the `discard_as_unraisable` stderr and
just writes a warning with its content instead.
* refactor: simpler Intel workaround, suggested by @laramiel
* fix: try version with impl to see if it is easier to compile
* docs: update README for ICC
Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
* Only allow integer type_caster to call __int__ or __index__ method when conversion is allowed
* Remove tests for __index__ as this seems to only be used to convert to int in 3.8+
* Take both `int` and `long` types into account for Python 2
* Add test_numpy_int_convert to assert tests currently fail, even though np.intc has an __index__ method
* Also consider __index__ as noconvert to a C++ integer
* New-style classes for Python 2.7; sigh
* Add some tests on types with custom __index__ method
* Ignore some tests in Python <3.8
* Update comment about conversion from np.float32 to C++ int
* Workaround difference between CPython and PyPy's different PyIndex_Check (unnoticed because we currently don't have PyPy >= 3.8)
* Avoid ICC segfault with py::arg()
* CI: Intel icc/icpc via oneAPI
Add testing for Intel icc/icpc via the oneAPI images.
Intel oneAPI is in a late beta stage, currently shipping
oneAPI beta09 with ICC 20.2.
* CI: Skip Interpreter Tests for Intel
Cannot find how to add this, neiter the package `libc6-dev` nor
`intel-oneapi-mkl-devel` help when installed to solve this:
```
-- Looking for C++ include pthread.h
-- Looking for C++ include pthread.h - not found
CMake Error at /__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:165 (message):
Could NOT find Threads (missing: Threads_FOUND)
Call Stack (most recent call first):
/__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:458 (_FPHSA_FAILURE_MESSAGE)
/__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindThreads.cmake:234 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
tests/test_embed/CMakeLists.txt:17 (find_package)
```
* CI: libc6-dev from GCC for ICC
* CI: Run bare metal for oneAPI
* CI: Ubuntu 18.04 for oneAPI
* CI: Intel +Catch -Eigen
* CI: CMake from Apt (ICC tests)
* CI: Replace Intel Py with GCC Py
* CI: Intel w/o GCC's Eigen
* CI: ICC with verbose make
* [Debug] Find core dump
* tests: use arg{} instead of arg() for Intel
* tests: adding a few more missing {}
* fix: sync with @tobiasleibner's branch
* fix: try ubuntu 20-04
* fix: drop exit 1
* style: clang tidy fix
* style: fix missing NOLINT
* ICC: Update Compiler Name
Changed upstream with the last oneAPI release.
* ICC CI: Downgrade pytest
pytest 6 does not capture the `discard_as_unraisable` stderr and
just writes a warning with its content instead.
* Use new test pinning requirements.txt
* tests: add notes about intel, cleanup
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
* Adding a valgrind build on debug Python 3.9
Co-authored-by: Boris Staletic <boris.staletic@gmail.com>
* Add Valgrind suppression files
- Introduce suppression file, populate it with a first suppression taken from CPython, and fix one leak in the tests
- Suppress leak in NumPy
- More clean tests!
- Tests with names a-e passing (except for test_buffer)
- Suppress multiprocessing errors
- Merge multiprocessing suppressions into other suppression files
- Numpy seems to be spelled with a big P
- Append single entry from valgrind-misc.supp to valgrind-python.supp, and make clear valgrind-python.supp is only CPython
Co-authored-by: Boris Staletic <boris.staletic@gmail.com>
* Enable test_virtual_functions with a workaround
* Add a memcheck cmake target
- Add a memcheck cmake target
- Reformat cmake
- Appease the formatting overlords - they are angry
- Format CMake valgrind target decently
* Update CI config to new action versions
* fix: separate memcheck from pytest
* ci: cleanup
* Merge Valgrind and other deadsnakes builds
Co-authored-by: Boris Staletic <boris.staletic@gmail.com>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
* Plug leaking function_record objects when exceptions are thrown
* Plug leak of strdup'ed strings in function_record
* Some extra comments about the function_record ownership dance
* Clean up the function_record better, in case of exceptions
* Demonstrate some extra function_record leaks
* Change DeleteStrings template argument to free_strings runtime argument in destruct(function_record *)
* Zero-state unique_function_record deleter object
* Clarify rvalue reference to unique_ptr parameter in initialize_generic
* Use push_back with const char * instead of emplace_back
* Ignore old-style __init__ warnings
* Simplify ignoreOldStyleInitWarnings with py::exec
* Only wrap single class_::defs to ignore DeprecationWarnings about old-style __init__
* Fix leak in the test_copy_move::test_move_fallback
* Fix leaking PyMethodDef in test_class::test_implicit_conversion_life_support
* Plumb leak in test_buffer, occuring when a mutable buffer is requested for a read-only object, and enable test_buffer.py
* Fix weird return_value_policy::reference in test_stl_binders, and enable those tests
* Cleanup nodelete holder objects in test_smart_ptr, and enable those tests
* Update pytest to 6.2.1 in tests/requirements.txt
* Pin pytest to last supported version for 3.5
* Suppress PytestUnraisableExceptionWarning and use sys.__unraisablehook__ instead of sys.unraisablehook
* Fix filterwarnings mark on old pytest and old Python versions
* Cleanup ignore_pytest_unraisable_warning decorator
* minor cleanup: fixing or silencing flake8 errors
* ci: lock CMake to non-Universal version
* Update .github/workflows/ci.yml
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
* Allow type_caster of std::reference_wrapper<T> to be the same as a native reference.
Before, both std::reference_wrapper<T> and std::reference_wrapper<const T> would
invoke cast_op<type>. This doesn't allow the type_caster<> specialization for T
to distinguish reference_wrapper types from value types.
After, the type_caster<> specialization invokes cast_op<type&>, which allows
reference_wrapper to behave in the same way as a native reference type.
* Add tests/examples for std::reference_wrapper<const T>
* Add tests which use mutable/immutable variants
This test is a chimera; it blends the pybind11 casters with a custom
pytype implementation that supports immutable and mutable calls.
In order to detect the immutable/mutable state, the cast_op needs
to propagate it, even through e.g. std::reference<const T>
Note: This is still a work in progress; some things are crashing,
which likely means that I have a refcounting bug or something else
missing.
* Add/finish tests that distinguish const& from &
Fixes the bugs in my custom python type implementation,
demonstrate test that requires const& and reference_wrapper<const T>
being treated differently from Non-const.
* Add passing a const to non-const method.
* Demonstrate non-const conversion of reference_wrapper in tests.
Apply formatting presubmit check.
* Fix build errors from presubmit checks.
* Try and fix a few more CI errors
* More CI fixes.
* More CI fixups.
* Try and get PyPy to work.
* Additional minor fixups. Getting close to CI green.
* More ci fixes?
* fix clang-tidy warnings from presubmit
* fix more clang-tidy warnings
* minor comment and consistency cleanups
* PyDECREF -> Py_DECREF
* copy/move constructors
* Resolve codereview comments
* more review comment fixes
* review comments: remove spurious &
* Make the test fail even when the static_assert is commented out.
This expands the test_freezable_type_caster a bit by:
1/ adding accessors .is_immutable and .addr to compare identity
from python.
2/ Changing the default cast_op of the type_caster<> specialization
to return a non-const value. In normal codepaths this is a reasonable
default.
3/ adding roundtrip variants to exercise the by reference, by pointer
and by reference_wrapper in all call paths. In conjunction with 2/, this
demonstrates the failure case of the existing std::reference_wrpper conversion,
which now loses const in a similar way that happens when using the default cast_op_type<>.
* apply presubmit formatting
* Revert inclusion of test_freezable_type_caster
There's some concern that this test is a bit unwieldly because of the use
of the raw <Python.h> functions. Removing for now.
* Add a test that validates const references propagation.
This test verifies that cast_op may be used to correctly detect
const reference types when used with std::reference_wrapper.
* mend
* Review comments based changes.
1. std::add_lvalue_reference<type> -> type&
2. Simplify the test a little more; we're never returning the ConstRefCaster
type so the class_ definition can be removed.
* formatted files again.
* Move const_ref_caster test to builtin_casters
* Review comments: use cast_op and adjust some comments.
* Simplify ConstRefCasted test
I like this version better as it moves the assertion that matters
back into python.
* Adding missing virtual destructors, to silence clang -Wnon-virtual-dtor warnings.
Tested with clang version 9.0.1-12 under an Ubuntu-like OS.
Originally discovered in the Google-internal environment.
* adding -Wnon-virtual-dtor for GNU|Intel|Clang
* demo kwarg with none(false)
* Reorder and extend tests for arg::none(false) in test_methods_and_attributes.py::test_accepts_none
* Fix arg::none() for keyword arguments
* Add changelog note
* Fix names of no_none_kw test functions
Co-authored-by: Yannick Jadoul <yannick.jadoul@belgacom.net>
* Demonstrate test_factory_constructors.py failure without functional changes from #2335
* Revert "Demonstrate test_factory_constructors.py failure without functional changes from #2335"
This reverts commit ca33a8021fc2a3617c3356b188796528f4594419.
* Fix test crash where registered Python type gets garbage collected
* Clean up some more internal structures when class objects go out of scope
* Reduce length of std::erase_if-in-C++20 comment
* Clean up code for cleaning up type internals
* Move cleaning up of type info in internals to tp_dealloc on pybind11_metaclass
* Deprecated public constructors of module
* Turn documentation comment of module_::add_object into valid doxygen documentation
* Move definition of PYBIND11_DETAIL_MODULE_STATIC_DEF and PYBIND11_DETAIL_MODULE_CREATE macros up
* Move detail::create_top_level_module to module_::create_extension_module, and unify Python 2 and 3 signature again
* Throw error_already_set if module creation fails in module_::create_extension_module
* Mention module_::create_extension_module in deprecation warning message of module_::module_
* Check scope's __dict__ instead of using hasattr when registering classes and exceptions, to allow registering the same name in a derived class scope
* Extend test_base_and_derived_nested_scope test
* Add tests on error being thrown registering duplicate classes
* Circumvent bug with combination of test_class.py::test_register_duplicate_class and test_factory_constructors.py::test_init_factory_alias
* tests: Don't run tests that often segfault
* tests: drop all cross module gil tests
* tests: try skipping all macOS Python 3.9 tests
* tests: drop macOS Python 3.9
* Fail on passing py::object with wrong Python type to py::object subclass using PYBIND11_OBJECT macro
* Split off test_non_converting_constructors from test_constructors
* Fix test_as_type, as py::type constructor now throws an error itself if the argument is not a type
* Replace tp_name access by pybind11::detail::get_fully_qualified_tp_name
* Move forward-declaration of get_fully_qualified_tp_name to detail/common.h
* Don't add the builtins module name in get_fully_qualified_tp_name for PyPy
* Add PYBIND11_BUILTINS_MODULE macro, and use it in get_fully_qualified_tp_name
* Remove code inside 'PYPY_VERSION_NUM < 0x06000000' preprocessor if branch
* fix: more cleanup
* Remove more references to PyPy 5.7 and 5.9 in the docs
* Update comment on PyUnicode_UTF* in PyPy
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
* tests: New test for ctypes buffers (pybind#2502)
* fix: fix buffer_info segfault on views with no stride (pybind11#2502)
* Explicit conversions in buffer_info to make clang happy (pybind#2502)
* Another explicit cast in buffer_info constructor for clang (pybind#2502)
* Simpler implementation of buffer_info constructor from Py_buffer.
* Move test_ctypes_buffer into test_buffers
* Comment on why view->strides may be NULL (and fix some whitespace)
* Use c_strides() instead of zero when view->strides is NULL.
c_strides and f_strides are moved from numpy.h (py::array)
to buffer_info.h (py::detail) so they can be used from the
buffer_info Py_buffer constructor.
* Increase ctypes buffer test coverage in test_buffers.
* Split ctypes tests and skip one which is broken in PyPy2.
* Allow function/functor passed to py::vectorize to return void
* Stealing @sizmailov's test and fixing unused argument warning
* Add missing std::move()
RVO doesn't work here because function return type is different from
actual returned type
* remove extra EOL
* docs: add a few details
* chore: pre-commit autoupdate
* Remove array_iterator, array_begin, and array_end (in detail namespace)
Co-authored-by: Sergei Izmailov <sergei.a.izmailov@gmail.com>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
* Add tests demonstrating the problem with deregistering pybind11 instances
* Fix deregistering of different pybind11 instance from internals
Co-authored-by: Yannick Jadoul <yannick.jadoul@belgacom.net>
Co-authored-by: Blistic <wots_wot@hotmail.com>
This changes enum reprs to look like `<Enum.name: value>` similarly to
the Python enum module.
This keeps the str of enums as `Enum.name`, like the Python enum module.
* feat: setup.py redesign and helpers
* refactor: simpler design with two outputs
* refactor: helper file update and Windows support
* fix: review points from @YannickJadoul
* refactor: fixes to naming and more docs
* feat: more customization points
* feat: add entry point pybind11-config
* refactor: Try Extension-focused method
* refactor: rename alt/inplace to global
* fix: allow usage with git modules, better docs
* feat: global as an extra (@YannickJadoul's suggestion)
* feat: single version location
* fix: remove the requirement that setuptools must be imported first
* fix: some review points from @wjacob
* fix: use .in, add procedure to docs
* refactor: avoid monkeypatch copy
* docs: minor typos corrected
* fix: minor points from @YannickJadoul
* fix: typo on Windows C++ mode
* fix: MSVC 15 update 3+ have c++14 flag
See <https://docs.microsoft.com/en-us/cpp/build/reference/std-specify-language-standard-version?view=vs-2019>
* docs: discuss making SDists by hand
* ci: use pep517.build instead of manual setup.py
* refactor: more comments from @YannickJadoul
* docs: updates from @ktbarrett
* fix: change to newly recommended tool instead of pep517.build
This was intended as a proof of concept; build seems to be the correct replacement.
See https://github.com/pypa/pep517/pull/83
* docs: updates from @wjakob
* refactor: dual version locations
* docs: typo spotted by @wjakob
* Wrap PYBIND11_OVERLOAD_NAME and PYBIND11_OVERLOAD_PURE_NAME in do { ... } while (false), and resolve trailing semicolon
* Deprecate PYBIND11_OVERLOAD_* and get_overload in favor of PYBIND11_OVERRIDE_* and get_override
* Correct erroneous usage of 'overload' instead of 'override' in the implementation and internals
* Fix tests to use non-deprecated PYBIND11_OVERRIDE_* macros
* Update docs to use override instead of overload where appropriate, and add warning about deprecated aliases
* Add semicolons to deprecated PYBIND11_OVERLOAD macros to match original behavior
* Remove deprecation of PYBIND11_OVERLOAD_* macros and get_overload
* Add note to changelog and upgrade guide
* feat: type<T>()
* refactor: using py::type as class
* refactor: py::object as base
* wip: tigher api
* refactor: fix conversion and limit API further
* docs: some added notes from @EricCousineau-TRI
* refactor: use py::type::of
* Added guards to the includes
Added new CI config
Added new trigger
Changed CI workflow name
Debug CI
Debug CI
Debug CI
Debug CI
Added flags fro PGI
Disable Eigen
Removed tests that fail
Uncomment lines
* fix: missing include
fix: minor style cleanup
tests: support skipping
ci: remove and tighten a bit
fix: try msvc workaround for pgic
* tests: split up prealoc tests
* fix: PGI compiler fix
* fix: PGI void_t only
* fix: try to appease nvcc
* ci: better ordering for slow tests
* ci: minor improvements to testing
* ci: Add NumPy to testing
* ci: Eigen generates CUDA warnings / PGI errors
* Added CentOS7 back for a moment
* Fix YAML
* ci: runs-on missing
* centos7 is missing pytest
* ci: use C++11 on CentOS 7
* ci: test something else
* Try just adding flags on CentOS 7
* fix: CentOS 7
* refactor: move include to shared location
* Added verbose flag
* Try to use system cmake3 on CI
* Try to use system cmake3 on CI, attempt2
* Try to use system cmake3 on CI, attempt3
* tests: not finding pytest should be a warning, not a fatal error
* tests: cleanup
* Weird issue?
* fix: final polish
Co-authored-by: Andrii Verbytskyi <andrii.verbytskyi@mpp.mpg.de>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Andrii Verbytskyi <averbyts@cern.ch>
* Add check if `str(handle)` correctly converted the object, and throw py::error_already_set if not
* Fix tests on Python 3
* Apply @rwgk's fixes to cherry-picked commits from #2392
The variables PYBIND11_HAS_OPTIONAL, PYBIND11_HAS_EXP_OPTIONAL, PYBIND11_HAS_VARIANT,
__clang__, __APPLE__ were not checked for defined in a minortity of instances.
If the project using pybind11 sets -Wundef, the warnings will show.
The test build is also modified to catch the problem.
* fix: support nvcc and test
* fixup! fix: support nvcc and test
* docs: mention what compilers fail
* fix: much simpler logic
* refactor: slightly faster / clearer
* ci: harden chrono test, mark another macos 4.9 dev failure
This should help with a little of the flakiness seen with the timing test
* Update tests/test_chrono.py
* Can also fail
Adding missing `bytes` type to `test_constructors()`, to exercise the code change.
The changes in the PR were cherry-picked from PR #2409 (with a very minor
modification in test_pytypes.py related to flake8). Via PR #2409, these
changes were extensively tested in the Google environment, as summarized here:
https://docs.google.com/document/d/1TPL-J__mph_yHa1quDvsO12E_F5OZnvBaZlW9IIrz8M/
The changes in this PR did not cause an issues at all.
Note that `test_constructors()` before this PR passes for Python 2 only
because `pybind11::str` can hold `PyUnicodeObject` or `PyBytesObject`. As a
side-effect of this PR, `test_constructors()` no longer relies on this
permissive `pybind11::str` behavior. However, the permissive behavior is still
exercised/exposed via the existing `test_pybind11_str_raw_str()`.
The test code change is designed to enable easy removal later, when Python 2
support is dropped.
For completeness: confusingly, the non-test code changes travelled through PR
Example `ambiguous conversion` error fixed by this PR:
```
pybind11/tests/test_pytypes.cpp:214:23: error: ambiguous conversion for functional-style cast from 'pybind11::detail::item_accessor' (aka 'accessor<accessor_policies::generic_item>') to 'py::bytes'
"bytes"_a=py::bytes(d["bytes"]),
^~~~~~~~~~~~~~~~~~~~
pybind11/include/pybind11/detail/../pytypes.h:957:21: note: candidate constructor
PYBIND11_OBJECT(bytes, object, PYBIND11_BYTES_CHECK)
^
pybind11/include/pybind11/detail/../pytypes.h:957:21: note: candidate constructor
pybind11/include/pybind11/detail/../pytypes.h:987:15: note: candidate constructor
inline bytes::bytes(const pybind11::str &s) {
^
1 error generated.
```
* tests: keep source dir clean
* ci: make first build inplace
* ci: drop dev setting (wasn't doing anything)
* tests: warn if source directory is dirty
* docs: move helpers to .github where allowed
* docs: more guidelines in CONTRIBUTING
* chore: update issue templates
* fix: review from @bstaletic
* refactor: a few points from @rwgk
* docs: more touchup, review changes
* tests: refactor and cleanup
* refactor: more consistent
* tests: vendor six
* tests: more xfails, nicer system
* tests: simplify to info
* tests: suggestions from @YannickJadoul and @bstaletic
* tests: restore some pypy tests that now pass
* tests: rename info to env
* tests: strict False/True
* tests: drop explicit strict=True again
* tests: reduce minimum PyTest to 3.1
To deal with exceptions that hit destructors or other noexcept functions.
Includes fixes to support Python 2.7 and extends documentation on
error handling.
@virtuald and @YannickJadoul both contributed to this PR.
Important gain: uniformity & therefore easier cleanup when we drop PY2 support.
Very slight loss: it was nice to have `str is bytes` as a reminder in this specific context.
* Modified Vector STL bind initialization from a buffer type with optimization for simple arrays
* Add subtests to demonstrate processing Python buffer protocol objects with step > 1
* Fixed memoryview step test to only run on Python 3+
* Modified Vector constructor from buffer to return by value for readability
These tests will also alert us to any behavior changes across Python and PyPy versions.
Hardening tests in preparation for changing `pybind11::str` to only hold `PyUnicodeObject` (NOT also `bytes`). Note that this test exposes that `pybind11::str` can also hold `bytes`.
If the default argument value is a class, and not an instance of a
class, `a.value.attr("__repr__")` raises a `ValueError`. Switching to
`repr(a.value)` makes this use case work.
Fixes#2028
* Fix undefined memoryview format
* Add missing <algorithm> header
* Add workaround for py27 array compatibility
* Workaround py27 memoryview behavior
* Fix memoryview constructor from buffer_info
* Workaround PyMemoryView_FromMemory availability in py27
* Fix up memoryview tests
* Update memoryview test from buffer to check signedness
* Use static factory method to create memoryview
* Remove ndim arg from memoryview::frombuffer and add tests
* Allow ndim=0 memoryview and documentation fixup
* Use void* to align to frombuffer method signature
* Add const variants of frombuffer and frommemory
* Add memory view section in doc
* Fix docs
* Add test for null buffer
* Workaround py27 nullptr behavior in test
* Rename frombuffer to from_buffer
* Change NAMESPACE_BEGIN and NAMESPACE_END macros into PYBIND11_NAMESPACE_BEGIN and PYBIND11_NAMESPACE_END
* Fix sudden HomeBrew 'python not installed' error
* Sweep difference in 'Class.__init__() must be called when overriding __init__' error message between CPython and PyPy under the rug
* Homebrew updated to 3.8 yesterday.
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
* Error out eval_file
* Enable dynamic attribute support for Pypy >= 6
* Add a test for dynamic attribute support
* Skip test for eval_file on pypy
* Workaround for __qualname__ on PyPy3
* Add a PyPy3.6 7.3.0 build
* Only disable in PyPy3
* Fix travis testing
* No numpy and scipy for pypy
* Enable test on pypy2
* Fix logic in eval_file
* Skip a few tests due to bugs in PyPy
* scipy wheels are broken. make pypy2 a failrue
Co-authored-by: Andreas Kloeckner <inform@tiker.net>
This adds support for a `py::args_kw_only()` annotation that can be
specified between `py::arg` annotations to indicate that any following
arguments are keyword-only. This allows you to write:
m.def("f", [](int a, int b) { /* ... */ },
py::arg("a"), py::args_kw_only(), py::arg("b"));
and have it work like Python 3's:
def f(a, *, b):
# ...
with respect to how `a` and `b` arguments are accepted (that is, `a` can
be positional or by keyword; `b` can only be specified by keyword).
* test pair-copyability on C++17 upwards
The stdlib falsely detects containers like M=std::map<T, U>
as copyable, even when one of T and U is not copyable.
Therefore we cannot rely on the stdlib dismissing std::pair<T, M>
by itself, even on C++17.
* fix is_copy_assignable
bind_map used std::is_copy_assignable which suffers from the same problems
as std::is_copy_constructible, therefore the same fix has been applied.
* created tests for copyability
* fix: Avoid conversion to `int_` rhs argument of enum eq/ne
* test: compare unscoped enum with strings
* suppress comparison to None warning
* test unscoped enum arithmetic and comparision with unsupported type
* Make `overload_cast_impl` available in C++11 mode.
Narrow the scope of the `#if defined(PYBIND11_CPP14)` block around overload_cast to only
cover the parts where C++14 is stricly required. Thus, the implementation in
`pybind11::details::overload_cast_impl` is still available in C++11 mode.
* PR #1581: Modify test to use overload_cast_impl, update docs and change log
This is only necessary if `get_internals` is called for the first time in a given module when the running thread is in a GIL-released state.
Fixes#1364
* Test dtype field order in numpy dtype tests
When running tests with NumPy 1.14 or later this test exposes the
"invalid buffer descriptor" error reported in #1274.
* Create dtype_ptr with ordered fields
* Fix casting of time points with non-system-clock duration on Windows
Add explicit `time_point_cast` to time point with duration of system
clock. Fixes Visual Studio compile error.
* Add test case for custom time points casting
In def_readonly and def_readwrite, there is an assertion that the member comes
from the class or a base class:
static_assert(std::is_base_of<C, type>::value, "...");
However, if C and type are the same type, is_base_of will still only be true
if they are the same _non-union_ type. This means we can't define accessors
for the members of a union type because of this assertion.
Update the assertion to test
std::is_same<C, type>::value || std::is_base_of<C, type>::value
which will allow union types, or members of base classes.
Also add a basic unit test for accessing unions.
* Fix async Python functors invoking from multiple C++ threads (#1587)
Ensure GIL is held during functor destruction.
* Add async Python callbacks test that runs in separate Python thread
I found that the numpy array tests already contained an empty-shaped
array test, but none with data in it.
Following PEP 3118, scalars have an empty shape and ndim 0. This
works already and is now also documented/covered by a test.
* Fixing order of arguments in call to PyErr_GivenExceptionMatches in pybind11::error_already_set.matches
* Added tests on error_already_set::matches fix for exception base classes
* Fix warning that not including a cmake source or build dir will be a fatal error (it is now on newest CMakes)
* Fixes appveyor
* Travis uses CMake 3.9 for more than a year now
* Travis dropped sudo: false in December
* Dropping Sphinx 2
- clang7: Suppress self-assign warnings; fix missing virtual dtors
- pypy:
- Keep old version (newer stuff breaks)
- Pin packages to extra index for speed
- travis:
- Make docker explicit; remove docker if not needed
- Make commands more verbose (for debugging / repro)
- Make Ubuntu dist explicit per job
- Fix Windows
- Add names to travis
This avoids GIL deadlocking when pybind11 tries to acquire the GIL in a thread that already acquired it using standard Python API (e.g. when running from a Python thread).
* Adds std::deque to the types supported by list_caster in stl.h.
* Adds a new test_deque test in test_stl.{py,cpp}.
* Updates the documentation to include std::deque as a default
supported type.
* Check default holder
-Recognize "std::unique_ptr<T, D>" as a default holder even if "D" doesn't match between base and derived holders
* Add test for unique_ptr<T, D> change
Pybind11 provides a cast operator between opaque void* pointers on the
C++ side and capsules on the Python side. The py::cast<void *>
expression was not aware of this possibility and incorrectly triggered a
compile-time assertion ("Unable to cast type to reference: value is
local to type caster") that is now fixed.
* Support C++17 aligned new statement
This patch makes pybind11 aware of nonstandard alignment requirements in
bound types and passes on this information to C++17 aligned 'new'
operator. Pre-C++17, the behavior is unchanged.
This PR brings the std::array<> caster in sync with the other STL type
casters: to accept an arbitrary sequence as input (rather than a list,
which is too restrictive).
* Fix for Issue #1258
list_caster::load method will now check for a Python string and prevent its automatic conversion to a list.
This should fix the issue "pybind11/stl.h converts string to vector<string> #1258" (https://github.com/pybind/pybind11/issues/1258)
* Added tests for fix of issue #1258
* Changelog: stl string auto-conversion
* Fix potential crash when calling an overloaded function
The crash would occur if:
- dispatcher() uses two-pass logic (because the target is overloaded and some arguments support conversions)
- the first pass (with conversions disabled) doesn't find any matching overload
- the second pass does find a matching overload, but its return value can't be converted to Python
The code for formatting the error message assumed `it` still pointed to the selected overload,
but during the second-pass loop `it` was nullptr. Fix by setting `it` correctly if a second-pass
call returns a nullptr `handle`. Add a new test that segfaults without this fix.
* Make overload iteration const-correct so we don't have to iterate again on second-pass error
* Change test_error_after_conversions dependencies to local classes/variables
This commit addresses an inefficiency in how enums are created in
pybind11. Most of the enum_<> implementation is completely generic --
however, being a template class, it ended up instantiating vast amounts
of essentially identical code in larger projects with many enums.
This commit introduces a generic non-templated helper class that is
compatible with any kind of enumeration. enum_ then becomes a thin
wrapper around this new class.
The new enum_<> API is designed to be 100% compatible with the old one.
object_api::operator[] has a powerful overload for py::handle that can
accept slices, tuples (for NumPy), etc.
Lists, sequences, and tuples provide their own specialized operator[],
which unfortunately disables this functionality. This is accidental, and
the purpose of this commit is to re-enable the more general behavior.
This commit is tangentially related to the previous one in that it makes
py::handle/py::object et al. behave more like their Python counterparts.
This commit revamps the object_api class so that it maps most C++
operators to their Python analogs. This makes it possible to, e.g.
perform arithmetic using a py::int_ or py::array.
* check for already existing enum value added; added test
* added enum value name to exception message
* test for defining enum with multiple identical names moved to test_enum.cpp/py
This PR adds a new py::ellipsis() method which can be used in
conjunction with NumPy's generalized slicing support. For instance,
the following is now valid (where "a" is a NumPy array):
py::array b = a[py::make_tuple(0, py::ellipsis(), 0)];
Catch v2 changed the `run(...)` signature to take a `char *argv[]`,
arguing partly that technically a `char *argv[]` type is the correct
`main()` signature rather than `const char *argv[]`.
Dropping the `const` here doesn't appear to cause any problems with
catch v1 (tested against both the cmake-downloaded 1.9.3 and Debian's
1.12.1 package) so we can follow suit.
* stl.h: propagate return value policies to type-specific casters
Return value policies for containers like those handled in in 'stl.h'
are currently broken.
The problem is that detail::return_value_policy_override<C>::policy()
always returns 'move' when given a non-pointer/reference type, e.g.
'std::vector<...>'.
This is sensible behavior for custom types that are exposed via
'py::class_<>', but it does not make sense for types that are handled by
other type casters (STL containers, Eigen matrices, etc.).
This commit changes the behavior so that
detail::return_value_policy_override only becomes active when the type
caster derives from type_caster_generic.
Furthermore, the override logic is called recursively in STL type
casters to enable key/value-specific behavior.
* Add basic support for tag-based static polymorphism
Sometimes it is possible to look at a C++ object and know what its dynamic type is,
even if it doesn't use C++ polymorphism, because instances of the object and its
subclasses conform to some other mechanism for being self-describing; for example,
perhaps there's an enumerated "tag" or "kind" member in the base class that's always
set to an indication of the correct type. This might be done for performance reasons,
or to permit most-derived types to be trivially copyable. One of the most widely-known
examples is in LLVM: https://llvm.org/docs/HowToSetUpLLVMStyleRTTI.html
This PR permits pybind11 to be informed of such conventions via a new specializable
detail::polymorphic_type_hook<> template, which generalizes the previous logic for
determining the runtime type of an object based on C++ RTTI. Implementors provide
a way to map from a base class object to a const std::type_info* for the dynamic
type; pybind11 then uses this to ensure that casting a Base* to Python creates a
Python object that knows it's wrapping the appropriate sort of Derived.
There are a number of restrictions with this tag-based static polymorphism support
compared to pybind11's existing support for built-in C++ polymorphism:
- there is no support for this-pointer adjustment, so only single inheritance is permitted
- there is no way to make C++ code call new Python-provided subclasses
- when binding C++ classes that redefine a method in a subclass, the .def() must be
repeated in the binding for Python to know about the update
But these are not much of an issue in practice in many cases, the impact on the
complexity of pybind11's innards is minimal and localized, and the support for
automatic downcasting improves usability a great deal.
The property returns the enum_ value as a string.
For example:
>>> import module
>>> module.enum.VALUE
enum.VALUE
>>> str(module.enum.VALUE)
'enum.VALUE'
>>> module.enum.VALUE.name
'VALUE'
This is actually the equivalent of Boost.Python "name" property.
- PYBIND11_MAKE_OPAQUE now takes ... rather than a single argument and
expands it with __VA_ARGS__; this lets templated, comma-containing
types get through correctly.
- Adds a new macro PYBIND11_TYPE() that lets you pass the type into a
macro as a single argument, such as:
PYBIND11_OVERLOAD(PYBIND11_TYPE(R<1,2>), PYBIND11_TYPE(C<3,4>), func)
Unfortunately this only works for one macro call: to forward the
argument on to the next macro call (without the processor breaking it
up again) requires also adding the PYBIND11_TYPE(...) to type macro
arguments in the PYBIND11_OVERLOAD_... macro chain.
- updated the documentation with these two changes, and use them at a couple
places in the test suite to test that they work.
This fixes the test code on big-endian architectures: the array support
(PR #832) had hard-coded the little-endian '<' but we need to use '>' on
big-endian architectures.
This updates the `py::init` constructors to only use brace
initialization for aggregate initiailization if there is no constructor
with the given arguments.
This, in particular, fixes the regression in #1247 where the presence of
a `std::initializer_list<T>` constructor started being invoked for
constructor invocations in 2.2 even when there was a specific
constructor of the desired type.
The added test case demonstrates: without this change, it fails to
compile because the `.def(py::init<std::vector<int>>())` constructor
tries to invoke the `T(std::initializer_list<std::vector<int>>)`
constructor rather than the `T(std::vector<int>)` constructor.
By only using `new T{...}`-style construction when a `T(...)`
constructor doesn't exist, we should bypass this by while still allowing
`py::init<...>` to be used for aggregate type initialization (since such
types, by definition, don't have a user-declared constructor).
* Fix segfault when reloading interpreter with external modules
When embedding the interpreter and loading external modules in that
embedded interpreter, the external module correctly shares its
internals_ptr with the one in the embedded interpreter. When the
interpreter is shut down, however, only the `internals_ptr` local to
the embedded code is actually reset to nullptr: the external module
remains set.
The result is that loading an external pybind11 module, letting the
interpreter go through a finalize/initialize, then attempting to use
something in the external module fails because this external module is
still trying to use the old (destroyed) internals. This causes
undefined behaviour (typically a segfault).
This commit fixes it by adding a level of indirection in the internals
path, converting the local internals variable to `internals **` instead
of `internals *`. With this change, we can detect a stale internals
pointer and reload the internals pointer (either from a capsule or by
creating a new internals instance).
(No issue number: this was reported on gitter by @henryiii and @aoloe).
- UPDATEIFCOPY is deprecated, replaced with similar (but not identical)
WRITEBACKIFCOPY; trying to access the flag causes a deprecation
warning under numpy 1.14, so just check the new flag there.
- Numpy `repr` formatting of floats changed in 1.14.0 to `[1., 2., 3.]`
instead of the pre-1.14 `[ 1., 2., 3.]`. Updated the tests to
check for equality with the `repr(...)` value rather than the
hard-coded (and now version-dependent) string representation.
PEP8 indicates (correctly, IMO) that when an annotation is present, the
signature should include spaces around the equal sign, i.e.
def f(x: int = 1): ...
instead of
def f(x: int=1): ...
(in the latter case the equal appears to bind to the type, not to the
argument).
pybind11 signatures always includes a type annotation so we can always
add the spaces.
In the latest MSVC in C++17 mode including Eigen causes warnings:
warning C4996: 'std::unary_negate<_Fn>': warning STL4008: std::not1(),
std::not2(), std::unary_negate, and std::binary_negate are deprecated in
C++17. They are superseded by std::not_fn(). You can define
_SILENCE_CXX17_NEGATORS_DEPRECATION_WARNING or
_SILENCE_ALL_CXX17_DEPRECATION_WARNINGS to acknowledge that you have
received this warning.
This disables 4996 for the Eigen includes.
Catch generates a similar warning for std::uncaught_exception, so
disable the warning there, too.
In both cases this is temporary; we can (and should) remove the warnings
disabling once new upstream versions of Eigen and Catch are available
that address the warning. (The Catch one, in particular, looks to be
fixed in upstream master, so will probably be fixed in the next (2.0.2)
release).
Pybind11's default conversion to int always produces a long on Python 2 (`int`s and `long`s were unified in Python 3). This patch fixes `int` handling to match Python 2 on Python 2; for short types (`size_t` or smaller), the number will be returned as an `int` if possible, otherwise `long`. Requires Python 2.5+.
This is needed for things like `sys.exit`, which refuse to accept a `long`.
This commit turns on `-Wdeprecated` in the test suite and fixes several
associated deprecation warnings that show up as a result:
- in C++17 `static constexpr` members are implicitly inline; our
redeclaration (needed for C++11/14) is deprecated in C++17.
- various test suite classes have destructors and rely on implicit copy
constructors, but implicit copy constructor definitions when a
user-declared destructor is present was deprecated in C++11.
- Eigen also has various implicit copy constructors, so just disable
`-Wdeprecated` in `eigen.h`.
py::class_<T>'s `def_property` and `def_property_static` can now take a
`nullptr` as the getter to allow a write-only property to be established
(mirroring Python's `property()` built-in when `None` is given for the
getter).
This also updates properties to use the new nullptr constructor internally.
A few fixes related to how we set `__qualname__` and how we show the
type name in function signatures:
- `__qualname__` isn't supposed to have the module name at the
beginning, but we've been putting it there. This removes it, while
keeping the `Nested.Class` name chaining.
- print `__module__.__qualname__` rather than `type->tp_name`; the
latter doesn't work properly for nested classes, so we would get
`module.B` rather than `module.A.B` for a class `B` with parent `A`.
This also unifies the Python 3 and PyPy code. Fixes#1166.
- This now sets a `__qualname__` attribute on the type (as would happen
in Python 3.3+) for Python <3.3, including PyPy. While not particularly
important to have in earlier Python versions, it's useful for us to be
able to extracted the nested name, which is why `__qualname__` was
invented in the first place.
- Added tests for the above.
The just-updated flake8 package hits a bunch of:
E741 ambiguous variable name 'l'
warnings. This commit renames them all from `l` to `lst` (they are all
list values) to avoid the error.
- For the debian/buster docker build (GCC 7/C++17) install and use the
system `catch` package; this also renames "COMPILER_PACKAGES" to
"EXTRA_PACKAGES" since it now contains a non-compiler package.
- Add a status message indicating the catch version being used for
compiling the embedded tests
- Simplify some bash code by using VAR+=" foo" to append (rather than
VAR="${VAR} foo"
- Fix CMAKE_INCLUDE_PATH appending: it was prepending the ':' but not
the existing $CMAKE_INCLUDE_PATH value and so would end up with
":/eigen-path" if CMAKE_INCLUDE_PATH was already set. (This wasn't
bug that was actually noticed since currently nothing else sets it).
This fixes a bug introduced in b68959e822
when passing in a two-dimensional, but conformable, array as the value
for a compile-time Eigen vector (such as VectorXd or RowVectorXd). The
commit switched to using numpy to copy into the eigen data, but this
broke the described case because numpy refuses to broadcast a (N,1)
into a (N).
This commit fixes it by squeezing the input array whenever the output
array is 1-dimensional, which will let the problematic case through.
(This shouldn't squeeze inappropriately as dimension compatibility is
already checked for conformability before getting to the copy code).
This changes the caster to return a reference to a (new) local `CharT`
type caster member so that binding lvalue-reference char arguments
works (currently it results in a compilation failure).
Fixes#1116
This also matches the Eigen example for the row-major case.
This also enhances one of the tests to trigger a failure (and fixes it in the PR). (This isn't really a flaw in pybind itself, but rather fixes wrong code in the test code and docs).
The current C++14 constexpr signatures don't require relaxed constexpr,
but only `auto` return type deduction. To get around this in C++11,
the type caster's `name()` static member functions are turned into
`static constexpr auto` variables.
E.g. trying to convert a `list` to a `std::vector<int>` without
including <pybind11/stl.h> will now raise an error with a note that
suggests checking the headers.
The note is only appended if `std::` is found in the function
signature. This should only be the case when a header is missing.
E.g. when stl.h is included, the signature would contain `List[int]`
instead of `std::vector<int>` while using stl_bind.h would produce
something like `MyVector`. Similarly for `std::map`/`Dict`, `complex`,
`std::function`/`Callable`, etc.
There's a possibility for false positives, but it's pretty low.
To avoid an ODR violation in the test suite while testing
both `stl.h` and `std_bind.h` with `std::vector<bool>`,
the `py::bind_vector<std::vector<bool>>` test is moved to
the secondary module (which does not include `stl.h`).
There are two separate additions:
1. `py::hash(obj)` is equivalent to the Python `hash(obj)`.
2. `.def(hash(py::self))` registers the hash function defined by
`std::hash<T>` as the Python hash function.
The lookup of the `self` type and value pointer are moved out of
template code and into `dispatcher`. This brings down the binary
size of constructors back to the level of the old placement-new
approach. (It also avoids a second lookup for `init_instance`.)
With this implementation, mixing old- and new-style constructors
in the same overload set may result in some runtime overhead for
temporary allocations/deallocations, but this should be fine as
old style constructors are phased out.
Creating an instance of of a pybind11-bound type caused a reference leak in the
associated Python type object, which could prevent these from being collected
upon interpreter shutdown. This commit fixes that issue for all types that are
defined in a scope (e.g. a module). Unscoped anonymous types (e.g. custom
iterator types) always retain a positive reference count to prevent their
collection.
The `latest` build remains as is, but all others are modified to:
* Use regular Python instead of conda. `pip install` is much faster
than conda, but scipy isn't available. Numpy is still tested.
* Compile in debug mode instead of release.
* Skip CMake build tests. For some reason, CMake configuration is very
slow on AppVeyor and these tests are almost entirely CMake.
The changes reduce build time to about 1/3 of the original. The `latest`
config still covers scipy, release mode and the CMake build tests, so
the others don't need to.
The main point of `py::module_local` is to make the C++ -> Python cast
unique so that returning/casting a C++ instance is well-defined.
Unfortunately it also makes loading unique, but this isn't particularly
desirable: when an instance contains `Type` instance there's no reason
it shouldn't be possible to pass that instance to a bound function
taking a `Type` parameter, even if that function is in another module.
This commit solves the issue by allowing foreign module (and global)
type loaders have a chance to load the value if the local module loader
fails. The implementation here does this by storing a module-local
loading function in a capsule in the python type, which we can then call
if the local (and possibly global, if the local type is masking a global
type) version doesn't work.
This reimplements the py::init<...> implementations using the various
functions added to support `py::init(...)`, and moves the implementing
structs into `detail/init.h` from `pybind11.h`. It doesn't simply use a
factory directly, as this is a very common case and implementation
without an extra lambda call is a small but useful optimization.
This, combined with the previous lazy initialization, also avoids
needing placement new for `py::init<...>()` construction: such
construction now occurs via an ordinary `new Type(...)`.
A consequence of this is that it also fixes a potential bug when using
multiple inheritance from Python: it was very easy to write classes
that double-initialize an existing instance which had the potential to
leak for non-pod classes. With the new implementation, an attempt to
call `__init__` on an already-initialized object is now ignored. (This
was already done in the previous commit for factory constructors).
This change exposed a few warnings (fixed here) from deleting a pointer
to a base class with virtual functions but without a virtual destructor.
These look like legitimate warnings that we shouldn't suppress; this
adds virtual destructors to the appropriate classes.
This allows you to use:
cls.def(py::init(&factory_function));
where `factory_function` returns a pointer, holder, or value of the
class type (or a derived type). Various compile-time checks
(static_asserts) are performed to ensure the function is valid, and
various run-time type checks where necessary.
Some other details of this feature:
- The `py::init` name doesn't conflict with the templated no-argument
`py::init<...>()`, but keeps the naming consistent: the existing
templated, no-argument one wraps constructors, the no-template,
function-argument one wraps factory functions.
- If returning a CppClass (whether by value or pointer) when an CppAlias
is required (i.e. python-side inheritance and a declared alias), a
dynamic_cast to the alias is attempted (for the pointer version); if
it fails, or if returned by value, an Alias(Class &&) constructor
is invoked. If this constructor doesn't exist, a runtime error occurs.
- for holder returns when an alias is required, we try a dynamic_cast of
the wrapped pointer to the alias to see if it is already an alias
instance; if it isn't, we raise an error.
- `py::init(class_factory, alias_factory)` is also available that takes
two factories: the first is called when an alias is not needed, the
second when it is.
- Reimplement factory instance clearing. The previous implementation
failed under python-side multiple inheritance: *each* inherited
type's factory init would clear the instance instead of only setting
its own type value. The new implementation here clears just the
relevant value pointer.
- dealloc is updated to explicitly set the leftover value pointer to
nullptr and the `holder_constructed` flag to false so that it can be
used to clear preallocated value without needing to rebuild the
instance internals data.
- Added various tests to test out new allocation/deallocation code.
- With preallocation now done lazily, init factory holders can
completely avoid the extra overhead of needing an extra
allocation/deallocation.
- Updated documentation to make factory constructors the default
advanced constructor style.
- If an `__init__` is called a second time, we have two choices: we can
throw away the first instance, replacing it with the second; or we can
ignore the second call. The latter is slightly easier, so do that.
An alias can be used for two main purposes: to override virtual methods,
and to add some extra data to a class needed for the pybind-wrapper.
Both of these absolutely require that the wrapped class be polymorphic
so that virtual dispatch and destruction, respectively, works.
In C++11 mode, `boost::apply_visitor` requires an explicit `result_type`.
This also adds optional tests for `boost::variant` in C++11/14, if boost
is available. In C++17 mode, `std::variant` is tested instead.
This udpates all the remaining tests to the new test suite code and
comment styles started in #898. For the most part, the test coverage
here is unchanged, with a few minor exceptions as noted below.
- test_constants_and_functions: this adds more overload tests with
overloads with different number of arguments for more comprehensive
overload_cast testing. The test style conversion broke the overload
tests under MSVC 2015, prompting the additional tests while looking
for a workaround.
- test_eigen: this dropped the unused functions `get_cm_corners` and
`get_cm_corners_const`--these same tests were duplicates of the same
things provided (and used) via ReturnTester methods.
- test_opaque_types: this test had a hidden dependence on ExampleMandA
which is now fixed by using the global UserType which suffices for the
relevant test.
- test_methods_and_attributes: this required some additions to UserType
to make it usable as a replacement for the test's previous SimpleType:
UserType gained a value mutator, and the `value` property is not
mutable (it was previously readonly). Some overload tests were also
added to better test overload_cast (as described above).
- test_numpy_array: removed the untemplated mutate_data/mutate_data_t:
the templated versions with an empty parameter pack expand to the same
thing.
- test_stl: this was already mostly in the new style; this just tweaks
things a bit, localizing a class, and adding some missing
`// test_whatever` comments.
- test_virtual_functions: like `test_stl`, this was mostly in the new
test style already, but needed some `// test_whatever` comments.
This commit also moves the inherited virtual example code to the end
of the file, after the main set of tests (since it is less important
than the other tests, and rather length); it also got renamed to
`test_inherited_virtuals` (from `test_inheriting_repeat`) because it
tests both inherited virtual approaches, not just the repeat approach.
Attempting to mix py::module_local and non-module_local classes results
in some unexpected/undesirable behaviour:
- if a class is registered non-local by some other module, a later
attempt to register it locally fails. It doesn't need to: it is
perfectly acceptable for the local registration to simply override
the external global registration.
- going the other way (i.e. module `A` registers a type `T` locally,
then `B` registers the same type `T` globally) causes a more serious
issue: `A.T`'s constructors no longer work because the `self` argument
gets converted to a `B.T`, which then fails to resolve.
Changing the cast precedence to prefer local over global fixes this and
makes it work more consistently, regardless of module load order.
This commit adds a `py::module_local` attribute that lets you confine a
registered type to the module (more technically, the shared object) in
which it is defined, by registering it with:
py::class_<C>(m, "C", py::module_local())
This will allow the same C++ class `C` to be registered in different
modules with independent sets of class definitions. On the Python side,
two such types will be completely distinct; on the C++ side, the C++
type resolves to a different Python type in each module.
This applies `py::module_local` automatically to `stl_bind.h` bindings
when the container value type looks like something global: i.e. when it
is a converting type (for example, when binding a `std::vector<int>`),
or when it is a registered type itself bound with `py::module_local`.
This should help resolve potential future conflicts (e.g. if two
completely unrelated modules both try to bind a `std::vector<int>`.
Users can override the automatic selection by adding a
`py::module_local()` or `py::module_local(false)`.
Note that this does mildly break backwards compatibility: bound stl
containers of basic types like `std::vector<int>` cannot be bound in one
module and returned in a different module. (This can be re-enabled with
`py::module_local(false)` as described above, but with the potential for
eventual load conflicts).
The builtin exception handler currently doesn't work across modules
under clang/libc++ for builtin pybind exceptions like
`pybind11::error_already_set` or `pybind11::stop_iteration`: under
RTLD_LOCAL module loading clang considers each module's exception
classes distinct types. This then means that the base exception
translator fails to catch the exceptions and the fall through to the
generic `std::exception` handler, which completely breaks things like
`stop_iteration`: only the `stop_iteration` of the first module loaded
actually works properly; later modules raise a RuntimeError with no
message when trying to invoke their iterators.
For example, two modules defined like this exhibit the behaviour under
clang++/libc++:
z1.cpp:
#include <pybind11/pybind11.h>
#include <pybind11/stl_bind.h>
namespace py = pybind11;
PYBIND11_MODULE(z1, m) {
py::bind_vector<std::vector<long>>(m, "IntVector");
}
z2.cpp:
#include <pybind11/pybind11.h>
#include <pybind11/stl_bind.h>
namespace py = pybind11;
PYBIND11_MODULE(z2, m) {
py::bind_vector<std::vector<double>>(m, "FloatVector");
}
Python:
import z1, z2
for i in z2.FloatVector():
pass
results in:
Traceback (most recent call last):
File "zs.py", line 2, in <module>
for i in z2.FloatVector():
RuntimeError
This commit fixes the issue by adding a new exception translator each
time the internals pointer is initialized from python builtins: this
generally means the internals data was initialized by some other
module. (The extra translator(s) are skipped under libstdc++).
This adds the infrastructure for a separate test plugin for cross-module
tests. (This commit contains no tests that actually use it, but the
following commits do; this is separated simply to provide a cleaner
commit history).
Currently types that are capable of conversion always call their convert
function when invoked with a `py::object` which is actually the correct
type. This means that code such as `py::cast<py::list>(obj)` and
`py::list l(obj.attr("list"))` make copies, which was an oversight
rather than an intentional feature.
While at first glance there might be something behind having
`py::list(obj)` make a copy (as it would in Python), this would be
inconsistent when you dig a little deeper because `py::list(l)`
*doesn't* make a copy for an existing `py::list l`, and having an
inconsistency within C++ would be worse than a C++ <-> Python
inconsistency.
It is possible to get around the copying using a
`reinterpret_borrow<list>(o)` (and this commit fixes one place, in
`embed.h`, that does so), but that seems a misuse of
`reinterpret_borrow`, which is really supposed to be just for dealing
with raw python-returned values, not `py::object`-derived wrappers which
are supposed to be higher level.
This changes the constructor of such converting types (i.e. anything
using PYBIND11_OBJECT_CVT -- `str`, `bool_`, `int_`, `float_`, `tuple`,
`dict`, `list`, `set`, `memoryview`) to reference rather than copy when
the check function passes.
It also adds an `object &&` constructor that is slightly more efficient
by avoiding an inc_ref when the check function passes.
`error_already_set` is more complicated than it needs to be, partly
because it manages reference counts itself rather than using
`py::object`, and partly because it tries to do more exception clearing
than is needed. This commit greatly simplifies it, and fixes#927.
Using `py::object` instead of `PyObject *` means we can rely on
implicit copy/move constructors.
The current logic did both a `PyErr_Clear` on deletion *and* a
`PyErr_Fetch` on creation. I can't see how the `PyErr_Clear` on
deletion is ever useful: the `Fetch` on creation itself clears the
error, so the only way doing a `PyErr_Clear` on deletion could do
anything if is some *other* exception was raised while the
`error_already_set` object was alive--but in that case, clearing some
other exception seems wrong. (Code that is worried about an exception
handler raising another exception would already catch a second
`error_already_set` from exception code).
The destructor itself called `clear()`, but `clear()` was a little bit
more paranoid that needed: it called `restore()` to restore the
currently captured error, but then immediately cleared it, using the
`PyErr_Restore` to release the references. That's unnecessary: it's
valid for us to release the references manually. This updates the code
to simply release the references on the three objects (preserving the
gil acquire).
`clear()`, however, also had the side effect of clearing the current
error, even if the current `error_already_set` didn't have a current
error (e.g. because of a previous `restore()` or `clear()` call). I
don't really see how clearing the error here can ever actually be
useful: the only way the current error could be set is if you called
`restore()` (in which case the current stored error-related members have
already been released), or if some *other* code raised the error, in
which case `clear()` on *this* object is clearing an error for which it
shouldn't be responsible.
Neither of those seem like intentional or desirable features, and
manually requesting deletion of the stored references similarly seems
pointless, so I've just made `clear()` an empty method and marked it
deprecated.
This also fixes a minor potential issue with the destruction: it is
technically possible for `value` to be null (though this seems likely to
be rare in practice); this updates the check to look at `type` which
will always be non-null for a `Fetch`ed exception.
This also adds error_already_set round-trip throw tests to the test
suite.
The instance registration for offset base types fails (under macOS, with
a segfault) in the presense of virtual base types. The issue occurs
when trying to `static_cast<Base *>(derived_ptr)` when `derived_ptr` has
been allocated (via `operator new`) but not initialized.
This commit fixes the issue by moving the addition to
`registered_instances` into `init_holder` rather than immediately after
value pointer allocation.
This also renames it to `init_instance` since it does more than holder
initialization now. (I also further renamed `init_holder_helper` to
`init_holder` since `init_holder` isn't used anymore).
Fixes#959.
This adds support for implicit conversions to bool from Python types
with `__bool__` (Python 3) or `__nonzero__` (Python 2) attributes, and
adds direct (i.e. non-converting) support for numpy bools.
If a class doesn't provide a `T::operator delete(void *)` but does have
a `T::operator delete(void *, size_t)` the latter is invoked by a
`delete someT`. Pybind currently only look for and call the former;
this commit adds detection and calling of the latter when the former
doesn't exist.
This changes the pointer `cast()` in `PYBIND11_TYPE_CASTER` to recognize
the `take_ownership` policy: if casting a pointer with take-ownership,
the `cast()` now recalls `cast()` with a dereferenced rvalue (rather
than the previous code, which was always calling it with a const lvalue
reference), and deletes the pointer after the chained `cast()` is
complete.
This makes code like:
m.def("f", []() { return new std::vector<int>(100, 1); },
py::return_value_policy::take_ownership);
do the expected thing by taking over ownership of the returned pointer
(which is deleted once the chained cast completes).
PR #936 broke the ability to return a pointer to a stl container (and,
likewise, to a tuple) because the added deduced type matched a
non-const pointer argument: the pointer-accepting `cast` in
PYBIND11_TYPE_CASTER had a `const type *`, which is a worse match for a
non-const pointer than the universal reference template #936 added.
This changes the provided TYPE_CASTER cast(ptr) to take the pointer by
template arg (so that it will accept either const or non-const pointer).
It has two other effects: it slightly reduces .so size (because many
type casters never actually need the pointer cast at all), and it allows
type casters to provide their untemplated pointer `cast()` that will
take precedence over the templated version provided in the macro.
In a Debug build, MSVC doesn't apply copy/move elision as often,
triggering a test failure. This relaxes the test count requirements
to let the test suite pass.
This updates the std::tuple, std::pair and `stl.h` type casters to
forward their contained value according to whether the container being
cast is an lvalue or rvalue reference. This fixes an issue where
subcaster casts were always called with a const lvalue which meant
nested type casters didn't have the desired `cast()` overload invoked.
For example, this caused Eigen values in a tuple to end up with a
readonly flag (issue #935) and made it impossible to return a container
of move-only types (issue #853).
This fixes both issues by adding templated universal reference `cast()`
methods to the various container types that forward container elements
according to the container reference type.
The std::pair caster can be written as a special case of the std::tuple
caster; this combines them via a base `tuple_caster` class (which is
essentially identical to the previous std::tuple caster).
This also removes the special empty tuple base case: returning an empty
tuple is relatively rare, and the base case still works perfectly well
even when the tuple types is an empty list.
When defining method from a member function pointer (e.g. `.def("f",
&Derived::f)`) we run into a problem if `&Derived::f` is actually
implemented in some base class `Base` when `Base` isn't
pybind-registered.
This happens because the class type is deduced from the member function
pointer, which then becomes a lambda with first argument this deduced
type. For a base class implementation, the deduced type is `Base`, not
`Derived`, and so we generate and registered an overload which takes a
`Base *` as first argument. Trying to call this fails if `Base` isn't
registered (e.g. because it's an implementation detail class that isn't
intended to be exposed to Python) because the type caster for an
unregistered type always fails.
This commit adds a `method_adaptor` function that rebinds a member
function to a derived type member function and otherwise (i.e. regular
functions/lambda) leaves the argument as-is. This is now used for class
definitions so that they are bound with type being registered rather
than a potential base type.
A closely related fix in this commit is to similarly update the lambdas
used for `def_readwrite` (and related) to bind to the class type being
registered rather than the deduced type so that registering a property
that resolves to a base class member similarly generates a usable
function.
Fixes#854, #910.
Co-Authored-By: Dean Moldovan <dean0x7d@gmail.com>
When casting to an unsigned type from a python 2 `int`, we currently
cast using `(unsigned long long) PyLong_AsUnsignedLong(src.ptr())`.
If the Python cast fails, it returns (unsigned long) -1, but then we
cast this to `unsigned long long`, which means we get 4294967295, but
because that isn't equal to `(unsigned long long) -1`, we don't detect
the failure.
This commit moves the unsigned casting into a `detail::as_unsigned`
function which, upon error, casts -1 to the final type, and otherwise
casts the return value to the final type to avoid the problematic double
cast when an error occurs.
The error most commonly shows up wherever `long` is 32-bits (e.g. under
both 32- and 64-bit Windows, and under 32-bit linux) when passing a
negative value to a bound function taking an `unsigned long`.
Fixes#929.
The added tests also trigger a latent segfault under PyPy: when casting
to an integer smaller than `long` (e.g. casting to a `uint32_t` on a
64-bit `long` architecture) we check both for a Python error and also
that the resulting intermediate value will fit in the final type. If
there is no conversion error, but we get a value that would overflow, we
end up calling `PyErr_ExceptionMatches()` illegally: that call is only
allowed when there is a current exception. Under PyPy, this segfaults
the test suite. It doesn't appear to segfault under CPython, but the
documentation suggests that it *could* do so. The fix is to only check
for the exception match if we actually got an error.
This fixes#856. Instead of the weakref trick, the internals structure
holds an unordered_map from PyObject* to a vector of references. To
avoid the cost of the unordered_map lookup for objects that don't have
any keep_alive patients, a flag is added to each instance to indicate
whether there is anything to do.
Fixes a race condition when multiple threads try to acquire the GIL
before `detail::internals` have been initialized. `gil_scoped_release`
is now tasked with initializing `internals` (guaranteed single-threaded)
to ensure the safety of subsequent `acquire` calls from multiple threads.
This commit allows multiple inheritance of pybind11 classes from
Python, e.g.
class MyType(Base1, Base2):
def __init__(self):
Base1.__init__(self)
Base2.__init__(self)
where Base1 and Base2 are pybind11-exported classes.
This requires collapsing the various builtin base objects
(pybind11_object_56, ...) introduced in 2.1 into a single
pybind11_object of a fixed size; this fixed size object allocates enough
space to contain either a simple object (one base class & small* holder
instance), or a pointer to a new allocation that can contain an
arbitrary number of base classes and holders, with holder size
unrestricted.
* "small" here means having a sizeof() of at most 2 pointers, which is
enough to fit unique_ptr (sizeof is 1 ptr) and shared_ptr (sizeof is 2
ptrs).
To minimize the performance impact, this repurposes
`internals::registered_types_py` to store a vector of pybind-registered
base types. For direct-use pybind types (e.g. the `PyA` for a C++ `A`)
this is simply storing the same thing as before, but now in a vector;
for Python-side inherited types, the map lets us avoid having to do a
base class traversal as long as we've seen the class before. The
change to vector is needed for multiple inheritance: Python types
inheriting from multiple registered bases have one entry per base.
Fixes#896.
From Python docs: "Once an iterator’s `__next__()` method raises
`StopIteration`, it must continue to do so on subsequent calls.
Implementations that do not obey this property are deemed broken."
Passing utf8 encoded strings from python to a C++ function taking a
std::string was broken. The previous version was trying to call
'PyUnicode_FromObject' on this data, which failed to convert the string
to unicode with the default ascii codec. Also this incurs an unnecessary
conversion to unicode for data this is immediately converted back to
utf8.
Fix by treating python 2 strings the same python 3 bytes objects, and just
copying over the data if possible.
Py_Finalize could potentially invoke code that calls `get_internals()`,
which could create a new internals object if one didn't exist.
`finalize_interpreter()` didn't catch this because it only used the
pre-finalize interpreter pointer status; if this happens, it results in
the internals pointer not being properly destroyed with the interpreter,
which leaks, and also causes a `get_internals()` under a future
interpreter to return an internals object that is wrong in various ways.
This reimplements the std::reference_wrapper<T> caster to be a shell
around the underlying T caster (rather than assuming T is a generic
type), which lets it work for things like `std::reference_wrapper<int>`
or anything else custom type caster with a lvalue cast operator.
This also makes it properly fail when None is provided, just as an
ordinary lvalue reference argument would similarly fail.
This also adds a static assert to test that T has an appropriate type
caster. It triggers for casters like `std::pair`, which have
return-by-value cast operators. (In theory this could be supported by
storing a local temporary for such types, but that's beyond the scope
of this PR).
This also replaces `automatic` or `take_ownership` return value policies
with `automatic_reference` as taking ownership of a reference inside a
reference_wrapper is not valid.
The new version of pytest now reports Python warnings by default. This
commit filters out some third-party extension warnings which are not
useful for pybind11 tests.
This commit also adds `doc()` to `object_api` as a shortcut for the
`attr("__doc__")` accessor.
The module macro changes from:
```c++
PYBIND11_PLUGIN(example) {
pybind11::module m("example", "pybind11 example plugin");
m.def("add", [](int a, int b) { return a + b; });
return m.ptr();
}
```
to:
```c++
PYBIND11_MODULE(example, m) {
m.doc() = "pybind11 example plugin";
m.def("add", [](int a, int b) { return a + b; });
}
```
Using the old macro results in a deprecation warning. The warning
actually points to the `pybind11_init` function (since attributes
don't bind to macros), but the message should be quite clear:
"PYBIND11_PLUGIN is deprecated, use PYBIND11_MODULE".
* Added template constructors to buffer_info that can deduce the item size, format string, and number of dimensions from the pointer type and the shape container
* Implemented actual buffer_info constructor as private delegate constructor taking rvalue reference as a workaround for the evaluation order move problem on GCC 4.8
At this point, there is only a single test for interpreter basics.
Apart from embedding itself, having a C++ test framework will also
benefit the C++-side features by allowing them to be tested directly.
All targets provided by pybind11:
* pybind11::module - the existing target for creating extension modules
* pybind11::embed - new target for embedding the interpreter
* pybind11::pybind11 - common "base" target (headers only)
Now that #851 has removed all multiple uses of a caster, it can just use
the default-constructed value with needing a reset. This fixes two
issues:
1. With std::experimental::optional (at least under GCC 5.4), the `= {}`
would construct an instance of the optional type and then move-assign
it, which fails if the value type isn't move-assignable.
2. With older versions of Boost, the `= {}` could fail because it is
ambiguous, allowing construction of either `boost::none` or the value
type.
MSVC by default uses the local codepage, which fails when it sees the
utf-8 in test_python_types.cpp. This adds the /utf-8 flag to the test
suite compilation to force it to interpret source code as utf-8.
Fixes#869
This extends py::vectorize to automatically pass through
non-vectorizable arguments. This removes the need for the documented
"explicitly exclude an argument" workaround.
Vectorization now applies to arithmetic, std::complex, and POD types,
passed as plain value or by const lvalue reference (previously only
pass-by-value types were supported). Non-const lvalue references and
any other types are passed through as-is.
Functions with rvalue reference arguments (whether vectorizable or not)
are explicitly prohibited: an rvalue reference is inherently not
something that can be passed multiple times and is thus unsuitable to
being in a vectorized function.
The vectorize returned value is also now more sensitive to inputs:
previously it would return by value when all inputs are of size 1; this
is now amended to having all inputs of size 1 *and* 0 dimensions. Thus
if you pass in, for example, [[1]], you get back a 1x1, 2D array, while
previously you got back just the resulting single value.
Vectorization of member function specializations is now also supported
via `py::vectorize(&Class::method)`; this required passthrough support
for the initial object pointer on the wrapping function pointer.
This attribute lets you disable (or explicitly enable) passing None to
an argument that otherwise would allow it by accepting
a value by raw pointer or shared_ptr.
This commit allows type_casters to allow their local values to be moved
away, rather than copied, when the type caster instance itself is an rvalue.
This only applies (automatically) to type casters using
PYBIND11_TYPE_CASTER; the generic type type casters don't own their own
pointer, and various value casters (e.g. std::string, std::pair,
arithmetic types) already cast to an rvalue (i.e. they return by value).
This updates various calling code to attempt to get a movable value
whenever the value is itself coming from a type caster about to be
destroyed: for example, when constructing an std::pair or various stl.h
containers. For types that don't support value moving, the cast_op
falls back to an lvalue cast.
There wasn't an obvious place to add the tests, so I added them to
test_copy_move_policies, but also renamed it to drop the _policies as it
now tests more than just policies.
Using a dynamic_cast instead of a static_cast is needed to safely cast
from a base to a derived type. The previous static_pointer_cast isn't
safe, however, when downcasting (and fails to compile when downcasting
with virtual inheritance).
Switching this to always use a dynamic_pointer_cast shouldn't incur any
additional overhead when a static_pointer_cast is safe (i.e. when
upcasting, or self-casting): compilers don't need RTTI checks in those
cases.
The Python method for /= was set as `__idiv__`, which should be
`__itruediv__` under Python 3.
This wasn't totally broken in that without it defined, Python constructs
a new object by calling __truediv__. The operator tests, however,
didn't actually test the /= operator: when I added it, I saw an extra
construction, leading to the problem. This commit also includes tests
for the previously untested *= operator, and adds some element-wise
vector multiplication and division operators.
Currently, `py::int_(1).cast<variant<double, int>>()` fills the `double`
slot of the variant. This commit switches the loader to a 2-pass scheme
in order to correctly fill the `int` slot.
Many of our `is_none()` checks in type caster loading return true, but
this should really be considered a deferral so that, for example, an
overload with a `py::none` argument would win over one that takes
`py::none` as a null option.
This keeps None-accepting for the `!convert` pass only for std::optional
and void casters. (The `char` caster already deferred None; this just
extends that behaviour to other casters).
This exposed a few underlying issues:
1. is_pod_struct was too strict to allow this. I've relaxed it to
require only trivially copyable and standard layout, rather than POD
(which additionally requires a trivial constructor, which std::complex
violates).
2. format_descriptor<std::complex<T>>::format() returned numpy format
strings instead of PEP3118 format strings, but register_dtype
feeds format codes of its fields to _dtype_from_pep3118. I've changed it
to return PEP3118 format codes. format_descriptor is a public type, so
this may be considered an incompatible change.
3. register_structured_dtype tried to be smart about whether to mark
fields as unaligned (with ^). However, it's examining the C++ alignment,
rather than what numpy (or possibly PEP3118) thinks the alignment should
be. For complex values those are different. I've made it mark all fields
as ^ unconditionally, which should always be safe even if they are
aligned, because we explicitly mark the padding.
Resolves#800.
Both C++ arrays and std::array are supported, including mixtures like
std::array<int, 2>[4]. In a multi-dimensional array of char, the last
dimension is used to construct a numpy string type.
We're current copy by creating an Eigen::Map into the input numpy
array, then assigning that to the basic eigen type, effectively having
Eigen do the copy. That doesn't work for negative strides, though:
Eigen doesn't allow them.
This commit makes numpy do the copying instead by allocating the eigen
type, then having numpy copy from the input array into a numpy reference
into the eigen object's data. This also saves a copy when type
conversion is required: numpy can do the conversion on-the-fly as part
of the copy.
Finally this commit also makes non-reference parameters respect the
convert flag, declining the load when called in a noconvert pass with a
convertible, but non-array input or an array with the wrong dtype.
`EigenConformable::stride_compatible` returns false if the strides are
negative. In this case, do not use `EigenConformable::stride`, as it
is {0,0}. We cannot write negative strides in this element, as Eigen
will throw an assertion if we do.
The `type_caster` specialization for regular, dense Eigen matrices now
does a second `array_t::ensure` to copy data in case of negative strides.
I'm not sure that this is the best way to implement this.
I have added "TODO" tags linking these changes to Eigen bug #747, which,
when fixed, will allow Eigen to accept negative strides.
If a bound std::function is invoked with a bound method, the implicit
bound self is lost because we use `detail::get_function` to unbox the
function. This commit amends the code to use py::function and only
unboxes in the special is-really-a-c-function case. This makes bound
methods stay bound rather than unbinding them by forcing extraction of
the c function.
Enumerations on Python 2.7 were not always implicitly converted to
integers (depending on the target size). This patch adds a __long__
conversion function (only enabled on 2.7) which fixes this issue.
The attached test case fails without this patch.
This removes the convert-from-arithemtic-scalar constructor of
any_container as it can result in ambiguous calls, as in:
py::array_t<float>({ 1, 2 })
which could be intepreted as either of:
py::array_t<float>(py::array_t<float>(1, 2))
py::array_t<float>(py::detail::any_container({ 1, 2 }))
Removing the convert-from-arithmetic constructor reduces the number of
implicit conversions, avoiding the ambiguity for array and array_t.
This also re-adds the array/array_t constructors taking a scalar
argument for backwards compatibility.
Python 3's `PyInstanceMethod_Type` hides itself via its `tp_descr_get`,
which prevents aliasing methods via `cls.attr("m2") = cls.attr("m1")`:
instead the `tp_descr_get` returns a plain function, when called on a
class, or a `PyMethod`, when called on an instance. Override that
behaviour for pybind11 types with a special bypass for
`PyInstanceMethod_Types`.
The Unicode support added in 2.1 (PR #624) inadvertently broke accepting
`bytes` as std::string/char* arguments. This restores it with a
separate path that does a plain conversion (i.e. completely bypassing
all the encoding/decoding code), but only for single-byte string types.
This commits adds base class pointers of offset base classes (i.e. due
to multiple inheritance) to `registered_instances` so that if such a
pointer is returned we properly recognize it as an existing instance.
Without this, returning a base class pointer will cast to the existing
instance if the pointer happens to coincide with the instance pointer,
but constructs a new instance (quite possibly with a segfault, if
ownership is applied) for unequal base class pointers due to multiple
inheritance.
When we are returned a base class pointer (either directly or via
shared_from_this()) we detect its runtime type (using `typeid`), then
end up essentially reinterpret_casting the pointer to the derived type.
This is invalid when the base class pointer was a non-first base, and we
end up with an invalid pointer. We could dynamic_cast to the
most-derived type, but if *that* type isn't pybind11-registered, the
resulting pointer given to the base `cast` implementation isn't necessarily valid
to be reinterpret_cast'ed back to the backup type.
This commit removes the "backup" type argument from the many-argument
`cast(...)` and instead does the derived-or-pointer type decision and
type lookup in type_caster_base, where the dynamic_cast has to be to
correctly get the derived pointer, but also has to do the type lookup to
ensure that we don't pass the wrong (derived) pointer when the backup
type (i.e. the type caster intrinsic type) pointer is needed.
Since the lookup is needed before calling the base cast(), this also
changes the input type to a detail::type_info rather than doing a
(second) lookup in cast().
We currently fail at runtime when trying to call a method that is
overloaded with both static and non-static methods. This is something
python won't allow: the object is either a function or an instance, and
can't be both.
Adding numpy to the pypy test exposed a segfault caused by the buffer
tests in test_stl_binders.py: the first such test was explicitly skipped
on pypy, but the second (test_vector_buffer_numpy) which also seems to
cause an occasional segfault was just marked as requiring numpy.
Explicitly skip it on pypy as well (until a workaround, fix, or pypy fix
are found).
Don't try to define these in the issues submodule, because that fails
if testing without issues compiled in (e.g. using
cmake -DPYBIND11_TEST_OVERRIDE=test_methods_and_attributes.cpp).
This adds support for constructing `buffer_info` and `array`s using
arbitrary containers or iterator pairs instead of requiring a vector.
This is primarily needed by PR #782 (which makes strides signed to
properly support negative strides, and will likely also make shape and
itemsize to avoid mixed integer issues), but also needs to preserve
backwards compatibility with 2.1 and earlier which accepts the strides
parameter as a vector of size_t's.
Rather than adding nearly duplicate constructors for each stride-taking
constructor, it seems nicer to simply allow any type of container (or
iterator pairs). This works by replacing the existing vector arguments
with a new `detail::any_container` class that handles implicit
conversion of arbitrary containers into a vector of the desired type.
It can also be explicitly instantiated with a pair of iterators (e.g.
by passing {begin, end} instead of the container).
When attempting to get a raw array pointer we return nullptr if given a
nullptr, which triggers an error_already_set(), but we haven't set an
exception message, which results in "Unknown internal error".
Callers that want explicit allowing of a nullptr here already handle it
(by clearing the exception after the call).
Many of the Eigen type casters' name() methods weren't wrapping the type
description in a `type_descr` object, which thus wasn't adding the
"{...}" annotation used to identify an argument which broke the help
output by skipping eigen arguments.
The test code I had added even had some (unnoticed) broken output (with
the "arg0: " showing up in the return value).
This commit also adds test code to ensure that named eigen arguments
actually work properly, despite the invalid help output. (The added
tests pass without the rest of this commit).
Fixes#775.
Assignments of the form `Type.static_prop = value` should be translated to
`Type.static_prop.__set__(value)` except when `isinstance(value, static_prop)`.
When make_tuple fails (for example, when print() is called with a
non-convertible argument, as in #778) the error message a less helpful
than it could be:
make_tuple(): unable to convert arguments of types 'std::tuple<type1, type2>' to Python object
There is no actual std::tuple involved (only a parameter pack and a
Python tuple), but it also doesn't immediately reveal which type caused
the problem.
This commit changes the debugging mode output to show just the
problematic type:
make_tuple(): unable to convert argument of type 'type2' to Python object
This commit adds `error_already_set::matches()` convenience method to
check if the exception trapped by `error_already_set` matches a given
Python exception type. This will address #700 by providing a less
verbose way to check exceptions.
The extends the previous unchecked support with the ability to
determine the dimensions at runtime. This incurs a small performance
hit when used (versus the compile-time fixed alternative), but is still considerably
faster than the full checks on every call that happen with
`.at()`/`.mutable_at()`.
This adds bounds-unchecked access to arrays through a `a.unchecked<Type,
Dimensions>()` method. (For `array_t<T>`, the `Type` template parameter
is omitted). The mutable version (which requires the array have the
`writeable` flag) is available as `a.mutable_unchecked<...>()`.
Specifying the Dimensions as a template parameter allows storage of an
std::array; having the strides and sizes stored that way (as opposed to
storing a copy of the array's strides/shape pointers) allows the
compiler to make significant optimizations of the shape() method that it
can't make with a pointer; testing with nested loops of the form:
for (size_t i0 = 0; i0 < r.shape(0); i0++)
for (size_t i1 = 0; i1 < r.shape(1); i1++)
...
r(i0, i1, ...) += 1;
over a 10 million element array gives around a 25% speedup (versus using
a pointer) for the 1D case, 33% for 2D, and runs more than twice as fast
with a 5D array.
This extends the trivial handling to support trivial handling for
Fortran-order arrays (i.e. column major): if inputs aren't all
C-contiguous, but *are* all F-contiguous, the resulting array will be
F-contiguous and we can do trivial processing.
For anything else (e.g. C-contiguous, or inputs requiring non-trivial
processing), the result is in (numpy-default) C-contiguous layout.
The only part of the vectorize code that actually needs c-contiguous is
the "trivial" broadcast; for non-trivial arguments, the code already
uses strides properly (and so handles C-style, F-style, neither, slices,
etc.)
This commit rewrites `broadcast` to additionally check for C-contiguous
storage, then takes off the `c_style` flag for the arguments, which
will keep the functionality more or less the same, except for no longer
requiring an array copy for non-c-contiguous input arrays.
Additionally, if we're given a singleton slice (e.g. a[0::4, 0::4] for a
4x4 or smaller array), we no longer fail triviality because the trivial
code path never actually uses the strides on a singleton.
Instead of a segfault. Fixes#751.
This covers the case of loading a custom holder from a default-holder
instance. Attempting to load one custom holder from a different custom
holder (i.e. not `std::unique_ptr`) yields undefined behavior, just as
#588 established for inheritance.
We can't support this for classes from imported modules (which is the
primary purpose of a ctor argument base class) because we *have* to
have both parent and derived to properly extract a multiple-inheritance
base class pointer from a derived class pointer.
We could support this for actual `class_<Base, ...> instances, but since
in that case the `Base` is already present in the code, it seems more
consistent to simply always require MI to go via template options.
Fixes#738
The current check for conformability fails when given a 2D, 1xN or Nx1
input to a row-major or column-major, respectively, Eigen::Ref, leading
to a copy-required state in the type_caster, but this later failed
because the copy was also non-conformable because it had the same shape
and strides (because a 1xN or Nx1 is both F and C contiguous).
In such cases we can safely ignore the stride on the "1" dimension since
it'll never be used: only the "N" dimension stride needs to match the
Eigen::Ref stride, which both fixes the non-conformable copy problem,
but also avoids a copy entirely as long as the "N" dimension has a
compatible stride.
Allows use of vectors as python buffers, so for example they can be adopted without a copy by numpy.asarray
Allows faster conversion of buffers to vectors by copying instead of individually casting the elements
* Add value_type member alias to py::array_t (resolve#632)
* Use numpy scalar name in py::array_t function signatures (e.g. float32/64 instead of just float)
The duration calculation was using %, but that's only supported on
duration objects when the arithmetic type supports %, and hence fails
for floats. Fixed by subtracting off the calculated values instead.
* Add `pytest.ini` config file and set default options there instead of
in `CMakeLists.txt` (command line arguments).
* Change all output capture from `capfd` (filedescriptors) to `capsys`
(Python's `sys.stdout` and `sys.stderr`). This avoids capturing
low-level C errors, e.g. from the debug build of Python.
* Set pytest minimum version to 3.0 to make it easier to use new
features. Removed conditional use of `excinfo.match()`.
* Clean up some leftover function-level `@pytest.requires_numpy`.
When using pybind::options to disable function signatures, user-defined
docstrings only get appended if they exist, but newlines were getting
appended unconditionally, so the docstring could end up with blank lines
(depending on which overloads, in particular, provided docstrings).
This commit suppresses the empty lines by only adding newlines for
overloads when needed.
This makes array_t respect overload resolution and noconvert by failing
to load when `convert = false` if the src isn't already an array of the
correct type.
Before this, `py::iterator` didn't do any error handling, so code like:
```c++
for (auto item : py::int_(1)) {
// ...
}
```
would just silently skip the loop. The above now throws `TypeError` as
expected. This is a breaking behavior change, but any code which relied
on the silent skip was probably broken anyway.
Also, errors returned by `PyIter_Next()` are now properly handled.
test_eigen.py and test_numpy_*.py have the same
@pytest.requires_eigen_and_numpy or @pytest.requires_numpy on every
single test; this changes them to use pytest's global `pytestmark = ...`
instead to disable the entire module when numpy and/or eigen aren't
available.
This commit largely rewrites the Eigen dense matrix support to avoid
copying in many cases: Eigen arguments can now reference numpy data, and
numpy objects can now reference Eigen data (given compatible types).
Eigen::Ref<...> arguments now also make use of the new `convert`
argument use (added in PR #634) to avoid conversion, allowing
`py::arg().noconvert()` to be used when binding a function to prohibit
copying when invoking the function. Respecting `convert` also means
Eigen overloads that avoid copying will be preferred during overload
resolution to ones that require copying.
This commit also rewrites the Eigen documentation and test suite to
explain and test the new capabilities.
Numpy raises ValueError when attempting to modify an array, while
py::array is raising a RuntimeError. This changes the exception to a
std::domain_error, which gets mapped to the expected ValueError in
python.
numpy arrays aren't currently properly setting base: by setting `->base`
directly, the base doesn't follow what numpy expects and documents (that
is, following chained array bases to the root array).
This fixes the behaviour by using numpy's PyArray_SetBaseObject to set
the base instead, and then updates the tests to reflect the fixed
behaviour.
Currently when we do a conversion between a numpy array and an Eigen
Vector, we allow the conversion only if the Eigen type is a
compile-time vector (i.e. at least one dimension is fixed at 1 at
compile time), or if the type is dynamic on *both* dimensions.
This means we can run into cases where MatrixXd allow things that
conforming, compile-time sizes does not: for example,
`Matrix<double,4,Dynamic>` is currently not allowed, even when assigning
from a 4-element vector, but it *is* allowed for a
`Matrix<double,Dynamic,Dynamic>`.
This commit also reverts the current behaviour of using the matrix's
storage order to determine the structure when the Matrix is fully
dynamic (i.e. in both dimensions). Currently we assign to an eigen row
if the storage order is row-major, and column otherwise: this seems
wrong (the storage order has nothing to do with the shape!). While
numpy doesn't distinguish between a row/column vector, Eigen does, but
it makes more sense to consistently choose one than to produce
something with a different shape based on the intended storage layout.
With the previous commit, output can be very confusing because you only
see positional arguments in the "invoked with" line, but you can have a
failure from kwargs as well (in particular, when a value is invalidly
specified via both via positional and kwargs). This commits adds
kwargs to the output, and updates the associated tests to match.
* Make tests buildable independently
This makes "tests" buildable as a separate project that uses
find_package(pybind11 CONFIG) when invoked independently.
This also moves the WERROR option into tests/CMakeLists.txt, as that's
the only place it is used.
* Use Eigen 3.3.1's cmake target, if available
This changes the eigen finding code to attempt to use Eigen's
system-installed Eigen3Config first. In Eigen 3.3.1, it exports a cmake
Eigen3::Eigen target to get dependencies from (rather than setting the
include path directly).
If it fails, we fall back to the trying to load allowing modules (i.e.
allowing our tools/FindEigen3.cmake). If we either fallback, or the
eigen version is older than 3.3.1 (or , we still set the include
directory manually; otherwise, for CONFIG + new Eigen, we get it via
the target.
This is also needed to allow 'tests' to be built independently, when
the find_package(Eigen3) is going to find via the system-installed
Eigen3Config.cmake.
* Add a install-then-build test, using clang on linux
This tests that `make install` to the actual system, followed by a build
of the tests (without the main pybind11 repository available) works as
expected.
To also expand the testing variety a bit, it also builds using
clang-3.9 instead of gcc.
* Don't try loading Eigen3Config in cmake < 3.0
It could FATAL_ERROR as the newer cmake includes a cmake 3.0 required
line.
If doing an independent, out-of-tree "tests" build, the regular
find_package(Eigen3) is likely to fail with the same error, but I think
we can just let that be: if you want a recent Eigen with proper cmake
loading support *and* want to do an independent tests build, you'll
need at least cmake 3.0.
* Make string conversion stricter
The string conversion logic added in PR #624 for all std::basic_strings
was derived from the old std::wstring logic, but that was underused and
turns out to have had a bug in accepting almost anything convertible to
unicode, while the previous std::string logic was much stricter. This
restores the previous std::string logic by only allowing actual unicode
or string types.
Fixes#685.
* Added missing 'requires numpy' decorator
(I forgot that the change to a global decorator here is in the
not-yet-merged Eigen PR)
Now that only one shared metaclass is ever allocated, it's extremely
cheap to enable it for all pybind11 types.
* Deprecate the default py::metaclass() since it's not needed anymore.
* Allow users to specify a custom metaclass via py::metaclass(handle).
In order to fully satisfy Python's inheritance type layout requirements,
all types should have a common 'solid' base. A solid base is one which
has the same instance size as the derived type (not counting the space
required for the optional `dict_ptr` and `weakrefs_ptr`). Thus, `object`
does not qualify as a solid base for pybind11 types and this can lead to
issues with multiple inheritance.
To get around this, new base types are created: one per unique instance
size. There is going to be very few of these bases. They ensure Python's
MRO checks will pass when multiple bases are involved.
Instead of creating a new unique metaclass for each type, the builtin
`property` type is subclassed to support static properties. The new
setter/getters always pass types instead of instances in their `self`
argument. A metaclass is still required to support this behavior, but
it doesn't store any data anymore, so a new one doesn't need to be
created for each class. There is now only one common metaclass which
is shared by all pybind11 types.
* Fixed compilation error when defining function accepting some forms of std::function.
The compilation error happens only when the functional.h header is
present, and the build is done in debug mode, with NDEBUG being
undefined. In addition, the std::function must accept an abstract
base class by reference.
The compilation error occurred in cast.h, when trying to construct a
std::tuple<AbstractBase>, rather than a std::tuple<AbstractBase&>.
This was caused by functional.h using std::move rather than
std::forward, changing the signature of the function being used.
This commit contains the fix, along with a test that exhibits the
issue when compiled in debug mode without the fix applied.
* Moved new std::function tests into test_callbacks, added callback_with_movable test.
