* Put bakein branch @ 18b72c0ffa on top of smart_holder branch:
Commands used:
```
git checkout bakein
git diff smart_holder > ~/zd
git checkout smart_holder
git checkout -b bakein_sh
patch -p 1 < ~/zd
git checkout smart_holder \
MANIFEST.in \
README.rst \
README_smart_holder.rst \
docs/advanced/smart_ptrs.rst \
ubench/holder_comparison.cpp \
ubench/holder_comparison.py \
ubench/holder_comparison_extract_sheet_data.py \
ubench/number_bucket.h \
ubench/python/number_bucket.clif
git add -A
```
* Add back README_smart_holder.rst in tests/extra_python_package/test_files.py
* Restore smart_holder_poc.h as-is on smart_holder branch (i.e. undo `PYBIND11_SMART_HOLDER_PADDING`, which was meant for stress-testing only).
* Insert `std::move()` as suggested by @laramiel
* `property_cpp_function_sh_*` named specializations, as suggested by @laramiel (https://github.com/pybind/pybind11/pull/5257#discussion_r1688346807)
* Call `property_cpp_function_classic` member functions, rather than inlining the implementations.
* Use `PYBIND11_HAVE_INTERNALS_WITH_SMART_HOLDER_SUPPORT` in holder_comparison.cpp (holder_comparison.py is NOT changed accordingly in this commit, i.e. can still only be run if the smart_holder functionality is available).
* Systematically rename `loaded_as` to `load_as` (`shared_ptr`, `unique_ptr`) as suggested by @laramiel
* Make change as suggested by @laramiel. This makes it much more obvious that the latest implementation of `smart_holder_from_unique_ptr()` accepts all existing `return_value_policy` enum values except `copy`.
* Resolve `BAKEIN_WIP: Rewrite comment.` for `property_cpp_function_*` specializations.
* Resolve `BAKEIN_WIP: Add comment to explain: This is meant for stress-testing only.`
* Resolve all remaining BAKEIN_WIP (in pybind11/cast.h).
Leave only two pairs of SMART_HOLDER_BAKEIN_FOLLOW_ON comments: refactoring of copyable_holder_caster, move_only_holder_caster. This is best left until after the smart_holder branch is merged into the master branch.
* Remove obsolete `using holder_type = smart_holder;` in `load_helper`
* Add SMART_HOLDER_BAKEIN_FOLLOW_ON comment for `internals::default_holder`
* README_smart_holder.rst update (line count reduced from 356 to 123).
* Manual line breaks to pre-empt undesired `clang-format`ing.
Informed by work under https://github.com/pybind/pybind11/pull/3683:
60b7eb410f59572e6559
* Manual curation of clang-format diffs involving source code comments.
Very labor-intensive and dull.
* Pulling .clang-format change from @henryiii's 9057962d40
* Adding commonly used .clang-format `CommentPragmas:`
* Ensure short lambdas are allowed
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
* override: Fix wrong caching of the overrides
There was a problem when the python type, which was stored in override
cache for C++ functions, was destroyed and the record wasn't removed from the
override cache. Therefor, dangling pointer was stored there. Then when the
memory was reused and new type was allocated at the given address and the
method with the same name (as previously stored in the cache) was actually
overridden in python, it would wrongly find it in the override cache for C++
functions and therefor override from python wouldn't be called.
The fix is to erase the type from the override cache when the type is destroyed.
* test: Pass by const ref instead of by value (clang-tidy)
* test: Rename classes and move to different files
Rename the classes and files so they're no too generic. Also, better place to
test the stuff is in test_virtual_functions.cpp/.py as we're basically testing
the virtual functions/trampolines.
* Add TODO for erasure code
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Wrap PYBIND11_OVERLOAD_NAME and PYBIND11_OVERLOAD_PURE_NAME in do { ... } while (false), and resolve trailing semicolon
* Deprecate PYBIND11_OVERLOAD_* and get_overload in favor of PYBIND11_OVERRIDE_* and get_override
* Correct erroneous usage of 'overload' instead of 'override' in the implementation and internals
* Fix tests to use non-deprecated PYBIND11_OVERRIDE_* macros
* Update docs to use override instead of overload where appropriate, and add warning about deprecated aliases
* Add semicolons to deprecated PYBIND11_OVERLOAD macros to match original behavior
* Remove deprecation of PYBIND11_OVERLOAD_* macros and get_overload
* Add note to changelog and upgrade guide
* Added guards to the includes
Added new CI config
Added new trigger
Changed CI workflow name
Debug CI
Debug CI
Debug CI
Debug CI
Added flags fro PGI
Disable Eigen
Removed tests that fail
Uncomment lines
* fix: missing include
fix: minor style cleanup
tests: support skipping
ci: remove and tighten a bit
fix: try msvc workaround for pgic
* tests: split up prealoc tests
* fix: PGI compiler fix
* fix: PGI void_t only
* fix: try to appease nvcc
* ci: better ordering for slow tests
* ci: minor improvements to testing
* ci: Add NumPy to testing
* ci: Eigen generates CUDA warnings / PGI errors
* Added CentOS7 back for a moment
* Fix YAML
* ci: runs-on missing
* centos7 is missing pytest
* ci: use C++11 on CentOS 7
* ci: test something else
* Try just adding flags on CentOS 7
* fix: CentOS 7
* refactor: move include to shared location
* Added verbose flag
* Try to use system cmake3 on CI
* Try to use system cmake3 on CI, attempt2
* Try to use system cmake3 on CI, attempt3
* tests: not finding pytest should be a warning, not a fatal error
* tests: cleanup
* Weird issue?
* fix: final polish
Co-authored-by: Andrii Verbytskyi <andrii.verbytskyi@mpp.mpg.de>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Andrii Verbytskyi <averbyts@cern.ch>
* fix: support nvcc and test
* fixup! fix: support nvcc and test
* docs: mention what compilers fail
* fix: much simpler logic
* refactor: slightly faster / clearer
* Fix warning that not including a cmake source or build dir will be a fatal error (it is now on newest CMakes)
* Fixes appveyor
* Travis uses CMake 3.9 for more than a year now
* Travis dropped sudo: false in December
* Dropping Sphinx 2
- clang7: Suppress self-assign warnings; fix missing virtual dtors
- pypy:
- Keep old version (newer stuff breaks)
- Pin packages to extra index for speed
- travis:
- Make docker explicit; remove docker if not needed
- Make commands more verbose (for debugging / repro)
- Make Ubuntu dist explicit per job
- Fix Windows
- Add names to travis
- PYBIND11_MAKE_OPAQUE now takes ... rather than a single argument and
expands it with __VA_ARGS__; this lets templated, comma-containing
types get through correctly.
- Adds a new macro PYBIND11_TYPE() that lets you pass the type into a
macro as a single argument, such as:
PYBIND11_OVERLOAD(PYBIND11_TYPE(R<1,2>), PYBIND11_TYPE(C<3,4>), func)
Unfortunately this only works for one macro call: to forward the
argument on to the next macro call (without the processor breaking it
up again) requires also adding the PYBIND11_TYPE(...) to type macro
arguments in the PYBIND11_OVERLOAD_... macro chain.
- updated the documentation with these two changes, and use them at a couple
places in the test suite to test that they work.
This reimplements the py::init<...> implementations using the various
functions added to support `py::init(...)`, and moves the implementing
structs into `detail/init.h` from `pybind11.h`. It doesn't simply use a
factory directly, as this is a very common case and implementation
without an extra lambda call is a small but useful optimization.
This, combined with the previous lazy initialization, also avoids
needing placement new for `py::init<...>()` construction: such
construction now occurs via an ordinary `new Type(...)`.
A consequence of this is that it also fixes a potential bug when using
multiple inheritance from Python: it was very easy to write classes
that double-initialize an existing instance which had the potential to
leak for non-pod classes. With the new implementation, an attempt to
call `__init__` on an already-initialized object is now ignored. (This
was already done in the previous commit for factory constructors).
This change exposed a few warnings (fixed here) from deleting a pointer
to a base class with virtual functions but without a virtual destructor.
These look like legitimate warnings that we shouldn't suppress; this
adds virtual destructors to the appropriate classes.
This allows you to use:
cls.def(py::init(&factory_function));
where `factory_function` returns a pointer, holder, or value of the
class type (or a derived type). Various compile-time checks
(static_asserts) are performed to ensure the function is valid, and
various run-time type checks where necessary.
Some other details of this feature:
- The `py::init` name doesn't conflict with the templated no-argument
`py::init<...>()`, but keeps the naming consistent: the existing
templated, no-argument one wraps constructors, the no-template,
function-argument one wraps factory functions.
- If returning a CppClass (whether by value or pointer) when an CppAlias
is required (i.e. python-side inheritance and a declared alias), a
dynamic_cast to the alias is attempted (for the pointer version); if
it fails, or if returned by value, an Alias(Class &&) constructor
is invoked. If this constructor doesn't exist, a runtime error occurs.
- for holder returns when an alias is required, we try a dynamic_cast of
the wrapped pointer to the alias to see if it is already an alias
instance; if it isn't, we raise an error.
- `py::init(class_factory, alias_factory)` is also available that takes
two factories: the first is called when an alias is not needed, the
second when it is.
- Reimplement factory instance clearing. The previous implementation
failed under python-side multiple inheritance: *each* inherited
type's factory init would clear the instance instead of only setting
its own type value. The new implementation here clears just the
relevant value pointer.
- dealloc is updated to explicitly set the leftover value pointer to
nullptr and the `holder_constructed` flag to false so that it can be
used to clear preallocated value without needing to rebuild the
instance internals data.
- Added various tests to test out new allocation/deallocation code.
- With preallocation now done lazily, init factory holders can
completely avoid the extra overhead of needing an extra
allocation/deallocation.
- Updated documentation to make factory constructors the default
advanced constructor style.
- If an `__init__` is called a second time, we have two choices: we can
throw away the first instance, replacing it with the second; or we can
ignore the second call. The latter is slightly easier, so do that.
This udpates all the remaining tests to the new test suite code and
comment styles started in #898. For the most part, the test coverage
here is unchanged, with a few minor exceptions as noted below.
- test_constants_and_functions: this adds more overload tests with
overloads with different number of arguments for more comprehensive
overload_cast testing. The test style conversion broke the overload
tests under MSVC 2015, prompting the additional tests while looking
for a workaround.
- test_eigen: this dropped the unused functions `get_cm_corners` and
`get_cm_corners_const`--these same tests were duplicates of the same
things provided (and used) via ReturnTester methods.
- test_opaque_types: this test had a hidden dependence on ExampleMandA
which is now fixed by using the global UserType which suffices for the
relevant test.
- test_methods_and_attributes: this required some additions to UserType
to make it usable as a replacement for the test's previous SimpleType:
UserType gained a value mutator, and the `value` property is not
mutable (it was previously readonly). Some overload tests were also
added to better test overload_cast (as described above).
- test_numpy_array: removed the untemplated mutate_data/mutate_data_t:
the templated versions with an empty parameter pack expand to the same
thing.
- test_stl: this was already mostly in the new style; this just tweaks
things a bit, localizing a class, and adding some missing
`// test_whatever` comments.
- test_virtual_functions: like `test_stl`, this was mostly in the new
test style already, but needed some `// test_whatever` comments.
This commit also moves the inherited virtual example code to the end
of the file, after the main set of tests (since it is less important
than the other tests, and rather length); it also got renamed to
`test_inherited_virtuals` (from `test_inheriting_repeat`) because it
tests both inherited virtual approaches, not just the repeat approach.
This adds a static local variable (in dead code unless actually needed)
in the overload code that is used for storage if the overload is for
some convert-by-value type (such as numeric values or std::string).
This has limitations (as written up in the advanced doc), but is better
than simply not being able to overload reference or pointer methods.
This allows a slightly cleaner base type specification of:
py::class_<Type, Base>("Type")
as an alternative to
py::class_<Type>("Type", py::base<Base>())
As with the other template parameters, the order relative to the holder
or trampoline types doesn't matter.
This also includes a compile-time assertion failure if attempting to
specify more than one base class (but is easily extendible to support
multiple inheritance, someday, by updating the class_selector::set_bases
function to set multiple bases).
With this change both C++ and Python write to sys.stdout which resolves
the capture issues noted in #351. Therefore, the related workarounds are
removed.
The current pybind11::class_<Type, Holder, Trampoline> fixed template
ordering results in a requirement to repeat the Holder with its default
value (std::unique_ptr<Type>) argument, which is a little bit annoying:
it needs to be specified not because we want to override the default,
but rather because we need to specify the third argument.
This commit removes this limitation by making the class_ template take
the type name plus a parameter pack of options. It then extracts the
first valid holder type and the first subclass type for holder_type and
trampoline type_alias, respectively. (If unfound, both fall back to
their current defaults, `std::unique_ptr<type>` and `type`,
respectively). If any unmatched template arguments are provided, a
static assertion fails.
What this means is that you can specify or omit the arguments in any
order:
py::class_<A, PyA> c1(m, "A");
py::class_<B, PyB, std::shared_ptr<B>> c2(m, "B");
py::class_<C, std::shared_ptr<C>, PyB> c3(m, "C");
It also allows future class attributes (such as base types in the next
commit) to be passed as class template types rather than needing to use
a py::base<> wrapper.
Adding or removing tests is a little bit cumbersome currently: the test
needs to be added to CMakeLists.txt, the init function needs to be
predeclared in pybind11_tests.cpp, then called in the plugin
initialization. While this isn't a big deal for tests that are being
committed, it's more of a hassle when working on some new feature or
test code for which I temporarily only care about building and linking
the test being worked on rather than the entire test suite.
This commit changes tests to self-register their initialization by
having each test initialize a local object (which stores the
initialization function in a static variable). This makes changing the
set of tests being build easy: one only needs to add or comment out
test names in tests/CMakeLists.txt.
A couple other minor changes that go along with this:
- test_eigen.cpp is now included in the test list, then removed if eigen
isn't available. This lets you disable the eigen tests by commenting
it out, just like all the other tests, but keeps the build working
without eigen eigen isn't available. (Also, if it's commented out, we
don't even bother looking for and reporting the building with/without
eigen status message).
- pytest is now invoked with all the built test names (with .cpp changed
to .py) so that it doesn't try to run tests that weren't built.
Problem
=======
The template trampoline pattern documented in PR #322 has a problem with
virtual method overloads in intermediate classes in the inheritance
chain between the trampoline class and the base class.
For example, consider the following inheritance structure, where `B` is
the actual class, `PyB<B>` is the trampoline class, and `PyA<B>` is an
intermediate class adding A's methods into the trampoline:
PyB<B> -> PyA<B> -> B -> A
Suppose PyA<B> has a method `some_method()` with a PYBIND11_OVERLOAD in
it to overload the virtual `A::some_method()`. If a Python class `C` is
defined that inherits from the pybind11-registered `B` and tries to
provide an overriding `some_method()`, the PYBIND11_OVERLOADs declared
in PyA<B> fails to find this overloaded method, and thus never invoke it
(or, if pure virtual and not overridden in PyB<B>, raises an exception).
This happens because the base (internal) `PYBIND11_OVERLOAD_INT` macro
simply calls `get_overload(this, name)`; `get_overload()` then uses the
inferred type of `this` to do a type lookup in `registered_types_cpp`.
This is where it fails: `this` will be a `PyA<B> *`, but `PyA<B>` is
neither the base type (`B`) nor the trampoline type (`PyB<B>`). As a
result, the overload fails and we get a failed overload lookup.
The fix
=======
The fix is relatively simple: we can cast `this` passed to
`get_overload()` to a `const B *`, which lets get_overload look up the
correct class. Since trampoline classes should be derived from `B`
classes anyway, this cast should be perfectly safe.
This does require adding the class name as an argument to the
PYBIND11_OVERLOAD_INT macro, but leaves the public macro signatures
unchanged.
- ICPC can't handle the NCVirt trampoline which returns a non-copyable
type, which is likely due to a constexpr/SFINAE issue. This disables
the test on that compiler so that at least the rest can be tested.
Use simple asserts and pytest's powerful introspection to make testing
simpler. This merges the old .py/.ref file pairs into simple .py files
where the expected values are right next to the code being tested.
This commit does not touch the C++ part of the code and replicates the
Python tests exactly like the old .ref-file-based approach.
