docs: known issues (#2565)

* docs: FAQ CMake updates

* docs: limitations

* ci: don't over label

* docs: update CHANGELOG, add a bit more structure

* ci: label PRs with more labels, and sooner

* docs: updates from @rwgk

* docs: address @YannickJadoul's points

.github/labeler.yml

@@ -0,0 +1,28 @@
+needs changelog:
+  - all:
+    - '!docs/changelog.rst'
+    any:
+    - 'include/pybind11/**/*'
+    - 'pybind11/**/*'
+    - 'tools/**/*'
+    - 'CMakeLists.txt'
+
+docs:
+  - any:
+    - 'docs/basics.rst'
+    - 'docs/benchmark.rst'
+    - 'docs/classes.rst'
+    - 'docs/compiling.rst'
+    - 'docs/faq.rst'
+    - 'docs/index.rst'
+    - 'docs/installing.rst'
+    - 'docs/limitations.rst'
+    - 'docs/reference.rst'
+    - 'docs/release.rst'
+    - 'docs/advanced/**/*.rst'
+    - 'docs/cmake/**/*.rst'
+
+
+ci:
+  - any:
+    - '.github/workflows/*.yaml'

.github/labeler_merged.yml

@@ -1,2 +0,0 @@
-needs changelog:
-  - all: ['!docs/changelog.rst']

.github/workflows/labeler.yml

@@ -1,15 +1,15 @@
 name: PR merged
 on:
   pull_request_target:
-    types: [closed]
+    types: [opened, reopened, synchronize, edited, closed]
 
 jobs:
   label-merged:
     name: Changelog needed
     runs-on: ubuntu-latest
-    if: github.event.pull_request.merged == true
+    if: "!(github.event.action == 'closed' && github.event.pull_request.merged == false)"
     steps:
     - uses: actions/labeler@main
       with:
         repo-token: ${{ secrets.GITHUB_TOKEN }}
-        configuration-path: .github/labeler_merged.yml
+        sync-labels: true

docs/changelog.rst

@@ -11,16 +11,7 @@ v2.6.0 (IN PROGRESS)
 
 See :ref:`upgrade-guide-2.6` for help upgrading to the new version.
 
-* ``py::module`` was renamed ``py::module_`` to avoid issues with C++20 when
+New features:
-  used unqualified, but an alias ``py::module`` is provided for backward
-  compatibility.
-  `#2489 <https://github.com/pybind/pybind11/pull/2489>`_
-
-* ``pybind11_add_module()`` now accepts an optional ``OPT_SIZE`` flag that
-  switches the binding target to size-based optimization regardless global
-  CMake build type (except in debug mode, where optimizations remain disabled).
-  This reduces binary size quite substantially (~25%).
-  `#2463 <https://github.com/pybind/pybind11/pull/2463>`_
 
 * Keyword-only arguments supported in Python 2 or 3 with ``py::kw_only()``.
   `#2100 <https://github.com/pybind/pybind11/pull/2100>`_
@@ -28,14 +19,17 @@ See :ref:`upgrade-guide-2.6` for help upgrading to the new version.
 * Positional-only arguments supported in Python 2 or 3 with ``py::pos_only()``.
   `#2459 <https://github.com/pybind/pybind11/pull/2459>`_
 
-* Access to the type object now provided with ``py::type::of<T>()`` and
+* ``py::is_final()`` class modifier to block subclassing (CPython only).
-  ``py::type::of(h)``.
+  `#2151 <https://github.com/pybind/pybind11/pull/2151>`_
-  `#2364 <https://github.com/pybind/pybind11/pull/2364>`_
 
 * Added ``py::prepend()``, allowing a function to be placed at the beginning of
   the overload chain.
   `#1131 <https://github.com/pybind/pybind11/pull/1131>`_
 
+* Access to the type object now provided with ``py::type::of<T>()`` and
+  ``py::type::of(h)``.
+  `#2364 <https://github.com/pybind/pybind11/pull/2364>`_
+
 * Perfect forwarding support for methods.
   `#2048 <https://github.com/pybind/pybind11/pull/2048>`_
 
@@ -45,11 +39,49 @@ See :ref:`upgrade-guide-2.6` for help upgrading to the new version.
 * ``py::hash`` is now public.
   `#2217 <https://github.com/pybind/pybind11/pull/2217>`_
 
-* ``py::is_final()`` class modifier to block subclassing (CPython only).
+* ``py::class_<union_type>`` is now supported. Note that writing to one data
-  `#2151 <https://github.com/pybind/pybind11/pull/2151>`_
+  member of the union and reading another (type punning) is UB in C++. Thus
+  pybind11-bound enums should never be used for such conversions.
+  `#2320 <https://github.com/pybind/pybind11/pull/2320>`_.
 
-* ``py::memoryview``  update and documentation.
+* Classes now check local scope when registering members, allowing a subclass
-  `#2223 <https://github.com/pybind/pybind11/pull/2223>`_
+  to have a member with the same name as a parent (such as an enum).
+  `#2335 <https://github.com/pybind/pybind11/pull/2335>`_
+
+Assisting in writing more correct code:
+
+* Error now thrown when ``__init__`` is forgotten on subclasses.
+  `#2152 <https://github.com/pybind/pybind11/pull/2152>`_
+
+* Throw error if conversion to a pybind11 type if the Python object isn't a
+  valid instance of that type, such as ``py::bytes(o)`` when ``py::object o``
+  isn't a bytes instance.
+  `#2349 <https://github.com/pybind/pybind11/pull/2349>`_
+
+* Throw if conversion to ``str`` fails.
+  `#2477 <https://github.com/pybind/pybind11/pull/2477>`_
+
+
+API changes:
+
+* ``py::module`` was renamed ``py::module_`` to avoid issues with C++20 when
+  used unqualified, but an alias ``py::module`` is provided for backward
+  compatibility.
+  `#2489 <https://github.com/pybind/pybind11/pull/2489>`_
+
+* Public constructors for ``py::module_`` have been deprecated; please use
+  ``pybind11::module_::create_extension_module`` if you were using the public
+  constructor (fairly rare after ``PYBIND11_MODULE`` was introduced).
+  **Provisional in 2.6.0rc1.**
+  `#2552 <https://github.com/pybind/pybind11/pull/2552>`_
+
+* ``PYBIND11_OVERLOAD*`` macros and ``get_overload`` function replaced by
+  correctly-named ``PYBIND11_OVERRIDE*`` and ``get_override``, fixing
+  inconsistencies in the presence of a closing ``;`` in these macros.
+  ``get_type_overload`` is deprecated.
+  `#2325 <https://github.com/pybind/pybind11/pull/2325>`_
+
+Packaging / building improvements:
 
 * The Python package was reworked to be more powerful and useful.
   `#2433 <https://github.com/pybind/pybind11/pull/2433>`_
@@ -57,7 +89,7 @@ See :ref:`upgrade-guide-2.6` for help upgrading to the new version.
   * :ref:`build-setuptools` is easier thanks to a new
     ``pybind11.setup_helpers`` module, which provides utilities to use
     setuptools with pybind11. It can be used via PEP 518, ``setup_requires``,
-    or by directly copying ``setup_helpers.py`` into your project.
+    or by directly importing or copying ``setup_helpers.py`` into your project.
 
   * CMake configuration files are now included in the Python package. Use
     ``pybind11.get_cmake_dir()`` or ``python -m pybind11 --cmakedir`` to get
@@ -70,12 +102,13 @@ See :ref:`upgrade-guide-2.6` for help upgrading to the new version.
   * ``pybind11-config`` is another way to write ``python -m pybind11`` if you
     have your PATH set up.
 
+
 * Minimum CMake required increased to 3.4.
   `#2338 <https://github.com/pybind/pybind11/pull/2338>`_ and
   `#2370 <https://github.com/pybind/pybind11/pull/2370>`_
 
-  * Full integration with CMake’s C++ standard system replaces
+  * Full integration with CMake’s C++ standard system and compile features
-    ``PYBIND11_CPP_STANDARD``.
+    replaces ``PYBIND11_CPP_STANDARD``.
 
   * Generated config file is now portable to different Python/compiler/CMake
     versions.
@@ -88,33 +121,36 @@ See :ref:`upgrade-guide-2.6` for help upgrading to the new version.
     ``CMAKE_INTERPROCEDURAL_OPTIMIZATION``, ``set(CMAKE_CXX_VISIBILITY_PRESET
     hidden)``.
 
+  * ``CUDA`` as a language is now supported.
+
   * Helper functions ``pybind11_strip``, ``pybind11_extension``,
     ``pybind11_find_import`` added, see :doc:`cmake/index`.
 
   * Optional :ref:`find-python-mode` and :ref:`nopython-mode` with CMake.
     `#2370 <https://github.com/pybind/pybind11/pull/2370>`_
 
-  * Uninstall target added.
+* Uninstall target added.
-    `#2265 <https://github.com/pybind/pybind11/pull/2265>`_ and
+  `#2265 <https://github.com/pybind/pybind11/pull/2265>`_ and
-    `#2346 <https://github.com/pybind/pybind11/pull/2346>`_
+  `#2346 <https://github.com/pybind/pybind11/pull/2346>`_
 
-* ``PYBIND11_OVERLOAD*`` macros and ``get_overload`` function replaced by
+* ``pybind11_add_module()`` now accepts an optional ``OPT_SIZE`` flag that
-  correctly-named ``PYBIND11_OVERRIDE*`` and ``get_override``, fixing
+  switches the binding target to size-based optimization if the global build
-  inconsistencies in the presene of a closing ``;`` in these macros.
+  type can not always be fixed to ``MinSizeRel`` (except in debug mode, where
-  ``get_type_overload`` is deprecated.
+  optimizations remain disabled).  ``MinSizeRel`` or this flag reduces binary
-  `#2325 <https://github.com/pybind/pybind11/pull/2325>`_
+  size quite substantially (~25% on some platforms).
-
+  `#2463 <https://github.com/pybind/pybind11/pull/2463>`_
-* Error now thrown when ``__init__`` is forgotten on subclasses.
-  `#2152 <https://github.com/pybind/pybind11/pull/2152>`_
-
-* ``py::class_<union_type>`` is now supported. Note that writing to one data
-  member of the union and reading another (type punning) is UB in C++. Thus
-  pybind11-bound enums should never be used for such conversion.
-  `#2320 <https://github.com/pybind/pybind11/pull/2320>`_.
 
 Smaller or developer focused features:
 
-* Moved ``mkdoc.py`` to a new repo, `pybind11-mkdoc`_.
+* Moved ``mkdoc.py`` to a new repo, `pybind11-mkdoc`_. There are no longer
+  submodules in the main repo.
+
+* ``py::memoryview`` segfault fix and update, with new
+  ``py::memoryview::from_memory`` in Python 3, and documentation.
+  `#2223 <https://github.com/pybind/pybind11/pull/2223>`_
+
+* Fix for ``buffer_info`` on Python 2.
+  `#2503 <https://github.com/pybind/pybind11/pull/2503>`_
 
 * If ``__eq__`` defined but not ``__hash__``, ``__hash__`` is now set to
   ``None``.
@@ -123,14 +159,6 @@ Smaller or developer focused features:
 * ``py::ellipsis`` now also works on Python 2.
   `#2360 <https://github.com/pybind/pybind11/pull/2360>`_
 
-* Throw error if conversion to a pybind11 type if the Python object isn't a
-  valid instance of that type, such as ``py::bytes(o)`` when ``py::object o``
-  isn't a bytes instance.
-  `#2349 <https://github.com/pybind/pybind11/pull/2349>`_
-
-* Throw if conversion to ``str`` fails.
-  `#2477 <https://github.com/pybind/pybind11/pull/2477>`_
-
 * Pointer to ``std::tuple`` & ``std::pair`` supported in cast.
   `#2334 <https://github.com/pybind/pybind11/pull/2334>`_
 
@@ -141,13 +169,20 @@ Smaller or developer focused features:
 * Added missing signature for ``py::array``.
   `#2363 <https://github.com/pybind/pybind11/pull/2363>`_
 
+* ``unchecked_mutable_reference`` has access to operator ``()`` and ``[]`` when
+  const.
+  `#2514 <https://github.com/pybind/pybind11/pull/2514>`_
+
 * ``py::vectorize`` is now supported on functions that return void.
   `#1969 <https://github.com/pybind/pybind11/pull/1969>`_
 
 * ``py::capsule`` supports ``get_pointer`` and ``set_pointer``.
   `#1131 <https://github.com/pybind/pybind11/pull/1131>`_
 
-* Bugfixes related to more extensive testing.
+* Fix crash when different instances share the same pointer of the same type.
+  `#2252 <https://github.com/pybind/pybind11/pull/2252>`_
+
+* Bugfixes related to more extensive testing, new GitHub Actions CI.
   `#2321 <https://github.com/pybind/pybind11/pull/2321>`_
 
 * Bug in timezone issue in Eastern hemisphere midnight fixed.
@@ -179,7 +214,13 @@ Smaller or developer focused features:
 * NVIDIA PGI compilers now supported and tested in CI.
   `#2475 <https://github.com/pybind/pybind11/pull/2475>`_
 
-* Extensive style checking in CI, with `pre-commit`_ support.
+* Extensive style checking in CI, with `pre-commit`_ support. Code
+  modernization, checked by clang-tidy.
+
+* Expanded docs, including new main page, new installing section, and CMake
+  helpers page, along with over a dozen new sections on existing pages.
+
+* In GitHub, new docs for contributing and new issue templates.
 
 .. _pre-commit: https://pre-commit.com
 

docs/faq.rst

@@ -27,18 +27,6 @@ The Python interpreter immediately crashes when importing my module
 
 See the first answer.
 
-CMake doesn't detect the right Python version
-=============================================
-
-The CMake-based build system will try to automatically detect the installed
-version of Python and link against that. When this fails, or when there are
-multiple versions of Python and it finds the wrong one, delete
-``CMakeCache.txt`` and then invoke CMake as follows:
-
-.. code-block:: bash
-
-    cmake -DPYTHON_EXECUTABLE:FILEPATH=<path-to-python-executable> .
-
 .. _faq_reference_arguments:
 
 Limitations involving reference arguments
@@ -275,17 +263,34 @@ been received, you must either explicitly interrupt execution by throwing
         });
     }
 
+CMake doesn't detect the right Python version
+=============================================
+
+The CMake-based build system will try to automatically detect the installed
+version of Python and link against that. When this fails, or when there are
+multiple versions of Python and it finds the wrong one, delete
+``CMakeCache.txt`` and then add ``-DPYTHON_EXECUTABLE=$(which python)`` to your
+CMake configure line. (Replace ``$(which python)`` with a path to python if
+your prefer.)
+
+You can alternatively try ``-DPYBIND11_FINDPYTHON=ON``, which will activate the
+new CMake FindPython support instead of pybind11's custom search. Requires
+CMake 3.12+, and 3.15+ or 3.18.2+ are even better. You can set this in your
+``CMakeLists.txt`` before adding or finding pybind11, as well.
+
 Inconsistent detection of Python version in CMake and pybind11
 ==============================================================
 
-The functions ``find_package(PythonInterp)`` and ``find_package(PythonLibs)`` provided by CMake
+The functions ``find_package(PythonInterp)`` and ``find_package(PythonLibs)``
-for Python version detection are not used by pybind11 due to unreliability and limitations that make
+provided by CMake for Python version detection are modified by pybind11 due to
-them unsuitable for pybind11's needs. Instead pybind provides its own, more reliable Python detection
+unreliability and limitations that make them unsuitable for pybind11's needs.
-CMake code. Conflicts can arise, however, when using pybind11 in a project that *also* uses the CMake
+Instead pybind11 provides its own, more reliable Python detection CMake code.
-Python detection in a system with several Python versions installed.
+Conflicts can arise, however, when using pybind11 in a project that *also* uses
+the CMake Python detection in a system with several Python versions installed.
 
-This difference may cause inconsistencies and errors if *both* mechanisms are used in the same project. Consider the following
+This difference may cause inconsistencies and errors if *both* mechanisms are
-CMake code executed in a system with Python 2.7 and 3.x installed:
+used in the same project. Consider the following CMake code executed in a
+system with Python 2.7 and 3.x installed:
 
 .. code-block:: cmake
 
@@ -303,10 +308,24 @@ In contrast this code:
     find_package(PythonInterp)
     find_package(PythonLibs)
 
-will detect Python 3.x for pybind11 and may crash on ``find_package(PythonLibs)`` afterwards.
+will detect Python 3.x for pybind11 and may crash on
+``find_package(PythonLibs)`` afterwards.
 
-It is advised to avoid using ``find_package(PythonInterp)`` and ``find_package(PythonLibs)`` from CMake and rely
+There are three possible solutions:
-on pybind11 in detecting Python version. If this is not possible CMake machinery should be called *before* including pybind11.
+
+1. Avoid using ``find_package(PythonInterp)`` and ``find_package(PythonLibs)``
+   from CMake and rely on pybind11 in detecting Python version. If this is not
+   possible, the CMake machinery should be called *before* including pybind11.
+2. Set ``PYBIND11_FINDPYTHON`` to ``True`` or use ``find_package(Python
+   COMPONENTS Interpreter Development)`` on modern CMake (3.12+, 3.15+ better,
+   3.18.2+ best). Pybind11 in these cases uses the new CMake FindPython instead
+   of the old, deprecated search tools, and these modules are much better at
+   finding the correct Python.
+3. Set ``PYBIND11_NOPYTHON`` to ``TRUE``. Pybind11 will not search for Python.
+   However, you will have to use the target-based system, and do more setup
+   yourself, because it does not know about or include things that depend on
+   Python, like ``pybind11_add_module``. This might be ideal for integrating
+   into an existing system, like scikit-build's Python helpers.
 
 How to cite this project?
 =========================

docs/limitations.rst

@@ -1,6 +1,9 @@
 Limitations
 ###########
 
+Design choices
+^^^^^^^^^^^^^^
+
 pybind11 strives to be a general solution to binding generation, but it also has
 certain limitations:
 
@@ -11,9 +14,43 @@ certain limitations:
 
 - The NumPy interface ``pybind11::array`` greatly simplifies accessing
   numerical data from C++ (and vice versa), but it's not a full-blown array
-  class like ``Eigen::Array`` or ``boost.multi_array``.
+  class like ``Eigen::Array`` or ``boost.multi_array``. ``Eigen`` objects are
+  directly supported, however, with ``pybind11/eigen.h``.
 
-These features could be implemented but would lead to a significant increase in
+Large but useful features could be implemented in pybind11 but would lead to a
-complexity. I've decided to draw the line here to keep this project simple and
+significant increase in complexity. Pybind11 strives to be simple and compact.
-compact. Users who absolutely require these features are encouraged to fork
+Users who require large new features are encouraged to write an extension to
-pybind11.
+pybind11; see `pybind11_json <https://github.com/pybind/pybind11_json>`_ for an
+example.
+
+
+Known bugs
+^^^^^^^^^^
+
+These are issues that hopefully will one day be fixed, but currently are
+unsolved. If you know how to help with one of these issues, contributions
+are welcome!
+
+- The test suite currently segfaults on macOS and Python 3.9.0 when exiting the
+  interpreter.  This was suspected to be related to the cross module GIL code,
+  but could be a broader Python 3.9.0 issue.
+  `#2558 <https://github.com/pybind/pybind11/issues/2558>`_
+
+- The ``cpptest`` does not run on Windows with Python 3.8 or newer, due to DLL
+  loader changes. User code that is correctly installed should not be affected.
+  `#2560 <https://github.com/pybind/pybind11/pull/2560>`_
+
+- There may be a rare issue with leakage under some compilers, exposed by
+  adding an unrelated test to the test suite.
+  `#2335 <https://github.com/pybind/pybind11/pull/2335>`_
+
+Known limitations
+^^^^^^^^^^^^^^^^^
+
+These are issues that are probably solvable, but have not been fixed yet. A
+clean, well written patch would likely be accepted to solve them.
+
+- Type casters are not kept alive recursively.
+  `#2527 <https://github.com/pybind/pybind11/issues/2527>`_
+  One consequence is that containers of ``char *`` are currently not supported.
+  `#2245 <https://github.com/pybind/pybind11/issues/2245>`_

docs/upgrade.rst

@@ -13,10 +13,19 @@ modernization and other useful information.
 v2.6
 ====
 
-The ``tools/clang`` submodule and ``tools/mkdoc.py`` have been moved to a
+Usage of the ``PYBIND11_OVERLOAD*`` macros and ``get_overload`` function should
-standalone package, `pybind11-mkdoc`_. If you were using those tools, please
+be replaced by ``PYBIND11_OVERRIDE*`` and ``get_override``. In the future, the
-use them via a pip install from the new location.
+old macros may be deprecated and removed.
 
+``py::module`` has been renamed ``py::module_``, but a backward compatible
+typedef has been included. This change was to avoid a language change in C++20
+that requires unqualified ``module`` not be placed at the start of a logical
+line. Qualified usage is unaffected and the typedef will remain unless the
+C++ language rules change again.
+
+The public constructors of ``py::module_`` have been deprecated. Use
+``PYBIND11_MODULE`` or ``module_::create_extension_module`` instead.
+**Provisional in 2.6.0rc1.**
 
 An error is now thrown when ``__init__`` is forgotten on subclasses. This was
 incorrect before, but was not checked. Add a call to ``__init__`` if it is
@@ -37,9 +46,13 @@ If ``__eq__`` defined but not ``__hash__``, ``__hash__`` is now set to
 ``None``, as in normal CPython. You should add ``__hash__`` if you intended the
 class to be hashable, possibly using the new ``py::hash`` shortcut.
 
-Usage of the ``PYBIND11_OVERLOAD*`` macros and ``get_overload`` function should
+The constructors for ``py::array`` now always take signed integers for size,
-be replaced by ``PYBIND11_OVERRIDE*`` and ``get_override``. In the future, the
+for consistency. This may lead to compiler warnings on some systems. Cast to
-old macros may be deprecated and removed.
+``py::ssize_t`` instead of ``std::size_t``.
+
+The ``tools/clang`` submodule and ``tools/mkdoc.py`` have been moved to a
+standalone package, `pybind11-mkdoc`_. If you were using those tools, please
+use them via a pip install from the new location.
 
 The ``pybind11`` package on PyPI no longer fills the wheel "headers" slot - if
 you were using the headers from this slot, they are available by requesting the