clang-tidy upgrade (to version 18) (#5272)

* `container: silkeh/clang:18-bookworm` in .github/workflows/format.yml

* clang-tidy auto-fix (trivial, in test only)

* Disable `performance-enum-size` (noisy, low value)

* Temporarily turn off 3 diagnostics (to be tackled one-by-one).

* Add explicit `switch` `default` to resolve clang-tidy `bugprone-switch-missing-default-case`

Debian clang version 18.1.8 (++20240718080534+3b5b5c1ec4a3-1~exp1~20240718200641.143)
Target: x86_64-pc-linux-gnu

tests/test_numpy_dtypes.cpp:212:5: warning: switching on non-enum value without default case may not cover all cases [bugprone-switch-missing-default-case]

* Add clang-17 and clang-18 testing.

* Add `NOLINTNEXTLINE(clang-analyzer-optin.core.EnumCastOutOfRange)` in test_tagbased_polymorphic.cpp

Debian clang version 18.1.8 (++20240718080534+3b5b5c1ec4a3-1~exp1~20240718200641.143)
Target: x86_64-pc-linux-gnu

tests/test_tagbased_polymorphic.cpp:77:40: warning: The value '150' provided to the cast expression is not in the valid range of values for 'Kind' [clang-analyzer-optin.core.EnumCastOutOfRange]

* Fix inconsistent pybind11/eigen/tensor.h behavior:

This existing comment in pybind11/eigen/tensor.h

```
                // move, take_ownership don't make any sense for a ref/map:
```

is at odds with the `delete src;` three lines up.

In real-world client code `take_ownership` will not exist (unless the client code is untested and unused). I.e. the `delete` is essentially only useful to avoid leaks in the pybind11 unit tests.

While upgrading to clang-tidy 18, the warning below appeared. Apparently it is produced during LTO, and it appears difficult to suppress. Regardless, the best way to resolve this is to remove the `delete` and to simply make the test objects `static` in the unit test code.

________

Debian clang version 18.1.8 (++20240718080534+3b5b5c1ec4a3-1~exp1~20240718200641.143)
Target: x86_64-pc-linux-gnu
________

```
lto-wrapper: warning: using serial compilation of 3 LTRANS jobs
lto-wrapper: note: see the ‘-flto’ option documentation for more information
In function ‘cast_impl’,
    inlined from ‘cast’ at /mounted_pybind11/include/pybind11/eigen/tensor.h:414:25,
    inlined from ‘operator()’ at /mounted_pybind11/include/pybind11/eigen/../pybind11.h:296:40,
    inlined from ‘_FUN’ at /mounted_pybind11/include/pybind11/eigen/../pybind11.h:267:21:
/mounted_pybind11/include/pybind11/eigen/tensor.h:475:17: warning: ‘operator delete’ called on unallocated object ‘<anonymous>’ [-Wfree-nonheap-object]
  475 |                 delete src;
      |                 ^
/mounted_pybind11/include/pybind11/eigen/../pybind11.h: In function ‘_FUN’:
/mounted_pybind11/include/pybind11/eigen/../pybind11.h:297:75: note: declared here
  297 |                     std::move(args_converter).template call<Return, Guard>(cap->f),
      |                                                                           ^
```

* Disable `bugprone-chained-comparison`: this clang-tidy check is incompatible with the Catch2 `REQUIRE` macro (26 warnings like the one below).
________

Debian clang version 18.1.8 (++20240718080534+3b5b5c1ec4a3-1~exp1~20240718200641.143)
Target: x86_64-pc-linux-gnu
________

```
/mounted_pybind11/tests/test_embed/test_interpreter.cpp:127:9: warning: chained comparison 'v0 <= v1 == v2' may generate unintended results, use parentheses to specify order of evaluation or a logical operator to separate comparison expressions [bugprone-chained-comparison]
  127 |         REQUIRE(ret == 42);
      |         ^
/build/tests/catch/catch.hpp:17670:24: note: expanded from macro 'REQUIRE'
 17670 | #define REQUIRE( ... ) INTERNAL_CATCH_TEST( "REQUIRE", Catch::ResultDisposition::Normal, __VA_ARGS__  )
       |                        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/build/tests/catch/catch.hpp:2710:47: note: expanded from macro 'INTERNAL_CATCH_TEST'
 2710 |             catchAssertionHandler.handleExpr( Catch::Decomposer() <= __VA_ARGS__ ); \
      |                                               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/mounted_pybind11/tests/test_embed/test_interpreter.cpp:127:9: note: operand 'v0' is here
  127 |         REQUIRE(ret == 42);
      |         ^
/build/tests/catch/catch.hpp:17670:24: note: expanded from macro 'REQUIRE'
 17670 | #define REQUIRE( ... ) INTERNAL_CATCH_TEST( "REQUIRE", Catch::ResultDisposition::Normal, __VA_ARGS__  )
       |                        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/build/tests/catch/catch.hpp:2710:47: note: expanded from macro 'INTERNAL_CATCH_TEST'
 2710 |             catchAssertionHandler.handleExpr( Catch::Decomposer() <= __VA_ARGS__ ); \
      |                                               ^~~~~~~~~~~~~~~~~~~
/mounted_pybind11/tests/test_embed/test_interpreter.cpp:127:17: note: operand 'v1' is here
  127 |         REQUIRE(ret == 42);
      |                 ^
/build/tests/catch/catch.hpp:17670:90: note: expanded from macro 'REQUIRE'
 17670 | #define REQUIRE( ... ) INTERNAL_CATCH_TEST( "REQUIRE", Catch::ResultDisposition::Normal, __VA_ARGS__  )
       |                                                                                          ^~~~~~~~~~~
/build/tests/catch/catch.hpp:2710:70: note: expanded from macro 'INTERNAL_CATCH_TEST'
 2710 |             catchAssertionHandler.handleExpr( Catch::Decomposer() <= __VA_ARGS__ ); \
      |                                                                      ^~~~~~~~~~~
/mounted_pybind11/tests/test_embed/test_interpreter.cpp:127:24: note: operand 'v2' is here
  127 |         REQUIRE(ret == 42);
      |                        ^
/build/tests/catch/catch.hpp:17670:90: note: expanded from macro 'REQUIRE'
 17670 | #define REQUIRE( ... ) INTERNAL_CATCH_TEST( "REQUIRE", Catch::ResultDisposition::Normal, __VA_ARGS__  )
       |                                                                                          ^~~~~~~~~~~
/build/tests/catch/catch.hpp:2710:70: note: expanded from macro 'INTERNAL_CATCH_TEST'
 2710 |             catchAssertionHandler.handleExpr( Catch::Decomposer() <= __VA_ARGS__ ); \
      |                                                                      ^~~~~~~~~~~
```

* Add 8 `// NOLINT(bugprone-empty-catch)`

* Resolve clang-tidy `bugprone-multi-level-implicit-pointer-conversion` warnings.
________
Debian clang version 18.1.8 (++20240718080534+3b5b5c1ec4a3-1~exp1~20240718200641.143)
Target: x86_64-pc-linux-gnu
________
```
pybind11/detail/internals.h:556:53: warning: multilevel pointer conversion from 'internals **' to 'const void *', please use explicit cast [bugprone-multi-level-implicit-pointer-conversion]

pybind11/detail/type_caster_base.h:431:20: warning: multilevel pointer conversion from 'void **' to 'void *', please use explicit cast [bugprone-multi-level-implicit-pointer-conversion]

pybind11/numpy.h:904:81: warning: multilevel pointer conversion from '_object *const *' to 'const void *', please use explicit cast [bugprone-multi-level-implicit-pointer-conversion]
pybind11/numpy.h:1989:39: warning: multilevel pointer conversion from 'typename vectorize_arg<const double *>::type *' (aka 'const double **') to 'void *', please use explicit cast [bugprone-multi-level-implicit-pointer-conversion]
pybind11/numpy.h:1989:39: warning: multilevel pointer conversion from 'typename vectorize_arg<const VectorizeTestClass *>::type *' (aka 'const VectorizeTestClass **') to 'void *', please use explicit cast [bugprone-multi-level-implicit-pointer-conversion]

pybind11/stl/filesystem.h:75:44: warning: multilevel pointer conversion from 'PyObject **' (aka '_object **') to 'void *', please use explicit cast [bugprone-multi-level-implicit-pointer-conversion]
pybind11/stl/filesystem.h:83:42: warning: multilevel pointer conversion from 'PyObject **' (aka '_object **') to 'void *', please use explicit cast [bugprone-multi-level-implicit-pointer-conversion]
```

* Introduce `PYBIND11_REINTERPRET_CAST_VOID_PTR_IF_NOT_PYPY` to resolve PyPy build errors:

```
In file included from /Users/runner/work/pybind11/pybind11/tests/test_stl.cpp:18:
/Users/runner/work/pybind11/pybind11/include/pybind11/stl/filesystem.h:75:17: error: no matching function for call to 'PyPyUnicode_FSConverter'
            if (PyUnicode_FSConverter(buf, reinterpret_cast<void *>(&native)) != 0) {
                ^~~~~~~~~~~~~~~~~~~~~
/Users/runner/hostedtoolcache/PyPy/3.10.14/x64/include/pypy3.10/pypy_decl.h:969:31: note: expanded from macro 'PyUnicode_FSConverter'
                              ^~~~~~~~~~~~~~~~~~~~~~~
/Users/runner/hostedtoolcache/PyPy/3.10.14/x64/include/pypy3.10/pypy_decl.h:970:17: note: candidate function not viable: cannot convert argument of incomplete type 'void *' to 'struct _object **' for 2nd argument
PyAPI_FUNC(int) PyUnicode_FSConverter(struct _object *arg0, struct _object **arg1);
                ^
/Users/runner/hostedtoolcache/PyPy/3.10.14/x64/include/pypy3.10/pypy_decl.h:969:31: note: expanded from macro 'PyUnicode_FSConverter'
                              ^
In file included from /Users/runner/work/pybind11/pybind11/tests/test_stl.cpp:18:
/Users/runner/work/pybind11/pybind11/include/pybind11/stl/filesystem.h:83:17: error: no matching function for call to 'PyPyUnicode_FSDecoder'
            if (PyUnicode_FSDecoder(buf, reinterpret_cast<void *>(&native)) != 0) {
                ^~~~~~~~~~~~~~~~~~~
/Users/runner/hostedtoolcache/PyPy/3.10.14/x64/include/pypy3.10/pypy_decl.h:971:29: note: expanded from macro 'PyUnicode_FSDecoder'
                            ^~~~~~~~~~~~~~~~~~~~~
/Users/runner/hostedtoolcache/PyPy/3.10.14/x64/include/pypy3.10/pypy_decl.h:972:17: note: candidate function not viable: cannot convert argument of incomplete type 'void *' to 'struct _object **' for 2nd argument
PyAPI_FUNC(int) PyUnicode_FSDecoder(struct _object *arg0, struct _object **arg1);
                ^
/Users/runner/hostedtoolcache/PyPy/3.10.14/x64/include/pypy3.10/pypy_decl.h:971:29: note: expanded from macro 'PyUnicode_FSDecoder'
                            ^
```

* clang-tidy auto-fix

* Fix silly oversight.

.clang-tidy

@@ -57,10 +57,12 @@ Checks: |
   readability-string-compare,
   readability-suspicious-call-argument,
   readability-uniqueptr-delete-release,
+  -bugprone-chained-comparison,
   -bugprone-easily-swappable-parameters,
   -bugprone-exception-escape,
   -bugprone-reserved-identifier,
   -bugprone-unused-raii,
+  -performance-enum-size,
 
 CheckOptions:
 - key:             modernize-use-equals-default.IgnoreMacros

.github/workflows/ci.yml

@@ -340,6 +340,12 @@ jobs:
           - clang: 16
             std: 20
             container_suffix: "-bullseye"
+          - clang: 17
+            std: 20
+            container_suffix: "-bookworm"
+          - clang: 18
+            std: 20
+            container_suffix: "-bookworm"
 
     name: "🐍 3 • Clang ${{ matrix.clang }} • C++${{ matrix.std }} • x64"
     container: "silkeh/clang:${{ matrix.clang }}${{ matrix.container_suffix }}"

.github/workflows/format.yml

@@ -41,7 +41,7 @@ jobs:
     # in .github/CONTRIBUTING.md and update as needed.
     name: Clang-Tidy
     runs-on: ubuntu-latest
-    container: silkeh/clang:15-bullseye
+    container: silkeh/clang:18-bookworm
     steps:
     - uses: actions/checkout@v4
 

include/pybind11/detail/internals.h

@@ -553,7 +553,7 @@ PYBIND11_NOINLINE internals &get_internals() {
         }
 #endif
         internals_ptr->istate = tstate->interp;
-        state_dict[PYBIND11_INTERNALS_ID] = capsule(internals_pp);
+        state_dict[PYBIND11_INTERNALS_ID] = capsule(reinterpret_cast<void *>(internals_pp));
         internals_ptr->registered_exception_translators.push_front(&translate_exception);
         internals_ptr->static_property_type = make_static_property_type();
         internals_ptr->default_metaclass = make_default_metaclass();

include/pybind11/detail/type_caster_base.h

@@ -428,7 +428,7 @@ PYBIND11_NOINLINE void instance::allocate_layout() {
 // NOLINTNEXTLINE(readability-make-member-function-const)
 PYBIND11_NOINLINE void instance::deallocate_layout() {
     if (!simple_layout) {
-        PyMem_Free(nonsimple.values_and_holders);
+        PyMem_Free(reinterpret_cast<void *>(nonsimple.values_and_holders));
     }
 }
 

include/pybind11/eigen/tensor.h

@@ -469,9 +469,6 @@ struct type_caster<Eigen::TensorMap<Type, Options>,
                 parent_object = reinterpret_borrow<object>(parent);
                 break;
 
-            case return_value_policy::take_ownership:
-                delete src;
-                // fallthrough
             default:
                 // move, take_ownership don't make any sense for a ref/map:
                 pybind11_fail("Invalid return_value_policy for Eigen Map type, must be either "

include/pybind11/numpy.h

@@ -901,7 +901,11 @@ public:
 
     template <typename T>
     array(ShapeContainer shape, StridesContainer strides, const T *ptr, handle base = handle())
-        : array(pybind11::dtype::of<T>(), std::move(shape), std::move(strides), ptr, base) {}
+        : array(pybind11::dtype::of<T>(),
+                std::move(shape),
+                std::move(strides),
+                reinterpret_cast<const void *>(ptr),
+                base) {}
 
     template <typename T>
     array(ShapeContainer shape, const T *ptr, handle base = handle())
@@ -1986,7 +1990,7 @@ private:
         // Pointers to values the function was called with; the vectorized ones set here will start
         // out as array_t<T> pointers, but they will be changed them to T pointers before we make
         // call the wrapped function.  Non-vectorized pointers are left as-is.
-        std::array<void *, N> params{{&args...}};
+        std::array<void *, N> params{{reinterpret_cast<void *>(&args)...}};
 
         // The array of `buffer_info`s of vectorized arguments:
         std::array<buffer_info, NVectorized> buffers{

include/pybind11/stl/filesystem.h

@@ -33,6 +33,13 @@
 PYBIND11_NAMESPACE_BEGIN(PYBIND11_NAMESPACE)
 PYBIND11_NAMESPACE_BEGIN(detail)
 
+#ifdef PYPY_VERSION
+#    define PYBIND11_REINTERPRET_CAST_VOID_PTR_IF_NOT_PYPY(...) (__VA_ARGS__)
+#else
+#    define PYBIND11_REINTERPRET_CAST_VOID_PTR_IF_NOT_PYPY(...)                                   \
+        (reinterpret_cast<void *>(__VA_ARGS__))
+#endif
+
 #if defined(PYBIND11_HAS_FILESYSTEM) || defined(PYBIND11_HAS_EXPERIMENTAL_FILESYSTEM)
 template <typename T>
 struct path_caster {
@@ -72,7 +79,8 @@ public:
         }
         PyObject *native = nullptr;
         if constexpr (std::is_same_v<typename T::value_type, char>) {
-            if (PyUnicode_FSConverter(buf, &native) != 0) {
+            if (PyUnicode_FSConverter(buf, PYBIND11_REINTERPRET_CAST_VOID_PTR_IF_NOT_PYPY(&native))
+                != 0) {
                 if (auto *c_str = PyBytes_AsString(native)) {
                     // AsString returns a pointer to the internal buffer, which
                     // must not be free'd.
@@ -80,7 +88,8 @@ public:
                 }
             }
         } else if constexpr (std::is_same_v<typename T::value_type, wchar_t>) {
-            if (PyUnicode_FSDecoder(buf, &native) != 0) {
+            if (PyUnicode_FSDecoder(buf, PYBIND11_REINTERPRET_CAST_VOID_PTR_IF_NOT_PYPY(&native))
+                != 0) {
                 if (auto *c_str = PyUnicode_AsWideCharString(native, nullptr)) {
                     // AsWideCharString returns a new string that must be free'd.
                     value = c_str; // Copies the string.

include/pybind11/stl_bind.h

@@ -180,7 +180,7 @@ void vector_modifiers(
                         v.end());
                 try {
                     v.shrink_to_fit();
-                } catch (const std::exception &) {
+                } catch (const std::exception &) { // NOLINT(bugprone-empty-catch)
                     // Do nothing
                 }
                 throw;

tests/constructor_stats.h

@@ -190,7 +190,7 @@ public:
                     t1 = &p.first;
                 }
             }
-        } catch (const std::out_of_range &) {
+        } catch (const std::out_of_range &) { // NOLINT(bugprone-empty-catch)
         }
         if (!t1) {
             throw std::runtime_error("Unknown class passed to ConstructorStats::get()");

tests/test_eigen_tensor.inl

@@ -147,33 +147,39 @@ void init_tensor_module(pybind11::module &m) {
 
     m.def(
         "take_fixed_tensor",
-
         []() {
             Eigen::aligned_allocator<
                 Eigen::TensorFixedSize<double, Eigen::Sizes<3, 5, 2>, Options>>
                 allocator;
-            return new (allocator.allocate(1))
+            static auto *obj = new (allocator.allocate(1))
                 Eigen::TensorFixedSize<double, Eigen::Sizes<3, 5, 2>, Options>(
                     get_fixed_tensor<Options>());
+            return obj; // take_ownership will fail.
         },
         py::return_value_policy::take_ownership);
 
     m.def(
         "take_tensor",
-        []() { return new Eigen::Tensor<double, 3, Options>(get_tensor<Options>()); },
+        []() {
+            static auto *obj = new Eigen::Tensor<double, 3, Options>(get_tensor<Options>());
+            return obj; // take_ownership will fail.
+        },
         py::return_value_policy::take_ownership);
 
     m.def(
         "take_const_tensor",
         []() -> const Eigen::Tensor<double, 3, Options> * {
-            return new Eigen::Tensor<double, 3, Options>(get_tensor<Options>());
+            static auto *obj = new Eigen::Tensor<double, 3, Options>(get_tensor<Options>());
+            return obj; // take_ownership will fail.
         },
         py::return_value_policy::take_ownership);
 
     m.def(
         "take_view_tensor",
         []() -> const Eigen::TensorMap<Eigen::Tensor<double, 3, Options>> * {
-            return new Eigen::TensorMap<Eigen::Tensor<double, 3, Options>>(get_tensor<Options>());
+            static auto *obj
+                = new Eigen::TensorMap<Eigen::Tensor<double, 3, Options>>(get_tensor<Options>());
+            return obj; // take_ownership will fail.
         },
         py::return_value_policy::take_ownership);
 

tests/test_modules.cpp

@@ -90,32 +90,32 @@ TEST_SUBMODULE(modules, m) {
         try {
             py::class_<Dupe1>(dm, "Dupe1");
             failures.append("Dupe1 class");
-        } catch (std::runtime_error &) {
+        } catch (std::runtime_error &) { // NOLINT(bugprone-empty-catch)
         }
         try {
             dm.def("Dupe1", []() { return Dupe1(); });
             failures.append("Dupe1 function");
-        } catch (std::runtime_error &) {
+        } catch (std::runtime_error &) { // NOLINT(bugprone-empty-catch)
         }
         try {
             py::class_<Dupe3>(dm, "dupe1_factory");
             failures.append("dupe1_factory");
-        } catch (std::runtime_error &) {
+        } catch (std::runtime_error &) { // NOLINT(bugprone-empty-catch)
         }
         try {
             py::exception<Dupe3>(dm, "Dupe2");
             failures.append("Dupe2");
-        } catch (std::runtime_error &) {
+        } catch (std::runtime_error &) { // NOLINT(bugprone-empty-catch)
         }
         try {
             dm.def("DupeException", []() { return 30; });
             failures.append("DupeException1");
-        } catch (std::runtime_error &) {
+        } catch (std::runtime_error &) { // NOLINT(bugprone-empty-catch)
         }
         try {
             py::class_<DupeException>(dm, "DupeException");
             failures.append("DupeException2");
-        } catch (std::runtime_error &) {
+        } catch (std::runtime_error &) { // NOLINT(bugprone-empty-catch)
         }
 
         return failures;

tests/test_numpy_dtypes.cpp

@@ -266,6 +266,8 @@ py::array_t<int32_t, 0> test_array_ctors(int i) {
             return fill(arr_t(buf_ndim1_null));
         case 44:
             return fill(py::array(buf_ndim1_null));
+        default:
+            break;
     }
     return arr_t();
 }

tests/test_opaque_types.cpp

@@ -65,7 +65,7 @@ TEST_SUBMODULE(opaque_types, m) {
 
     m.def("return_unique_ptr", []() -> std::unique_ptr<StringList> {
         auto *result = new StringList();
-        result->push_back("some value");
+        result->emplace_back("some value");
         return std::unique_ptr<StringList>(result);
     });
 

tests/test_tagbased_polymorphic.cpp

@@ -74,6 +74,7 @@ std::vector<std::unique_ptr<Animal>> create_zoo() {
     // simulate some new type of Dog that the Python bindings
     // haven't been updated for; it should still be considered
     // a Dog, not just an Animal.
+    // NOLINTNEXTLINE(clang-analyzer-optin.core.EnumCastOutOfRange)
     ret.emplace_back(new Dog("Ginger", Dog::Kind(150)));
 
     ret.emplace_back(new Chihuahua("Hertzl"));