mirror of
https://github.com/pybind/pybind11.git
synced 2025-02-16 21:57:55 +00:00
When casting to an unsigned type from a python 2 `int`, we currently cast using `(unsigned long long) PyLong_AsUnsignedLong(src.ptr())`. If the Python cast fails, it returns (unsigned long) -1, but then we cast this to `unsigned long long`, which means we get 4294967295, but because that isn't equal to `(unsigned long long) -1`, we don't detect the failure. This commit moves the unsigned casting into a `detail::as_unsigned` function which, upon error, casts -1 to the final type, and otherwise casts the return value to the final type to avoid the problematic double cast when an error occurs. The error most commonly shows up wherever `long` is 32-bits (e.g. under both 32- and 64-bit Windows, and under 32-bit linux) when passing a negative value to a bound function taking an `unsigned long`. Fixes #929. The added tests also trigger a latent segfault under PyPy: when casting to an integer smaller than `long` (e.g. casting to a `uint32_t` on a 64-bit `long` architecture) we check both for a Python error and also that the resulting intermediate value will fit in the final type. If there is no conversion error, but we get a value that would overflow, we end up calling `PyErr_ExceptionMatches()` illegally: that call is only allowed when there is a current exception. Under PyPy, this segfaults the test suite. It doesn't appear to segfault under CPython, but the documentation suggests that it *could* do so. The fix is to only check for the exception match if we actually got an error. |
||
---|---|---|
.. | ||
pybind11 |