Merge pull request #356 from TrentHouliston/master

Add in casts for c++11s chrono classes to pythons datetime

CMakeLists.txt

@@ -161,6 +161,7 @@ if (PYBIND11_INSTALL)
   set(PYBIND11_HEADERS
     include/pybind11/attr.h
     include/pybind11/cast.h
+    include/pybind11/chrono.h
     include/pybind11/common.h
     include/pybind11/complex.h
     include/pybind11/descr.h

docs/advanced.rst

@@ -760,6 +760,85 @@ Please refer to the supplemental example for details.
     length queries (``__len__``), iterators (``__iter__``), the slicing
     protocol and other kinds of useful operations.
 
+C++11 chrono datatypes
+======================
+
+When including the additional header file :file:`pybind11/chrono.h` conversions
+from C++11 chrono datatypes to python datetime objects are automatically enabled.
+This header also enables conversions of python floats (often from sources such
+as `time.monotonic()`, `time.perf_counter()` and `time.process_time()`) into
+durations.
+
+An overview of clocks in C++11
+------------------------------
+
+A point of confusion when using these conversions is the differences between
+clocks provided in C++11. There are three clock types defined by the C++11
+standard and users can define their own if needed. Each of these clocks have
+different properties and when converting to and from python will give different
+results.
+
+The first clock defined by the standard is ``std::chrono::system_clock``. This
+clock measures the current date and time. However, this clock changes with to
+updates to the operating system time. For example, if your time is synchronised
+with a time server this clock will change. This makes this clock a poor choice
+for timing purposes but good for measuring the wall time.
+
+The second clock defined in the standard is ``std::chrono::steady_clock``.
+This clock ticks at a steady rate and is never adjusted. This makes it excellent
+for timing purposes, however the value in this clock does not correspond to the
+current date and time. Often this clock will be the amount of time your system
+has been on, although it does not have to be. This clock will never be the same
+clock as the system clock as the system clock can change but steady clocks
+cannot.
+
+The third clock defined in the standard is ``std::chrono::high_resolution_clock``.
+This clock is the clock that has the highest resolution out of the clocks in the
+system. It is normally a typedef to either the system clock or the steady clock
+but can be its own independent clock. This is important as when using these
+conversions as the types you get in python for this clock might be different
+depending on the system.
+If it is a typedef of the system clock, python will get datetime objects, but if
+it is a different clock they will be timedelta objects.
+
+Conversions Provided
+--------------------
+
+C++ to Python
+    - ``std::chrono::system_clock::time_point`` → ``datetime.datetime``
+        System clock times are converted to python datetime instances. They are
+        in the local timezone, but do not have any timezone information attached
+        to them (they are naive datetime objects).
+
+    - ``std::chrono::duration`` → ``datetime.timedelta``
+        Durations are converted to timedeltas, any precision in the duration
+        greater than microseconds is lost by rounding towards zero.
+
+    - ``std::chrono::[other_clocks]::time_point`` → ``datetime.timedelta``
+        Any clock time that is not the system clock is converted to a time delta. This timedelta measures the time from the clocks epoch to now.
+
+Python to C++
+    - ``datetime.datetime`` → ``std::chrono::system_clock::time_point``
+        Date/time objects are converted into system clock timepoints. Any
+        timezone information is ignored and the type is treated as a naive
+        object.
+
+    - ``datetime.timedelta`` → ``std::chrono::duration``
+        Time delta are converted into durations with microsecond precision.
+
+    - ``datetime.timedelta`` → ``std::chrono::[other_clocks]::time_point``
+        Time deltas that are converted into clock timepoints are treated as
+        the amount of time from the start of the clocks epoch.
+
+    - ``float`` → ``std::chrono::duration``
+        Floats that are passed to C++ as durations be interpreted as a number of
+        seconds. These will be converted to the duration using ``duration_cast``
+        from the float.
+
+    - ``float`` → ``std::chrono::[other_clocks]::time_point``
+        Floats that are passed to C++ as time points will be interpreted as the
+        number of seconds from the start of the clocks epoch.
+
 Return value policies
 =====================
 
@@ -826,7 +905,7 @@ The following table provides an overview of the available return value policies:
 |                                                  | ``keep_alive<0, 1>`` *call policy* (described in the next section) that    |
 |                                                  | prevents the parent object from being garbage collected as long as the     |
 |                                                  | return value is referenced by Python. This is the default policy for       |
-|                                                  | property getters created via ``def_property``, ``def_readwrite``, etc.)    |
+|                                                  | property getters created via ``def_property``, ``def_readwrite``, etc.     |
 +--------------------------------------------------+----------------------------------------------------------------------------+
 
 .. warning::

docs/basics.rst

@@ -297,6 +297,10 @@ as arguments and return values, refer to the section on binding :ref:`classes`.
 +---------------------------------+--------------------------+-------------------------------+
 | ``std::function<...>``          | STL polymorphic function | :file:`pybind11/functional.h` |
 +---------------------------------+--------------------------+-------------------------------+
+| ``std::chrono::duration<...>``  | STL time duration        | :file:`pybind11/chrono.h`     |
++---------------------------------+--------------------------+-------------------------------+
+| ``std::chrono::time_point<...>``| STL date/time            | :file:`pybind11/chrono.h`     |
++---------------------------------+--------------------------+-------------------------------+
 | ``Eigen::Matrix<...>``          | Eigen: dense matrix      | :file:`pybind11/eigen.h`      |
 +---------------------------------+--------------------------+-------------------------------+
 | ``Eigen::Map<...>``             | Eigen: mapped memory     | :file:`pybind11/eigen.h`      |

include/pybind11/chrono.h

@@ -0,0 +1,160 @@
+/*
+    pybind11/chrono.h: Transparent conversion between std::chrono and python's datetime
+
+    Copyright (c) 2016 Trent Houliston <trent@houliston.me> and
+                       Wenzel Jakob <wenzel.jakob@epfl.ch>
+
+    All rights reserved. Use of this source code is governed by a
+    BSD-style license that can be found in the LICENSE file.
+*/
+
+#pragma once
+
+#include "pybind11.h"
+#include <cmath>
+#include <ctime>
+#include <chrono>
+#include <datetime.h>
+
+// Backport the PyDateTime_DELTA functions from Python3.3 if required
+#ifndef PyDateTime_DELTA_GET_DAYS
+#define PyDateTime_DELTA_GET_DAYS(o)         (((PyDateTime_Delta*)o)->days)
+#endif
+#ifndef PyDateTime_DELTA_GET_SECONDS
+#define PyDateTime_DELTA_GET_SECONDS(o)      (((PyDateTime_Delta*)o)->seconds)
+#endif
+#ifndef PyDateTime_DELTA_GET_MICROSECONDS
+#define PyDateTime_DELTA_GET_MICROSECONDS(o) (((PyDateTime_Delta*)o)->microseconds)
+#endif
+
+NAMESPACE_BEGIN(pybind11)
+NAMESPACE_BEGIN(detail)
+
+template <typename type> class duration_caster {
+public:
+    typedef typename type::rep rep;
+    typedef typename type::period period;
+
+    typedef std::chrono::duration<uint_fast32_t, std::ratio<86400>> days;
+
+    bool load(handle src, bool) {
+        using namespace std::chrono;
+
+        // Lazy initialise the PyDateTime import
+        if (!PyDateTimeAPI) { PyDateTime_IMPORT; }
+
+        if (!src) return false;
+        // If invoked with datetime.delta object
+        if (PyDelta_Check(src.ptr())) {
+            value = type(duration_cast<duration<rep, period>>(
+                  days(PyDateTime_DELTA_GET_DAYS(src.ptr()))
+                + seconds(PyDateTime_DELTA_GET_SECONDS(src.ptr()))
+                + microseconds(PyDateTime_DELTA_GET_MICROSECONDS(src.ptr()))));
+            return true;
+        }
+        // If invoked with a float we assume it is seconds and convert
+        else if (PyFloat_Check(src.ptr())) {
+            value = type(duration_cast<duration<rep, period>>(duration<double>(PyFloat_AsDouble(src.ptr()))));
+            return true;
+        }
+        else return false;
+    }
+
+    // If this is a duration just return it back
+    static const std::chrono::duration<rep, period>& get_duration(const std::chrono::duration<rep, period> &src) {
+        return src;
+    }
+
+    // If this is a time_point get the time_since_epoch
+    template <typename Clock> static std::chrono::duration<rep, period> get_duration(const std::chrono::time_point<Clock, std::chrono::duration<rep, period>> &src) {
+        return src.time_since_epoch();
+    }
+
+    static handle cast(const type &src, return_value_policy /* policy */, handle /* parent */) {
+        using namespace std::chrono;
+
+        // Use overloaded function to get our duration from our source
+        // Works out if it is a duration or time_point and get the duration
+        auto d = get_duration(src);
+
+        // Lazy initialise the PyDateTime import
+        if (!PyDateTimeAPI) { PyDateTime_IMPORT; }
+
+        // Declare these special duration types so the conversions happen with the correct primitive types (int)
+        using dd_t = duration<int, std::ratio<86400>>;
+        using ss_t = duration<int, std::ratio<1>>;
+        using us_t = duration<int, std::micro>;
+
+        return PyDelta_FromDSU(duration_cast<dd_t>(d).count(),
+                               duration_cast<ss_t>(d % days(1)).count(),
+                               duration_cast<us_t>(d % seconds(1)).count());
+    }
+
+    PYBIND11_TYPE_CASTER(type, _("datetime.timedelta"));
+};
+
+// This is for casting times on the system clock into datetime.datetime instances
+template <typename Duration> class type_caster<std::chrono::time_point<std::chrono::system_clock, Duration>> {
+public:
+    typedef std::chrono::time_point<std::chrono::system_clock, Duration> type;
+    bool load(handle src, bool) {
+        using namespace std::chrono;
+
+        // Lazy initialise the PyDateTime import
+        if (!PyDateTimeAPI) { PyDateTime_IMPORT; }
+
+        if (!src) return false;
+        if (PyDateTime_Check(src.ptr())) {
+            std::tm cal;
+            cal.tm_sec   = PyDateTime_DATE_GET_SECOND(src.ptr());
+            cal.tm_min   = PyDateTime_DATE_GET_MINUTE(src.ptr());
+            cal.tm_hour  = PyDateTime_DATE_GET_HOUR(src.ptr());
+            cal.tm_mday  = PyDateTime_GET_DAY(src.ptr());
+            cal.tm_mon   = PyDateTime_GET_MONTH(src.ptr()) - 1;
+            cal.tm_year  = PyDateTime_GET_YEAR(src.ptr()) - 1900;
+            cal.tm_isdst = -1;
+
+            value = system_clock::from_time_t(std::mktime(&cal)) + microseconds(PyDateTime_DATE_GET_MICROSECOND(src.ptr()));
+            return true;
+        }
+        else return false;
+    }
+
+    static handle cast(const std::chrono::time_point<std::chrono::system_clock, Duration> &src, return_value_policy /* policy */, handle /* parent */) {
+        using namespace std::chrono;
+
+        // Lazy initialise the PyDateTime import
+        if (!PyDateTimeAPI) { PyDateTime_IMPORT; }
+
+        std::time_t tt = system_clock::to_time_t(src);
+        // this function uses static memory so it's best to copy it out asap just in case
+        // otherwise other code that is using localtime may break this (not just python code)
+        std::tm localtime = *std::localtime(&tt);
+
+        // Declare these special duration types so the conversions happen with the correct primitive types (int)
+        using us_t = duration<int, std::micro>;
+
+        return PyDateTime_FromDateAndTime(localtime.tm_year + 1900,
+                                          localtime.tm_mon + 1,
+                                          localtime.tm_mday,
+                                          localtime.tm_hour,
+                                          localtime.tm_min,
+                                          localtime.tm_sec,
+                                          (duration_cast<us_t>(src.time_since_epoch() % seconds(1))).count());
+    }
+    PYBIND11_TYPE_CASTER(type, _("datetime.datetime"));
+};
+
+// Other clocks that are not the system clock are not measured as datetime.datetime objects
+// since they are not measured on calendar time. So instead we just make them timedeltas
+// Or if they have passed us a time as a float we convert that
+template <typename Clock, typename Duration> class type_caster<std::chrono::time_point<Clock, Duration>>
+: public duration_caster<std::chrono::time_point<Clock, Duration>> {
+};
+
+template <typename Rep, typename Period> class type_caster<std::chrono::duration<Rep, Period>>
+: public duration_caster<std::chrono::duration<Rep, Period>> {
+};
+
+NAMESPACE_END(detail)
+NAMESPACE_END(pybind11)

tests/CMakeLists.txt

@@ -9,6 +9,7 @@ set(PYBIND11_TEST_FILES
   test_alias_initialization.cpp
   test_buffers.cpp
   test_callbacks.cpp
+  test_chrono.cpp
   test_class_args.cpp
   test_constants_and_functions.cpp
   test_eigen.cpp

tests/test_chrono.cpp

@@ -0,0 +1,59 @@
+/*
+    tests/test_chrono.cpp -- test conversions to/from std::chrono types
+
+    Copyright (c) 2016 Trent Houliston <trent@houliston.me> and
+                       Wenzel Jakob <wenzel.jakob@epfl.ch>
+
+    All rights reserved. Use of this source code is governed by a
+    BSD-style license that can be found in the LICENSE file.
+*/
+
+
+#include "pybind11_tests.h"
+#include "constructor_stats.h"
+#include <pybind11/chrono.h>
+
+// Return the current time off the wall clock
+std::chrono::system_clock::time_point test_chrono1() {
+    return std::chrono::system_clock::now();
+}
+
+// Round trip the passed in system clock time
+std::chrono::system_clock::time_point test_chrono2(std::chrono::system_clock::time_point t) {
+    return t;
+}
+
+// Round trip the passed in duration
+std::chrono::system_clock::duration test_chrono3(std::chrono::system_clock::duration d) {
+    return d;
+}
+
+// Difference between two passed in time_points
+std::chrono::system_clock::duration test_chrono4(std::chrono::system_clock::time_point a, std::chrono::system_clock::time_point b) {
+    return a - b;
+}
+
+// Return the current time off the steady_clock
+std::chrono::steady_clock::time_point test_chrono5() {
+    return std::chrono::steady_clock::now();
+}
+
+// Round trip a steady clock timepoint
+std::chrono::steady_clock::time_point test_chrono6(std::chrono::steady_clock::time_point t) {
+    return t;
+}
+
+// Roundtrip a duration in microseconds from a float argument
+std::chrono::microseconds test_chrono7(std::chrono::microseconds t) {
+    return t;
+}
+
+test_initializer chrono([] (py::module &m) {
+    m.def("test_chrono1", &test_chrono1);
+    m.def("test_chrono2", &test_chrono2);
+    m.def("test_chrono3", &test_chrono3);
+    m.def("test_chrono4", &test_chrono4);
+    m.def("test_chrono5", &test_chrono5);
+    m.def("test_chrono6", &test_chrono6);
+    m.def("test_chrono7", &test_chrono7);
+});

tests/test_chrono.py

@@ -0,0 +1,116 @@
+
+
+def test_chrono_system_clock():
+    from pybind11_tests import test_chrono1
+    import datetime
+
+    # Get the time from both c++ and datetime
+    date1 = test_chrono1()
+    date2 = datetime.datetime.today()
+
+    # The returned value should be a datetime
+    assert isinstance(date1, datetime.datetime)
+
+    # The numbers should vary by a very small amount (time it took to execute)
+    diff = abs(date1 - date2)
+
+    # There should never be a days/seconds difference
+    assert diff.days == 0
+    assert diff.seconds == 0
+
+    # We test that no more than about 0.5 seconds passes here
+    # This makes sure that the dates created are very close to the same
+    # but if the testing system is incredibly overloaded this should still pass
+    assert diff.microseconds < 500000
+
+
+def test_chrono_system_clock_roundtrip():
+    from pybind11_tests import test_chrono2
+    import datetime
+
+    date1 = datetime.datetime.today()
+
+    # Roundtrip the time
+    date2 = test_chrono2(date1)
+
+    # The returned value should be a datetime
+    assert isinstance(date2, datetime.datetime)
+
+    # They should be identical (no information lost on roundtrip)
+    diff = abs(date1 - date2)
+    assert diff.days == 0
+    assert diff.seconds == 0
+    assert diff.microseconds == 0
+
+
+def test_chrono_duration_roundtrip():
+    from pybind11_tests import test_chrono3
+    import datetime
+
+    # Get the difference betwen two times (a timedelta)
+    date1 = datetime.datetime.today()
+    date2 = datetime.datetime.today()
+    diff = date2 - date1
+
+    # Make sure this is a timedelta
+    assert isinstance(diff, datetime.timedelta)
+
+    cpp_diff = test_chrono3(diff)
+
+    assert cpp_diff.days == diff.days
+    assert cpp_diff.seconds == diff.seconds
+    assert cpp_diff.microseconds == diff.microseconds
+
+
+def test_chrono_duration_subtraction_equivalence():
+    from pybind11_tests import test_chrono4
+    import datetime
+
+    date1 = datetime.datetime.today()
+    date2 = datetime.datetime.today()
+
+    diff = date2 - date1
+    cpp_diff = test_chrono4(date2, date1)
+
+    assert cpp_diff.days == diff.days
+    assert cpp_diff.seconds == diff.seconds
+    assert cpp_diff.microseconds == diff.microseconds
+
+
+def test_chrono_steady_clock():
+    from pybind11_tests import test_chrono5
+    import datetime
+
+    time1 = test_chrono5()
+    time2 = test_chrono5()
+
+    assert isinstance(time1, datetime.timedelta)
+    assert isinstance(time2, datetime.timedelta)
+
+
+def test_chrono_steady_clock_roundtrip():
+    from pybind11_tests import test_chrono6
+    import datetime
+
+    time1 = datetime.timedelta(days=10, seconds=10, microseconds=100)
+    time2 = test_chrono6(time1)
+
+    assert isinstance(time2, datetime.timedelta)
+
+    # They should be identical (no information lost on roundtrip)
+    assert time1.days == time2.days
+    assert time1.seconds == time2.seconds
+    assert time1.microseconds == time2.microseconds
+
+
+def test_floating_point_duration():
+    from pybind11_tests import test_chrono7
+    import datetime
+
+    # Test using 35.525123 seconds as an example floating point number in seconds
+    time = test_chrono7(35.525123)
+
+    assert isinstance(time, datetime.timedelta)
+
+    assert time.seconds == 35
+    assert 525122 <= time.microseconds <= 525123