Arrow: Better dates and times for Python
crsmithdev.com
crsmithdev.com
Arrow.to() converts to a new timezone? And .replace() applies a relative offset!? Replace the hour field of this object with -1 should not return an object having hour=11. Arrow.get() is doing some kind of quadruple duty, neither of which would I describe as "getting."
And what about that class name? Arrow as the name of a package is fine, but what do you expect someone to make of <Arrow [...]> -- what's wrong with arrow.DateTime?
Great work on making and releasing something, but this API is surprising -- as in, one would be unable to predict how it works. I will continue using python-dateutil
Naming should be much clearer than this, with the intention to make code sort-of read like English text as much as possible. Arrow is a fine name[0], but the actual class/object names should be descriptive.
Also, not only the fact that .replace() can apply relative time-offsets, the quirky way it signifies it, using singular and plural keyword-parameters, it's a "clever" solution but it's not at all intuitive. When I figured it out, it was more of a relief that at least .replace() does not always apply relative time-offsets (which would have been terrible naming).
I wish it was more pythonic, on the whole, agreeing with the way people expect things to behave in Python modules. It says it's inspired by Requests, which gets that very much right, so I was getting my hopes up a little.
Arrow.get() is totally named the wrong thing. It does more like a .create(), maybe I would've put that functionality into the constructor. A method called .get() should implement something very similar to what most .get* methods do on standard types and in the standard library. Which is, extract part of an object that is a container-type (optionally with defaults, etc). Of course that operation does not make sense on an Arrow object (and please don't try), so it probably just shouldn't have a .get() method.
[0] actually it's not. I've found myself accidentally muscle-memory-typing "Array" a couple of times already. I'd probably have renamed this module early on in the process for that reason alone.
Not to be confused with Delorean (https://github.com/myusuf3/delorean) which is a lot like Arrow.
I strongly agree with the principles you chose - specificaly - all time storage and math should be done in UTC (the 'unicode' of time). Timezones only come in when you display or parse times (the 'encodings' of time).
I hope more and more libraries apply this idea. It makes the API a lot simpler and error retardant.
Good job!
I strongly agree with the principles you chose - specificaly - all time storage and math should be done in UTC
Having worked on a few data cleaning projects to re-purpose old scientific datasets, I can tell you that the quality of these data would have suffered markedly, and many corrections would not have been possible if the TZ/offsets had been discarded, Eg: "The geolocation for these records cross a TZ boundary here but the UTC offset doesn't reflect that until here so that explains the 1hr gap in the record where the user realized her mistake half-way into the new TZ so we should add an hour to these 860 records"
Yes, it would be better if raw data was recorded in UTC in the first place but when working with such data the ability to make decisions and inferences from the TZ is very useful.
I maintain that time and location are orthogonal. If the application needs both, store both - properly in separate fields. That's better than shoving the granular location in the form of a timezone in the time field.
From an evidence-based perspective though, a full "chain of custody" has to be maintained (as far as alterations go), providence of information can be very important.
If you care about reproducibility, how raw data has been manipulated from the original, you'd better have some record of it.
For date-times almost all public datasets standardize on ISO8601. Any datetimes captured by persons or processes which aren't working in UTC from the very beginning will have the TZ offset in it.
Normalizing to UTC is up to the data consumer to do as they please, but some get very upset if you try to say that an event was recorded as happening at 03:00 if the person or process actually recorded 13:00+10:00. It may seem subtle and boring and pointless, but the fact is that you may be seen to be changing the facts.
We've got a complete clusterfuck in our legacy code because somebody decided that UTC was not the right answer, nor was timezones quite, and so somehow made the heinous mistake of creating a "local UTC" and separately storing the TZ. The mind boggles.
I'm still straightening that mess out.
People, computers care not for your "calendar time"--usually, you want physical time and can fix the rest at the view level.
It is in fact best to store the local time and the location (i.e., not "+01:00", but for example "Europe/Amsterdam"). Not UTC with or without time zone. For example, if you have an appointment in the future, and the time zone or daylight savings time rules change, this is the only way that the appointment can still be at the correct moment. And time zone rules do change.
And this way you can still represent absolute UTC time if you wish. With another representation, you cannot represent human-society time, which is often what you actually want.
IOW, I think storing local time and location as a rule is not a good idea. The rule should be to use UTC and store location if needed. Only if that doesn't work (e.g. '5 minutes after the game is over') come up with some other representation.
I was fairly sure ISO8601 with correct timezone information for every item, was completely sufficient for all my time needs, so if theres something it fails at, id like to know more.
Have you looked at the moment.js manual ? They have a lot of really useful examples (I believe Arrow was inspired by Moment).
I wrote cpppo (https://github.com/pjkundert/cpppo.git) to work with industrial data communications in various timezones. The cpppo.history module assures fast and consistent handling (and serialization/deserialization) of time series data across a selected set of consistent timezone abbreviations.
If speed and consistency is an issue it may be of interest.
I bring up dmc for discussion, because I started it right after finding arrow. Building a full wrapper like this hadn't occurred to me until I saw arrow do it.
> Python’s standard library and some other low-level modules have near-complete date, time and time zone functionality but don’t work very well from a usability perspective:
Why be consistent with something that suffers from usability problems?
Not that it's a big deal to loop in external dependencies or anything, I just see newer (and experienced) Python devs trip up on stuff like TZ aware/naive times. Also, dateutil's relativedelta is so nice compared to the built-in timedelta!
That's on point, and you're not alone. It's a little frustrating when you have to take a detour to get something done. There's that extra hurdle. That said, user friendliness, developer friendliness, and maintainability are all about using the best tools for the job. If arrow is that tool, then so be it.
Definitely. The most unfortunate part of this is that people who are newer to the ecosystem won't really know where to go or what they're even looking for, necessarily.
Unlike datetime.now() and datetime.utcnow(), arrow.now() produces a datetime that includes tzinfo. As long as that's on there and correct, I think the risk you describe is low.
"""calpaterson's awesome project"""
import arrow
arrow.localnow = arrow.now
del arrow.nowDocumentation is very educational too well worth the read. http://delorean.readthedocs.org/en/latest/
They kept things down to just DateTime (timezone aware, but only supports local and UTC) and Duration, and assumed that anything more advanced should be outside the core.
I'm not sure if they got it right or not, but we haven't seen anyone want to replace the built in types yet. I think we've always expected a Joda-Time like library eventually, but hopefully DateTime and Duration are good, so it's additive.
(I'm on the Dart team, but not on the core lib team)
While it doesn't have all the bells and whistles you would get with a 3rd party library like the one posted here it does have an advantage that date & time are both first class datatypes:
>> type? 6-Aug-2014
== date!
>> type? 01:20:59 ; 1 hour, 20 mins and 59 secs
== time!
>> 6/8/2014 - 5/8/2014
== 1
>> difference 6/8/2014 5/8/2014
== 24:00
>> yesterday: now - 1
== 5-Aug-2014/13:12:33+1:00
>> yesterday/day
== 5
>> one-hour-twenty-mins: 1:20
== 1:20
>> one-hour-twenty-mins/minute
== 20
>> one-hour-twenty-mins/hour
== 1
>> two-hours: one-hour-twenty-mins + 0:40
== 2:00
What's the difference between .utcnow() and .now()? Is one supposed to be an alias or is .now() default to the time the platform is set to?
I've opened an issue regarding calling convention. What do HNer think?
The arrow.get function tries a bit too hard to be user-friendly ; it lets me wondering if a value could have an unexpected interpretation.
One use-case that seem to be missing is time deltas. There is a support for time iteration but I don't see a good way to transform two timestamps into a time interval.
I just fired up ipython and tried:
>>> t1 = arrow.now()
>>> t2 = arrow.now()
>>> t2-t1
datetime.timedelta(0, 4, 864716)
In my experience, the main issues are: [1] How do we store the date / time fields in the database tier [2] How do we display those fields to users in multiple time zones and cultures and gather validated input from those cultures [3] How do we do date / time math in a simple manner
Issue [1] has implications for manipulating data directly (i.e. querying or reporting). Should the data be stored in UST or in some "local timeframe"?
Issue [2] always seems to trip some developers: how do we render a time or date in a way that makes sense in a global world (i.e. "8/5/2014" vs. "5/8/2014" vs. "5-8-2014" or 3:26 PM vs 3:26 etc).
Issue [3] is always a pain too, based on whatever decisions were made for [1] and [2].
Test of awesomeness: if your code works in Hebrew or Thai cultures!
I've implemented something similar in a query language I wrote, where dates are represented similar to ISO 8601 format:
d'2014-12-01'
t'2014-12-01 12:52'
t'2014-12-01 12:52 PST'
t'2014-12-01T12:52:20.0820Z'
t'2014-12-01T12:52:20.0820+08.00'Do you have so many time literals in your code?
When you don't need to do that, a function call is a much better answer. There are many things that could be string literals but aren't because they don't have to be.
My use case is the iPython astrophysicist use case, where users tend to be very interactive with their dates and times. In fact, that's why I made the query language, because it was necessary for dataset discovery (i.e. return datasets between this time and that time). Of course, we often use a variety of types of times, from Julian day to Mission Event Time (where an arbitrary epoch is chosen that's usually close to some significant event, usually full funding, mission launch, first light, etc..)
Of course we could always just use function calls, but the ability to have literals is really nice.
Maybe I'm just a dreamer, but a similar extension to javascript and JSON would be really nice as well, given that you need a schema to understand the proper parsing of dates, otherwise you end up with a string, number, additional type field, or a fallback to Hungarian notation.
Why would you do it by hand anyway when Babel already does it? http://babel.pocoo.org/docs/api/dates/#babel.dates.format_ti...
There's an obligatory xkcd somewhere on this...