Tinderbox Date data and timezone offsets

Tinderbox Tasks
1

[admin edit] These posts split from topic Looking for way to set alarm to ToDo (list) / external app?

I have been assuming, wrongly, that Tinderbox, like some other applications, stores an ISO date-time string that doesn’t change just because I change the time zone in Date & Time Preferences on the Mac.

Created for a sample note, for example, changes from “2019-07-03T16:01:26+08:00” to “2019-07-03T04:01:26-04:00” when I switch the time zone from Taipei to Boston. These are equivalent, of course, as shown by converting both to coordinated universal time. (I assume a similar conversion to UTC is easily done in JavaScript, perhaps built-in).

AppleScript to convert to UTC (click to reveal) 

to convertToAsDateUTC(isoDate)
	set text item delimiters to {"-", "T", "+"}
	set asDate to current date -- a "placeholder" date; actual values substituted in below
	tell isoDate's text items to set {asDate's year, asDate's month, asDate's day} to {item 1, item 2, item 3}
	tell isoDate's text item 4 to set asDate's time to hours * (word 1) + minutes * ((word 2) + (word 3) / 60)
	set asDate to asDate + hours * (isoDate's text 20 thru 22) * -1 # time zone offset
end convertToAsDateUTC


-- TEST -----------------------------------------------------------

set strDate1 to "2019-07-03T16:01:26+08:00"
set strDate2 to "2019-07-03T04:01:26-04:00"

set asDate1 to convertToAsDateUTC(strDate1)
set asDate2 to convertToAsDateUTC(strDate2)

return {asDate1, asDate2}

That I think means the time zone in which a note was created, if important, needs to be tracked in other ways and the original local date and time, if important, needs to be reconstructed. Fair enough, now that I think I understand it.

But as @ComplexPoint suggests, summer time and its effect on historical dates is a knotty problem. How does JavaScript deal with that? Can both AppleScript and JavaScript access NSDate and NSCalendar?

2

As long as we are delegating to well-tested dateTime libraries it probably doesn’t matter too much which particular library we use.

JavaScript for Automation, like AppleScript, can use more or less all of the NS classes, but in the case of dates, JS has also has its own Date object, which is subject, if anything, to even more intensive use and testing than the excellent NSDate, because of the context of the web, and the competitive refinement and optimisation of JS interpreters in the course of commercial competition between Chrome, Safari, Mozilla etc etc

(JS for Automation – osascript JS– uses the same JS interpreter as Safari and the JSContexts which we can create through the ObjC interfaces)

3

Say one has historical dates in Tinderbox. Some originally were entered while in another time zone but all have now been “converted” automatically to the time zone to which the Mac was set when the Tinderbox document was last opened and saved. Can one use the “well-tested” libraries to take these historical dates and, given a time zone offset from UTC, convert them back to the orginal local date and time, taking into account whether summer time was or was not in effect on each historical date in the original time zone?

If so, would love to know how that is done, as I’ve encountered this situation.

(Edit. For JavaScript I found this: “Javascript date objects are always in the local time zone. If a date and time is provided in a different time zone, it must be dealt with manually as the date object’s time zone offset is read only. Consequently, there may be issues if daylight saving is observed in one location but not the other.” Still true? True for “NS classes” as well?)

4

Nullius in verba – there’s never a substitute for concrete testing with specific examples, but perhaps that formulation, with its image of zone->zone surface level translation:

“converted” automatically to the time zone to which …

Might risk introducing some confusion.

The NSDate docs summarise well what date libraries do – they internally represent the date in a ‘deep’ format independent of particular time zones, and only translate from deep structure to surface structure when required, in the light of the time zone information presented at translation time.

NSDate objects encapsulate a single point in time, independent of any particular calendrical system or time zone.

We need date libraries rather than strings because the former translate dates into a time-zone independent format.

5

Thanks for the explanations.

“converted automatically to the time zone to which the Mac was set"

Could perhaps have been more fully expressed as follows.

"When a date-time is entered into a date attribute in Tinderbox that point in time is represented by an ISO string containing the local date and time in the time zone to which the Mac is set at the time of entry with an appended offset from that original time zone to UTC.

When the time zone setting on the Mac is changed, Tinderbox automatically changes the original ISO string to express the same point in time as the local date and time in the time zone to which the Mac is now set with an appended offset from the new local time zone to UTC."

Converting the old and new ISO date strings to UTC, either via a library or in AppleScript via a simple handler like the one above, as demonstrated, shows that Tinderbox does the right thing and the two different strings represent the same point in time.

The practical problem that I’ve encountered is this. Working with the “new” ISO string that Tinderbox automatically generates to replace the original one, how does one, for a series of historical dates, get back to the original local date-time in the original time zone? If one knows the time zone offset for the original time zone (and knows how to retrieve the “new” time zone offset from the new ISO string, as in the handler posted above), basic AppleScript time arithmetic quite easily gives the answer, without having to make a trek to the library.

But, but … what about summer time? For a series of historical dates it’s sometimes observed in one time zone but not in the other, messing up the arithmetic. Do the libraries automatically account for that? See the edit to my post above to include an old reference to JavaScript that suggests, sadly, that summer time “must be deal with manually.” Is that still the case for JavaScript? Is it also the case for the “NS classes”?

1 Like
6

Neither NSDate nor JS Date hold dates as strings (they store them essentially as numbers representing a time-zone independent moment, an offset from some arbitrary but absolute moment).

We always need to think in terms of two separate translations:

  1. Translation into a numeric representation that is independent of calendar system and time-zone delimitations.
  2. Translation from an absolute (numerically encoded) moment to a string representation language based on a particular calendar and a particular segment of a time-zone system.

(Fortunately, there is never any kind of string to string translation)

Any constraints will be relevant to one of these two arrows (String -> Date) and (Date -> String)

Any constraints will mainly operate on the (String -> Date) morphism.

Did the original string contain sufficient information for translation into an absolute and calendar/zone independent moment ?
Did it, for example actually include any potentially relevant time zone or summer time information ?

Such input date strings will seldom be complete up to the millisecond, eliding perhaps:

  • the specifics of the zone,
  • the state of any Summer Time adjustment
  • any information about time of day
  • or some level of granularity in the time (minutes but not seconds, seconds but not milliseconds)

In that case the (String -> Date) translation will deal with absent information by normalising to a default, typically for example, some kind of zero.

In the case of both NSDate JS Date that zero would, I think, be:

  • UTC when the source lacks a zone and summer time state
  • Midnight when it lacks a time
  • 0 minutes when it lacks minutes
  • 0 milliseconds etc when it lacks milliseconds.

In other words, no information is discarded, and absent information is represented by a zero.

Finally the second translation/morphism (Date -> String), which is conducted only on demand and “just in time” in response to request for displays and XML serialisations etc:

  • Never discards information unless asked to (e.g. we might ask for date without time)
  • Can only use what information was offered to translation 1 (String -> Date)
7

For an example, if I replace ‘never’ with ‘tomorrow’ in a StartDate field now,

Tinderbox:

  • asks NSDate for the number of seconds (down to a sub-millisecond approximation) elapsed since the start of the champagne pop at Greenwich at the turn of this century,
  • Adds 24 hours worths of seconds to that and stores the result as a new NSDate

(i.e. perhaps something analogous to an incantation like NSDate.init.timeIntervalSinceNow(86400) where 86400 is a day of seconds.)

When I save the file, the serialization to XML asks that NSDate value for an ISO8601 representation of itself, and the NSDate (containing a number of seconds since the UCT millennium champagne cork) generates the string:

2019-07-04T13:58:17+01:00

(Assuming that I sit on a summer balcony in London, EU)

which is written out in the XML (without loss of information),
and can be read back into an NSDate value when the file is next opened.

(and equally, of course, the locale-specific string displayed in GUI date attribute fields is also a string generated on-demand and just-in-time as an NSDate -> String translation from the numeric offset in seconds from UCT cork pop).

8

Try entering two notes with historical StartDates, including date and time, one before summer time starts, one after summer time starts.

Note the ISO date strings retrieved by script.

Then close the document and switch to a time zone the doesn’t have summer time. Taipei will work for that.

Open the document and note the ISO date strings and displayed dates in Tinderbox.

Then, without closing the document, switch the time zone back to the one in which the dates are entered.

Note the ISO date strings and displayed dates in Tinderbox.

As expected? Shouldn’t it be the date-times as originally entered?

(For me this isn’t academic; I go back and forth between a zone with summer time and one without and want to be able to reconstruct local date-times if Tinderbox doesn’t preserve the original date-time string. )

9

That does seem unlikely to be prudent.

It relies on Tinderbox resampling the user locale settings, which may be cached in the program’s run-time state - you would have to ask @eastgate, but it wouldn’t particularly surprise me if the user locale was, for example, re-read from the system only in response to particular event triggers, such as file open.

(see, for example currentLocale | Apple Developer Documentation)

10

Oddly enough, this probably will work; as it happens, we do listen for a locale-change notification because changing locale with Tinderbox open invalidated calendar objects.

That meant Tinderbox could crash when arriving at a new airport, but only after a long plane flight. (This was a bear to find!)

11

Here are results I get… If I close the document, change time zones, and reopen, Tinderbox alters the ISO string for a historical date by changing only its time zone offset, which results in a date-time that is not equivilant to the original. The date displayed in Tinderbox remains on the original local time, not the new local time (intended?). Then when I switch back to the original time zone in which the date was entered the ISO string is restored to its original state. If I don’t close the document when changing time zones then Tinderbox changes the date string in unexpected ways. It doesn’t seem to make a difference whether summer time was or was not in effect in one of the time zones on the historical date. Tinderbox seems to be ignoring that.

Any chance of leaving the ISO date string itself unchanged when changing time zones? With the original intact it will be much easier to reconstruct local time as needed (including dealing with summer time, if relevant).

Details (which I tried, but failed, to hide behind a disclosure triangle fixed by admin, @mwra):

I. Summer time not yet in effect for either time zone

A. Forgetting to close document before switching time zones:

Time zone set to Taipei
StartDate input as 3/8/19, 1:00 AM
ISO string to script: “2019-03-08T01:00:00+08:00”

Close document and switch time zone to New York
Reopen document
ISO string script retrieves: “2019-03-08T01:00:00-04:00” (unexpected; not an equivalent date-time)
Date displayed in Tinderbox KeyAttributes: 3/8/19, 1:00 AM

Switch time zone back to Taipei (without closing document)
ISO string script retrieves: “2019-03-08T14:00:00+08:00” (unexpected)
Date displayed in Tinderbox KeyAttributes as 3/8/19, 2:00 AM. (unexpected)

B. Being careful to close document before switching time zones:

Time zone set to Taipei
StartDate input as 3/8/19, 1:00 AM
ISO string retrieved by script: “2019-03-08T01:00:00+08:00”

Close document and switch time zone to New York
Reopen document
ISO string retrieved by script: “2019-03-08T01:00:00-04:00” (unexpected; not an equivalent date-time)
Date displayed in Tinderbox KeyAttributes: 3/8/19, 1:00 AM (expected? still on Taipei time)

Close document and switch time zone back to Taipei
ISO string to script: “2019-03-08T01:00:00+08:00” (expected)
Date displayed in Tinderbox KeyAttributes: 3/8/19, 1:00 AM (expected)

I. Summer time in effect in one time zone but not the other

A. Forgetting to close document before switching time zones:

Time zone set to Taipei
StartDate input as 3/10/19, 1:00 AM
ISO string retrieved by script: “2019-03-10T01:00:00+08:00”

Close document and switch time zone to New York
Reopen document
ISO string retrieved by script: “2019-03-10T01:00:00-04:00” (unexpected; not equivalent date-time)
Date displayed in Tinderbox KeyAttributes: 3/10/19, 1:00 AM (unexpected? Still on Taipei time)

Switch time zone back to Taipei (without closing document)
ISO string retrieved by script: “2019-03-10T14:00:00+08:00” (unexpected)
Date displayed in Tinderbox KeyAttributes: 3/10/19, 2:00 PM (unexpected)

B. Being careful to close document before switching time zones:

Time zone set to Taipei
StartDate input as 3/10/19, 1:00 AM
ISO string to script: “2019-03-10T01:00:00+08:00”

Close document and switch time zone to New York
Reopen document
ISO string to script: “2019-03-10T01:00:00-04:00” (unexpected; not equivalent date-time)
Date displayed in Tinderbox KeyAttributes: 3/10/19, 1:00 AM (expected? Still on Taipei time)

Close document and switch time zone back to Taipei
ISO string to script: “2019-03-10T01:00:00+08:00” (expected)
Date displayed in Tinderbox KeyAttributes: 3/10/19, 1:00 AM (expected)

Script used was:

tell front document of application "Tinderbox 8" to return value of attribute "StartDate" of selected note
12

Orthogonally to your question to @eastgate, if your reference to a ‘historical date’ is to something iconic and meaningful but chronologically incomplete like 14 July 1789 (incomplete only in the sense that it lacks any time and ‘zone’ specifics)

then … might you be better served by a string attribute than by a date attribute (which doesn’t hold a string at all, though its NSDate-type contents can be serialised to a string on request, for GUI display and file saves etc ) ?

It could be that for some of your purposes, a date-type attribute is really not what you are after, and that any computations you do want to base on strings representing dates could be done through action script, applescript etc

Returning to your starting point:

I have been assuming, … , that Tinderbox, like some other applications, stores an ISO date-time string that doesn’t change just because I change the time zone in Date & Time Preferences on the Mac.

It certainly can, when that is what you want – just put the ISO 8601 string in a string-type attribute rather than in a date-type attribute.

(and, of course, even when you do use a date-type attribute, its time-zone independent contents is not affected by locale changes. The only thing that is locale-sensitive is the response given to requests for stringification)

13

I don’t entirely follow. As described (you might try replicating on your machine) when I’ve entered a date-time in, say, Taipei, that is represented by an ISO string that correctly represents that point in time, when I close the document and switch the time zone setting on my Mac to say, New York, and reopen the document, the date string changes and now describes a different point in time (the time zone offset has changed but the date and time components remain as they were). Is there an advantage to that? It seems “wrong” because the point in time described by the ISO date string has changed simply because I changed a setting on my Mac. Isn’t it better to leave the ISO date string unchanged, so that it represents the correct point in time, and base calculations on that?

14

The first thing to clarify is that no strings are involved.

Once you’ve entered a date into a date attribute, is is translated, in the light of your locale, into a locale-independent (and calendrical system independent) numeric value (an NS Date object), which represents an absolute point in time, independent of all time zones.

That numeric value, which is not a string, is not subsequently affected by any time zone shifts. Once translated into that absolute reference to a sub-millisecond moment, beyond region and calendrical culture it is immutable.

Whenever a stringification of it is requested (for GUI display, or for saving into an XML file), that translation is locale sensitive, but the underlying value is unchanged.

If it turned out that the numeric value of the NS Date (seconds since millennial pop) was changing, that would be a bug.

15

If you want to experiment with this kind of thing, and check whether or not Tinderbox is successfully leaving the NSDate value unchanged, you can get the numeric value of an NSDate using AppleScript.

Here, for example, we initialise an NSDate value to the current run-time instant, (translated to universal time in the light of our locale settings) , and then obtain its floating point numeric value. Not an integer because it represents seconds, but allows for fractions of seconds at 1/10000 resolution.

https://developer.apple.com/documentation/foundation/nsdate?language=objc

use AppleScript version "2.4"
use framework "Foundation"
use scripting additions

on run
    
    tell current application
        set nsDte to init() of alloc of its NSDate
    end tell
    
    -- As floating point number
    set floatSeconds to timeIntervalSinceReferenceDate() of nsDte
    
    -- We need to clear ObjC pointer references
    -- or it may not be possible to save our script
    set nsDte to missing value
    
    return floatSeconds
end run
16

And having done the tests, I agree with you :slight_smile:

It seems that when Tinderbox opens a file, reading its XML serialisation of dates from ISO8601 back into NSDate objects, it discards the zone components of the ISO8601 strings in the file, and uses the system NSLocale zone instead.

@eastgate is that by design ?

(I can see how some might want it for diary use – sitting in London and making note of an 11am appointment in Beijing next week, I guess I might want to see 11am in the current GUI display, and still see it as 11am when I arrive in Beijing.

For other uses of course, discarding time zones might look like data loss)

17

I can only see and talk reasonably intelligently about the “string” part of the elephant so forgive my fixation on them.:grinning: The description of what may be going on in the bowels when the strings are read into date objects sounds good to me.

Talking about the strings, note that the behavior appears to be different when the document is not closed before switching time zones. The date and time part of the string change as well as the TZ offset. Better. But the resulting ISO string can still end up representing a different point in time than the original when summer time is effect now but wasn’t on that date in the past. And the original time zone information is lost. (see details below).

Keeping the original ISO string accessible to a script and reading that into date calculations instead of a modified string would avoid loss of time zone information and would also avoid the problem of ending up with strings that represent different points in time, leading to an unexpected shift in date-times when switching to another time zone and then back again.

Details on changing ISO date strings when switching TZ without closing document

Before summer time is in effect in one time zone

Taipei TZ
Entered as 3/8/19, 1:00 AM
“2019-03-11T01:00:00+08:00”

Change to NY TZ (without closing doc)
String: “2019-03-07T12:00:00-04:00” (not equivalent; and lose original TZ information)
Display: 3/7/19, 12:00 PM (expected)

Back to Taipei TZ (without closing doc)
String: “2019-03-08T01:00:00+08:00” (expected)
Display: 3/8/19, 1:00 AM (expected)

After summer time is in effect

Taipei TZ
Entered as 3/15/19, 1:00 AM
“2019-03-15T01:00:00+08:00”

Change to NY TZ (without closing doc)
“2019-03-14T13:00:00-04:00” (equivalent; but lose original TZ information)
Display: 3/14/19, 1:00 PM (expected)

Change back to Taipei TZ (without closing doc)
“2019-03-11T01:00:00+08:00” (expected)
Display: 3/11/19, 1:00 AM (expected)

18

There does seem to be an asymmetry, at least in this build – zone information is written out to XML in when a .tbx document is saved, but ignored when the document is opened.

Of course, if enter your date (in any format) in a string attribute instead of a date attribute, it should round trip (in saving and opening) without any data loss.

19

Yes, thanks, one way to preserve the original. Not optimal, of course, because of much more typing, no date picker, etc., and no calculations or plotting on timeline possible.

I was thinking another possibility might be to preserve the original ISO date string by remembering to run action code something like:

$MyDateString=$StartDate

That preserves the original ISO date string.

But after switching time zones (without closing document) and running…

$StartDate=$MyDateString

… then using a script to retrieve the value for the attribute $StartDate

The retrieved ISO Date string is neither the same as the original nor does it represent the same point in time as the original. (The date time components are the same as the original; only the time zone offset has changed.)

20

Just in case there is a general ISO8601-reading function in Tinderbox code which (by design or otherwise) discards time zones, I would (for the moment at least) tend to use AppleScript / JS, and either the NS classes or JS Date, for the String -> Date process.

(Not sure if invoking osacript -e “…” with runCommand would work - I haven’t tried)