AutoFetching and DEVONthink - some ideas

[DaveM (Dave Mee)]

God knows I love to import from DEVONthink! The truth is that I don’t even write in DEVONthink, I just index files. But it is a very flexible way of working. I can write in an ever-shifting roster of CRIMPed editors.

Having spent the weekend jiggling around a long outline constructed entirely of DEVONthink imports, I have a few thoughts I’d like to share. Some of them may be feature requests, even.

They way I operate is to have each item set to AutoFetch, so I can rely on Tinderbox updating, and my own discipline not to pull myself up by my own rug.

1. AutoImport of .md renders text as rich text - I’d love to see a mechanism to make it remain plain text (I’m importing everything as .txt in DEVONthink, but then it becomes a matter of where the magic interpretation happens - some editors will treat .txt as .md, but quickview won’t).

2. AutoImporting of text where a line begins with an asterisk treats it as a list item (even if it’s not). This could be nearly bypassed, I think, if there was some mechanism related to 1 available.

3. AutoImporting an existing note also imports the file extension as part of the name - I’d love for this not to happen (or for there to be a flag to strip it on import) - it can’t be scripted away, as any changes to the contents of the autoimported note will clobber it again.

   3a. To which point, a mechanism to not rename a note on update would be useful, as I have a quite obtuse process between the two applications.

4. unrelated, but…! the addition of a note to flag it as ‘dead’ or ‘descriptive’ would be really useful, if it did two things: stopped it counting towards $SiblingOrder (I’d like to be able to have an item in a list but to not have it participate in the export and counting process, without fully deleting it from the document; I can strip items out using templates and properties, but using SiblingOrder in templates means they still leave numbered ghosts behind). Maybe there could be a new property, $ExtantSiblingOrder, and a $ExtantSibling flag to write them out of the family, without impacting existing use of $SiblingOrder?

Just some thoughts and ideas I’ve had after managing a 300-item, markdown-based document in DEVONthink and Tinderbox.

[eastgate]

@DaveM

4. unrelated, but…! the addition of a note to flag it as ‘dead’ or ‘descriptive’ would be really useful,

I think you might achieve this by defining a boolean $Descriptive and a function that steps through the children in the container and assigns $ExtantOrder in sequence, omitting $Descriptive notes. The container might call this function as a rule or edict.

[mwra (Mark Anderson)]

I think there is an answer to all these, but the way to think of it is to use available features rather than add niche tweaks to existing one. So, starting with the last.

Issue 4. $SiblingOrder. This does and should, for all sorts of reasons count the number of siblings. If you need different, it is easy to set and calculate such. This shifts the problem of why can’t you use Tinderbox view the count you want. So let’s assume you add a user boolean $IsDeadSibling, default false^†. If a sibling note is regarded as non-extant (dead) your set its to true (how/where is a separate discussion to have). Now the container’s rule or edict sets $ExtantSiblingOrder:

$ExtantSiblingOrder = sum_if(children, $IsDeadSibling==false,1);

You can now use $ExtantSiblingOrder in the counters Displayed Attributes table, in a Display Expression, etc., etc.

Issue #1 import text format. Normally, $Text default to being styled (‘rich’) text. By using things like a Markdown template and applying it to a new note before the $Text is added this can avoid the problem. But if the 4Text is already there, this map can fix the problem:

$Text = $Text.plain;

(sets $Text to the plaintext version of itself replacing the styled version)

Or, look at AutoFetch’s $AutoFetchCommand. I don’t have the DEVONthink end set-up to test. But you could try getting the $AutoFetchCommand to set the Markdown prototype which likely takes care of issue #1 but if not, use the above code.

Issue #2. The current default/expected house style is that Markdown-style bullet notation (asterisk or bullet (Cmd+8 at link beginning) is intended. This meshes with quick lists (for export). So, Quick Lists. Not here yet but likely soon, a $QuickLists system boolean will give finer control (akin to $SmartQuotes, etc.). So you have a today workaround and a more formal simple solution coming soon.

$QuickLists, a boolean attribute, enables and disables quickest parsing. – ed.

Issue #3. Assuming we set a prototype at issue #2 above, we generate a Display Expression (and thus $DisplayName) that strips the file extension. The exact code for that is a separate question as it depends whether(a) all imports have a file extension and the maxim length of said extension. But if the sans-extension name is saved into $MyString, then the $DisplayExpression is as simple as $MyString (N.B. no semicolon terminator used in a Display Expression). Now the DEVONthink note imported as “Great Idea.txt” has the name Great Idea.txt but $DisplayName of Great Idea which is what you see in you map, uptime etc (N.B. the Text pane top shows $Name and $Not $DisplayName)

Issue #3a. The above takes care of this as you see $DisplayName and not name so the problem goes away.

Note how this uses existing features and for very limited effort compared to a built-in option (must be an option else it screws up other user’s work), tested, and maintain, and changed if AutoFetch
(or DEVONthink behaviour change). In the above method you just tweak the code in you prototype.

TIP. If yuo do embrace any of the ideas above, once tested and happy, make a note of what you did. My experience is this is the sort of detail I quickly forget. So note, the setting changes needed and put it wherever you won’t loose it—DEVONthink?

HTH

†. Tip for making booleans. Make the false state the most likely and default state. Thus you set them to true to make the note as different. Thus a name like IsDeadSibling implies that a true value means the note is considered dead, else live and in use. No need to go double-check the meaning. If you have to do so it indicates you might want to reconsider your first choice of attribute name!

[DaveM (Dave Mee)]

Mark(s), thank you both: @mwra, there’s a stack of stuff I had no idea about there.

I agree that using existing features is better than adding endless new edge cases, but didn’t know of the functionality you pointed towards.

④ SiblingOrder can wait for now - I can come back and amend this note later.

① $AutoFetchCommand - I did not know about this. I have a tree of items that are already set to a Markdown-descended Prototype, but this immediately fixes points around ③ (and 3a, for which I do not have a fancy glyph).

While this works great for other features, the import is still an issue: items like headings, bold, etc are magically consumed into RTF text, so changing the note back to $Text.plain eliminates the hashes and asterisks used for formatting, so an ongoing output will not restore the markdown. So close!

② I’d expect a bullet from "* ", (asterisk followed by space) but not from “*Text” (asterisk followed immediately by non-space), which is much more an opening italic than a list to my fully-markdown eyes. I will leave this as a cosmetic point for now.

Thanks for all the great suggestions, off on a ① $AutoFetchCommand exploration!

[mwra (Mark Anderson)]

Thanks for the questions! I don’t recall talk of $AutoFetchCommand for ages, but there it is, lurking at the back of the tool box. Indeed, I’m really glad to hear it worked (as I didn’t have time to fully test).