Thinking through an idea - am I modelling this the wrong way, mentally?

[DaveM (Dave Mee)]

I have a giant Tinderbox document with a few branches in it. One branch is an outline of my PhD thesis, each node of which pulls in markdown documents stored in DEVONthink via autofetch. While this may sound obtuse, it works well for me, as I can edit the documents on DEVONthink To Go on my iPad and then update when I get back on my mac.

One of the things I do, as a consequence of this approach, is to leave me notes to deal with later inline with the text. {xyz: a note} {q: a question}, as I don’t use these parentheses elsewhere (other than through Bookends).

I would like to build an agent to sift through the Tinderbox document, looking for these markers. What I’d love to be able to do is extract the section question as a new note, link it to the original paragraph (or note, at the least), and capture these in a separate tree in the main Tinderbox document.

Am I thinking about this all wrong? Should I instead be splitting my notes down more granularly to capture these questions and notes to make it less crazy to aggregate elsewhere? I’m happy to write code to do this, but before I embark on an unrealisable mission, I’d like to do a quick sanity check.

Thanks for insights for anyone who has approached something similar, and not done it badly!

[MartinBoycott-Brown]

Would it be possible to just use double brackets around the question so that would create a new note with that text as the $Name?

Personally, I find there is some value in using a complete piece of text as the $Name of a note (e.g. “depression is widespread among military personnel” rather than “depression in military”). But I’m mainly using Obsidian these days, where it might make more sense.

[DaveM (Dave Mee)]

This genius thought had never crossed my mind; thank you for the suggestion, Martin!

Okay, I added {{q:this is a double-bracket note}} to a note (which I’m editing in iaWriter as a .txt file, which lives in a DEVONthink indexed directory, so has a unique DEVONthink URL. The note in Tinderbox is updated with the new text neartly, the {{…}} is removed, and a new note is created under this imported note. However, it takes on the name of the note it is a child of (rather than the text between {{ and }}).

It’s doing a lot of things I’d like it to do already, but I have to do some research into how this note/link extraction works in Tinderbox, which I’ve not really looked at much. There’s also some possible madness happening on my side, as child notes of the parent note undergo some transformation and inheritance as it is.

But thanks for this suggestion; it’s a great idea I’d not thought about, and possibly using what was intended for one use in Tinderbox for another. Something to squander my weekend over!

[mwra (Mark Anderson)]

I wonder what is processing the {{... }}. I understand curly brackets are reserved characters in Markdown although they have no formalised Markdown meaning—at least in the base Markdown flavour.

The characters don’t—unless I’m mistaken—have any processing intent for Tinderbox within $Text (as discrete from any Markdown behaviour).

That might suggest some other tool in the chain is doing the substitution. I note this only because while clearly it has had good outcome, I find it useful to know what did the work if only to guide triage on the day when the process doesn’t work as expected.

Meanwhile, Tinderbox can absolutely check through $Text for {...} or {{...}} sections of body text. There is no one right way so a place to start is to make a list of the different codes you wish to detect.

[PaulWalters]

Curly brackets – {…} or {{…}} – have no meaning in MultiMarkdown, which is what DEVONthink uses, so there should be no conflict with the OP’s use of DEVONthink to Go. The curly brackets have meaning in some of the more esoteric and less-commonly-used versions of Markdown. Seems unlikely that the OP would have a problem with their scheme.

Edit: there is a case for using Critic Markup in DEVONthink notes, which does rely on { } curly braces. And DEVONthink’s “transclusion in the preview” effect mentioned by @brookter below.

[mwra (Mark Anderson)]

@PaulWalters

Seems unlikely that the OP would have a problem with their scheme.

Good. Thanks

[satikusala (Dr. Michael Becker)]

@DaveM

One branch is an outline of my PhD thesis, each node of which pulls in markdown documents stored in DEVONthink via autofetch. While this may sound obtuse, it works well for me, as I can edit the documents on DEVONthink To Go on my iPad and then update when I get back on my mac.

Oh, I like this!!! Can you demonstrate this at a meetup some time?

[MartinBoycott-Brown]

I’m glad the thought made some sense! One never quite knows if what one says will be useful or useless to someone else.

[DaveM (Dave Mee)]

@satikusala

Oh, I like this!!! Can you demonstrate this at a meetup some time?

oh god I’m going to have to shave aren’t I?

Sure, I’d like to figure out what’s going on with those {{}} child notes as I think that’d be a really useful aspect of this. The overall workflow is planning and outlining in TB, linking to .txt files in DEVONthink (using an indexed directory so it’s on disk, with bonus points for Hookmark linking of all three), and mapping the TB notes URL field to the DT item URLs. DT’s magic is that the URLs remain consistent, so I can edit the same files when they sync elsewhere - I’m using iaWriter on my Mac and DT itself on my iPad - and then when I sit down to neaten things up with Bookends and overall editing on my Mac, a quick DT sync and TB agent update brings everything together.

Some years ago I begged for the edit panel to show multiple notes together, and allow editing of them - this is why. I select the notes in the outline, which then gives me a huge Markdown document in the edit view, which I copy-and-paste into iaWriter for Word export into Mellel for final wrestling and citation injection.

The only real barrier is that edits have to be done through the .txt files only, as changes don’t propagate outwards from Tinderbox - I considered this at one point (someone here some years back produced a bidirectional DT system), but decided I already had enough debugging of my own without introducing a potential new set of misinteractions.

I’m hoping to get a non-decrepit mac as soon as Apple release the new machines, for the next month or so. I will hopefully have figured out what’s happening with {{}} by then, which would be great. {} is already use by Bookends for cite-as-you-write annotation, which is why I hijacked their markup for my manual annotations (which I’d need to be able to insert in plain text files, for my workflow to make sense across devices and systems).

[PaulWalters]

I’m as confused as @mwra as to how {{...}} automatically creates a so-called zip-link in a note inside a watched DEVONthink group, in Tinderbox. FWIW, even if [[...]], the “normal doubled square brackets” are used in a watched note, a new linked note is not created.

Could you, @DaveM, post a small video showing this in action?

Or perhaps due to time dialation you are using some Tinderbox version from the future?

[DaveM (Dave Mee)]

Well. I’m as surprised as everyone else about this feature. And having tested it anew, it has let me down. I’d never really used ziplinks before, and deliberately kept clear of any documentation around them, so I was unsurprised to learn that they had some magical superpowers of which I was not aware.

There is a small risk there is some residual code I had half drafted to pull out text in {{…}} blocks, which may have triggered it instead (my test earlier today ran this on plain, I prototyped notes, so I suspect that I may have half-built the extraction code earlier and forgot about it).

It was so exciting for a minute. Sorry to be a source of disappointment, particularly about my temporal manipulation skills. My thesis suffers equally from this.

[brookter (David Bertenshaw)]

I’m not sure whether this helps, but {{document name}} is the DT3 syntax for file transclusion.

From the Help file:

File Transclusion: A special feature of DEVONthink’s Markdown handling is support for file transclusion. This provides the capability of displaying the contents of one file inside the contents of a Markdown file. This supports displaying the contents of: plain text, rich text, HTML, formatted notes, sheet, or more commonly, other Markdown files. For example, you may have several chapters of a book written in individual Markdown files. Use transclusion to view them all as one document with out the need to merge or copy and paste the content between files, similar to some popular writing applications.

To use transclusion, just enter the desired document’s name or item link between double braces, e.g., {{Chapter 1}}. Transclusion supports filenames with and without extensions, relative and absolute links, as well as item links. It also supports the Obsidian syntax, e.g., ![[Chapter 2.md]].

Is it possible that this is being translated into Tinderbox’s footnote feature?

[PaulWalters]

@brookter

Is it possible that this is being translated into Tinderbox’s footnote feature?

Good point, but I cannot cause the {{bracketed}} text to create a note or footnote in the case of a watched DEVONthink group. The transclusion effect occurs in DEVONthink’s markdown preview mode – it doesn’t actually occur in the source file itself. It is the source file that is watched when Tinderbox watches a DEVONthink group.

[brookter (David Bertenshaw)]

I’ve just tried and I can’t get it work either – but it’s the only place I can think that DT3 uses {{ }} so thought there may be some connection.

Interestingly, sometimes (but not always), dragging a Markdown file which includes words in double square brackets ([[example]]) from DT3 will create a new sibling note called example in Tinderbox. I presume this is expected – I’d never noticed it before.

[PaulWalters]

@DaveM’s case for parsing notes for interlinear comments, tasks, or other, might be addressed with
stream processing.

[satikusala (Dr. Michael Becker)]

@DaveM

oh god I’m going to have to shave aren’t I?

Cool! Can you join this Sunday’s meetup?

[mwra (Mark Anderson)]

Yes, and I’m reminded better documentation/description of that process is on the spike at aTbRef HQ. I’m aware that whilst the individual operators are usefully described discretely, they are rarely used on their own but rather as part of a chain. I’ll think on this.

[DaveM (Dave Mee)]

Okay, I’m doing that weird thing of replying to my comment. Happy new solar year, everyone!

Just to review what I’m trying to do: I have a large outline of text notes. When I’m feverishly writing in them, I use strings surrounded by { } to distinguish them, following Bookend’s model for inline citations. So I may have a note that says

This whole issue is nonsense {q:is it actually nonsense} because I say so

and what I’m trying to do is collect any text strings that match {q:…} in notes under a particular writing branch, make a new note under a followup-question branch, using the text between the {} characters as the name and which links back to the original.

I’ve realised I can’t just alias notes, because I want subsets of text from them, and a single note may contain multilple {q:…} items.

I have some very rudimentary code that I think should work (sat in an edict of the note that will be used to accumulate these questions).

There’s a ‘slog’ function to (s)log so I can debug this, and then I open with this:

function slog(gripe) {
	$Text('/log') += gripe+"\n";
}

var whereLingeringQuestionsLive = $Path(this);

var candidates = find($IsAlias==false & descendedFrom('/New Model Thesis')==true);

slog("candidates = "+candidates); 

candidates.each(candidate) {
	slog("\nfound candidate with path :"+$Path(candidate) ); 

	if($Text(candidate).contains('q:') { 

and I get a dump of a list of all the notes, concatenated as I’d expect:

candidates = [/New Model Thesis/ Challenges of Social Media Research;/New Model Thesis/ Challenges of Social Media Research /Introduction;/New Model Thesis/ Challenges of Social Media Research /Introduction/Research Questions; (trimmed for brevity)

But as I .each() my way through the list, hoping to see some indication of notes, all I get is

found candidate with path :

I would expect candidate to be a full note reference I could extract a $Path() from, but it seems to be just blank.

As I type this out, am I coming to the realisation that ‘find’ is returning a string rather than a list (as I’d consider it in more fully JavaScript environment)? E.g, “1;2;3;” rather than (“1”,“2”,“3”), if that makes any sense? Do I need to massage that candidates list somehow?

Thanks for any thoughts on the first day of (20+25)²

[mwra (Mark Anderson)]

@DaveM

that ‘find’ is returning a string rather than a list

find() returns a list (List-type data) of $Path values for every matched item (see). Of course, depending how you pass the list it might appear as a String-type (especially if only on match!).

@DaveM

E.g, “1;2;3;” rather than (“1”,“2”,“3”)

Assumption error. Not every language uses JavaScript precepts. The opening form above is correct for Tinderbox!

Over time—because we users asked for (demanded!) finer automation control—just depending on the app’s auto-coercion of (assumed) data type has potentially got harder. It’s us, not the app!

The default data type for var (see) is String-type. So your code might better use:

var:list candidates = find(...

Small snit, hardly worth mentioning but in:

...descendedFrom('/New Model Thesis')...

the quotes are redundant. Quotes are used to indicate the user is passing a $Name value. If pusing a $Path, quoting it just makes the app check twice not that today’s Macs notice the difference. BUt, from a perspective of coding intent, there in no need t encode fill paths. An example from using export (all are the same):

^include(/Template/My-page)^

But, using just $Name

include("My-page")^

Redundant quoted path:

^include"(/Template/My-page")^

all resolve to template called ‘My-page’.

Not sure if that helps. Just ask if not.

[mwra (Mark Anderson)]

Retracing your steps, all seems to work although the final if() fails as it is missing its closing parentheses, i.e. if( not if().

This edict works for me :

function slog(gripe) {
	$Text('/log') += gripe+"\n";
}

var candidates = find(!$IsAlias & descendedFrom(/New Model Thesis));

$Text =;
$Text('/log') =;

candidates.each(candidate) {
	slog("\nfound candidate with path :"+$Path(candidate) ); 

	if($Text(candidate).contains('q:')) { 
		$MyNumber+=1;
		$Text += candidate + "\n";

	}
}

I’ve tidied up the find queries a bit and added an alias into the text folder as a potential false positive. Here’s my test document: new-model-thesis-1.tbx (153.5 KB)