How to name $Title and move $Text for notes containing bracketed titles?

jprint714 · October 22, 2022, 4:50pm

Last year, fellow Tinderbox users generously provided me with a stamp code to extract a few words from the top of a note’s text (in the body) to create note titles, and then move the notes’ text (which appears as the titles) into the notes’ text body.

That stamp code is:

$Text=$Name;
$Name=$Text.words(11)

It still works great! But I’m seeking advice for a process that would enable me to query and stamp notes whose title is bracketed with some kind of syntax, e.g., { } or ( ) or something else.

Here’s the background context…

I’ve created a process in which I annotate documents, render the annotation notes, export them into an OPLM format, and then import them into Tinderbox. When I import the notes into TBX, the notes’ text resides in the title – which is why I use that stamp to name the $Title and move the $Text.

But in some cases, I’m able to create a title for notes (e.g., notes for timelines, “1492: Columbus sailed the ocean blue”), and so I’m thinking it might make sense to query those notes with an agent, and then apply a different name $Title and move the $Text to them. So, here are specific questions…

I know that regular brackets are used for Tinderbox’s Ziplinks. So, what kind of bracket syntax I should use instead, that would be better to apply for note titles – and would work best for the process I’m seeking?
I assume I’m seeking an agent using regex query code for this purpose. Is that correct? If so, could I get some suggestion on the construction of that code for locating notes whose titled is bracketed in some way, e.g., {1492: Columbus sailed the ocean blue} or (1492: Columbus sailed the ocean blue) ?
Finally, what new stamp code should I use for notes with bracketed titles, so that I can remove the brackets (leaving the title intact w/o the brackets), and moving the text underneath it from the title area to the text body?

Thanks so much!

eastgate · October 22, 2022, 5:22pm

To restate your problem in order to ensure I understand it, you have some notes with titles like this:

Name: {1492: Columbus Sails} Columbus departs from Palos, Spain

and you want to transform them thus:

Name: 1492: Columbus sails
Text: Columbus sails

Tinderbox offers a bunch of ways to do this. You’re thinking of a stamp, but let’s use an agent instead. I think we’re looking at three steps:

Gather the notes we want — notes with brackets
Extract the title, which is inside the brackets
Extract the text, which follows the brackets

I think for now we will ignore edge cases, such as notes with unpaired brackets, or nested brackets, and so forth. We can handle those later, or just fix any mistakes by hand.

step 1

The agent query finds the notes in which we’re interested. I think those are notes with brackets {}. Depending on your use, you might want to look for () or or some other delimiters. You might want to restrict the agent to looking in a specific part of the document

inside(/inbox) & ....

or to looking for specific kinds of notes

$Prototype=="Task" & ....

But I think we can leave that aside for now.

Our query, then, can be simple:

$Name.contains("}")

We could check for the opening bracket, too, but that might be more work than it’s worth.

$Name.contains("}") & $Name.contains("{")

#step 2

Now, what does the agent do to the notes it finds? You’re right: we could use a regular expression in the query and extract back references that way. I’m going to do it a little differently.

I want to make an agent action that splits the text at the closing bracket “}”.

$MyList=$Name.split("}");

This makes a list of substrings, divided at the “}” character, and stores that list in $MyList. That does almost all the work we need to do! All that’s left is to move the parts where they belong:

$MyList=$Name.split("}");
   $Text=$MyList[1];
   $Name=$MyList[0];

Almost there! We have one glitch left: the opening “{” remains in the name. We’ll delete it:

$MyList=$Name.split("}");
   $Text=$MyList[1];
   $Name=$MyList[0].replace("{", "");

Again, you could do this with regular expressions just as well, and that’s what most people would reach for first. I think this approach is a little cleaner, especially since regular expressions think that characters such as “( ) ” have special meanings and so require a special signal to remove their specialness.

One warning: A perfectionist might be tempted to try to write a regular expression that ensures that braces are properly paired and that handles nested braces. That’s won’t work, because that’s a problem beyond the capacity of regular expressions. It’s easy to forget that! But you aren’t going to encounter such stuff in the wild, so just don’t do it.

satikusala · October 23, 2022, 2:14pm

Hey there, great sample of the use of split and back-references. Once crucial point. Curly brackets are a RegEx operator. For @eastgate example to work you need to escape the brackets with a backslash \.

I’ve created a sample file with three approaches to the problem. Also, it includes a logging example. I will go over things today in today’s meetup, as I think it holds a ton of valuable lessons.

This lesson is in response to a post by @jprint714

How to name $Title and move $Text for notes containing bracketed titles?. This is a wonderful tutorial that gets at the root of action code, logging, text transformation, note name path management and more

Description

Parse the text in an attribute and place the different parts in different attributes.

Learning Outcome

Know how to use .split operator
Appreciate the retention of your source data until you’re ready to destroy it
Missed typed operators can lead to data being inexplicably destroyed (see LO2)
Logging is your friend, helps you see what’s going on.
Commenting code is your friends, it helps your future-self
There is more than one way to do something
Understand RegEx operational characters and how to escape them
Tinderbox’s syntax for finding keys in a list or dictionary, e.g. [0], [1].
User of designators, e.g. a note name, path, to enhance action code.
The use of variables
Appreciate the value of cleaning up after yourself
Bonus: writing in markdown gives a ton of flexibility
Bonus: some visual affordance really help, e.g., views, separators, and badges
Bonus: replacing tricky characters, e.g., “;”, in operations with a different delimiter to complete a process and then later refers back to the original.
Bonus: Triggering functions.

Objective:

Parse this text “{1492: Columbus Sails} Columbus departs from Palos, Spain” that is in the $Name of a note.

Place the text between the{} in the $Name
Place the remaining text in $Text
Do it in a way that leaves no lingering data

The Solution

See the stamps. I’ve produced several for the exercise:

Reset
Parse Name Original 0, the stamp that does not work
Parse Name Original 1, simple stamp
Parse Name Original 2 Modified, handles “;” in text.
Parse =Name Refined Eg 1 Use Attribute Placeholder , uses attributes as placeholder
Parse Name Refined Eg 2 Use Variable uses variables instead of attributes.
Tigger Parse Note, triggers a function to parse the note

See the functions, I’ve created two functions.

fParseNote, parses the note on a boolean trigger
fReset, resets the note on a boolean trigger

TBX L - Meetup 23OCT22 Using Split to Parse Data.tbx (303.1 KB)

jprint714 · October 23, 2022, 4:33pm

Oh wow. Thank you so much, @eastgate and @satikusala – this looks amazing! I’m grateful for your generous help…

I’m excited to put this to work, but first wanted to further clarify the way the text appears so that I’m applying the right code to fix it. Using the same example, the lines of text would appear like…

{1492: Columbus Sails}
Columbus departs from Palos, Spain (near Huelva) in southern Spain, on August 3, 1492, in command of three ships: the Niña, the Pinta and the Santa Maria. His crew mostly came from surrounding towns such as Lepe and Moguer.

In order words, whenever I create a bracketed title, the text is in the first line, and the remaining text falls directly underneath it – but all of the text still appears in the title section of the note.

BTW, I’ve been using this process with stamps (as I noted in our exchange on the other post), but I’m happy to do this through another process. I’m still learning Tinderbox, so I’m not clear on the pros/cons of doing it through ways other than stamps.

satikusala · October 23, 2022, 7:38pm

Here is the meetup from today: Tinderbox Meetup- Meetup 23OCT22 Working with Action Code, a Step-by-Step Walkthrough

jprint714 · October 24, 2022, 5:48pm

This is tremendous… Thank you, guys! I’ve been watching the the meet up recap, and trying to absorb all of the lessons.

Now I’m going to throw you a curve! @satikusala and a few others know that I’m using this process for annotated notes that I’m importing into Tinderbox.

You’ve just provided me with an elegant solution for preserving titles for notes with { } (curly brackets). (Again, I’m happy to use < >'s, parentheses or anything else for bracketing these titles.) As I said, I’m mostly using this syntax around note titles with dates, for notes that are part of my timelines.

However, I still need to apply a stamp to name the $Title and move the $Text for the notes without the curly brackets as well.

Sure, I could manually locate those notes and apply the old stamp to them. But is there another way that I could do this, that would also integrate some of the automated functions you created for the notes with curly brackets?

I’m curious to hear your answer, and thank you again for all of your help!

satikusala · October 24, 2022, 6:15pm

I don’t remember. your old code. However, this should be simple enough. You can string action code with functional or stamp() actions. I assume you have a prototype for the imported notes. Add a boolean that shows the default is true, e.g. $IsNeedsProcessing. Add your modified function to the prototype and have the function turn $IsNeedsProcessing to false once it is complete. Doing this will automatically trigger the action code when you import the notes. Or, you bring the important notes into a simple folder that has an OnAdd to trigger your function. You can then have another function move them to wherever you want them to go in your file after they’ve been imported. I can’t be more specific without knowing your context

mwra · October 24, 2022, 8:30pm

But, IIRC, in the demo above, .replace() is used. So, notes with no curly brackets won’t be affected. IOW, the code works for notes whether or not_ they contain the problematic characters. The same applies at the reverse stage.

If you are getting something different, could you post a small TBX showing the problem as I suspect there may be another factor at play.

jprint714 · October 26, 2022, 2:47am

Thanks! I’ll answer your question first, and then walk you through my process for importing notes.

The stamp code I’ve used extract a few words from the top of a note’s text (in the body) to create note titles, and then move the notes’ text (which appears as the titles) into the notes’ text body is simply:

$Text=$Name;
$Name=$Text.words(11)

I’ve created three prototypes that are nearly identical in every way but their appearance: pArticles; pDocuments; pInterviews. These prototypes have the same attributes; they just have different names (of course, for the source data) and colors and badges for quick identification.

After I’ve annotated documents (in a PDF format) in MarginNote, I render them in OmniOutliner (through a process I’ve shared with @satikusala and @mwra), and then export them with OmniOutliner in an OPML format. During the OPML export process, I change the prefix so that any files will include the following file naming conventions: REN_ART_2022_10_25-Columbus sailing (REN for rendered; ART for article) or REN_DOC_2022_10_25-Columbus sailing historical data (DOC for document) or REN_INT_2022_10_25-Interview with august historian about Columbus sailing (INT for interview).

Then I create and name a folder (container) in Tinderbox using the OPML file name, e.g., REN_ART_2022_10_25-Columbus sailing. And then I open the OPML file in Tinderbox, select all notes, copy them, and then paste them into the target TBX container.

I believe I once set up an approach in which Tinderbox would apply the right prototype to notes inside of a container, depending on the parent containers prefix (i.e., REN_ART / REN_DOC / REN_INT).

As I said, I’m open to suggested approaches by fellow users for applying stamps and selecting the appropriate prototypes or creating an automated process that selects the appropriate prototype for the notes, then applies the stamp code to the notes so that titles in the curly brackets remain as titles, while the remaining text outside the right curly bracket is moved to the text body and the notes without bracketed titles undergo the original title-shortening-text-moving process (for which the code above was designed).

I hope this makes sense. I’m happy to clarify this further and answer any questions.

Thank you again for all of your help!

mwra · October 26, 2022, 8:01am

One thought. Make a prototype which you apply to the new containers you make to subsequently hold the pasted-in OPML-generated notes. Give the prototype’s $OnAdd this code:

if($Name.beginsWith("REN_ART"){$Prototype="pArticles"};
if($Name.beginsWith("REN_Doc"){$Prototype="pDocuments"};
if($Name.beginsWith("REN_ART"){$Prototype="pInterviews"};

Some notes on the choices above:

Setting the prototype via OnAdd means the test runs only one.
Putting the ONAdd code in a prototype means you only need create the code once
Why not use .contains()? .beginsWith is a simple literal string value check, i.e. not a regex, so simpler faster. You could use .contains() if you prefer.

So the process, once you’ve made the new prototype:

make a new prototype, e.g. “pFolder” - choose a name that makes sense to you
make your new import container and set it to your new prototype
paste your new notes into the container and their prototypes are automatically set.

jprint714 · October 26, 2022, 6:15pm

This is brilliant. Thank you so much, @mwra !

I realize this is a somewhat elementary question, so apologies in advance… In describing my annotated notes import process I failed to mention how I’m creating folders (containers) for these notes under a parent folder titled “I M P O R T S.” So, I’m wondering if it’s possible – and advisable – to simply include $OnAdd code, and the if($Name.beginsWith sequence you suggested, within that parent folder’s Action, Rule or Edict section?

Is there any other functional difference between inserting the code into one of those sections versus creating a prototype with the $OnAdd code, and applying it to the parent “I M P O R T S” folder?

I want to first nail that down that phase of work. But the next question is: would it be possible also include code for the Parse name stamp that @satikusala created – so that the $OnAdd can first apply the appropriate prototype to the notes and then parse the names, and move the text? Or would combining those two functions be problematic?

Like said, I’m aiming to first work on the first phase of the $OnAdd before I test the Parse name functions; just want to determine if it’s viable to combine those functions within a single $OnAdd sequence.

Thanks again!

eastgate · October 26, 2022, 9:16pm

No difference at all; if the note has no $OnAdd, it will inherit from its prototype.

mwra · October 26, 2022, 10:16pm

Yes. If, the ‘I M P O R T S’ container contains a container per import into which the OPML-generated notes are added, the ‘I M P O R T S’ OnAdd doesn’t fire as the OPML imports are not children of that container but grandchildren. My solution was designed—using the information you gave—to be applied to the container holding the new OPML-generated notes, not their grandparent.

But, if you set OnAdd for ‘I M P O R T S’ to:

$Prototype="pFolder";

Then any container created a child of ‘I M P O R T S’ will inherit pFolder’s ONAdd so when the OPML-generated notes are added to it the prototype’s ONAdd will set the right prototype for the new children.

Tip, if struggling to imagine the process, just actually test it in a small TBX. Otherwise you drown in imagined hypothetical questions about problems that likely won’t arise.

Sadly, you’ve not told us what those are so an answer would be a complete guess. Why not take a step back and post a small TBX with your starting problem and the prototypes and functions etc., that you want to use? It’s hard to give a helpful answer if part of the question is missing. Don’t forget we can’t see your file, nor do we know your design assumptions unless they are stated.

jprint714 · October 26, 2022, 10:24pm

You’re right, you’re right… I’ll first try experimenting a bit more, using the process you’ve suggested, and then I’ll circle back with more concrete follow-up questions.

Thanks again.

jprint714 · December 16, 2022, 11:58pm

Thanks again, @satikusala for producing such a comprehensive step-by-step explanation of your process. I’ve been applying your parsing code to notes, and generally feel I understand what works (and why) for notes that contain { } 's.

I hate to throw in another variable, but…how would you suggest I amend one of the stamps so that it can apply the following function for notes w/o { } 's :

$Text=$Name;
$Name=$Text.words(11)

Thanks!

mwra · December 17, 2022, 12:16pm

I’m not sure you’ve described the problem correctly. The stamp code you quote works fine to set $text to the $Name value and then set $Name to the first 11 words of $Text.

The presence or absence of curly brackets (i.e. { or }) has no effect, as this test TBX shows:title-stamp.tbx (394.2 KB)

In the ‘before’ section there are two notes with long titles. In the ‘after’ section, the same note but with the ‘Set Title’ stamp applied (using the code you supplied above). The first of the examples has no brackets of any kind. The second has a pair each of parentheses, square brackets and braces.

So what is the actual problem you are trying to fix?

jprint714 · December 20, 2022, 5:03am

You’re right. I haven’t explained this well, and I’m sorry about that. I’ve also been refining how I input and format annotated text, which affects how notes are formed on the Tinderbox end.

I’m attaching the following screenshot, which will hopefully clarify what I’m seeking…

As you can see, there are notes with curly bracket headers, and notes without headers (those I’ve selected) – w/o curly brackets or otherwise – that are just a giant block of text. In the past, I just applied the following stamp to all of my notes:

$Text=$Name;
$Name=$Text.words(11)

Essentially, I’m trying to tell Tinderbox

Please convert note-headers, in curly brackets (though again, I’m happy to use any syntax) into the note-title, then
remove the curly brackets, and
move (or parse) all remaining text (underneath it) to the text pane.
Also, when no such curly bracketed note-headers exist, just apply the regular stamp (above) so that I can have a note title consisting of the first 11 words – and then move all the text to the text pane.

Please let me know if I can further clarify what I’m seeking. Thanks!

mwra · December 20, 2022, 2:06pm

Thanks, that’s fine. So our example is like this:

{1945: End of World War Il}
1945 Establishment of UNO; End of World War Il; Hiroshima and Nagasaki experience the first dropping of the Atom Bomb; Death of President Roosevelt.

So the patten is that we start with a title ($Name) where the format is:

start of the string
*{
title string (not containing cully brackets)
}
line break (zero or more times - in case we forgot to add that in upstream in the process)
string holding $Text.

So we make a stamp like this - in my demo doc (link below) it is called “Set title - curly brackets”. Code:

if($Name.contains("^\{([^\}]+)\}\n*(.+$)")){
   $Name = $1;
   $Text = $2;
}

Why use an if()statement? Because it allows us to use a regular expression to set up back references to extract just the title and just text from the original note title— the $1 and '$2` you see in the later lines of code.

The regex code explained:

^. Start the match at the very beginning of the source string.
\{. Match a literal opening curl bracket.
(. Open a back reference (this will be $1).
[^\}]+. Match one or more characters that isn’t a closing curly bracket (i.e. the desired $Name text).
). Close a back reference (close $2).
\}. Match a literal closing curl bracket.
\n*
(. Open a back reference (this will be $2)
.+$. Match one or more characters to the end of the source string (i.e. the desired $Text text)
). Close a back reference (close $2)

Having made the back-references we can the set $Name and $Text to the appropriate back reference.

When you add the stamp code to Tinderbox, it may colour and show a yellow highlight like so:

The syntax colouring is not wrong, in its logic. In normal action code unbalanced parentheses/brackets would be a problem, but here they occur inside a regex string and we can just ignore the warning^†.

OK, so here is the stamp and my earlier demo file updated with a new before/after example and a new stamp (only for the new example): title-stamp.tbx (446.6 KB)

Does that help?

†. This is more an issue for those copy/pasting code without an understanding of it, thus the explanation, as the yellow highlight is there to warn you about issues like unmatched brackets. The syntax colouring/code warnings are isn’t intended to try and understand regex (which can be incredibly complex), nor to spot and ignore embedded regex.

jprint714 · December 20, 2022, 7:26pm

Thank you very much for this. I’ve downloaded and tested your title-stamp file, and it works perfectly. Thank you!

I feel like I’ve drifted far off course from @satikusala’s original parting script, but I think that’s largely due to the way I’ve reformatted the title-header – which is entirely on me. But I did so because it works better for annotating and transferring notes from my annotation app to Tinderbox. My apologies to @satikusala for taking this process in another direction. I promise your tutorial wasn’t in vain!

I hope you don’t mind, but I’ve got some quick questions, just so I can better understand Tinderbox…

My initial through was that I could combine code – perhaps by using if – in such a way that a stamp would be able to carry out the note naming process with curly brackets, and then proceed with the $Title and move $Text for notes without curly bracketed headers. Did you create two stamps because one can’t combine those functions or it just works better as a two-stamp process? Or was it to break down the processes for me?

Also, does your Action prototype in your title-stamp file have any impact on the execution of the script? It doesn’t appear to me that it does; I’m asking just because I don’t want to be overlooking anything. Thanks!

I’m thinking that I might copy these stamps into one of my template files, but edit their names and sequence (e.g., A - Set title - curly brackets ; B - Set title ) since it seems like I should aim to first apply that stamp to notes with curly bracketed headers, and then the other stamp to those without.

I feel like I’ve entered a whole new realm of Tinderbox by using if ! It’s a bit scary, and exciting.

Thank you very much, again!

satikusala · December 20, 2022, 11:29pm

Hey, congrats on your progress. I 'm not sure I understand your question.