Looking for an efficient way to move data from interconnected notes into another container

[bardiccollege (Simon)]

IIRC there is an easy well established function of Tinderbox to automate the export of data from one note to another.

Essentially I want to pick related notes, then run a function that collects all those selected notes data (paragraphs of text) and pipes it into another note just for combining the data together.

This would let me pick and choose which notes go together under various contexts. Then I have all the relevant data in the same place, and I can process it separately from the individual notes.

What is this sort of task called, and where can I go to read about it?

• thanks

[mwra (Mark Anderson)]

There is no formal task name here apart from a general sense of incremental formalisation. You are essentially doing four generic tasks. Thanks for the image you posted, which gives some broad idea of the context.

Step 1. Setting the searchable data. The general start is using regex on note $Text. With practice, a better approach is to record metadata (i.e. an attribute value) for notes matching known formalisms. Either a single-value attribute, e.g. String, if a note has only one such match in this axis of enquiry. Or, if a note may match multiple topics, use a Set^†.

Step 2. Searching for the notes of interest. This can be done via an agent or action code (edict or stamp). the learner-level approach will tend to the agent, but if doing this a lot for many discrete target notes, working within action code is a cleaner approach IMO. But, there is not single ‘right’ way. The choice is partly the individually users comfort/competence with a particular approach. The query eventually run will either look (via regex) for a word in $Text or match a pre-stored attribute value (as in step 1 above). Whichever approach is taken, also consider the sort order of the agent container or the action once list generated as that affects set 4.

Step 3. Setting up the target note. The code at step 4 will need a note to which to write/update. That note could either be pre-made or created on first demand.

Step 4. For the agent approach, iterate the child aliases and append their $Text to the $Text of the target note. Via action code you will collect a list of matching notes, iterate the list and again append source $Text to that of the target note.

The fact there is no single ‘wizard;’ for the above is to be expected, consistent with how Tinderbox is designed. Consider the degree of choice in the above. What is need will vary from person to person and task to task even if the general task is the same. So building a generate push-button ‘function’ for this would be an exercise in futility. Instead, Tinderbox offers a wide range of query and text handling tools (in action code) to let the user build their workflow.

If you want more detailed assistance on any of the above, it will help massively towards giving pertinent advice/code, if you are a little clearly as to the nature of the queries you envisage and how/where you will store the result. The more specific the assistance requested, the more detail as to the source task will help.

†. The advantage of a Set over a list is a Set automatically de-duplicates. Thus a discrete list value can only be recorded only once per note. A set also automatically sorts values A–Z. A List allows duplicates and deliberately maintains order of entry. These choices have an impact on downstream use of the data.

[bardiccollege (Simon)]

Thank you! This gives me a solid starting point.

I recall hearing about note consolidation in one of the Tinderbox Meetups, and I had the impression that it was a simple one- or two-step process using action code to extract text from notes with inbound or outbound links.

Based on your explanation, it seems a more structured approach would be better. I could:

1. Create an attribute to mark the notes I want to consolidate.

2. Run action code to extract text from these marked notes into a pre-made consolidation note.

One follow-up question: Can I format the extracted text with headers to clearly separate the content from each source note?

[eastgate]

@bardiccollege

I recall hearing about note consolidation in one of the Tinderbox Meetups, and I had the impression that it was a simple one- or two-step process using action code to extract text from notes with inbound or outbound links.

This is, in the abstract, pretty simple. For example, you might write:

create("/MyNewNote");
$Text(/MyNewNote)=$Text(Tolstoy notes)+"\n"+$Text(Gogol notes)

Of course, this can be elaborated in lots of ways! But the idea is pretty straightforward.

@bardiccollege

Can I format the extracted text with headers to clearly separate the content from each source note?

You can add a header easily enough, for example:

+"\n\n"+$Name(theNote)+"\n\n"

It’s trickier to format the name, and usually it’s just easier to apply formatting yourself. But it’s possible if you really need it.

---

This sort of consolidation is sometimes the step before export and final revisions. If that’s the case, you might also consider HTML export which excels at this sort of task, and in which formatting titles and such is both easy and flexible.

[mwra (Mark Anderson)]

@bardiccollege

Can I format the extracted text with headers to clearly separate the content from each source note?

Yes, though explaining it (e.g. with code) makes more sense in the context of specimen data.

FWIW, I attach a small TBX (further below) used to test some ideas above. I’m, aware you image above suggests you’re working in Map view^† but for sake of clarity this task is much easier to do/test in outline view, even if you work on the outcome in map view. Here what the demo looks like.

I’m assuming we’re marking notes of interest in a user attribute. I’ve used a String ($Topic) and an alternate Set ($Topics) just to show the idea. Depending on which you use (String or Set) a slightly different query syntax is needed. In your own work you can name the attribute as you choose.

The ‘agent testing string’ takes thins one step further as the search target is parameterised. Thus instead of coding “Tariffs” into the query, the query uses the value ($MyString) of the agent’s attribute ‘MyString’.

The agent’s edict is then:

var:string vString;
vString = collect(children,$Text).format("\n");
var:string vPath = "/Topics/"+$MyString;
$Text(vPath) = vString;

The data is written to note /Topics/Trade, though if working on a map you might want to forego the use of the ‘Topics’ container although it helps for testing (as the map might have 100s of notes and finding the new one to check can take a moment).

Were you wanting to build the note on the fly @eastgate’s post above, indicates the rough pattern. Note that create() won’t make the same note twice, i.e. if it already exists.

The second agent just shows the different query yo use if searching a Set or List type attribute.

The ‘Note using edict’ forgoes the agent and does the querying within the note. Again the edict is used to populate the target note:

var:string vString;
vString = collect(find($Topic==$MyString(that)&!$IsAlias),$Text).format("\n");
var:string vPath = "/Topics/"+$MyString;
$Text(vPath) = vString;

The result is in note ‘Trade’. Though for a different topic and via a different collection method not the $Text of ‘Tariffs’ and of ‘Trade’ is essentially similar.

@bardiccollege

Can I format the extracted text with headers to clearly separate the content from each source note?

Yes, I added a new edit note to the demo ‘Note using edict with text headings’ which populates note ‘Trade—with heading’ via the following edict:

var:list vList;
// we add to target name to avoid other demo
var:string vPath = "/Topics/"+$MyString+"—with headings";
vList = find($Topic==$MyString(that)&!$IsAlias);
var:string vString=;
vList.each(aNote){
   vString+="Text from note at path: "+aNote+"\n";
   vString+=$Text(aNote)+"\n";
   vString+="~~~~~~~~~~\n\n";
};
$Text(vPath) = vString;

Note that to add the per-source note headings a slightly different approach is needed. This reflects the guidance in my first answer and in @eastgate’s answer that you decide the structure. So there is no notion of a ‘heading’. But, if you want to insert text into the output yet which is not derived from the (being processed) source note’s body text this is possible. As this additional test shows.

You might also ask yourself: do I need all this?. For instance, in the demo, if you select all the aliases and look at the text pane to you see:

Now you know that it might be enough. However, only you can tell that. This is why advice here may often seem loose-edged. Fellow users here can only respond to the information provided, which (innocently) may not be the full picture. Then again, the ‘full picture’ can be an emergent property, i.e. as you dig in the picture slowly emerges. It also explains why building to much, too early in terms of process, often means a do-over unless one is re-running the exact same scenario as before.

If templated to ask “Can I have the heading formatted”, as in ‘rich text’ bold/italic etc. It is possible but not productive. I’d ask myself do I truly need it or think so just because I imagined it might be so based on working in WYSIYG word processors. If you want text formatting you may do well to consider the preview approach alluded to in a previous answer.

HTH

Edit: Forgot to add the demo tbx

Here is the demo file: collecting-text-1.tbx (159.6 KB). NOTE: there is no ‘readme’ as the TBX is explained above. Writing a fully documented demo would take more time than is spare at present!

†. If so, and for reasons not to dive into here, there are significant upsides to not building your main map at root level. If you have done this it is easy to make a new note that with be the map container. Then select only the items you don’t want/need on the map (e.g. not /Hints, /Templates, etc.) then drag the selection onto your new note. the drop moves all the selected item to the child map of the new container.

[satikusala (Dr. Michael Becker)]

@bardiccollege

IIRC there is an easy well established function of Tinderbox to automate the export of data from one note to another.

Simon, how are you progressing? This would be a GREAT meetup topic. Perhaps we could have you join and you could share with us what you’re trying to ultimately accomplish.

Also, there a lots of way to pull in and repurpose content. I’ve not had a chance to read the entire piece here. Are you looking to “move data” or are you looking to “report” data. If you’re looking to report it, have you tried developing templates to pull the data and create the formatting of this data the way you want it?

[bardiccollege (Simon)]

OK, so an update: Sorry this got put in cold storage for a bit.

Today I tried creating a working rule & edict to do this but nothing has worked yet. I tried a lot of things, and not one of them worked - it was all using rules and edicts.

Instead of making a long post I’ll post a short test that also failed. I think. if I can get your help to make this test work, I can try again with the rules and edicts.

Test steps:

1. insert this code into the notes rules box

$TestOutput = "Current Note: " + $Name

This is supposed to place the note name into the text box of the note itself.

This did not work for me.

an image of the UI I’ve been working with

[satikusala (Dr. Michael Becker)]

Hey @bardiccollege, you action code works just fine.

Tinderbox is doing exactly what you are asking it to do. I’m not clear, however, what is it you actually wan to do. Also, there are MUCH EASIER WAYS to work with action code than the getinfo window, e.g., the inspector (high CMD+1). I’m working right now, if you’re free, maybe we can hop on a zoom. I’ll DM you.

[bardiccollege (Simon)]

Thanks for the help. I dont use zoom so the forum is all I can use right now

With this test I just want to see if the action code works. What I want to happen is to see the notes name “Rules or Edicts” appear in its own notes field.

This is my first foray into action code so I’ll hunt down a few of the TBX videos and see if I can get it figured out with them. The documentation says I need to verify that my note template or property settings display computed properties.

[satikusala (Dr. Michael Becker)]

@bardiccollege

With this test I just want to see if the action code works. What I want to happen is to see the notes name “Rules or Edicts” appear in its own notes field.

Hey there, sounds good. As you can see in the above image, the action code does work. The Attribute $TestOutput has the text “Current Note: Note.”

Good luck.

[bardiccollege (Simon)]

Ok, I have made progress (FWIW Im searching through your TBX videos on Youtube and using Marks Tinderbox Way book as reference with ChatGPT to help explain the code)

I have got the note printing its own name in the $text field; but when I try to generate a linked notes $text into the receiving notes $Text field I just receive a "Incoming Linked Notes Text:

true"

A boolean I believe. So I think Im almost there, what have I not understood?

[mwra (Mark Anderson)]

@bardiccollege

1. insert this code into the notes rules box

$TestOutput = "Current Note: " + $Name

This is supposed to place the note name into the text box of the note itself.

OK, the code you placed in $Rule for a note has the code:

$TestOutput = "Current Note: " + $Name;

Assuming the current note’s title (i.e. $Name) is “Test note”, then the result is the user attribute $TestOutput holds this value:

Current Note: Test Note

N.B. this assumes you’ve made a string-type attribute called $TestOutput. If you haven’ty, nothing happens as the code won’t, in case supposed, autogenerate the missing attribute. It needs to exist before using the code^†.

Even if $TestOutput is populated, your opening intent was to put the value “into the text box of the note itself”, i.e. into attribute $Text of the note. to do that the code you needed is:

$Text = "Current Note: " + $Name;

On the left side of the equation you set the target attribute, which for the note’s text area is $Text.

†. There are ways to test for the existence of an attribute, and/or create an attribute via action code, but that is best taken as a separate question.

[mwra (Mark Anderson)]

@bardiccollege

the $text field

withuot meaning to sound pedantic, but as your starting out at action code, be aware that attribute names, and action code operators )e.g. collect()) are case-sensitive. Action code treats $Text and $text as addressing two different attributes. I make the point as this isn’t necessarily obvious at first encounter.

If in doubt over case/pselling or syntax see up to date listing at aTbRef: a list of system attributes](System Attribute List), and a list of action code operators.

[mwra (Mark Anderson)]

@bardiccollege

A boolean I believe. So I think Im almost there, what have I not understood?

I suspect GPT isn’t as clever at reading documentation as it thinks. It’s great at writing readable text, less sure footed at inferring things.

You want the

You have code:

$Text = "Note: " + $Name + "\n\nIncoming Linked Notes Text: \n" + linkedTo(this) .collect ($Text).format ("\n\ n" )

The active call is this:

linkedTo(this) .collect ($Text).format ("\n\ n" )

but this isn’t doing what you want. Firstly, if you look at the documentation for
linkedTo(scope[, linkTypeStr]), you will see it tests outgoing links. But you want to test *incoming links, to you want linkedFrom(scope[, linkTypeStr]). This is where reading the primary references direct rather than using a GPT obfuscator will likely help you get started more quickly.

But this still doesn’t help as linkedTo() and linkedFrom() are simply boolean tests as to whether a link (optionally of specified link type) exists. So the test returns true or false with doesn’t give you a meaningful list of paths off which to work.

What you want is a list of all the notes linking to the current note and from that to get their $Text and to concatenate each text with a double line break. Or, easier, use links() and in a single command get a list of the $Text values of all notes that link (with a link of any type) to the current note. Thus:

// make a list type variable
var:list vList;
// set variable to list of $Text for every note linking to this note
vList = links(this).inbound..$Text;
// set initial static string for $Text
$Text =  "Note: " + $Name + "\n\nIncoming Linked Notes' $Text: \n\n";
// format the list as a series of paragraphs
// and add a per-source marker close line with some line breaks
$Text += vList.format("\n--------\n\n");

The result is like so:

Here is the document above: text-via-links01.tbx (135.6 KB)

[bardiccollege (Simon)]

Excellent, this does just what I needed - thank you.

[bardiccollege (Simon)]

Just for fun I asked GPT to analyze the code you made without the comments and asked if it would do what I was requesting. This is its response:

In general, yes—this snippet should collect the $Text of all inbound links and append it to the current note’s $Text. A few details to confirm:

1. Tinderbox Version

• links(this).inbound and the double-dot (…) syntax are newer Tinderbox features (introduced around version 9). Make sure you’re on a version that supports them.

2. Link Directions

• links(this).inbound gathers notes that link to this note. If your links are in the opposite direction (i.e., you want the text from notes that this note links out to), you’d use links(this).outbound.

3. Circular References

• If the inbound notes are also computing their own $Text in a way that references this note, you could get into recursion. Usually this won’t happen unless you’ve set up something similar on the inbound notes.

If all you need is to display the text from each inbound note, this code looks good. Just verify you’re using a recent Tinderbox release, and confirm the links point in the direction you expect. If you see unexpected results (like empty text or “true”), double-check that the inbound notes actually have non-empty $Text.

[mwra (Mark Anderson)]

If you read into the links() syntax you will be see that you can also filter by link type. I didn’t include that in the above so as not to over complicate things.

But if you change the link type of the link for “Text 3” to “simple note” to anything other than “Links to” and then alter this line in the $Rule code:

vList = links(this).inbound."Links to".$Text;

You will see that the added filter means that only inbound links that are also of type ‘Links to’ are matched.

links() is an odd operator and one of a kind in syntax terms as its arguments (i.e. inputs) are separated by periods. In all other operators, arguments are a comma-delimited list inside the (optional) parentheses after the operator name. The so-called ‘dot-operators’, e.g. List.count() that reports the number of items in ‘List’, came later to Tinderbox. They both use ‘dot’ joins but in the latter case you are chaining operators (commands) such that the result/output of the first becomes the input of the next-chained operator, i.e. one of its arguments.

The latter is why you will see two forms of many operators. Thus count(collect(children, $Name)) and collect(children, $Name).count() do the same but the latter is easier to write (correctly) and read back as code. If both forms occur, the dot-operator version is usually a later addition to the former. Thus count(List) came in v4.0.0 and List.count() came in v7.0.0. In aTbRef, the listings for system attributes and action code operators list the version when that feature was added as it can sometimes help make sense of other people’s code, especially if written some years back before newer methods were added.

This an be useful and you can use link types to give more nuance to your ‘collecting’ text of relevance. If you need multiple link types you’d need to create a link per type: multiple links between the same notes are allowed. I mention the latter as if going that route it is likely a signal to you that you should consider storing those types as some sort of attribute and using other action codes to get to the same eventual outcome in terms of building a $Text.