$Onadd link to container + inherit all meta-data (esp. tags)

Thanks again for these great answers (this probably the most helpful forum I ever participated in)! I think I have to do a bit more explaining…

Background: For a book project I’m working with many different kinds of sources (scientific publications, laws, interviews, newspaper articles etc in English and Chinese). In the past I’ve been using tags and filename-based metadata to find common themes (based on/inspired by Grounded Theory). I applied tags to both, the original reference (eg an article) and the small, paragraph-based text files mentioned above. At a later stage, when doing analysis on a specific topic, I usually searched for specific combinations of tags (or tags and other metadata). In a way, this is also what I intend to do here. Actually, I expect to mostly work with the Attribute Browser once the initial analysis (=assigning tags and metadata to the exploded notes/smaller text files) is done.

Idea: So this is how I thought to do it: 1. Import of the reference from Devonthink (Prototype “reference”); 2. Import of all small comment files into this reference, which then inherit certain key attributes and are linked to the reference. 3. Rearrange notes with similar natured content next to each other and create a “tag” note (see the example below, the tag here is “modernization”). 4. The code I’m looking for should add the tag modernization to the five blue notes (“notes on notes”). 5. The parent container (“reference”) of all these notes should then collect all the tags it contains (such as “modernization”, “land grab” etc, see also second example). I think this will provide me several avenues to find what I’m looking for once all references have been added (via Attribute Browser, Map View, Roadmap etc). 1 to 3 are solved, I’m struggling with 4 and 5.

Issues: Regrettably, $MySet=links(find($Prototype==“tag”)).outbound…$Tags; $Tags=$MySet; does not seem to work either. It only returns the tags of the parent note (in this case “test”) but not “modernization”, which is what I was looking for:

Thanks also for the advice regarding the difference between rule and edict - in my pervious project I had several thousand notes and I expect this one to be even more complex. Once I have the correct query I’ll turn this into an edict (I would like to avoid a Stamp as I would have to invoke this manually, am I correct?).

Very interesting – thank for sharing!

4 The code I’m looking for should add the tag modernization to the five blue notes (“notes on notes”). 5 The parent container (“reference”) of all these notes should then collect all the tags it contains (such as “modernization”, “land grab” etc, see also second example).

If I were doing (4), I would use a QuickStamp if adding the tag is a one-off action, or a Stamp, if the action needs to be repeated more often, or an agent, or a smart adornment’s $OnAdd action. Any of these is a once-and-done approach, which I think is what happens in Tinderbox or elsewhere when adding a tag to a note or a file or whatever.

For (5) I think agents or AB or both are best – at least that’s what I’ve found when I do this sort of searching-for-patterns-and-relationships.

(BTW, DEVONthink’s See Also feature is also excellent for this sort of thing, but is limited (thus far) because it does not consider tags when suggesting related notes.)

BTW, in The Tinderbox Way, Mark Bernstein discusses in depth using Tinderbox for searching for structure and relationships. There are also numerous threads on this in the legacy Tinderbox forum.

Noticing emergence is the most interesting effect of note taking. It is very interesting to see how others develop their heuristics to achieve that.

Yes (short, as I don’t want to overload this thread). In basic terms Edicts are conceptually like rules - in coding terms & scope of action - but run less aggressively. Simplistically, if you only change $Tags once a week (or once ever), you don’t needs to be checking the attribute value every second! Given that manually refreshing agents also does a “run-all-edicts now-once-only” it’s a nice halfway house between a non-stop rule and an only manually invoked stamp (though you might use the same action code in either).

Well, I just fixed the code you mentioned whereas the pictures above describe a different set of notes. I can’t tell from the images alone but I’m assuming prototype ‘notes on note’ is blue and prototype ‘tag’ is orange. From your description and image, you want all blue notes to add to their $Tags the $Name of the orange note to which they are linked (link direction: blue->orange). From the illustration we could assume blue notes only link to a single orange note, although that seems unlikely in the eventual dataset. You need to clarify this assumption.

I also assume the picture above is the map of a single reference’s child notes, i.e. your notes on that reference. However, unless the (grounded theory) tags - the orange notes - are discrete to a reference reference (i.e. map), they need to occur in more than one map.

So despite the picture, the real model here is an N-to-N relationship of blue and orange items. Notes may have more than one tag and tag notes may be referenced by more than one note. Is this correct.

I ask, before trying to give any code, as there are nuances relating to linkage and aliases. Basic-type (i.e. note-to-note) links are shared by a note and its aliases except where an alias has basic links (in or out) of its own. That difference is per-alias, i.e. one alias might share the original’s basic links whereas another would not.

My hunch is you need a container with all the ‘tag’ notes in it and then you create aliases as required and place them in the child map of each reference. Thus the maps for ‘Reference A’ and ‘Reference B’ would both contain an alias of the note representing the tag ‘modernization’. The original of the latter would reside separately in a container holding all your (grounded theory coding) tags.

So we get something like this:

I personally recommend (in a project of this eventual size) using prototype names that are self-evidently not real (content) note names. As the DT import uses the built-in ‘Reference’ prototype i can rename that (or I’d give it a sensible prototype name too). Although we want to harvest tag values via linkage relationships, we might have other purposes for links, so we’ll make a new custom link type called ‘tag’ but we’ll hid the label to make the map clean. It’s a different colour and line type (dotted) to a normal link so can easily be parsed by eye on a map. Before we can use the new link type (or re-set existing links to use it), we must create it:

We’ll (re-set) our annotation-to-tag links to use the ‘tag’ link type and now we see how each reference uses aliases of the tag notes. Here’s the map in ‘Reference A’:

Note how the link between the two annotations (an ‘untitled’ type text link, in this case) is clearly different from the ‘tag’ type links in dotted magenta.

…to be continued

OK, see Tag-demo.zip. I’ve added a user attribute set $TaggedAnnotations and I’ve also added $Tags to the ‘Reference’ prototype’s key attributes.

Each of the the 3 prototype now has an edict.

Annotation notes add to $Tags the $Name of any ‘pTag’ prototyped note to which they link by a link of type ‘tag’:

$Tags=links.outbound."tag".$Name;

References collected a (unique value) set of all tags used by their child (annotation) notes:

$Tags = collect_if(children,$Prototype=="pAnnotation",$Tags);

Tag notes collect a Set of all the names of annotations that used them:

$TaggedAnnotations=collect(find($Prototype=="pAnnotation"&$Tags.contains($Name(that))),$Name);

You could extend the last, to make the tag note list the using annotations by adding this code to the edict:

$Text=$TaggedAnnotations.format("\n");

You could gather $Path rather than $Name. Further customisation is left as an exercise for the reader!

To run all edicts on demand, use File menu → Update All Agents or shortcut Cmd+Ctrl+=. Make some changes to the doc then try out updating the edicts.

Using the above, here’s Attribute Browser looking at which annotations use which tags:

Or look at tags and which annotations use them. In this case we’re using the view’s options query feature (query: $Prototype=="pAnnotation") to filter so only annotations are shown:

1 Like

Thanks for this great answer, again! Most important things first: I think you’ve helped me to put together a workable system now. Will see how it performs in the coming weeks.

Now for the details:

You were mostly right with your assumptions. The blue notes are of the prototype “notes on notes”. However, I was not using $Name of the orange “tag” notes - this is prototype with the single key attribute $Tags. First I did this to keep things simple but it turned out to be useful in unimagined ways. And indeed, an annotation note (using your term here) might have more than one tag.

You were also right about the need to use tags on more than one reference. Perhaps I’m doing something wrong, but I found the creation of an alias and moving it into the reference a bit tedious. It somehow breaks the flow of briefly inspecting each note and quickly adding a tag/link to it. Adding a new note and writing a short title, which may or may not be turned into a tag seems much quicker. Using the Attribute Browser with just $Tags as query I should get a nice overview of all tags in use.

I very much liked your idea about using a special type of “tag”-link. So, for the annotation notes I use (via an edict):

$MySet=links.outbound."tag".$Tags; $Tags=$MySet;

And for the references I also went with your suggestion:

$Tags = collect_if(children,$Prototype=="notes on notes",$Tags);

My workflow for the initial analysis now looks like this:

  1. Import reference and import exploded text files (annotation notes) into reference container.
  2. Open outline view.
  3. Start reading notes from top to bottom, identify topics.
  4. Press Enter to create a note for the topic, change it to prototype “tag”, link annotation note to this “tag” note.
  5. Continue until done.
  6. Switch to Map View, rearrange map. Done.

This provides me (in theory) with:

  1. A map view of the article content in thematic clusters.
  2. Tagged annotation notes with important metadata of the reference.
  3. A reference that also has all tags of the annotation notes.

However, I do see some weird behaviour. Sometimes annotation notes list tags they should not have. When I click on them, they instantly show the correct value. This happens in Attribute Browser (example 1) and Outline View (example 2). Also, at least in one case the tag of an annotation note is not added to the reference (despite having pressed the “Update Agent Now”, example 3).

Example 1

Example 2

Example 3

You said you didn’t like aliases, so in may be you’re making tag notes of the same name in different maps. Tinderbox allows the latter but don’t expect queries and action-based linking to work correctly. If Tinderbox is trying to match a note called ‘motivation’ and there is more than one (original) note of that name Tinderbox will always pick/link to the first by $OutlineOrder. My demo used aliases specifically to work around that issue.

If I’ve misread the problem, my apologies. To diagnose further, the explanation and grabs—though helpful—aren’t enough detail. We really need to see the code either in a version of your file (perhaps cut down) or a new demo but which manifests the same problem. That allows both reviewing/testing code and giving a common frame of reference.

HTH :slight_smile:

I don’t think this is should be an issue. First, right now all tag names are unique (I only experimented with this one article) and second, I’m not calling any of these tag notes by $Name. This is the edict of the annotation notes:

$MySet=links.outbound."tag".$Tags; $Tags=$MySet;

I’ve prepared a demo version of my current setup. research 2017 demo.tbx (178.9 KB)

Perhaps I should use a stamp, as @PaulWalters suggested (thanks!), to also increase the reliability of the process?

Thanks again for taking the time to look into this!

1 Like

Thank you for the document – very useful to see the work itself.

I see this. When selecting a note in an Outline (e.g., “p323 the end of the tax burden.txt”) I see that it has $Tags=taxes, and then when selecting another note the $Tags column for “p323 the end of the tax burden.txt” displays several different values for $Tags for that note.

Perhaps this is because of the $Rule for that note: $MySet=links.outbound..$Tags; $Tags=$MySet;?

Also, I don’t understand why some notes with $Prototype==“notes on notes” have that rule but other notes do not? Did you change your mind somewhere in the development process about the rule and forget to either add the rule to notes or forget to zero it out?

Only about 20% of the notes with $Prototype==“notes on notes” have a rule – and the rules vary from note to note – why is that?

The document is so dynamic, with so many rules executing – maybe that is causes a problem? Sending it to Mark Bernstein might give you an useful second opinion and advice?

I think you might have found the problem! I was experimenting with rules before switching to edicts. All of those rules should not be there (somehow they survived the changes to the prototype - some I might have changed by hand) - without them it seems to work just fine. Thanks!

Good deal. Thanks.

BTW, probably moot but I had left your file open in Tinderbox and went about doing other things. A hour later I noticed that Tinderbox had hung. I’m guessing it was because of the continual iteration of the rules. So moving them to edicts was probably a good design choice.

[sorry for slow reply - ongoing broadband issues]

TBX open on my MBA seemed stable. Paul’s observation is pertinent as a reminder that when changing from rules to more controlled methods, you want a to-do note to do clean up the rules. An agent query for $Rule (i.e does this item’s $Rule have a non-default value) should find them all and an action of $Rule=; (reset the rule) should fix the issue. I do note you have an agent in your ‘PrepWork’ container. The title is in italics normally implying this is an alias but I can’t find an original elsewhere in the doc (looking at the XNML source it is not marked as an alias.

1 Like

In addition to Mark A’s suggestion, I suggest a couple of fit-and-finish modifications to the document that can improve readability and future-proof it.

  1. It’s good practice to put prototypes into a root container /Prototypes and templates into /Templates
  2. Don’t mix agents and code notes, or plain notes into these special containers
  3. I would advise not naming notes or prototypes with operators in the name (e.g, reference + text) since some actions can parse that string and might interpret the character “+” as a real operator. This is not a danger in your current file, as far as I can tell. If you want to show operators or special characters in your names, use a $DisplayExpression.

Which in turn reminds me, if you’re using link-based actions be careful about your naming of notes. Although Tinderbox is happy with quite exotic note names, be aware that certain characters can cause problems in relation to link actions.

Titles like “collectives / communes” can be problematic as Tinderbox’s link parser might assume 'communes is a child of ‘collectives’. I re-iterate, for general use this doesn’t matter but if you’re going to query paths via query/action code you need to be conservative.

I can’t see under the hood but experience says the following should be regarded as unsafe in note names if doing link-based actions: ampersand, left and right parentheses (i.e. normal brackets), and forward slash. There may be others - without documentation it is hard to tell.

These hints are very much appreciated and have been acted on! :slight_smile:

1 Like

@laotang, I’ve tested your very good “research 2017 demo” file and am wondering if it would be possible for you to upload an updated version, with the corrections you’ve implemented?

This may well be the right paht for me to move part of my work to TB. Thank you so much.

Dear Rene Trappel :
Thanks in advance.
This article relating to my area of interest.

TBD, I am afraid I am lost. Can you help me?
I can not prove the difference between “research 2017.tbx” and “The Brain.app.”, (if possible?)
I am a newbie for Tinderbox.

I will put in an effort to use Tinderbox for the best.
Tinderbox acts like everything is hunky-dory, I believe.

I accustomed to use The Brain.app
when something a little out of the ordinary using DEVONthink Pro.
I will import my data from DEVONthink to TheBrain.
I am a devotee using MORE, but now under OSX, I lost this background.

It seem worth a try “research 2017.tbx” at an early date.

Could you explain more about up-close view of your landing site?

I apologize to you for my rudeness, again.
Truly Yours. WAKAMATSU kunimitsu

I’m out in the woods camping (literally) and will get back to your questions early next week. Using this setup for several weeks now I can also add some thoughts about user experience in the real world.

René

Sure, as soon as I get back. The main difference is that in the document posted here there still had been some notes with rules attachted to them (from previous experimentation) - which conflicted with the edicts.

Dear René,
Thanks in advance.

I am looking forward to it.
I will appreciate your kindness in making
to get messages in the most dynamic form possible.

Yours, WAKAMATSU