$Onadd link to container + inherit all meta-data (esp. tags)

Q & A - getting started with Tinderbox
1

Hi,

this one might be a bit tricky. I want to set up a prototype for a note/container (“reference”) so that as soon as I add a number of notes to this container they:

  1. Change to a certain prototype (“notes on notes”, this is the easy part);
  2. Inherit all metadata/attributes from the parent container “reference”;
  3. Are linked linked to the parent container “reference”.

Some background info: I read a lot of PDFs with PDF Expert (iOS) or Skim (macOS). Usually I highlight and comment the interesting parts of these PDFs and export these sections as a text file. Afterwards I chop this file into many smaller files, one for each highlighted section (I have a shell script for that). I do this, because it improves the chances that DevonThink’s AI will find similar content. Now I want to import all of these small text files into Tinderbox and relate them to their source - for use in later projects.

Any help or ideas are very much appreciated.

1 Like
2

(1) and (3) are easy, by setting the container’s $OnAdd to

$Prototype="protoMe";linkTo(parent,"Clunk");

or whatever looks like that.

However “Inherit all metadata/attributes from the parent container ‘reference’;” needs explanation from you. In theory, hundreds of attributes could be “inherited” – so “all” cannot be the requirement, really.

But, “inheritance” or assignment of “some” or “specific” parent attribute values to a child is not difficult. For example, if you want an attribute of the child to be assigned the value of the parent (or another note’s) specific attribute you can use additional $OnAdd actions such as

$Name(this)=$Name(parent)

but be very careful not to set off a chain of unintended consequences.

Another few approaches, that do not require adding notes to container, when you want to facilitate linking notes (e.g., your annotations) to source documents (e.g., a note representing your PDF), is to use stamps, or to use smart adornments.

**Stamps:**You could have a stamp for each source document and when you use the stamp against annotation notes the stamp could set up the proper linkTo() relationships and attribute inheritance.

Smart Adornments: The smart adornment approach does the same but in this case it is an adornment on a map that represents the document and dropping notes on the adornment triggers the adornment’s $OnAdd actions to set links, etc.

Agents: And, finally, you could use agents to find annotations according to specific queries and allocate the found notes to source documents and create the links.

Extra Credit

Here’s a tip that works with your dragged-in DEVONthink notes. If you keep your annotations for a PDF in a group in your DEVONthink database, and that group is named after the title of your document:

The Story of Mars and Belinda
--- Note 1
--- Note 2

and in DEVONthink you have enabled tagging for the group “The Story of Mars and Belinda”, then each of “Note 1” and “Note 2” will have tags in DEVONthink with the group’s name.

When you drag those two notes into Tinderbox, then the set attribute $Tags is automatically loaded with a tag The Story of Mars and Belinda, viz.:

and you can use an agent that queries $Tags for that value to locate the relevant notes for your document named “The Story of Mars and Belinda”.

Your agent can also throw the notes into the container that started this thread, by setting $Container() for them.

2 Likes
3

[deleted - basically posted together with but repeated @Paul’s much better post above :grinning:]

4

Thanks for the great answers. I experimented with the code Paul suggested and it worked quite well. Ultimately I went with:

$Prototype=“notes on notes”; linkTo(parent); $URL(this)=$URL(parent); $Authors(this)=$Authors(parent); $Titel(this)=$Titel(parent); $Tags(this)=$Tags(parent);

Now its seems that I need to figure out how to have the size of notes automatically adjust to the DisplayName. Additional space between the notes would also be great :slight_smile:

Bildschirmfoto 2017-05-18 um 13.43.47.png1920×1080 225 KB

PS. Thanks again, Paul, for the alternative ideas on how to solve my problem (esp. Extra Credit). Somehow, if it is more than one DT tags, I found that these tags are not correctly/reliably imported into Tinderbox.

5

To resize notes see the Note menu. There are options to expand (all selected objects):

  • Expand Horizontally (Map view only - otherwise greyed out). Tells Tinderbox to attempt to widen the note’s map icon to display the whole note Name (title). The selected note’s icon is expanded at the right side, maintaining existing height and X/Y origin. May be used in conjunction with Expand Vertically (below). The revised Map note width is retained for the test of the session and persisted if the TBX file is saved. This option is available when multiple notes are selected.
  • Expand Vertically (Map view only - otherwise greyed out). Tells Tinderbox to attempt to increase the depth (height) of the note’s map icon to display the whole note Name (title). The selected note’s icon is expanded downward, maintaining existing width and X/Y origin. May be used in conjunction with Expand Horizontally (above). The revised Map note height is retained for the test of the session and persisted if the TBX file is saved. This option is available when multiple notes are selected.
  • Expand Proportionally. Tells Tinderbox to expand notes proportionately if the note name is long.

For map view layout, see the View menu → Arrange sub-menu, and its further sub-submenu Cleanup. Most actions in these two menu affect only the current selections. If no selection is made cleanup actions apply to the whole current map. Be aware that for tasks such as assigning a common width, separation etc., the first (or first and second) selected items are used as the reference for measurement. Band-box (drag) select usually results in the top left item being the ‘first’. If in doubt, Cmd+click successive items to ensure the correct frame of reference.

6

Good solutions.

It appears your $DisplayExpression must be fairly complex. Have you considered putting all that info in the notes’ $HoverExpression instead.

A judicious use of $DisplayExpression, $HoverExpression, $Caption, $Subtitle and Adornments and $KeyAttributes can make maps very informative “at a glance” without shoving a lot of info into one or the other of those display-related attributes. Personally, I use $DisplayExpression more for Outline purposes, since outline lines wrap automatically and expand as needed.

1 Like
7

Further to @PaulWalters’ point about complex display expressions (DE). It can help, as your doc grows, to use a rule (or possibly an edict) to pre-compile the DE into string stored in a user string attribute and then make the DE use the latter:

$MyDisplayString = ($Price*4)+ $SomeString.substr(1,4)+[...lots of action code...];
$DisplayExpression = $MyDisplayString;

DE is fine for normal simple things like appending a container’s $ChildCount to the ten of the container’s title, or the like. But if DE is doing 3 or 4 manipulations for each of 0s or 00s of items, the above method is worthy of consideration.

2 Likes
8

I have a similar workflow, but the script I use is kind of a headache. Any chance you’d be willing to share yours?

9

There are several scripts posted in DEVONthink’s forum for this explosion of notes.

I’ll find the links and post them here.

And of course Tinderbox’s own Note > Explode can be used, but then you have to fiddle with getting the URL and Tags into the exploded note.

10

Thanks for all the suggestions in regard to $DisplayExpression and/or $HoverExpression. Truth be told, there is nothing in both attributes yet - but these two attributes might be the ones I’ve been looking for. The names of the imported files/notes in the screenshot above look a bit strange because I’ve adopted a particular naming scheme for those exploded text files years ago (and all other files, for that matter - inspired by this post: File System Infobase Manager, works great with Hazel). After the import I manually used the Cleanup function.

@derekvan Sure. This is zerhacken.sh (it just sounds better in German)

#!/bin/bash
csplit -f note-no_ -n 3 -k "$1" /^-/ {100}
rm "$1"
for i in * ; do mv "$i" "`grep -e "^- " $i | sed -e "s/^- //"`.txt" ; done

Warning: This will also delete the original file.

How to use: Textfiles need to be in particular shape to work. My summary files are usually written in Markdown (many benefits including the possibility to import into outliners such as workflowy) and start with

# Title
## Author, Year

These lines will be ignored. Next are the highlighted sections, for example:

- page no. Title/Summary for the section
    - highlighted content
- page no. Title/Summary for the section
    - highlighted content

The script will turn each pair of Title+highlighted content into a single file with the name of “page no. Title/Summary for the section”. It is important that the title line starts with an - and that highlighted content is indented or at least does not start with -.

1 Like
11

This seems a familiar issue - packing a title with extra (sortable) metadata. Once the data is in Tinderbox , you can of course unpack that metadata and pare back to the basic textual title. sorting is still possible as you simply sort the container on the attribute holding the data (be it a date, book name, page number, etc.) and note Tinderbox supports 2-level sorting (sort by X then by Y within each category of X).

Thus a complex imported note title (in pseudocode form), i.e. $Name like:

"yyyy-mm-dd_bookname_pageref_Incremental Formalisation"

Might end up as:

$Name: Incremental Formalisation
$RefDate: dd/mm/yyyy (user Date-type attribute)
$RefName: bookname (user String-type attribute, or Number-type if something like an ID #)
$RefPage: pageref (user Number-type attribute)
$OriginalTitle: yyyy-mm-dd_bookname_pageref_Incremental Formalisation ((user String-type attribute)

…etc. I’ve found this helps map use by giving me shorter, more readable titles. In turn that allows smaller map icons and slightly better data density - especially on smaller screens. Of course, things like $HoverExpression could be used to show $OriginalTitle or some other concoction of metadata you’d like to see on mouse-over, without having to change your current selection.

If nothing else, unpacking an overloaded title also makes analysis within Tinderbox easier as you can scope queries more tightly and avoid having to use regex-based string searches on note titles.

12

Improving the visibility of metadata was one of the things that tempted me to try Tinderbox. Is there an easy way to do this automatically (slice up $Name into $Name, $RefDate etc)?

13

The easiest way it to do a tabular import (tab-delim or CSV) - see here and here. Explode (here) also allows you to do actions on the imported notes.

However, in you case the data was pulled from DEVONThink into Tinderbox. In such a case you’re likely down to using regex ($Text.contains() - see here). It takes a bit of thought but isn’t difficult to do if you’ve some regex expertise. Tip: rather than using a rule, I’d experiment using a stamp (a once-only run of action code on the selected item(s)) and then use a rule/agent once confident of the outcome.

If you don’t mind starting over, you might find it easier in the round (and for subsequent import) to script DEVONThink to produce a TSV or CSV file that imports and maps the data directly. Note the tips in the links above re attribute names and pre-defining attribute pre-import to ensure correct data type.

If you want to go the route of extracting from text, I’d suggest starting a new thread seeded with a sample $Text and indicating what data you wish to pull (and/remove) from $text and the target attributes.

I hope that helps. :grinning:

14

Thanks for the extensive and detailed back-and-forth here. Instructive for those of us just following from the sidelines.

1 Like
15

DEVONthink out-of-the box includes CSV scripts for export, which can be easily customized, and there are dozens of user-contributed options for CSV, TSV, and OPML export in the DEVONthink forum. Similarly, there are contributed scripts in legacy Tinderbox forum. Over the past 10 years both forums have addressed export / import in robust detail.

I suggesting preferring one of the various custom OPML scripts that users there have contributed over CSV or TSV. Well-written OPML exports can handle Unicode and text blocks better than CSV – CSV can go very bad very fast when it encounters punctuation. Stick with OPML. Tinderbox plays very well with OPML

1 Like
16

Thanks Mark for the detailed answer. I played around with OPML and the explode feature and will keep both in mind.

I’m still in the early days of setting up my Tinderbox database, so I haven’t imported a lot right now. It might be easier to redo the import with much shorter filenames (Tinderbox quit on me several times today and seems to appreciate those shorter filenames). And I really do like the improved visibility:

Bildschirmfoto 2017-05-19 um 21.53.07.png1920×1080 188 KB

The blue notes are imported from Devonthink. I arranged them around similar topics, which I marked with a note (land grab, taxes, modernization in the example above). Would it be possible that all notes linked to these topical notes inherit their $Tags? In the example above the notes “p322 the red line”, “322 zengjian guaogou” etc would all inherit the tag “land grab”. And if possible, would it be better to do this with an Agent (who continuously updates - I’m afraid Tinderbox chokes when I add more than thousand notes) or something else?

I probably should have started a new post for this question :wink:

17

Although the OPML spec has very few defined fields/attributes, I’ve tested and Tinderbox seems to import all (most?) custom attributes in the OPML records to system attributes or create user attributes for them. The mapping is case-sensitive. If using this route, as with TSV/CSV I’d set up the ‘custom’ attributes as user attribute pre-import so you can ensure the correct data type is configured before any data is added (i.e. avoid lists importing as strings, etc.). If any import attributes are lists, ensure you set a semi-colon delimiter in the source OPML to ensure the values unpack correctly when imported to Tinderbox.

Yes, though the immediate question is how many notes, how often and is this is one-pass-per-note event (i.e. will existing tags need to update. One route (whether stamp/rule/edict) is to find() all notes linked from that topic and set their $Tags to be the $Tags value of the topic note. If using an agent, the find part becomes the query and the rest is the action. Which route you takes depends on how often $Tags change/notes get added. IOW, there’s no correct way - one method will prove best fit for for your workflow.

[edit - typo]

18

So I went with a rule for the “notes on notes” Prototype to experiment with this:

$MySet=links.outbound..$Tags; $Tags=$MySet;

However, I do have difficulties to limit this collection of tags to those linked notes which are of the “tag” prototype.

I tried:

$MySet=links($Prototype=="tag").outbound..$Tags; $Tags=$MySet;

and

if (linkedTo($Prototype=="tag") {$MySet=links.outbound..$Tags; $Tags=$MySet;};

Both worked not as intended and also added all tags of the parent/container, which is a note of the prototype “reference”.

I also tried

$MySet=links(siblings).outbound..$Tags; $Tags=$MySet;

Which returned all tags of all siblings, not only the ones the note was linked to.

And this one, which did not work at all:

$MyString=find(links.outbound..$Name & $Prototype=="tag");
$MySet=links($MyString).outbound..$Tags; $Tags=$MySet;

If this works as intended I intend to add a rule for the “reference” prototype:

$MySet=links.inbound..$Tags; $Tags=$MySet;

This should automatically collected and add all relevant tags to a reference based on my analysis of the exploded summary files.

19

I would split this calculation. One possibility is to have an edict

$LinkedNoteTags=;$LinkedNoteTags=links.inbound..$Tags

that, because it is an edict, is only applyied periodically to the parent “note on notes” and would reduce the calculation burden in your document.

Folks have lots of intended purposes for tagging – it might be helpful to understand how you are using tags in your analysis. Are you merely interesting in viewing the tags of linked notes – in that case, an attribute browser might be just a valuable as these rules.

If your document is crashing Tinderbox you might have introduced some recursive or circular calculations. You should be able to get well beyond 1,000 notes. If you have crash logs, send them to Eastgate directly, with a copy of your document, for analysis and insight.

20

The links() operator (syntax here) needs a path or list of paths from its first input (the bit in the parentheses). If the parentheses are empty or missing the pass is assumed to be ‘this’ note, i.e. the currently selected note> I think the filter you were looking for is a find() query:

$MySet=links(find($Prototype==“tag”)).outbound…$Tags; $Tags=$MySet;

(N.B. not tested).

In this sort of context, I’d caution against a rule with code like the above or with 00s/000s of notes it can slow things up while every note’s constantly re-evaluating its linkages. Go with an edict or stamp. Edicts can be run on demand (by calling File menu → Update agents).

The way to find all unique $Tags values for notes using the prototype ‘tag’ is to use collect(), which returns a List. Thus:

$MySet = collect(find($Prototype=="tag")),$Tags);

or:

$MyList = collect(find($Prototype=="tag")),$Tags).unique;

I endorse the advice not to overlook the Attribute Browser view as, depending on your real underlying task, it may avoid adding lots of code you don’t really need.

1 Like