Tinderbox Forum

Agent to calculate notes' Relative Vertex Orders


(Enrico Scarpella) #1

Hello everyone,

I have spent the past several days trying to do something with Tinderbox, but I am very new to it and success has so far eluded me; I was therefore wondering whether someone could be so kind to help me out or at least point me in the right direction.

I would like create a Tinderbox Agent that calculates automatically for each note in a document the note’s Relative Vertex Order (RVO), i.e. the total number of links (inbound + outbound) for a given note divided by the total number of links in the document, and that attributes different colors to notes with different RVOs.

For example, notes with no links (RVO=0) could be, say, violet; notes with 0<RVO≤0.2 could be blue; notes with a 0.2<RVO≤0.4 could be green; notes with a 0.4<RVO≤0.6 could be yellow; notes with a 0.6<RVO≤0.8 could be orange; notes with a 0.8<RVO<1 could be red; and notes with RVO=1 (very unusual) could be white.

I think the AgentAction should be, for example:

$Color=“yellow”

But I am really stuck on the Query… So far, I have:

($InboundLinkCount+$OutboundLinkCount)/…>0.4 & ($InboundLinkCount+$OutboundLinkCount)/…<=0.6

But I don’t know whether parentheses can be used in Tinderbox, and I don’t know what Attribute to use at the denominator (…); is there an Attribute that counts the total number of links in a document?

Thank you for your consideration.

Best regards,
Enrico


Links counts and attributes
(Paul Walters) #2

Perhaps

$TotalLinks = sum(descendants,$InboundLinkCount)+sum(descendants,$OutboundLinkCount)

would yield the denominator if all the notes were inside a container at the root. Call that container, say “Base”.

If you want to include text links with basic links, then use $OutboundLinkCount rather than $PlainLinkCount.


Edit: adjusted example to conform with my final post, below


(Enrico Scarpella) #3

Thank you very much for the prompt reply, Paul.

Currently, the notes are not inside a container, but I could always create one and move them into it; however, the notes are all at the same hierarchical level: they neither have ancestors nor descendants. But perhaps I misunderstand the meaning of “descendants” in your suggestion.

I have noticed that the Document Inspector does provide the total number of links, so the number I would need at the denominator is somehow calculated by Tinderbox; only, I don’t know whether the formula used for that calculation can be made accessible to or spelled out for an Agent.

Thank you very much again.

Best regards,
Enrico


(Paul Walters) #4

“descendants” refers to all notes inside a container, regardless of whether those notes are siblings (same hierarchical level) or child notes of other notes inside the top-level container. So the formula provided grabs anything inside the top-level container.

The formula provided is using an attribute ($PlainLinkCount) “somehow calculated by Tinderbox only”.

Have you tried to use the suggestion?


(Enrico Scarpella) #5

“descendants” refers to all notes inside a container, regardless of whether those notes are siblings (same hierarchical level) or child notes of other notes inside the top-level container. So the formula provided grabs anything inside the top-level container.

Thank you for the clarification.

The formula provided is using an attribute ($PlainLinkCount) “somehow calculated by Tinderbox only”.

I am not sure what you mean by “somehow calculated by Tinderbox only”: that’s not what I wrote. Could you please clarify?

Have you tried to use the suggestion?

Yes, I have, with both $OutboundLinkCount and $PlainLinkCount, but neither matches the total number of links provided by the Document Inspector. I will try it again with a document with fewer notes, so that I can count the number of links manually and see which, among the Document Inspector, the sum(descendants,$OutboundLinkCount) and the sum(descendants,$PlainLinkCount), provides the correct total number of links.

Thank you again.

Best regards,
Enrico


(Paul Walters) #6

Looking at the OP, I believe the denominator calculation should be

$TotalLinks=sum(descendants,$InboundLinkCount)+sum(descendants,$OutboundLinkCount);

Using that in the $Rule for the “Base” note, there are 3 Inbound and 3 Outbound links, of which 1 of the Outbound links originates from the “Base” note (where it is Inbound to the “B” note). The calculation does not include the “Base” note’s In- or Outbound links – you will have to decide if that matters to you. Therefore, among the descendants, the rule calculates 3 Inbound + 2 Outbound == 5 links.

The Info tab in the Document inspector counts “links” not edges.The Info inspector reports “3 links” because it is counting Plain links + Text links + Web links – and excludes “prototype” links. In this example there are 2 Plain links and 1 Text link == 3 total links per the inspector.

If you are counting edges, then you will want to add Inbound + Outbound links. If you are counting link pairs only, then count either but not both.

If you use the suggestion to put the document inside a root container (“Base” in this example) then do not have any links from or to the “Base” note. In other words the example above demonstrates what not to do.

LinkCounting.tbx (58.2 KB)

(Note, the suggestions above are not the solution, but refer only to the “how to calculate the denominator” question. You’ll need use an agent or stamp or other mechanism for the rest. Good luck with your experiment.)


(Mark Anderson) #7

I think there were a few miss-assumptions at the start of this thread, so have made a new thread here to describe what links are/aren’t counted and whether they are available to action code.

I presume this refers to the link count in the Info tab of the Tinderbox Inspector. As further described in my new thread (above), this count can not be accessed via action code.

Note that the value of $TextLinkCount is (internal) text-type links plus (external) web-type links. Thus if you need the count of only text-type links, use an expression like:

$TextInternalLinkCount = $TextLinkCount-$WebLinkCount

Thus, if a note has two text links and one web link, the note’s $TextLinkCount value would be 3 whilst the user attribute $TextInternalLinkCount would have a value of 2.

Note that $OutboundLinkCount excludes Web links, as the target is external to the Tinderbox document. Conversely, $TextLinkCount includes Web links: its value is (internal) text links plus (external) web links.
Thus if you need the count of only text-type links, use an expression like $TextInternalLinkCount = $TextLinkCount-$WebLinkCount. IOW, if a note has two text links and one web link, the note’s $TextLinkCount value would be 3 but user attribute $TextInternalLinkCount would have a value of 2.

Here, I’ve slightly modified @PaulWalters’ document to add a few edge cases for testing purposes:

  • the ‘Base’ container has no links.
  • ‘a’ has basic links to ‘b’ and ‘c’ and a text link to ‘c’.
  • ‘b’ has no links, but two aliases. One alias has a basic link to ‘a’, and the other a basic link to ‘c’.
  • ‘c’ has a single web link.
  • ‘c’ uses a prototype.

Notes:

  • The Inspector’s link count appears one high as it includes the one web link in ‘c’. IOW, there is no calculated value for only in-document links.
  • The prototype ‘Test’ is used by one not (‘c’) and thus has one basic link. But because it is of link type ‘prototype’ it is not counted in the overall Inspector link total.
  • The overall sum of $OutboundLinkCount does correctly sum discrete internal-only links (also omitting prototype links).

(Paul Walters) #8

Very useful mods. Thanks @mwra.

Nor does it need to be accessed. What link “counts” count depend on the intention of the user. We’re not sure what meaning for “count” the OP has in mind, but as @mwra and my posts point out, there is more than enough data available via system attributes and action code to satisfy the apparent use case that started this discussion;


(Paul Walters) #9

I have another suggested approach to the denominator. If the OP reads what’s been posted here and there about how to interpret “Link Count” in the Inspector’s Info tab, and if the analysis that the OP wishes to produce is not something that needs much repetition, then a simple approach might be to

  1. Create a user attribute, say “MyLinkCount”
  2. When needed, inspect the value shown in the Info Tab for “Link Count” and then insert that number in the Default field for the $MyLinkCount attribute in the User Attributes inspector.

As long as $MyLinkCount is not modified in any other way in the document (not in an agent, in a KA field in a note, etc.) then the OP could do this calculation with confidence:

($InboundLinkCount+$OutboundLinkCount)/$MyLinkCount

This sort of hardcoding of an attribute value is acceptable if it reduces action or rule churn in the document; if the analysis is not something that needs doing continually; and if the author remembers to update the $MyLinkCount variable when needed.


(Enrico Scarpella) #10

Dear Paul, Dear Mark,

I really don’t know how to thank you for all your help, but I will say it anyway: thank you so very much!

I am going to start working right away with the material you have so kindly provided; should you think it would be of interest, I will report back in the forum my findings.

I would just like to end by saying that the support you, the other users in this forum and Eastgate provide is not only notoriously outstanding but critical, and frankly a key component of my decision — and I am sure of that of many other users’ — to purchase that amazing, but sometimes intimidating, piece of software that is Tinderbox. So thank you very much for being so generous with your time.

Best regards,
Enrico


(Paul Walters) #11

Many people will find this very interesting. Please let us know how it’s going.


(Enrico Scarpella) #12

Dear Paul, Dear Mark, Dear Users,

First of all, let me thank you again, Paul and Mark, for all your very useful help. Should you be interested, please find attached the result of my efforts (I have also attached a screenshot of the map view). In it, you will see:

I have created a User Attribute called TotalLinkCount, which is defined as:

$TotalLinkCount=sum(all,$InboundLinkCount)

This is slightly different from what suggested in two respects:

  1. I found in aTbRef that I could use “all” to indicate all the notes in a file, so I used that instead of using “descendants”, which — if I am not mistaken — requires placing the notes in a container.
  2. To calculate the total number of links, it’s sufficient to count all the inbound links (or all the outbound ones) for all the notes in a document because any given link is outbound for a note and inbound for another; counting both gives you twice the number of links in a document.

I have also created a User Attribute called RelativeNoteOrder (RNO), which is defined as:

$RelativeNoteOrder=($InboundLinkCount+$OutboundLinkCount)/$TotalLinkCount

This is a measure of how connected a note is within a network of interlinked notes.

Finally, I have created five Agents that attribute different colors to notes with RNOs that fall within different intervals:

  • Red, for 0<RNO≤0.1.
  • Bright Red, for 0.1<RNO≤0.2.
  • Orange, for 0.2<RNO≤0.3.
  • Poppy, for 0.3<RNO≤0.4.
  • White, for 0.4<RNO≤0.5.

Of course it’s possible that all of this could be achieved with a single Agent, but this already does what I need. And of course this is just an example: one could define more or fewer, broader or narrower RNO intervals, and attribute to them an entirely different set of colors, shapes, etc. Finally, it could be modified to reveal notes that receive proportionally more (or fewer) links than those they send out, i.e. notes that are referred to more frequently than they refer to (or the other way around).

In my case, this helps me visualize emerging patterns that would otherwise take much longer to identify or that would be missed altogether. So, again, thank you, Paul and Mark, for helping me accomplish that, and thank you, Eastgate, for the only software that allows doing that.

Best regards,
Enrico


(Enrico Scarpella) #13

And of course I forgot to attach the files… Apologies: here they are.

Best regards,
Enrico

RNO.tbx (86.8 KB)


(Paul Walters) #14

@enricoscarpella, this is all very clever. :1st_place_medal:

Thank you for posting the explanation, screenshot and sample file. These “going the extra mile” steps are always very helpful for everyone who reads this forum.

It’s also helpful to see how an initial post evolves into a solution – you always have more insight into what the answer looks like than the rest of us observers.

I realize your sample file is not the production file, but a couple suggestions to help scale the concept.

  • First, I think a prototype would be called for that contains the $Rule, etc.
  • It’s sometimes helpful when you have an information-rich color-coding scheme to accompany that with the calculation result itself – so you could consider displaying $RelativeNoteOrder on the map in the notes’ $Subtitle or in a $HoverExpression or even in the notes’ $DisplayExpression.
  • I would suggest not calculating a global result such as $TotalLinkCount within the $Rule for each and every note. Calculate it in one note only (say, one named TLC) and reference the value calculated in that note as $TotalLinkCount(“TLC”). This will reduce churn.
  • Finally, if you scale to a large number of notes, with a lot of calculations occur simultaneous on a large scale, you could see performance issues. So, consider make the agents into Edicts, etc.

(Mark Anderson) #15

Thoroughly recommend the suggestions in @PaulWalters post above. They might seem tangential or extra work at first sight but make real sense over time and more so if your document is more than a few 10s of notes.

Indeed, as the doc grows having 5 agents just do this task, rather than just one—or use edicts—adds to the moving parts plus you need to consider if agents doing unrelated tasks are matching aliases inside these agents rather than your actual intended target. Put another way, as a document grows in size and complexity, consider your agent use carefully and be mindful of edge cases.

The issue about root vs container is worked around with prototypes. The doc level $TotalLinkCount may actually include links related to thinks outside the actual task at hand - often the case as your document matures (joys of emergent structure). If you give all your work notes, i.e. the ones of interest, a common prototype or a common attribute values (e.g. $Tags or some discrete user attribute value) then you can sum those and agent-query (or edict-find) only those notes.

If it seems 5 agents are the only way to ‘see’ notes of that type, be open to thinking again. Attribute Browser view might be far more efficient. For instance, if only those 5 groups use non-default $Color, then set an AB view to use a query for a prototype (if used) and $Colour!=normal; set the view’s grouping based on $Color. Now, the AB view will only show notes from those 5 groups and bin them by $Color value.

Please, don’t read this as some binary ‘agent bad’, ‘other good’ argument. As the demo doc kindly shows above, agents are a really good initial way of teasing out structure. For those going the next step—to scale and/or longevity—the later suggestions apply. I’ve some docs with 00s of agents. They’re there for a purpose but do mean I need to think carefully about unintended query effects due to the sheer number of alias notes they create.

[edit: typos/sense]


(Enrico Scarpella) #16

Dear Paul (@PaulWalters) , Dear Mark (@mwra) ,

Thank you very much for your very kind and encouraging words. And thank you very much also for the additional suggestions, which I by no means consider tangential or superfluous and which — I hope — I have taken into full account (please see below): for now, I only have a couple of Tinderbox documents with the few tens of personal and professional notes that I have been collecting since my recent purchase of Tinderbox (only a few weeks ago), but the intention is to move to Tinderbox the notes I had been collecting over the years in TiddlyWiki (and had been visualizing in TiddlyMap), so your suggestions are particularly welcome and even more appropriate.

I very much appreciate the suggestion to reduce unnecessary calculations, so I have now created and attached a revised Tinderbox document (“Revised RNO”) in which I moved the $Rule to a Prototype (“RNO Prototype”), from which I removed the calculation of the $TotalLinkCount, which is now performed only in the Reference note “Total Link Count” and to which the Prototype refers. I also removed the Agents and transferred their queries and actions to an Edict in the Prototype (I certainly don’t need those queries/actions to run continually in the background).

I wasn’t entirely satisfied with the fact that so much information is encoded in the color of the notes, but I did not know how to add the required redundancy in visual coding: typically, I would use different shapes together with different colors — e.g., hexagon and red, lozenge and bright red — but I already use different shapes to distinguish different Prototypes in my own Tinderbox document (not in the attached document, however, where there is a single Prototype), so the suggestion to display the $RelativeNoteOrder in the $Subtitle and $HoverExpression is very welcome and I have integrated it into the attached document (please see attached screenshot “Revised RNO - Map View”).

It is equally very welcome the friendly reminder to be mindful of unwanted effects as the size and complexity of the document increases, in view of which I explored the suggestion to use the Attribute Browser. I am not sure if I executed the suggestion properly, but in the attached document the notes are sorted by color bins in alphabetical order — bright red, orange, poppy, red, etc. (please see attached screenshot “Revised RNO - Attribute Browser View”) — which is not particularly helpful to me. Would you know whether it is possible to reorder those color bins according to a custom order? I was not able to find any information to that extent.

Finally, I have a question on a related topic; I will ask it here, but if you think it should instead be posted separately, please let me know and I will gladly do that. I find the Roadmap View very informative, so much so that I would like to have that information always visible. I was able to detach the Roadmap View from the location where it normally appears, so that it now floats like the Inspector; however, unlike the Inspector, the Roadmap View does not seem to be updated if I select a different note from that which was selected when the Roadmap View was first invoked. Am I doing something wrong? If not, would you know whether there is a way to have the information in the Roadmap View always visible and updated whenever a new note is selected (i.e. without invoking the View — either through the menu or through the assigned shortcut — after having selected a note)?

As much as I value your input, please do not feel you need to help me any further: you have already been exceedingly gracious and very generous with your time, and for that I thank you very much again.

Best regards,
Enrico

Revised RNO.tbx (68.2 KB)


(Paul Walters) #17

Thank you for continuing to share you work as it develops, @enricoscarpella. This thread is a very good example of “incremental formalization” – the sort of experimentation, adjustment, and development of your note taking concept that you are doing. It will be helpful as a good case for others learning Tinderbox or looking to do new things with the software.

Binning results by color is sometimes difficult to interpret unless the meaning of each color is immediately obvious. In this case I could imagine putting the file aside for awhile then coming back to it and not remembering what those colors denote. If I were doing this, I wouldn’t bin the results by color, I would do it by the word or phrase that denotes what that color means.

I don’t believe anything different can be done with the Roadmap, at this time.


(Mark Anderson) #18

Roadmap can shift focus - double-click a link in either column of the dialog - but it doesn’t ‘follow’ focus when that is changed in the main document window. IIRC, having the view follow focus, as per the the Inspector has been raised so is on the ‘stack’ of feature requests but there’s no harm in raising it directly with Eastgate so they can gauge interest (there are always more requests that resource to make them). The new hyperbolic view, which is still in development might be an answer to this issue. Essentially it is a visualisation of the document’s link network, eliding the hierarchy of the underlying outline.

I’d add my thanks for you sharing your progress. Embracing incremental formalisation (i.e. emergent structure) can seem confusing at first sight so worked examples help those starting out.

This next idea wants careful use, but you could make and name your own custom colours; they could even be the same colour as existing one. IOW, you could make a new colour that is the same visual colour as, for instance, ‘bright red’ but you might call your ‘point-two’ (I’d suggest not using numbers ). Now, the note would remain the same visual colour as above but is you use those new colour instead of the existing ones your AB view grouping would list groups like ‘point-two’ rather than ‘bright-red’. Whether you use different colour shades for these groups or simply use same colour but different name I leave as an exercise for the reader.


(eastgate) #19

I’m away from Eastgate world HQ for a few days. But I think live updates might be added to Roadmap. I’d like to know more about the application; perhaps by email to bernstein@astgate.com


(Enrico Scarpella) #20

Thank you very much, @eastgate, for your interest. I will then follow your suggestion and will contact you separately by email with more details.

Best regards,
Enrico