Another scenario showcasing the importance of DEVONthink-like replicates: the WordNet use case. The synset music#5, medicine#4 which branches out to mutually exclusive hypernyns yet related synset hierarchy cannot be showcased in Tbx due to one level limitation of aliases. WordNet is the most accurate representation of the English lexicon and a powerful tool for cross-reference of hierarchical relationships. Relationships in Tbx are visually flat in both the outline and attribute browser. Hyperbolic view will loose its purpose on large data sets. Most features in TBx are also geared towards text body content and not so much towards container text and, in my opinion, this is a tremendous limitation especially for those that parse and derive relationships from chunks of broken body text - paragraphs, sentences, etc. Hierarchical visualization still rulesā¦WordNet lexical database proves exactly that.
ālooseā or āloseā? I certainly donāt believe hyperbolic view is necessarily constrained by node count. Indeed, it is upon my larger documents that Iāve been playing with its use.
Anyway, hyperbolic view doesnāt care about the (stored) outline of the notes, as it only plots linked items. In an outline parent/child relationships are assumed but not formalised as Tinderbox links (though one may add such if desired).
$Text (body text) vs $Name (container text)? I would say quite the opposite. Indeed, Tinderboxās original design envisaged notes with small texts with little if any styling. Contrast long runs of styled (rich) text with embeds/inclusions as some would now prefer. Indeed, in my current large Tinderbox project most notes have no $Text at all and work is not suffering because of it. Thereās nothing wrong with $Text, but itās a more project leaning more on the link structure of the hypertext and the ātextā is essentially all metadata (attribute values). Using $Text instead of metadata would make the project much harder as most queries would need to regex mine $text instead of doing (easier/less computational) look-ups and tests of attribute values.
Other than the full unicode support and expanded visual styling coming on the back of the new v6+ app re-build I donāt think feature balance is overly towards note $Text.
As a long term forum mod/admin in this and previous Tinderbox user forums, my experience is text ($Text) related issues feature disproportionately in usersā questions. But we are a broad church, people discuss the issues that interest or bother them and we try to answer them. Tinderbox has a lot of features. Even after 14 years of writing and curating aTbRef I still find things Iāve missed, misunderstood or simply forgotten.
I do see the point of replicates, though they seem a functional sticking plaster for outlines. If we need replicates, it might imply weāre holding overly tightly to a particular view/organising style. For instance, if you link items appropriately you could use hyperbolic view to traverse the āoutlineā without the need to bother about the storage (outline). Doing DAM work a while back, I saw this outline constraint run through attempts to support early photo metadata. Apps used to have complex (outline) trees of keywords but, apart form where strict vocabularies were used, outlines fell from favour due to the weakness described above.
Perhaps we all need to explore and help improve the hyperbolic view (still very new) as a better way to understand note relationships.
Hey āknowā=now 
Concept mapping is what Hyperbolic view is to Tbx, in theory. And hyperbolic view on an entire controlled vocabulary is not much of use when you have over 8000 concepts with relationships between them; but thatās my view on it.
And that is exactly my point! Why canāt $Name also be metadata (attribute values)? Every piece of textual information can be grouped in some way and to its exhaustive limit. A paragraph of textual body content can be parsed to create an exhaustive taxonomy. That can only be created by grouping narrower concepts. Once thatās accomplished, every piece of information becomes its own āblockā which in turn becomes mutually exclusive to any other concept. You can find all the relationships in all the text you want but you will never āunstickā multiple paragraph content from multiple containers to fit the specificity of the narrowest of concepts. Anyhow, I grasp that the majority of people donāt think in terms of conceptual blocks and prose is the norm
I concur! ā¦ (extra dots required for reply minimum character length )
You can see all you want but the information will still remain static. Containers are not used for a predetermined scheme in my view. They are used as sticking points once information and relationships emerge. Itās a complicated subject matter which requires some understanding of linguistic concepts such as lexical semantics and a few other in the field of linguistics. And the courage to let go of the idea of textual matter. hierarchies are more powerful than youād thinkā¦
For sure. One discussion to have, as the view matures is to scope the view, in terms of how far out you plot. Unless you want to explore the whole, it may you might want to just look N links out from focus which inherently reduces the number of notes/links to plot. In this regard, I think hypertext tools in general are lacking.
Back in v5, I built a TBX for a client that was an 8 level outline with c.35,000 items IIRC. The performance issue is long outlines. IOW, the whole outlineāas currently expandedāthat usually extends beyond the part of it seen on screen and which we generally want to be able to scroll up and down at speed.
Where there is some nesting (e.g. aTBRefās source doc) I find keeping branches folded helpsāas the overall expanded outline remains smaller. For big projects, I work out of a root level container, i.e. such that all the ācontentā (as opposed to templates, prototypes and general notes, etc.) can be āfoldedā away in one place. Indeed, by hoisting that container (Focus view) in outline view it inherently reduces the immediate maximum scope (outline size) of the piece upon which you are working. sometimes it works better (in performance terms) if one uses several tabs scoped to different parts of the outline instead of scrolling around the whole thing. Iāll admit, the latter is my natural behaviour but Iām learning the the former so as to help not overload the view visualisation.
What are you doing with the WordNet corpus that is specific to Tinderbox? I ask out of genuine interest. Tinderbox isnāt a formal concept mapping tool in that it lets its user map out relationships but doesnāt intend to provide some formalisms of CM such as might be expected in a formal CM tool.
Iāve found Tinderbox is best for discovering structure. Once so discovered, it can be that āespecially for large datasetsāthat some more specialised tool is a better place to build out the full data structure.
It is. I wonder if out terminology/frame if reference is aligned? The $Name attribute is the ātitleā of a note. It is used to title items in the view pane (more strictly, $DisplayName is used) and it is used as the title at the top of the text pane ($DisplayName is not used here, as sometime you need to be able to see the actual $Name). So $Name is metadata. If doubting, add $Name to a noteās key attributes table.
$Name has a different role when it comes to linknig within Tinderbox. Although, under the hood, a link uses note $ID values, generally we either drag links in the UI or use action code. Action code uses a noteās path (IDs were invisible to the user until c.v5+), i.e. $Path or its $Name as a proxy. An implicit assumption is that $Name is unique within the current TBX, otherwise the full $Path is needed). A confusing factor that the linking code doesnāt allow for escapingā certain common non-alphanumeric characters such as parentheses and semi-colons, as these are misconstrued as code rather than content. Not a problem if planning your novel, more of a problem if the note title ($Name) is a reference to something like a scientific paper. You can bowdlerise the $Name to aid automated linking, but then you need to ensure you retain the true title (normally in a separate user attribute) for all other purposes. Of course if a note becomes a container its $Name is now part of its childrensā $Path, so the same care needs to be taken.
ā Sadly, Iāve yet to ascertain and document the true full set of ābadā characters for $Name/$path value with respect to links.
Most wonāt trip across this issue of titles and link-safe characters, but when you do (as in my own personal research work) it is a bit of a headache. But as an edge case, itās not what the majority of users face so canāt be presumed to be high up on a fix list (as I suspect it involves code very much at the guts of the app and so not the area for light surgery).
Amen. I didnāt mean to appear to disparage them, not least as I work primarily in outline for my personal research work (due mainly to the size of the datasets).
Hmmmā¦very good tip. Never thought of that in terms of performance.
Our thinking is alignedā¦Tbxās features are geared primarily to work with $Text rather than $Name. For instance, you can create #tags in your text pane, create a stamp, and then automatically extract textual #tags into attribute tags - kudos to @JFallows. A paragraph can be in the form of prose or in the form of a list - list paragraph, etc. Since each paragraph has a main idea, each main idea can become a container for that paragraph and each sentence can turn into multiple containers itself e.g. a sentence that explains, reduces, and clarifies. Those verbs can become containers to the main topic of your paragraph. Hence, the idea of exhaustive taxonomy. That paragraph is exhaustively parsed to its most specific semantic relationship. Now, the idea of extracting #tags from text to attribute tags using a stamp is no longer viable in this circumstance. Most of my main concepts have turned into containers. However, it would still be of great use if I could still create a stamp and extract the #tags from the container $Name.l but that is no longer possible. This is obviously very specific to my case - even though Tbx encourages small chunks of body textā¦sense the irony here?. The idea is that what remains of most of my $Text is a description of the most specific concept/topic/object and therefore the $Text has become 80% $Name. Semantically, the $Name is no different than $Text. Iāve simply arranged the Text (Just as one can create a list paragraph from a sequential item list in running prose paragraph) and the Iāve structured the text into into containers (as sections contain paragraphs so can main ideas contain sentences)ā¦of course Tbx cannot differentiate between my choices because it can only think in terms of $Text. The point is that $Text means very little in the way I use it but most features in Tbx favor $Text. I donāt know if I was capable to explain well enough to make more senseā¦I hope I did
Also think Codes and Quotes in Atlas.ti! The tool understands that paragraphs are meaningless to a user and thatās why you can code quotes and arrange them hierarchically. Again, it allows breakdown of information to its most specific concept/topicā¦
I parse content based on semantic relationships and WordNet groups words in terms of synonyms (called synsets) and senses (all senses a word might have). In the English lexicon, these synsets cannot all be mutually exclusive. If im not mistaken, I believe WordNEt has a hierarchal depth up to 16. Some, of these synsets are mutually inclusive in different domains/concepts and at different hierarchal levels. I constantly run into a need to classify in different containers based on WordNEt and with thousands of notes I canāt worry about manually duplicating repeating concepts in difference containers. Tbx allows for max 1 level cloning/replicating/aliasing of containers. Considering WordNet comprises a system 16 levels deep, it becomes difficult to manage a concept that needs to live multiple levels deep and across multiple groupings.
TBH, neither had I until I did by mistake. One wrinkle though, if doing linking, all tabs share the same link park, but separate document windows do not. There seems no gain in the latter and Iāve made a feature request that all windows (i.e. all tabs) share their link park. But for now, the preceding is the status quo.
Thanks - and interesting to hear about the task.
I may be misunderstanding, but the description doesnāt imply a need for an alias to support descendant context asāif Iāve understood correctlyāyou are testing if a term is āināā a particular container so its descendants are moot. I can see itās a pain to all lots of aliases across the corpus but itās actually less content than discrete notes.
ā Be aware that, for legacy reasons, the inside() query is more expansive than you might initial consume. Itās true not only if the original note is in that container but if an alias of that note is there. By comparison, the same is not true of descendants and descendedFrom(). To save asking, there is already a feature request for a more restrictive version of the inside() query that only counts originals as a valid match.
In Tinderbox vernacular the tags are in $Tags; this represents the general notion iin many apps of a single grab-bag of terms (or ākeywordsā) relating to an item. So for a note tilted āSome Noteā, you can get a list of the noteās tags like so:
$MyList=$Tags("Some Note");
$MyList now holds a list of those tags.
So letās assume the container is title āTopic Xā. You stamp used on itās descendants would be:
$Tags = $Tags("Topic X");
In fact there are a host of different ways, some more automatic, via which this can be done. Prototypes/Inheritance is another method for approaching this.
Where I may be missing something is the meaning of the term #tags. I think this refers to the method, especially in some simple note tools without discrete metadata fields, to mark a word (or underscore_joined_phrase) as a keyword/tag. I only ask because if wrong, can you set me straight as to your meaning. But Iām unclear as to what the ātagsā in a name. I guess you have a note called āSome term #tagA #tag_Bā. Is this so.
Not for Tinderbox. When you refer to these in the app, the $Name is the title of the note, the $Text is the noteās body copy, though thereās no reason the title ($Name) string canāt be repeated in the $Text as the first paragraph (or any p[art of $Text).
I donāt believe Tinderbox has ever ha, as a design intent, a decomposition of $Text. You can do various tests and manipulations on $Text. To Tinderbox a paragraph is, to my understanding any substring delimited by one or more consecutive hard line returns. there is do defined concept of a sentence (youād need to regex parse for all possible grammatically allowedā” sentence end markers)
ā” Remembering things like BrE/AmE differences on closing quotes before/after closing punctuation.
I donāt understand what these āchoicesā are. Could you explain a bit more?
All interesting stuff. Iād agree that an 8k plus item outline is going to require careful useful to remain responsive, but Iām not seeing anything impossible yet.
If it helps us better understand the issue, buy all means post a small TBX with enough info to explain the sorts of tasks youāre trying to do and canāt, or if easier post a before (āWhat I haveā) and after (āWhat I wantā) file. Perhaps, if doing so start a new thread to avoid undue topic drift here (conscious weāve already wandered some way from replicants)
My hunch is overall size is impeding normal initial-stage process formalisation (in terms of use of Tinderbox features to best advantage), quite naturally leading to frustration and lack of process. Anyway, if we fellow users can help, just ask away!