Putting content of a Summary Table into the text of a note

[Paul_Zahara]

Hi, I’ve had to prepare a bunch of YouTube videos to change from face to face teaching to online teaching, and I’ve got a note for each video. One of the properties of those notes is $YouTubeTime and another is $YouTubeURL to hold the elapsed time and URL of the notes.

I’ve set up a summary table to display the name, time and URL of each note, and now I want to export that table of information to another application. Nothing fancy, I just want to copy and paste the summary table into a text document. Unfortunately I can’t figure out how to do that.

As long as I’m asking, I would like to set the subtitle of the container holding the summary table to display the number and total elapsed time of the the videos in the container. I can’t get the syntax of the “sum” function to work with time. What am I missing?

Thanks for your help.
Paul

[entropydave (David)]

I’m not sure of a Tinderbox way to do this, but if I wanted a quick solution I would take a screenshot of the summary table, suitably zoomed, and then run OCR on it. Applications like Condense and Prizmo, to name two, can do this.

[mwra (Mark Anderson)]

Let’s do the last first as it is he smaller task.

@Paul_Zahara

set the subtitle of the container holding the summary table to display the number and total elapsed time of the the videos in the container

Let’s do the constituent parts first. The number of items is the count of children with a $YouTubeTime set. The latter check ensures any child notes (or containers ) for other purposes are ignored, so we use sum_if(). The code, returning a number, is:

$MyNumber = sum_if(children, $YouTubeTime, 1);

The second parameter is a short-form test for an attribute having a value. The last parameter simple add 1 to the sum for each child matching the test in the second parameter.

I’m assuming $YouTubeTime is an interval and and sum()/sum_if() work with numbers. Also, I don’t believe intervals can be added/subtracted—I need to run a test, and have a call due now. I’ll report back in a bit.

(BTW the solution to the other part, which I’ll also do later is to use an envelope/letter method to export the container summary as a text table)

Back in a bit, but don’t let that stop anyone else finishing off in the meantime.

[Paul_Zahara]

Thanks for the quick response Mark.

I understand that your code assigns a value to the variable $MyNumber and your explanation of the syntax of the code makes sense. Thanks.

I still have a bit of a problem in that I don’t know how to deploy the code. If I put your code in the Subtitle window in the Inspector I understand that I won’t be assigning the code to the Subtitle property. What I don’t know is how to get the code to assign the value of $MyNumber to the $Subtitle property.

Sorry for the very basic question.

I’ve had Tinderbox for years but I’ve only scratched the surface and now I’ve decided I need to put the effort into learning how to use Tinderbox to its full potential. I could turn out to be a pain, but I’ll try to only ask questions after I’ve referred to the documentation and tried to solve the problem myself!

[mwra (Mark Anderson)]

(waiting for call). Testing, intervals can be summed, so the next bit of code is:

$MyInterval = sum(children, $YouTubeTime);

Here we can use plain sum() as items with no interval set will add a zero value to the total.

Now to assemble the subtitle. I suggest using an Edict rather than a rule. Why? This count isn’t constantly changing so we don’t need an always on rule. Edicts run less often but can still be run on demand (see link on edicts). $Edict code for the container:

$Subtitle =  (sum_if(children, $YouTubeTime!="00:00", 1)) + " videos, total duration: " + (sum(children, $YouTubeTime));

In test I found i needed to use the long form of an attribute test to ensure only children with an interval were counted. The result looks like this:

If you want to further format the interval, look at Interval.format()

[mwra (Mark Anderson)]

This is from a demo, i’ll upload shortly:

Above is our table with $Subtitle (via container’s $Edict). Also shown is the configuration for the summary table (see the stored $TableHeading and $TableExpression in the container’s Key Attributes in the demo file).

Now the export template. I’m going to use TSV (tab-delimiter) text. For copy paste we don’t need to export as we can copy the ‘export’ table from the container’s Preview or HTML tab (you may need to turn these on Window menu → Show Text Pane Selector).

Export - outline view is best for looking at this:

The ‘envelope’ template TSV-table-wrapper has these two lines of code:
Lesson Duration (hh:mm:ss) Youtube URL

Lesson	Duration (hh:mm:ss)	Youtube URL
^children("TSV-table-item")^

The ‘letter’ template TSV-table-item has these two lines of code:

^if($YouTubeURL)^^value($Name)^	^value($YouTubeTime)^	^value($YouTubeURL)^
^endIf^

The ^if^ conditional—which encloses the line return at the end of each line of the table—is used to exclude non-lesson (based on them having no $YouTubeURL value.

Set the container of the table to use the TSV-table-wrapper template using the Export Inspector.

Now set the container text pane tab to ‘HTML’ or ‘Preview’ and select/copy the text. Note the clipboard contents will be styled text so if pasting to an TRF document or spreadsheet, use a plain text paste or paste-and-match-style. Here is our ‘exported’ table pasted into Excel:

And, here’s the demo: Container-table-demo.tbx (94.5 KB)

[Paul_Zahara]

Thanks very much Mark. That worked exactly as you explained.

[Paul_Zahara]

Thanks Mark
It’s late here in Australia. I’ll get back to you tomorrow after I’ve worked my way through your example.

[Paul_Zahara]

Mark
It’s the next day here and I’ve had a chance to work my way through your example/demo. It worked very well, but I have to say that if left to my own devices, I never would have figured out how to export my information.

From time to time I get excited about the capabilities of Tinderbox, then I come across a minor problem like exporting the contents of a summary table and all my enthusiasm deflates. A simple task like this shouldn’t need to have a multi step solution. I will need to keep a special file on this solution so I can remember all the steps next time I need to export a summary table again. That’s probably harder than I really want to work to maintain Tinderbox as a key part of my day to day workflow.

I greatly appreciate your quick response and the thorough and clear way you answered my question. You enabled me to fix something that would have taken hours to resolve by hand.

Thanks again.

[mwra (Mark Anderson)]

Good. I’m glad it helped.

@Paul_Zahara

A simple task like this

Simplicity is in the eye of the beholder. I’ve been helping in this forum and previous ones for c.15 years and this has never come up. Container summary tables are in-app [sic] displays. The simplicity you cite is the fact that in the minds eye we can ‘easily’ see the letters/numbers on screen as a table elsewhere. Don’t worry, this isn’t push-back. I’m just a fellow user, albeit with more Tinderbox experience, I too have to figure stuff out.

The solution I put in the demo is easily imported into other docs as it is those two templates doing the work. But, if you are going to be doing this a lot and want some sort of table export function or copy mechanism, send an email with a feature request to info@eastgate.com explaining your task and asking if it can be made easier.

[Paul_Zahara]

Mark I see the number of responses you have made on this forum and I’m truly impressed at the effort you put into helping people, and with the quality of answers you provide.

I grabbed onto the summary table as it seemed like the best way to get the information I needed, but maybe there is a better way that I missed. If this is the first time this issue has arisen, then people probably have a better solution to the underlying task.

My fundamental objective was to collect the contents of a couple of specific user properties that I had defined, and to export the values of those user properties. Is there a better/more elegant way to achieve this?

Thanks again
Paul

[mwra (Mark Anderson)]

Thanks. I totally get the frustration of hitting a brick wall on something that intuitively feels simple. To your question

@Paul_Zahara

My fundamental objective was to collect the contents of a couple of specific user properties that I had defined, and to export the values of those user properties. Is there a better/more elegant way to achieve this?

If you want to work primarily within Tinderbox, the map is a good place to start, with affordances like the container tables and plots. It should be said that these are not designed for heavyweight analysis. For that, you’ll want to export data.

If you are going to be exporting a lot, and especially if exporting lots of notes at one, you’ll want to get familiar with Outline view.

Some folk only tend to use one view type exclusively, which is fine, but there’s conversely no downside to using different views for different tasks. Two useful points to note. Different tabs can have different scope (part of the overall doc) and/or different selections, i.e. Tab a can have a different selection to Tab B. ‘Background’ tabs, the ones not currently being viewed, are no overhead (there is no rendering going on).

Although the note $Text space RTF does allow tables in $Text these are not, as far as I know, programmable. So, the latter are not suitable for your needs.

@Paul_Zahara

I grabbed onto the summary table as it seemed like the best way to get the information I needed, but maybe there is a better way that I missed.

‘Exporting’ via the Preview or HTML text pane tabs. Think of this like a scripted output, leveraging the app’s export mechanism to construct formatted data structures but without having to generate an export file.

However, if you’d find the latter easier than selecting copying from the HTML tab, I can modify the previous demo to show how. So folk hate one approach (no export!) other hate the other (just export!).

The envelop/letter technique is actually very powerful and adaptable. If you only ever summarise the same attributes, you need never alter them. You can copy them to your app support templates folder and then use them with any TBX file have open.

If you need to use different attributes, I can explain how where to do that. There isn’t a generic tutorial for this as most who need it, need it as the point where a generalised solution is too far from their current understanding to be useful.

So, what sort of summaries do you need?

• do you need column headings
• do the column attributes change with each use?
• do you need rich text, plain text, some other formatting for use pasting into the target app
• if creating a data table (as opposed to text looking like a table, to you want comma-separated (CSV), tab-delimited (TSV), or some other formatting.

These are all things the computer can’t guess (OK some could be if copying on screen data), and so it helps to indicate what you want and then we can find you an easier solution.

Many things initially seem hard because people are applying logic used in other apps. It’s often just as inflexible (or over-flexible) but doesn’t seem so as we are used to a particular method which thus seems the ‘norm’.

Tinderbox is a toolbox for notes, meaning ‘some assembly required’ but most of which you’ll then never touch again (and which can be copied to other files). As importantly, it is much easier to revise or improve your solution should the need arise.

[Paul_Zahara]

Mark
This conversation has been incredibly useful. I agree with your point that each user brings their own preconceived approach to solving problems based on their experience and expertise. A change of view point to suit the new tool is extremely valuable, and you have certainly opened my eyes to new ways to use Tinderbox.

My starting point with Tinderbox was as a note taking app. As I gained a bit of experience with Tinderbox I began to see the power buried under the hood, and my thought process was that user properties gave me the opportunity to attach metadata to each note. Tinderbox has a great toolbox for using that metadata to sort, filter, slice and dice notes, based on the metadata.

My next thought was to use the metadata as a valuable resource in itself. In the example you worked through for me, the YouTubeURL’s were the relevant data I needed on a specific day. On another day I might need the total time of all the videos. So, to answer your direct question “So, what sort of summaries do you need?” I think it would be useful to be able to generate a table with the note name in the first column and then append one or more subsequent columns with a header containing a user property name, with the value of the user property in the “cell” for that note.

I don’t really care about the specific export technique so long as I have a table I can insert into another document (Scrivener, Word, Excel etc) so a CSV format would probably be best.

In my mind the ability to work with the data in the note, plus the metadata in the user properties (and built in properties) makes Tinderbox a unique way to manage complex projects with layers of information on each note.

Thanks for listening.

Paul

[mwra (Mark Anderson)]

Thanks that’s useful. I’ll deal with re-usability first. The upside of not exporting is you have less work to do (setting file extension, etc.) so unless you need an exported file, I’d stick with the method above.

Re-using the templates in another file. Use app Help menu → Reveal Support Folder in Finder, which opens a Finder window on ~/Library/Application Support/Tinderbox/templates. If the templates folder (case-sensitive name) is missing, create it. Make two plain text files with the same name as the two templates in the above demo: I use a '.txt. file extension as it reminds me I’m using a library template and not a built-in template. Once in the support folder, these images now appear in the bottom section of the File menu → Built-in Templates menu, below a ruled line. The templates can now be added to any new document via this menu, also creating the necessary prototypes/containers. These templates import using the saved filename (including the file extension) but you can rename them if desired once in your TBX doc.

You can also copy-paste templates between open docs but for continual re-use, the above probably suits you better.

Altering the exported table. OK for this you will have to edit the existing templates and I think i’ve a kludge for that. New post in a bit…

[mwra (Mark Anderson)]

Try this updated version of the earlier demo: Container-table-demo.tbx (108.4 KB)

Changes. I’ve now made pairs of templates for both TSV and CSV (CSV can be a pain if you’ve literal commas or straight double quotes in your data). Each now has either $TableHeading or $TableExpression exposed as a Key Attribute. There are a new series of stamps:

• Populate stamps. Select you container with the summary table. This stamp pastes the container’s $TableHeading or $TableExpression to the 4 templates as appropriate.
• Set CSV Headings. With a heading template selected, this takes the template’s $TableHeading data and creates the correct CSV headings as the template (i.e. its $Text).
• Set CSV Rows. With an item template selected, this takes the template’s $TableExpression data and creates the correct CSV row cell data as the template (i.e. its $Text).
• Set TSV Headings. With a heading template selected, this takes the template’s $TableHeading data and creates the correct TSV headings as the template (i.e. its $Text).
• Set TSV Rows. With an item template selected, this takes the template’s $TableExpression data and creates the correct TSV row cell data as the template (i.e. its $Text).
• Clear templates. This undoes the first stamp, i.e. so you’re not storing unneeded data between sessions.

Of course if you only ever use CSV or only TSV, you could make the first stamp and the two heading/row stamps into one. However, I’ve done it this way in the demo to help folk understand the process.

Assumptions:

• The stamps assume the template names/paths as used in the demo file. If you alter either you will need to review and update the stamps accordingly.
• If your $TableExpressions use more complex per-cell action code you should double check the ‘item’ template code result and correct the code accordingly.

Unlike templates, the is not (yet) an app support folders for Stamps, but you can drag-drop from one doc to another or save out to Finder. To drag between files drag from the Stamp Inspector in the source document onto the view pane of the target document. To add a stamp from Finder, draw the stamp file onto the recipient doc’s view pane.

[Paul_Zahara]

Hi Mark

I’ve followed your steps through very carefully and I now think I understand how you approached the problem. Thanks for the lesson and time you took on this.

If you don’t mind I’d like to get your thoughts on some of the more philosophical issues this problem exposed. I’m wondering if I don’t get the whole philosophy of Tinderbox. I’ve not read the book “The Tinderbox Way” so I just bought it, but in the meantime can you please help me understand if I’ve missed something.

To me, one of the key defining features of Tinderbox that sets it apart from the zillion other note taking applications is its ability to assign metada to a note. A lot of metadata if you want. I think its great that the user has so much control over the type, amount and form of the metadata. Given this great resource why aren’t the tools and commands focussed on helping the user to work with the metadata?

Your solution to my problem is effective but it’s certainly not obvious. So, my issue is “Am I trying to coerce Tinderbox to do something it doesn’t want to do because of my user mindset, or is there some other reason that Tinderbox doesn’t play nicely with its own metadata?”

I know you are not the author of Tinderbox but I think my intention is to submit this whole thread as a “feature request” and before I do I want to be sure I’m not missing something obvious.

Thanks again.
Paul

[eastgate]

I don’t want to preempt Mark Anderson, to whom this question is addressed. I’ll be intertested to learn his thinking.

Still, this is (as you say) my fault, not his. I think, though, that you’ve hit on an edge case – and interesting edge case that is obvious to you, but that wasn’t obvious in the design.

Summary Tables were originally an improvised expedient to help expose some metadata in map view. They’re a visualization tool. It’s certainly not unreasonable to want to export them, copy them, and so forth. But that’s not what we were thinking: what we wanted to do is to show some of the metadata inside the container.

HTML Export was intended as the main export path. It’s flexible and powerful. When it was first designed, hardly anyone knew HTML but we all confidently expected that everyone in 2020 would know HTML, just as we assume everyone knows Google and Twitter. That world turns out not to be the world we got; it might have been nice had we got it, but we didn’t.

Tabular data exchange among programs is at present a mess. That’s not our fault, it’s Microsoft’s. But again, this is the world we have.

Summary Tables are not as popular as I would have thought: or, at any rate, that’s my impression. Your scenario is new, but it’s significant.

[Paul_Zahara]

Thanks for jumping into the conversation. I’m interested that you see this as an edge case. Obviously my experience as a casual user ( now looking to dive a bit deeper in) is different to your established user base. If the ability to layer data and metadata is not an important functionality in Tinderbox for most users then I’m missing the point of differentiation that Tinderbox offers.

I love that I can map my ideas in map view, but there are lots of mind mapping applications out there. Similarly I love that I can outline ideas in outline view, but again there are lots of outliners. The ability to join up the functionality of note taking, outlining and mapping was what convinced me to buy Tinderbox in the first place.

Later on I discovered the “under the hood” capabilities of properties and the ability to manage my data with those properties. At that point I thought “Great, I don’t need an outliner, I don’t need a mind mapping app and I can use Tinderbox to store information and to manage the projects that information belongs to.” I figured that was the reason some people raved about Tinderbox, but now it turns out I’ve misunderstood again.

Summary Tables are a cool feature but what surprises me is that I am the first person to ask for the ability to export them. If the ability to layer data and metadata into a coherent package was a happy byproduct of your design, then I need to read your book to understand what your intentions were. I am looking forward to getting to the bottom of this!

[PaulWalters]

@Paul_Zahara

Obviously my experience as a casual user ( now looking to dive a bit deeper in) is different to your established user base. If the ability to layer data and metadata is not an important functionality in Tinderbox for most users then I’m missing the point of differentiation that Tinderbox offers.

Don’t be harsh on yourself. There is no “received wisdom” with regard to “the point” of Tinderbox. Tinderbox is a vastly liberal tool. A case where metadata (attributes of a note) are the center of focus in a note (which I believe is your understanding), or a case where a note’s text has centrality and metadata is a completment to the text. Or a case where links of all sorts are central, or a case where links are a complement to the metadata, or any other case. These are all abstractions, of a sort, that one can use Tinderbox to instantiate. No use case is “the right way”. Nothing is required.

For some, Tinderbox maps are the whole point. For others, not at all. Neither aspect is better or worse or intended or not intended or recommended or not recommended or standard or not standard. Same with your take on Tinderbox versus anyone else’s. Many voices have been reflected in the software in the course of Tinderbox’s evolution. As a result, some of the software is very opinionated for whatever historical reasons came up in the past.

Please give it more time – you are not on the wrong track, and perhaps Tinderbox might need to become more opinionated in your direction. That’s @eastgates’s decision, of course, but one thing I’ve learned over time about Tinderbox is “if I can imagine it, Tinderbox can probably be tuned to do it”.

[mwra (Mark Anderson)]

Sorry for radio silence today, I’ve been busy in the vegetable garden all days (veg is on full stream at last) and am now setting up for a quiet anniversary dinner. So, an answer will come tomorrow. Nothing explosive—I have no real quibbles with comments thus far.

To save having to say it later, my views—such as they are—are heavily influenced by assisting in this community for some years. That doesn’t make me smarter. More experienced, perhaps. However, the aspect of such experience that people often miss is that I spend as much time figuring out other people’s problems as I do my own. No praise needed there - I do it because I choose to. But the problems of others are illuminating as one has no personal, emotional investment in it. In the same way, maintaining aTbRef has both been a schooling in the practicalities of hypertext structure and a lesson in humility. The resource is never finished, and to my sadness occasionally plain wrong. I’m often halfway through writing a comment along the lines of “You misunderstand…” only to realise I do too!

Proper answer tomorrow. Have a great day.