Exporting a JSON

Hi -

Some perhaps naive questions…I’m trying to create a json output file from a subtree of notes. Following some examples found in the forum I’ve been able to get most of the way there but there are a few things I’m struggling with…

  1. Some of my notes are nested within containers. I’m not sure how to preserve the hierarchy of the notes as I export to JSON? If I use something like ^Descendents, I can get a flattened list of all the notes but I lose the nesting…

  2. Some of my attributes are sets or dictionaries. Is there an easy way to transform from Tinderbox’s notation to the JSON notation? I did a bit of experimenting with the .each operator but didn’t have much luck…

Any thoughts and examples would be appreciated…

Thanks…

Lots of contextual questions which would be most easily solved if you could upload a small TBX with the export templates you’re using and content that shows the problematic structure (and user attributes as necessary).

Are you using the ‘envelope and letter’ technique for your templates? If not you should give that a try. That way, the inner ‘letter’ template simply recursively keeps calling itself all the way down the outline.

One thing to watch for is trailing commas at the end of array or object lists. The easiest way is to add such an ending to all items then trim the tailing comma (or comma & space) once the list is complete.

Thanks Mark,

Here is a sample file (building a glossary of terms for sustainability reporting). I think I am using the 'envelope and letter ’ technique - but perhaps incorrectly (or at least not to its potential). A couple of points to look at in the file.

A) the attribute “synonyms” is a dictionary where each name:value pair is a term:strength.
B) the attribute “contexts” is a set of one or more named contexts (just strings).
C) I created JSON GlossaryList and JSON GlossaryItems templates.

Thanks,

dan
12-3 JSON Subset.tbx (322.5 KB)

2 Likes

Big :hearts: for someone actually, at last, using Export pane rather than Preview (doc opens in text/Export pane) to check export. Seriously, doesn’t happen enough. Preview shows that export worked/failed, but Export tells you why.

1 Like

shouldn’t the JSON array instead of looking like this:

"Contexts": ["Sustainability,Test" ]

look like this:

"Contexts": ["Sustainability","Test" ]

with a small change in the template:

"Contexts": ["^value($contexts.format('\",\"') )^" ]

And the last one (Synonyms) has a { at the start and a ] at the end:
"Synonyms": {"^value($synonyms.format(","))^"]

This should work with the Synonyms:
“Synonyms”: ["^action($MyString=$synonyms.format(',\"');$MyString=$MyString.replace(':','\":');)^^value($MyString)^]

returns

"Synonyms": ["foo":50,"goo":100,"moo":25]

2 Likes

Detlef,

Thank you very much! As a relative newbie, I think what was confusing me is the behavior of the set or dictionary - when is it treated as a single string and when is it treated as a vector (or array) that you can operate on…

Between Mark’s suggestion to handle the nesting and your response I think I have a workable JSON! Thank you all very much!

Now the next naive question…is their a recommended way to call a RESTful interface from within Tinderbox or do folk use JavaScript external to Tinderbox? Just starting to explore options…

Thanks!

dan

If the RESTful interface uses GET, the built-in AutoFetch facility does what you want. Construct the $URL, auto fetch it into the text of a note, and then parse that.

If you need something more complex — POST, OAuth, whatever — then run up a script in whatever language you prefer, or just call curl.

Viewed as a string, a List, a Set or a Dictionary are all semi-colon delimited lists of items. A Dictionary differs in that the list items are key:value colon delimited pairs of strings rather than a single string value.

There isn’t a list-to-JSON operator but the new action code user-defined functions (see video of today’s meet-up) arriving in v9.1.0 should make that an easier process.

I’m still chipping away at your TBX. As seen in your kind TBX example you definitely aren’t using the envelope-letter process, as will be apparent when I’m done. I’m mainly trying to remind myself of the more arcane edge-cases of JSON formats and nesting whilst trying to get JSON valid nesting to work. Currently ending up with to many or too few parent object closures. but i’ll get there.

1 Like

Well, that was interesting… and for unexpected reasons. It turns out that just pasting any old stuff from styled web-pages and then exporting to JSON is … problematic. So, clean your data before use (more on this below). Also, if sharing file, if you like to use valid by non-standard attribute names (e.g. examples vs. Examples) it is a good idea to to state that up front. I got really confused in debugging as it looked like the attributes were incorrectly defined.

A(n probably incomplete) list of work done to get your data to work:

  • added envelope-letter style of export with the “JSON GlossaryList” envelope template for the root export note and the recursing “JSON GlossaryItems” letter template for all other exporting notes.
  • I added level-respecting indenting for the code to aid reading of the exported JSON. Doesn’t affect parsers but very helpful to human readers (e.g. for de-bugging).
  • All $Text had to be carefully scrubbed of all sorts of rubbish formatting (I’m guessing pasted in from a web page) that was confusing jsonEncode() no end.
  • moved content notes out of the /Templates container and descendants, where they have no place to be (I’m guessing the result of failed experimentation). suffice it to say this isn’t how template relate to exporting notes. These unwanted notes are saved in /bin
    in my test doc
  • Valid JSON doesn’t seem to like empty exports or trailing line-break/whitespace. Clean your $Text before starting to do formatted export of this type. Top tip: unless you are sure you want formatting, never paste content from other sources using Cmd+V, but instead is Ctrl+Opt+Cmd+V (paste and match style). The latter saves you later having to strip out dodgy web-derived text styling that likely you never really needed on the first place.
  • I used https://jsonlint.com for JSON validation.
  • Template “JSON GlossaryItems” will make more sense when viewed in a text editor like BBEdit with line-wrapping turned off. The long code lines are needed to deal with correct JSON encoding involving a fair amount of conditional wrappers.

Here is you’re file, emitting valid JSON for container “Glossary Terms”: 12-3 JSON Subset-ed01.tbx (321.6 KB)

Apologies, I forgot to change the TBX back to dark mode. I had to turn it off as it gives me eye-strain and headaches. Not Tinderbox’s, just dark mode in general.

That also reminds me, your templates were incorrectly set up so I fixed the $Text formatting there as well. Remember, unless you are really, really sure you want to paste source formatting, Paste and match style is nearly always a better method to use.

Sorry this solution took so long. It took peeling back several layers of cross-cutting errors which took some amount of de-construction to figure out.

Mark,

Thank you VERY much. There seem a number of things for me to learn from what you have done.

There are some mystery’s I’d like to understand better in your template (and some comments):

a) Not sure I understand the purpose of ^if()^ - is it to demarcate a compound statement?

b) The indentation is great - and I appreciate that you are addressing the real possibility that an attribute may not be present.

c) I was unaware of Ctrl+Opt+Cmd+V (paste and match style) - will try to do this; that said, is there an easy way to strip style from the $text as it is added into Tinderbox?

d) is it possible to get the $ID of a parent (from the child)? That might be another important thing for synchronization.

e) Not quite sure how my templates were incorrectly set up? I must have done something accidental to get notes created under the template - I’ll investigate…

Now on to the next challenge - playing with RESTful interfaces…

^if(query)^…^endIf^ is the export code equivalent if the action code if(query){…}. Same role in both cases. If the query evaluates as true, the delimited code is executed. If If the query evaluates as false, no code is execute. Both export and action code offer an ‘else’ branch.

The generic logic is:
if(query) code to run if query is true / else / code to run if query is false

Added how? Unfortunately the question is too loosely stated to answer clearly. Note that $text refers to the user attribute whilst $Text refers to main body text of a note Attribute names are case sensitive. Indeed, whilst you can use use case you like for attributes (price, Price, PRICE, etc.) it’s a good idea to follow Tinderbox norms especially if you want others to understand your code. At the same time, others value using naming styles more familiar from other apps they use. All said, it’s a personal choice. System attributes must be used in the case defined by the app (not the user)

Yes. Read up on references and designators, especial the ‘parent’ designator.

Some had the wrong prototypes, this ended up with the wrong font style (e.g. you couldn’t see where the insertion cursor was). Your template handling the descendant notes “JSON GlossaryItems” was trying to call ^descendants()^ ( list of descendants) rather than use the envelope-letter mechanism to which I linked in a previous post. As a result it wasn’t working as you envisaged.

Fear not, you aren’t doing anything wrong per se, but having bad luck with your guesses. Pretty much all the questions above are covered in Help, the app’s Help menu tutorials, or aTbRef. Part of the secret is not trying to do too many things at once. For instance, I’d start with things like how to fetch an attribute value from a different note. Then when you need to use that technique in the template it is no longer a mystery.

Although not strictly the case, it’s a good starting assumption to assume all app-defined codes (Attribute’s names, action code operators, export code, designators, etc.) are case-sensitive. That way you won’t inadvertently colour outside the lines—at least not before you chose to doing so because you know where it is safe to do so. :slight_smile:

Thanks Mark,

a) On the fourth line down in the items template it reads:

Blockquote
^if()^^indent(“\t”,$OutlineDepth(parent)+1)^“id”:
Blockquote
So I was wondering about the ^if()^

b) sorry, I meant $Text - especially when the text is pasted in.

c) Ok - I had found the designator - but without experimentation didn’t see that parent.ID would work - I’m afraid I didn’t realize that attributes of a designator could be retrieved via the dot notation.

Thank you - this has definitely been an interesting learning experience…

dan

Ah, classic one-more edit error. sorry. Having figured the main task out, I realised I might as well wrap each per-note attribute in a test that it has a value so non-validating blanks don’t occur. I’d meant the line (and beginning of the next line) to read:

...
^if($ID)^^indent("\t",$OutlineDepth(parent)+1)^"ID": "^value($ID)^", 
^endIf^...

IOW ^if()^^if($ID)^. But in this line the test for an attribute value is pointless as a value for system attribute ‘ID’ is one of the things every object (note, agent, adornment, etc.) must have. FWIW, it’s how two notes in the same map.container can have the same title ($Name) and still work. So either use the above, or if you prefer:

...
^indent("\t",$OutlineDepth(parent)+1)^"ID": "^value($ID)^", 
^if($Name)^...

This per-attribute test includes the command line-break after each (non-last) attribute. Validators won’t mind a lack linebreak but a comma or lack of one when needed will break less forgiving validators. Some JSON validators/parsers are more relaxed about extra/missing commas accepting that most humans don’t code and even those that do make mistakes.

Here is the former of the above version implemented in my earlier file. I was confused as I didn’t know I’d made that error of omission under the questions was asked is a close-ended fashion. Sorry about the confusion! :slight_smile:

File: 12-3 JSON Subset-ed02.tbx (321.9 KB)

Note: yes, I know the trst for a $ID vaiue is un-needed, but I suspect most later readers wanting to re-use the ‘pattern’ of the template might not know that. So the more verbose template , if copied, wont’ tip them into exactly the error that 's hard to spot because it’s “all just code”.

HTH