Exporting to JSON - two problems, with suggested answers, and a question

Exporting from Tinderbox
1

Given a note with immediate children, I want to export that note in JSON format to facilitate its import into an application written in Javascript. Say there are three children, with titles: “Title 1”, “Title 2”, “Title 3”, and multiline text, starting Text 1; then the JSON output might be:

{ "notes": [ 
   {"title": "Title 1",  "text": "Text 1....."},
   {"title": "Title 2",  "text": "Text 2....."},
   {"title": "Title 3",  "text": "Text 3....."}
] }

The first problem: removing the linefeeds from the multiline text values.
With a naive template, e.g.

{ 
   "title": "^title^",
   "text": "^text^"
}

the ^text^ for text includes linefeeds that then appear in the JSON source, and make it invalid.
My fix for this was to use ^value to evaluate a replace call on the $Text attribute, i.e.:

{ 
   "title": "^title^",
   "text": "^value($Text.replace("\n","\\n"))^" 
}

This could be generalised, as it happens for the source text I’m working considering linefeeds is sufficient.

Note: I’ve switched off the MarkupText and QuoteHTML export options for the notes I’m exporting to JSON.

The second problem: each child note in the JSON list needs a comma separator, except the last.

That is I need all but the last child note to end in a ‘,’ to separate it from the next note in the list.
The parent note export template is:

{ "notes": [^children(/templates/json_note)] }

The way I got the comma to only appear between child notes was to add a conditional at the end of the
json_note template:

{ "title": "^title^",
  "text": "^value($Text.replace("\n","\\n"))^",
} ^if(^nextSibling!=^lastSibling),^endif

Which includes a comma unless the next sibling note (of the current child note) is not also the last.
This should get me most of the way there. The last child in the list won’t have a ‘,’ appended (which is what I wanted)
but the penultimate child will also miss a comma - disaster. Only no, this template in practice works in so far as it outputs the commas between every child, but not after the last. I can’t see why this is.

If I change the template to:

{ "title": "^title^",
   "text": "^value($Text.replace("\n","\\n"))^",
   "next": "^nextSibling",
   "last": "^lastSibling"
 } ^if(^nextSibling!=^lastSibling), ^endif

I get:

{ "title": "Introduction - p1",
  "text": "some text...",
  "next": "data/Introduction_-_p2.html",
  "last": "data/Introduction_-_p3.html"
} ,

{ "title": "Introduction - p2",
  "text": "more text...",
  "next": "data/Introduction_-_p3.html",
  "last": "data/Introduction_-_p3.html"
} ,

{ "title": "Introduction - p3",
  "text": "blah blah...",
  "next": "data/Introduction_-_p3.html",
  "last": "data/Introduction_-_p3.html"
}

It’s a puzzle… I’d really like to understand what is going on.

The question: In general, what is best practice for exporting to JSON from Tinderbox?

HTML export is well catered for, and the documentation suggests that it generalises for JSON export too.
For creating a document that will be the final output, like a web page, HTML is a reasonable choice, but for
output that will feed into another system JSON is (I suggest) better.

2

It’s some while back, but IIRC I’m probably responsible for asking for a number of the JSON-targetted features that are now in the app such as jsonEncode(). Although the JSON spec is fairly clear, my experience is the workflow (apps, OSs, networks) through which the data passes have as much bearing hare as the headline formate: crossing some boundaries can have unexpected and generally invisible effects until the last step fails.

IIRC, your second point is a request somewhere on the ‘already asked’ spike. There is a way around, which might seem counter-intuitive form the context of your current templates. Make a list (assuming it is not huge amounts of data) and then use List.format() to join the elements as that puts the delimiter (in this case the string ",\n") only between list items.

However, I’m definitely in the ‘+1’ camp for an export code look-ahead method to suppress an unwanted trailing join as in the JSON case as these sorts of tasks otherwise work well in the envelope-letter recursing from of export. As export of an item maybe conditional it is often not enough to try and test (via user code) if a ‘next’ item exists. therefore it would be useful to be able to ask the app to do the test when it has the data. IOW, keep adding this unless it is the last item. If necessary, the app could add ad normal and then, on detecting ‘list’ end, step back and remove the last-added delimiter string. I guess, given the List.format approach, the trade-oof is the engineering cost of the new feature against the likely (few) number of us who might use it. FWIW, when I was doing a lot more JSON export I recall I just post-processed on the command line to clip unwanted trailing commas.

3

Thanks Mark :slight_smile:

I’ve updated the template to export a note to JSON, first to use the jsonEncode() operator you suggested.
Secondly I’ve changed the condition for deciding if a join between two notes is needed to something that not only seems to work in practice, but also looks right in theory. Here is the new version:

{
 "id": "^value($ID)",
 "title": "^title^",
  "text": "^value(jsonEncode($Text))^"
} ^if(^value(!last("$Path(parent)", 1))^), ^endif

I’m using the last() operator. The second argument defaults to 1, so it’s superfluous in this template. This version also includes an id, which I needed because I’m using the exported JSON with the reactjs JavaScript library; I expect other JS libraries would also need some unique ID for each note.

1 Like
4

Thanks for the positive follow-up (useful for others following your path). last() is a neat touch. Be aware its is talking about actual children of the original (by default anywhere). Things can get more complex when exporting via an agent or such. IOW, just something to bear in mind if your exports bet more complex. As it is looks like as is well, which is a good result.

6

I can’t get “Tags”: [“reseña”, “política”, “lingüística”].
Not with "Tags": [^value($Tags.format(", "))], nor when I apply, escaping the quotation mark with \:
List/Set.format("listPrefix","itemPrefix","itemSuffix","listSuffix")

Any advice on what code expression I should be looking at?

7

The is no specific method for formatting lists to JSON format. But, using your example, this code (in an export template) creates the desired output effect:

"Tags" : ["^value($Tags.format('","'))^"]

Note that all the quotes outside ^value()^ are treated as string literals. Within that code double-quotes would normally be used as string delimiters. But, as we want to use double quotes in the output form the export code, we need to use single quotes instead. Note that the opening and closing sets of double-quotes are supplied outside the export code as .format() doesn’t have a method to only provide a start-of-list,item-join and list-end strings.

In an action, as opposed to an export template, I’d create the same output text this way:

'"Tags": ["'+$Tags.replace(';','", "')+'"]';

Tested in v7.3.1.

1 Like
8

I take good note of the use of single quotes. I understand the elements of both your expressions, but not why one is for an action and the other for an export template. I’ll read about it on my own.
Thank you for your help and enjoy a happy new year.

9

The primary difference is export codes (e.g. ^value()^ are no longer used in action code). Also whereas an evaluated export code’s output is simply inserted into the template, an action needs to concatenate (join together) the literate strings with the output of the action codes.

On reflection you can use the .replace() or .format() interchangeably as long as you’re careful about your use of single/double quotes.

Sidenote, a quote - single or double - is a character you can’t escape in action code by using a backslash.

1 Like