Simple Pandoc Template

The goal is simply to provide a way to use Pandoc in Tinderbox. In this template, Pandoc is relevant for two different tasks:

  • Preview the Markdown text making use of Pandoc’s Citeproc
  • Export the text of the selected notes to files — to a single or to multiple files — in different formats (md, epub, pdf, latex, opml, wikicode, pptx, etc.)

In order to do this, two things are needed: the TbxConfig note and an export stamp.

With the stamp you can…

  • Export each one of the selected notes with or without their children (as part of each file or as part of multiple individual files). (@satikusala, this should address the issue raised in this topic).

From the TbxConfig note you can…

  • Update the preview command in case you decide to make changes to the configuration (click on the checkbox; it will quickly go back to being unchecked and the $HTMLPreviewCommand will have been updated).
  • Choose your export folder.
  • Point to your bibliography file*.
  • Choose your bibliography style.
  • Choose the Pandoc’s conversion templates.
  • Choose the converted file’s extension.
  • Activate/deactivate conversion options (e.g. table of contents, citations as links to corresponding bibliographical reference).
  • Choose the fonts for the preview and Export (only for pdf conversion).
  • Choose from different CSS styles available and that will be automatically used in Preview.
  • Include children, if the note has any, when exporting the selected notes.
  • Choose to export the children of the selected notes as individual files

This template assumes that all of the writing will be done using Markdown. In case you want to use something different, you my need to make a few adjustments.

Latest version will always be in this Github repository.

* If you never used Pandocs bibliography feature, consider giving it a try. It is both useful and easy to use. Especially if you are using Bibdesk.


Updates

2021-05-01: The new version doesn’t require any script to be downloaded or set up. It’s all there already.

  1. Download the file
  2. Go to TbxConfig to set up…
    • the path to the bibliography file
    • the path to the template
    • OR leave empty if you are not using any bibliography file or templates.
  • To update the HTMLPreviewCommand just click on the checkbox in UpdatePreviewNow.
  • To export a note, just select it and run the stamp.

Simple Pandoc Template 02.tbx (488.2 KB)

8 Likes

Thank you, Bernardo! This looks most interesting and useful. I look forward to trying it out.

1 Like

Yes, I love what you’ve done here.

All, @Bernard-0 has developed a comprehensive approach for processing Tinderbox notes through Pandoc. As noted, his attached file is primarily set up for processing markdown (.md) files. I can attest, however, that it is simple enough to also process .html files using pandoc. Also, PLEASE don’t let all the configuration options in the file intimidate you (it certainly intimidated me when I first look at it). Pandoc is extremely powerful, you can do a lot with it. However, all the options may create a paradox of choice, and with too much choice you may end up doing nothing.

To this end, I’d like to add to this thread a barebones TBX to Pandoc file/process. Here are the steps to use it.

Besure to INSTALL PANDOC

First off, huge thanks to @rtalexander who taught me all this.

  1. Launch your terminal, use the built-in one with your mac found in Utilities or install https://iterm2.com/ which is a VERY nice enhanced terminal
  2. Install Homebrew, https://brew.sh/. Homebrew is a shell to help run command-line apps on your Mac.
  3. Install Pandoc, once you have Homebrew installed type this in your terminal “brew install pandoc”. Homebrew will automatically go out and install the latest version of pandoc. NOTE: this same step can be used to update pandoc too.

#TEST PANDOC
Now that you have pandoc ready to go you can test it out.

  1. Create a pandoc scratchpad directory for yourself, a place to process your files. The easiest way to do this is from the Finder (you can do it from the terminal tool, simply type “mktdir pandocSP” at the root level). I put my at the home directory of my Mac using finder. the path to this directory is /users/mysernmae/pandocSP/.
    image
  2. Make sure you’re in your new directory, type pwd to see what dir your in. Type “cd pandocSP” if you need to move yourself to your new directory.
  3. Test processing a sample file in pandoc.
    Here are two sample files you can try, one markdown file and one HTML file. Download and uncompress these files in your pandocSP directory.

PandocTestFiles.zip (45.7 KB)

Unzip this file and put the files in your pandocSP dir.

a. Convert markdown to PowerPoint
In the command-line type: “pandoc -s PRESTEST.md -o PRESTEST.pptx”

This will create a PPTX file.

b. Convert HTML to DOCX
In the command-line type: “pandoc -s SAMPLEFILE.html -o SAMPLEFILE.docx”

c. Covnert HTML to PDF*
If you want to do this, you first need to install basictex, from your command line, type brew install basictex and then follow the prompts. Once it is installed you can type in the In the command-line type: “pandoc -s SAMPLEFILE.html -o SAMPLEFILE.pdf”.

*Note: converting the PDF is taking a two-step process. Need to try to debug this with @Bernard-0 or @rtalexander

The steps above will convert the the files. the “-s” flag is telling pandoc to create a standalone file. The “-0” flag is telling pandoc to create and output files, e.g. “.pptx” or “.docx” or “.pdf”.

image

You see? Pandoc converted the two files with the above commands.

Going through the steps above makes you a command-line user. Congrats!!! :tada::partying_face:

#Now Use Tinderbox to Do It
Your next step is to have Tinderbox pass the pandoc flags and file names to pandoc. You do this using a stamp that will 1) create a file in your dir, 2) paste your notes into this file, 3) construct the files names and pandoc run command, and 4) run the run command.

Here is the stamp:

var theExportedFile;
var theText;
var theCovertedFile;
theExportedFile=$exportDir("TBXConfigNote")+$ExportFileName+$fileType;
theCovertedFile=$exportDir("TBXConfigNote")+$ExportFileName+$convertToType;
$theConversionCommand=$pandocDir("TBXConfigNote")+"pandoc -s "+theExportedFile+" -o "+ theCovertedFile;
theText=exportedString(this,$HTMLExportTemplate);
runCommand("pbcopy ",theText);
runCommand("touch " + theExportedFile); 
runCommand("pbpaste > " + theExportedFile);
runCommand($theConversionCommand);

The pbcopy command line copys the text of your notes to the clipboard.
The touch command line creates a file on your Mac.
The pbpaste command line pastes the clipboard to the newly created file.
The conversation command line runs pandoc.

Here is a sample TBX and sample files you can play with to make this work.

PandocTestFiles.zip (45.7 KB)
TBX L _ Barefones TBX and Pandoc Example.tbx (111.5 KB)

The only difference between this file and @Bernard-0’s is that I’ve removed all the niceties, e.g. changing of fonts, templates, using the bibliography etc. Tinderbox and Pandoc together, as @bernard demonstrates, can do some amazing things together. To enhance the pandoc conversion you simply need to pass in the command line additional refinement flags to enhance the standalone file and output file, like --toc which adds a table fo contents to your word doc. enhance the Tinderbox side you can add file name version control attributes, have it remove the source files, and a whole bunch of other things (will save this easter eggs for later).

NOTE: When working with markdown you need to make sure there is at least one space after each bullet on your slide notes and one space before the speaker notes.

4 Likes

@satikusala, thank you very much for bringing up all these important details that I left out. It also reminded me that I left out a crucial step: for the export stamp to work, one has to download the applescript and point the correct location at the TbxConfig note.

Thanks. Download from where? I don’t see it on your github? What does the script do?

I added it to the first post :wink:

Oh, I see, it is the script to export children individually. It is in your TBX. Missed that. Thanks.

1 Like

Hi! Thanks @Bernard-0 , the template you provided seems to be exactly what I was looking for, the easy way to export my notes to markdown files whenever I want. BUT as I am not a programmer and my skills in computer science field are very limited, I don’t really know how to make your template work :confused: . Thanks to @satikusala guide (BTW, I love your TBX tutorials!) I managed to install home-brew and Pandoc but I’ve got lost after copying stamp to my tinderbox project… Would anyone be so kind to explain in plain English what next steps I should take to make it work? What exactly I should do with provided template to export my individual notes into markdown files?
I’m new to Tinderbox (using it merely for about 2-3 months) and still struggling with its more complex functions, but seeing its potential keeps me motivated to learn more about it :slight_smile:
Thanks! :blush:

2 Likes

These are great! Thanks for the great explanations!

@ShiJianhui if you’re writing in “rich text” in Tinderbox and just looking for an easy way to export to Markdown files (and don’t currently need the other Pandoc options nicely demonstrated in this thread) then you may find it convenient to use the short AppleScript posted and explained here .

Script repeated here (improved by ideas from this thread, e.g. "borrowed" idea to use `touch`)
-- Have an export folder ready. Select Tinderbox notes and run. 
-- Assumes Pandoc is installed at /usr/local/bin/. See https://pandoc.org/installing.html
-- NB: overwrites any like-named .md files in that folder

set prefix to "@" -- character(s) used to distinguish internal links from external ones

set pandocCmd to "/usr/local/bin/pandoc -f html -t markdown_mmd" -- html to MultiMarkdown
set sedCmd to "sed -E 's/" & prefix & "(\\[.+\\]).+\\)/[\\1]/g;t'" -- grab anchor, surround by [[ ]]
set cmdStr to pandocCmd & " | " & sedCmd -- assemble the "pipe"

set theFolder to (choose folder with prompt "Choose a folder to receive the exported MD files")

tell front document of application "Tinderbox 8"
	repeat with aNote in selections
		tell aNote
			set theFilePath to POSIX path of theFolder & (value of attribute "Name") & ".md" -- name file after note
			set theHTML to evaluate with "exportedString(this,$HTMLExportTemplate)"
			set theMMD to do shell script "echo " & quoted form of theHTML & " | " & cmdStr
			do shell script "touch " & quoted form of theFilePath -- create file if doesn't exist
			do shell script "echo " & quoted form of theMMD & "> " & quoted form of theFilePath -- write to file
		end tell
	end repeat
end tell

Virtually no assembly is required within Tinderbox. Add the default export template at File > Built-in Templates > HTML, then select Tinderbox notes you want, and click run.

3 Likes

@sumnerg Thanks a lot, the solution you provided is perfect for my needs! :relaxed:

1 Like

There is a key insight in this thread: there are many ways to export (i.e. publish) your data out of Tinderbox.

As with anything your output is dependent on your input, and both have a relationship with your short-term along-term intentions and goals. Are you simply taking notes? Are you writing short-form content? Writing long-form content? Do you need stylized content? In preview or just for ouput? Outlines? Is your work academic in nature, and need footnotes and references? Do you need map visuals, or will the outline suffice? Are you developing a solution of interrelated notes, like an industry map, where links and associations are important, as well as attributes to help manage the relationship between notes. All the choices you make in to address these work efforts will influence how you use Tinderbox. The beauty of Tinderbox, however, is in its flexibility. You can go down one path and simply pivot to another with relative ease.

You can use basic export templates to publish your work

If you’re looking to get markdown out, you don’t need an AppleScript. If you want MD to HTML then apply the HTML template to your MD note and Tinderbox will conver it. If you want MD to MD then apply a text template for your export*.

You can mix templates.

You can enhance templates with action code to transform your content on the fly.

You can work with utilities like Pandoc and Applescript to auotmagically transform your data.

Again, what you do depends on your immediate and long-term needs and your general skill level with Tinderbox and other utilities, like Applescript.

*##Example of exporting markdown out of tinderbox
See the attached example.

  1. Witere you note in Markdown
  2. Apply the markdown prototype
  3. Apply a template that only has # ^value($Name)^ and ^value($Text)^ in it (see tMarkdown in the attahced).

TBX L - Working with Markdown in Tinderbox.tbx (96.5 KB)

If you want to export your markdown out as HTML, then apply the HTML template to your note.

3 Likes

Basic export templates easily handle Markdown → HTML, and Markdown → Markdown. Thanks for the clarifying examples.

But I’ve found RTF → Markdown (writing “normally” in Tinderbox then exporting to Markdown files) more complicated. You can get easily from RTF → HTML (a traditional strength of Tinderbox). But then you have to go through Pandoc to get to Markdown, and use regex to “fix” note-to-note text links that are rendered with an .html extension even if you specify .md.

A short AppleScript currently seems to be the most efficient way to get from RTF to multiple individual Markdown files, especially if you don’t want to export the entire Tinderbox document. A bonus is that this will work with other Tinderbox documents too. You don’t have to configure each new document.

I looking forward to working through and trying to absorb all the great material upthread. Thanks!

3 Likes

This can be done, but nice built-in support is coming.

7 Likes

Thanks @satikusala for this explanation! It let me have a better understanding of how Tinderbox works! The solution @sumnerg provided is the best for my needs i.e. “writing “normally” in Tinderbox then exporting to Markdown files”. I like to have notes written in RTF, but sometimes when I’m not on my desktop, I want to be able to access and edit my notes with an iPad.

You bet! You can do RTF to Markdown too, but it is a bit more tedious. As noted above, it looks like @eastgate will be a better solution for us in TBX 9.

2 Likes

Just updated the template. Now there is practically no setup to do. Just download and enjoy.

1 Like

Thank you Bernard-o. Is the Github version also updated?

Tom

Not yet! I will update it shortly. :wink:

EDIT: done.

1 Like

Very cool…you’ve got a lot of cool stuff going on here wit RegEx. :slight_smile:
BTW, question for you, do you have any idea on how to set the PDF export in pandoc to create a one-page PDF, like DevonThink can do? I want to be able to create a PDF of exported TBX files without there being a page break in the middle of the PDF file.

Hey Michel, the conversion to PDF in Pandoc can happen by two different mechanisms: HTML and LaTeX. So it all depends on what you are trying to accomplish and by what means. If you are trying with LaTeX, perhaps you could try changing the document class:

-V documentclass:report

or

-V documentclass:article

For more info, see this and this.

And you could remove headers and footers:

-V pagestyle:empty

Latex can be a pain if you still don’t know your way around it, but this could get you started. HTML could be easier, of course, since you know your way around it. Just check Pandoc’s documentation for more info on this method (I never use it).