Tinderbox Forum

Zettelkasten compatibility with Markdown apps

In another thread I asked about how to make a Tinderbox file for a Zettelkasten, which would also export files in plain text Markdown formats that would retain compatibility with other ZK apps, like the Archive or etc.

I have had some moderate success, but it requires some particular ways of working.

First, it’s necessary to make sure to add the date stamp and double bracket wiki link syntax manually to any in-text link that is made. Further, at this point, note-to-note links won’t carry over to the Markdown files. I think changes may need to be made to TBX export before that can be arranged.

Second, it’s possible to use Keyboard Maestro along with Tinderbox’s AppleScript support to automate this linking procedure. If you use this macro, what you do is this:

  1. Be typing in a note where you want to create a link to another note.
  2. Navigate to that target note
  3. Invoke the macro (I use the keyboard shortcut CMD-SHIFT-CTRL-OPT-L)
  4. Macro should take you “back” to the previous note. It may be that you’ve navigated to a few notes looking for the right one, so you don’t exactly get back to the original note. If this is the case, no worries, just select that original note manually.
  5. Press the “CTRL” key (you can change this of course, it’s just the cue to KM to continue with the macro)
  6. Macro should paste the Name of the target note, create a text link, the prepend the line with the date stamp wiki link expected by Markdown ZK apps.

Third, you can now export via HTML export and get markdown files named appropriately. Nice!

I’d love to see other ZK fans tweak this or offer other suggestions. I think this is rich territory.

Here’s a demo Tinderbox.

And here’s the KM macro image:

6 Likes

Thank you. BTW, .tbx files can be dragged directly into postings in the forum, for sharing. No need to use a file sharing services.

@derekvan, I’m late to this conversation, but very interested in experimenting with TBX as my Zettlekasten.

The dropbox links to the TBX seems broken. Would you be willing to repost it?

My fear in using TBX is I don’t want to get stuck if I decide to move to another “slip box.” However, in my attempts to play around it looks like I’m solving it, but I’d be curious to see what you’ve done.

Edit: I was able to download the Keyboard Maestro script

zkn_testv3.tbx (109.6 KB)

Here you go. Sorry about that!

1 Like

It turns out I’m too dumb to fully grok the tinderbox export mechanisms. But this prompted me to follow through on the promise of tinderbox not locking you in via having parsable xml files. Turns out the file format is quite beautiful and parsable! I thus created some code that takes all my tinderbox notes and spits out directories of markdown files, replete with wikilinks. I did this to play with noteplan3 a bit, but the files are equally readable (and links work) in obsidian and devonthink. For the sake of anyone who wants to do something similar I’ve added my code below.

A few notes:

  • this only takes the plain text part of each note, ignores any rtf formatting
  • it assumes that the link text and target id name are the same thing (which was true for me due to creating all links via zip links; would be quite easy to move away from that assumption).
  • assumes that interesting items can be selected via their prototypes.

The code is in R - would be just as easy in python or ruby or whatever, but I went with what I’ve used more recently. Needs the tidyverse, xml2, and stringi packages.

Really not sure this is useful to anyone, but just in case, here it is:

library(xml2)
library(tidyverse)
library(stringi)

### function definitions

add_wikilinks <- function(ID, text, links_df) {
  # find relevant links
  links <- links_df %>% filter(sstart>0 & sourceid==ID) %>% arrange(sstart)
  # return existing text if no links were found
  if (nrow(links)==0) {
    return(text)
  }
  
  # create substring: takes the link location and surrounds it by [[]]
  # this assumes that the source name and link text are the same.
  # then returns the modified text string
  wikified <- paste0('[[',
                     stri_sub(text, from=links$sstart+1, length=links$slen),
                     ']]')
  return(stri_sub_replace_all(text,
                              from=links$sstart+1,
                              length=links$slen,
                              value=wikified))
}

tbx_links <- function(tbx_xml) {
  # grab all link entries
  links <- xml_find_all(tbx_xml, ".//link")
  # convert link endtries to data frame, keeping link name sourceid, destid, and start and length
  links_df <- data.frame(
    name=links %>% xml_attr("name"),
    sourceid=links %>% xml_attr("sourceid"),
    destid=links %>% xml_attr("destid"),
    sstart=links %>% xml_attr("sstart") %>% as.numeric(),
    slen=links %>% xml_attr("slen") %>% as.numeric()
  )
  return(links_df)
}

tbx_items <- function(tbx_xml, links_df) {
  
  # grab all item entries
  items <- xml_find_all(tbx_xml, ".//item")
  
  # convert to dataframe
  items_df <- data.frame(
    # get item ID
    ID=items %>% xml_attr("ID"), 
    # get item prototype
    proto=items %>% xml_attr("proto"), 
    # find first Name attribute nested below this item
    Name=items %>% xml_find_first(".//attribute[@name='Name']") %>% xml_text(),
    # find first text entry nested below this item
    text=items %>% xml_find_first(".//text") %>% xml_text()
  ) %>% # now add the wikilinks and header text
    mutate(wikilinks=map2_chr(ID, text, add_wikilinks, links_df=links_df),
           wikitext=paste0('# ', Name, '\n\n', wikilinks))
  return(items_df)
}

write_to_md <- function(items_df, outpath, prototypes) {
  # check output directories, create if necessary
  dirs <- paste(outpath, prototypes, sep="/")
  for (d in dirs) {
    if (!dir.exists(d)) {
      dir.create(d, recursive = TRUE)
    }
  }
  
  # take the items
  items_df %>% 
    # keep just the one with the prespecified prototypes
    filter(proto %in% prototypes) %>% 
    # and write each to file
    split(.$ID) %>% 
    map(~ write_file(.x$wikitext, file=paste0(outpath, "/", .x$proto, "/", .x$Name, ".md"), append = FALSE))
}

### Run it for my data

# start by reading my tbx file
litnotes <- read_xml("~/Dropbox/litnotes.tbx")
# create a dataframe of all the links
links_df <- tbx_links(litnotes)
# create a dataframe of all the items; this includes wikilinkifying
items_df <- tbx_items(litnotes, links_df)
# write out those items with Zettel and Reference prototypes
write_to_md(items_df, "testout", c("Zettel", "Reference"))
1 Like

Welcome to community Jason !Thansk for the code

I do use Noteplan3 (a bit) , the code is beyond my capabilities(to run or to understand) but few screenshots on how they look in NP3 would be great !

@jpl Can you describe exactly how one is to use this script. Is this an AppleScript? What steps should someone go through to execute it? I’m sorry if I missed this above.

By no means are you, or anyone, a failure on the point of export, or on any point for that matter. If you can write this xml parser, you can do Tinderbox export, with a little guidance.

There are no dumb questions. Just because you can’t do it now does not mean you can’t do it later? Also, if you’ve never been exposed to some of the concepts, like HTML or Markdown, there is no reason why you should expect yourself to know what to do. Output software like Word and PowerPoint have trained us to not have to look under the hood. Tinderbox give us the ability to look under the hood and at first pass what we see is unfamiliar, but by no means learnable.

Getting to the contribution stage (aka publishing stage) of what I call the 4Cs of knowledge making takes training and experience. Tinderbox is such a powerful tool that people are often afflicted with the paradox of choice. There are so many ways to publish out of Tinderbox, the method you choose all depends on both your immediate needs and context and your long-term aspirations and goals.

When thinking about how to publish your content out of Tinderbox there are a number of factors to consider, including 1) how stylized do you need the content?, 2) how often will you need to perform an operation?, 4) are you simply grabbing the $Name and $Text, or are you wanting to pull in more variables, 5) are you looking to pull and transform attributes to create new meaning through the export, 6) how much time do you have, 7) are you interested in learning more about HTML and coding, 8) is this for yourself or someone else, and so on.

I have a few trainings on export:

http://forum.eastgate.com/t/tinderbox-training-video-29-working-with-export-templates-part-2/4207/8

http://forum.eastgate.com/t/tinderbox-training-video-20-working-with-export-templates-part-3/4075/2

http://forum.eastgate.com/t/tinderbox-training-video-31-integrating-export-code-action-code-in-your-text/4233/31

http://forum.eastgate.com/t/tinderbox-training-video-30-working-with-export-code-to-pull-attribute-values-into-your-notes/4213/2

I’ve also started to contribute some templates.

You most certainly got this! I’ll be working on creating more videos on this topic.

1 Like

Apologies, I was indeed too brief.

This code is in R, an open source scripting/programming language (mainly in use by statisticians). To use the code, you would:

  • download and install R from here: (it tells me I can’t include links - search for R cran and it’s the first link)

  • once installed, start R. To make your life easier, you could first download and then use the free RStudio IDE: (again, no link allowed - search for RStudio)

  • in R or Rstudio, install the required packages:

    install.packages(‘tidyverse’)
    install.packages(‘xml2’)
    install.packages(‘stringi’)

(note, I think the quotes above are being weirdly substituted - so retype rather than copy paste, maybe)

  • then create a new source file (in RStudio: File->New File->R Script)
  • copy and paste my code from the message above. At the very bottom, there are a few lines which show how I ran it on my tbx file. You’ll need to change the filename inside read_xml, and you’ll almost certainly want to change the prototypes on the very last line and possibly the output directory.
  • once saved, source and run that code.

Could all be turned into an easier to use command line script (but would still need R).

Hrm - just made a screenshot but am told I’m not allowed to upload it, nor to add links. I guess I’m too new a user on the forum for that. So below is what an example Markdown output looks like, in case that’s mildly helpful.

# 5-HT related cre lines

Most 5-HT genes are ubiquitously expressed early in development, and hence KOs are embryonic or early post-natal lethal. From [[Scott et al 2005]], citing refs 13, 17.

Pet-1 is expressed in all mid-brain 5-HT synthesizing neurons and its expression precedes the appearance of serotonergic traits. Pet-1 expression is restricted to 5-HT neurons and their post mitotic precursors. [[Scott et al 2005]]

These two lines show similar cell-type specificity, but the penetrance of transgende expression is different, with SERT-Cre staining more TpH+ cells [[Cardozo Pinto et al 2019]]. This is consistent with papers (citing refs 56 and 57) showing that about 30% of 5-HT neurons develop independently of Pet-1 expression.

These two lines show similar axonal targets, suggesting that it is not necessarily distinct 5-HT populations in the Dorsal Raphe that express Pet-1 [[Cardozo Pinto et al 2019]]. (But the authors point out that their methods might have a ceiling effect and not find subtle differences).

ePet-Cre labels 5-HT neurons at earlier times in development than SERT-Cre (ref 65) [[Cardozo Pinto et al 2019]].

Dorsal Raphe 5-HT neurons show differential response to motivational stimuli on different timescales [[Cardozo Pinto et al 2019]] citing refs 22, 24, 26

The Dorsal Raphe also contains a population of dopaminergic neurons

See also:
[[development of 5-HT neurons]]
1 Like

Hey there, I got R and R Studio installed. I’m not clear on how to install the packages. What steps should I take to do this?

Maybe a moderator @mwra will change permissions. Would be good to see screenshots and links.

Start Rstudio. There should be a terminal window (bottom left, I think). In there type these lines, pressing enter after each line:

install.packages("tidyverse")
install.packages("xml2")
install.packages("stringi")

(Being careful about quote substitution in this forum)

Oh, I see. got it, it was not intuitive.

Got it installed and running. Have updated the file name and path. When I run the script, what is supposed to happen next? Where does the output of the script go?

In the folder that you specified on the last line; in the example it would be “testout” (relative path to where-ever you are running the script from). Also remember that it will only take notes with the right prototype - in my example they were either “Zettel” or “Reference”; so make sure that is updated too to reflect your own note prototypes.

Got it. I’m getting this error. Any ideas?

You need to run everything in that file - the easiest thing to do is to click the “Source on Save” button that you see up top, then save the file with the name of your choice. Every time you save it the code will then run.

(And I need to step away from my computer, so apologies for any delayed answers to other problems that might crop up)

Thanks for posting the script, @jpl. I am also still trying to make it work and getting this error:

index ranges must be sorted and mutually disjoint

I will give it a try again later. Let me know if you have any ideas about what I could change to make it work :wink:

Ok, got it almost working.
image
For some reason, the notes from my TBX file are not being exported.

All very interesting. Thanks. Always wanted to have an excuse to play with R.