@derekvan, I’m late to this conversation, but very interested in experimenting with TBX as my Zettlekasten.
The dropbox links to the TBX seems broken. Would you be willing to repost it?
My fear in using TBX is I don’t want to get stuck if I decide to move to another “slip box.” However, in my attempts to play around it looks like I’m solving it, but I’d be curious to see what you’ve done.
Edit: I was able to download the Keyboard Maestro script
It turns out I’m too dumb to fully grok the tinderbox export mechanisms. But this prompted me to follow through on the promise of tinderbox not locking you in via having parsable xml files. Turns out the file format is quite beautiful and parsable! I thus created some code that takes all my tinderbox notes and spits out directories of markdown files, replete with wikilinks. I did this to play with noteplan3 a bit, but the files are equally readable (and links work) in obsidian and devonthink. For the sake of anyone who wants to do something similar I’ve added my code below.
A few notes:
The code is in R - would be just as easy in python or ruby or whatever, but I went with what I’ve used more recently. Needs the tidyverse, xml2, and stringi packages.
Really not sure this is useful to anyone, but just in case, here it is:
library(xml2)
library(tidyverse)
library(stringi)
### function definitions
add_wikilinks <- function(ID, text, links_df) {
# find relevant links
links <- links_df %>% filter(sstart>0 & sourceid==ID) %>% arrange(sstart)
# return existing text if no links were found
if (nrow(links)==0) {
return(text)
}
# create substring: takes the link location and surrounds it by [[]]
# this assumes that the source name and link text are the same.
# then returns the modified text string
wikified <- paste0('[[',
stri_sub(text, from=links$sstart+1, length=links$slen),
']]')
return(stri_sub_replace_all(text,
from=links$sstart+1,
length=links$slen,
value=wikified))
}
tbx_links <- function(tbx_xml) {
# grab all link entries
links <- xml_find_all(tbx_xml, ".//link")
# convert link endtries to data frame, keeping link name sourceid, destid, and start and length
links_df <- data.frame(
name=links %>% xml_attr("name"),
sourceid=links %>% xml_attr("sourceid"),
destid=links %>% xml_attr("destid"),
sstart=links %>% xml_attr("sstart") %>% as.numeric(),
slen=links %>% xml_attr("slen") %>% as.numeric()
)
return(links_df)
}
tbx_items <- function(tbx_xml, links_df) {
# grab all item entries
items <- xml_find_all(tbx_xml, ".//item")
# convert to dataframe
items_df <- data.frame(
# get item ID
ID=items %>% xml_attr("ID"),
# get item prototype
proto=items %>% xml_attr("proto"),
# find first Name attribute nested below this item
Name=items %>% xml_find_first(".//attribute[@name='Name']") %>% xml_text(),
# find first text entry nested below this item
text=items %>% xml_find_first(".//text") %>% xml_text()
) %>% # now add the wikilinks and header text
mutate(wikilinks=map2_chr(ID, text, add_wikilinks, links_df=links_df),
wikitext=paste0('# ', Name, '\n\n', wikilinks))
return(items_df)
}
write_to_md <- function(items_df, outpath, prototypes) {
# check output directories, create if necessary
dirs <- paste(outpath, prototypes, sep="/")
for (d in dirs) {
if (!dir.exists(d)) {
dir.create(d, recursive = TRUE)
}
}
# take the items
items_df %>%
# keep just the one with the prespecified prototypes
filter(proto %in% prototypes) %>%
# and write each to file
split(.$ID) %>%
map(~ write_file(.x$wikitext, file=paste0(outpath, "/", .x$proto, "/", .x$Name, ".md"), append = FALSE))
}
### Run it for my data
# start by reading my tbx file
litnotes <- read_xml("~/Dropbox/litnotes.tbx")
# create a dataframe of all the links
links_df <- tbx_links(litnotes)
# create a dataframe of all the items; this includes wikilinkifying
items_df <- tbx_items(litnotes, links_df)
# write out those items with Zettel and Reference prototypes
write_to_md(items_df, "testout", c("Zettel", "Reference"))Welcome to community Jason !Thansk for the code
I do use Noteplan3 (a bit) , the code is beyond my capabilities(to run or to understand) but few screenshots on how they look in NP3 would be great !
@jpl Can you describe exactly how one is to use this script. Is this an AppleScript? What steps should someone go through to execute it? I’m sorry if I missed this above.
By no means are you, or anyone, a failure on the point of export, or on any point for that matter. If you can write this xml parser, you can do Tinderbox export, with a little guidance.
There are no dumb questions. Just because you can’t do it now does not mean you can’t do it later? Also, if you’ve never been exposed to some of the concepts, like HTML or Markdown, there is no reason why you should expect yourself to know what to do. Output software like Word and PowerPoint have trained us to not have to look under the hood. Tinderbox give us the ability to look under the hood and at first pass what we see is unfamiliar, but by no means learnable.
Getting to the contribution stage (aka publishing stage) of what I call the 4Cs of knowledge making takes training and experience. Tinderbox is such a powerful tool that people are often afflicted with the paradox of choice. There are so many ways to publish out of Tinderbox, the method you choose all depends on both your immediate needs and context and your long-term aspirations and goals.
When thinking about how to publish your content out of Tinderbox there are a number of factors to consider, including 1) how stylized do you need the content?, 2) how often will you need to perform an operation?, 4) are you simply grabbing the $Name and $Text, or are you wanting to pull in more variables, 5) are you looking to pull and transform attributes to create new meaning through the export, 6) how much time do you have, 7) are you interested in learning more about HTML and coding, 8) is this for yourself or someone else, and so on.
I have a few trainings on export:
http://forum.eastgate.com/t/tinderbox-training-video-20-working-with-export-templates-part-3/4075/2
I’ve also started to contribute some templates.
You most certainly got this! I’ll be working on creating more videos on this topic.
Apologies, I was indeed too brief.
This code is in R, an open source scripting/programming language (mainly in use by statisticians). To use the code, you would:
download and install R from here: (it tells me I can’t include links - search for R cran and it’s the first link)
once installed, start R. To make your life easier, you could first download and then use the free RStudio IDE: (again, no link allowed - search for RStudio)
in R or Rstudio, install the required packages:
install.packages(‘tidyverse’)
install.packages(‘xml2’)
install.packages(‘stringi’)
(note, I think the quotes above are being weirdly substituted - so retype rather than copy paste, maybe)
Could all be turned into an easier to use command line script (but would still need R).
Hrm - just made a screenshot but am told I’m not allowed to upload it, nor to add links. I guess I’m too new a user on the forum for that. So below is what an example Markdown output looks like, in case that’s mildly helpful.
# 5-HT related cre lines
Most 5-HT genes are ubiquitously expressed early in development, and hence KOs are embryonic or early post-natal lethal. From [[Scott et al 2005]], citing refs 13, 17.
Pet-1 is expressed in all mid-brain 5-HT synthesizing neurons and its expression precedes the appearance of serotonergic traits. Pet-1 expression is restricted to 5-HT neurons and their post mitotic precursors. [[Scott et al 2005]]
These two lines show similar cell-type specificity, but the penetrance of transgende expression is different, with SERT-Cre staining more TpH+ cells [[Cardozo Pinto et al 2019]]. This is consistent with papers (citing refs 56 and 57) showing that about 30% of 5-HT neurons develop independently of Pet-1 expression.
These two lines show similar axonal targets, suggesting that it is not necessarily distinct 5-HT populations in the Dorsal Raphe that express Pet-1 [[Cardozo Pinto et al 2019]]. (But the authors point out that their methods might have a ceiling effect and not find subtle differences).
ePet-Cre labels 5-HT neurons at earlier times in development than SERT-Cre (ref 65) [[Cardozo Pinto et al 2019]].
Dorsal Raphe 5-HT neurons show differential response to motivational stimuli on different timescales [[Cardozo Pinto et al 2019]] citing refs 22, 24, 26
The Dorsal Raphe also contains a population of dopaminergic neurons
See also:
[[development of 5-HT neurons]]Hey there, I got R and R Studio installed. I’m not clear on how to install the packages. What steps should I take to do this?
Start Rstudio. There should be a terminal window (bottom left, I think). In there type these lines, pressing enter after each line:
install.packages("tidyverse")
install.packages("xml2")
install.packages("stringi")
(Being careful about quote substitution in this forum)
Oh, I see. got it, it was not intuitive.
Got it installed and running. Have updated the file name and path. When I run the script, what is supposed to happen next? Where does the output of the script go?
In the folder that you specified on the last line; in the example it would be “testout” (relative path to where-ever you are running the script from). Also remember that it will only take notes with the right prototype - in my example they were either “Zettel” or “Reference”; so make sure that is updated too to reflect your own note prototypes.
You need to run everything in that file - the easiest thing to do is to click the “Source on Save” button that you see up top, then save the file with the name of your choice. Every time you save it the code will then run.
(And I need to step away from my computer, so apologies for any delayed answers to other problems that might crop up)
Thanks for posting the script, @jpl. I am also still trying to make it work and getting this error:
index ranges must be sorted and mutually disjoint
I will give it a try again later. Let me know if you have any ideas about what I could change to make it work 
Ok, got it almost working.
For some reason, the notes from my TBX file are not being exported.
All very interesting. Thanks. Always wanted to have an excuse to play with R.
I’ve tried to run it, but I’m getting error
ERROR: ' ' does not exist in current working directory ('/Users/shijianhui').
I don’t really get what I’m doing wrong 
Here’s a small version of my notes that I used for testing this code; maybe anyone who is interested in trying it out can make sure that they get results from this one first to make sure that there are no issues with the install (or my instructions). Then, if still keen, I can try and help debug any issues that occur (post tbx file here or dm me?) in other files that I likely did not think of when only testing against my notes.
sample-tbx-file.tbx (103.7 KB)
After running the code above on this sample file the output should look like this:
testout/
testout/Reference
testout/Reference/Hellstrom et al 2012.md
testout/Reference/Zhang et al 2013.md
testout/Zettel
testout/Zettel/maternal care regulates a host of NGFI-A sensitive genes.md
testout/Zettel/maternal effects on hippocampal glucocorticoid receptors mediated by increased 5-HT turnover NGFI-A expression.md
Of course, I should reiterate as @satikusala nicely summarized in his documentation/videos of Tinderbox export and @mwra has documented in depth elsewhere , my xml parser is not likely ever going to compete with proper exports. It was me struggling with export plus, mostly, trying to put the no-data-lockin principle to the test (which it passed with flying colours - I now know I can get very useful data out of tinderbox even if I somehow lose access to tinderbox). Still, happy to make that script more usable if anyone thinks they would benefit.