Publish from Tinderbox to DEVONthink


(Pat Maddox) #1

Warning: This is beta software!

It does the job fine for me, but is still a bit rough around the edges. If you don’t understand how it’s built, you might get confused. Ask for help on the forum and I’ll do my best to help you out.


I love taking notes with Tinderbox, don’t you? It gives me a nice structured way of looking at information, and then I can export it to take it with me or share it with others.

Ever since DEVONthink To Go was released, I’ve wanted to use it to take my Tinderbox notes with me. So I run File->Export->as HTML, index the folder in DEVONthink, and off I go, right? Not quite…

There’s one key problem that prevents me from effectively taking my Tinderbox notes on the road:

Indexed folders in DEVONthink kind of suck, unless that’s what you really want.

You’ve got a giant Bookends library that you want to search with DEVONthink? Great, index the Bookends folder. Just don’t bother using DEVONthink’s Copy item link feature. As soon as you move or rename a file on disk, the x-devonthink-item:// link breaks. And I love those links! You can add them to your task manager and calendar, and they always take you to the record you want – if it’s an imported record. So how do you make that work with Tinderbox?

Take notes in Tinderbox. Publish to DEVONthink. Maintain the link.

Publishing from Tinderbox to DEVONthink is easy, with the code provided in this document. With it, you can:

  • press one button to publish a note from Tinderbox to DEVONthink
  • automatically link the Tinderbox note to the created DEVONthink record
  • update your DEVONthink records with the press of a button
  • file the record wherever you like in your DEVONthink database
  • use the x-devonthink-item:// link anywhere you want, confident that it will always work

Two ways to publish

1. The quick-start method (just edit this document!)

The quickest way to publish from Tinderbox to DEVONthink is to simply use this document. Add a note and enable its $PublishToDEVONthink attribute (the p_DEVONitem prototype has it enabled by default). Then run Stamps->Publish to DEVONthink. If everything goes well, DEVONthink will have a new HTML record and your note will now have its $SourceURL attribute set to the x-devonthink-item:// URL. Click it to open the record in DEVONthink.

2. Publish from existing documents

Already have a document with a bunch of notes in it? No problem. It’s a little bit more involved than the quick-start method, but it should still only take a few minutes. Here’s what you do:

  1. Add a boolean user attribute named PublishToDEVONthink
  2. Make sure you have an HTML template defined (you can use File->Built-In Templates->HTML if you don’t have any)
  3. Add the Code prototype: File->Built-In Prototypes->Code
  4. Copy the Stamps container into your document
  5. Copy the Applescripts container into your document
  6. Create a stamp with the following action code: action($Text("/Stamps/Publish to DEVONthink"))

Run the stamp on your note!

What gets published

  • $Name
  • $URL
  • HTML representation of the note, as defined by $HTMLExportTemplate property

If you make changes to your note, you can update the record in DEVONthink simply by publishing the note again.

Links between notes become DEVONthink links

One of my main goals with this project is to build a personal wiki with Tinderbox – and then take it with me wherever I go. To do that, the pages need to be able to link to one another… which is precisely what this document does.

Assuming that you have published your notes to DEVONthink, any links will automatically be converted to x-devonthink-item:// links.

For this to work, you may need to publish your notes twice.

The first pass imports the note into DEVONthink. This sets $SourceURL on the note. Before a note has a $SourceURL, any incoming links will not work in DEVONthink. Here’s an example:

  • Note A links to Note B. Neither has been published to DEVONthink.
  • Publish Note A. The link to Note B will not be converted to a DEVONthink link, because Note B has not been published and does not have a $SourceURL
  • Publish Note B. It now has a $SourceURL. The published DEVONthink record for Note A still has a broken link, because it was published before Note B.
  • Publish Note A again. It picks up the $SourceURL from Note B, and the published DEVONthink record has a working x-devonthink-item:// link

The simplest solution is to publish all your notes twice each time you add or link new notes.

I hope to improve this functionality in the future to make it faster and more automatic.

Possible issues

There are probably some edge cases, and for now it’s best to only publish top-level notes with distinct names. But it’s working well for me so far…

Implementation note

To get this working quickly, and to minimize the number of steps required to make this work for existing documents, the x-devonthink-item:// linking functionality uses $MyString as a temporary storage area. If you have agents that suddenly go out of whack, it may be because they’re relying on $MyString. I’ll see what I can do to avoid using it in the future.

Performance concerns

I use find() to look up all the notes that have been published to DEVONthink. Then I iterate through them and replace the exported paths with the x-devonthink-item:// URLs. This is a lot slower than just looking for the links that are in the note itself. But I think that’s the only way to support links that are provided by child notes.

Version history

0.4.0

  • NEW Add $PublishToDEVONthink attribute for explicit publish selection
  • NEW Set DEVONthink’s URL field to Tinderbox note link (forum request)
  • FIX Linking to and from notes in containers
  • FIX Link issue when publishing from aliases
  • FIX Respect DEVONthink’s import destination configuration (forum report)

0.3.0

0.2.0

  • Replace path-based text links with DEVONthink links – e.g. <a href="Publish_to_DEVONthink.html">Check it out!</a> becomes <a href="x-devonthink-item://919FE3FC-88CD-4E6A-804A-F4DDA4F2C114">Check it out!</a>

0.1.0

  • Basic publish to DEVONthink functionality

Upcoming Development

  • Delete from DEVONthink
  • Tooling to update an existing document
    • Stamp to check for necessary infrastructure
    • Stamp to update components (stamps, applescripts, prototypes)

Get the code on GitHub or download publish-tinderbox-to-devonthink_0_4_1.tbx (122.8 KB)


Grabbing a bunch of tags
(Paul Walters) #2

VERY clever @pat – and very useful and extensible concept.

Thank you for sharing this one!


(Pat Maddox) #3

(updated to include a fix for notes that include a ' in the $Name)


Thanks :slight_smile: I got really excited once I discovered that /usr/bin/osascript can receive its script via standard input. The idea of using Tinderbox to construct applescripts on the fly looks to be very powerful…

I’ve wanted this sort of publish-to-DEVONthink functionality for a couple years now, and the osascript insight made it possible to implement a first version in an afternoon.


(Pat Maddox) #4

Well I am ecstatic to share an updated version (in the original post) which includes automatic x-devonthink-item:// linking when you link between Tinderbox notes.

I have wanted this for years! Added bonus: the x-devonthink-item:// trickery takes place only when you publish notes to DEVONthink. So if you want to use a single Tinderbox file to manage a public website, and publish a local copy to DEVONthink, you can!

If you link to any unpublished notes, no x-devonthink-item:// links will be created, because the note doesn’t have a DEVONthink record yet. But once you publish those notes to create a record, and then publish the linking notes again, you’ll get those reliable x-devonthink-item:// links.


(Paul Walters) #5

(Comments were too confusing. Deleted)


(Pat Maddox) #6

Thanks @PaulWalters for the suggestions :slight_smile:

Yep makes sense. I’ll probably stick it on GitHub and tag distinct versions.

Can you talk about this a bit more? I’m not quite sure what you mean.

I don’t think so… the opposite is true though. If note 2 has a text link to note 1 and note 2 gets published before note 1, then the link fails – note 1 doesn’t have a $SourceURL at that point and so the path-based link doesn’t get substituted with the x-devonthink-item:// URL.

This is documented in the section Use text links between notes… and generate DEVONthink links:

The first time you publish these two notes to DEVONthink, you’ll get the standard Tinderbox file-based link. But once the notes have been published to DEVONthink, publishing a second time will replace those links with DEVONthink links.

but I can see how that is probably unclear.

I don’t… that’s the whole point of automatically substituting $SourceURL for links in the rendered HTML.

This:

<a href="Publish_to_DEVONthink.html">Check it out!</a>

Becomes this:

<a href="x-devonthink-item://919FE3FC-88CD-4E6A-804A-F4DDA4F2C114">Check it out!</a>

Perhaps I’m misunderstanding you though.

Is that something you want? I don’t personally care to export to RTF. But if it’s easy enough to do then perhaps I could add a setting. I love how the rendered HTML looks on DEVONthink desktop and DTTG. Plus there’s room for styling.

Fair enough. I can add a “hey, my link’s broken!” section to the documentation. I am trying to think of a way to indicate broken links in some way… the first case being when I’ve linked to a note that isn’t published (as discussed previously in this post), and the second being the cases you just brought up.

Perhaps I need to clearly communicate that this is beta software… the functionality does the job for me, but it’s a bit rough around the edges and if you don’t understand the machinery then you might get stuck – in which case you’ll have to be comfortable asking for help here.


(Paul Walters) #7

If I don’t export all the notes at the same time, the text links (link from one note to another) do not work in DEVONthink. This is because of DEVONthink’s handling of HTML links. I don’t know how to explain it differently.

I’ll fix it on my side, anyway, since I am adding my own routine for RTF notes.


(Pat Maddox) #8

Export all the notes and twice, it should be all good.

The first export is what produces the $SourceURL. Before a note has a $SourceURL, there’s no way to replace links to that note. But once a note has a $SourceURL, when you export a note that links to it then the path-based link gets replaced with the $SourceURL as shown above.

In the version uploaded to the original post, this behavior only applies to text links. If you have agents or use ^include^ to get links then those won’t be replaced. But I have a local version that fixes this (it’s a bit slower and I’m not sure how it will scale though).


(Paul Walters) #9

Sorry, @pat, I cannot get things to work that way.


(Pat Maddox) #10

Hrm, do you have the version that has the note Use text links between notes… and generate DEVONthink links?

I’ll upload the latest version soon, which has a fix for non-text links… but text links should work fine in the above version. So that’s strange.

If you upload the file (here, PM, or pat@patmaddox.com) then I can take a look and see what’s wrong.


(Pat Maddox) #11

@PaulWalters I updated the original post with 0.3.0 which has a fix for non-text links. It may address other link-based issues. Try it out and let me know if it works for you. And if not, I hope you’ll share your document so that I can investigate and see what’s wrong.


(Paul Walters) #12

Version 0.3.0 always puts the exported notes into the Global Inbox. The earlier version respected the DEVONthink setting in Preferences > Import > Destination, which I always set to “Select Group” since the Global Inbox is nothing but a useless dumping ground.

Somewhere in this code the preference is getting ignored

  ^if(^value($MyString("/DT configuration"))^)^
    set theGroup to get record with UUID "^value($MyString("/DT configuration"))^"
  ^else^
    set theGroup to incoming group
  ^endif^

(Pat Maddox) #13

Okay I’ll fix that. In the mean time, if you look in Applescripts/CreateDEVONthink record there’s the create record with line that ends in in theGroup. If you delete in theGroup at the end of that line then it will use DTPO’s configuration rather than the global inbox.


(Andreas Grimm) #14

This is just awesome, @pat. Thank you for this!

As far as the super-cool-link-feature is concerned: In Devonthink I do see the links created between notes in Tinderbox. However, clicking those inter-note-links in Devonthink does not render any effect other thank giving me the system-error-honk.

Ideas?
Cheer!


(Pat Maddox) #15

hrm, might be something similar to what @PaulWalters is experiencing. I’m not sure what’s going on. Can you paste the HTML link that’s not working? Is the href getting replaced with an x-devonthink-item:// link or not?


(Paul Walters) #16

I’ll try again to explain the HTML errors that appear in the file. First, I added a text link to the top level note “Publish to DEVONthink” in the text of the note “Links between notes become DEVONthink links”. Then I published the first note, and then I published the second note – separately.

The link that now appears in the HTML for the second note (“Links between notes become DEVONthink links”) is

<p>This is a link to <a href="../x-devonthink-item://2D3A180D-60A1-41E4-9AF9-F7F33BE5490F">
Publish to DEVONthink</a></p>

which DEVONthink interprets as a nonsensical link to a folder inside the database package – something that could never work.


(Pat Maddox) #17

I found a bug with links to/from notes inside containers, that I think is fixed but I’m currently vetting in my own usage. The HTML you shared above looks like that’s what’s happening – thanks for sharing that.

Does it work properly if you only link to/from top-level notes?


(Paul Walters) #18

No it does not work properly.

Here’s the full link that DEVONthink interprets from the HTML (with personal info redacted).

Internote HTML links is a longstanding problem with DEVONthink, BTW.


(Pat Maddox) #19

Hrm I’m at a loss. I uploaded a video walkthrough in case it helps. Perhaps this is what you’re already doing though…

The only other thing I can think of is that perhaps it’s a version problem. I’m running DTPO 2.9.16, Tinderbox 7.3.0, on OS X 10.10.5. Other than that, maybe there’s some DEVONthink configuration difference that I’m unaware of.


(Paul Walters) #20

(Another too-long-to-be-useful post. Deleted.)