Exporting annotations from DEVONthink to Tinderbox

Don’t have time at the minute to test this, but I think if you make the delimiter "Page " then you would get notes that look like this:

3:
processus


3:
psychic de deliason


etc.

At the risk of hijacking the thread, I’ve always loved the trademark @dominiquerenauld pastel tones for your notes. Did you make the shades yourself, or are they variations of the named Tinderbox options?

2 Likes

Thank you Derek. I finally add myself a page number to each note. It doesn’t take me a lot of time and I can at the same time review my notes. I mustn’t lose sight of the fact that I take notes with Tinderbox precisely in order to map my notes.

1 Like

I play with the colour palette, shadow including.

2 Likes

Thanks, @dominiquerenauld. Information presented beautifully seems to make it more meaningful.

Can verify that if you…

  1. In Devonthink, select Tools > Summarize Highlights > Rich Text on a PDF w/highlights
  2. Drag the created file into Tinderbox
  3. Explode on delimiter “Page” and check “Delete delimiter”

…you get what you suspect, @derekvan, with the name of the note being the page number.

3 Likes

It works and the link leads correctly to the related page. Thanks.

Dear Beck,
could you please clarify where is this option?

See Exploding notes and more precisely the Explode pop-over. Do ask if still stuck.

1 Like

How do you set the delimiter “Page”? Thanks

If you read this post (above) you will see the data imports with lines like “Page 2:”. In this post you will see that the inityal big note is split into notes starting like so “6:”.

We can thus deduce the original note was broken on the delimiter string "\nPage " and the delimiter deleted )as per the controls described here. Give it a try. :slight_smile:

I’m facing the same challenge as @dominiquerenauld, to whit to import the the file output through the Summarize Highlights in Devonthink v3. In this case there are 210 notes and I do this quite often so I’ve spent more time how to streamline the process. I share my solution as it could be useful for other TB users who also store and highlight their documents in DevonThink:

  • Generate a Summarize Highlights output file in DevonThink selecting the RTF output option
  • Run the AppleScript found under the following DT community thread to convert the RTF links into their URL equivalents e.g. spell out the links
  • Create a blank TB file and a single note to serve as the basis for the import
  • Select and copy the RTF text (with URLs) into the $Text field of the TB note. This serves as the basis for the import and helps me get around the automatic import mechanism for TB.
  • Apply the TB Explode function to the note using the following settings to generate a single TB note for each highlight
    image

NB: the structure of a DT3 Summarise Highlight file uses a Return to identify each highlight and 2 tabs to identify the link to the PDF file/page, the highlighted text and (optional) the user defined comment for each highlight. See the example below which includes 1) the URL link in blue, 2) the highlighted text from the PDF file and 3) my comment. The 2nd link at the bottom belongs to the next highlight but is included to provide the overall structure of the file.

If all goes well you should have exploded notes that look something like the screenshot below with each note title given by the link to the corresponding page in the Devonthink PDF document.

image

  • My goals is now transform all of the imported notes into the structure that allows me to analyse their content within TB. Specifically I want to 1) Extract and store the link in my own key attribute $DTURL, 2) Set the $Name of the note with my comment text and 3) set $Text to the quoted text from the PDF document. For this I use the Stamp code below applied to each of the exploded notes. NB: For the case where I’ve highlighted text in the PDF file but not provided my own comment about the highlight there will be only two elements to deal with, the text and DT link. Here I simply set the $Name to the highlighted text.

$MyList=$Text.replace(“;”,“\;”).split(“\t{2}”);
if($MyList.size==3){$DTURL=$MyList.at(0);$Text=$MyList.at(1);$Name=$MyList.at(2);};
if($MyList.size==2){$DTURL=$MyList.at(0);$Name=$MyList.at(1);$Text=“”;};

The final result looks something like the screenshot below with all notes now properly labelled and containing a link back to the original highlight in the PDF file stored in DevonThink. From there on I can analyse within Tinderbox and make sense of the notes in the usual way.

2 Likes

@mdavidson that’s an interesting process – but, wow, a lot of heroics just to get something imported into Tinderbox. If only DEVONtechnologies structured and formatted their report in a more user-friendly manner. Or realized that people want to export annotations to use them some place else, so give us options. I use the heck out of DEVONthink, but it is also in the “my way or no way” category of design.

This is why I use MarginNote for annotation and getting data into Tinderbox. MN’s formatting is horrible, but at least it provides several means to export annotations in note-by-note chunks.

I agree on the effort although worth it when there are many notes to deal with (or the process will be repeated many times).

To be fair the developers of DevonThink did try to address some of my suggestions and for instance updated the RTF output format to include a DevonThink URL for each highlight (instead of only when the page changed). With this I have the information to link back to the source document and page. Unlike @dominiquerenauld I’m also keen to map the chunks of information into their respective key attributes (for instance filling a URL key attribute with the DT link).

Ideally the splitting would be done in DT itself and then imported in a transparent way to TB. Failing this at least the format from DT is useable and stable. Additional functions in the Explode dialogue such as being able to choose the last sentence as the $Name or capturing RegEx matches would have helped simplify the process too.

1 Like

Hi,
using the Summarize Highlights in Devonthink v3 but with “Markup” as format works very well for me while importing the highlights into TB.

I use a small script the convert the Markup I got from DEVONthink into individual notes in Tinderbox. Each note contains a link to the document in DT.

I enclose a demo file - right click on the “DTExport01” demo and choose the stamp “DoProcessDTHighlights” and you get the notes for this export.

The script is simple:

function parseDTHighlights_db(theRawNoteName,thePathToTheNote){
	var:string theDTExport  = $Text(theRawNoteName);
	var:string theAuthorRaw = $theDTExport.extract("^#[^#]\[.*");
	var:string theURLToPDF  = theAuthorRaw.extract("\((.*)\)");
	var:string theAuthor    = theAuthorRaw.extract("\[(.*)\]");
	var:number thePage;
	var:string theNoteURL;
	var:list   theHighlights;
	var:string theHighlight;
	var:string newNote;

	var:list   allNotes     = theDTExport.extractAll("## (\[[^#]*)");
	allNotes.each(singleNote){
		thePage    = singleNote.extract("\[Page (.*)\]");
		theNoteURL = singleNote.extract("\]\((.*)\)\n");

		// each page can have more then one annotation
		theHighlights = singleNote.extractAll("\* \{==(.*)==\}");
		// process all highlights found
		theHighlights.each(singleOne){
			theHighlight = singleOne.extract("\* \{==(.*)==\}");
			newNote = create(thePathToTheNote, theHighlight.replace("[/@#]","|"));
			$Prototype(newNote)       = "dtHighlight";
			$Text(newNote)            = theHighlight;
			$DEVONthinkPage(newNote)  = thePage;
			$DEVONthinkLabel(newNote) = theAuthor.trim;
			$ReferenceURL(newNote)    = theNoteURL;
			$DEVONthinkNouns(newNote) = theHighlight.nounList;
		};
	};
};

And - just to be complete - same for an export out of a Kindle ePub. You send the annotations from the Kindle app (iOS) via Mail and copy this into the Tinderbox note. Here the demo is called “KindleExport01”. Now choose the stamp named “DoProcessKindleHighlights” and again you get the notes for this export.

HighlightsDemo.tbx (228.4 KB)

6 Likes

I’m playing with the nounList function to auto-create links between the annotations I imported.
The function to explode the nodes (using the DT example) creates a list of nouns for each note (see my post before).
A new function creates links between all notes that share one of those nouns.

function doSetDTLinksBetweenNotes_db(theRawNoteName,thePathToTheNote) {
	var:list theNounsList = collect(children(thePathToTheNote),$DEVONthinkNouns).unique.isort;
	var:list theLinkNotes;
	var:string targetNote;
	var:string sourceNote;
	var:number noteCount;

	if(theNounsList.count > 0) {
		theNounsList.each(singleNoun){
			theLinkNotes = collect_if(children(thePathToTheNote),$DEVONthinkNouns.icontains(singleNoun),$Path);
			if(theLinkNotes.count >= 2) {
				// found at least two notes with the same nouns
				noteCount = 0;
				theLinkNotes.each(noteToLink){
					if(noteCount < 1) {
						targetNote = noteToLink;
					} else {
						sourceNote = noteToLink;
					}
					createLink(sourceNote, targetNote);
					noteCount += 1;
				};
			};
		};
	};
};

This will give me a map like this - without a single keystroke - all done by Tinderbox:

Here is a demo - first use the stamp “DoProcessDTHighlights” on the note “DTExport01” and then run the stamp “DoProcessDTLinks” on the note “DTExport01”.

HighlightsDemo.tbx (266.3 KB)

2 Likes

Hi Detlef
Thank you for posting your HighlightsDemo.tbx file above.
I have been trying to understand your code for the KindleExport and DevonthinkExport.

The Devonthink demo works perfectly as described but…I am having trouble with the KindleExport probably because it is using the German version.

Would it be possible for you to repost the raw German Kindle html Text that you used to import into Tinderbox? If I am not mistaken, the kindleExport included example is the processed Kindle file after it has been stamped with your code.

Comment: When I export from Kindle, it give me a raw html file.
Thanks in advance
Tom

Upps - the note with the Kindle export should contain text… I upload a new demo file - sorry. My script deletes the text of the source note after processing it. I’ll change that later.

The parser works with the html export you received by mail from your Kindle.

Here (function parseKindleHighlights_db):

		thePage       = singleHighlight.extract("Seite ([0-9]+)");
		thePosition   = singleHighlight.extract("Position ([0-9]+)");
		highlightType = singleHighlight.extract("Markierung|Notiz|Lesezeichen");
		theColor      = singleHighlight.extract("Markierung?\(<span class=\x22(.*)\x22");

You have to replace the German terms “Seite”, “Position”, “Markierung|Notiz|Lesezeichen” and “Markierung?” with the English version. Or you send me a demo export and I will create a EN version for you.
HighlightsDemo.tbx (337.7 KB)

2 Likes

Dear Detlef,
would you be so truly kind to make a very short video (when you have a moment) recording the following sequence?

  1. from DT pdf with highlights (and annotations?) 2. export in markdown 3. opening/parsing-exploding the DT export 4. running your two scripts.

I have downloaded it, but frankly I do not know where to start…

  • On the other side the possibility of linking automatically the highlights is really an interesting thing semantically.
  • Pity there is no way to go back to the exact page of the pdf (unless there is a way for you to integrate also what Beck Tench suggests in her post).
  • Pity there is not a link back to bookends

MANY thanks for your great support !

There are many pdf annotation apps runs on ipad, such as Adobe Acrobat, Xodo, Foxit, Drawboard, PDF Expert, PDFelement, etc.
I recommend PDF Expert, it’s my favorite for many years.

1 Like