Sending data to Quarto via runCommand()

Tinderbox Tasks
1

Dear Dr.Mark Anderson,
Quarto is not just an online-only app.
I installed the app as follows.
I download and install quarto-1.3.450-macos.pkg on my Mac OS11.7.9 BigSur.
Or in /usr/local/Caskroom/quarto/1.3.450
You can also [brew install --cask quarto] install using brew cask.
[Install Quarto](A Quarto tip a day - Quarto on Homebrew 20Using%20a%20private%20tap%3A,private%20tap.%20brew%20tap%20rundel%2Fquarto-cli%20brew%20install%20quarto)

I have a question from here.
Any help would be greatly appreciated.

I would like to execute the “quarto preview” command from Tinderbox via Terminal
I am searching for a way to do that.
From which attribute can the RunCommand be quoted?

I finally found it by searching from your aTbRef source.
Action Code > Operators > Action Operator Functional Types > Function actions >
runCommand(commandStr[, inputsStr, dirStr])
I do not know how to write those commands.
Can I borrow your wisdom?
Sincerely yours, WAKAMATSU

quarto-test.tbx (137.9 KB)

2

You need it in the purpose of a live preview, I suppose? I use Quarto too, but simplify my workflow a lot: I copy/paste my text and compile it. But
 I don’t answer your question.

3

Cher Mr.Dominique Renauld,
Merci pour votre message.
Je compile essentiellement quarto directement via le terminal.
J’essaie actuellement l’aperçu quarto de spacemacs (0.999.0@28.2).
La compilation vers safari via spacemacs est lente.
Il met trop de temps à s’afficher.
Je suis habituĂ© Ă  l’environnement utilisant spacemacs,
donc je voudrais continuer à l’utiliser tel quel.

J’aimerais pouvoir rendre depuis Tinderbox.
Le chemin vers la réalisation sera-t-il long ?
SincĂšrement,WAKAMATSU
P.S.
Je vous serais reconnaissant si vous pouviez me dire comment utiliser le quarto.

1 Like
4

OK, this might take a little discussion. First, to check chect have the right app, I assume we are talking about https://quarto.org, which turns out to be a plotting library like plot.ly and such. As you are a musician, I had thought Quarto might be a music notation app :slight_smile: .

The first step here is to know what command line you need to create. So, if you were not using Tinderbox, what would you need to type at the command line to get the output you want. Once you have a working command line in Terminal, you can then work out how Tinderbox can populate the command’s arguments using runCommand(). But, I repeat, make sure the command line works in terminal before trying to use it via runCommand() as it makes it easier to de-bug if there is a problem.

Taking a different approach, and seeing that Quarto is an online plotting library, you might think to try the new v9.6.0 poster feature, see Map Posters. This might be another and possibly simpler way to visualise your information.

1 Like
5

Dear Dr. Mark Anderson,
Thank you for your reply.
I will follow your instructions and read the instructions
in Map Posters carefully.

I am executing the following.
The command from the Terminal,
after specifying the location of the file
type ‘quarto preview note.qmd’ and wait for the compilation process to proceed.
The only thing you have to do is wait for the compilation process to proceed.
In other words, in this case, what we would like you to tell us is.
How to specify the “$ cd file location”,
The command line “$quarto preview note.qmd” to type next.
Where to write it.
How do I quote runCommand()?
Where do I quote from?

I want to write out the steps by which a quarto file is compiled from the terminal.
(Sample file example is ja.qmd)

  1. start Terminal.

  2. $ cd ~/quarto

  3. $ quarto preview ja.qmd

  4. (wait patiently while the compilation runs)
    Starting python3 kernel . .Done
    Executing ‘ja.ipynb’ Cell 1/1
 .Done
    pandoc to: latex (pdf-engine: lualatex)
    metadata 

    running lualatex - 1
    running lualatex - 2
    Output created: en.pdf

  5. Watching files for changes
    Browse at http://localhost:xxxx/web/viewer.html

  6. view.html is displayed from safari.
    During this time, ja.pdf is created in the folder cd ~/quarto.

What procedures should I carry out to port the above process to Tinderbox?
What is the best way to do this?
Yours truly, WAKAMATSU
P.S.
Would not it be better to branch the posts in this thread under a different title?
I have no knowledge of how to change branches directly from this location.

P.S.2
I have been able to compile note.qmd from Emacs 28.2 in the same way.

6

So you want to send quarto preview ja.qmd via runCommand(). So, using variables for clarity:

var:string vCmd = "quarto preview ja.qmd";
runCommand(vCmd);

if you want to pass the command’s flags input as a separate Tinderbox argument you need to use the xarg prefix (see more), like so:

var:string vCmd = "xargs quarto";
var:string vArgs= "preview ja.qmd";
runCommand(vCmd, vArgs);

I don’t have quarto installed(or rime to do so just now), so please be aware I’ve not tested this, but expect it to work.

7

Dear Dr.Mark Anderson,
Thank you for your guidance on the two compilation methods in quarto.
For me, runCommand() is still an unknown world.
I do not have enough knowledge, so I can reach the level of
understanding the procedure,I would like to take a slow look at
your suggestions and learn from them.

Inspired by your suggestion yesterday, I tried the following.
The premise was written in quarto-test.tbx from Tinderbox
Export the note (named “ja”) and the document created as a file
with a “.qmd” extension called “ja.qmd”.
Commands from outside the notebook in quarto-test.tbx, using Apple script.
Executed, Compile the ja.qmd file from the terminal and run up
to the display Safri(ver.16.6), it was executed successfully
(it’s a matter of course).
The script I used is written as follows.

tell application “Terminal”
activate
set currentTab to do script “cd ~/Documents/Tinderbox9/2023quarto”
do script “quarto preview ja.qmd” in currentTab
delay 0.3
end tell

This is the end of my report on my experiments.
Well, there are many things I would like to ask.
Successfully compiled AppleScript from outside of Tinderbox,
From inside Tinderbox, where and how to specify this AppleScript
to be commanded and acted upon, If I write, will I get the same result?
Sincerely, WAKAMATSU

8

Thank you for the update. The whole point of runCommand() is you don’t need the AppleScript and Terminal. runCommand() ‘speaks’ directly to the Mac’s Unix shell. As you’d need to use runCommand to call your AppleScript, it is extra work for no reward.

I assume, because you didn’t report it as working, that the runCommand() example I gave you didn’t work. Looking at your AppleScript script example there is a path element (that wasn’t mentioned in your original example). So have you tried a command like:

var:string vCmd = "xargs quarto";
var:string vArgs= "preview ja.qmd";
var:string vPath = "~/Documents/Tinderbox9/2023quarto";
runCommand(vCmd, vArgs, vPath);

Depending on how big complex your Quarto file is, you can actually skip exporting that and just pass the exported data (currently saved to ja.md) directly into runCommand().

So a pseudo example (as I don’t have all the data & apps to try it):

var:string vCmd = "xargs quarto";
var:string vArgs= 'preview "'+exportedString("ja")+'"';
var:string vPath = "~/Documents/Tinderbox9/2023quarto";
runCommand(vCmd, vArgs, vPath);

I’d recommend getting the first example method (of the two in the this post) to work before trying the second. The whole point of the approach is to change only a little part of the process at a time.

It is difficult to be precise on what works as whilst we are all using Tinderbox, not all Mac unix shells are configured the same way. This is why starting with a Terminal command that works is a got starting play when trying to figure out a runCommand() value to use. The possible parts are:

  • the shell command (e.g. like quarto) - does it run from anywhere
  • the command’s flags (e.g. preview) that trigger different behaviours in the command
  • other input such as data file(s) which might, in Terminal, be supplied via stdIn or as an argument (alongside the flag(s).
  • the relative location of data file(s) to where runCommand() executes in the shell. By default this is you home folder (~), so if calling a data file (above) which isn’t at ~ you need to supply the path to it (either full path or path relative to ~.

Anyway, whilst I’m glad your AppleScript test worked, it is not part of the solution here. Think of runCommand() as similar to being able to type into Terminal directly from inside Tinderbox.

9

Note: I’ve moved this to a thread of its own with a more useful title.

Your test note includes lots of Quarto customisation (i.e that other users might not have.). I’d suggest exporting the simplest note possible to Quarto as that increases the chance of other people being able to test your code. If a simple export works, then you can add in the code you really want confident that the export will work.

I hope that helps :slight_smile:

10

Dear Dr.Mark Anderson,
Thank you very much for your detailed explanation.
I am sorry that all the questions are too elementary.
The big question for me at the moment is,
where in the Tinderbox should I put the command
which you have given me when I use runCommand()?
I do nt know this exact where adding in, that is a reason why
I experimented with a different approach using AppleScript.

How can I put this in my Tinderbox so that I can take my first steps,
where can I put my hand, where can I catch it, so that
I can start walking like a baby?

Please forgive me for bothering you and taking up valuable time
from you on an elementary matter.
I hope you can provide me with more information.
Sincerely, WAKAMATSU

11

Dear Mr. Dominique Renauld,
I have a little question.
It is about how you use quarto.
Are you processing documents written in Tinderbox with quarto?
Q01 :
If so, how do you change the extension in the export to specifying it?

Q02 : Do you use HTMLexportTemplate?
If you are using, when creating the “.qmd” extension
How to write extra description when exporting from html avoiding it?
Q03 :
If possible, a sample of your Tinderbox file
I would appreciate it if you could present it.
Yours, WAKAMATSU
P.S.
Export the file as Text, not HTML
quarto preview when renaming the file with a “.qmd” extension
Commands from Terminal were enabled cleanly.

12

Ah. My apologies. First to understand is that runConmmand() is an [action code] operator (Action Code). So you can use runCommand() in any place where you use actions : rules, edicts, agent actions, on add actions, stamps.

My suggestion for initial testing is to use a stamp. To give you confidence, make a new note, add attribute MyNumber as a Displayed Attribute and apply this stamp (as from the documentation):

$MyNumber = runCommand("wc -w","hello world");

After running the stamp, the note’s $MyNumber is 2. Why, because the number of words in the string ‘hello world’ is 2.

Congratulations. You now know the workings of runCommand(). This is the first of the small steps you asked for. You have now passed a command from Tinderbox it the OS command line and got a calculated output without using the Terminal.

You should be see that what you pass as arguments to the runCommand() operator changes what gets done. Depending on your task you can use runCommand() with:

runCommand( ...);

or without:

$SomeAttribute = runCommand( ... );

a left side argument.

13

Dear Monsieur WAKAMATSU,

To answer you:

Q01:

I don’t export my document. I don’t use any extension from Tinderbox. But, you can do it of course with Tinderbox. Instead of doing that, I just copy and paste my document — including my YAML and my separators — into Quarto. Then, I « knit » the whole and get a nice pdf.

But, I know that you directly « compile » your document from Tinderbox. There are one video Michael Becker @satikusala made about it and some reflections here in the Tinderbox forum.

Q02:

I don’t use HTMLexportTemplate.

Q03:

You’ll find below a file I made to illustrate my words and some pictures to see the whole.

Plan article Quarto.tbx (192.0 KB)

Plan_article_Quarto
Plan_article_Quarto1761×1048 73.4 KB

1
1947×1217 84.7 KB

2
2941×1211 66.3 KB

As you can see, I use a Latex template in my YAML because I do need to output a PDF that looks like an already accepted and published article. This trick helps me a lot. But, you can customize your export template within Tinderbox.

1 Like
14

Dear Dr. Mark Anderson,
Thank you for your detailed explanation.
I apologize for bothering you with my lack of understanding, everytime.
Last time, I set the general idea as “Action Code” and
I created a Stamp using the method which you presented,
but maybe because my writing style was bad, it did not work for me.
This time, after thoroughly understanding it, I will try runCommand again.
Yours, WAKAMATSU

15

Dear Mr.Dominique Renauld,
Thank you for your detailed answer.
I am happy to know that you are using the LaTeX Template.
I am a LaTeX enthusiast.
The reason is LaTeX description method and music
This is because it creates the same world as the specification method in the score.
I will read and learn from your tbx file.
Hurry up and thank you.
Sincerely yours, WAKAMATSU
P.S
My YAML LaTeX Template using lualatex engine.

16

OK, here is the example from above:

runCommand-example.tbx 2023-08-22 10-32-01
runCommand-example.tbx 2023-08-22 10-32-011960×1110 296 KB

And the TBX file shown above: runCommand-example.tbx (212.6 KB)

Test files for test #2 (zip): test.zip (29.9 KB)

Select note “Test” and run stamp “Example 1” on it. Observe that it works.

Next make a new TBX, and without just copy/pasting from my demo, re-create the same test for yourself and prove it works. You will now know the basic principle of runCommand().

Now, in the original file, read the note for test 2 and install the test folders/files from the zip. Select ‘Test 2’ and one at a time try all 4 variants of stamp “Example 2” (a through d). This exercise shows how you deconstruct a single command string into both the 2 and 3 argument forms of runCommand() use.

As before, try recreating the second set of tests in a new TBX so you get used to the syntax.

Note: runCommand() simply communicates with the macOS command line. If you pass the command line the wrong command it will not work: “garbage in, garbage out”. It is important to understand that for non-OS-default command line tools, it is not possible to know that they are installed the same way on every Mac. This is why using the Terminal app to test commands is a good start. If your command doesn’t work in Terminal, it will not work via runCommand(). Note also that a command in Terminal may need re-formatting (i.e. same information, different layout) to meet runCommand’s documented argument syntax.

Tip: to avoid having to code in long literal paths, the 3-argument form of runCommand helps a lot. Note, via the second example, that if using several directories, the base directory you pass as the third argument must be a common parent of the other folders. So, in example #2 it is ‘test’ from which only the ‘x’ and ‘y’ sub-paths are needed. In reality, ‘test’ might me a folder 14 levels deep in your documents, which is why the 3-argument version of runCommand() syntax is so useful.

Summary, until you can re-create this simple demo yourself, and make it work, don’t waste time on anything more complex as you will have no understanding of why it fails. :slight_smile:

1 Like
17

Dear Dr. Mark Anderson,
Thank you for your detailed explanation.
I apologize again, for bothering you with my lack of understanding, everytime.
I understand the first step you taught me.
I tried to move on, but it was all failure.
I was trying to find out what I didn’t know.
「Code examples using attributes with a ‘My’ prefix」
「format(dataStr, formatStr[, additionalArguments])」
「List/Set.at(itemNum)」
「aTbRef’s naming conventions」
「String.replace(regexMatchStr, replacementStr)etc.etc

There was a response to ja.qmd only once, “Coincidentally?”
The contents were only the information of “html export” was added.
For this reason, in the evening, I was going to give up the approach it.

After seeing your “how to approach” in your exercises,
I am determined to make even more efforts in the future.
I am very grateful to you.
Yours, WAKAMATSU

18

Dear Mr. Wakamatsu

One thing to keep in mind when using runcommand is that, unlike the normal terminal, the PATH of the command is limited.

var:string vCmd="echo $PATH;";
$MyString=runCommand(vCmd);
$MyString  -> /usr/bin:/bin:/usr/sbin:/sbin (in my case)

I think quarto is installed in /usr/local/bin, so when you call it, you need to set for PATH before calling quarto.

export PATH=/usr/local/bin:$PATH;quarto preview

But, calling quarto preview directly from Tinderbox did not work.

If you are calling a command that does not stop, the next runcommand waits for it. Also, exporting notes was not executed.

When quarto preview is run, the process remains(for the browser), so if you want to edit and preview again, you have to kill the process from the terminal.

After my trial and error, I was able to get the PDF and browser display to update automatically when I overwrite the qmd file from Tinderbox by running quarto preview in the terminal in advance and displaying it in the browser.

19

I think it is now clear that if using runCommand() you don’t need to involve the Terminal as both call directly to the macOS Unix shell.

I looked at your original TBX and there were a lot of mis-configuration issues:

  • the example note had no export template (I added one)
  • the example note had been set to not export! (I re-enabled that)
  • there was an export command set that could not run (I removed that.

Now the note can be exported and make a ‘note.qmd’ which I’m exporting to my home folder (~). As I have now got the homebrew Quarto installed, I used this command in Terminal (note how I first test in Terminal):

quarto preview note.qmd

and get feedback like:

Starting python3 kernel...Done

Executing 'note.ipynb'

pandoc 
  to: latex
  output-file: note.tex
  standalone: true
  pdf-engine: lualatex
  variables:
    graphics: true
    tables: true
  default-image-extension: pdf
  toc: true
  toc-depth: 3
  number-sections: true
  
metadata
  block-headings: true
  title: My title
  author: 苄束äč…仁慉
  lang: ja
  jupyter: python3
  documentclass: ltjsarticle
  classoption: 'lualatex,ja=standard'
  citations-hover: true
  footnotes-hover: true
  bibliography:
    - ./references1.bib
  editor: visual
  
File ./references1.bib not found in resource path

So, while the command failed as I don’t have any of your Python customisations installed, but it indicates that quarto is basically working. Next I added a stamp:

$Text(/log) = runCommand("quarto preview note.qmd");

Running this in Tinderbox returns this message:

/bin/sh: line 1: quarto: command not found

Clearly the environemnt variables don’'t have quarto in the PAT, but this command teels us where Quarto lives:

which quarto

which returns:

/usr/local/bin/quarto

So the stamp becomes:

$Text(/log) = runCommand("/usr/local/bin/quarto preview note.qmd");

which now runs quarto on the command line as desired but fails due to missing components (python, etc.) that are nothing to do with Tinderbox or runCommand().

I note that whilst drafting this, @sazanamix has added some useful information re the shell PATH variable.

Here is your file with the fixes to the export: quarto-test -1.tbx (225.5 KB)

If you export ‘note’ to your home folder and run the stamp, I believe quarto should compile for you.

20

Dear Dr.Mark Anderson & Mr.Takeshi Otsuka,
Thank you for all your support.
I tried using the revised quarto-test-1.tbx.
From the terminal first by the method pointed out by Mr.Takeshi Otsuka
I started it, and once it was finished, I ran a stamp on Tinderbox.
Append to quarto-test-1.tbx and export note.qmd
Run Stamp again.
For the most part, I was able to confirm that it works up to Preview.
From now on, it will be time for daily flute practice in the morning.

I would like to express my sincere gratitude to you for reporting
that the function you taught has been fulfilled.
With warm regards,WAKAMATSUAugust 23, 2023 11:03:39 JST