I’ve noticed that when I am actively using Tinderbox or have it open in the background for an extended period that after about an hour or so I’m alerted by a message that I am reaching the maximum memory of my RAM and Swap file setup. It happens most often when I am setting up a new file or re-configuring prototypes in an existing file. This causes me to stop work save and close all apps and run an app to free up RAM. I’m using a 14" MacBook Pro (14-inch, 2021) with macOS Monterey Version 12.5, Chip Apple M1 Pro, Memory 32 GB. This problem may be caused by some other process but Tinderbox is so far the only common denominator. Does anyone have recommendations for a default set up my computer when using Tinderbox to reduce these occurrences or any knowledge of other apps or Apple background processes that may be causing this problem?
I don’t have an M1 Mac, but there were reports of a system level memory leak,eg here. Couldn’t tell you if Apple has fixed it.
I have the same MBPro age/spec and don’t see this. My main ‘drain’ are:
- macOS ‘WindowServer’ processes’ longstanding bug in its ability not to release resources properly when windows/app are closed. (FWIW, fixing this does need a full OS re-boot)
- Safari. Having many tabs open, especially media intensive sites. Closing/re-opeing and using menu History → Reopen All Windows from Last Session should purge Safari memory.
Whilst Tinderbox can use several GB of memory with big docs open, this is recovered when the doc is closed or failing that, closing the app. That said, it might be something to do with your specific manner of use in Tinderbox—a common mis-presumption is all users do the same. In fact, as Tinderbox is a toolbox, we often use different app features in different combinations so you might be doing something regularly that others don’t (do often).
I note this as if you are seeing excessive—as in much more than expected—RAM usage by Tinderbox my first reaction would be to save/close docs and then quit Tinderbox, check resource usage falls. Then re-open and look at usage levels. Note that just closing all TBX docs doesn’t necessarily release the app’s resource holdings. Why? Both OS and app will figure you’re likely to start over so holds back the resources for now (though the OS can grab them from the app is under stress—if all works correctly!).
Setting up a new file suggests to me a file with few notes so it is unlikely a cause of resource use. RE-configuring prototypes, though likely in a larger file, is also not obviously a user of resources.
How to disagnose further? For years, due to running external 4k monitors and the macOS WindowServer bug, I’ve always monitored CPU/RAM usage via the Apple ‘Activity Monitor’ app or more recently via ‘iStats Menus’ (see App Store). The makes it easy to see which apps are hitting the CPU or taking current (RAM) resource. Both offer a means to quit/force quit processes—though don’t force quit maacOS stuff without a little research first. Given that, re-booting your whole OS is an over-burdensome way of doing what re-starting the Tinderbox app will achieve.
So (via the above utils) keep an eye on ongoing resources by Tinderbox and try to narrow down use/file(s) that reliably trigger the problem.
If you find a particular document with excessive RAM usage, I’d contact formal Tinderbox tech support directly (tinderbox@eastgate.com) as they may want more info about your system and this is best not done though a public forum.
HTH 
I’ve experienced in the past. Often I’ll find that I have action code that is running amok and creating a lot of unnecessary processes or data creation elements.
Yes, this can be the case, though of course it is a user generated error. In this context, the secondary tab of the Tinderbox Inspector (here) can be your friend.
Another user-misstep that may use up a lot of resource is loose queries polling too many things. For instance you might want action code to ask notes to link to any note that, for example, contains their note in $Text. Simple, right? But, if your doc has thousands of notes and each note is now searching every other note and doing a regex-based .contains() operation on it, it actually takes time resource. In this case, much of the load can be avoided by tighter queries, i.e. run only on the notes that need to do this and them polling only those notes that are realistically target notes): both those lessen the overall impact. Also, don’t to these ops in always-on agents as it’s wasted CPU cycles if you’re running complex task that don’t need doing. I would stress I’m talking about large/complex docs, ‘my first TBX doc’ doesn’t need to worry about such things.
Turning off, even just to diagnose things, the ‘Always update agents’ on the File menu. also be aware that if you have a 5,000 notes document and make an agent that matches each, there are note c.10k, not 5k, objects to now search.
So, if you do thing an issue is action code malfunctioning or simply doing more work that you imagined, I’d look at the Agents and Rules Inspector and see if everything there is updating slowly, if at all. If so, you need to start disabling or removing action code to see where the glitch is. I generally use a stamp for any once only or occasional-but-complex stamp as I know it can only run once for each stamped notes. Or all other things—agents, rules, edicts—I try to add the code via prototypes so that it is easier to disable ot delete code if things get out of control
One question regarding this. My impression is, after changing action code fir an agent, the agent is executed immediately. Independent of the deactivated automatic execution. Correct?
Also, agents are executed when a file is opened?
I am asking, because I was fiddeling with a corrupt file today. It was a calculation error in the action code, which was executed immediately after updating the code and opening the file.
Changing the query causes the agent to run (when Return key is pressed or the query is enabled/disabled in the Action/query Inspector). Not sure if changing agent action does the same, but assume the new code is used for all $AgentAction calls after the change.
Yes (as are all edicts) but note disabled agents are not run.
Thanks for your feedback!
Thanks for these tips. I’m pretty sure you are right that it something to do with macOS windowing processes. I’ll use the Activity monitor as suggested.
That is the very thing I need to check because I am running various action codes on different notes in the files I’m using. Thanks
If you run external monitors—especially 4k ones—at any resolution other than their default zoom you will get a memory leak in WindowServer that can only be cleared with a re-boot. the bug’s been around in macOS since at least OS 10.14.
I have the same problem. And I almost don’t use any agents. Tinderbox in a course of several days increases consumption of Ram up to several Gb. Wrote about that on tech support e-mail, but recommendation was only to close tinderbox completely and reopen files again. If I remember correctly I got answer that they know about bug for long and will probably fix it somewhere in the future. So for a while the only solution is to close tinderbox completely every several days as was mentioned earlier.
Edit: added qualification on effect of queries as relating to the size of the TBX doc, to reflect @eastgate’s subsequent observation on this (which I agree!)
From the CPU/RAM monitoring apps I run, it is clear that for many apps—including Tinderbox—closing an app’s documents but not the app itself does not necessarily cause allocated resources to be freed up—so it appears. My understanding is those ‘unused’ resources may yet be re-appropriated by the OS if it runs short.
Regardless, closing re-opening apps is a habit I’ve acquired as the more I use then without a re-start, the more their OS resource allocation ticks up. Biggest offender? Web browsers. These days, unless you’ve active edit sessions in page(s), they all let to close and then re-open the previous windows/tabs. Again I commend using Apple’s Activity Monitor (free) or iStat Menus (££) if trying to get a bead on general performance.
It’s been some years, but I have occasionally had docs with seeming run-away resource use (CPU and/or memory) and those cases it generally boiled down to unintended user error. Often it was me writing overly expansive queries or overly complex actions, were I thought I was doing a ‘simple’ task but had configured it—albeit unintentionally—in a way that created a vast amount of work for the app.
This is a good observation, as if you’ve 100s of agent running it might be a prompt to revisit your doc’s structure. However, even a single agent (or action find(query)) can slow things to a crawl (clarification: in large TBX docs†), as a single agents query/task can kick off a significant amount of processing activity. We also easily overlook aliases. A query tests—within the scope of the query—all notes and all aliases. The fact agent queries only alias any note/alias group once doesn’t alter the fact that in getting to that point the number of objects to test may be large.
In some of my research docs, this happens regularly, but because—even with minimal and well designed agents—I’m doing a lot of work. Closing the doc releases some resource, closing the app releases it all.
I too initially thought all this is a bug, but I’m not sure that charged description (i.e. stupid/bad software). A more useful question to ask is is there a way the app can possibly detect the users asking it to do something needing too much resource rather than start the process and run out of road. As my experience increases, I think that’s a harder ask than we imagine.
If you’re only using one/a few agents, by all means make a new thread and post the queries/actions as we might spot some corrections/optimisations that might avoid such problems as you mention. If doing so, it would be useful to know the overall note/alias counts and the number of agents/rules/edicts as they are all factors in the mix.
HTH
†. I should clarify that in a small document, the effects of queries (assuming valid syntax) are not noticeable. The step change can occur if rather than 00s of note you have 000s or 0,000s of notes.
The current status is that we don’t know of any memory leaks. But leaks do occur from time to time, and when we find them, we do our best to resolve them. (As may be the case here, leaks that arise from unusual situations, such as unparseable actions, are especially tricky to discover.)
Note that (a) most people can work without difficulty even when some apps have a large memory footprint; (b) the memory footprint reported by Activity Monitor is frequently an over-estimate; and (c) the workaround of closing and reopening Tinderbox on occasion is really not terribly onerous in most cases.
But, if you have a document in which memory usage seems to increase indefinitely or unreasonably, by all means send it here so we can take a look.
A note on find() “slowing things to a crawl”.
Keep in mind that @mwra is working with some huge Tinderbox documents. By definition, find examines every note. If the task you’re performing is complicated — searching for a regular expression with multiple wildcards — that means doing something complicated many thousands of times, every few seconds.
My advice is, don’t worry too much about performance until you find yourself with a performance headache.
Thank you for your detailed reply! Also thank you @eastgate
Looks like I need to try to understand this issue. So, I looked at my main document that contains 1367 notes, 139 aliases, 89 adornments and nil agents. Nevertheless, I see 195 rules in Inspector and didn’t even suppose that I have that much. Sorry for stupid question, but how can I find those notes with rules? I have examined every prototype and didn’t find even one prototype with rule (I have such in other documents).
Many times I tried to figure out how to use agents and didn’t find useful way. Basically, I use Tinderbox to broaden and deepen my knowledge in specific complicated topic, and containers, adornments and links do all the work that I need. As I understood people use agents mainly when they need to deal with constant flow of routine information in which agents can find something particular of interest or even some patterns. But when you deal with complicated non routine information which you immediately should classify somehow (by container and adornment and some links) I still can’t see how to use agents. Even for searches which I constantly do. Simple search not by agent is usually enough. Because it is not persistent searches.
Yes that document which I mentioned in previous post starts from 300 mb of RAM and usually goes up to 6-7 GB in several days when I notice it in iStat. Once it consumed around 14 GB of RAM which I couldn’t believe because I have only 16 GB of RAM.
An agent :), but before we come to that, why might there be a lot of rules? Prototypes for a start. If the prototype has a rule, that is inherited by all notes using that prototype. If the prototype is used by 100 notes, then there will be 101 (the ‘1’ is the prototype) rules from that alone. Or if you duplicate a note with a rule, you now have one more rule.
How to find notes with rules? An agernt with this query
$Rule
or in longer form:
$Rule!=""
We are asking the agent to alias any note where the value of the attribute named Rule is not the default value of an empty cell.
I know your doc has no prototype rules, but to round out the above. To find rules that are not inherited form a prototype:
$Prototype=="" & $Rule!=""
This first checks the note doesn’t use a prototype and then checks for a rule. In the earlier example of a prototype with 100 inheritors (and there only being one prototype in use) then if the first agent found 105 items, the second would expect 4 (105 - 101). These would be where the rule had been set ‘locally’ in the note and not inherited.
FWIW, if you’ve 195 rules and lots are similar that is a strong hint you might want to use a prototype, even if only as a way to ensure all the notes are using the same code and not minor (unintended) variations.
We just found one! But seriously, if you don’t need agents, you’re doing fine. Tinderbox is a toolbox with more tools than any one of us likely needs. So different users push hard on different sub-sets of tools in the box and that’s fine.
Rules are always-on, running continuously. Think of the rules as a list of tasks: as soon as the last is run Tinderbox immediately starts the list again. That’s a slight-oversimplification as Tinderbox needs to allow gaps for other things (e.g. running agents). However, it helps show that what rules do they do incessantly. Thus the effect of a small glitch my compound rapidly. It also means that the cause of the individual glitch may be hard to see in the moment (as it’s small!).
Thank you very much!
Actually I missed two prototypes which was found by your query that contains two extremely simple rules about which I forgot. They are:
if($Checked=="true") {$Container="/archived_tasks"; $Color="green";}
if($Checked=="true") {$BorderColor="green";$Border=5;}
First prototype contains first rule and second — second. So I can’t see how it can have such effect on RAM.
Both those rules have no syntax issues and as you say are simple. I doubt these are the cause of anything troublesome.