By the way… I’ve noticed on the latest version of Script Debugger (6.0.4) this will now actually crash the application, even if it isn’t running in debug mode.
(Crashing on exception: Error -2706 received from OSAID instance)
I have another customer with a massive script that is experiencing a similar crash. I suspect something has changed in AppleScript so that it is reporting errOSAInternalTableOverflow errors in places where it did not in the past. Please try this pre-release build and let me know how it goes . If the crash persists, please submit a crash report.
OS X’s Gatekeeper is asking me to move the app to the Trash. Is this to be expected?
“Script Debugger Lite” is damaged and can’t be opened.
I would like to ask a general question regarding stack overflow.
I encounter Stack Overflow message and hangs in SD occasionally when I am in debug mode for long scripts (>1000 lines of codings excluding spaces, comments and definition of properties and variables) that involves manipulating the properties (including the plain text content of the records) and arrays of >1000 records of an application.
Is there any way to tweak the system ( e.g. clear the system memory) to prevent the Stack Overflow from happening?
Thanks
Absolutely – for a larger stack, just choose the -l JavaScript switch on the osascript command line.
If we destructively test the stack size running JXA for Automation in VSC or Atom, we get:
[31721, Maximum call stack size exceeded.]
from the following code:
(() => {
'use strict';
const go = i => {
try {
return go(1 + i)
} catch (e) {
return [i.toString(), e.message];
}
}
return go(0)
})();
Script Editor turns out to allow for a much smaller stack space (about one order of magnitude).
["3032", "Maximum call stack size exceeded."]
for the same JS code.
But even that is drastically shrunk by using -l AppleScript rather than -l JavaScript
We get only:
"Recursion limit encountered at 502"
from:
-- recursionDepth :: () -> IO String
on recursionDepth()
script go
on |λ|(i)
try
|λ|(1 + i)
on error
"Recursion limit encountered at " & i
end try
end |λ|
end script
go's |λ|(0)
end recursionDepth
on run
recursionDepth()
end run
(Script Debugger and Script Editor give the same result for this code)
AppleScript has many charms, but a large stack is not, alas, among them.
In short, no, there isn’t.
Thank you @ComplexPoint. But I’m afraid I don’t understand a quarter of what you explained? I worked on MS VBA 28 years ago (Excel and Access mostly), and that’s why I am comfortable to write a long script with lots of array’s manipulation. But I have only picked up programming again during the last 12 months and only on AppleScript while starting to read Shane’s book on ASObjC more recently.
I am guessing what you and Shane are saying is that AppleScript has allocated much smaller memory space for some sort of table of pointers to keep track of what’s happening to variables in the program than the other language such as Javascript?
Thank you to both Shane and you.
The TL;DR is that the architecture of the AppleScript interpreter goes back to the early 1990’s and was designed to make use of only a small amount of memory.
It’s a delightful glue language for working on a miniature scale, but begins to error, intermittently or ineluctably, when:
- Code begins to grow long (c. 1000 loc +)
- Data structures grow large (as you have seen)
- Recursion goes deep (as above).
If you need to work with Apple Events, and with the Foundation, Appkit etc libraries, at at a slightly larger scale, then you may need to experiment with the JavaScript interface to osascript. (You can begin to the explore JS Automation library from Script Editor by switching the language tab at top left).
Pretty much. In terms of large lists, ASObjC often helps enormously.
Thanks again. In fact, I am using ASObjC more and more for arrays operations by coding examples from you, your book, and others in MacScripter. It seems the only limitation for ASObjC is that it can’t operate on list (or list of list) that contains reference objects of an application.
Thanks very much for the explanation.
To be fair, your script only starts counting a level or two down and i and (presumably) the error return address are stored on the stack each time. If the script’s rewritten this way …
property i : 0
on recursionDepth()
set i to i + 1
recursionDepth()
end recursionDepth
on run
set i to 0
try
recursionDepth()
on error
"Recursion limit encountered at " & i
end try
end run
… the result is "Recursion limit encountered at 734". Admittedly, though, that’s still a long way short of 31721. 
On the other hand, 734 return addresses isn’t very much actual memory at all and I wonder if the result has any real meaning. 
But it’s usually possible to write scripts in such a way that the limit’s never reached.
Thanks!
Can u kindly point me to where I can read/learn about such info?
Or it’s could be as simple as setting the lists back to {} after they are not used anymore or something like that? I never have issue (yet) in running the script, but stack overflow happened while I’m in the debug mode of SD occasionally.
In my case, I don’t use recursions much and only for less that 4 levels (for folders hierarchy) if they are used. I do have to manipulate list of lists that are usually less than 1000 rows and with less than 20 items in each row and of which some of the items are also a list, or records of an app, or a string with several hundreds to a few thousands words.
Not looking for a specific answer but a pointer to the right direction…
Usually true of the specific case of recursion, but not of code length or data size.
Chewing over results is the business of ideology – more revealing of the chewer than of what lies behind the results 
Experimentation is in the business of asking questions, and these questions are comparative:
What happens if we apply the same method to different languages and contexts ?
You feel that adjusting the method (but only applying it to one language) asks nature a useful question ?
(To be honest, the variation, even for AS in isolation, is still not great, and still reveals comparatively little space for a language to breathe in.)
(comparative being the relevant category)
Even in Script Editor, we do have much more resource to play with (code length and data size as well as recursion depth) if we reach for JavaScript.
You are doing very interesting work with the automation of DEVONthink, but your projects are now reaching the scale at which AppleScript becomes less reliable and less useful. Perhaps less relevant.
I would certainly recommend continued use of AppleScript for smaller things, not least because of the excellent Script Debugger product, but you probably would need to reach for a different tool if you wanted to pursue those larger and more ambitious DEVONthink projects.
(We would be misleading you if we gave the impression that scrupulously avoiding recursion would suffice to make AppleScript significantly more scaleable.
The shallow recursion stack is a useful and measurable indication of the limited memory available to the AppleScript interpreter, but it is not the only constraint).
PS you are far from alone in hitting these limits, there are various discussions on Stack Overflow (if Nigel will forgive the hint of something recursive here), for example:
I first became conscious if it while writing a large script for search and querying the OmniFocus database.
It impinged again with a DSL for quicker reorganising and formatting of nested diagrams in OmniGraffle.
JavaScript I first tried when TaskPaper moved the bulk of its scripting into a JS Context. Since then Omni group has done the same with OmniGraffle, OmniOutliner, and most recently OmniFocus, which, inter alia, allows them to provide rich scriptability for these applications on iOS as well as macOS.
JS rewards a bit of experimentation I think – a useful complement to AppleScript.