I’ve been using UI Scripting with AppleScript for a few years, and have an app that works great under High Sierra though Big Sur, but was having issues under Monterey. What I have discovered is either a new security feature or UI scripting bug.
So under normal situations I can access UI Elements in apps just fine, however when the screensaver is active or the display is asleep, these elements are no longer accessible… I get an error stating that the element cannot be found. ei, “text 1 of group 2 of window 1 of application… etc.”
This not normal behavior on Big Sur or earlier.
Specifically my app reads Notification banners and alerts me of certain situations. At first I thought it was Notification center blocking access, but then I tried accessing an UI element on a window of an app that I also normally access but that also failed with an error. Again only when the screensaver was active or the display was asleep.
Anybody out there willing to confirm my findings? If this is a new feature if Monterey, then my app is doomed. Or I’ll have to work around it somehow. Ugh.
I cannot confirm general problems with UI-Scripting related to screensaver or sleeping display. I just did some testing with Safari in a repeat loop and didn’t notice any abnormalities during the screensaver.
However, I can confirm that there are significant issues or “changes” (especially with the ControlCenter) since the update to Monterey.
One example:
tell application "System Events"
tell application process "ControlCenter"
repeat with theElement in menu bar items of menu bar 1
if (exists attribute "AXIdentifier" of theElement) then
if value of attribute "AXIdentifier" of theElement = "com.apple.menuextra.wifi" then
tell theElement
perform action "AXPress"
end tell
log (count of entire contents of window 1)
--> 0
set theElements to entire contents of window 1
log (count of theElements)
--> 0
set theElements to entire contents of window 1
log (count of theElements)
--> 9
end if
end if
end repeat
end tell
end tell
This is not a time problem! One can wait infinitely via delay-statement. The first query for the count will always answer with „0". It seems that the Reference is set only on the second set-statement. Without the Script Debugger I would never have thought of it!.
Another problem is that the ControlCenter in some situations does not respond to click events from the script (without any error). Not even on clickAt events!
You can try to assign the UI elements (step by step) to variables beforehand, and possibly multiple times. Maybe this helps. For me it helped.
Note: To run the example, the WLAN menu must be enabled in the system settings.
I’m still seeing the display sleep / screensaver issue. I’ve tried several things that work until the screensaver kicks in. Try setting screensaver to come on after 1 minute and then make a script that reads the value of any app’s text or button element. It will work when the screen is awake, but then add a 90 sec delay to the start of the script and let the screensaver activate. When you waken your computer from the screensaver after the 90 seconds you will see that the script failed.
This worries me since I have have a great app that works using an idle loop and runs all day in the background. It fails when the screensaver is active or the display is asleep.