My automated "click" Applescript has just stopped working

AppleScript
,
#1

Right up until a week ago, my simple Applescript that had to simply click on a specific coordinate on screen just stopped working.

Here is the Applescript:

tell application "System Events"
click at {693, 619}
end tell

And I automate it’s trigger at a specific time of day with the help of osascript (recommended for Applescripts) using the following plist file.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.mouseclick.plist</string>
    <key>ProgramArguments</key>
    <array>
        <string>/usr/bin/osascript</string>
        <string>/Users/mugenvoid/Downloads/mouseclick.scpt</string>
    </array>
    <key>StartCalendarInterval</key>
    <dict>
        <key>Hour</key>
        <integer>6</integer> <!-- Adjust the hour as needed -->
        <key>Minute</key>
        <integer>10</integer> <!-- Adjust the minute as needed -->
    </dict>
</dict>
</plist>

I’ve added this plist file to the LaunchDaemons folder of my core Library folder. This will ensure it’s a global daemon, so it should have the highest chance of always triggering correctly. Beyond that, I’ve given all the relevant access and permissions (Accessibility, Automation, disk access etc) for this setup to work correctly.

To ensure the job is loaded correctly and functional, I use an application called Launchcontrol 2. Over there, it is always loaded in correctly and the status shows as OK.

Despite all these checks and balances, the script just doesn’t click on the specific coordinate no more. It was always working perfectly fine up until a week ago and I haven’t made any changes whatsoever to this whole setup.

The Launchcontrol 2 application has some launchd(8) logs of the job trigger event at the designated time that I would like to share. Perhaps it will help with the debugging. Here it is:

2024-10-22 06:10:03.808584 <Notice> internal event: WILL_SPAWN, code = 0
2024-10-22 06:10:03.808688 <Notice> service state: spawn scheduled
2024-10-22 06:10:03.808692 <Notice> service state: spawning
2024-10-22 06:10:03.808727 <Notice> launching: xpc event
2024-10-22 06:10:03.810543 <Notice> xpcproxy spawned with pid 28683
2024-10-22 06:10:03.810593 <Notice> internal event: SPAWNED, code = 0
2024-10-22 06:10:03.810599 <Notice> service state: xpcproxy
2024-10-22 06:10:03.810718 <Notice> internal event: SOURCE_ATTACH, code = 0
2024-10-22 06:10:03.827675 <Notice> service state: running
2024-10-22 06:10:03.827703 <Notice> internal event: INIT, code = 0
2024-10-22 06:10:03.827709 <Notice> job state = running
2024-10-22 06:10:03.827726 <Notice> Successfully spawned osascript[28683] because xpc event
2024-10-22 06:10:10.104597 <Notice> exited due to exit(0), ran for 6293ms
2024-10-22 06:10:10.104617 <Notice> service state: exited
2024-10-22 06:10:10.104630 <Notice> internal event: EXITED, code = 0
2024-10-22 06:10:10.104633 <Notice> job state = exited
2024-10-22 06:10:10.104665 <Notice> service inactive: com.mouseclick.plist
2024-10-22 06:10:10.104681 <Notice> service state: not running

To the best of my knowledge, I’ve checked everything possible from my end and am still faced with this roadblock. I’d really appreciate your support here!

Condensed view of what I’ve tried so far:

  • Under the Privacy & Security settings section, I enabled Accessibility for osascript, launchcontrol and the script editors I’ve used for the Applescript and the plist file. See image attached → https://i.sstatic.net/XWDVLn7c.png
  • Under the Privacy & Security settings section, under Automation, the requisite access has been given to Osascript and relevant applications. See image attached → https://i.sstatic.net/7AfVqYSe.png
  • Ensured the job is loaded correctly with no errors and that it is loaded in as a global daemon (by adding the plist file to the LaunchDaemons foler in my core Library folder). See image attached → https://i.sstatic.net/yk7U9ls0.png
  • Verified the script file and plist file and found no syntax errors (code used for both has been shared above).
  • Ran the script file from the Script Editor itself to confirm if it’s working fine and got the following results. See image attached → https://i.sstatic.net/tRea9jyf.png
#2

Quick note for anyone clicking the image links: you need to copy the link itself and open in a new browser tab; clicking the link itself produces an error message.

#3

Thanks for calling that out for all the readers. This was my only option to link as many images as I needed because being a new user, I can’t attach more than one image embed per post unfortunately.

Coming back to the main topic, would you have an idea as to why my AppleScript has stopped working since a week ago?

#4

Hey everyone,

Bumping this up. Any ideas on what might be causing this?

For more context, the click is supposed to happen on a Google Chrome window that I keep fully active and in the foreground in the moments leading upto the trigger in order to ensure nothing breaks.

And still the script fails to trigger properly and actually click…

#5

So is the script itself not working anymore when you’re running it, say, from Script Editor? Or it’s only the daemon that’s not working?

#6

I can’t say which is not working, which is what I’m trying to find out.

When I execute the script on the Script Editor, it displays results as I’ve highlighted in the screenshot I’ve attached in my original post. Sharing the image here again.

image
image1406×1453 121 KB

So it doesn’t look like the script is malfunctioning (atleast that’s what I think).
And therefore, the suspicion now falls on the daemon/automation not working properly.

But in all honesty, it could be either like I mentioned above. The Script Editor results are surely there to stay, but nothing gets “clicked” when I execute the script.
The automation does execute at the designated time according to the launchd(8) logs but no click happens.

So yea, I’d appreciate some help in understanding where the problem lies.

#7

To check if the script itself is working, specify a coordinate at which a clickable UI element is located (you can get the coordinates by using the Cmd-Snit-4-drag approach; I’m sure other ways exist, too).

I’m pretty sure your script is working fine.

And what is located at {693, 619}? Are you sure the coordinates of the desired element didn’t change?

#8

  1. What changed in your system a week ago ?
    e.g. New updates to macOS or Chrome installed ? Changes to any system settings ?

  2. How do you know your script stopped working ? What is not happening now that was happening a week ago ?
    i.e. What does the script actually do ? It clicks on a Chrome window for what purpose ?

This isnt necessary, as you can direct click commands at specific processes and UI elements, which, apart from being more reliable than coordinate-based targetting, means that the Chrome window only needs to exist, but needn’t be at the front or even visible to you (although it shouldn’t be minimised or off-screen).

#9

click has unfortunately had some reliability issues with coordinates.

You can use the very useful command-line utility cliclick (i.e. CLI-Click) from AppleScript to click at specified coordinates. It’s available on Homebrew.

to clickAt(global_coordinates)
	set coordinates_s to "=" & ((item 1 of global_coordinates) div 1 as string) & ",=" & ((item 2 of global_coordinates) div 1 as string)
	do shell script quoted form of CLICLICK_PATH & " c:" & coordinates_s
	return global_coordinates
end clickAt

-- Example usage.
my clickAt({123, 456})

Just make sure you replace CLICLICK_PATH with the path to the binary.

#10

Any details about the nature of the problem and the affected OS versions ?

#11

Confirmed that the script works just fine.
When I “start” the job in Launchcontrol, I’ve confirmed it will perform a click at the UI element present in the specified coordinate.
I made sure to position the application such that a interactable UI element was present at the configured coordinates.

As for what is located at {693, 619}, I mean to click a “Save” button at those coordinates on a certain website. I use the Chrome browser to visit that website and keep the concerned page in my active view, cause the click will only target active applications.

This setup used to work perfectly fine without any hitches up until a week ago. Didn’t change a thing.

#12

I didn’t change anything in the script configuration and if I recall correctly, my system software hadn’t been updated yet back then.
So I have no clue what changed. Could be the new system update most likely, eventhough it happened afterwards.
Another hypothesis is that more than a week ago, I added another script that clicks a different coordinate at a separate time. But I don’t see why this should be a problem at all.

As mentioned, the click is not happening anymore at the specified coordinates at the given time.
I have two separate scripts that click on two different elements of a webpage’s UI at different times. Both of them are not working anymore.
I’ve confirmed through Launchcontrol that when I start the job ad-hoc, the click does happen at the specified coordinates. But when the jobs runs automated at the given time, the click just doesn’t happen for some reason.

I am still rudimentary in my knowledge of Applescript, which is why I could only come up with this simple code. And it was working perfectly fine till now, so I didn’t see a reason to add more complex targeted code.

I’m somewhat aware of how to bring an application to the forefront, but can you tell how I can go about targeting a specific UI element in that application?

#13

I did come across this suggestion when I first started scoping out the feasibility of automating a click at a specified time and at a specific coordinate.
But chose to go forward with the simpler option because it started working for me just fine.

Now that I’ve invested some time into figuring this problem out, I’m more keen on arriving at a solution with my existing setup intact.
I’m still stunned as to why a perfectly functional setup would just stop working out of the blue for seemingly no reason.

#14

FWIW, the System Events click command generally works as expected, except when the target coordinates are within a web page that is displayed in Google Chrome. For example:

--this script run by way of the Script Menu on Sequoia 15.0.1

--this works
tell application "System Events"
	click at {612, 153} --the "Macscripter" item in the Bookmark bar of Google Chrome
end tell

--this doesn't work
tell application "System Events"
	click at {690, 270} --the "Latest" button on the Macscripter site displayed in Google Chrome
end tell

--this doesn't work (but does work with Safari)
tell application "System Events"
	click at {515, 147} --the "Google" link on the "Google" site displayed in Google Chrome
end tell

--this works
tell application "System Events"
	click at {25, 15} --the Apple icon
end tell

--this works
tell application "System Events"
	click at {1730, 644} --the Open button of the File > Open dialog of TextEdit
end tell

--this works
tell application "System Events"
	click at {954, 563} --the More Info button of the About This Mac menu item
end tell
#15

Thanks for the reply!

So far until a week ago, the clickat command was clicking stuff within Chrome just fine.
But now no more.

Would you have any idea how to proceed further from here?
I’ve run out of ideas on how to make this click automation work. I just want to click at certain coordinates at certain times of the day. Never realized it would be this complicated…

#16

Just to confirm, I tried to click in a web site in Google Chrome with a script run in Script Debugger, Script Editor, FastScripts, and the Script Menu. This failed in all instances with an assistive-access error. However, all four of these as well as Google Chrome have assistive-access permissions set on. I don’t know how to fix this.

You could try Safari, which works with the script in my testing. I don’t have any JavaScript skills but that might be an option to investigate. I agree that this is much more difficult than it should be.

#17

Interesting. I tried a click on Chrome from Script Editor and it worked here on Ventura.

But regardless, you can also try Keyboard Maestro (not free but a must-have anyway, in my view).

You can create a macro to click at a specific location AND specify the time for running the macro (as well as many other options).

#18

Salazarian. The following script will click on a button of a web site displayed in Google Chrome. This script only works with some web sites, but if it’s not private, you could post the web site you’re using and button name, and I could see if this script can be made to work.

--click on "Log In" button
--https://www.macscripter.net/

clickClassName("d-button-label", 0)

to clickClassName(theClassName, elementnum)
	tell application "Google Chrome"
		execute tab 1 of window 1 javascript "document.getElementsByClassName('" & theClassName & "')[" & elementnum & "].click();"
	end tell
end clickClassName
#19

Thanks for all the inputs guys!

I managed to get it resolved. Turn out that for some reason, the Google Chrome UI wasn’t clickable by “System Events”. When I opened the same webpage where I needed the click to happen on Safari, everything worked fine and dandy like before.
So it seems like the core issue here resided with Google Chrome. Some recent update must’ve made it’s interface non-clickable by “System Events”.
That’s it! Nothing was wrong with the script or the automation, it was Google Chrome all along.

#20

It’s been a while since click was a consistent issue… I want to say around macOS 10.9 or so.

However, it still doesn’t always play nicely with applications that don’t use standard macOS accessibility controls (remember, click returns a UI element which is dependent on inspectibility within the Accessibility frameworks). Anecdotally, it seems like 3rd-party applications that don’t use the standard Cocoa UI are more likely to have issues.

Case in point.

The reliable solution is to use either the low-level C API (via an executable like cliclick) or ASObjC (with FastScripts if cold-start performance is important). FWIW, I’ve done extensive work with GUI scripting in some weird environments, and cliclick with the above handler has been flawless for me for the past decade.