Understand that for sure - haven’t gotten around to that one myself. I wasn’t thinking anything too fancy (no Accessibility framework), just simple image matching - i.e. getCoordinatesOfImageOn Screen(path_to_PNG). All that would do is let you know you’re clicking what you intend to click.
Very much so – it’s a common intermediate AppleScript question. Don’t know if anyone’s actually released any tool to get the current keyboard modifiers.
For some reason, keystroke sends the numbers as key codes for the number pad area of a full keyboard, rather than the number keys in the row above the QWERTY line.
I was getting intermittently garbled output when using keystroke with a remote desktop application, because as it turned out, the output depended on the state of the Num Lock key in the remote machine, which wasn’t consistent between connections (or visible/controlled by me).
So, it was toss-up whether keystroke "4" would type a 4 key or press a left arrow key. It took me a long time to figure out what was going on.
I wrote a little handler that fixed this, and also allows key commands to be pooled. For example:
-- Type "Testing123", wait 0.2 seconds, press the Escape key, press command-w
my sendKeys({"Testing123", 0.2, 53, {"w", {command down}}})
There are other issues, though, including that caps lock being enabled affects the output, and the fn key isn’t handled. Worse, perhaps, is that physically pressing a modifier key on the keyboard (such as command) will affect whether keystroke "q" types a q or quits the frontmost application.
You’re correct! It’s possible with ASOC. I’ve had the executable around since before ASOC existed.
I do still think it would make sense to bundle this with the other mouse actions in FastScripts, and have it return AppleScript-style coordinates (rather than the inverted y-axis that requires further calculation & getting the screen size).
Yes, that works, but also requires ASOC, and as you point out, AppleScript doesn’t have useful operators for bitmasks - it’s much easier & faster in ObjC or Swift.
This is a really common question for intermediate AppleScript users, has no native AppleScript implementation, and is useful in a lot of contexts (e.g. checking for modifier key release before using the built-in keystroke, or checking that no modifier keys are pressed before launching an application, etc.). For that reason, I think it could be useful for Daniel to include in FastScripts, especially if he adds a keystroke-like command.
Yes, almost anything is possible with ASOC! But the main emphasis of these features in FastScripts is to facilitate easier, more streamlined workflows. Similar to the regular expression support, which can also be done through ASOC but which is made way simpler by FastScripts.
For the modifier flags I envision FastScripts returning a list of enumerators, so testing for command key for example would look something like:
current keyboard modifiers contains command
I can definitely see how this would be a useful tool to have “at hand” for FastScripts users.
Regardless, the issue with both of the above solutions is that they require reaching for ASOC, which can have a 200-400 ms delay on first run as the bridging files are loaded (which is more of a problem with GUI scripting). This delay is experienced every time you run using osascript from the command line or using the native Script Menu, since they spawn a new process each time they are invoked.
As far as I know, FastScripts is the only script runner-type application that preloads the bridging files to prevent this delay.
Exactly – that’s why I think these make sense.
That looks great. keystroke has enumerations for command, option, control, and shift (no caps, fn key, escape key), but I can’t think of any good reason the codes or names would need to match, even if you did end up creating your own version of keystroke.
Oh yeah I’m sure that Daniel’s implementation of pure AppleScript functions in FastScripts will be more user-friendly and convenient than dealing with ASOC.
This version includes the requested additions for “current keyboard modifiers” and “current mouse position”.
It also exposes an optional parameter on click mouse to control the “cadence” of event dispatch. I mentioned earlier that events are dispatched with a bit of default stagger time, 0.01 seconds by default. The “with event cadence” parameter allows you to specify a different cadence either because you know it can be done faster, or (more likely) you discover it needs to happen more slowly.
Mouse Automation is a very exciting addition to FastScripts. I write many AppleScript scripts to automate various tasks. This involves a lot of GUI scripting. I will re-write some of my existing scripts to incorporate FastScript’s mouse automation and get back to you with my comments.
Based on my experience I can tell you that the addition of the ability to specify event cadence is very important. Moving the mouse programmatically frequently occurs faster than the GUI can accommodate.
I have two suggestions for additional features.
The execution of a GUI script can be “broken” if the user manually moves the cursor (i.e. using the mouse or trackpad.) I have attempted to learn how to disable user input during the execution of the script, using AppleScriptObjC, but have yet to succeed. This would make a nice addition to FastScript’s mouse automation; or if anyone knows how to do so, I would appreciate the information.
Going along with the above would be the option of hiding the cursor during execution of a GUI script. It might also make the script’s actions seem more “natural,” i.e. more like a built in function. I have attempted to use NSCursor’s hide: method. However, the cursor is unhidden when (I believe) the script is run by another application, in my case, FastScripts.
I have not tried making the script into an application in order to keep the cursor hidden, but it would be a nuisance to need to make every script an application.
Thank you, Daniel, for continuing to support and develop FastScripts. I have used it for such a long time that I don’t even recall when I began to do so! It is incredibly useful. And you nearly provide it at no cost. My cost per usage amount is sensibly zero. If only all the applications I use were of its quality.
@dmbrown Great to hear that you will also appreciate the enhancements! Thanks for the kind words about FsstScripts, it’s been a real joy for me to be able to maintain and extend it over the many years (decades!).
Interesting suggestions for additional enhancements, I’m not sure if I can “block out” the user during the event automation, but I’ll look into that possibility! I will also look into the possibility of hiding the cursor. Could be difficult since other apps can also show it proactively, but since FastScripts (or more accurately, its script runner) will be actively running throughout the process, maybe it can aggressively re-hide it and hope for the best.
Daniel - I’ve been working with FastScripts’s Mouse Automation. The cadence is an issue, even using “with event cadence.” For example, have a Script Debugger or Script Editor document open and arrange all your windows such that when the cursor is at {100, 100}, it is over the Desktop. Now execute the (nice and elegant) script:
tell application “FastScripts” to click mouse {100, 100} with event cadence X
where the value for X can be anything you want.
Clicking at {100, 100} should bring Finder to the front, but it does not. The following works:
tell application “FastScripts”
move mouse to {100, 100}
delay 0.1
click mouse
end tell
From what I can tell, it appears that “with event cadence” is only applicable to dragging. I think it would be good if “with event cadence” was applied between the two parts of “click mouse”, the move to the location and the subsequent click. This would eliminate manually adding the delay.
As you know, GUI scripts are frequently peppered with short delays. I have been writing GUI scripts for a long time. When I get a new computer, some of the scripts need to have their delay’s increased, presumably a result of the new computer being faster than the last. I think it would be useful to have the ability to set the default cadence in FastScripts’s Settings. This would allow users to set a default that is compatible with their computer and usage.
Even more flexibility would be to also have the ability to set the cadence as needed, such as
tell application “FastScripts”
set default cadence to 0.1
…
set default cadence to 0.5
…
end tell
Thanks for the feedback - I’ll have to look at how I can improve the cadence behavior. Don’t let me ship with this broken! If i neglect to fix it perfectly please let me know I did have the idea to offer a script-level setting like “default event cadence” … that would give FastScripts the ability to infer how much to slow down in a variety of circumstances.
I think that’s a great idea, provided you can make the default stay contained within a given script (otherwise, I can see it causing issues if some scripts explicitly set it but others don’t, and end up inheriting different default values). It would be very efficient for anything involving virtual machines / remote desktop connections.
I agree completely. I’ve done a fair amount of GUI scripting myself and the delays required when controlling the mouse programmatically are often longer than you’d expect. Some applications need the mouse to hover in place for x number of milliseconds before clicking, others need the mouse movement itself to take x number of milliseconds to move to the point you want to click.
This update includes a new script property: “user interaction disabled”. If you set this on “me” in your script:
set user interaction disabled of me to true
FastScripts will block all input events until the script finishes running, or until you set the property back to false.
This is a little risky since a bug in the script, or in FastScripts, could lead to input being blocked to the extent that a user can’t cancel the script or do anything else with their Mac. I’ll have to think about how to mitigate that, but I thought it would be worth sharing with the “bleeding edge” testers here for now.
Note: I wasn’t able to prevent “Mouse moves” from happening. But at least the user can’t interfere by clicking or keyboard pressing while the interaction is disabled.
As far as the “click mouse” situation described by @dmbrown above - I think the problem there is that I’m not doing an implicit ‘move mouse’ before the first click. I am injecting a “cadence” delay between each event, but in the case of “click mouse” it’s currently just clicking at the designated spot without moving there first. Adding an implicit move should introduce the desired cadence delays, I hope!
When looking into disabling user input, I thought of the same risks. I intended to implement the disabling in such a way that prior to doing so, a separate background application would be started. Its only purpose was a “fail-safe” timer; in 30 seconds it would check if input was disabled and, if so, re-enable it. The GUI scripts I use would not need input disabled any longer than that. Perhaps that gives you some ideas.
I never figured out how to disabled user input, so I can not tell you how the envisioned “fail-safe” performed. I am interested to know how you did it, if you are comfortable divulging such information.
Thanks, you’ve given me an idea that I think is good: I will change the property to a command and offer an optional parameter “with timeout” or similar, so that the process for disabling user input will be something like:
disable user interaction with timeout 30
Which would achieve the goal you described of having user interaction automatically restored after 30 seconds.
As for how I achieved the disabling of interaction, I’m using a feature of the OS called “event taps” which are designed to both intercept and suppress events at the “HID” level - that is, where the events enter the system, more or less.
That looks good. This looks like it will work even if e.g. Script Debugger is running the script during development/debugging.
With the previous implementation (using a script property), I’m assuming that FastScripts had to be the application running the script?
When it comes to the “default cadence”, I think you’ll have the same issue? If it’s a script property, then the default cadence will only work when FastScripts is running it, not during development/debugging (of which there’s a lot with GUI scripting) in Script Debugger. That’ll be more of a problem, since it’ll affect timing in the script.
Will the “with timeout” command launch a separate application as the “timer”? The idea being (perhaps obvious) that even if the the script editor, a script application or FastScripts itself hangs or crashes, the independent timer application will live on.
I appreciate the information about event traps. I was looking into using NSEvents somehow, but never did get very far.
The way I’m approaching it now, the user interaction is disabled by the script runner process itself, which means that it actually won’t work when run from Script Debugger or another launcher besides FastScripts. I’ll have to think about that a bit - it would obviously be nice for the functionality to work when run from outside of FastScripts, but bundling the behavior up with the script execution itself has some advantages.
Whatever I end up doing, I don’t think there needs to be a separate app to “monitor” the suppressing process. The way it works, events can only be suppressed if the event filter is actively running, so I could build into the suppression code a timeout value that it would check on its own and uninstall itself when the time is right.
I did have the idea to offer a script-level setting like “default event cadence” … that would give FastScripts the ability to infer how much to slow down in a variety of circumstances.