FaceTime Audio Call UI Scripting

I have a script to get the Caller from the FaceTime Audio Call notification. And it works just fine on Sonoma:

use AppleScript version "2.4" -- Yosemite (10.10) or later
use scripting additions

set theCaller to ""
tell application "System Events"
	if exists static text 3 of group 1 of UI element 1 of scroll area 1 of group 1 of window 1 of application process "NotificationCenter" then
		set theCaller to value of static text 3 of group 1 of UI element 1 of scroll area 1 of group 1 of window 1 of application process "NotificationCenter"
	else
		if exists static text 1 of group 1 of UI element 1 of scroll area 1 of group 1 of window 1 of application process "NotificationCenter" then
			set theCaller to value of static text 1 of group 1 of UI element 1 of scroll area 1 of group 1 of window 1 of application process "NotificationCenter"
			if theCaller is "from your iPhone" then set theCaller to value of static text 2 of group 1 of UI element 1 of scroll area 1 of group 1 of window 1 of application process "NotificationCenter"
		end if
	end if
end tell

return theCaller

But I upgraded to Sequoia…
UI Browser shows nothing except standard apple menu bar in application process FaceTimeNotificationExtension of application "System Events"

Temporarily I use a workaround with Keyboard Maestro, but it gives me not the exact caller names

Maybe ASObjC?

We’re in the middle of grappling with this very issue in another thread.

Sequoia has significantly changed how notifications are displayed, and it’s no longer possible to directly access their text through AppleScript. The swift helper executable I posted should work for this.

(Also, FWIW – you’d need to inspect /System/Library/CoreServices/NotificationCenter.app with UI Browser, not System Events. It won’t help you in this situation, though.)

1 Like

Yes, I made your script to work (using PFAssistive framework), I do get the string “Title, SubTitle, Test 123”, but don’t have a clue how to do it with FaceTime…

So, after a bit of head bashing since trying to help @Tree_Frog, I have since made some new discoveries.

@Silverstone, I initially had a somewhat similar issue. I maintained an Alfred workflow that allowed people to make calls and send iMessages via Alfred. There was an option to skip the confirmation of the call (would click call button for you), which seems to have changed EVERY OS X/macOS release that I can remember. Sequoia has been the most unforgiving of all the OSs in this regard.

Anyway, I eventually managed to find a way to automate that damn button! (I know you aren’t asking about the button, but bear with me)

Every tool I tried to use was a lie. Accessibility Inspector, UI Browser 3, UIElementInspector, whatever tool comes with BetterTouchTool, etc… I basically just had to recursively loop through everything until I eventually found what I need.

Before I get to the scripts, here is something I noticed:

  1. Everything basically needs Accessibility permissions. I gave said permissions to Terminal.app, Script Editor.app, Xcode.app, etc… I have even gone as far as exporting scripts as “Applications” in Script Editor and giving them accessibility permissions too (if necessary). Swift binaries? Accessibility Permissions too.

  2. If you do not do #1, then you will get that application process FaceTimeNotificationExtension of application "System Events"

  3. I can’t exactly test calling myself with FaceTime audio because the UI changes to me being the caller and not the recipient. (I assume you are talking about the call recipient, right?)

The script to click on the call button:

# iPhone Call
#do shell script "open 'tel://0123456789'"

# FaceTime Audio Call
do shell script "open 'facetime-audio://0123456789'"

tell application "System Events"
    # Gives about 3 seconds max for the notification/button to appear
    # Takes my old Mac around 2.75 seconds
    set countdown to 15 -- Set the number of seconds for the countdown
	
    repeat while countdown > 0
	    try
		    click button 1 of group 4 of group 1 of group 1 of group 1 of group 1 of group 1 of group 1 of group 1 of group 1 of scroll area 1 of group 1 of group 1 of window "Notification Center" of application process "NotificationCenter"
		    exit repeat
	    on error errorMessage number errorNumber
		    return errorMessage
	    end try
	    delay 0.25
	    set countdown to countdown - 1
    end repeat
end tell

From the notification (that I create to call someone) here is how you get their name, the call recipient (or their phone number if not a contact):

tell application "System Events"
    try
	    set callerName to description of static text 1 of group 1 of group 1 of group 1 of group 1 of group 1 of group 1 of group 1 of group 1 of scroll area 1 of group 1 of group 1 of window "Notification Center" of application process "NotificationCenter"
    	
	    return callerName
    	
    on error errorMessage number errorNumber
	    return "Error: " & errorMessage
    end try
end tell

I would imagine getting the incoming caller’s name would be quite similar. If you know of a way to test this, then I can maybe try and see what I can do to help if time permits.

1 Like

Thanks to @Hirvi74 I remade a script for Sequoia.
The numeration of static texts remained the same.

When this script is run during a call (before answered) it gets the Name of the caller (to whom you call, or who calls you) as it is in your Address Book. Or phone number if there are no Name in Address Book.

It may be used e.g. to rename an mp3 record file or classify it into callers folders (tags)

set theCaller to ""
tell application "System Events"
	if exists static text 3 of group 1 of group 1 of group 1 of group 1 of group 1 of group 1 of group 1 of group 1 of scroll area 1 of group 1 of group 1 of window "Notification Center" of application process "NotificationCenter" then
		set theCaller to value of static text 3 of group 1 of group 1 of group 1 of group 1 of group 1 of group 1 of group 1 of group 1 of scroll area 1 of group 1 of group 1 of window "Notification Center" of application process "NotificationCenter"
	else
		if exists static text 1 of group 1 of group 1 of group 1 of group 1 of group 1 of group 1 of group 1 of group 1 of scroll area 1 of group 1 of group 1 of window "Notification Center" of application process "NotificationCenter" then
			set theCaller to description of static text 1 of group 1 of group 1 of group 1 of group 1 of group 1 of group 1 of group 1 of group 1 of scroll area 1 of group 1 of group 1 of window "Notification Center" of application process "NotificationCenter"
			if theCaller is "from your iPhone" then set theCaller to description of static text 2 of group 1 of group 1 of group 1 of group 1 of group 1 of group 1 of group 1 of group 1 of scroll area 1 of group 1 of group 1 of window "Notification Center" of application process "NotificationCenter"
		end if
	end if
end tell

return theCaller

PS
But what is interesting for me, @Hirvi74, how did you manage to get this info?

@Silverstone

Yay! I am glad you were able to get something working. I feel like Apple is making it harder every year.

But what is interesting for me, @Hirvi74, how did you manage to get this info?

So, I won’t even pretend like I am some AppleScript savant. Basically, I used ChatGPT-4o, to help write me various scripts that recursively drill down through attribute hierarchies. One thing I think that helps LLMs with this task is sharing everything you can with them. I would upload screenshots of UI Browser and Accessibility Inspector hierarchies, upload every attempt I have tried, and I would specify details like the macOS version too. Also, make sure to describe the problem you are trying to solve in detail. Then GPT would basically spit out scripts, with debugging dialog messages, and I would feed the returns back to GPT.

GPT won’t get this correct on the first attempt. It’s going to take a bit of back and forth – like +10 tries, if not more. You have to work with it though. For what it is worth, at the time of posting this message, I found other LLMs to be awful at AppleScript – Claude Sonnet 3.5 was worse than useless, for example.

Here is an example of a one of GPT’s attempts about two steps before it figured it out. See how it drills deeper down? I would have probably never figured it out on my own.

tell application "System Events"
    try
	    -- Get a reference to the Notification Center process
	    set notificationCenter to application process "NotificationCenter"
    	
	    -- Make sure Notification Center is accessible
	    tell notificationCenter
		    -- List out all elements within the Notification Center
		    set allElements to entire contents of window "Notification Center"
	    	
		    -- Iterate through all elements and display their class and value
		    repeat with element in allElements
			    try
				    display dialog "Class: " & (class of element as string) & ", Value: " & (value of element as string)
			    end try
		    end repeat
	    end tell
    on error errorMessage number errorNumber
	    display dialog "Error: " & errorMessage
    end try
end tell

Read the ‘Events’ output in Script Editor, and you can see how it starts to drill down deeper each iteration.

1 Like