Filtering folder contents with UTI

ionah · December 7, 2017, 8:58am

I have 2 script handlers in a library for filtering the content of a folder.
The first one is using file extensions and is very fast.
The second one uses the type identifier but is slower because it relies on loops to gather the UTI from each file.

Is it possible to make the following handler work with UTIs?

use AppleScript version "2.5"
use framework "Foundation"
use framework "AppKit"
use scripting additions

on filteredContents:theFolder withExtensions:wExtensions |returning|:returnType recursive:wRecursive
	set theFolder to current application's |NSURL|'s fileURLWithPath:(POSIX path of theFolder)
	set theFileManager to current application's NSFileManager's defaultManager()
	if wRecursive = false then
		set theURLs to theFileManager's contentsOfDirectoryAtURL:theFolder includingPropertiesForKeys:{} options:4 |error|:(missing value)
	else
		set theURLs to (theFileManager's enumeratorAtURL:theFolder includingPropertiesForKeys:{} options:6 errorHandler:(missing value))'s allObjects()
	end if
	set thePred to current application's NSPredicate's predicateWithFormat:"pathExtension IN %@" argumentArray:{wExtensions}
	set theURLs to theURLs's filteredArrayUsingPredicate:thePred
	if returnType = "name" then return (theURLs's valueForKey:"lastPathComponent") as list
	if returnType = "path" then return (theURLs's valueForKey:"path") as list
	if returnType = "url" then return theURLs
	return theURLs as list
end filteredContents:withExtensions:|returning|:recursive:

my filteredContents:"/Applications" withExtensions:{"applescript", "scpt", "scptd", "app"} |returning|:"name" recursive:(0 as boolean)

I tried this but, naturally, it returns this error: Unable to parse the format string "UTI-CONFORMS-TO IN %@"

 use AppleScript version "2.5"
use framework "Foundation"
use framework "AppKit"
use scripting additions

on filteredContents:theFolder withUTI:wUTI |returning|:returnType recursive:wRecursive
	set theFolder to current application's |NSURL|'s fileURLWithPath:(POSIX path of theFolder)
	set theFileManager to current application's NSFileManager's defaultManager()
	set keysToRequest to current application's NSURLTypeIdentifierKey
	if wRecursive = false then
		set theURLs to theFileManager's contentsOfDirectoryAtURL:theFolder includingPropertiesForKeys:{keysToRequest} options:4 |error|:(missing value)
	else
		set theURLs to (theFileManager's enumeratorAtURL:theFolder includingPropertiesForKeys:{keysToRequest} options:6 errorHandler:(missing value))'s allObjects()
	end if
	set thePred to current application's NSPredicate's predicateWithFormat:"UTI-CONFORMS-TO IN %@" argumentArray:{wUTI}
	set theURLs to theURLs's filteredArrayUsingPredicate:thePred
	if returnType = "name" then return (theURLs's valueForKey:"lastPathComponent") as list
	if returnType = "path" then return (theURLs's valueForKey:"path") as list
	if returnType = "url" then return theURLs
	return theURLs as list
end filteredContents:withUTI:|returning|:recursive:

my filteredContents:"/Applications" withUTI:{"public.image", "com.adobe.pdf"} |returning|:"name" recursive:(0 as boolean)

ShaneStanley · December 7, 2017, 11:19am

The short answer is no. You can only filter on properties (or methods that take no arguments), and UTIs aren’t properties. You need to get the UTI for each file, and then check for conformance.

ionah · December 7, 2017, 12:46pm

This is my handler for UTI filtering.
@ShaneStanley, do you have any comment?

use AppleScript version "2.5"
use framework "Foundation"
use framework "AppKit"
use scripting additions

on filteredContents:theFolder withUTI:wUTI |returning|:returnType recursive:wRecursive
	set theFolderURL to current application's NSURL's fileURLWithPath:(POSIX path of theFolder)
	
	set keysToRequest to {current application's NSURLTypeIdentifierKey}
	set theFileManager to current application's NSFileManager's defaultManager()
	
	-- get all items in folder descending into subfolders if asked
	if wRecursive = true then
		set allURLs to (theFileManager's enumeratorAtURL:theFolderURL includingPropertiesForKeys:keysToRequest options:6 errorHandler:(missing value))'s allObjects()
	else
		set allURLs to theFileManager's contentsOfDirectoryAtURL:theFolderURL includingPropertiesForKeys:keysToRequest options:4 |error|:(missing value)
	end if
	
	-- build an array with URL and UTI for each item
	set keyValuesArray to current application's NSMutableArray's new()
	repeat with oneURL in allURLs
		set keyValue to (oneURL's resourceValuesForKeys:keysToRequest |error|:(missing value))
		if keyValue ≠ missing value then
			set keyValue to keyValue's mutableCopy()
			(keyValue's setObject:oneURL forKey:"theURL")
		end if
		(keyValuesArray's addObject:keyValue)
	end repeat
	
	-- buid the pedicate array
	set predArray to current application's NSMutableArray's new()
	repeat with aKind in wUTI
		(predArray's addObject:(current application's NSPredicate's predicateWithFormat_("%K UTI-CONFORMS-TO %@", current application's NSURLTypeIdentifierKey, aKind)))
	end repeat
	
	-- filter the values array
	set thePredicate to current application's NSCompoundPredicate's orPredicateWithSubpredicates:predArray
	set filteredArray to (keyValuesArray's filteredArrayUsingPredicate:thePredicate) --'s |count|()
	
	-- extract the URLs and convert to an AppleScript list if asked
	set theURLs to (filteredArray's valueForKey:"theURL")
	if returnType = "name" then return (theURLs's valueForKey:"lastPathComponent") as list
	if returnType = "path" then return (theURLs's valueForKey:"path") as list
	if returnType = "url" then return theURLs
	return theURLs as list
end filteredContents:withUTI:|returning|:recursive:

my filteredContents:"/Applications" withUTI:{"public.image", "com.adobe.pdf"} |returning|:"file" recursive:true

ShaneStanley · December 7, 2017, 10:33pm

Looks reasonable to me. For a single UTI, I’d probably dispense with the predicate and array of dictionaries, and just test each URL’s UTI using NSWorkspace’s -type:conformsToType:.

NigelGarvey · December 8, 2017, 12:39pm

This is the fastest I’ve been able to get it, building the URL array directly from URLs which satisfy the predicate:

use AppleScript version "2.5"
use framework "Foundation"
use framework "AppKit"
use scripting additions

on filteredContents:theFolder withUTI:wUTI |returning|:returnType recursive:wRecursive
	set theFolderURL to current application's NSURL's fileURLWithPath:(POSIX path of theFolder)
	
	set keysToRequest to current application's NSArray's arrayWithObject:(current application's NSURLTypeIdentifierKey)
	set theFileManager to current application's NSFileManager's defaultManager()
	
	-- get all items in folder descending into subfolders if asked
	if wRecursive = true then
		set allURLs to (theFileManager's enumeratorAtURL:theFolderURL includingPropertiesForKeys:keysToRequest options:6 errorHandler:(missing value))'s allObjects()
	else
		set allURLs to theFileManager's contentsOfDirectoryAtURL:theFolderURL includingPropertiesForKeys:keysToRequest options:4 |error|:(missing value)
	end if
	
	-- build an OR predicate to test each URL's UTI against all the specified ones.
	set predArray to current application's NSMutableArray's new()
	repeat with aKind in wUTI
		(predArray's addObject:(current application's NSPredicate's predicateWithFormat_("%K UTI-CONFORMS-TO %@", current application's NSURLTypeIdentifierKey, aKind)))
	end repeat
	set thePredicate to current application's NSCompoundPredicate's orPredicateWithSubpredicates:predArray
	
	-- build an array of those URLs whose UTIs satisfy the predicate
	set theURLs to current application's NSMutableArray's new()
	repeat with oneURL in allURLs
		set keyValue to (oneURL's resourceValuesForKeys:keysToRequest |error|:(missing value))
		-- This works even if keyValue is missing value.
		if ((thePredicate's evaluateWithObject:keyValue) as boolean) then (theURLs's addObject:oneURL)
	end repeat
	
	if returnType = "name" then return (theURLs's valueForKey:"lastPathComponent") as list
	if returnType = "path" then return (theURLs's valueForKey:"path") as list
	if returnType = "url" then return theURLs
	return theURLs as list
end filteredContents:withUTI:|returning|:recursive:

my filteredContents:"/Applications" withUTI:{"public.image", "com.adobe.pdf"} |returning|:"file" recursive:true

ionah · December 8, 2017, 1:30pm

Many thanks, Shane & Nigel: the modification speeds up significantly the script.
evaluateWithObject: is great.

ShaneStanley · December 8, 2017, 10:56pm

I second that – very nice.

NigelGarvey · December 9, 2017, 1:32pm

It turns out it’s possible to speed things up slightly more by keeping note of the UTIs found to conform and not to conform so that they don’t need to be tested every time they come up. AppleScript text seems to be the most efficient means for this:

use AppleScript version "2.5"
use framework "Foundation"
use framework "AppKit"
use scripting additions

on filteredContents:theFolder withUTI:wUTI |returning|:returnType recursive:wRecursive
	set theFolderURL to current application's NSURL's fileURLWithPath:(POSIX path of theFolder)
	
	set typeIdentifierKey to current application's NSURLTypeIdentifierKey
	set keysToRequest to current application's NSArray's arrayWithObject:(typeIdentifierKey)
	set theFileManager to current application's NSFileManager's defaultManager()
	
	-- get all items in folder descending into subfolders if asked
	if wRecursive = true then
		set allURLs to (theFileManager's enumeratorAtURL:theFolderURL includingPropertiesForKeys:keysToRequest options:6 errorHandler:(missing value))'s allObjects()
	else
		set allURLs to theFileManager's contentsOfDirectoryAtURL:theFolderURL includingPropertiesForKeys:keysToRequest options:4 |error|:(missing value)
	end if
	
	-- build an OR predicate to test each URL's UTI against all the specified ones.
	set predArray to current application's NSMutableArray's new()
	repeat with aKind in wUTI
		(predArray's addObject:(current application's NSPredicate's predicateWithFormat_("self UTI-CONFORMS-TO %@", aKind)))
	end repeat
	set thePredicate to current application's NSCompoundPredicate's orPredicateWithSubpredicates:predArray
	
	-- build an array of those URLs whose UTIs satisfy the predicate …
	set theURLs to current application's NSMutableArray's new()
	-- … keeping AS texts listing the UTIs tried so that they don't need to be tested again.
	set conformingUTIs to ""
	set unconformingUTIs to ""
	repeat with oneURL in allURLs
		set thisUTI to end of (oneURL's getResourceValue:(reference) forKey:typeIdentifierKey |error|:(missing value))
		-- It's only necessary to test this UTI for conformity if it hasn't come up before.
		set thisUTIAsText to linefeed & thisUTI & linefeed
		if (unconformingUTIs contains thisUTIAsText) then
			-- Do nothing.
		else if (conformingUTIs contains thisUTIAsText) then
			-- Add this URL to the output array.
			(theURLs's addObject:oneURL)
		else if ((thePredicate's evaluateWithObject:thisUTI) as boolean) then -- This works even if thisUTI is missing value.
			-- Add this URL to the output array and append the UTI to the conforming-UTI text.
			(theURLs's addObject:oneURL)
			set conformingUTIs to conformingUTIs & thisUTIAsText
		else
			-- Append this UTI to the unconforming-UTI text.
			set unconformingUTIs to unconformingUTIs & thisUTIAsText
		end if
	end repeat
	
	if returnType = "name" then return (theURLs's valueForKey:"lastPathComponent") as list
	if returnType = "path" then return (theURLs's valueForKey:"path") as list
	if returnType = "url" then return theURLs
	return theURLs as list
end filteredContents:withUTI:|returning|:recursive:

my filteredContents:"/Applications" withUTI:{"public.image", "com.adobe.pdf"} |returning|:"file" recursive:true

ionah · December 9, 2017, 3:13pm

What a brilliant construction!
Bravo!

NigelGarvey · December 9, 2017, 8:45pm

There is one further minor refinement — if you can bear it — which is to collect the qualifying URLs in a referenced list and only to make an array version of this at the end in order to derive the final output:

use AppleScript version "2.5"
use framework "Foundation"
use framework "AppKit"
use scripting additions

on filteredContents:theFolder withUTI:wUTI |returning|:returnType recursive:wRecursive
	set theFolderURL to current application's NSURL's fileURLWithPath:(POSIX path of theFolder)
	
	set typeIdentifierKey to current application's NSURLTypeIdentifierKey
	set keysToRequest to current application's NSArray's arrayWithObject:(typeIdentifierKey)
	set theFileManager to current application's NSFileManager's defaultManager()
	
	-- get all items in folder descending into subfolders if asked
	if wRecursive = true then
		set allURLs to (theFileManager's enumeratorAtURL:theFolderURL includingPropertiesForKeys:keysToRequest options:6 errorHandler:(missing value))'s allObjects()
	else
		set allURLs to theFileManager's contentsOfDirectoryAtURL:theFolderURL includingPropertiesForKeys:keysToRequest options:4 |error|:(missing value)
	end if
	
	-- build an OR predicate to test each URL's UTI against all the specified ones.
	set predArray to current application's NSMutableArray's new()
	repeat with aKind in wUTI
		(predArray's addObject:(current application's NSPredicate's predicateWithFormat_("self UTI-CONFORMS-TO %@", aKind)))
	end repeat
	set thePredicate to current application's NSCompoundPredicate's orPredicateWithSubpredicates:predArray
	
	-- build a list of those URLs whose UTIs satisfy the predicate …
	script o
		property theURLs : {}
	end script
	-- … keeping AS texts listing the UTIs tried so that they don't need to be tested again.
	set conformingUTIs to ""
	set unconformingUTIs to ""
	repeat with oneURL in allURLs
		set thisUTI to end of (oneURL's getResourceValue:(reference) forKey:typeIdentifierKey |error|:(missing value))
		-- It's only necessary to test this UTI for conformity if it hasn't come up before.
		set thisUTIAsText to linefeed & thisUTI & linefeed
		if (unconformingUTIs contains thisUTIAsText) then
			-- Do nothing.
		else if (conformingUTIs contains thisUTIAsText) then
			-- Add this URL to the output list.
			set end of o's theURLs to oneURL
		else if ((thePredicate's evaluateWithObject:thisUTI) as boolean) then -- This works even if thisUTI is missing value.
			-- Add this URL to the output list and append the UTI to the conforming-UTI text.
			set end of o's theURLs to oneURL
			set conformingUTIs to conformingUTIs & thisUTIAsText
		else
			-- Append this UTI to the unconforming-UTI text.
			set unconformingUTIs to unconformingUTIs & thisUTIAsText
		end if
	end repeat
	
	-- Get an array version of the URL list and use this to derive the final output.
	set theURLs to current application's NSArray's arrayWithArray:(o's theURLs)
	if returnType = "name" then return (theURLs's valueForKey:"lastPathComponent") as list
	if returnType = "path" then return (theURLs's valueForKey:"path") as list
	if returnType = "url" then return theURLs
	return theURLs as list
end filteredContents:withUTI:|returning|:recursive:

my filteredContents:"/Applications" withUTI:{"public.image", "com.adobe.pdf"} |returning|:"file" recursive:true

ionah · December 13, 2017, 8:16am

Thanks Nigel!
On my MacPro 2013, compared to my first version, the gain is almost 50%.

NigelGarvey · December 13, 2017, 10:24am

Glad you like it. I found the original script interesting.

If you happen to know when writing the callling script that a folder contains files more likely to conform to one of your UTIs than to another, you can get the absolute best performance by arranging the UTI list in the call from most likely conformance to least. But it’s unlikely to ruin your life if you don’t have this information beforehand.