Copying moving deleting and manipulating files


(Ed Stockly) #1

This is kind a best practices question.

What is the fastest, most reliable way to copy, move, delete and otherwise manipulate files.

I have scripts that use Finder; System Events and shell scripting, and some of the older scripts (using finder) are painfully slow.

What are the best recommendations?

Are there any libraries with file handling commands?

Does ASObjC have any souped-up methods?


(Mark Alldritt) #2

When bulk file operations are required, I prefer shell commands that operate on multiple files at once, either via wildcards or by listing multiple file paths in a single mv/cp/rm command. Any time you are issuing sequence mv/cp/rm commands from AppleScript, things tend slow down.

From a shell script you can also exploit concurrency by forking multiple commands into their own processes. With this you can approach the bandwidth limits of your hardware. But, its difficult to synchronize if you need to know when operations have completed and AppleScript’s do shell script does not play well with this so be careful to ensure all forked processes complete before returning to back AppleScript.


#3

Not sure if any of these are relevant, but FWIW some composable Lego bricks:

use AppleScript version "2.4"
use framework "Foundation"
use scripting additions

-- Write a string to the end of a file. 
-- Returns true if the path exists 
-- and the write succeeded. 
-- Otherwise returns false.
-- appendFile :: FilePath -> String -> IO Bool
on appendFile(strPath, txt)
    set ca to current application
    set oFullPath to (ca's NSString's stringWithString:strPath)'s ¬
        stringByStandardizingPath
    set {blnExists, intFolder} to (ca's NSFileManager's defaultManager()'s ¬
        fileExistsAtPath:oFullPath isDirectory:(reference))
    if blnExists then
        if intFolder = 0 then
            set oData to (ca's NSString's stringWithString:txt)'s ¬
                dataUsingEncoding:(ca's NSUTF8StringEncoding)
            set h to ca's NSFileHandle's fileHandleForWritingAtPath:oFullPath
            h's seekToEndOfFile
            h's writeData:oData
            h's closeFile()
            true
        else
            -- text appended to folder is undefined
            false
        end if
    else
        if doesDirectoryExist(takeDirectory(oFullPath as string)) then
            writeFile(oFullPath, txt)
            true
        else
            false
        end if
    end if
end appendFile

-- Write a string to the end of a file. 
-- Returns a Just FilePath value if the 
-- path exists and the write succeeded. 
-- Otherwise returns Nothing.
-- appendFileMay :: FilePath -> String -> Maybe IO FilePath
on appendFileMay(strPath, txt)
    set ca to current application
    set oFullPath to (ca's NSString's stringWithString:strPath)'s ¬
        stringByStandardizingPath
    set strFullPath to oFullPath as string
    set {blnExists, intFolder} to (ca's NSFileManager's defaultManager()'s ¬
        fileExistsAtPath:oFullPath isDirectory:(reference))
    if blnExists then
        if intFolder = 0 then -- Not a directory
            set oData to (ca's NSString's stringWithString:txt)'s ¬
                dataUsingEncoding:(ca's NSUTF8StringEncoding)
            set h to ca's NSFileHandle's fileHandleForWritingAtPath:oFullPath
            h's seekToEndOfFile
            h's writeData:oData
            h's closeFile()
            Just(strFullPath)
        else
            Nothing()
        end if
    else
        if doesDirectoryExist(takeDirectory(strFullPath)) then
            writeFile(oFullPath, txt)
            Just(strFullPath)
        else
            Nothing()
        end if
    end if
end appendFileMay

-- createDirectoryIfMissingMay :: Bool -> FilePath -> Maybe IO ()
on createDirectoryIfMissingMay(blnParents, fp)
    if doesPathExist(fp) then
        Nothing()
    else
        set e to reference
        set ca to current application
        set oPath to (ca's NSString's stringWithString:(fp))'s ¬
            stringByStandardizingPath
        set {bool, nse} to ca's NSFileManager's ¬
            defaultManager's createDirectoryAtPath:(oPath) ¬
            withIntermediateDirectories:(blnParents) ¬
            attributes:(missing value) |error|:(e)
        if bool then
            Just(fp)
        else
            Nothing()
        end if
    end if
end createDirectoryIfMissingMay

-- doesDirectoryExist :: FilePath -> IO Bool
on doesDirectoryExist(strPath)
    set ca to current application
    set oPath to (ca's NSString's stringWithString:strPath)'s ¬
        stringByStandardizingPath
    set {bln, int} to (ca's NSFileManager's defaultManager's ¬
        fileExistsAtPath:oPath isDirectory:(reference))
    bln and (int = 1)
end doesDirectoryExist

-- doesFileExist :: FilePath -> IO Bool
on doesFileExist(strPath)
    set ca to current application
    set oPath to (ca's NSString's stringWithString:strPath)'s ¬
        stringByStandardizingPath
    set {bln, int} to (ca's NSFileManager's defaultManager's ¬
        fileExistsAtPath:oPath isDirectory:(reference))
    bln and (int ≠ 1)
end doesFileExist

-- doesPathExist :: FilePath -> IO Bool
on doesPathExist(strPath)
    set ca to current application
    ca's NSFileManager's defaultManager's ¬
        fileExistsAtPath:((ca's NSString's ¬
            stringWithString:strPath)'s ¬
            stringByStandardizingPath)
end doesPathExist

-- filePath :: String -> FilePath
on filePath(s)
    ((current application's ¬
        NSString's stringWithString:s)'s ¬
        stringByStandardizingPath()) as string
end filePath

-- fileSize :: FilePath -> Either String Int
on fileSize(fp)
    script fs
        on |λ|(rec)
            |Right|(NSFileSize of rec)
        end |λ|
    end script
    bindLR(my fileStatus(fp), fs)
end fileSize

-- fileStatus :: FilePath -> Either String Dict
on fileStatus(fp)
    set e to reference
    set {v, e} to current application's NSFileManager's defaultManager's ¬
        attributesOfItemAtPath:fp |error|:e
    if v is not missing value then
        |Right|(v as record)
    else
        |Left|((localizedDescription of e) as string)
    end if
end fileStatus

-- getCurrentDirectory :: IO FilePath
on getCurrentDirectory()
    set ca to current application
    ca's NSFileManager's defaultManager()'s currentDirectoryPath as string
end getCurrentDirectory

-- getDirectoryContents :: FilePath -> IO [FilePath]
on getDirectoryContents(strPath)
    set ca to current application
    unwrap(ca's NSFileManager's defaultManager()'s ¬
        contentsOfDirectoryAtPath:unwrap(stringByStandardizingPath of wrap(strPath)) |error|:(missing value))
end getDirectoryContents

-- getFinderDirectory :: IO FilePath
on getFinderDirectory()
    tell application "Finder" to POSIX path of (insertion location as alias)
end getFinderDirectory

-- getHomeDirectory :: IO FilePath
on getHomeDirectory()
    current application's NSHomeDirectory() as string
end getHomeDirectory

-- getTemporaryDirectory :: IO FilePath
on getTemporaryDirectory()
    current application's NSTemporaryDirectory() as string
end getTemporaryDirectory

-- listDirectory :: FilePath -> [FilePath]
on listDirectory(strPath)
    set ca to current application
    unwrap(ca's NSFileManager's defaultManager()'s ¬
        contentsOfDirectoryAtPath:(unwrap(stringByStandardizingPath of ¬
            wrap(strPath))) |error|:(missing value))
end listDirectory

-- readFile :: FilePath -> IO String
on readFile(strPath)
    set ca to current application
    set e to reference
    set {s, e} to (ca's NSString's ¬
        stringWithContentsOfFile:((ca's NSString's ¬
            stringWithString:strPath)'s ¬
            stringByStandardizingPath) ¬
            encoding:(ca's NSUTF8StringEncoding) |error|:(e))
    if e is missing value then
        s as string
    else
        (localizedDescription of e) as string
    end if
end readFile

-- readFileMay :: FilePath -> Maybe String
on readFileMay(strPath)
    set ca to current application
    set e to reference
    set {s, e} to (ca's NSString's ¬
        stringWithContentsOfFile:((ca's NSString's ¬
            stringWithString:strPath)'s ¬
            stringByStandardizingPath) ¬
            encoding:(ca's NSUTF8StringEncoding) |error|:(e))
    if e is missing value then
        Just(s as string)
    else
        Nothing()
    end if
end readFileMay

-- setCurrentDirectory :: String -> IO ()
on setCurrentDirectory(strPath)
    if doesDirectoryExist(strPath) then
        set ca to current application
        set oPath to (ca's NSString's stringWithString:strPath)'s ¬
            stringByStandardizingPath
        ca's NSFileManager's defaultManager()'s ¬
            changeCurrentDirectoryPath:oPath
    end if
end setCurrentDirectory

-- Split a filename into directory and file. combine is the inverse.
-- splitFileName :: FilePath -> (String, String)
on splitFileName(strPath)
    if strPath ≠ "" then
        if last character of strPath ≠ "/" then
            set xs to splitOn("/", strPath)
            set stem to init(xs)
            if stem ≠ {} then
                Tuple(intercalate("/", stem) & "/", |last|(xs))
            else
                Tuple("./", |last|(xs))
            end if
        else
            Tuple(strPath, "")
        end if
    else
        Tuple("./", "")
    end if
end splitFileName

-- takeBaseName :: FilePath -> String
on takeBaseName(strPath)
    if strPath ≠ "" then
        if text -1 of strPath = "/" then
            ""
        else
            set fn to item -1 of splitOn("/", strPath)
            if fn contains "." then
                intercalate(".", items 1 thru -2 of splitOn(".", fn))
            else
                fn
            end if
        end if
    else
        ""
    end if
end takeBaseName

-- takeDirectory :: FilePath -> FilePath
on takeDirectory(strPath)
    if strPath ≠ "" then
        if character -1 of strPath = "/" then
            text 1 thru -2 of strPath
        else
            set xs to init(splitOn("/", strPath))
            if xs ≠ {} then
                intercalate("/", xs)
            else
                "."
            end if
        end if
    else
        "."
    end if
end takeDirectory

-- takeExtension :: FilePath -> String
on takeExtension(strPath)
    set xs to splitOn(".", strPath)
    if length of xs > 1 then
        "." & item -1 of xs
    else
        ""
    end if
end takeExtension

-- takeFileName :: FilePath -> FilePath
on takeFileName(strPath)
    if strPath ≠ "" and character -1 of strPath ≠ "/" then
        item -1 of splitOn("/", strPath)
    else
        ""
    end if
end takeFileName

-- tempFilePath :: String -> IO FilePath
on tempFilePath(template)
    (current application's ¬
        NSTemporaryDirectory() as string) & ¬
        takeBaseName(template) & ¬
        text 3 thru -1 of ((random number) as string) & ¬
        takeExtension(template)
end tempFilePath


-- writeFile :: FilePath -> String -> IO ()
on writeFile(strPath, strText)
    set ca to current application
    (ca's NSString's stringWithString:strText)'s ¬
        writeToFile:(stringByStandardizingPath of ¬
            (ca's NSString's stringWithString:strPath)) atomically:true ¬
            encoding:(ca's NSUTF8StringEncoding) |error|:(missing value)
end writeFile

-- writeFileMay :: FilePath -> String -> Maybe FilePath
on writeFileMay(strPath, strText)
    set ca to current application
    set strFullPath to stringByStandardizingPath of ¬
        wrap(strPath)
    if wrap(strText)'s writeToFile:(strFullPath) atomically:false ¬
        encoding:(ca's NSUTF8StringEncoding) |error|:(missing value) then
        Just(unwrap(strFullPath))
    else
        Nothing()
    end if
end writeFileMay

-- File name template -> string data -> temporary path
-- (Random digit sequence inserted between template base and extension)
-- writeTempFile :: String -> String -> IO FilePath
on writeTempFile(template, txt)
    set strPath to (current application's ¬
        NSTemporaryDirectory() as string) & ¬
        takeBaseName(template) & ¬
        text 3 thru -1 of ((random number) as string) & ¬
        takeExtension(template)
    -- Effect
    writeFile(strPath, txt)
    -- Value
    strPath
end writeTempFile

-- intercalate :: String -> [String] -> String
on intercalate(sep, xs)
    set {dlm, my text item delimiters} to {my text item delimiters, sep}
    set s to xs as text
    set my text item delimiters to dlm
    return s
end intercalate

-- splitOn :: String -> String -> [String]
on splitOn(strDelim, strMain)
    set {dlm, my text item delimiters} to {my text item delimiters, strDelim}
    set xs to text items of strMain
    set my text item delimiters to dlm
    return xs
end splitOn

-- unwrap :: NSObject -> a
on unwrap(objCValue)
    if objCValue is missing value then
        missing value
    else
        set ca to current application
        item 1 of ((ca's NSArray's arrayWithObject:objCValue) as list)
    end if
end unwrap

-- wrap :: a -> NSObject
on wrap(v)
    set ca to current application
    ca's (NSArray's arrayWithObject:v)'s objectAtIndex:0
end wrap

-- Just :: a -> Just a
on Just(x)
    {type:"Maybe", Nothing:false, Just:x}
end Just

-- Nothing :: () -> Nothing
on Nothing()
    {type:"Maybe", Nothing:true}
end Nothing

-- Left :: a -> Either a b
on |Left|(x)
    {type:"Either", |Left|:x, |Right|:missing value}
end |Left|

-- Right :: b -> Either a b
on |Right|(x)
    {type:"Either", |Left|:missing value, |Right|:x}
end |Right|

-- Tuple (,) :: a -> b -> (a, b)
on Tuple(a, b)
    {type:"Tuple", |1|:a, |2|:b}
end Tuple

(Shane Stanley) #4

There’s my FileManagerLib. It’s pretty fast, but it’s also convenient: it has terminology, and you can pass aliases, files, HFS paths or POSIX paths.

It also includes several other file commands, like contents of and sort items. And it’s fairly robust — it does things like replacements atomically.

Download it here:

https://www.macosxautomation.com/applescript/apps/Script_Libs.html


(Ed Stockly) #5

So far it seems pretty fast. I just wish the result of copied or moved items were alias(es) to the item(s) moved rather than a boolean.


(Shane Stanley) #6

Where were you when I was testing? :face_with_raised_eyebrow:

How would an option of returning a POSIX path do?


(Ed Stockly) #7

I think that was before I discovered the magic of libraries :wink:

Yes a POSIX path would do nicely. (my personal preference is alias, but its easy enough to coerce) I spent a few hours with it last night and have a few other suggestions, if you don’t mind…

• Getting file name; file extension from a path (faster than finder, less clunky than using TIDs)
• Processing (copy; move; delete) lists of items (does it do that already?)

I’m not clear on the results for making new folder. If the item exists, but isn’t a folder it trips an error; if the item exists and is a folder it returns false; if the item doesn’t exist, it creates a folder and returns true. Is that all correct? Ideally, whether the item existed or it had to create it, I think it should simply return the folder’s path. (if I need to know ahead if the folder exists you have a couple commands for that) An error should be fine when the item exists but isn’t a folder, but that error should be distinguishable from other potential errors (invalid directory, etc.)

FWIW, I used it on an older script that relied on Finder and moved files from various network volume to other network volumes and local drives. Used to take over 5 minutes, and occasionally crashed Finder. Now it takes less than a minute and finder is out of the picture.


(Shane Stanley) #8

The issue here is what to do when there’s a problem with one or more files. Do I return an error and leave the user to work out what succeeded and what didn’t? Should it stop going any further when one item fails?


(Ed Stockly) #9

Hmmm, what does Finder do? My preference would be if it’s a list of items moved or copied it should return a list of references to the moved or copied files, and if there was an error with any or all of those items, then the corresponding item in the return list should be the errorText and/or err number. Send it a list of 7 items and get a list of 7 back, but some of the items in the reply could be error messages.


(Ed Stockly) #10

I think I might have found a terminology conflict with “Finder” and this lib.

My script has this

tell application "Finder"
 if exists item dailyFolderName of localDesktop then
do some stuff
end tell

The problem is that your command is “exists item” and the finder’s command is “exists”

Since my command is building this file reference on the fly, that creates a conflict, no?


(Shane Stanley) #11

Throws an error and leaves it to you.

Then you’re going to have to check every item every time.

I’m not convinced skipping a simple repeat loop up front — which you’re going to need for the result anyway — is worth the effort.


(Ed Stockly) #12

FWIW, here’s my solution.


on CopyItem(itemToCopy, itemName, itemDestination, withReplacing)
   local itemToCopy, itemName, itemDestination, withReplacing
   if itemName is "" then set itemName to NameOfItem(itemToCopy)
   set itemCopied to copy item itemToCopy ¬
      to folder itemDestination ¬
      replacing withReplacing
   if itemCopied then
      try
         return ((itemDestination as text) & itemName) as alias
      on error
         return "error"
      end try
   else
      return "error"
   end if
end CopyItem

on MoveItem(itemToMove, itemName, itemDestination, withReplacing)
   if itemName is "" then set itemName to NameOfItem(itemToMove)
   set itemMoved to move item itemToMove ¬
      to folder itemDestination ¬
      replacing withReplacing
   if itemMoved then
      try
         return ((itemDestination as text) & itemName) as alias
      on error
         return "error"
      end try
   else
      return "error"
   end if
end MoveItem

on CopyListOfItems(ItemsToCopy, destinationForItems, withReplacing)
   local ItemsToCopy, destinationForItems, withReplacing, copiedItems, thisItem, itemName
   set copiedItems to {}
   repeat with thisItem in ItemsToCopy
      set thisItem to thisItem as alias
      set itemName to NameOfItem(thisItem)
      set the end of copiedItems to my CopyItem(thisItem, itemName, destinationForItems, withReplacing)
   end repeat
   return copiedItems
end CopyListOfItems

on moveListOfItems(ItemsTomove, destinationForItems, withReplacing)
   local ItemsTomove, destinationForItems, withReplacing, copiedItems, thisItem, itemName
   set movedItems to {}
   repeat with thisItem in ItemsTomove
      set thisItem to thisItem as alias
      set itemName to NameOfItem(thisItem)
      set the end of movedItems to my MoveItem(thisItem, itemName, destinationForItems, withReplacing)
   end repeat
   return movedItems
end moveListOfItems


on NameOfItem(thisItem)
   local thisItem, itemName
   try
      set thisItem to thisItem as alias
   on error
      return ""
   end try
   set saveTID to AppleScript's text item delimiters
   set AppleScript's text item delimiters to {":"}
   set itemName to the last text item of (thisItem as text)
   if itemName is "" then set itemName to the text item -2 of (thisItem as text)
   set AppleScript's text item delimiters to saveTID
   return itemName
end NameOfItem

I may consolidate these single item/list of items into single handlers, but for now this works.

And holy moly is it fast. I have a script that’s been running every day for years, and I have it run an hour before anyone comes in to make sure it’s done before we start working. It often takes over half an hour to execute.

Now, it’s just a couple minutes and most of that is other apps doing their thing.


(Ed Stockly) #13

I could resolve this by using
exists folder…
or
exists file…

But instead I’ll just take it out of the finder tell and use your lib to do it.


(Shane Stanley) #14

You have the same issue with move. The only foolproof solution is to use obscure or obtuse terms, which kind of defeats the purpose of terminology. In this case the aim is to provide a replacement for the Finder, so I’m happy to live with the problem. If you really want to use the Finder as well, you can use parentheses:

tell application "Finder"
 if exists (item dailyFolderName) of localDesktop then
do some stuff
end tell

(Ed Stockly) #15

Said no appleScripter ever!

For me the issue was changing an existing script with a number of those finder commands to start using your lib.

Took me a while to figure out what the issue was.

Looking at the dictionary I can see a few other commands where it could crop up.


(Mark Alldritt) #16

I’m coming to this late, but can’t you return all the needed information in the error:

  • offending object: the file that failed to move/copy/rename
  • partial result: the files that were successfully moved/copied/renamed

Then, those that care can examine these elements of the error and respond accordingly.


(Shane Stanley) #17

Indeed I can. Didn’t occur to me :roll_eyes:


(Ed Stockly) #18

Just notice a small error in the dictionary for the create folder command:

create folder at
create folder at (verb)Create a new folder. (from File Suite)
FUNCTION SYNTAX
set theResult to create folder at any ¬
use name text
RESULT booleanThe POSIX path of the new folder. Only throws an error if the name is already in use by other than a directory.

The result is either a boolean (I think) or the POSIX path. Can’t be both… or can it?


(Shane Stanley) #19

Right — the result type should be text.


(Ed Stockly) #20

That would work!

But, if it encountered an error halfway through the list would it stop, or would it keep going and process all it could and return the two lists in the error?