I’ve got three new projects that I’m starting this week and I’m posting them here in case anyone has any advice or shortcuts, or can help me from reinventing the wheel. I’ll post a new message for each project.
All comments are welcome!
Projection 1: Text replacement pairs
I have numerous scripts that read large text files and make replacements. The replacements are done with handler calls:
set myText to ReplaceText(textToFind,replacementText, myText)
a typical call would look like this:
set programListings to ReplaceText (“X Files”, “The X-Files”, programListings)
To add, remove or edit the text replacements I open the script and edit. What I would like to do is make this user friendly, so a non-scripter could easily create his own list of text replacements.
Most of these scripts have dozens of find/replace pairs, some have hundreds.
I’m thinking of using MyriadTables to do this, and that part seems fairly straight forward. My question is what would be the best way to store and access the saved data. In the past I would make them properties and rely on AppleScript.
Any suggestions?
Project 2: Securely saving passwords
We have a system where the user sets a single username/password that works for many things (email; system access; web sites; network volumes).
Each time a users change their passwords they have to also change it in multiple appleScripts, and sometimes the only way they know they didn’t change it in a particular script is that script fails (and if they get enough fails they have to have an administrator unlock their account).
To resolve this I’d like to have a single appleScript app that any of the dozens of scripts and applets installed on the users’ macs could access to get the updated password and use it .
For the user this would mean changing their password in one single place and it would work everywhere. If a sign-on is denied the user would be prompted if they want to reset their password.
I started a version of where the PasswordResetApp simply saves the password as a property, but I know now that that’s not secure.
Any suggestions?
Projects 3: Sharing scripts on multiple macs across a network
I have many scripts and applets that live in numerous places on my mac. Some in the System Scripts menu, stored in various subfolders for apps; some in various application menus; some in specific folders where they’re used as droplets or applets.
If I update one of these scripts I have to go to each mac where that script is used, navigate to it and replace it.
What I would like to do is store all the current versions of each of these scripts in a single network directors, and the user could easily run a script (or it could automatically run each day) that would look for new versions and automatically install them on the user’s mac. (It would be fine if the user get’s prompted for their password or authentication).
But the main goal is to keep all the scripts on everyone’s systems up to date.
(If I get this working, there will probably be several versions; one for every mac; and versions for the scripts for the specific projects various users are working on.)
Any suggestions?
I use filemaker pro a lot with applescript. you can have a table in the database that holds the user names and passwords that the AS can lookup before running. you can have an interface that has applescript scripts attached to buttons. you can then have accounts in FMP in which people are only allowed to run certain scripts. This also makes it easy to deploy script updates. you can make a single user FMP database per person that you send out, or if really going all the way, use filemaker server so that all your changes are live to other users. just my 2 cents…
Thanks, I should have added that I can’t assume the client macs for any of these projects have any third-party or other installed software.
The solutions must be able to run from basic mac setups. (Of course, if the user is working on a particular task, they will have the apps needed for that task installed (Mostly Adobe creative suite; Text Wranger/BBEdit; Excel; Word; Safari; Chrome; Firefox) Also, most of the macs are running on Yosemite (10.10.4; 10.10.5).
Eventually, I assume, El Capitan and Sierra will be certified, so whatever we use will need to be able to be upgraded to use on those systems).
Hi Ed,
Here are my recommendations.
Project 1: The shell command ‘sed’ is your best friend. My company gets a data text file every day that contains 50-100K lines with 30 fields each. We make extensive find/replace changes to each file with AS calls to ‘sed’, then use the file to update a database. Best of all, ‘sed’ is screaming fast compared to any other technique we’ve tried.
Project 2: You could use either a plist preference file or a script library that would be accessed by all of the users’ scripts. While less intuitive, you could also use ‘store script’ and ‘load script’ from Standard Additions to manage a file that contained the username and password.
Project 3: I’ve got nothing and would love to know the solution myself.
Good luck,
Stan C.
It depends a bit on whether you want them to be portable, or even editable outside your app. But basically you want to save them in a file in a folder in ~/Library/Application Support.
Saving them as text is probably not a good idea because of delimiter issues, so you can either write the container object (list or record), or use a property list.
I’m going to argue with that…
First, it has potential issues with Unicode. You might not hit them, but then again you might. Second, on the issue of speed, it is fast – but you often lose a lot of the advantage because of the overhead of calling do shell script.
I don’t know if you want to save previously used “find and replaces” In case you do I’ve included AppleScript routines I’ve used for a long time that can do that easily. These are routines I used to save script preferences but that can easily be extended by saving each line of “find and replaces” with a return appended to it. Then you can read until a return is encountered and keep doing that until all of them are read back in. In the end this saves a tab separated line of text. If you are interested in this and it doesn’t make sense let me know. These routines are pretty general. They can be used on any type of data that can be turned into a string.
use AppleScript version "2.4" -- Yosemite (10.10) or later
use scripting additions
on ListToCharSperatedText(TheList, TheDelimiter)
script ReturnObj
property successful : false
property TheString : ""
end script
try
set OldASDelimiters to text item delimiters of AppleScript
on error errMsg number errNum
display dialog "Error " & (errNum as string) & " occured getting AppleScript delimiters." & return & return & errMsg buttons {"OK"} default button {"OK"} with title "Error"
set successful of ReturnObj to false
return ReturnObj
end try
try
set text item delimiters of AppleScript to {TheDelimiter}
set TheString of ReturnObj to TheList as string
set text item delimiters of AppleScript to OldASDelimiters
set successful of ReturnObj to true
return ReturnObj
on error errMsg number errNum
set text item delimiters of AppleScript to OldASDelimiters
display dialog "Error " & (errNum as string) & " occured while converting a list to characters." & return & return & errMsg with title "Error"
set successful of ReturnObj to false
return ReturnObj
end try
end ListToCharSperatedText
on CharacterSperatedTextToList(TheString, TheDelimiter)
script ReturnObj
property successful : false
property TheList : {}
end script
if (TheString = "") then
set TheList of ReturnObj to {}
set successful of ReturnObj to true
return ReturnObj
end if
try
set OldASDelimiters to text item delimiters of AppleScript
on error errMsg number errNum
display dialog "Error " & (errNum as string) & " occured getting AppleScript delimiters." & return & return & errMsg buttons {"OK"} default button {"OK"} with title "Error"
set successful of ReturnObj to false
return ReturnObj
end try
try
set text item delimiters of AppleScript to {TheDelimiter}
set TempList to text items of TheString
set text item delimiters of AppleScript to OldASDelimiters
if ((last item of TempList) = "") then
set TempList to reverse of rest of reverse of TempList
end if
set TheList of ReturnObj to TempList
set text item delimiters of AppleScript to OldASDelimiters
set successful of ReturnObj to true
return ReturnObj
on error errMsg number errNum
set text item delimiters of AppleScript to OldASDelimiters
display dialog "Error " & (errNum as string) & " occured while converting characters to a list." & return & return & errMsg with title "Error"
set successful of ReturnObj to false
return ReturnObj
end try
end CharacterSperatedTextToList
on SeachAndReplace(SearchStr, ReplaceStr, TheStr)
set OldASDelimiters to text item delimiters of AppleScript
set text item delimiters of AppleScript to {SearchStr as string}
set TheList to text items of TheStr
set text item delimiters of AppleScript to {ReplaceStr as string}
set TheNewStr to TheList as string
set text item delimiters of AppleScript to OldASDelimiters
return TheNewStr
end SeachAndReplace
on WriteDataToDisk(TheData, FolderPath, FileName)
script ReturnObj
property successful : false
end script
set FilePath to FolderPath & "TheData"
tell application "Finder"
if (not (exists (item FilePath))) then
make new file at (folder FolderPath) with properties {name:"TheData"}
end if
end tell
try
write (TheData as text) to file FilePath
set successful of ReturnObj to true
return ReturnObj
on error errMsg number errNum
display dialog "Error " & (errNum as string) & " occured while writing to the disk." & return & return & errMsg buttons {"OK"} default button "OK" with title "Error"
set successful of ReturnObj to false
return ReturnObj
end try
end WriteDataToDisk
on ReadDataFromDisk(FilePath)
script ReturnObj
property successful : false
property TheData : ""
end script
tell application "Finder"
if (not (exists (item FilePath))) then
display dialog "The file to be read from can not be found." buttons {"OK"} default button "OK" with title "Error"
end if
end tell
try
set DataRead to read file (FilePath)
set TheData of ReturnObj to DataRead
set successful of ReturnObj to true
return ReturnObj
on error errMsg number errNum
display dialog "Error " & (errNum as string) & " occured while writing to the disk." & return & return & errMsg buttons {"OK"} default button "OK" with title "Error"
set successful of ReturnObj to false
return ReturnObj
end try
end ReadDataFromDisk
--------------------------------
-- These 2 routines allow you to write a list of values to the disk with a single write, & read that same list back with a single read. Here is a sample of how it works.
set TheResult to ListToCharSperatedText({1, 2, 3, 4}, "*")
if (not (successful of TheResult)) then return false
TheString of TheResult --> "1*2*3*4"
set TheResult1 to CharacterSperatedTextToList("1*2*3*4", "*")
if (not (successful of TheResult1)) then return false
TheList of TheResult1 --> {"1", "2", "3", "4"}
--------------------------------
set FolderPath to "Bills second iMac HD:Users:bill:Desktop:Destination Folder:"
set FileName to "TheData"
set FilePath to FolderPath & FileName
set Value1 to "1"
set Value2 to "A"
set Value3 to true
set TheResult1 to ListToCharSperatedText({Value1, Value2, Value3}, tab)
if (not (successful of TheResult1)) then return false
set ValueToWriteToDisk to TheString of TheResult1
set TheResult2 to WriteDataToDisk(ValueToWriteToDisk, FolderPath, FileName)
if (not (successful of TheResult2)) then return false
set TheResult3 to ReadDataFromDisk(FilePath)
if (not (successful of TheResult3)) then return false
set TheResult4 to CharacterSperatedTextToList(TheData of TheResult3, tab)
if (not (successful of TheResult4)) then return false
return TheList of TheResult4
I don’t think you’re going to find any remotely secure option without at least some third-party help (e.g., using the Keychain and my BridgePlus lib).
With password you need to save them to something secure. Unless you use some third party stuff Applescript can’t save anything securely. The way I get around that is to have the script send the information to some place that is secure. Most of the time I’ve done this it has been with FileMaker Pro. But any secure data storage will work. My theory is to get the information and store it someplace safe as soon as possible and never store the password anywhere except the secure location. I’ve always read the user input in with a script and when done I even zero out the info in the variable I stored the user response in and end the script right after that. That keeps people from searching memory after script quit to find unreclaimed memory that has the secure info in it, and I have had people search memory and do just that in the past.
Bill
For project 3 I’m not sure I understand what you want done. Because centralized update systems with scattered various scripts taking direction from the centralized update system is done a lot and is not hard to implement.
The simplest solution is to have a place on the network where updates are posted. As soon as a script sees there is an update it makes changes and if successful to reports back to this central location it received the update and implemented it. The person in charge can then see what has been updated.
Depending on your needs you can make a dynamic script that can take directions and update or reconfigure itself. That is a lot more complicated to implement. The more common way is for a script to discover an update exists and calls another script to update itself. The update script moves the old script to the trash, and puts the new script in it’s place. You can also write a script that can create a brand new script on the fly and just dynamically generate AppleScript code (inserting anything that is specific to a particular script), then put it into into a script file and save it to where the old script was with the same name as the old one. This is the method I’ve done most often.
Of course moving scripts or saving scripts requires the script be able to handle the network permissions. But Applescript has ways to do that.
There are a lot of ways to go about this kind of stuff. Without more specifics there are are just too many ways to implement this for me to describe them all.
The overall idea is that you decide if you want scripts handling events in many location and controlled from a central location or if you want a centralized script that handles Macs in many different places. The centralized script that does everything from one place suffers from the problem that it breaks a lot when something is changed a the network. And I’ve seen many people who just change something and not tell the technical person. But having lots of script around suffers from the problem of the technical person having more problems knowing how the various scripts are doing. Especially when the user quits the script, a reboot is required and the script does not restart after the reboot, the script crashes …
My solution to this last problem is to have a script report every so often that it is still running and I have a central script that tracks which scripts are reporting and which ones have failed to report back. Also I never leave scripts running on a Mac the user can open and read. Sometimes users really like to tinker and that can cause a lot of problems.
I mentioned a lot of things because I don’t know your constraints or requirements are.
Bill
If one is dealing with Unicode, there are workarounds. For example, U+200B can be specified by breaking it up into its three component bytes:
sed 's/\xe2\x80\x8b//g' inputfile
Attribution: Dennis Williamson
Extensively changing a 13 MB text file with 60K rows takes a couple of minutes vs. a couple of hours for other methods. But with very small files, you’re right—there’d be no benefit to using ‘sed’.
I just remembered another technique that I’ve used for text files with up to a few thousand lines—text item delimiters. Split the file using your “find” text as the delimiter, then rejoin the text using your “replace” text as the delimiter. This is shockingly fast for an AS-only method.
Stan C.
Let me take another bite at this, because how to store values between runs is becoming a greater issue as more people start signing their code or using AppleScriptObjC.
It’s actually not too hard to store values in the preference file any applet (or script bundle) already has. You are obviously limited to types that can be stored in property lists, but for basics like strings and numbers, and lists and records thereof, it’s pretty simple.
Here’s an example saving a list of values:
use AppleScript version "2.4"
use scripting additions
use framework "Foundation"
-- standard prefs files are based on bundle IDs, so the script needs to be a bundle...
if my id = missing value then error "This code works only in an applet or script bundle."
if current application's id = my id then -- must be running as applet
set theDefaults to current application's NSUserDefaults's standardUserDefaults()
else -- being run by editor or other host app
set theDefaults to current application's NSUserDefaults's alloc()'s initWithSuiteName:(my id)
end if
-- set the factory value, used if nothing found
theDefaults's registerDefaults:{searchAndReplaceList:{}}
-- retrieve list
set theList to (theDefaults's objectForKey:"searchAndReplaceList") as list
-- do your stuff here
-- save list
theDefaults's setObject:theList forKey:"searchAndReplaceList"
For a more complete solution capable of storing different classes, you probably want to look at hiding most of the code in a script library. I cover the topic fairly extensively in my book. But for simpler requirements, the above should be all you need.
So what would you suggest instead of using the shell script with sed ?
I’m sorry, but it’s 2016 – there should really be no “if”. As soon as you use something as simple as proper quote or an en-dash, you’re effectively into Unicode. (And that workaround really says it all…)
I’m not sure what “other methods” you’ve tried, but I’m tempted to challenge you to a race 
Seriously, there was a time I’d agree with you. But AppleScriptObjC is quite zippy for this sort of thing. Here’s a simple example:
use AppleScript version "2.4" -- Yosemite (10.10) or later
use framework "Foundation"
use scripting additions
set theContents to current application's NSString's stringWithContentsOfFile:"/Users/shane/Desktop/TerminologyLog.txt" encoding:(current application's NSUTF8StringEncoding) |error|:(missing value)
set theContents to theContents's stringByReplacingOccurrencesOfString:"NSArray" withString:"^$0^" options:(current application's NSRegularExpressionSearch) range:{0, theContents's |length|()}
theContents's writeToFile:"/Users/shane/Desktop/TerminologyLog2.txt" atomically:true encoding:(current application's NSUTF8StringEncoding) |error|:(missing value)
The code here reads in a 16MB file, replaces about 22,000 instances of “NS[A-Z]rray”, and saves the result to a new file.
How long do you think that might take?
Given that he’s doing it within AppleScript, AppleScriptObjC is a logical choice. Or one of the scripting additions, if he’s comfortable with that.
Hi Shane,
I won’t try to guess, but ‘sed’ just did 60067 changes of “INLINE” TO “outhouse” in a 60068-line, 13 MB file in 4.04 seconds, according to Debugger’s script timer. Here’s my code:
do shell script ("sed -e 's:INLINE:outhouse:g' " & sourceFilePath & " > " & targetFilePath)
When I said earlier that it would take a couple of minutes, that’s for a larger task involving several hundreds of thousands of changes.
I’m beyond curious as to how your ASObjC did. Don’t keep us waiting.
Thanks,
Stan C.
About 0.28 seconds. iMac late 2014, 3.5 GHz, Fusion drive.
(To be fair, you can probably subtract about 0.2 from your value because you’re running it in an editor.)