Ever since updating to 11.7.5 on a 2014 Mini, a couple of my scripts that use the curl command stopped working.
In troubleshooting it, taking one element at a time, I first tried manually encoding the URL portion I was accessing that had spaces in it - that I had relied on “quoted form of” to handle. Using “%20” for each space character, the script worked again.
Can anyone confirm that “quoted form of” is broken in BS 11.7.5?
It’s likely a quoting issue in your code. Can you post the AppleScript code & an example of a problematic URL?
This script has been running as-is for 4? years now. This is the first time it’s ever failed in this way, and between the last time it ran Friday morning, and when it tried to run Saturday - and failed, the only thing that transpired was that Mac being updated to 11.7.5 from 11.7.4. Bear in mind this is just plucking the pertinent pieces out of the script, and masking domain and customer info:
One example of what “strNotification”, “intPara” and therefore “strData” would contain:
MAT67522_759968.hl_04\JobFiles\AM JobFiles\Perfecting_MAT67522 AMLayout.pdf (50.20M bytes)
set strData to paragraph intPara in strNotification as text # from an email saved as text file via Switch
set AppleScript's text item delimiters to "\\"
set strJobName to text item 1 in strData
set AppleScript’s text item delimiters to “" – delimit based on "”
set strJob1 to text item 1 in strJobName as text – grab just the job number
set AppleScript’s text item delimiters to “.” – delimit based on “.”
set strJob1 to text item 1 in strJob1 as text – grab just the job number
set AppleScript’s text item delimiters to “MAT” – delimit based on “MAT”
set strJob2 to text item 2 in strJob1 as text – grab just the job number
set AppleScript’s text item delimiters to “”
set theDirectory to (“/[DOMAIN1].[DOMAIN2]/User Mailboxes/[ID]/Received/” & strJobName & “/JobFiles/AM JobFiles/”) as text # yes, I know, if we had control of this, we’d change that space to _
set theFile to (“Perfecting_” & strJob1 & " AMLayout.pdf" as text)
set strresult to my hndftpdownload(theDirectory, theFile, strJobName)
on hndftpdownload(theDirectory, theFile, strJobName)
set user to “xxxxxxx” – FTP account user
– Shane Stanley KungFu here to encode the “@” symbol in the password – it cannot be escaped with "" as that doesn’t work in the command line
set thePassword to current application’s NSString’s stringWithString:“Collegi@te” – FTP account password
set thePassword to (thePassword’s stringByAddingPercentEncodingWithAllowedCharacters:(current application’s NSCharacterSet’s URLPasswordAllowedCharacterSet())) as text – use ObjectiveC to encode the “@” character
set theURL to “ftp.[DOMAIN].inet” – the base portion of the URL
set strFilePath to quoted form of (theDirectory & theFile as text) -- get quoted form to escape other characters
set localdirectory to quoted form of ("/Volumes/Commercial/H/Hamburger_4101/Litho_Flat_Sheets/Received_ArtWork/" & strJobName & "/" as text) -- get quoted form to escape other characters
try
do shell script ("curl ftp://" & user & ":" & thePassword & "@" & theURL & strFilePath & " >> " & localdirectory & (quoted form of theFile)) -- execute shell script using 'curl' command
return "Complete"
on error
return "Error" -- this never gets executed BTW
end try
end hndftpdownload
It’s difficult to debug this when it’s not formatted as code - single quotes, double quotes, and double hyphens are modified by the website.
If you surround your code with three consecutive backticks, it should display correctly.
I believe this part would result in an invalid path such as /Volumes/Commercial/H/Hamburger_4101/Litho_Flat_Sheets/Received_ArtWork/'theFile.pdf', with quotes surrounding theFile. You should do quoted form of (localdirectory & theFile) instead.
Edit: localdirectory also shouldn’t be the quoted form. The concatenation of quoted forms of strings includes quote characters, so you’re ending up with an invalid path.
I must have forgotten to include that line, that’s my very point anyway - I am using “quoted form of”, and it appears to be the part that’s broken. Again, this script has been running for 4 years perfectly fine. The only thing that has changed is the OS. Can someone at least test “quoted form of” under 11.7.5 and report back if it’s working?
The line is right here
Tested it on a different Mac running 11.7.5, same issue. Tested it on 11.7.4 on another, works.
quoted form of returns a string, so you should be able to capture the value during a debugging run in Script Debugger (or put in a log statement) to see if it matches the expected value. If you can correct the display of your code in post #3 then I can have another look.
I have 2 confirmations showing it’s broke in my previous post.
What’s the conclusion then? localdirectory is a “safe” string so I’m not using “quoted form of” on it
Can you step through the script in Script Debugger and look at the values of those paths?
You should be able to tell if they’re formatted correctly.
Two things: Updated to 11.7.6, still broke. If I build the entire string before passing to “do shell script”, this is what the string looks like:
curl ftp://username:C0llab0r%40te@ftp.xx.inet’/XXXXXXX.XXXXXX.INET/User Mailboxes/HNJ_FI/Received/FI to COL for HAMBURGER/’
So maybe it’s not that “quoted form of” isn’t doing its thing, but CURL isn’t handling it properly
After .inet, there is a ', and another one at the end. This means that there is a quoted string being joined with the path, resulting in an invalid path that includes ’ in places where it shouldn’t be.
In this line, you are joining localdirectory (a quoted form already, 3 lines above) with (quoted form of theFile). Thus, the resulting text is /path/to/localdirectory/'path/to/file', with the single quotes included, as you observed.
If you change that line to get the quoted form of the entire path, it should work, i.e:
do shell script ("curl ftp://" & user & ":" & thePassword & "@" & theURL & strFilePath & " >> " & quoted form of (localdirectory & theFile))
Likewise, set strFilePath to quoted form of (theDirectory & theFile as text) should become set strFilePath to theDirectory & theFile as text
Hey Folks,
An essential rule-of-the-thumb I’ve learned over several decades of AppleScripting is to never, ever compose a shell command inside a do shell script statement.
I compose the command first and then run the composed command with do shell script.
set theURL to "https://www.google.com/"
set shCmd to "curl -Ls " & theURL
# do shell script shCmd without altering line endings
This way I can eyeball the composed command and test it in the Terminal if necessary, before I enable it in my AppleScript.
I learned to use this method long ago after spending hours trying to debug a problem.
-Chris
I’ll try reconstructing the string as suggested. BUT, anyone care to hazard an explanation for why it has worked for 4 years as-is?
Honestly, I’m not sure – maybe Apple changed the behavior of concatenation, or the way curl handles paths might’ve changed, but both of those seem unlikely. I’m also thinking it could be the switch from bash to zsh, but that happened with Catalina, so again not sure. In any case, I hope this change fixes it for you.
The path that’s passed to the “do shell script” should contain single quotes around the section that is “quoted form of”, no? curl refuses to accept that. Tried re-configuring the strings and simply doesn’t work. The final string within double quotes as passed to the shell contains single quotes around the “quoted form of” section, is that not correct?