How Do We Combine XPath with NSDictionary?

JMichaelTX · March 29, 2018, 1:25am

Continuing the discussion from JavaScript in Browser vs ASObjC XPath:

@ShaneStanley has given us some great examples of how to use both XPath in a NSXMLDocument, and in getting ValueForKey in a NSDictionary. I now need to combine the two.

I have a complicated XML document (from a Keyboard Maestro Macro Action) in which I need to find, and change, the text in certain XML keys.

So, my basic question is how do I:

use a relative XPath to find the key of interest
get its <string> value,
change it (I can do this part)
update the XML with this change.

Here are some typical XML paths I need to find:

Many differenct Actions, but the most common
<key>Variable</key><string>ABC__Item1</string>

Prompt for User Input
<key>Variables</key><array<dict><key>Default</key><string>|%Variable%ABC__Item1%|Choice 1|Choice 2</string>

Action Name that has been renamed by user to include the Var Name
<key>ActionName</key><string>Variable ABC__Item1 in Action Title</string>

Set Variable to Calculation
<key>Text</key><string>ABC__Item2 + 100</string>

All of these are relative paths. There are usually several XML levels (3–5+) above each of these.

Here’s a example of the full path:

<plist version="1.0">
<dict>
  <key>Conditions</key>
  <dict>
    <key>ConditionList</key>
    <array>
      <dict>
        <key>ConditionType</key>
        <string>Variable</string>
        <key>Variable</key>
        <string>ABC__Item1</string>

where the relative path of interest is

        <key>Variable</key>
        <string>ABC__Item1</string>

In case you need it, here is an attachment of the full XML:
Example KM Action XML for Variable Prefix.xml.zip (1.5 KB)

The combination of these two ASObjC features/tools is huge, and will enable some processing not otherwise possible.

Many, many thanks.

BTW, I did try to figure this out myself, searching first Shane’s great book Everyday AppleScriptObjC, Third Edition, then this forum, all of my ASObjC script files, and, of course, the 'net.

ShaneStanley · March 29, 2018, 2:22am

The best answer is that you don’t. As in, load the NSXMLDocument, use XPath queries to get the nodes you want, modify those nodes or their elements, then save the modified document. Involving NSDictionary is just going to complicate things.

ShaneStanley · March 29, 2018, 2:33am

OK, I’ve just had a look at your file, and it’s a property list. So you are probably better off to go the other way — forget about NSXMLDocument, read it using NSPropertyListSerialization to create a mutable dictionary, edit that, then save it again as a new property list.

Property lists are a specialized form of XML, and they are better dealt with differently.

JMichaelTX · March 29, 2018, 3:37am

OK, but I think that leaves me where I started: How do I find the key of interest?
If not via XPath, then how?

ShaneStanley · March 29, 2018, 8:08am

I’m not sure I understand the question — you have to know the key, because that’s what you’re looking for.

If you look at the XML, you will see that a property list uses key itself as a key. That makes parsing via XPath difficult.

JMichaelTX · March 29, 2018, 10:32pm

For example, suppose I have this as part of my XML string:

Prompt for User Input
<key>Variables</key><array<dict><key>Default</key><string>|%Variable%ABC__Item1%|Choice 1|Choice 2</string>

I want to search for the key defined by this path:
(note this is a relative path. There are XML levels above this one.)

<key>Variables</key><array<dict><key>Default</key>

Get its string value:
|%Variable%ABC__Item1%|Choice 1|Choice 2

I will then do a change (probably using RegEx) of, for example:
“ABC__”

TO:
“Local__”

Then I need to update the XML with this change.

This example is in the XML file I attached above.

Can you please show me some sample code, using either XPath OR NSDictionary, on how to do this?

Thanks.

ShaneStanley · March 29, 2018, 11:16pm

When you read that file as a property list, you end up with a dictionary. There’s no way to search a dictionary, other than walking through it. But assuming the format is fixed, you know all you need.

Run this and look at theDict in Script Debugger’s Best view:

use AppleScript version "2.4" -- Yosemite (10.10) or later
use framework "Foundation"
use scripting additions


set thePath to "/Users/shane/Desktop/Example KM Action XML for Variable Prefix.xml" -- change to suit
set newPath to "/Users/shane/Desktop/Example KM Action XML for Variable Prefix modified.xml"
set theData to current application's NSData's dataWithContentsOfFile:thePath
-- convert to Cocoa object
set {theDict, theError} to current application's NSPropertyListSerialization's propertyListWithData:theData options:(current application's NSPropertyListMutableContainersAndLeaves) |format|:(missing value) |error|:(reference)
if theDict is missing value then error (theError's localizedDescription() as text)

From there, you can see how to access the variable by working down the dictionary’s contents. For example:

set theVariables to (theDict's objectForKey:"ElseActions")'s firstObject()'s objectForKey:"Variables"
set thisVariable to theVariables's firstObject()'s objectForKey:"Default"

Because the code specified NSPropertyListMutableContainersAndLeaves and this is a mutable class, this will be an NSMutableString, so you can edit it in place using NSMutableString methods. For example:

thisVariable's replaceOccurrencesOfString:"Choice 1" withString:"Choice 3" options:0 range:{0, thisVariable's |length|()}

When you have finished, you can write it to a new (or the same) property list file. So putting that all together:

use AppleScript version "2.4" -- Yosemite (10.10) or later
use framework "Foundation"
use scripting additions

set thePath to "/Users/shane/Desktop/Example KM Action XML for Variable Prefix.xml" -- change to suit
set newPath to "/Users/shane/Desktop/Example KM Action XML for Variable Prefix modified.xml"
set theData to current application's NSData's dataWithContentsOfFile:thePath
-- convert to Cocoa object
set {theDict, theError} to current application's NSPropertyListSerialization's propertyListWithData:theData options:(current application's NSPropertyListMutableContainersAndLeaves) |format|:(missing value) |error|:(reference)
if theDict is missing value then error (theError's localizedDescription() as text)
-- get the value
set theVariables to (theDict's objectForKey:"ElseActions")'s firstObject()'s objectForKey:"Variables"
set thisVariable to theVariables's firstObject()'s objectForKey:"Default"
-- it's mutable, so modify it
thisVariable's replaceOccurrencesOfString:"Choice 1" withString:"Choice 3" options:0 range:{0, thisVariable's |length|()}
-- write to file
set {theData, theError} to current application's NSPropertyListSerialization's dataWithPropertyList:theDict |format|:(current application's NSPropertyListBinaryFormat_v1_0) options:0 |error|:(reference)
if theData is missing value then error (theError's localizedDescription() as text)
set theResult to theData's writeToFile:newPath atomically:true

You may need to change NSPropertyListBinaryFormat_v1_0 to NSPropertyListXMLFormat_v1_0, although I doubt it.

JMichaelTX · March 30, 2018, 12:14am

OK, thanks. I see this:

I want to change the Conditions > ConditionList > Variable key, so I tried this:

which gave me this error:

-[__NSCFDictionary firstObject]: unrecognized selector sent to instance 0x60800106e400

What am I doing wrong?

I’m trying to work with your NSDictionary example, but my inclination is that using XML with XPath would be easier. It is easier to specify the XPath directly to an element (or actually an array of elements) than it is to step through the NSDictionary one object at a time.

Also, I know the XPath for some objects that is a relative path, where there may be many different Parent Nodes. So, rather than having to figure out every full Parent Path for all cases, I could just use one relative XPath. Make sense?

Also, in both cases, I need to test for a “not found” condition. What’s the best way to do that?

Thanks.

ShaneStanley · March 30, 2018, 1:31am

You’re treating a dictionary as if it were an array. Cut out the firstObjects()'s.

OK. You have the code from elsewhere showing how to read it as XML. Write the XPath query, and I’ll show you how to use it.

In theory, yes. But the XPath query is going to be ugly because the style of XML involved doesn’t use different keys.

Test for missing value, or where arrays are involved, arrays whose |count|() is 0.

JMichaelTX · March 30, 2018, 4:20am

Thanks, Shane.

Here’s my test script which is working, thanks to all other other XML and XPath help you’ve given me.
The result, for now, is the changed text (changed Prefix of KM Variable).
This XPath is for one use case. I will have several XPaths to test in each execution of the script. But this is a good one for now:

XPath for <key>Variable</key><string>SomeVariableName</string>
//key[.='Variable']/following-sibling::string[1]

Now I need to know how to update the XML after I have changed the “SomeVariableName” into something else.


use AppleScript version "2.4" -- Yosemite (10.10) or later
use framework "Foundation"
use scripting additions

property LF : linefeed

--- XPath for <key>Variable</key><string>ABC__Item1</string> ---
(*
  This is a RELATIVE XPath.  It will find all occurances of the Node that meet that path.
*)

set xpathStr to "//key[.='Variable']/following-sibling::string[1]"

--- Path to Example XML Code from KM Action ---
set xmlPath to "~Documents/Test/Example KM Action XML for Variable Prefix.xml"

set nsDataFromXML to current application's NSData's dataWithContentsOfFile:xmlPath

(*
  In my production script, I will get the XML by script from KM, not from a file.
  So, please verify this is how to use/convert that.
  
set nsXMLStr to current application's NSString's stringWithString:actionXMLStr
set nsDataFromXML to nsXMLStr's dataUsingEncoding:(current application's NSUTF8StringEncoding)
*)

set {nsXML, theError} to current application's NSXMLDocument's alloc()'s initWithData:nsDataFromXML options:(current application's NSXMLDocumentTidyHTML) |error|:(reference)
if nsXML = missing value then error (theError's localizedDescription() as text)

-- Search XML for List of Nodes Based o XPath ---

set {nodeArray, theError} to (nsXML's nodesForXPath:xpathStr |error|:(reference)) ##JMTX
if nodeArray = missing value then
  return "[ERROR]" & LF & (theError's localizedDescription() as text)
end if

--- Get Text of Node as List ---

set nodeValueArray to {}
repeat with oNode in nodeArray
  set end of nodeValueArray to (oNode's stringValue() as text)
end repeat

set nodeList to nodeValueArray as list

--- Use Satimage.osax to Change Variable Name Prefix ---
set changedList to change "ABC__" into "Local__" in nodeList

return changedList

(*
OK, I've changed the data.  How do I get this back into the XML?
*)

ShaneStanley · March 30, 2018, 6:20am

You modify the node. So get the node, get its stringValue(), make the new string, then use setStringValue: to modify the node.

set nodeValueArray to {}
repeat with oNode in nodeArray
	set nodeString to oNode's stringValue()
	set nodeString to (nodeString's stringByReplacingOccurrencesOfString:"ABC__" withString:"Local__") -- or whatever
	(oNode's setStringValue:nodeString)
end repeat

The node is still an element of your document, so all you need to do is save the document at the end.

Then use:

set {nsXML, theError} to current application's NSXMLDocument's alloc()'s initWithXMLString:xmlString options:(current application's NSXMLDocumentTidyHTML) |error|:(reference)

JMichaelTX · March 30, 2018, 10:31pm

OK, great! That works well.

How do I convert the changed nsXML to a normal AppleScript string?

I’ve tried a number of things, done a bunch of searching, but to no avail.

ShaneStanley · March 30, 2018, 10:48pm

Use:

set theXML to nsXML's XMLString() as text

Or:

set theXML to (nsXML's XMLStringWithOptions:someOptions) as text

where someOptions is one of the NSXMLNodeOptions. But the former will probably be all you need here.

JMichaelTX · March 30, 2018, 11:15pm

I need to get the tidy formatting, like the XML was as input.
What option do I use?

I’ve tried NSXMLDocumentTidyHTML without success:

set changedXMLStr to (nsXML's XMLStringWithOptions:(current application's NSXMLDocumentTidyHTML)) as text

-- also tried NSXMLDocumentTidyXML

ShaneStanley · March 30, 2018, 11:30pm

If it’s going to be consumed by KM, I doubt that you do. It will only require that it’s a valid property list.

Same as me: trial and error

JMichaelTX · March 30, 2018, 11:45pm

Technically, you’re correct. I’d just like the tidy version for viewing to validate that the changes were properly made.

Thanks for everything. I think I’ve got a proof-of-concept script working now.

Just one more question (I hope): Is there an easy way to use RegEx in the change xml node step?

set nodeString to (nodeString's stringByReplacingOccurrencesOfString:"ABC__" withString:"Local__") -- or whatever
  (oNode's setStringValue:nodeString)

JMichaelTX · March 31, 2018, 12:02am

I thought BBEdit had this built-in, but evidently they removed it at some point. IAC, here is a BBEdit Text Filter that will do the job:

BBEdit text filter to replace Markup → Tidy → Reflow XML

ShaneStanley · March 31, 2018, 12:54am

Did you try NSXMLNodePrettyPrint?

Same as anywhere else:

	set nodeString to (nodeString's stringByReplacingOccurrencesOfString:"ABC(_+)" withString:"Local$1" options:(current application's NSRegularExpressionSearch) range:{0, nodeString's |length|()})

JMichaelTX · March 31, 2018, 2:35am

Nope, didn’t know about that. Thanks.

ShaneStanley:

JMichaelTX:

Is there an easy way to use RegEx in the change xml node step?

Same as anywhere else:
	set nodeString to (nodeString's stringByReplacingOccurrencesOfString:"ABC(_+)" withString:"Local$1" options:(current application's NSRegularExpressionSearch) range:{0, nodeString's |length|()})

OK, that’s great! That will be much better than using an external app or osax for RegEx.

OK, barring any unforeseen issues, I think I’m done here!

Here’s my (hopefully) final test script.
Thank you so very, very much Shane. Could not have done this without you. I have solved a problem and learned much.


use AppleScript version "2.4" -- Yosemite (10.10) or later
use framework "Foundation"
use scripting additions

-- classes, constants, and enums used
property NSUTF8StringEncoding : a reference to 4
property NSString : a reference to current application's NSString
property nsCurApp : a reference to current application

property LF : linefeed
property ptyDebug : false

--- XPath for <key>Variable</key><string>ABC__Item1</string> ---
(*
  This is a RELATIVE XPath.  It will find all occurances of the Node that meet that path.
*)
set xpathStr to "//key[.='Variable']/following-sibling::string[1]"
(*
---------------------------------------------
GET XML FROM VARIABLE
---------------------------------------------
In my production script, I will get the XML by script from KM, not from a file.
Then use:

set {nsXML, theError} to nsCurApp's NSXMLDocument's alloc()'s initWithXMLString:actionXML options:(nsCurApp's NSXMLDocumentTidyHTML) |error|:(reference)
*)
-----------------------------
--- Get XML From File
-----------------------------
--- Path to Example XML Code from KM Action ---
set xmlPath to "/Users/Shared/Dropbox/SW/DEV/Projects/[KM] Script to Change Variable Names/Example KM Action XML for Variable Prefix.xml"
set nsDataFromXML to nsCurApp's NSData's dataWithContentsOfFile:xmlPath
set {nsXML, theError} to nsCurApp's NSXMLDocument's alloc()'s initWithData:nsDataFromXML options:(nsCurApp's NSXMLDocumentTidyHTML) |error|:(reference)
-------- END Get XML From File ------------------------

if nsXML = missing value then error (theError's localizedDescription() as text)

--- Now We Have the XML as an ObjC NSXMLDocument ---

-- Search XML for List of Nodes Based on XPath ---

set {nodeArray, theError} to (nsXML's nodesForXPath:xpathStr |error|:(reference)) ##JMTX
if nodeArray = missing value then
  return "[ERROR]" & LF & (theError's localizedDescription() as text)
end if

--- Change Each Node Value That was Found ---

set changedList to {}
repeat with oNode in nodeArray
  set nodeString to oNode's stringValue()
  
  --- Normal Change Text ---
  --  set nodeString to (nodeString's stringByReplacingOccurrencesOfString:"ABC__" withString:"Local__") -- or whatever
  
  --- RegEx Example of Change ---
  set nodeString to (nodeString's stringByReplacingOccurrencesOfString:"ABC(_+)" withString:"Local$1" options:(nsCurApp's NSRegularExpressionSearch) range:{0, nodeString's |length|()})
  
  --- Update the XML ---
  (oNode's setStringValue:nodeString)
  
  set end of changedList to nodeString as text
end repeat

--set changedXMLStr to nsXML's XMLString() as text
set changedXMLStr to (nsXML's XMLStringWithOptions:(nsCurApp's NSXMLNodePrettyPrint)) as text

if ptyDebug then
  tell application "BBEdit"
    activate
    set oDoc to make new text document with properties {contents:changedXMLStr, source language:"xml"}
  end tell
end if

return changedList

If anyone else has questions about this, about XPath, I’ll be more than happy to try to answer them.
Just keep in mind that I to am still learning.