Exporting Apple Notes Attachments

JMichaelTX · October 7, 2017, 11:44pm

Continuing the MacScripter.net discussion at
Export selected notes from Notes.app w/ attachments, import to EN

@NigelGarvey and @koenigyvan, I hope you guys don’t mind if we have the discussion here. I find Macscripter.net forum to be tedious as hell to use, especially since you can’t upload images, and you can’t quote selected parts of a post.

This part of your script is failing for me, running macOS 10.11.6:

           -- Get alias specifiers for any attachments.
           set attachmentFiles to {}
           repeat with thisAttachment in myAttachments
               -- Get this attachment's content identifier (cid) from Notes.
               tell application "Notes" to set thisCID to content identifier of thisAttachment
               -- Use it to look up the corresponding NSURL in the lookup dictionary and store the result as a file specifier.
               set end of attachmentFiles to (attachmentLookup's valueForKey:(thisCID)) as alias
           end repeat

This line:
tell application "Notes" to set thisCID to content identifier of thisAttachment
returns missing value.

SD6 Screenshot:

I tried using id but that did not work either.

Here’s the actual path to the attachment:
/Users/jimunderwood/Library/Group Containers/group.com.apple.notes/Media/8EB8E540-FD61-465A-AC04-F65F785984F0/CSS Selector Attribute Comparisons.docx

Obviously, we can get the name of the attachment, but I don’t see anything that relates to this folder:
8EB8E540-FD61-465A-AC04-F65F785984F0

Any ideas?

For ease of reference, here’s Nigel’s complete script:

(*
  ====================================================
    [EN] Import Apple Notes into Evernote
  ====================================================
  
  DATE:    2013-10-24
  AUTHOR: d.b.walker
  
  REVISED BY:  JMichaelTX on 2016-03-28 to make BUG fix. <[url]https://discussion.evernote.com/topic/64814-apple-notes-app/#comment-395941[/url]>
  
  REF:
    â€¢ Importing from Apple Mail.app's Notes - Mac Help - Evernote User Forum       
    â€¢ [url]https://discussion.evernote.com/topic/4046-importing-from-apple-mailapps-notes/?do=findComment&comment=236445[/url]
  
  Posted 24 Oct 2013
  Modified this script to work with Mavericks Notes, which is no longer in the mail app.
  Added the original creation and modification dates
  Added multiple tags - replace with your own
  Did not add the long note name fix (I needed to preserve my note names)
  ====================================================
  
  FURTHER DEVELOPED BY: Nigel Garvey 2017-03-21/22/23, based on information in the Evernote fora, to allow a choice of Notes source folder(s) and to handle attachments.
  REVISION BY NG, 2017-03-29: "Note" folder names in the temporary desktop hierarchy for attachments now based on the notes' names. Any path delimiters in potential folder names now replaced with dashes.
  FURTHER REVSION 2017-04-03/4: The attachment files list is now populated with aliases to Notes's copies of the files.
  
  CAVEATS:
    1. I don't have Evernote and can't test that part of the code.
    2. Only intended for use with Notes's "On my Mac" account.
    3. Any attachments are simply "appended" to the Evernote notes in the order they happen to be returned by Notes.
    4. The effect in Evernote of Notes's references to the attachments in the note HTML is unknown.
    5. Although the script works for me, there's a possibility it may not correctly identify some file URLs. 
*)

use AppleScript version "2.5" -- Mac OS 10.11 (El Capitan) or later. (For NSURL coercions to alias.)
use framework "Foundation"
use scripting additions

main()

on main()
  -- User choice of one or more Notes folders (by name).
  tell application "Notes"
    activate
    set folderNames to name of folders
    set chosenFolderNames to (choose from list folderNames with multiple selections allowed)
    if (chosenFolderNames is false) then error number -128 -- Cancel button.
  end tell
  
  -- Get an NSDictionary which has the attachment CIDs as keys and the corresponding file URLs as values.
  set attachmentLookup to getAttachmentLookup()
  
  -- Repeat with each chosen folder name:
  repeat with i from 1 to (count chosenFolderNames)
    -- Get all the notes in the folder with this name.
    set thisFolderName to item i of chosenFolderNames
    tell application "Notes" to set theNotes to notes of folder thisFolderName
    
    -- Repeat with each note in the folder:
    repeat with j from 1 to (count theNotes)
      set thisNote to item j of theNotes
      
      tell application "Notes"
        -- Get the relevant note data.
        set myTitle to the name of thisNote
        set myText to the body of thisNote
        set myCreateDate to the creation date of thisNote
        set myModDate to the modification date of thisNote
        set myAttachments to the attachments of thisNote
      end tell
      
      -- Get alias specifiers for any attachments.
      set attachmentFiles to {}
      repeat with thisAttachment in myAttachments
        -- Get this attachment's content identifier (cid) from Notes.
        tell application "Notes" to set thisCID to content identifier of thisAttachment
        -- Use it to look up the corresponding NSURL in the lookup dictionary and store the result as a file specifier.
        set end of attachmentFiles to (attachmentLookup's valueForKey:(thisCID)) as alias
      end repeat
      
      tell application "Evernote"
        
        set myNote to create note with text myTitle title myTitle notebook "Imported From Notes" tags ["imported_from_notes"]
        set the HTML content of myNote to myText
        
        repeat with thisFile in attachmentFiles
          tell myNote to append attachment thisFile
        end repeat
        
        set the creation date of myNote to myCreateDate
        set the modification date of myNote to myModDate
        
      end tell
      
    end repeat
    
  end repeat
  
end main

-- Create a lookup dictionary (NSDictionary) which has all the available attachment CIDs as keys and URLs to the corresponding files as values.
on getAttachmentLookup()
  set |âŒ˜| to current application
  set fileManager to |âŒ˜|'s class "NSFileManager"'s defaultManager()
  -- Get an NSURL to the user Library folder and thence to the folder containing Note's database file(s).
  set userLibraryFolderURL to (fileManager's URLForDirectory:(|âŒ˜|'s NSLibraryDirectory) inDomain:(|âŒ˜|'s NSUserDomainMask) appropriateForURL:(missing value) create:(false) |error|:(missing value))
  set NotesDBFolderURL to userLibraryFolderURL's URLByAppendingPathComponent:("Containers/com.apple.Notes/Data/Library/Notes")
  -- Or of course:
  (* set NotesDBFolderPath to |âŒ˜|'s class "NSString"'s stringWithString:("~/Library/Containers/com.apple.Notes/Data/Library/Notes")
  set NotesDBFolderPath to NotesDBFolderPath's stringByExpandingTildeInPath()
  set NotesDBFolderURL to |âŒ˜|'s class "NSURL"'s fileURLWithPath:(NotesDBFolderPath) *)
  -- On my system, the database file of interest has a name ending with "-wal".
  set databaseCandidates to fileManager's contentsOfDirectoryAtURL:(NotesDBFolderURL) includingPropertiesForKeys:({}) options:(|âŒ˜|'s NSDirectoryEnumerationSkipsHiddenFiles) |error|:(missing value)
  set walFilter to |âŒ˜|'s class "NSPredicate"'s predicateWithFormat:("path ENDSWITH '-wal'")
  set databaseURL to (databaseCandidates's filteredArrayUsingPredicate:(walFilter))'s firstObject()
  
  -- The file's contents are binary data, but this hack involves treating them as ISO Latin1 encoded text.
  set databaseText to |âŒ˜|'s class "NSString"'s stringWithContentsOfURL:(databaseURL) encoding:(|âŒ˜|'s NSISOLatin1StringEncoding) |error|:(missing value)
  -- Set a regex to scan for instances of attachment content identifiers (CIDs) in angle brackets ("<xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx@home>" on my machine, simply "<xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx>" on Yvan's) and of URLs to files in Notes's attachments folder hierarchy ("file:///Users/username/Library/Containers/com.apple.Notes/Data/Library/CoreData/Attachments/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/filename").
  set searchRegex to |âŒ˜|'s class "NSRegularExpression"'s regularExpressionWithPattern:("(?<=<)[[:alnum:]]{8}-[[:alnum:]]{4}-[[:alnum:]]{4}-[[:alnum:]]{4}-[[:alnum:]]{12}(?:@home)?(?=>)|file:///Users/[^/]++/Library/Containers/com\\.apple\\.Notes/Data/Library/CoreData/Attachments/[[:alnum:]]{8}-[[:alnum:]]{4}-[[:alnum:]]{4}-[[:alnum:]]{4}-[[:alnum:]]{12}/[\\u0021-\\u007f]++") options:(0) |error|:(missing value)
  -- For additional security, find out where the first CID occurs in the "text".
  set searchStart to (databaseText's rangeOfString:("<[[:alnum:]]{8}-[[:alnum:]]{4}-[[:alnum:]]{4}-[[:alnum:]]{4}-[[:alnum:]]{12}(?:@home)?>") options:(|âŒ˜|'s NSRegularExpressionSearch))'s location()
  -- Start the search proper from there and get the ranges of all the matches. (No harm if none.)
  set matchRanges to (searchRegex's matchesInString:(databaseText) options:(0) range:({searchStart, (databaseText's |length|()) - searchStart}))'s valueForKey:("range")
  
  -- The matches should be alternating instances of CIDs and file URLs, but the code below can easily be modified if this turns out not always to be the case.
  -- Extract the CIDs to one array and NSURL versions of the file URLs to another.
  set cidPrefix to |âŒ˜|'s class "NSString"'s stringWithString:("cid:")
  set theCIDs to |âŒ˜|'s class "NSMutableArray"'s new()
  set theURLs to |âŒ˜|'s class "NSMutableArray"'s new()
  set i to 1
  set j to 2
  set matchCount to matchRanges's |count|()
  repeat until (j > matchCount)
    set thisCID to databaseText's substringWithRange:(item i of matchRanges)
    set thisURL to databaseText's substringWithRange:(item j of matchRanges)
    tell theCIDs to addObject:(cidPrefix's stringByAppendingString:(thisCID))
    tell theURLs to addObject:(|âŒ˜|'s class "NSURL"'s URLWithString:(thisURL))
    set i to j + 1
    set j to i + 1
  end repeat
  
  -- Make and return an NSDictionary with the CIDs as the keys and the NSURLs as the values.
  -- The CID/NSURL pairs will no doubt be duplicated in the lists, but not in the dictionary.
  return |âŒ˜|'s class "NSDictionary"'s dictionaryWithObjects:(theURLs) forKeys:(theCIDs)
end getAttachmentLookup

NigelGarvey · October 9, 2017, 11:47am

On my own 10.11.6 machine, an attachment’s content identifier value is a string like “cid:hhhhhhhh-hhhh-hhhh-hhhh-hhhhhhhh@home”, where the hex digits are different from the id of the attachment and the same as the name of the folder in which the attachment file’s stored. Looking back through the off-list correspondence I had with Yvan in April while the MacScripter thread was current, he was getting missing value too. But he was running Sierra and his notes were in iCloud. Now on my new Sierra machine, I’m getting missing value too. The machine was signed up to iCloud with no apparent opt-out during its initial set-up, but I believe I’ve managed to sign out of that now. Still, although I clicked the Cancel button in the “Turn on iCloud” dialog which appeared when I first opened Notes in Sierra, the “On My Mac” account and its contained folders, notes, and attachments now all have “ICAccount”, “ICFolder”, “ICNote”, or “ICAttachment” in their IDs instead of just “Folder”, “Account”, “Note”, or “Attachment”.

The ~/Library/Group Containers/group.com.apple.notes/ folder exists on my El Capitan machine, but there’s no Media folder in it. The attachments I set up to work on the script originally appeared at ~/Library/Containers/com.apple.Notes/Data/Library/CoreData/Attachments/hhhhhhhh-hhhh-hhhh-hhhhhhhh/filename.extn. On my Sierra machine, these attachments have been duplicated to ~/Library/Group Containers/group.com.apple.notes/Media/hhhhhhhh-hhhh-hhhh-hhhhhhhh/filename.extn, with different folder names.

JMichaelTX · October 9, 2017, 11:41pm

Nigel, thank you for your reply.

It looks like the Notes attachments folder is dependent on macOS and on iCloud being turned on/off.

I do have Notes checked in the System Preferences > iCloud panel, and I’m running El Capitan (macOS 10.11.6).

I do have the folder you used:
~/Library/Containers/com.apple.Notes/Data/Library/CoreData/Attachments
but there is only one file in it, from a Note created in Aug 2015.

I have found a workaround for my setup that is not perfect, but may be acceptable.
I simply do a bash find in the ~/Library/Group Containers/group.com.apple.notes/Media folder looking for an exact match with the attachment file name. Obviously, if the same attachment (by name) exists in more than one Note, then the user will have to choose which one to associate with the Note.

I was hoping to include the file mod date in the find search, but I can’t find the attachment date anywhere in Notes. You haven’t run across this date by chance, have you?

NigelGarvey · October 10, 2017, 11:51am

I’m afraid not. The only dates mentioned in Notes’s dictionary are the creation and modification dates of the notes themselves. On my machines, the attached files have the same dates as the originals which were dragged onto the notes to create the attachments. Presumably the folders containing the attached files have creation and modification dates somewhere between those of the associated notes, but I wouldn’t care to write code to find them on that basis immediately….

JMichaelTX · October 10, 2017, 9:05pm

NIgel, thanks for the confirmation.

Thanks for the idea.

Actually it might not be that difficult.
For each file found that matches attachment name, get the parent folder and its mod (or create) date, and compare that the Note dates. But it is still not deterministic.

I’ve settled for attaching all files with the same name into the new Evernote Note, and set an Evernote tag (“Dup.Attach”) flagging it. I think dup names are very unlikely, so having a few Evernote Notes to check afterward doesn’t seem like much pain.

tornado · December 2, 2017, 1:50am

Hi JMIchaelTX,

Would you mind putting your script up somewhere I could use it? I’m having the same issues trying to export attachments from Notes and this would save me some time.

Thanks!

Joe

DearOldDrDad · January 28, 2022, 9:02pm

Yeah, I was working on a similar project to export Notes to Filemaker. Imagine my chagrin when I discovered that the name consists of the first 70ish characters of the first line (paragraph) of the note, and ‘body’ consists of subsequent lines (paragraphs). I can’t find any (good) way with Applescript to get the rest of the first line!

So I believe your script will be truncating first lines and losing data.

Well, what I wound up doing was showing the note, scripting the GUI for the ‘select all’ menu item and the ‘copy’ menu item, then sending the clipboard contents to Filemaker. Crude and slow.

Incidentally, what a miserable thing it is. If you tell the note to show, it will select the note by the title, and ‘select all’ will select all the notes, and ‘copy’ will be disabled. You have to tell the application to show the note, in order to put the text insertion point into the shown note.