Getting File/Folder Sizes: System Events vs Info For

JMichaelTX · January 2, 2018, 4:45am

Continuing the discussion from Handler to Determine IF Text Contains Item in List:

In the above topic we had quite a disucssion about using the deprecated command info for.

I’m finding I can’t trust anytinig in the ASUG about the performance of infor for

In the above topic I showed that info for and System Events required about the same amount of time to get a file size. But now, I’d tested both getting the size of a large folder, my users folder which is about 51GB.

info for vs System Events

System Events takes nearly 6X (5.86) longer info for.

I found comparable times when running from Script Debugger 6.0.7 (6A217) on macOS 10.11.6.

I also tried this on my large DropBox account which has thousands of files, and got similar results.

So, in spite of Apple’s warning that info for will be slow in getting the size of large folder, I have found it is the fastest method I know of.

Do any of you know a faster method?

Here are my test scripts:

info for Script

use AppleScript version "2.5" -- El Capitan (10.11) or later
use scripting additions

set itemPath to "/Users/jimunderwood" ### CHANGE to your folder, or any LARGE folder

set itemSize to size of (get info for itemPath)
set itemSizeGB to itemSize / (1024 ^ 3)

return itemSizeGB

System Events Script

use AppleScript version "2.5" -- El Capitan (10.11) or later
use scripting additions

set itemPath to "/Users/jimunderwood" ### CHANGE to Your Users folder, or other large folder

set itemAlias to (POSIX file itemPath) as alias

--- Get Size in Bytes of ENTIRE Folder ---
tell application "System Events"
  set itemSize to size of itemAlias
end tell

set itemSizeGB to itemSize / (1024 ^ 3)

return itemSizeGB

--- System Events ---
-->-0.184760604054 GB
-->-198194854 bytes

--- Finder (from the Info panel) ---
-->56.93 GB 
-->55,636,440,453 bytes

--- info for ---
-->51.815 GB
-->5.5636314471E+10 bytes

--- Bash Script du ---
-->0.102789670229 GB
-->110369568 bytes

System Events is returning a NEGATIVE size for my users folder, and the ~/Documents folder
WHY???
It returns a proper positive number for all other folders.

Which is the Correct Size ???

Each method returns a different size. Which should we believe?

For complete reference here’s my shell script:

Bash du Script



set cmdStr to "du -s /users/jimunderwood"
set itemSizeStr to (do shell script cmdStr)

set itemSize to (first word of itemSizeStr) as number
set itemSizeGB to itemSize / (1024 ^ 3)

return itemSizeGB

-->0.102789670229 GB
-->110369568 bytes

ShaneStanley · January 2, 2018, 7:24am

I think you’re misreading the warning. It’s not a warning that it’s a slow method per se, rather that it can take some time for large folders, and therefore it’s quickest to exclude it unless size is the thing you’re actually after.

In fact, I’m not surprised it’s the fastest method. I suspect it uses a Carbon function called FSGetCatalogInfoBulk(). This was always the fastest way, and when Apple deprecated it in 10.8, there were many developers who complained that there was no Cocoa method that was so fast. I used it in early versions ASObjC Runner, and when I changed to the Cocoa method after 10.8, it was definitely slower. Believe it or not, even for me, speed isn’t everything But I’m sure there are apps still using it.

With 10.13 and APFS, Apple renewed the advice to move away from the Carbon disk APIs. They did not say they would break, but they did say that developers should expect them to be slower in some cases. I’ve seen one minor case where there’s been a subtle change of behavior, but I haven’t done any timings.

A bug? It looks like it’s failing with very large values (I wonder if LLONG_MAX is the limit). Please log a bug with Apple.

Obviously not the System Events result for very large folders. I think the du result is in blocks, so probably has to be multiplied by 512, which means some rounding.

FWIW, here’s an ASObjC method that’s even slower . It has the merit of returning the same result as info for (and what you see in the Finder), but is not using deprecated APIs. And you can change the key to return actual on-disk size, and to exclude/include metadata files.

use AppleScript version "2.4"
use scripting additions
use framework "Foundation"

on sizeOfFolder:folderPosixPath
	set theURL to current application's |NSURL|'s fileURLWithPath:folderPosixPath
	set totalSize to 0
	set theEnumerator to current application's NSFileManager's defaultManager()'s enumeratorAtURL:theURL includingPropertiesForKeys:{current application's NSURLTotalFileSizeKey} options:0 errorHandler:(missing value)
	repeat with nextURL in theEnumerator's allObjects()
		set {hasResult, theResult} to (nextURL's getResourceValue:(reference) forKey:(current application's NSURLTotalFileSizeKey) |error|:(missing value))
		if theResult is not missing value then
			set totalSize to totalSize + (theResult as real)
		end if
	end repeat
	return totalSize
end sizeOfFolder:

You can’t say you lack choices…

Of course there’s argument about what these values even mean on APFS volumes.

JMichaelTX · January 2, 2018, 7:57am

Perhaps so, but this seems very clear to me – that Apple is recommending that we use System Events rather than info for to get file/folder sizes:

From the ASLG section on info for:

Special Considerations
Because info for returns so much information, it can be slow, and because it only works on one file at a time, it can be difficult to use. The recommended technique is to use System Events or Finder to ask for the particular properties you want.

Of course, Apple is also recommending Finder here, which we all know is a terrible approach.

IAC, we now know that info for is in fact the fastest method to get file size, or entire folder contents size. Turns out it is also the easiest method to use.

Thanks for the suggestion, but since I am, and will continue to be, using info for, I’ll skip this exercise in futility. If Apple has not figured out this for themselves by now I doubt my single report would have an impact.

If I were to do anything, it would be to request that they do NOT remove info for. Shouldn’t this be obvious since it is the fastest/easiest method?

Good to know that it confirms the info for results.

Thanks, as always, for your great insight and feedback, Shane.

ShaneStanley · January 2, 2018, 8:44am

No, it’s not obvious. They haven’t removed it – they’ve just noted that it’s deprecated. To remove the deprecation notice, they would have to recode it using non-deprecated methods – at which point it would most likely be slower than other methods, because as well as getting the size, it also gets other information.

Things get fixed because people log bug reports, not because Apple figures things out for themselves.

ShaneStanley · January 2, 2018, 11:04am

And one more method for completeness, using BridgePlus. This comes closest to info for in speed, but returns both the file size and the on-disk allocated size:

use scripting additions
use framework "Foundation"
use script "BridgePlus"
load framework

set fileAliasOrPath to "~/Desktop"
set theResult to current application's SMSForder's sizeInfoForFile:fileAliasOrPath
theResult as record

sphil · January 2, 2018, 1:07pm

This is worth reiterating. For those that aren’t developers it might not be obvious, but once s/w reaches a certain level of complexity (and you’re way beyond that level once you’re talking about anything from Apple), no one is looking to “fix things”, both because of time and because of responsibility (aka ‘blame’).

As I understand it, all bugs that get reported get sent to the Apple team responsible for that area of development. The teams have regular meetings and decide which bugs they’ll act on and which they won’t. The number of reports for a particular BR isn’t the only criterion, but it’s certainly one of them.

From a user’s point of view (and that includes 3rd party devs), bug reporting is like the lottery: you gotta be in it to win it.

JMichaelTX · January 3, 2018, 8:55am

Thanks Shane. I can confirm that your ASObjC script yields exactly the same result as my info for script. Of course, it did take a bit longer: 1 min 25 sec

This also exactly matches the results of info for, but takes 23.08 sec vs the 6.485 sec for info for.

So, info for seems to be the best in terms of accuracy and speed, in spite of it being deprecated for years. I guess I’ll stick with it until/if it breaks, or Apple explicitly announces that it is being removed in the next macOS.

BTW, one cool feature about info for is that if you need file/folder properties other than size, then you can exclude size calculation from the command:
Note the without size clause.

Use `info for` to get Reference to File/Folder Object with all Properties except Size

set fileAliasOrPath to "/users/jimunderwood"
set oFolder to info for fileAliasOrPath without size
return oFolder

This took < 0.005 sec in SD6.

The other cool thing is that the file/folder reference can be any of these: POSIX Path, Alias, HFS Path.

NigelGarvey · January 4, 2018, 1:34pm

I’ve been looking at this myself for a couple of days, on and off, on my 10.13.2 system (HFS). The results are quite perplexing!

I began by seeing if I could speed up Shane’s script …

use AppleScript version "2.4"
use scripting additions
use framework "Foundation"

on sizeOfFolder(folderPosixPath)
	set theURL to current application's |NSURL|'s fileURLWithPath:folderPosixPath
	set keyArray to current application's NSArray's arrayWithObject:(current application's NSURLTotalFileSizeKey)
	
	set totalSize to 0
	set theEnumerator to current application's NSFileManager's defaultManager()'s enumeratorAtURL:theURL includingPropertiesForKeys:keyArray options:0 errorHandler:(missing value)
	repeat with nextURL in (theEnumerator's allObjects())
		set theResult to (nextURL's resourceValuesForKeys:(keyArray) |error|:(missing value))
		if theResult is missing value then
		else
			set theResult to theResult as record
			if (count theResult) > 0 then set totalSize to totalSize + (theResult's NSURLTotalFileSizeKey)
		end if
	end repeat
	return totalSize
end sizeOfFolder

… but only managed to knock a few of hundredths of a second off the timings in tests with my Documents folder. Then I wrote scripts to test the speed and results of all of the folder-size methods known to me, compiling all of them from text in Script Editor, saving them unrun, and using the resulting script files in both Script Debugger and Script Editor. Each script was run three times in each editor and the fastest timing used, along with the size result returned on the first run with that timing. The timer was a loaded library script which uses NSDate.

The size returned for my Home folder is obviously different every time, because the contents of its Library folder change from second to second. But the results for both info for and System Events are clearly wrong, the former returning missing value and the latter returning sizes much lower than those from the remaining methods. Jim was getting negative sizes for his Home folder and I suspect that System Events returns just the low 32 bits of 64-bit results. With my Home folder, all the scripts except for one using the Finder are slower in Script Editor than in Script Debugger — Shane’s spectacularly so! I’m totally at a loss to explain to why his script runs so much slower in SE when my minor variation doesn’t.

When run against just my Documents or Desktop folders, all the methods return the same size results and, with the exception of Shane’s script, are faster in Script Editor than in Script Debugger! Interestingly, the results for my variation on Shane’s script, which returns integers when the numbers are small enough, are reported in decimal form in Script Editor, but in exponential form in Script Debugger. Although the total size of my Desktop is embarrassingly much larger than that of my Documents folder, and indeed a greater number of items is involved, the scripts are all faster against the Desktop than against the Documents folder.

The shock horror hands down winner in terms of speed and reliability turns out to be the Finder!

	{time, size} in SD	{time, size} in SE

Home folder:

info for:	{8.136, missing value}	{9.08, missing value}
System Events:	{12.979, 1.447432964E+9}	{13.109, 1.44743554E+9}
Shane's script:	{46.262, 3.1512214435E+10}	{58.497, 3.1512221235E+10} -- !
My version:	{44.307, 3.1512227217E+10}	{44.619, 3.1512140907E+10}
Bridge Plus:	{7.354, 3.1512148507E+10}	{7.575, 3.1512152843E+10}
Finder:	{0.007, 3.1459650227E+10}	{0.004, 3.1459650227E+10}


Documents folder:

info for:	{0.056, 6.5202481E+8}	{0.053, 6.5202481E+8}
System Events:	{0.122, 6.5202481E+8}		{0.117, 6.5202481E+8}
Shane's script:	{0.797, 6.5202481E+8}	{1.031, 6.5202481E+8}
My version:	{0.761, 6.5202481E+8}	{0.756, 652024810}  -- !
Bridge Plus:	{0.043, 6.5202481E+8}	{0.041, 6.5202481E+8}
Finder:	{0.008, 6.5202481E+8}	{0.005, 6.5202481E+8}

Desktop:

info for:	{0.041, 2.048825986E+9}	{0.039, 2.048825986E+9}
System Events:	{0.088, 2.048825988E+9}	{0.085, 2.048825988E+9}
Shane's script:	{0.745, 2.048825988E+9}	{0.954, 2.048825988E+9}
My version:	{0.703, 2.048825988E+9}	{0.695, 2048825988}  -- !
Bridge Plus:	{0.032, 2.048825988E+9}	{0.03, 2.048825988E+9}
Finder:	{0.004, 2.048804887E+9}	{0.002, 2.048804887E+9}

ShaneStanley · January 4, 2018, 11:56pm

Fascinating. The time differences between the editors is probably mostly down to differing instrumentation overhead. And perhaps SE recognizing my style . I can’t explain the real-integer difference at all.

But the Finder results smell awfully of caching. That would also explain the slightly differing results.

JMichaelTX · January 5, 2018, 4:42am

Hey Nigel, I was hoping that this might catch your interest.

That is strange. Let me make sure I understand. Are you saying the info for returns missing value for your Home folder? If so, I never saw that in SD6, but never tried it in SE. I’ll do that now. I get exactly the same result in SE as I do in SD6.

Could you please post your script for the Finder.

ccstone · January 5, 2018, 5:23am

Shane’s right. It’s caching.

I got missing values for both size and physical size until I manually did Cmd-I in the Finder on my home folder.

-Chris

NigelGarvey · January 5, 2018, 11:50am

Thanks for everyone’s responses.

That must be it!

There seems to be a range of numbers that are too large to be AppleScript integers, but which are integers if arrived at by integer addition. SE displays the results in integer form; SD in exponential form. Both report the results as being integers.

set n to 2.048825988E+9 as integer -- Typed in as 'set n to 2048825988 as integer'
log class of result --> (*real*)

set n to "2048825988" as integer --> 2.048825988E+9
log class of result --> (*real*)

-- Get smaller, integer values and derive the number by addition.
set {nDiv10, nMod10} to {n div 10, n mod 10 as integer}
log result --> (*204882598, 8*)
set s to 0
repeat 10 times
	set s to s + nDiv10
end repeat
set s to s + nMod10 --> 2048825988 in SE's result pane; 2.048825988E+9 in SD's.

-- log {s, class of s} --> (*2.048825988E+9, integer*) in both apps.

For the home folders on both my iMac (10.13.2) and MacBook Pro (10.11.6), it returns a record, the value of whose size property is missing value. My home folders are apparently not nearly as large as yours, so there must be some other reason for the result on my machines than sheer … er … size.

tell application "Finder" to set s to size of home

Yeah. When I tried it again this morning, I kept getting exactly the same result I posted yesterday until I opened an information window on the folder. The script would have to be something like this …

tell application "Finder"
	set theFolder to home
	tell theFolder
		open its information window
		set s to its size
		close its information window
	end tell
end tell
return s

… although the information window itself may need time or encouragement to update. But given the constant housekeeping going on in the machine, there’s no point in trying to get an accurate size figure for the home or library folders anyway. Whichever method you use, the result’s likely to be out of date by the time the script reports it.

ShaneStanley · January 5, 2018, 12:31pm

There are 64-bit integer types defined —both signed and unsigned — I’m just not sure where they’re used (and why not more often).

It will. Try again next time you restart, or choose a bigger folder like /Applications.