Searching for a file with a distinctive word in its name should be straightforward, but here I show some weird problems that could catch you out. I’m very grateful to Sam for drawing my attention to this, and welcome all and any rational explanations of what’s going on.
In some accounts of ancient Greek mythology, Cleta (Κλήτα) was one of the two Charites or Graces, alongside Phaenna. Her name apparently means renowned, and is still occasionally used as a first name today. It’s not the sort of word that should give Spotlight any cause for concern, and should prove easy to find.
Demonstration
To see the problems it can cause, create a folder somewhere accessible, in ~/Documents perhaps, and create half a dozen files with the names shown below.
Now open a new Finder window, and set it to Find mode using that command at the foot of the File menu. Then type into its search box the letters cleta
Only four of the files in that folder are found, excluding the first two, despite the fact that all their names clearly contain the search term.
Now clear the search box, and in the search criterion below, set it to find Name contains cleta, which you might have thought would be the same as the previous search.
Now all six files are found successfully.
You can try other variations of the file name to see which can be found using the search box, and which remain hidden. For example, 1995z_spectacletable_01.txt also appears susceptible to this problem, suggesting that other examples might have the form [digits]_[chars]cleta[chars]_[digits].[extension]
Separators
There are some other oddities at work as well, that you can see in the four file names that haven’t yet played hide and seek. So far I’ve been using Spotlight to find file names that simply contain the characters cleta. Now extend that to cletapainting
While you would expect the second of those to appear, Spotlight has elided the hyphen embedded in the first, as if it wasn’t there. Although Spotlight doesn’t provide a simple way to search for discrete words in file names, that’s a feature readily accessible in several third-party search utilities, including Find Any File and HoudahSpot. If you use Spotlight much, both of those are essentials, and you may wish to add Alfred as well.
As expected, Find Any File has no problems in finding all six test files when looking for names containing cleta
Set it to find names containing the wordcleta, though, and it recognises spaces, hyphens and underscore _ characters as word separators, but doesn’t oblige with CamelCase, whether or not you capitalise its initial character.
Conclusions
Avoid using the characters cleta in file names, as they can confuse Spotlight.
Leave the search box in the Finder’s Find window empty and construct your search in the lower search bars instead.
Spotlight can overlook hyphens in file names, but does treat them as word separators.
Searching for words in file names can treat spaces, hyphens and underscore _ characters as word separators, but can’t cope with CamelCase.
Spotlight’s rules are largely unwritten. Apple’s brief account is here, and doesn’t even mention the name Cleta.
My thanks again to Sam for providing me with the example of cleta that made this possible if apparently highly improbable.
Postscript
For those who think this all works as they expect, try the following file name:
A basic Mac system consists of the Mac itself and external storage for its backups, and is by far the most popular configuration. For many folk backing up the whole of its Data volume is wise, but that isn’t always the most economical. If the Data volume contains large items that don’t need to be backed up as often as its working folders, that can waste space. This article shows how you can make it more efficient without additional cost or hardware.
Backups and local snapshots
Most good backup utilities including Time Machine also make local snapshots of the volumes they back up. Let’s say your Data volume contains 100 GB of files that either change little or don’t need to be backed up as frequently as the rest. One proven strategy for minimising the time and storage required for backups is to add those to the exclusion list, and back them up separately, maybe only once a week. You can do that to another volume on external storage, provided you ensure there’s sufficient space for both that and your normal automatic backups.
What that doesn’t do is keep those 100 GB out of the frequent snapshots made of the Data volume. While you can exclude files and folders from backups, snapshots always include everything in that volume, without exclusions. The only way to save the space they add to snapshot size is to move them to another volume that doesn’t get snapshots made of it. But your Mac’s standard disk layout doesn’t provide any spare volume for that.
This could apply to all sorts of relatively static data that doesn’t need Time Machine’s automatic hourly backups, including Virtual Machines and some large media libraries, although you won’t then be able to share these in iCloud Drive, which would require them to be in your Data volume.
Boot disk layout
Standard layout of the internal SSD of an Apple silicon Mac running Sequoia or earlier is shown below.
Intel Macs have the same Apple APFS container with the Boot Volume Group in it, but the other two containers are replaced by a single small EFI partition.
Adding another partition or container is possible, but not recommended as it has a fixed size, and lacks the flexibility of a volume. It also risks disturbing the three existing partitions/containers. As they’re essential for the Mac to start up successfully, you don’t want to meddle with them.
In practice, the best place to add a new volume is inside the third container, the one already holding the System and Data volumes. Add that in Disk Utility once you’ve decided the next two steps.
Limit volume size
Your new volume is going to share space in its container with all the existing volumes, including both System and Data. It’s usually wise to impose a maximum limit on the size it can grow to, to avoid compromising any of those. When you add the new volume, put a sensible limit on its Quota Size.
Encryption
Although Apple’s documentation isn’t explicit, volumes added to the boot container aren’t protected by FileVault, unlike the Data volume. If you want your extra volume to be encrypted, you’ll have to format it in APFS (Encrypted). Whether that’s accelerated by the hardware in the Secure Enclave isn’t clear, and on Apple silicon Macs it’s hard to tell the difference, as you should get similar full speed performance from your extra volume to that of the Data volume.
Setting it up
Open Disk Utility, ensure its View options are set to Show All Devices, then select the Container holding the boot volumes. Click the + tool to add the new volume.
Give the volume a name, then click on the Size Options… button.
Enter your chosen Quota Size, as the maximum you want to allow the extra volume to use on the boot SSD, and click OK.
Then select whether you want it formatted in plain APFS, or encrypted, and click the Add button.
If you’ve opted for APFS (Encrypted) you’ll then be prompted to enter the encryption password. Unlike FileVault, there’s no option for a Recovery Key, or for iCloud Recovery.
When you first unlock the extra volume, you’ll be given the option to save its password to your keychain. That confirms this isn’t being performed by FileVault, as that protects its encryption keys in the Secure Enclave.
There are a couple of quirks:
If you try unmounting the extra volume using the Finder’s contextual menu, macOS might try to unmount all volumes on the boot disk, and warn you that it can’t. Simply cancel those warnings, and the extra volume should unmount fine. If you’re worried by this, unmount the volume in Disk Utility, which isn’t as silly.
You can use the Finder contextual menu to encrypt or decrypt the volume if you change your mind.
Summary
To save space in local snapshots made for backups of your Data volume, move bulky items that you back up separately to an extra volume alongside the Data volume.
Set a Quota Size on the extra volume to limit the maximum space it can take.
Use plain APFS or APFS (Encrypted) as the extra volume can’t be protected by FileVault.
If you encrypt the volume, safeguard its password as there’s no recovery option if you lose it.
The extra volume performs as well as any other volume on the internal SSD, and is far faster than using external storage.
Apple has just released an update to XProtect for all supported versions of macOS, bringing it to version 5298. As usual, Apple doesn’t release information about what security issues this update might add or change.
This version adds new rules for MACOS.ADLOAD.CODEP, MACOS.ADLOAD.BYTE.B and MACOS.PIRRIT.OP.OBF, and amends the rule for MACOS.PIRRIT.BM.OBF.
You can check whether this update has been installed by opening System Information via About This Mac, and selecting the Installations item under Software.
A full listing of security data file versions is given by SilentKnight and SystHist for El Capitan to Sequoia available from their product page. If your Mac hasn’t yet installed this update, you can force it using SilentKnight or at the command line.
If you want to install this as a named update in SilentKnight, its label is XProtectPlistConfigData_10_15-5298.
Sequoia systems only
This update has already been released for Sequoia via iCloud. If you want to check it manually, use the Terminal command sudo xprotect check
then enter your admin password. If that returns version 5298 but your Mac still reports an older version is installed, you may be able to force the update using sudo xprotect update
I have updated the reference pages here which are accessed directly from LockRattler 4.2 and later using its Check blog button.
I hope that you enjoyed Saturday’s Mac Riddles, episode 308. Here are my solutions to them.
1: One of two of the three at the start, he left for 12 years with the successor before returning for one more thing.
Click for a solution
Steve Jobs (1955-2011)
One of two of the three at the start (co-founder of Apple with Steve Wozniak and Ronald Wayne), he left for 12 years with the successor (from 1985-1997 he ran NeXT) before returning (in 1997, when Apple bought NeXT) for one more thing (his catch-phrase used to introduce a new product at the end of a keynote).Wikipedia.
2: Writer for Bannister and Crun who originated and named it without a mouse.
Click for a solution
Jef Raskin (1943-2005)
Writer for Bannister and Crun (he first worked for Apple as a contract writer through his company Bannister and Crun) who originated and named it (he created and named the Macintosh project in 1979) without a mouse (he originally disliked the mouse).Wikipedia.
3: First to copy and paste, then changed Pascal and Newton, but was always modeless.
Click for a solution
Larry Tesler (1945-2020)
First to copy and paste (he devised these when working at Xerox PARC), then changed Pascal (he worked with Niklaus Wirth to develop Object Pascal for Lisa and Mac) and Newton (he led development of Apple’s Newton device), but was always modeless (throughout his career he eschewed modal interfaces).Wikipedia.
The common factor
Click for a solution
They’re three of the most influential people responsible for the development of the Mac.
Apple has just released an update to XProtect for all supported versions of macOS, bringing it to version 5297. As usual, Apple doesn’t release information about what security issues this update might add or change.
This version adds a single new rule for MACOS.SOMA.L.
You can check whether this update has been installed by opening System Information via About This Mac, and selecting the Installations item under Software.
A full listing of security data file versions is given by SilentKnight, LockRattler and SystHist for El Capitan to Sequoia available from their product page. If your Mac hasn’t yet installed this update, you can force it using SilentKnight, LockRattler, or at the command line.
If you want to install this as a named update in SilentKnight, its label is XProtectPlistConfigData_10_15-5297.
Sequoia systems only
This update has now been released for Sequoia via iCloud, as of 1930 GMT. If you want to check it manually, use the Terminal command sudo xprotect check
then enter your admin password. If that returns version 5297 but your Mac still reports an older version is installed, you may be able to force the update using sudo xprotect update
I have updated the reference pages here which are accessed directly from LockRattler 4.2 and later using its Check blog button.
I hope that you enjoyed Saturday’s Mac Riddles, episode 307. Here are my solutions to them.
1: Workshop exhibition with thunder and lightning from the A13 in 68.58 cm.
Click for a solution
Studio Display
Workshop (a studio) exhibition (a display) with thunder and lightning from the A13 (it contains an Apple A13 Bionic chip with CPU cores named Thunder and Lightning) in 68.58 cm (27 inches).
2: First person seed vessel contact takes music wherever.
Click for a solution
iPod Touch
First person (I) seed vessel (a pod) contact (touch) takes music wherever (it does). (The first model came with a Samsung S5L8900 ARM SoC.)
3: Notes of brief communication from a pioneering mathematician.
Click for a solution
MessagePad
Notes (in a pad) of brief communication (a message) from a pioneering mathematician (Sir Isaac Newton). (It came with an ARM 610 processor running at 20 MHz.)
In computing, the term Quality of Service is widely used to refer to communication and network performance, but for Macs it has another more significant meaning, as the property that determines the performance of each thread run on your Mac, most importantly in Apple silicon chips.
Processes and threads
Each process running on your Mac consists of at least one thread. Threads are single flows of code execution run on one CPU core at a time, sharing virtual memory allocated to that process, but with their own stack. In addition to the process’s main thread, it can create additional threads as it requires, which can then be scheduled to run in parallel on different cores. As all recent Macs have more than one core, processes with more than one thread can make good use of more than one core, and so run faster.
Take the example of a file compressor. If it’s coded so that it can perform its compression in four threads that can be run simultaneously, then it will compress files in roughly a quarter of the time when it runs on four CPU cores, compared with running on a single core (ignoring input and output to disk).
That only works when those four cores are all free. If your Mac is also trying to build its Spotlight indexes at the same time, the threads doing that will compete with those of your compression app. That’s where the thread’s Quality of Service (QoS) settings come in, as they assign priority. On Apple silicon Macs, a thread’s QoS will also help determine whether it’s run on its Performance or Efficiency cores.
Standard QoS settings
QoS is set by the process, and is normally chosen from the standard list:
QoS 9 (binary 001001), named background and intended for threads performing maintenance, which don’t need to be run with any higher priority.
QoS 17 (binary 010001), utility, for tasks the user doesn’t track actively.
QoS 25 (binary 011001), userInitiated, for tasks that the user needs to complete to be able to use the app.
QoS 33 (binary 100001), userInteractive, for user-interactive tasks, such as handling events and the app’s interface.
There’s also a ‘default’ value of QoS between 17 and 25, an unspecified value, and in some circumstances you might come across others used by macOS.
These are the QoS values exposed to the programmer. Internally, macOS uses a more complex scheme with different values.
CPU core type
When running apps on Intel Macs, because all their CPU cores are identical, QoS has more limited effect, and is largely used to determine priority when there are threads queued for execution on a limited number of cores.
Apple silicon Macs are completely different, as they have two types of CPU core, Efficiency (E) cores designed to use less energy and normally run at lower frequencies, and Performance (P) cores that can run at higher frequencies and deliver maximum performance, but using more energy.
QoS is therefore used to determine which type of core a thread should be run on. Threads with a QoS of 9 (background) are run on E cores, and can’t be promoted to run on P cores, even when there are inactive P cores and the E cores are heavily loaded. Threads with a QoS of 17 and above will be preferentially run on P cores when they’re available, but when they’re all fully occupied, macOS will run them on E cores instead. In that case, the E cores will be run at higher frequencies for better performance with less economy.
If your Apple silicon Mac has a base variant chip with 4 E and 4 P cores, this results in the following:
apps with a total of up to 4 threads at high QoS will be scheduled and run at full speed on the P cores;
when those P cores are all busy with high QoS threads, running another thread will then result in that being run on the E cores, and slightly slower than it would on a P core;
a total of 8 high QoS threads can thus be run on P and E cores together;
when running low QoS background threads on E cores, a maximum of 4 can be run at any time when the E cores are available, but those threads can’t spill over and run on the P cores, even if those are idle.
Controls
As QoS is normally either set by the process for its threads, or for services in their LaunchDaemon or LaunchAgent property list, the user has little direct control. A few apps now provide settings to adjust the QoS of their worker threads. Among those in the compression utility Keka, together with a couple of my own utilities such as the Dintch integrity checker.
In Keka’s settings, you can give its tasks a maximum number of threads, and even run them at custom Quality of Service (QoS) if you want them to be run in the background on E cores, and not interrupt your work on P cores.
Dintch has a simple slider, with the green tortoise to run it on E cores alone, and the red racing car at full speed on the P cores.
App Tamer and taskpolicy
The great majority of threads run at low QoS on the E cores are those of macOS and its services like Spotlight indexing. When a thread has already been assigned a low QoS, there’s currently no utility or tool that can promote it so it’s run at a higher QoS. In practice this means that you can’t accelerate those tasks.
What you can do, though, is demote threads with higher QoS to run at low QoS, more slowly and in the background. The best way to do this is using St. Clair Software’s excellent utility App Tamer. If you prefer, you can use the taskpolicy command tool instead. For instance, the command taskpolicy -b -p 567
will confine all threads of the process with PID 567 to the E cluster, and can be reversed using the -B option for threads with higher QoS (but not those set to low QoS by the process).
That can be seen in this CPU History window from Activity Monitor. An app has run four threads, two at low QoS and two at high QoS. In the left side of each core trace they are run on their respective cores, as set by their QoS. The app’s process was then changed using taskpolicy -b and the threads run again, as seen in the right. The two threads with high QoS are then run together with the two with low QoS in the four E cores alone.
Virtualisation
Although Game Mode does alter the effects of QoS and core allocation, its impact is limited. The one significant exception to the way that QoS works is in virtualisation.
macOS Virtual Machines running on Apple silicon chips are automatically assigned a high QoS, and run preferentially on P cores. Thus, even when running threads at low QoS, those are run within threads on the host’s P cores. This remains the only known method of electively running low QoS threads on P cores.
Key points
Threads are single flows of code execution run on one CPU core at a time, sharing virtual memory allocated to that process, but with their own stack.
Apps and processes set the Quality of Service (QoS) for each of the threads they run.
On Apple silicon chips, low QoS of background results in that thread being run on E cores alone.
Higher QoS threads are preferentially allocated to P cores, but when they aren’t available, that thread will be run on E cores at high frequency.
Some apps now provide controls over the QoS of their worker threads.
App Tamer and taskpolicy let you demote high QoS threads to be run with low QoS on the E cores, but can’t promote low QoS threads to run faster on P cores.
Virtual machines run all threads at high QoS as far as the host Mac is concerned.
I hope that you enjoyed Saturday’s Mac Riddles, episode 306. Here are my solutions to them.
1: Where I left my heart in our words since 2015.
Click for a solution
San Francisco
Where I left my heart (I Left My Heart in San Francisco) in our words since 2015 (when it became the system font across all Macs and Apple’s devices, now abbreviated to SF).
2: The first windy city from Susan Kare until 1997.
Click for a solution
Chicago
The first (the first system font on the Mac from 1984) windy city (Chicago) from Susan Kare (she designed the font) until 1997 (when it was replaced by Charcoal).
3: Bigelow and Holmes brought great clarity for the millennium.
Click for a solution
Lucida Grande
Bigelow and Holmes (it was designed by Charles Bigelow and Kris Holmes) brought great (grande) clarity (the origin of Lucida) for the millennium (it was adopted as the system font from 1999-2000 onwards).
In my introduction to the macOS document versioning system, I explained that a document could lose all its saved versions when moved to another volume. This sequel provides more detail about how you can preserve or lose those saved versions.
When you save the first version of a document, a record is created in the hidden database on the same volume as that containing that document. That record refers to the file not by its name or path, but by its inode, its unique file system number. Anything that preserves that inode number will thus tend to keep its saved versions; anything that creates a new file with its different inode number is guaranteed to lose all versions.
What’s safe
The following actions are version-safe:
moving the file anywhere within the same volume;
renaming the file, changing its permissions, adding extended attributes, adding a custom icon, or editing the file’s data;
creating a Finder Alias or symbolic link to the file;
rolling back a local snapshot, which will return the versions to the same state as at the moment that snapshot was made; volume snapshots include the whole version database, and preserve inode numbers;
‘cloning’ the whole volume to make an identical copy of everything in it;
if the file is stored inside a disk image, then that disk image can be copied or moved safely, or backed up; that also applies to files stored inside Virtual Machines.
What loses versions
The following actions are destructive of versions:
moving the file to a different volume; the original file on its original volume will retain its versions, but they won’t copy across to the new volume;
duplicating the file; the duplicate, an APFS clone file, will have no versions at all;
saving the file as a new file, with a Save As command;
compressing or archiving the file; the copy in the archive won’t have any versions;
saving the file to a file system like ExFAT or MS-DOS, which don’t support versions;
backing the file up to another volume, as versions can’t even be backed up by Time Machine;
moving or copying the file over a network.
If you try to save versions on a file system that doesn’t support them, you aren’t warned when saving those versions, as those appear to remain cached. Normally the first warning is given when you try to close the file.
If you want to keep versions, click on Cancel and save that file to a volume that does support versions.
The version database can only be created on a volume that has sufficient space to support it. This doesn’t normally affect working with regular APFS volumes, but can be a problem if you’re intending to store the file in a disk image or sparse bundle. You should find that a minimum size of 1 GB will support limited versions, but 500 MB is definitely too small for support, and will result in a warning when you try to close the file.
iCloud Drive
The behaviour of versions in iCloud Drive might appear confusing unless you remember the rule that they are saved for that file’s inode number. Here I’ll look at the example of a file that’s in a different folder from iCloud Drive (or Documents in iCloud) and is moved to iCloud Drive.
If that file is already in the current Data volume, that’s the same volume as local copies of what’s in iCloud Drive, so moving it to iCloud Drive keeps it within the same volume, and versions are preserved. If that file is evicted from local storage, that only removes the data for that file, and doesn’t change its inode number. When the file is downloaded again and opened, its versions are still there.
On another Mac connected to the same iCloud Drive, though, the versions are on a different volume on another Mac, so if that opens the file, there are no saved versions available. If that Mac adds its own versions to the file, they will be saved locally, and will be accessible to that Mac.
Saving versions to files already in iCloud Drive is more complex, as different versions of macOS, iOS and iPadOS have saved some into iCloud, so I will look at that in a separate article.
Preserving versions whatever
The only way that I know to preserve all the versions of any file is to save each of them individually in a folder, numbered so that the original versions can be recreated on another volume. While you can use my utility Revisionist to do that, a simpler drag-and-drop approach is provided by Versatility.
There’s no screenshot to show: all you do is drag a file with versions onto Versatility’s blue landing pad. You’ll then be prompted to name and locate the folder it creates containing the file’s versions. Once made, you can move them around like any other Mac folder, back them up, Zip the folder and send it to a colleague.
To reconstitute the original, simply drop the folder onto Versatility’s pad. You’ll then be prompted for the name and location of the file to be saved. That file will contain all the versions saved in the original, ready to use.
If you wish, you can edit the versions in the archive folder created by Versatility. When the file is reconstituted the individual versions will simply be reassembled in numerical order.
I’m also delighted to confirm that both Versatility and Revisionist are fully compatible with macOS Sequoia.
Key point
Versions are linked to the inode number of their file. Actions that preserve that on the same volume should retain those versions. Anything that creates a new inode number or uses a different volume won’t retain those versions.
So far, I have looked at slow launching in two apps, Pages and Calibre. Many of the complaints about the slowest to launch have been levelled at apps in Affinity’s popular suite. Thanks to the efforts of Alain, who generously obtained and provided me with log extracts, I can now try to explain why some apps may launch extremely slowly, occasionally taking more than 30 seconds, and on some Macs several minutes.
Previous work
In the first of my investigations, I demonstrated how Pages and Calibre could take a long time checking frameworks in their app bundle, and how that appeared to account for modest delays. I then looked more closely at Calibre, and found an association between slow launching and cache misses when checking frameworks. One possible explanation for that is the recomputation of SHA-256 hashes used to generate the CDHashes that both identify and verify protected code, and in my third article I showed how that could account for large differences seen in older models.
Affinity Designer 2
Alain has provided me with two particularly valuable log extracts from his MacBook Pro M1 Pro running macOS Monterey. Both were collected during launches of Affinity Designer 2, one from a very slow launch taking close to 30 seconds, and the other from a normal, brisk launch taking little more than a second.
Differences between those are stark: the slow launch took 23.5 seconds from mouse-click to loading user preferences for the app, of which 22.3 seconds was accounted for by framework checks, all of which were cache misses. The fast launch took a total of only 1.0 second and didn’t record any framework or other security checks at all, as it was performed about 12 minutes after the slow launch took place.
Framework checks
Framework checks performed during the slow launch consisted of a similar sequence of log entries to those seen in Pages and Calibre: 10.943986 AppleSystemPolicy Waking up reference: 173
10.944007 AppleSystemPolicy Thread waiting on reference 173 woke up
10.944014 AppleSystemPolicy evaluation result: 173, allowed, cache, 1745876410
10.952412 AppleMobileFileIntegrity AMFI: constraint violation /Applications/Affinity Designer 2.app/Contents/Frameworks/liblibpersona.dylib has entitlements but is not a main binary
10.961105 amfid Entering OSX path for /Applications/Affinity Designer 2.app/Contents/Frameworks/liblibpersona.dylib
10.982192 Security SecTrustEvaluateIfNecessary
10.987489 Security SecTrustEvaluateIfNecessary
11.007538 Security SecTrustEvaluateIfNecessary
11.011106 Security SecTrustEvaluateIfNecessary
11.012004 com.apple.syspolicy.exec Recording cache miss for <private>
20.898736 AppleSystemPolicy Waking up reference: 174
Entries here differ slightly from excerpts for Pages and Calibre, as these are from Monterey rather than Sonoma.
In that case, the time interval between ‘waking up’ the two references is the checking cycle time for liblibpersona.dylib in the app’s Frameworks folder, an amazing 9.955 seconds. That dylib is 1.08 GB in size, and the largest of all the components in the app’s frameworks. If that period was accounted for by computing the dylib’s SHA-256 hash, that would represent a rate of 108 MB/s.
Checking cycle times varied greatly, ranging from 0.03-9.96 seconds over a total of 61 frameworks. As the total size of the Frameworks folder is 2.3 GB, the overall rate is 103 MB/s, similar to that for liblibpersona.dylib alone.
What’s being checked?
Log entries don’t provide any information as to what SecTrustEvaluateIfNecessary is likely to perform on these frameworks that can take anything from 0.03-9.96 seconds for each framework. Many of them are far too short to involve any online check, and in any case those are distinctive in their log entries. Malware scan using any known Yara rules is most unlikely, as:
XProtect Yara rules commonly include file size limits, resulting in few rules applying to larger files, and more rapid completion.
Known checks using Yara rules are all well-recorded in log entries, and the source of those rules is stated clearly.
Yara scans are normally reported with their result.
Scan results are succinct and hardly likely to be lost in a ‘cache miss’.
The most likely activity to account for these long checking times is computation of SHA-256 hashes for the contents of each item in the app’s Frameworks folder. Thus, these occasional extremely long launch times are most probably due to time taken ‘evaluating trust’ by computing hashes for protected code in the Frameworks folder in the app bundle, when those hashes have been flushed from their cache.
Why so extremely slow?
Several events have been credited with causing very long launch times, among them starting the Mac up, and installing updates to XProtect data. It appears most likely that the latter might act indirectly, by flushing at least some of the caches, which may be intentional.
Workarounds
The only strategy that does appear to stop long launch times in most cases is to disable SIP and, in the case of Apple silicon Macs, to set the security mode to Permissive. As I have noted on several occasions, that results in major changes in app security as well. It also disables some features that may be required, including Wallet and Apple Pay. It’s worth being aware that, once so disabled, if those features are to be restored they have to be created from scratch again, and can’t simply be toggled back on. Other side-effects of Permissive Security mode include the refusal of some App Store apps to run.
Those who find the occasional extremely slow app launch unacceptable, but wish to continue to run in Full Security, can pre-warm apps likely to be affected at the start of each session, so making it unlikely that subsequent launches during that session will be slowed. So far, reports of extremely slow launches in Sequoia appear less common, so there may also be benefits to updating macOS if that’s feasible.
Conclusions
The most likely cause of extremely slow app launches is security checking of frameworks and similar within the app bundle, when cached checks aren’t available.
Time taken for those checks is dependent on the size of the protected code being checked, and is most probably attributable to the computation of SHA-256 hashes.
Pre-warming the app by launching it early in each session is a feasible strategy to avoid delays in launching it later.
Although these extreme delays appear to be most common on Intel Macs, they can still affect older Apple silicon chips such as the M1 Pro.
I’m very grateful to Alain and Kristian who have generously run tests and provided their logs for my analysis, and their discussion.
The versioning system built into macOS for the last 14 years must be among its least used features, with many simply ignoring it, and some even blocking it from working. Yet among those who do use its features, it’s probably used more often than backups, and to greater effect.
A couple of weeks ago, when I was working on the code for AppexIndexer, I realised that the changes I was making to enable its search feature were getting out of hand, and should I need to undo them would have required a lot of work. Because the code editor in Xcode supports macOS versioning, I paused my editing for a few moments, opened that document’s versions and stepped back through them to reach the version saved just before I had started making those changes. I then saved that version as a separate fallback copy and resumed my editing.
All this had happened since the last Time Machine backup, which would have been of little use had I needed to revert all those changes. Even automatic hourly backups are too infrequent to capture changes as we make them in active documents. This is just what the versioning system is intended to capture, although to make best use of it, you’ll need third-party software.
Standard access
Most apps that edit documents now support the macOS versioning system by default. Some allow you to auto-save documents as you’re working on them, but all should create a new version each time you save a document. Normally, you might access saved versions using the Revert To… command in the app’s File menu, where you can Browse All Versions… in a full-screen window resembling Time Machine’s interface to it backups.
Although the design looks impressive, it’s not the only means of browsing a document’s saved versions, and in many respects doesn’t support the power and flexibility in the versioning system.
How it works
Many macOS apps that edit documents are based on AppKit, supporting actions in their File menus including Save and Revert To…. Each time you save a document, that automatically uses the Foundation API to save a copy in the hidden version database located in the folder .DocumentRevisions-V100 at the root of the volume on which that document is stored. When you Browse All Versions macOS finds all the previous versions of that document in the volume’s database and displays them, making them available to revert to. Although document-based apps developed using SwiftUI are still relatively unusual, they too should provide full support for versions and the same menu commands.
This relies on:
support in the editing app;
the document being stored in a volume in APFS or HFS+ format; ExFAT, MS-DOS and other file systems don’t normally work, and can’t save versions;
the hidden folder .DocumentRevisions-V100 containing the local version database for that volume.
Although a document’s versions are normally accessed through an app that can edit that document, that isn’t a requirement, and there are several alternatives that can access and manage any versions, including Versions from the App Store, and my own free utilities Revisionist, Versatility and Deep Tools, whose use I will explain in further articles.
On APFS volumes, the version database appears to use clone files to good effect, in preserving changed storage blocks for each saved version, so using a minimum of space.
How it doesn’t work
The biggest limitation of versions is that they don’t move with the document. When you copy or move a document with versions to a different volume, its versions don’t go with it, although moving it within the same volume preserves them intact. Any document only retains the versions created during its editing on its current volume.
Say you have been working on a file named MyBigDocument when it’s stored in your ~/Documents folder, and there are versions saved for it there. Copy that to an external disk, and that copy now has no versions at all. Edit it there, and it will build its own collection of versions on that disk, but you won’t be able to access those if you open MyBigDocument from your ~/Documents folder. You can use my utilities to move its versions around from volume to volume, or to another Mac, but left to its own devices, macOS doesn’t do that for you.
Versions become complicated in iCloud Drive. iPhones and iPads also use versions, but their behaviour in iCloud storage has changed over time. I will explore current behaviour in Sequoia in a separate article.
One common fear of using versioning is that the database can accumulate a seemingly unlimited number of versions of files, and consume large amounts of space. You can get a good estimate of how large a volume’s version database folder is using the Terminal command sudo du -sh [pathtovolume]/.DocumentRevisions-V100
followed by authentication. The correct path to use for that on the current Data volume is /System/Volumes/Data/.DocumentRevisions-V100.
macOS doesn’t provide any means of editing what’s stored in the versions database, nor of weeding old versions. On one of my Macs, its internal Data volume currently has several documents for which there are more than 100 versions, with a maximum of 153. However, the versions database on that volume is only 38 MB for a total of more than 35,000 files in ~/Documents.
Practical use
In addition to my example at the start of this article, I quite frequently use versions as an extended undo. Some minutes into an editing session I might realise that an earlier change such as deleting paragraphs needs to be undone. Rather than working my way back through a long series of Undo commands, the easiest way to fix that is to open the document’s versions using Revisionist, locate an earlier version containing the deleted paragraphs, copy those from that version and paste them into the current document that I’m editing.
Some documents have important previous versions that I want to preserve when I move them across to another Mac or to external storage. For those I drag and drop the document onto Versatility to create a folder containing all that document’s versions, move that to the other Mac, and drop that folder onto Versatility there. That recreates the original document with all its versions stored safely in its new location.
When certain documents have been very active, and are now complete so I don’t want to preserve all those old versions, I can clear them all by moving that document to another volume, and moving it back again. If I want to reduce the versions but keep some of them for posterity, then I open them using Revisionist and delete the old versions I don’t want to retain any longer.
Most important of all, I know that so long as I have saved a document, if I want to recover any previous version I can do so using Revisionist without having to look through my Time Machine backups. With macOS versioning, Command-S doesn’t just save my work, it also maintains that document’s change history.
Key points
Many apps support macOS versions. They normally feature a Revert To… command in their File menu, although not all do.
Save (Command-S) creates a new version of a document automatically, and maintains its change history.
Use additional apps like Revisionist and Versatility to get the most out of versions.
A document’s versions don’t move with it to another volume or Mac, but can be preserved using Versatility.
Documents can accumulate a large number of versions when they’ve been saved many times.
Delete all a document’s versions by moving it to another volume, then moving it back again.
Using versions gives you finer detail of changes that are lost in backups.
Apps, frameworks and other items come in bundles, structured hierarchies of folders and files that look like a single file in the Finder. This article explains what bundles are and shows how they are structured.
Bundles came to Mac OS X from its NeXTSTEP parent, not Classic Mac OS. The latter assembled its apps from many resources stored in the app’s resource fork. NeXTSTEP and Mac OS X instead built their apps from component files arranged in a series of directories. Those bundles gathered together all the files required for apps and similar items, into what was then presented to the user as a single item. Modern app bundles are descended from ‘new-style’ bundles introduced in Mac OS X, which had a minimum requirement of a top-level folder named Contents, and inside that a property list named Info.plist.
Bundles and packages
When a folder has a name with an extension for a standard bundle type, such as .app, .bundle, .plugin or .kext, the Finder will display and treat it as if it were a file, also offering to Show Package Contents in its contextual menu. For historical reasons, bundles with the extension .framework aren’t treated as bundles, but are displayed and navigated as if they were still folders.
Although bundles have similarities with packages, Apple draws a clear distinction between them, even if the Finder doesn’t. Directory-based documents and similar items that are used to contain multiple files for Rich Text Format Directory (RTFD) and other documents don’t conform to the rules for bundles, and should be referred to as packages to avoid confusion. Look inside an RTFD package, for example, and you’ll see the structure:
Contents, a directory containing a PkgInfo file
files containing graphics to be included within the documents
TXT.rtf, a Rich Text Format file containing the styled text.
which has some common features, but is structured differently and lacks both an Info.plist file and executable code.
Apps and frameworks
The fundamental requirements for an app or related bundle, including those for app extensions or appexes, are:
The whole bundle’s contents are within a single top-level folder named Contents.
Within Contents is an Info.plist property list containing information and settings for the bundle.
There’s executable code, normally in the form of a Mach-O binary inside the MacOS folder.
For historical and compatibility reasons, there’s a PkgInfo file containing a Classic Mac OS type and creator code for the bundle, e.g. APPL and ???? for an app.
However, frameworks have a different and more complex structure, lacking a Contents folder, providing at least one version of the framework, and burying the Info.plist file deeper in the hierarchy. This is shown in full later.
App bundles can contain frameworks and other app bundles, as can framework bundles. In the worst case you may need to dive deep into a series of folder hierarchies to reach your goal.
Info.plist
The information property list Info.plist contains key-value pairs in XML format, for attributes such as:
bundle name (CFBundleName),
name of bundle executable (CFBundleExecutable),
version (CFBundleShortVersionString),
document types handled by the app (CFBundleDocumentTypes),
app identifier (CFBundleIdentifier),
other metadata.
Because of the importance of its contents, code signatures protect the Info.plist from being changed in any way.
App bundle
In addition to the minimum requirements of an Info.plist, executable code, and PkgInfo file inside its Contents folder, apps almost invariably contain several additional folders, among them:
Resources is universal, and contains any of a rich range of images, localised versions of menu and other text contents, the app icon and any for custom document types, storyboards for GUI elements, Help books, and more.
PlugIns is now common, to contain appexes in particular.
Library, containing one or more sub-folders with QuickLook and Spotlight support for custom document types, any login items or property lists for LaunchServices, and other support and extensions for the main app.
Apps obtained through the App Store should contain a _MASReceipt folder with the store receipt inside it.
Signed apps will have a _CodeSignature folder containing the CodeResources file with signature data.
Notarized apps may have a notarization ticket in a CodeResources file, but that isn’t required for notarization to work.
Some apps may have an embedded.provisionprofile file containing their provisioning profile.
Other folders, such as Frameworks for nested framework bundles.
This layout is shown below, with the essentials in light red.
Framework bundle
In comparison, framework bundles are deeper, lack a top-level Contents folder, contain at least one version of the framework (although two or more are now exceptional), and come with symbolic links. This is shown below, again with the essentials in light red.
Three symlinks merit explanation:
Versions/Current links to A, the current (and only) version of the framework.
a symlink with the same name as the framework’s code is at the top level of the bundle, alongside the Versions folder. That links through the Current symlink, giving the path Current/A/.
a symlink named Resources is also at the top level of the bundle, alongside the Versions folder. That too links through the Current symlink, giving the path Current/A/Resources.
Thus paths to the Info.plist file include Resources ➜ Info.plist via the top-level Resources symlink, Versions/Current ➜ Resources via the Current symlink, and Versions/A/Resources without using any symlinks. This demonstrates how confusing this layout can appear in the Finder.
Frameworks commonly include other nested frameworks, appexes in their PlugIns folder, and a diverse range of other resources including localisations in the Resources folder.
Conclusions
Bundles are formal structures of folders and files used for apps and their relatives, and for frameworks. Packages used for RTFD and other documents have different structural requirements.
App bundles have a top-level Contents folder, inside which is an Info.plist property file, and executable code inside a MacOS folder.
Framework bundles don't have a Contents folder, but contain versions of the framework, normally only one named A. They use important symlinks to provide standard access paths to their contents, and their Info.plist file is inside their Resources folder.
Info.plist files and other contents are protected by code signatures and can't be modified without breaking the signature. However, they contain valuable information about the app or framework.
Building bundles by hand is prone to error, causing bugs. When possible, leave it to Xcode.
Apple has just released an update to XProtect for all supported versions of macOS, bringing it to version 5296. As usual, Apple doesn’t release information about what security issues this update might add or change.
This version adds a single new rule for MACOS.DOLITTLE.RDRGOCON, and amends the rules for MACOS.8032420, OSX.HMining.D and MACOS.SOMA.D.
You can check whether this update has been installed by opening System Information via About This Mac, and selecting the Installations item under Software.
A full listing of security data file versions is given by SilentKnight, LockRattler and SystHist for El Capitan to Sequoia available from their product page. If your Mac hasn’t yet installed this update, you can force it using SilentKnight, LockRattler, or at the command line.
If you want to install this as a named update in SilentKnight, its label is XProtectPlistConfigData_10_15-5296.
Sequoia systems only
This update has also been released for Sequoia via iCloud. If you want to check it manually, use the Terminal command sudo xprotect check
then enter your admin password. If that returns version 5296 but your Mac still reports an older version is installed, you may be able to force the update using sudo xprotect update
I have updated the reference pages here which are accessed directly from LockRattler 4.2 and later using its Check blog button.
Updated 1928 GMT 22 April 2025 – now available fully via iCloud.
I hope that you enjoyed Saturday’s Mac Riddles, episode 304. Here are my solutions to them.
1: Glance at a miniature image or foretaste.
Click for a solution
Quick Look
Glance (a quick look) at a miniature image (a thumbnail) or foretaste (a preview).
2: Desktop gadget should give your beer a creamy head.
Click for a solution
Widget
Desktop (where it can go) gadget (a widget) should give your beer a creamy head (it’s also the name of a device put in some canned beers to inject nitrogen and give them a creamy head).
3: Merchant of medics ensures indexing of contents.
Click for a solution
mdimporter
Merchant (an importer) of medics (MDs) ensures indexing of contents (what it does for Spotlight indexes).
The common factor
Click for a solution
They are all supported by app extensions (appexes).
Presenting complex verbal information in an accessible way remains one of the great challenges in interface design. This article explains how SwiftUI List Views offer solutions that are superior to those provided in AppKit, and why I have chosen to use them in several of my recent apps, such as LogUI and AppexIndexer.
Each of the apps I discuss here has a common problem: it has to display a series of text records consisting of fields whose contents can be completely empty or expand into long paragraphs. Thus the condensed length of each row, representing a single record, can be as short as 30 characters, or as long as hundreds, some with embedded line-breaks. Fields contain contrasting data, such as datestamps, flowing text and UUIDs. The user needs to scan rows rapidly, following patterns in fields and their contents, and to read each carefully.
There are some ground rules, notably:
rows must remain visually separate,
the order of fields in each row must be the same,
each field must be displayed in full and not truncated,
fields must be visually distinct across each row,
field width must be adjusted automatically to avoid horizontal scrolling.
There are centuries of experience in print design of tables, to which computers add dynamic resizing of columns and ready use of colour.
In some circumstances, AppKit’s Table View works well, here in Activity Monitor. However, adjustable column widths can’t overcome one problem shown here, where uniform column width wastes space for short process names, and truncates others.
This shows how poorly a Table View copes with log entries in Console.
My best solution using AppKit is for rows of untruncated text, with colour used to distinguish fields within them. Unfortunately, some rows inevitably overflow into multiple lines, and may require very wide windows to remain accessible.
This is SwiftUI’s List View in action with a log excerpt in LogUI. Although it might appear desirable to allow manual adjustment of field width, it’s more practical to provide options to change which fields are displayed, and so accommodate narrower windows. Sometimes line breaking in fields isn’t good, but I think this is a problem of content (computing terms being formed by concatenating series of words) rather than view design.
This is AppexIndexer, where I use an easily distinguished emoji to mark a significant field positioned in the middle of some rows. This aids navigation when scanning rows quickly, but doesn’t rely on the reading of the emoji.
Here’s another example, this time using many icons drawn from SF Symbols rather than pure text. This works well with fewer rows, but rendering many of those icons in thousands of rows may not be as efficient.
For macOS, SwiftUI continues to suffer serious omissions, and in some circumstances isn’t yet a good fit. But its List Views are a compelling reason for using SwiftUI in many native Mac apps. Further details are given in the references below, and the Appendix provides example source code for a basic implementation in SwiftUI.
Apple has just released an update to XProtect for all supported versions of macOS, bringing it to version 5295. As usual, Apple doesn’t release information about what security issues this update might add or change.
This version adds a single new rule for MACOS.SOMA.BYTE.SEQUENCE.B.
You can check whether this update has been installed by opening System Information via About This Mac, and selecting the Installations item under Software.
A full listing of security data file versions is given by SilentKnight, LockRattler and SystHist for El Capitan to Sequoia available from their product page. If your Mac hasn’t yet installed this update, you can force it using SilentKnight, LockRattler, or at the command line.
If you want to install this as a named update in SilentKnight, its label is XProtectPlistConfigData_10_15-5295.
Sequoia systems only
This update has also been released for Sequoia via iCloud. If you want to check that manually, use the Terminal command sudo xprotect check
then enter your admin password. If that returns version 5295 but your Mac still reports an older version is installed, you can force the update using sudo xprotect update
I have updated the reference pages here which are accessed directly from LockRattler 4.2 and later using its Check blog button.
I hope that you enjoyed Saturday’s Mac Riddles, episode 303. Here are my solutions to them.
1: Harvard or Yale cipher brought emoji and Ugaritic.
Click for a solution
Unicode
Harvard or Yale (a university or uni) cipher (a code) brought emoji and Ugaritic (two of its supported character sets).
2: US standard for 128 Roman characters now over 60.
Click for a solution
ASCII
US standard (initially ASA X3.4-1963, later an ANSI standard) for 128 Roman characters (originally consisted of only 128 characters including a basic Roman alphabet) now over 60 (first published in 1963, it turns 62 years old this year).
3: Two plus 128 more came in 1989, gained a euro in 1998, and still supported.
Click for a solution
MacRoman
Two plus 128 more (it consists of the 128 characters in ASCII, plus 128 more including punctuation, symbols and diacritics) came in 1989 (first appeared in System 6.0.4 in that year), gained a euro in 1998 (the only change made since introduction), and still supported (it is, although now encoded in UTF-8).
The common factor
Click for a solution
They are text encodings that have been used in Mac OS.
For many years, most types of disk image were inefficient in their use of storage space, as they occupied their full size on disk. Until recently, when you created a 5 GB read-write UDIF disk image, one of the most popular, it invariably took up 5 GB in storage, even when empty. This also applied to the raw disk images used by Virtual Machines: give a VM 100 GB, and that’s just what it took on disk. With the introduction of sparse files in APFS, this has changed, and many disk images now only take the space they need. I’m not sure exactly when this change occurred, as Apple still doesn’t appear to have documented it, but it seems to have changed with macOS Monterey.
This is easiest to see with a plain read-write disk image, created using DropDMG or Disk Utility.
Disk image
Here’s one I made earlier, a whole 350 GB in size. When it’s created, it’s automatically attached and mounted at full size. For the sake of example, I then copied a large IPSW to it, so it wasn’t entirely empty.
Unmount it and Get Info on the disk image and you’ll see it does still take up a full 350 GB on disk. Mount it again, though, and APFS works its magic. You can see this in LogUI, or the custom log extract provided by Mints.
When unmounted again it has shrunk down to take little more than the size of the IPSW file in it, at just over 17 GB. That’s less than 5% of its nominal size, without using any compression.
It’s worth looking through entries in the log made by APFS for the mount process. First, APFS checks whether the data store for the disk image is already sparse: 01.470 container_backingstore_is_sparse:1652: Image url file:///Volumes/LaCie2tb/350gbudif.dmg Image path /Volumes/LaCie2tb/350gbudif.dmg
01.470 container_backingstore_is_sparse:1659: Image /Volumes/LaCie2tb/350gbudif.dmg is a flat file, do not consider as sparse
It then sets it to sparse, ready for sparsification: 01.475 handle_apfs_set_backingstore:6207: disk9s1 Set backing store as sparse
01.475 handle_apfs_set_backingstore:6240: disk9 Backing storage is a raw file
Space Manager performs an initial scan for free blocks without any Trimming: 01.479 spaceman_scan_free_blocks:4136: disk9 scan took 0.004272 s (no trims)
01.479 spaceman_fxc_print_stats:477: disk9 dev 0 smfree 81258479/85398014 table 4/452 blocks 81258479 32766:20314619:79974226 100.00% range 35869:85362145 99.95% scans 1
Space Manager then scans and Trims free storage blocks, taking just over 0.7 second to complete: 02.196 spaceman_scan_free_blocks:4106: disk9 scan took 0.717433 s, trims took 0.715705 s
02.196 spaceman_scan_free_blocks:4110: disk9 81258479 blocks free in 25 extents, avg 3250339.16
02.196 spaceman_scan_free_blocks:4119: disk9 81258479 blocks trimmed in 25 extents (28628 us/trim, 34 trims/s)
02.196 spaceman_scan_free_blocks:4122: disk9 trim distribution 1:0 2+:0 4+:0 16+:0 64+:0 256+:25
VM
What happens with an Apple silicon VM is a bit more complicated, and harder to observe. This time the virtualisation app should create the disk image inside the VM bundle as a sparse file to begin with, then copy into that what’s needed for the VM, so skipping the first mount stage and Trimming during the second mount.
The result is the same, though, with a 350 GB VM taking just 22 GB on disk. Inspect that disk image using my free utility Precize, and you’ll see that economy confirmed, and the Sparse File flag set.
Conclusions
For plain read-write disk images and those inside VMs to be sparse files:
they must contain a suitable raw disk image, such as UDIF read-write;
the host file system must be APFS, as HFS+ doesn’t support sparse files;
for normal disk images, they must be stored on an SSD that supports Trimming;
there must be sufficient free space in the disk image;
the guest file system can be APFS, either plain or encrypted, or HFS+J;
for normal disk images, they must have been mounted at least once since first being created.
Login
