Apple has just released an update to XProtect for all versions of macOS from El Capitan or so, bringing it to version 5273.
Apple doesn’t release information about what security issues this update might add or change. This adds Yara definitions for MACOS.DOLITTLE.CT, MACOS.SHEEPSWAP.CT and MACOS.SOMA.CT using a new format of rule, with each rule given a UUID and listing SHA256 hashes of file size.
You can check whether this update has been installed by opening System Information via About This Mac, and selecting the Installations item under Software.
A full listing of security data file versions is given by SilentKnight, LockRattler and SystHist for El Capitan to Sequoia available from their product page. If your Mac hasn’t yet installed this update, you can force it using SilentKnight, LockRattler, or at the command line.
If you want to install this as a named update in SilentKnight, its label is XProtectPlistConfigData_10_15-5273.
If you’ve upgraded to Sequoia and are still stuck at a version number of 0 or 5272, you can either leave macOS to catch up with this in its own good time, or you can force an update by typing into Terminal sudo xprotect update
then entering your admin password.
I have updated the reference pages here which are accessed directly from LockRattler 4.2 and later using its Check blog button.
In the previous article, I outlined what extended attributes do, and how they work in macOS. I also started to explain how some are considered ephemeral, while others are persistent. This article continues from there, by documenting how macOS decides what to do with them when a file containing xattrs is copied.
Although Apple does now explain a little about this in the context of the FileProvider framework and syncing with cloud services, the only useful documentation is provided in man xattr_name_with_flags, and two source code files that are part of the open source copyfile component.
In 2013, as part of its enhancements for iCloud in particular, Apple added support for flags on xattrs to indicate how those xattrs should be handled when the file is copied in various ways. Rather than change the file system, Apple opted for what’s perhaps best seen as an elegant kludge: appending characters to the end of the xattr’s name.
If you work with xattrs, you’ve probably already seen this in those whose name ends with a hash # then one or more characters: that’s actually the flags, not part of the name, what Apple refers to as a ‘property list’. To avoid confusion I won’t use that term here, but refer to them as xattr flags. A common example of this is com.apple.lastuseddate#PS, which is seen quite widely. In recent years, Apple has added one flag, B, and there’s another to come with Sequoia.
Xattr flags
Flags can be upper or lower case letters C, N, P, S or B, and invariably follow the # separator, which is presumably otherwise forbidden from use in a xattr’s name. Upper case sets or enables that property, while lower case clears or disables that property. There are currently (macOS 14.6.1) five properties:
C: XATTR_FLAG_CONTENT_DEPENDENT, which ties the flag and the file contents, so the xattr is rewritten when the file data changes. This is normally used for checksums and hashes, text encoding, and position information. The xattr is preserved for copy and share, but not in a safe save.
P: XATTR_FLAG_NO_EXPORT, which doesn’t export or share the xattr, but normally preserves it during copying.
N: XATTR_FLAG_NEVER_PRESERVE, which ensures the xattr is never copied, even when copying the file.
S: XATTR_FLAG_SYNCABLE, which ensures the xattr is preserved during syncing with services such as iCloud Drive. Default behaviour is for xattrs to be stripped during syncing, to minimise the amount of data to be transferred, but this will override that.
B: XATTR_FLAG_ONLY_BACKUP, which keeps the xattr only in backups, including Time Machine (added recently).
These operate within another general restriction of xattrs: their name cannot exceed a maximum of 127 UTF-8 characters.
Defaults
macOS provides a standard ‘whitelist’ of default flag settings for different types of xattr. These aren’t contained in a configuration file, but are baked into the xattr flag code, where as of macOS 14.6.1 the following default flags are set for different types of xattr (* here represents the wild card):
com.apple.quarantine – PCS
com.apple.TextEncoding – CS
com.apple.metadata:kMDItemCollaborationIdentifier – B
com.apple.metadata:kMDItemIsShared – B
com.apple.metadata:kMDItemSharedItemCurrentUserRole – B
com.apple.metadata:kMDItemOwnerName – B
com.apple.metadata:kMDItemFavoriteRank – B
com.apple.metadata:* (except those above) – PS
com.apple.security.* – S
com.apple.ResourceFork – PCS
com.apple.FinderInfo – PCS
com.apple.root.installed – PC
Copy intents
Also contained in the source code is a table of intents, that explains how different types of copy are affected by different combinations of xattr flag. Currently, those are:
XATTR_OPERATION_INTENT_COPY – a simple copy, preserves xattrs that don’t have flag N or B
XATTR_OPERATION_INTENT_SAVE – save, where the content may be changing, preserves xattrs that don’t have flag C or N or B
XATTR_OPERATION_INTENT_SHARE – share or export, preserves xattrs that don’t have flag P or N or B
XATTR_OPERATION_INTENT_SYNC – sync to a service such as iCloud Drive, preserves xattrs if they have flag S, or have neither N nor B
XATTR_OPERATION_INTENT_BACKUP – back up, e.g. using Time Machine, preserves xattrs that don’t have flag N
Use
If you want a xattr preserved when it passes through iCloud, you therefore need to give it a name ending in the xattr flag S, such as co.eclecticlight.MyTest#S. Sure enough, when xattrs with that flag are passed through iCloud Drive, those xattrs are preserved even if the default rule would treat them differently. Similarly, to have a xattr that is stripped even when you just make a local copy of that file, append #N to its name.
There’s a further limit imposed on xattrs synced by FileProvider, including those for iCloud Drive, that strips all individual xattrs that are larger than a certain size. Apple gives that as “about 32KiB total for each item”, and my measurements performed in the recent past put that at about 32,650 bytes, slightly less than 32,767.
In itself, this information is valuable if you ever use any metadata stored in xattrs. It’s used in my intergrity-checking utilities Dintch, Fintch and cintch to ensure the xattr containing a file’s hash isn’t stripped by passage through iCloud Drive, for instance. On Tuesday morning next week, once Sequoia has been released, I’ll explain how Apple has extended this system to achieve something that many have been wishing for.
One of the innovative features in classic Mac OS was its use of resource forks, allowing structured metadata to be attached to any file. When Mac OS X merged that with the more traditional Unix approach adopted by NeXTSTEP, those were nearly lost. Classic Mac apps were restructured from storing most of their components, including their executable code, in their resource fork, when Mac OS X flattened those into an app bundle consisting of a hierarchy of separate files in folders, without any resources.
For the first four years of Mac OS X resource forks were reluctantly tolerated, until the solution came in 10.4 with the introduction of extended attributes, including one to contain what had previously been stored in the resource fork, which became an extended attribute or xattr with the name com.apple.ResourceFork.
All files in HFS+ and APFS (and other file systems) contain a fairly standard set of metadata known as attributes, information about a file such as its name, datestamps and permissions. Xattrs are extensions to those that contain almost any other type of metadata, the first notable xattr coming in Mac OS X 10.5, named com.apple.quarantine. That contains quarantine information for apps and other files downloaded from the internet, in a format so ancient that the quarantine flag is stored not in binary but as text.
The quarantine xattr provides a good demonstration of some of the valuable properties of xattrs: it can be attached to any file (or folder) without changing its data, and isn’t included when calculating CDHashes for code signatures. It can thus be added safely without any danger of altering the app or its code, although it does change the way in which macOS handles the code, by triggering security checks used to verify it isn’t malicious. Once those have been run, the flag inside the quarantine xattr can be changed to indicate it has been checked successfully.
Far from being a passing phase, or dying out as some had expected, xattrs have flourished since those early days. This has happened largely unseen by the user: few change anything revealed in the Finder’s Get Info dialog, although they’re used to store some forms of visible metadata such as Finder tags, and the URL used to download items from the internet. Editing xattrs is normally performed silently: you’re not made aware of changes in the quarantine xattr, and in most cases the only way to manage xattrs is to use the xattr command tool, or one of very few apps like xattred that can edit and manage them.
Examples
Among the well-known and important xattrs you can encounter are:
com.apple.quarantine the quarantine xattr, containing a quarantine flag
com.apple.rootless marks items individually protected by System Integrity Protection (SIP)
com.apple.provenance contains data about the origin of apps that have been quarantined
com.apple.metadata:kMDItemCopyright records copyright info
com.apple.metadata:kMDItemWhereFroms the origin of downloaded file as a URL
com.apple.metadata:_kMDItemUserTags Finder tags
com.apple.TextEncoding reveals text file encoding
com.apple.ResourceFork a classic Mac resource fork
Storage
In APFS and HFS+, xattrs aren’t stored with file data, nor with a file or folder’s normal attributes.
For smaller extended attributes up to 3,804 bytes, their data is stored with the xattr in the file system metadata. Larger extended attributes are stored as data streams, with separate records, but still separately from the file data. Apple doesn’t give a limit on the maximum size of xattrs, but they can certainly exceed 200 KB, and each file and folder can have an effectively unlimited number of them.
Persistence
Most file systems to which macOS can write either handle xattrs natively (HFS+, APFS), or macOS uses a scheme to preserve them, as in the hidden files written to FAT and ExFAT volumes. NFS is an important exception, and files copied to NFS will have all their xattrs stripped. Neither are extended attributes unique to Macs: most file systems used by Linux support them, and even Windows can at a push.
Because xattrs contain a wide range of metadata, some are treated as being ephemeral, others as persistent. Moving files with xattrs around within the same volume shouldn’t affect their xattrs, as that takes place within the same file system. Copying files to another volume, even if both use APFS, may leave some xattrs behind if they’re considered to be ephemeral.
The most complex situation is when a file with xattrs is moved to iCloud Drive. The Mac that originated that file is likely to retain most if not all of its xattrs, because the local copy remains within the same volume and file system. However, not all xattrs are copied up to iCloud storage, so other Macs accessing that file may only see a small selection of them. The rules for which xattrs are to be preserved during file copying, including in iCloud Drive, are baked into macOS, and the subject of the next article.
I hope that you enjoyed Saturday’s Mac Riddles, episode 272. Here are my solutions to them.
1: Used by courting birds with a haven for video and audio to replace the others.
Click for a solution
DisplayPort
Used by courting birds (a courtship display) with a haven (a port) for video and audio (it can carry both) to replace the others (intended to replace the answers to 2 and 3, as well as VGA and others).
2: 506 Romans can handle analogue and digital to display.
Click for a solution
DVI
506 Romans (DVI in Roman numerals = 506) can handle analogue and digital (has both DVI-I for analogue support, and DVI-D digital-only) to display (it’s for video output to displays and TVs).
3: With CTA-861, 19 pins and five connectors, it’ll carry all your media, even HDCP.
Click for a solution
HDMI
With CTA-861 (the standards it uses for video and more), 19 pins (in its connectors) and five connectors (it supports five different connectors now), it’ll carry all your media (it will), even HDCP (a form of DRM for use over HDMI).
The common factor
Click for a solution
They are all video interfaces that have been supported by Macs.
With macOS Sequoia fast approaching from the horizon comes the question as to how to upgrade and update, whether to Sequoia or one of its recent predecessors. If you’re happy to go with what Software Update offers, then that’s usually simplest and most efficient. This article considers what you should do if you want something different, from updating to any previous version, to using a single installer to update several different Macs.
Procedures given here should work with all versions of macOS from Monterey onwards. They may work too with Big Sur, but its installers weren’t always as reliable, so you should there be well-prepared to have to migrate from a backup in case the installation creates a fresh, empty Data volume instead of firmlinking up to your existing one.
Which installer?
As Apple discontinued standalone updater packages when it introduced Big Sur, the choice now is between downloading the full Installer app, and performing the process in Recovery mode. The latter severely limits your choice to what it’s prepared to offer, so you’re almost certainly going to need to obtain the full Installer for the version of macOS you want. Rather than use the Installer app provided in the App Store, download the Installer package from the links given by Mr. Macintosh. Those provide a package that’s easier to store and move around, unlike the Installer app itself. It will typically be a little over 13.5 GB, and works on both Intel and Apple silicon Macs.
Standard procedure
As with any update or upgrade, first ensure you have a full recent backup before starting. If anything does go wrong during the procedure you’ll then be able to perform a fresh install and migrate from that backup.
Unless you want to install everything afresh and migrate from your backup, don’t try erasing either your System or Data volume. You’d have to do that in Recovery mode anyway, limiting your options as to which version of macOS you can install unless you create a bootable installer first.
Double-click the installer package to launch it in the Installer utility. The default is to save the Installer app to your current Applications folder, which should work fine as long as you remember to delete it once you’ve finished. Once complete, launch that Installer app and follow its instructions.
When macOS restarts at the end of the process, check the version now running, confirm that your Data volume has survived intact, and run SilentKnight to ensure that all security data files are up-to-date.
Recovery
Intel Macs have a slight advantage when it comes to installing macOS in Recovery mode, as depending on the keys held during startup, you should be able to coax a choice of versions out of an Intel system. Unless you simply want to install or update to the current version, though, you’ll probably want to avoid doing so in Recovery.
There’s another good reason for not using Recovery, in that delivery of installers to Macs running in Recovery can be painfully slow, and you may well be in for a longer wait than if you downloaded the Installer direct.
However, if you want to erase the current boot volume group on your Mac’s internal storage so you can install a fresh copy of macOS and restore the contents of its Data volume from backups, Recovery is normally the best place to do that. Apple works through the process for Intel Macs, and Apple silicon models. The key step is to select the Macintosh HD boot volume group and click on the Erase tool to perform Erase Volume Group.
When the SSV was first introduced in Big Sur, there were many problems resulting from erasing just one volume in the boot volume group. If that happened to be the System volume, when macOS was installed it created a new firmlinked Data volume, leaving the existing Data volume as an orphan. That was usually done in a misguided attempt to have a fresh install of the System volume and SSV while keeping the existing contents of the Data volume, but doesn’t do that. Every installation of the SSV in any given version of macOS since Big Sur is identical, so it isn’t necessary to erase it, but simply to install or update macOS.
Bootable installer disk
Another traditional way to install macOS is using a bootable installer disk, normally a USB ‘thumb’ drive, although you can also create a small HFS+ volume for the purpose on an external SSD. Apple provides detailed instructions for doing this using a range of versions of macOS.
In many cases, installing a version of macOS older than the one that’s currently running requires this, as old Installers usually fail to run in newer macOS. Unfortunately, on Apple silicon Macs, this isn’t the powerful tool that it once was, as the Mac doesn’t boot fully from the external disk, and as a result it has no role in dealing with problems with internal storage.
Virtual Machines on Apple silicon
Installer apps and Recovery installs both work fine in virtual machines running on Apple silicon hosts. However, there’s one special circumstance you need to beware of. One of the major new features in virtualisation in Sequoia is support for iCloud and some other services dependent on Apple ID. If you want to use those, then the VM must be created new in Sequoia, using a Sequoia IPSW image. You can’t update or upgrade an existing VM from a previous version of macOS and use iCloud services in it.
Summary
If you can, use Software Update to update or upgrade macOS, as it minimises download size and is simplest.
If you want to perform a different update, or run one installer on several Macs, download and use the appropriate Installer package.
If you want to erase the existing system including all your data, use Recovery mode to erase the whole volume group, then install macOS and migrate from your backup.
Never erase only your Mac’s System volume, as that will orphan its current Data volume.
If you want to downgrade to an older version of macOS, you’ll probably need to do so from a bootable installer disk.
If you want a VM to use iCloud, then create a fresh VM using a Sequoia IPSW, as an upgraded VM can’t access iCloud.
Although we won’t know for sure until Apple releases the upgrade to macOS Sequoia next month, once again it will probably be presented as an update rather than a macOS upgrade. This means that, instead of Software Update downloading a complete Sequoia installer app, if you do choose to upgrade, it will be run through Software Update the same way that it might have updated from 14.5 to 14.6.
Although this is more efficient, resulting in a smaller update and faster completion, it also opens up the possibility of human error: what if you accidentally opt to upgrade, or click on SilentKnight’s Install all updates button? This article explains how you can stop that upgrade from completing, and how you could upgrade using SilentKnight instead of Software Update.
Updating or upgrading macOS
When SilentKnight has completed checking for updates, if there’s a macOS update or upgrade available you can install it if you wish, from within SilentKnight. Although my personal preference is to hand macOS updates over to Software Update in Settings, SilentKnight should also work fine, up to a point.
To test this, I opened a VM running Sonoma 14.6, with XProtect 5270 and XPR 140. When it had found all three updates available, I clicked on the Install all updates button, just as I have always advised you not to! SilentKnight proceeded to download the macOS 14.6.1 update, but once that was complete it failed to install it. It then proceeded to download the XProtect and XPR updates, which it did successfully install on its own.
There was a vague notification that “Some updates could not be installed”, and the VM was left in 14.6, with XProtect and XPR correctly updated.
At that stage, Software Update stated the 14.6.1 update was available, offering a Restart Now button. When I clicked on that, the VM restarted and installed the 14.6.1 update successfully.
SilentKnight doesn’t provide the handy progress indicator that Software Update does, but just turns its busy spinner until the updates have finished. So you may still prefer to install macOS updates using Software Update. However, the end result should be just the same if you let SilentKnight do it, and finish off the installation using Software Update.
Downloading updates
Using another copy of the same VM running Sonoma 14.6 with outdated XProtect and XPR, I set SilentKnight’s settings to download but not install updates, then clicked the Download all updates button.
This left all the updates uninstalled, but there was no sign of them in the standard /Library/Updates folder as documented for softwareupdate. I looked high and low for those updates, but was unable to find them anywhere. I therefore recommend that you don’t use this option until someone has worked out where those downloaded updates are kept.
Unwanted macOS updates
If you click either the Install all updates or Download all updates button and one of the updates is for macOS, that will leave Software Update poised to complete the installation. If you didn’t want to install that macOS update, there is a way that you can now persuade Software Update to forget that it has been downloaded and is waiting, ready to install. This is most useful if you didn’t intend updating macOS, and now want to undo the process.
Shut your Mac down, then start it up in Safe mode. Leave it there for a minute or so, then restart it back into normal mode. Those uninstalled updates should now have been flushed, and Software Update is back to where it started.
Summary
You should now be able to install macOS updates using SilentKnight if you wish. When warned that some updates weren’t installed, open Software Update settings and complete the installation there using the Restart Now button.
Don’t use SilentKnight’s setting to only download but not install updates, as the downloaded updates can’t be found and used.
If you inadvertently click the Install all updates button and want to reverse that for a macOS update, let the download complete, shut down, start up in Safe mode, wait a minute, then restart in normal mode.
These apply to Apple silicon Macs, and are untested in Intel Macs, although there’s no reason to believe they should differ there.
Apple has just released an update to XProtect for all versions of macOS from El Capitan or so, bringing it to version 5272. Apple has now released this for Sequoia betas as well, using their new update mechanism.
Apple doesn’t release information about what security issues this update might add or change. This makes a small amendment in the Yara definitions to the detection signature for MACOS.d98ded3, and adds another rule to those to detect MACOS.DOLITTLE, in MACOS.DOLITTLE.DOFSTRGT.
You can check whether this update has been installed by opening System Information via About This Mac, and selecting the Installations item under Software.
A full listing of security data file versions is given by SilentKnight, LockRattler and SystHist for El Capitan to Sonoma available from their product page. If your Mac hasn’t yet installed this update, you can force it using SilentKnight, LockRattler, or at the command line.
If you want to install this as a named update in SilentKnight, its label is XProtectPlistConfigData_10_15-5272.
If you’re running a Sequoia beta and are still stuck at 5271, don’t worry if sudo xprotect check
doesn’t offer you the update to 5272. Ignore it and run sudo xprotect update
and with any luck it will update your Mac to 5272.
I have updated the reference pages here which are accessed directly from LockRattler 4.2 and later using its Check blog button.
LM Studio 是一款将目前主流大模型 LLM 元素打包在一起的工具,可以让你在自己的电脑上,“0 门槛”运行本地大语言模型 LLM,并且用起来就像 ChatGPT 那样。支持 windows、macos、Linux。
LM Studio is an easy to use desktop app for experimenting with local and open-source Large Language Models (LLMs). The LM Studio cross platform desktop app allows you to download and run any ggml-compatible model from Hugging Face, and provides a simple yet powerful model configuration and inferencing UI.
傻瓜、一站式部署本地大语言模型,大概就是打开电脑 > 双击运行程序 > 开始提问 > 获得 ai 回答这样三步走。
I hope that you enjoyed Saturday’s Mac Riddles, episode 270. Here are my solutions to them.
1: The periodical is secure when charged through this attractive force.
Click for a solution
MagSafe
The periodical (mag or magazine) is secure (safe) when charged through this attractive force (it’s a charging lead attached by magnetism).
2: Quick flash followed by thunder for power adaptor and more.
Click for a solution
Lightning
Quick (lightning) flash followed by thunder (lightning) for power adaptor and more (what it’s for).
3: Might of military planes would have been magic for 8 and X until cancelled because of heat.
Click for a solution
AirPower
Might of military planes (air power) would have been magic for 8 and X (it was a wireless charging system announced with iPhones 8 and X in 2017) until cancelled because of heat (Apple cancelled it in 2019 apparently because it got too hot).
The common factor
Click for a solution
They have been used to charge Macs and Apple devices, although the last never made it to market.
It wasn’t until System 7 in 1991 that the Mac gained any feature that let the user create links to files or folders. Without a command shell, Unix traditions of symbolic and hard links weren’t available. Apple belatedly added what it termed an alias, created using the Make Alias command in the Finder’s File menu. This made a document-like object bearing the name of its original with alias appended, displayed in italics to distinguish it from the original. To help users locate the original file for an alias, the Finder’s Get Info dialog gained a button to Find Original, later moved to the File and contextual menus.
Later versions of classic Mac OS added refinements to this transformative feature, including the selection of a new target for a broken alias, creation by drag-and-drop of a file or folder with the Command and Option keys held, and a distinctive arrow badge to both the item’s icon and the pointer during drag-creation.
Internally, the alias was a small file of less than 5 KB size containing opaque data with more information than just the path to the original. Aliases were designed to support free movement within the same file system, at the time an HFS volume (or partition, as they’re the same in HFS and HFS+).
Aliases transferred across to Mac OS X, where the more adventurous could open Terminal and create both symbolic and hard links instead. It has remained a sign of the lasting schism between the GUI and Unix internals of Mac OS X that there are no standard command tools supporting Finder aliases, and there’s no way to create or maintain symbolic or hard links in the Finder.
Something happened to aliases when they transitioned into Mac OS X, and their size started to rise steadily until they reached 1-5 MB in El Capitan.
Back in Mac OS X 10.4 Tiger they were still fairly small, this one at 48 KB. Note the Select New Original… button in the Get Info dialog.
Aliases were all very good for the user, but OS X needed something similar that could be stored in property lists and other files, so the alias format was rejigged into more generalised form as the bookmark, introduced in OS X 10.6 in 2009. In early 2012 with 10.7.3, bookmarks could be security-scoped with permission on a per-app or per-document basis, to enable their use with the app sandbox. By OS X 10.9 Mavericks in 2013, bookmarks were in widespread use throughout the system.
In El Capitan (2015), bookmark size often reached 1 MB, while the Select New Original… button remained the same in the Get Info dialog.
macOS Sierra brought a major revision to both aliases and bookmarks, reducing their size substantially, but bringing other problems. As late as 10.12.1 they were still having problems resolving some links. The end result was worth the struggle, though, as they have since become more robust than ever, with a typical size of only 1 KB, making them almost as efficient as symbolic links.
Although aliases and bookmarks are still intended to be treated as opaque, their contents have become more accessible. Among the useful values each contains are:
_NSURLPathKey gives the full path to the file,
_NSURLFileIDKey gives the inode number of the file,
_NSURLBookmarkURLStringKey gives the file’s full URL,
NSURLCreationDateKey gives the file’s creation timestamp,
NSURLIsRegularFileKey indicates whether it’s a normal file,
NSURLIsPackageKey indicates whether it’s a package rather than a file,
_NSURLBookmarkSecurityScopeCryptoKeyKey is used if this is a Security-Scoped Bookmark, most used with sandboxed apps.
I even have a couple of utilities that work with them. Among the many features of Precize is the ability to generate bookmarks, and to analyse and resolve them. For those who want to bridge the divide between the GUI and command line, alisma is a tool that can both create aliases and resolve them to paths. Finally, Alifix helps you refresh and identify broken aliases.
Since its first public release seven years ago in High Sierra, APFS has changed greatly. Connect a bootable external disk with Sonoma installed to a Mac running macOS 10.13 and it won’t know what to make of it. That’s because many of the features used by APFS today didn’t exist until Catalina and Big Sur. This article explains how your Mac can cope with running such different versions of APFS, and how to avoid the problems that can arise when a Mac can start up in two or more different versions of macOS.
Fences
Disk structures, APFS and macOS fall into three main phases:
High Sierra and Mojave, which can run 32-bit code, boot from a single integrated volume, and don’t understand Signed System Volumes (SSVs); they run APFS versions 748.x.x to 945.x.x.
Catalina (macOS 10.15), which can’t run any 32-bit code, and boots from a group of volumes where the system is contained in its own read-only volume; it runs APFS version 1412.x.x.
Big Sur and later, which can’t run any 32-bit code, and boot from an SSV that’s a specially sealed snapshot on the System volume; they run APFS version 1677.x.x and later.
Fuller and finer details of the changes with different versions of macOS are given in the Appendix at the end.
If you need your Mac to run more than one version of macOS, it’s easiest and most compatible if the versions installed come from only one of those phases. Two phases are more of a challenge, and all three needs great care to ensure that one version doesn’t damage the file systems of another.
When you do need to run macOS from two or more phases, the cleanest and safest way is to only mount disks and volumes from one phase at a time. For example, when running Mojave, if you unmount disks and volumes for all later versions of macOS, then you won’t see any notifications warning you of Incompatible Disk. This disk uses features that are not supported on this version of macOS. Unfortunately, unless all your boot systems are on external disks, this isn’t easy to achieve.
Warnings
Generally speaking, APFS tools including those run by Disk Utility (including fsck_apfs, used by First Aid) are backward-compatible, but may not be as reliable when a tool from an older version of macOS is run on a newer version.
First Aid and fsck_apfs normally report versions of APFS tools used on the volumes and containers they’re checking. These draw attention to any potential for incompatibilities, such as: The volume [volume name] was formatted by diskmanagementd (1412.141.1) and last modified by apfs_kext (945.250.134).
telling you that volume was created by macOS 10.15, and last changed by macOS 10.14.
Other messages you might see include warnings like: warning: container has been mounted by APFS version 2236.141.1, which is newer than 1412.141.3.7.2, which is less helpful.
If you’re going to run multiple versions of macOS on the same Mac, you’ll have to get used to those. For reference, APFS versions decode to macOS versions as:
249.x.x is the beta release from macOS 10.12 Sierra
748.x.x is 10.13 High Sierra
945.x.x is 10.14 Mojave
1412.x.x is 10.15 Catalina
1677.x.x is 11 Big Sur
1933.x.x is 12 Monterey up to 12.2
1934.x.x is 12.3 Monterey and later
2142.x.x is 13 Ventura
2235.x.x is 14 Sonoma up to 14.3
2236.x.x is 14.4 Sonoma and later
2311.x.x and later is macOS 15 Sequoia beta.
Dangers
To minimise the risk of any problems arising, any manipulation of the file system should be performed by Disk Utility, the diskutil command tool or others, for the same or later version of macOS. Thus, if you have Mojave and Big Sur installed, you could use tools from either to maintain Mojave file systems, but should only use those for Big Sur when maintaining Big Sur’s file systems.
This becomes more difficult when file system maintenance needs to be performed in Recovery mode. Once the Mac has started up in Recovery, check the version of macOS before opening Disk Utility, using fsck_apfs, diskutil or anything similar.
This is one situation where versions of macOS with an SSV can become tricky, because of their different associations with Recovery systems. On Apple silicon Macs, Big Sur doesn’t use a paired Recovery volume, whereas Monterey and later do. To be confident that the Mac starts up in the correct version of Recovery, it should have been running that version normally before you start it up in Recovery. Once in Recovery, check the version of macOS before proceeding to work with its file systems.
Appendix: APFS and macOS version details
APFS major version numbers change with major version of macOS:
macOS 10.12 has APFS version 0.3 or 249.x.x, which shouldn’t be used at all.
10.13 has 748.x.x, which doesn’t support Fusion Drives, but has basic support for volume roles.
10.14 has 945.x.x, the first version to support Fusion Drives.
10.15 has 1412.x.x, the first version to support the multi-volume boot group, and introduces extended support for volume roles, including Data, Backup and Prelogin.
11 has 1677.x.x, the first version to support the SSV, and Apple silicon. On M1 Macs, it doesn’t support the paired Recovery volume, though.
12 has 1933.x.x until 12.2, thereafter 1934.x.x, which support the paired Recovery volume on Apple silicon.
13 has 2142.x.x, and is probably the first to support trimming of UDRW disk images and their storage as sparse files.
14 has 2235.x.x, until 14.3, thereafter 2236.x.x.
Minor version numbers increment according to the minor version of macOS, and patch numbers wander without pattern.
I hope that you enjoyed Saturday’s Mac Riddles, episode 269. Here are my solutions to them.
1: Palmtop with gravity was first with arm for five years.
Click for a solution
Newton
Palmtop (what it was, although Apple coined the term PDA) with gravity (first elaborated as the law of universal gravitation by Sir Isaac Newton) was first with arm (it was Apple’s first device to use an ARM processor) for five years (released in 1993, discontinued 1998).
2: Component framework took aim at OLE for a couple of years.
Click for a solution
OpenDoc
Component framework (what it was) took aim (it was part of the foundation of the AIM Alliance between Apple, IBM and Motorola) at OLE (it was a competitor for Microsoft’s OLE) for a couple of years (released in 1995, discontinued 1997).
3: Blue-sky lab from Larry’s education brought quick things for over a decade.
Click for a solution
Advanced Technology Group (ATG)
Blue-sky lab (what it was) from Larry’s education (it sprung from the late Larry Tesler’s Education Research Group) brought quick things (responsible for Color QuickDraw, QuickTime and its VR and 3D versions, and more) for over a decade (formed in 1986, closed in 1997).
The common factor
Click for a solution
They were all cancelled by Steve Jobs on his return to Apple in 1997.
When a solution works once, we tend to keep using it even though it may no longer be necessary, or may no longer be possible in the same way. This article is about one example, how we used to update macOS using Combo updaters in the days before Big Sur. Now those are no longer available, should we perform a fresh install of macOS whenever there’s an update?
How macOS updates worked
Before we upgraded to Big Sur, macOS updates were relatively simple, and based on installer packages, structured collections of files that are copied to the boot volume in accordance with scripts. Once that copying is complete and the boot volume contains the files forming the new version of macOS, the Mac can be rebooted from those.
Take the last of those major versions of macOS, Catalina. In the diagram below, I show in miniature what happened during the course of its first two minor updates, as an example.
Updating from Mojave to Catalina 10.15.0 was performed using installer packages that completely replaced everything in its system, then created in a new System volume, as shown at the top. This was referred to as an upgrade, as the whole system was installed afresh.
Then (skipping a Supplemental Update) came the first minor update to 10.15.1 on 29 October 2019. For the sake of this example, consider that includes a new version of the cat command tool. Rather than that update containing the whole system all over again, it only contains and installs what has changed, including the new cat binary.
Then, on 10 December 2019, Apple released the update to 10.15.2, containing another set of changed files, this time perhaps replacing the cp command tool. There are two ways to install that update, either as a delta update or a Combo. The delta update contains the changes from 10.15.1, here cp; the Combo update contains everything that has changed since 10.15.0, both that in 10.15.1, here cat, and in 10.15.2 with cp.
Combo updates tackled what was then a common problem, when a previous update hadn’t been installed correctly. If the update to 10.15.1 didn’t actually install the new cat and we then installed the delta update for 10.15.2, the latter wouldn’t have worked properly. As there was limited error checking in those old updaters, we would then be using a flawed installation of macOS. By installing the 10.15.2 Combo updater instead of the delta, the chance of errors like that were significantly reduced, and we were thoroughly impressed by updating using the Combo rather than delta update.
In fact, it was seldom necessary to install the Combo update every time, as most updates went well and errors weren’t as common as it seemed. What I have always recommended has been to install the delta update first, and only to resort to the Combo version if that didn’t work properly.
Big Sur and the SSV
Catalina’s novel boot volume group, with its separate System and Data volumes, was only an intermediate step to Big Sur, which brought the greatest structural change in Mac operating systems since the introduction of Mac OS X twenty years earlier. Here, the great majority of macOS runs from a mounted snapshot with a read-only file system separate from that of user files on the paired Data volume, and requires a fundamentally different method of installation and updating.
In addition to updating the contents of the system, from Big Sur onwards installers and updaters have considerably more to do. Those tasks include creating the new snapshot to contain the bootable copy of the system, firmlinking that System snapshot with its paired Data volume, building a tree of cryptographic hashes so that the snapshot can be sealed, and verifying the signature of its seal against Apple’s requirement.
If we follow Big Sur’s first couple of updates, you’ll see how they have changed. On 14 December 2020, the update to 11.1.0 might have gone through similar changes to Catalina, with the replacement of its cat command tool. Once that had been installed on the System volume, a snapshot was made of that System volume, hashes were computed for all its contents, and assembled into a tree. At the top of the tree, a hash of all the hashes in the next layer down forms its seal, which is then hashed again to form the signature of the Signed System Volume (SSV). The signature is compared against that set by Apple for the whole volume, and only if they’re identical is the System snapshot accepted for use.
There’s no room here for the slightest error if the signature on the Signed System Volume (SSV) is going to match its requirement. Within the SSV are additional changes, such as large dyld caches, which serve to make this more complicated, and more recently this includes cryptexes containing system components like Safari that are kept outside the SSV so they can be updated outside full macOS updates.
When that was updated again to 11.2.0 on 1 February 2021, perhaps with a new version of the cp tool, the new installer repeats this process of creating a new snapshot, building its tree of hashes, sealing them, creating the signature, and comparing that with Apple’s signature. This means that each and every update is verified to be perfect, and no errors can be tolerated.
No Combo updates
In the new system, from Big Sur onwards, Apple only provides Combo updates for those updating from versions of macOS older than the previous version, and only when required. If you were to update straight from 11.0 to 11.2, then your Mac downloads all that it requires for that update. If your Mac is already running 11.1.0, there’s no option to install the changes in 11.1.0 again, nor could you need to, as its installation on 11.1.0 has already been verified as being identical to Apple’s reference.
If you felt that wasn’t adequate, then the only option is to install the whole of 11.2.0 from scratch, and running the Big Sur Installer app will do that for you. However, there’s no point in doing that, as the outcome will be identical, as verified by its signature match against Apple’s. All that would do is take longer, and increase wear on your Mac’s internal SSD.
That’s why Apple doesn’t provide the user with Combo updates any more, because they’re now superfluous.
When Apple introduced the Signed System Volume (SSV) in Big Sur, the size of macOS updates rose to their highest ever, as the overhead required for each update reaching 2.2 GB for Intel models and 3.1 GB for Apple silicon. Since then, they have steadily improved in efficiency. This article shows how they have performed in Sonoma, and asks whether Apple has abandoned its Rapid Security Responses (RSRs).
To keep my charts clearer, I here show the two architectures separately.
Total update size
This chart shows cumulative sizes of updates to macOS on Intel Macs from 10.14 Mojave, with its traditional single boot volume, through to macOS Sonoma 14.6. Each point represents the cumulative sum of all updates to that major version of macOS required to reach that minor version. Thus the point for 14.2 is the total of update sizes for 14.1, 14.1.1, 14.1.2 and 14.2. Sizes used aren’t those reported by Software Update, but those of the download itself, as reported in articles here, indexed on this page. That has made a significant difference for some updates for Apple silicon Macs, where the first reported update size has excluded additional architecture-specific overhead. RSRs have been excluded, as they’ve been duplicated in subsequent patch or minor updates. Lines shown are best fits by linear regression.
Update sizes rose markedly from Mojave, with its single boot volume, to Catalina, with its boot volume group, and again to a peak in Big Sur, with the SSV. They fell again as Monterey introduced greater efficiency, and Ventura and Sonoma have been almost identical, and smaller than Mojave.
Apple silicon Macs started with the huge updates of Big Sur, which were even larger than those for Intel models, and benefitted from the improved efficiency of Monterey and Ventura. Unlike Intel Macs, though, Sonoma has seen further reduction in update sizes, although in each update they remain significantly larger than those for Intel models.
Big Sur took a total of 36.7 GB on Intel, and a remarkable 50.07 GB on M1; Sonoma has taken 14.1 GB on Intel, and 21.2 on M1. On Intel, Sonoma required 38% of the update size as Big Sur; on M1, that proportion was slightly higher at 42%. Apple silicon updates were 136% the size of those for Intel models in Big Sur; in Sonoma that ratio has risen to 150%.
Minimum update sizes were about 400 MB for Intel, and 820 MB for Apple silicon, which must be close to the overhead required for macOS updates for the two architectures. These reflect the size of the ‘Update Brain’ required to install each update, and all bundled firmware updates. The latter have been steadily reducing as Intel updates have supported fewer models without T2 chips, but have remained the same if not grown for the substantially larger firmware updates for Apple silicon’s Secure Boot.
Unscheduled updates
Sonoma reached version 14.6 without an excessive number of patch updates. The number of patch versions released between the x.0 and x.6 scheduled versions ranges from 4 (Monterey) to 6 (Bug Sur, Ventura), with Sonoma taking just 5, although 14.6 has been closely followed by 14.6.1 in what appears to be a bug fix rather than the first security update.
What has been surprising is that Apple has released no RSRs for Sonoma, not one. The last RSR was the second released for 13.4.1 more than a year ago, on 12 July 2023. RSRs were introduced to Ventura in May 2023 as a method of replacing some components of macOS that aren’t installed in the SSV but in separate cryptexes, cryptographically verified disk images stored on the hidden Preboot volume. They were intended to let Apple promulgate important fixes to vulnerabilities in Safari, WebKit, and some other bundled components without waiting for a full macOS update. Unfortunately, Apple’s second RSR in July last year was fumbled badly when it broke Safari access to many popular websites including Facebook, and had to be replaced three days later.
Although it’s never clear when a patch update could have been issued as an RSR, Sonoma 14.1.2 should have been a candidate as it’s only reported to have fixed two vulnerabilities in WebKit, that appeared to have been addressed in Safari-only updates for macOS 12 and 13. It looks now as if Apple’s initial enthusiasm for RSRs has cooled, and they’re unlikely to be significant in the future.
Standalone updaters
Updates to macOS Catalina and earlier were also available in downloadable standalone installer packages. Apple discontinued those, amid much protest from users, when it introduced Big Sur. Although there were some suggestions that they might return in the future, as the new update mechanism matured, there has still been no sign of them. Currently the only form of updater available is the full macOS Installer app, typically over 13 GB in size. This remains a shortcoming compared with Catalina and earlier, but it appears that standalone updaters aren’t missed by most Mac users, or at least that Feedback to Apple has been insufficient to produce a response.
Major updates
Prior to macOS Ventura, upgrading to the first release of the next major version of macOS required downloading its full installer app. Although larger than necessary to perform the upgrade, it was relatively error-proof, as the user had to intentionally start the app’s installation process, which could also be cancelled once it was in progress.
Ventura was the first major version for which Software Update used a mechanism similar to that for minor updates, and didn’t download a discrete macOS installer app. As Apple hadn’t warned of that, it caught some users by surprise, when they found their Mac was being upgraded almost automatically. This was repeated with the release of Sonoma, and the problem compounded when some users appear to have been upgraded without their consent.
As Tom Bridge has pointed out, this requires smaller downloads that install considerably faster, but, on Intel Macs at least, don’t require user consent or authentication.
Apple is likely to employ the same update mechanism when it releases Sequoia and takes us on to the next cycle.
I hope that you enjoyed Saturday’s Mac Riddles, episode 267. Here are my solutions to them.
1: Where the store requires apps and kids to play in safety and grip the rails.
Click for a solution
sandbox
Where the store requires apps (it’s a requirement of the App Store) and kids to play in safety (a sandbox) and grip the rails (also used by railway locos to apply to slippery rails to improve grip).
2: Benefit of a rightful claim to go beyond the confines of 1.
Click for a solution
entitlement
Benefit of a rightful claim (an entitlement) to go beyond the confines of 1 (what an app entitlement does for a sandboxed app).
3: ISO 688 could be intermodal folder for 1, but only in ~/Library.
Click for a solution
container
ISO 688 could be intermodal (a shipping container) folder for 1 (what it is in macOS), but only in ~/Library (the only place you should find them).
The common factor
Click for a solution
They are parts of the app sandbox system in macOS.
The kernel at the heart of macOS is XNU, standing for X is Not Unix, as, however much macOS might have in common with Unix, once you scratch its surface it’s all very different. Several of you commented recently that what I referred to as a user’s UniqueID should have been termed User ID as it would be in Unix. But dig deeper and you’ll see that macOS does indeed refer to that internally as the UniqueID, because that’s how it’s named for Directory Services in macOS.
Older versions of macOS used to provide advanced options for users in Users & Groups settings, where you could even change a user’s UUID. Then a bright spark thought it would be fun to tell users to change that in a misguided bid to improve their privacy. When someone who seems to know what they’re writing about tells you to do that, you go and try it, don’t you? Only changing a user’s UUID is catastrophic, and most who did so ended up having to rebuild the contents of their Mac from scratch to put the damage right. The lesson is that you should never try anything you don’t understand, for which there’s no simple undo, and which might have serious side-effects. Please don’t do that: it’s such a crazy idea that it’s malicious in intent. As a result, Apple has progressively removed those dangerous advanced user settings, to keep them out of harm’s way.
But there are times when you do need to make such changes, or at least check what current settings are. For those you’ll need Directory Services’ editor, Directory Utility, which is carefully hidden in /System/Library/CoreServices/Applications. Whatever you do, please don’t authenticate to Directory Utility, to ensure that you can’t inadvertently change anything that could give you grief.
Modern Directory Services have a long history that I’ll explain on Saturday morning, but they’re basically descended from NeXTSTEP’s NetInfo and the Lightweight Directory Access Protocol (LDAP) implemented in Apple’s successor Open Directory. These flourished in the days of Mac OS X Server, but since then have carried on quietly serving macOS with information about users, groups, and a whole lot more.
Directory Utility may be hidden away, but it’s still well documented, and its Help book is worth browsing. Unless your Mac is connected to a server delivering it LDAPv3, Active Directory or similar services, once you’ve selected the Directory Editor at the top, view Users in the node /Local/Default, as your Mac’s local directory. There you’ll find your personal details, including your UUID as GeneratedUID, your Home folder as NFSHomeDirectory, long user name as RealName, short name as RecordName, and user ID as UniqueID.
Listed as users are many of the services your Mac connects to on your behalf, including the App Store, Apple Pay, Find My and more. Each of these has its own UniqueID, PrimaryGroupID, and more. There may also be hidden users that you thought had been removed, and shared folders from years ago. Don’t give way to any temptation to try ‘cleaning’ this up, as it’s all too easy to wreak havoc unintentionally.
In the past, Directory Utility was commonly used to enable the root user and change its password, features that may still available today in its Edit menu once you’ve authenticated. At one time this was useful, but shouldn’t be used any more unless you’re advised to by Apple.
Directory Utility is now mainly used when integrating a Mac with a network directory server, including Open Directory (formerly in Mac OS X Server), LDAP on a Linux or other server, and Microsoft’s Active Directory. For the advanced user it gathers important information in one place, and every once in a blue moon it can unscramble a Mac or user account that would otherwise require starting from scratch. But please don’t authenticate and start making changes, as you’ll most likely regret it.
I hope that you enjoyed Saturday’s Mac Riddles, episode 266. Here are my solutions to them.
1: Skipper or proprietor who might also live there as one of the users.
Click for a solution
owner
Skipper (sometimes referred to as the ‘owner’ of a ship or aircraft) or proprietor (owner) who might also live there (owner-occupier) as one of the users (what the owner is).
2: Cluster or band such as staff or wheel.
Click for a solution
group
Cluster (a group) or band (a musical group) such as staff or wheel (two common groups in permissions).
3: The rest may be half or significant to everyone else.
Click for a solution
others
The rest (others) may be half (the other half) or significant (significant other) to everyone else (as in permissions).
The common factor
Click for a solution
They are the three classes used to set permissions.
Last Tuesday, as the uproar over the CrowdStrike catastrophe was still subsiding, and alongside the fourth developer beta of Sequoia, Apple quietly provided its first release candidate for macOS 14.6. Don’t be surprised if that ships early next week, alongside Ventura 13.6.8 and Monterey 12.7.6, making it the earliest release of the sixth minor version in the annual macOS cycle since Mojave. For the last four years, from Catalina to Ventura, the last full update marking the start of two years of security-only maintenance has taken place in September.
To put this into context, it’s worth revisiting how Apple numbers macOS versions and how that fits into its annual release cycle:
The first digits in the version number indicate the major version, currently 14 for Sonoma, which changes around September-October of each year with the start of each new cycle.
The second digit indicates the minor version, currently 5 for Sonoma, and starts from 0 with the first major release in September. Minor updates are scheduled well in advance, and culminate in the sixth as the last regular update (Catalina was an exception in running to 7), and the start of that version’s first year of security-only updates.
The third digit indicates the patch version, used for urgent unscheduled fixes between those minor versions. The previous release of Sonoma before 14.5 was 14.4.1, which fixed a few urgent bugs and security vulnerabilities. For security-only updates after the x.6 release, this marks each security update.
Thus the first year of Sonoma is expected to run from its first release as 14.0 on 26 September last year, through 14.1 to 14.6, its last full update, shortly before the release of Sequoia. Sonoma then enters its first year of security-only updates in 14.6.1, and its second year in the autumn/fall of 2025 with versions 14.7, 14.7.1, and so on, until it becomes unsupported after a total of three years.
Sonoma has been running early throughout its release cycle, starting with its first release, and squeezed in 14.2 well before Christmas. While this could indicate that Apple doesn’t intend putting it into security-only maintenance until after version 14.6.1 or even 14.7 in September, around the time of Sequoia’s initial release, that would mark a significant change in the annual cycle. You also have to wonder how many non-security fixes could be prepared for release during August, when most of Apple’s software engineers are fully extended in finishing off the major new versions of macOS, iOS, iPadOS, watchOS, tvOS and visionOS for release the following month.
I expect next week will bring the release of Sonoma 14.6, the last general update and start of its two years in security-only maintenance, together with Ventura 13.6.8 as the end of its first year of security-only fixes, and Monterey 12.7.6, its swan song as it becomes unsupported. Those should pave the way for Sequoia 15.0 in September, bringing support for a batch of new Macs featuring M3 and M4 chips, to include an updated Mac Studio, Mac mini, and MacBook Pros.
If your Mac is still running Ventura or Monterey and is supported by a more recent version of macOS, now is the time to make a decision about whether and when to upgrade. Even if Monterey does get one more final security update, it’s almost certain to fall well short of that provided for Ventura and Sonoma. If you were intending to upgrade to Sonoma, then it’s not likely to have any further general fixes after this forthcoming update, but only to receive security updates from August onwards.