This article describes one change that has caught out some using macOS Sequoia, and considers what has changed in Sequoia Virtual Machines (VMs).
periodic has been removed
After many years of deprecation, the periodic scheduled maintenance command tool has been removed from macOS 15.0. In its heyday, periodic was responsible for running daily, weekly and monthly maintenance and housekeeping schedules including rolling the system logs. Over that time, macOS has been given other means for achieving similar ends. For example, logs are now maintained constantly by the logd service, and aren’t retained by age, but to keep the total size of log files fairly constant. I don’t think that Sonoma performed any routine maintenance using periodic.
If you use periodic, then the best option is to use launchd with a LaunchAgent or LaunchDaemon. If you’d prefer to use cron, that’s still available but is disabled in macOS standard configuration.
Sequoia VMs: AI
Sequoia VMs created from an IPSW image of Sequoia (rather than upgraded from Sonoma or earlier) running on Sequoia hosts are the first to gain access to iCloud features. Now that 15.1 has been released with AI, I’ve been trying to discover whether that can also be used in a VM. So far, my 15.1 VM has sat for hours ‘preparing’, but AI still hasn’t activated on it. I suspect that, for the present, AI isn’t available to VMs. If you have had success, please let me know.
Sequoia VMs: macOS builds
My test 15.1 VM has also behaved strangely. It was originally created in 15.0, updated successfully to 15.0.1, then to 15.1, where it was running build 24B83, the version released generally on 28 October. Later that week Software Update reported that a macOS update was available, and that turned out to be a full install of 15.1 build 24B2083, released on 30 October for the new M4 Macs. This VM is hosted on a Studio M1 Max!
Installation completed normally, and that VM now seems to be running the new build perfectly happily, although it hasn’t proved any help in activating AI.
Don’t be surprised if your 15.1 build 24B83 VMs behave similarly. If anyone can suggest why that occurred I’d be interested to know, as it’s generally believed that build 24B2083 has been forked to support only M4 models.
Apple has just released an update to XProtect for all supported versions of macOS, bringing it to version 5279. As usual, Apple doesn’t release information about what security issues this update might add or change.
Relative to the last version released for all supported versions of macOS (5278), this version makes a small amendment to the detection rule for MACOS.PIRRIT.CHU.
You can check whether this update has been installed by opening System Information via About This Mac, and selecting the Installations item under Software.
A full listing of security data file versions is given by SilentKnight, LockRattler and SystHist for El Capitan to Sequoia available from their product page. If your Mac hasn’t yet installed this update, you can force it using SilentKnight, LockRattler, or at the command line.
If you want to install this as a named update in SilentKnight, its label is XProtectPlistConfigData_10_15-5279.
For Sequoia only: there’s no sign of this update being made available in iCloud, which now returns an XProtect version of 5278. If you download and install it using Software Update, softwareupdate or SilentKnight, then once that’s complete you need to update the primary XProtect bundle in Terminal using the command sudo xprotect update
then entering your admin password. If you’re unsure what to do, this article explains it comprehensively and simply.
I have updated the reference pages here which are accessed directly from LockRattler 4.2 and later using its Check blog button.
If you encounter problems with QuickLook not creating Thumbnails or Previews properly, one of the first steps is to discover which code is responsible for generating those for QuickLook. Prior to macOS Sequoia, the standard way to do that was using the command tool qlmanage, among whose options is -m, to list all the qlgenerators available on your Mac. If you’ve tried that in Sequoia, you’ll surely have noticed that no longer works.
qlmanage
Since Catalina, Apple has been encouraging developers to switch away from qlgenerators to app extensions to create custom Thumbnails and Previews for QuickLook, and Sequoia is the first version of macOS that can’t use third-party qlgenerators. I have noticed some document types that only a few weeks ago in Sonoma still used custom thumbnails and full previews, but now can’t do so, although others continue to work normally.
These are controlled in the Quick Look item in Login Items & Extensions in General settings.
That should list all third-party app extensions providing this service, and enabling the right one(s) could fix some of those problems. But it turns out this list isn’t complete, and doesn’t in any case tell you which app extension handles which file type. For those, you’d normally turn to qlmanage, but its -m option can only see the qlgenerators in macOS, and no third-party app extensions at all. In fact, qlmanage is now of little help for anything related to QuickLook. I’ve gone back through Sonoma and Ventura, and qlmanage there is no different: although it does list third-party qlgenerators, none of those provided in app extensions appear in its list.
QuickLook app extensions
As far as I can discover, Apple doesn’t provide any equivalent of qlmanage that can report on QuickLook app extensions. The closest it comes is in the pluginkit tool, that can list all app extensions known to macOS. With a bit of tweaking, its -m option can reveal which of those use the QuickLook SDKs for Thumbnails or Previews.
Armed with the appex bundle path from pluginkit, you can then inspect the Info.plist in each, where there’s an array of QLSupportedContentTypes giving the UTIs of all file types supported by that appex. Although I’m sure someone could implement that in a shell script, this seemed an ideal task for my free utility Mints.
Mints and QuickLook
Version 1.20 of my free utility Mints is now available from here: mints120
from Downloads above, from its Product Page, and via its auto-update mechanism.
This adds a twenty-fifth button to the app’s control window, named QuickLook, at the bottom left. Click on that and Mints will open a new window and fill it with information about all the qlgenerators and QuickLook appexes your Mac knows about.
For qlgenerators, you’re given the file UTI, the path to the qlgenerator file, and (when available) its version number, e.g. com.adobe.pdf/System/Library/QuickLook/PDF.qlgenerator (1002.2.3)
App extensions are divided into two, the first are those providing Previews, and the second those for Thumbnails, e.g. com.apple.applescript.text/Applications/PreviewCode.app/Contents/PlugIns/Code Previewer.appex
This is an appex provided in one of Black Pyramid Software’s superb Preview series, in PreviewBundle 2 from the App Store (highly recommended).
You will see a few entries like Safari’s [none]/System/Volumes/Preboot/Cryptexes/App/System/Applications/Safari.app/Contents/PlugIns/SafariQuickLookPreview.appex
with an appex that doesn’t have a list of file types in QLSupportedContentTypes.
Checking UTIs
It’s easy to guess which UTIs represent many file types, but some are a bit more cryptic. For those, copy and paste the UTI into the UTI field of my free UTIutility and it will give you clues as to its identity, including file extensions.
Unfortunately, some of the system qlgenerators support generic UTIs such as public.audio/System/Library/QuickLook/Audio.qlgenerator (1002.2.3) public.image/System/Library/QuickLook/Image.qlgenerator (1002.2.3) public.movie/System/Library/QuickLook/Movie.qlgenerator (1002.2.3)
which clearly cover broad ranges of more specific file types, but don’t provide any more specific information.
How to identify QuickLook extensions
List installed QuickLook extensions using Mints’ QuickLook button.
Identify the file’s UTI using UTIutility.
Locate the UTI in the list of extensions.
If no match is found, check UTIs listed in UTIutility as Conforms.
Check Quick Look item in Login Items & Extensions in General settings, to ensure that extension is enabled.
Next up for Mints is a feature to explore app extensions. I may be a little longer on that one.
QuickLook is the subsystem in macOS responsible for providing two types of document preview, small Thumbnails and full Previews. If you’ve already upgraded to Sequoia, you’ll have noticed that some document types are no longer displayed with their custom Thumbnails or Previews. This article explains what has happened, and how it should work in the future.
As I’ll detail on Saturday morning, QuickLook (or Quick Look) is the latest in a series of methods for providing custom icons and previews for documents, that started back in the initial versions of Classic Mac OS. macOS ships with its own code to generate Thumbnails and Previews for a wide range of standard file types, from text and PDF to audio and movies. To extend these to other types, developers are encouraged to provide their own code.
Prior to macoS 10.15 Catalina in 2019, the display of Thumbnails was supported by the QuickLook framework. From Catalina onwards, this is provided by a new framework named QuickLook Thumbnailing. The older framework is documented here, and had been deprecated for some years. Its replacement is documented here. To extend these, the older framework used QuickLook generators with the extension .qlgenerator, but in the newer framework this function is provided by QuickLook preview extensions, in particular Thumbnail Extensions, that were explained to developers at WWDC in 2019.
As with most deprecated features, eventually the time comes for Apple to remove support for the old, and for QuickLook generators that has occurred in macOS 15.0 Sequoia. From now on, QuickLook Generator plugins no longer work. Oddly, those provided by macOS in /System/Library/QuickLook are still named with the old extension of .qlgenerator, but all custom support now has to use the new framework in App Extensions.
To check whether an app is still trying to use an old QuickLook Generator, look inside the app bundle in Contents/Library/QuickLook. If you see one or more .qlgenerator bundles there, then those no longer work in Sequoia. Instead, you should see new Thumbnail Extensions in Contents/PlugIns, where you should see App Extension bundles with names ending in something like Thumbnail.appex and QuickLook.appex. Some of the better apps provide both QuickLook Generators for compatibility with Mojave and earlier, and App Extensions for more recent macOS.
If the app you rely on to generate custom QuickLook Thumbnails and Previews doesn’t yet come with those App Extensions, contact their Support and ask them when they’re going to implement the changes brought five years ago in Catalina. Particularly if you’re paying them a subscription, it’s time they caught up. Until they do, I’m afraid those Thumbnails and Previews simply won’t work in Sequoia, and you’ll continue to see generic icons rather than Thumbnails.
Apple has made great play over the privacy provided in its new AI tools. If you’ve just updated your Apple silicon Mac to Sequoia 15.1 and are wondering how you can check on this for Writing Tools, this article explains how.
When running on a capable Mac, with an M-series chip, macOS captures details of all AI use in its Apple Intelligence Report (AIR). Control and access that from its new entry in Privacy & Security settings, where you’ll find it towards the end, just above the final Security section. Open that, and you’ll see you can set the Report Duration to 15 minutes, 7 days, or turn it off altogether. As report sizes can grow quickly with a little use of Writing Tools, I suggest you start off with 15 minutes, or you might get overwhelmed.
When you want to browse a report, simply click on the button to Export Activity, and save the AIR report.
Apple Intelligence Reports are written out to JSON files that can be viewed using a text editor if you don’t have a specialist JSON editor. They’re usually bulky, and much of their content may be encoded binary that’s of little meaningful use. However, at the start you’ll see a series of modelRequests.
Each modelRequest begins with the timestamp of the request, given in decimal seconds since 1970. That’s followed by a UUID, information on the prompt template used, and shortly after that is the text that was extracted and used by Writing Tools. For longer passages of text, you may see that it’s divided up into a series of shorter sections that match the paragraphs given in a summary.
After that input text, the language localisation is given, currently en_US as other variants and languages won’t be available until macOS 15.2 later this year. Next, the response is provided, as inserted into the Writing Tools or text window. That section ends with:
model, the name of the AI model used, such as com.apple.fm.language.instruct_server_v1.text_summarizer, and the version.
clientIdentifier, such as com.apple.WritingTools.xpc.WritingToolsViewService for normal use of Writing Tools in an app.
executionEnvironment, currently expected to be PrivateCloudCompute, which tells you where the AI processing took place.
After the list of modelRequests, you’ll probably see a long series of privateCloudComputeRequests full of incomprehensible data for sepAttestations and provisioningCertificateChains, part of the validation information for use of PrivateCloudCompute. If this all seems a little long-winded, try looking in the logs when Writing Tools are in use!
I’m very grateful to Tim, who has drawn my attention to these reports, and points out that use of PrivateCloudCompute appears confined to macOS at the moment. A similar report is also available for iOS 18.1, but iPhones don’t appear to rely on PrivateCloudCompute in the same way.
We must remember that, while Apple considers Writing Tools now ready for general use, it’s still officially a beta-release, and over the coming months is likely to undergo significant change. This poses the question of whether Writing Tools will run on-device in the future, something only Apple can answer. What appears to happen at present is that the only local processing that takes place is tokenisation of text to prepare it for remote processing using Apple’s PrivateCloudCompute service, which actually performs the heavy lifting before returning its results to the Mac. However, macOS also appears to wake up the slumbering Neural Engine (ANE) for most Writing Tools services. Why that happens remains a mystery.
If you want to watch progress as AI features develop in macOS, you may find Apple Intelligence Reports a useful way to track that. If you do come across entries that seem to have used on-device services instead of PrivateCloudCompute, please let us know.
The macOS 15.1 update is the first scheduled update since Sequoia’s release last month, and brings with it a great many fixes as expected. From user reports, it’s believed to complete correcting problems reported with networking in 15.0, some of which were addressed in 15.0.1, although Apple hasn’t confirmed that.
Apple’s release notes are helpfully more detailed than usual, and include the following:
new Writing Tools, but only for Apple silicon Macs set to US English as their primary language, with Siri also set to US English,
a new-look Siri with Type to Siri for those who don’t want to talk to it, richer language understanding and context, and Apple product knowledge,
Photos can find by description, and now features Clean Up to remove unwanted parts,
Notifications has summaries, and a new Reduce Interruptions focus,
Mail and Messages have Smart Reply for suggested responses,
Notes has transcription summaries,
iPhone Mirroring now supports drag and drop.
To clarify the requirement to get Writing Tools and other AI to work, the Mac must have an Apple silicon chip (M1 to M4), and:
in System Settings, General, Language & Region, the Primary language must be set to English (US), although any other language can be set secondarily, and made the current language in the keyboard menu, and
in Apple Intelligence & Siri, the Language set for Siri Requests must be English (United States), although you can turn Listen forOff if you don’t want to converse with Siri vocally.
Once those are set, you should be able to turn Apple Intelligence on. There will then be a short period on the waiting list, for macOS to download the additional models required. You’ll be notified when it’s ready to use.
Security release notes are available here, and list 50 entries, none of which Apple suspects may already have been exploited.
iBoot firmware on Apple silicon Macs is updated to version 11881.41.5, and T2 firmware to 2069.40.2.0.0 (iBridge: 22.16.11072.0.0,0). The macOS build number is 24B83, with kernel version 24.1.0. There are no firmware updates for Intel Macs without T2 chips.
Significant changes seen in bundled apps include:
Books, to version 7.1
Freeform, to version 3.1
iPhone Mirroring, to version 1.1
Mail and Messages, large build increments
Music, to version 1.5.1
News, to version 10.1
Passwords, to version 1.1
Photos, large build increment
Reminders, large build increment
Safari, to version 18.1 (20619.2.8.11.10)
Shortcuts, large build increment
TV, to version 1.5.1
Tips, to version 15.1.
Inevitably, there are many build increments in components related to Apple Intelligence. Other significant changes to /System/Library include:
Audio/Plug-Ins/HAL MacAudio driver, to version 510.2
CoreServices Desk View app, to version 2.0
CoreServices Siri app, to version 3401.24.3.14.7
Significant changes across many AGX and AppleEmbeddedAudio kernel extensions
A new AppleT8140 kernel extension
APFS is updated to version 2313.41.1
Many public frameworks have updated build numbers, among them FileProvider
A new ImagePlayground public framework, which has moved from being private, in anticipation of the new app coming in macOS 15.2
Many private frameworks have substantial increments in build numbers, particularly Biome, Cloud, Email, Mail, Photo, Photos, Spotlight and FileProvider
A new DesignLibrary private framework.
Although this isn’t a particularly large update, it does come with the first wave of AI features, and a wide range of other improvements and bug fixes.
Updated 2030 GMT 1 November 2024 with info on non-T2 Intel firmware updates.
As expected, Apple has released the update to macOS 15.1 Sequoia, together with security updates to bring Sonoma to version 14.7.1, and Ventura to 13.7.1. There should also be Safari updates to accompany the latter two.
The Sequoia update is around 2.9 GB for Apple silicon Macs, and about 2.4 GB for Intel models.
As expected, this brings the first release of Writing Tools, in the first wave of new AI features, only for Apple silicon Macs using US English as both their primary language, and that set for Siri. Apple hasn’t got round to providing any list of new or changed features, and you may find that offered by Software Update is the same as for 15.0.
iBoot firmware on Apple silicon Macs is updated to version 11881.41.5, T2 firmware to 2069.40.2.0.0 (iBridge: 22.16.11072.0.0,0), and Safari to 18.1 (20619.2.8.11.10).
I will post details of changes found later tonight.
Early this coming week those fortunate enough to have an Apple silicon Mac will be able to update Sequoia to version 15.1 and try out Writing Tools at last. I’ve been testing and using them since the first beta of 15.1 back when the daylight lasted well into the night. I’ve been surprised at how lukewarm some have been at their arrival, particularly as most of those comments have come from those who make a living from their writing.
Perhaps the best way to show what Writing Tools can do is with an example, using the text from yesterday’s article A brief history of Mac firmware, a little over a thousand fairly technical words. I concentrate here on Writing Tools’ summary modes rather than its rewriting features, although they’re also thoroughly sound and useful in the right context.
Summary
Producing a succinct single paragraph summary is a challenge for even the most literate of humans. I’m impressed with Writing Tools’ 62 word offering of: Mac firmware has undergone significant changes, from the Macintosh ROM in Classic Macs to Open Firmware in PowerPC models and EFI in Intel Macs. EFI, later evolving to UEFI, provides hardware access and boot services, while Apple silicon Macs employ a verified chain of trust for enhanced security. Firmware updates are now integrated into macOS installers and updaters, ensuring security and accessibility.
Key points
With so much information in the original, a key point summary should provide a better reflection of my original thousand words. Rather than reproducing the full 250 words generated by Writing Tools, here’s an excerpt covering one of the more challenging sections:
Firmware Security Issue: Researchers demonstrated proof-of-concept attacks on Mac firmware, highlighting potential vulnerabilities.
Firmware Update Policy Change: Apple transitioned from separate firmware updates to delivering them through system upgrades and updates.
Firmware Security Concern: In 2017, Duo Labs researchers discovered many Macs running outdated firmware, raising security concerns.
Apple’s Response: Apple hired security researchers and developed eficheck, a tool to check and report firmware versions to Apple.
I don’t think I could better that.
List
The third type of summary suitable for that article is a detailed list. This strips everything else out to leave terse factual sentences. For the whole text, it reduces to 833 words, a reflection on its already dense information. You’re unlikely to want to use this with longer passages, but here’s the same section given as key points above:
In March 2015, two security researchers from LegbaCore demonstrated proof-of-concept attacks on the BIOS of several computers.
The attacks could have been used to implant malicious code.
Later that year, Kovah and Trammell Hudson turned their attention to Macs.
They demonstrated a firmware worm named Thunderstrike 2.
For the first nine years of Intel Macs, Apple had provided EFI firmware updates separately from updates to OS X.
In 2015, Apple changed the way that it supplied firmware, delivering it only as part of system upgrades and updates.
Although older separate firmware updates are still available, those were the last.
In 2017, Rich Smith and Pepijn Bruienne of Duo Labs discovered that many Macs were running outdated firmware.
Their concern was about the security risk posed by outdated firmware.
Apple had already been busy hiring Xeno Kovah and Corey Kallenberg who started work there in November 2015, and Nikolaj Schlej, another firmware security researcher, who joined them the following August.
They developed a new tool eficheck, released in High Sierra on 25 September 2017.
eficheck checked current firmware against a local database of versions known to be ‘good’, and with the user’s permission sent a report to Apple in the event that it found discrepancies.
Table
The fourth summary option is to generate a table. Unfortunately, my example wouldn’t produce a useful table without substantial additional knowledge. However, I’ve found this useful on long passages from fiction, where it can summarise relationships between different characters, and similar tasks.
On device and on target
Once Sequoia 15.1 has been released and I’ve had a chance to explore the internals of Writing Tools further, I’ll look at its processing and energy costs. Two important features distinguish it from other contemporary AI tools: all data remains on-device throughout, and it’s primarily using your text rather than a large language model built from vast quantities of text garnered from around the internet.
Privacy doesn’t generally worry me particularly, as much of what I write on Macs is destined in some way or another to be published, whether it’s in an article here, one in the magazines that I write for, or source code that will be built into apps. However, I do take exception to others making money out of my labours without my express consent, so I’ll generally be only too happy to keep my AI on-device.
I also think it’s important to draw a clear distinction between what Writing Tools offers, and the likes of ChatGPT. Now that I’m testing Sequoia 15.2 beta, I have been looking at that contrast. While you can’t ask Writing Tools questions (why would you want to when you have the whole text and its summaries?), I thought I’d see how ChatGPT answered one of my stock test questions for AI: what is the SSV?
At my first asking, ChatGPT didn’t have sufficient context, and told me that it’s a side-by-side vehicle, so I refined my question to what is the SSV in macOS?
Although much of its answer was correct and informative, the second sentence stated with complete confidence that the SSV was introduced in macOS Catalina, which is of course completely incorrect, as Catalina has a read-only System volume but not a Signed System Volume as was introduced in Big Sur. But you’d only spot that serious factual error if you already knew the answer.
Give me Writing Tools and my own fact-checking, thank you.
Apple provided developers with two Release Candidates of macOS Sequoia 15.1 this week. Provided there are no serious problems that come to light in the second of those, it’s likely that 15.1 will be released early next week, probably on Monday 28th. This article looks at what that brings, whether it’s safe to upgrade to Sequoia yet, and what comes next.
All supported Macs
Traditionally, the x.1 update is scheduled to be released about a month after the initial upgrade to a new major version of macOS, and brings with it the first wave of bug fixes, and a few features that weren’t quite ready in time.
Although there are reports of some other bugs in Sequoia, by far the most disruptive have been those affecting networking. Apple fixed the most serious of those in 15.0.1, released on 4 October, but some have continued to experience problems. Opinion from those testing betas of 15.1 are that it does resolve all those, and for the great majority should be ready for general use, provided that third-party apps are compatible. So if you normally wait for the x.1 version to be released before considering upgrading, this should fit the bill.
Apple does provide a list of fixes for developers, although as there’s no mention of any networking problems there, I suspect this isn’t of much help to users.
Apple silicon Macs
For those whose Macs run an M-series chip, the main interest in 15.1 is the first batch of Apple Intelligence features. Over the coming months, these should include:
Writing Tools, a suite of mainly on-device features for summarising and rewriting text.
Image Playground, producing synthetic images such as Genmoji, again using on-device methods.
Siri and related enhancements for user assistance, using on-device methods.
ChatGPT access, for more general AI features using text.
App-specific enhancement to Photos, including Clean Up, and others.
Of those, 15.1 brings Writing Tools and some other enhancements, but doesn’t bring Image Playground or ChatGPT. Although some have claimed that makes 15.1 little better, that understates the value and quality of Writing Tools for many.
Writing Tools should be accessible to pretty well any recent app that displays significant amounts of text. Although I haven’t intended the lower text view in SilentKnight to support them, Writing Tools are available there from the contextual menu (Control-click). They work great with all the text editors I have tested, including TextEdit, BBEdit, CotEditor, Pages, my Rich Text editor DelightEd, and even in my PDF viewer Podofyllin.
The initial release of Writing Tools in 15.1 does have language and regional limitations. It requires that your Mac’s primary language, as set in Language & Region settings, is set to English (US), although you can still switch to a secondary language such as English (UK) if you prefer. The other key control is in the new Apple Intelligence & Siri settings, where Siri’s language needs to be English (United States). As I don’t like Siri’s spoken interface, I have disabled that by setting the Listen for control to Off, and instead enabled a Keyboard shortcut to open Siri’s interactive window.
Apple has announced future support for non-US variants of English, and next year for other primary languages. However, Writing Tools still work excellently on British English, even that of Charles Dickens, with the settings described above.
When you have updated or upgraded to Sequoia 15.1, I suggest you download text versions of books by your favourite author(s) from Project Gutenberg and explore features in Writing Tools using those as prose sources.
Future Sequoia updates
Apple has this week released the first beta-test of Sequoia 15.2, with most if not all of the other Apple Intelligence features, including Image Playground and ChatGPT. Assuming testing proceeds well and there are no serious problems, this is likely to be released in the first couple of weeks in December. Although not confirmed yet, this should open supported languages to include most major regional variants of English.
Slated for next year is the extension of Apple Intelligence to cover French, German, Italian, Japanese, Korean, Portuguese, Spanish, Vietnamese, and others. However, these features aren’t likely to appear in the countries of the EU this year, and Apple hasn’t yet indicated when that’s expected.
For those concerned about on- and off-device AI and privacy, all the standard features of Writing Tools and Image Playground involve on-device processing, and don’t send your data to remote servers. If you choose to enable ChatGPT access, then that is handled off-device, but is opt-in, and requires a separate sign-in process to access either an anonymised free account or an existing ChatGPT account. You can also require confirmation of any Siri requests handled with ChatGPT before sending any information off-device.
Apple has already published a list of fixes in the first beta of 15.2, although it remains to be seen what it does for users.
M4 Macs
Apple has also signalled that it will be releasing new Macs next week, widely rumoured to be the first to use the M4 chip.
Summary
Sequoia 15.1 early next week, probably on 28 October, with Writing Tools in US English, and remaining networking bug fixes.
Sequoia 15.2 already in beta, probably for release in early December, with Image Playground, ChatGPT, and the remainder of this first wave of AI tools, including most other English variants.
In macOS Mojave and earlier, all Macs booted from a single integrated volume, containing system and user files arranged in directories based on those common to many Unix systems. That started to change in Catalina, when that volume was divided into two, System and Data, and continued in Big Sur, when macOS started booting from a specially sealed and signed snapshot, the Signed System Volume or SSV. Changes since Big Sur have been smaller and more subtle, with the introduction of the paired recovery system and cryptexes. This article guides you around the volumes and directories that compose the boot volume group in macOS 15 Sequoia.
Internal SSD partitions
All Macs are now intended to be started up from their internal SSDs through a secure boot process that tries to guarantee the integrity of all loaded code from the boot ROM and firmware through to the kernel and macOS. This is simpler in an Intel Mac, whose internal SSD is divided into two partitions, one for EFI, the other for the boot volume group.
Device names given here are examples, and those seen are likely to differ.
The boot volume group is composed of:
The System volume, used to build the SSV, which is unmounted during normal running.
The Signed System Volume, a signed and sealed snapshot of the System volume, containing the immutable System.
The Data volume, by default named Macintosh HD – Data, mounted within the System directory tree, and writable by the user.
The small Preboot volume, used to store cryptexes.
The paired Recovery volume, containing a disk image of the Recovery system.
The VM volume as the backing store for virtual memory.
Possibly an Update volume that appears to have been used when upgrading to versions of Big Sur. This won’t be present on more recent Macs, and appears to have been disused since macOS 11.
The SSV is created from the System volume during macOS installation and update, and features a tree of cryptographic hashes verifying its integrity, and its signature to verify that its contents match those expected for that version of macOS.
To accommodate the more advanced secure boot of Apple silicon Macs, their internal SSDs are divided into three partitions, with an extra six volumes beyond the boot volume group.
Device names given here are examples, and those seen are likely to differ.
Apple silicon Macs have two Recovery systems: their primary Recovery is run from a disk image in the paired Recovery volume in the active boot volume group, as with Intel models. A previous version of that disk image is stored in the Apple_APFS_Recovery partition/container, where it’s available as the Fallback Recovery system, only in Apple silicon Macs. The contents of the Apple_APFS_ISC partition are small, and largely support secure boot with extended anti-replay technology (xART) for the secure enclave, and other internal features.
Another more curious difference between architectures is that the default name of the Data volume differs: in Intel Macs, it’s normally Macintosh HD – Data, while Apple silicon Macs use just Data.
Inside the boot volume group
macOS now consists of three components:
the SSV, providing the root file system, and consisting of the main immutable components
the Data volume, mounted within the root and linked to it by firmlinks, consisting of user-writable directories
cryptexes, signed and sealed disk images mounted from the Preboot volume during the boot process, containing components that can be updated without creating a new SSV, notably including Safari and its supporting frameworks, shared caches for dynamic loading and, in Apple silicon Macs, those to support Rosetta 2.
These can be traced using the volume IDs and inode numbers of directories and files within the unified file system presented to the user, and summarised in the diagram below.
Volume IDs and inode numbers given here are only examples, and you’re likely to see different ones.
This shows the location of major directories across the three components, with those located in the SSV shown in pink, those of the Data volume in blue, and contributions from cryptexes in green. The effect of firmlinks and ‘grafting’ of cryptexes into the file system is now complex. One important example is how what appears to be the single top-level Applications directory is assembled from:
bundled apps in the SSV, located in /System/Applications,
user-installed apps in /Applications in the Data volume,
Safari.app mounted from the Preboot volume in the App cryptex.
Although the SSV is identical in both Intel and Apple silicon Macs, the two architectures are now differentiated by their cryptexes. Those for Intel Macs contain only dyld shared caches with x86 support; those for Apple silicon Macs provide both ARM and x86 support, as well as AOT shared caches, for Rosetta 2.
Firmlinks
System firmlinks are still the same as listed previously in /usr/share/firmlinks and haven’t changed since Catalina:
/Applications <-> Applications
/Library <-> Library
/System/Library/Caches <-> System/Library/Caches
/System/Library/Assets <-> System/Library/Assets
/System/Library/PreinstalledAssets <-> System/Library/PreinstalledAssets
/System/Library/AssetsV2 <-> System/Library/AssetsV2
/System/Library/PreinstalledAssetsV2 <-> System/Library/PreinstalledAssetsV2
/System/Library/CoreServices/CoreTypes.bundle/Contents/Library <-> System/Library/CoreServices/CoreTypes.bundle/Contents/Library
/System/Library/Speech <-> System/Library/Speech
/Users <-> Users
/Volumes <-> Volumes
/cores <-> cores
/opt <-> opt
/private <-> private
/usr/local <-> usr/local
/usr/libexec/cups <-> usr/libexec/cups
/usr/share/snmp <-> usr/share/snmp
/AppleInternal <-> AppleInternal (on Apple engineering systems only) in each case shown as the system volume path and the Data volume path which are firmlinked together.
Directory layout
Given the volume ID and inode numbers of directories and files, together with that list of firmlinks, it’s possible to construct a full directory tree for the conjoined SSV, Data volume and cryptexes. A simplified version of that is in the diagram below.
This uses the same colour convention, with pink for the SSV, blue for the Data volume, green for cryptexes, adding red for the mountpoint of the Data volume, and yellow for firmlinked items. This demonstrates graphically how the contents of the main Applications folder are gathered from three sources.
Previous boot volume architectures from High Sierra to Monterey are explained here.
Of the two most dependable types of disk image, read-write (UDRW) disk images are the simpler. The only options you can set are its internal file system, whether the container is encrypted, and its maximum capacity. From there on, macOS will automatically Trim the disk image when it’s mounted, and adjust the size it takes on disk accordingly.
Sparse bundles can be more complex to configure in the first place, and may need routine maintenance to ensure their size on disk doesn’t grow steadily with use. This article considers how significant those complications are.
Band size
Most if not all storage divides data into blocks; in the case of SSDs, the default block size in APFS is 4,096 bytes, and that’s effectively the unit of storage in the single file of a read-write disk image.
Its equivalent in a sparse bundle is the maximum size that each of its band files can reach, normally set to 8.4 MB in macOS Sequoia and its predecessors, and equivalent to slightly more than 2,000 SSD blocks. Smaller band sizes are more efficient in the space required to store the contents of a sparse bundle, but require more band files for the same quantity of data stored. Experience with older versions of macOS suggests that it’s not a good idea for the total number of band files to exceed 100,000, but I’m not aware of any recent evidence to support that.
When creating very large sparse bundles, of the order of TB, macOS now may adjust the band size to limit their number. Some users report that those sparse bundles perform better as a result, for instance when being used to store backups on a network. There don’t appear to be any advantages to using band sizes less than 8.4 MB.
Neither DropDMG nor Disk Utility currently appear to allow any control over band size; Spundle and hdiutil not only let you specify custom band sizes when creating sparse bundles, but also allow you to change band size by copying an existing sparse bundle into a new one, which could be useful if you were changing the maximum size of a sparse bundle.
Compaction
Each time a writeable APFS or HFS+ file system is mounted in a disk image, including both sparse bundle and read-write types, storage blocks that are no longer in use should be returned for reuse by the process of Trimming. This is automatic, and in the case of single-file read-write disk images, is the only maintenance that occurs.
Because sparse bundles add another layer of band files over SSD storage, they too require periodic maintenance in the process of compaction. To compact a sparse bundle, macOS scans the band files and removes those that are no longer being used by the file system in the sparse bundle. This only applies to sparse bundles with APFS or HFS+ file systems, though, and isn’t guaranteed to free up any space. One small catch is that, by default, compaction won’t take place on a notebook running on battery power; hdiutil and Spundle provide an option to override that.
Space efficiency in use
I’ve been unable to find any objective comparison between space efficiency of modern read-write disk images and sparse bundles, so have performed my own on a USB4 external SSD connected to my Mac Studio M1 Max. I created two test cases of 125 GB images on a freshly-formatted APFS volume, one a sparse bundle with the default 8.4 MB band size, the other a read-write disk image (UDRW). In macOS Sequoia 15.0.1, the initial size they took on disk was 14.8 MB for the sparse bundle, and 336 MB for the disk image.
Using my utility Stibium, I then wrote and deleted a series of files in each, as follows:
160 files of 2 MB to 2 GB written totalling 53 GB
40 largest of those, sizes 600 MB to 2 GB, were then deleted, leaving 120
another 160 files written totalling 53 GB
40 largest of those deleted, leaving a total of 240
another 160 files written totalling 53 GB
40 largest of those deleted, leaving a total of 360
another 160 files written totalling 53 GB
40 largest of those deleted, leaving a total of 480
another 160 files written totalling 53 GB, leaving a total of 640 files.
At that point, the sparse bundle had 104.08 GB stored in 12,408 band files, and the read-write disk image had 91.63 GB stored in its single file. I then deleted all files and folders in each of the images, and unmounted and remounted each image several times to ensure Trimming had completed. The sparse bundle then had 1.12 GB in 135 band files, and offered 124.66 GB of free space when mounted; the read-write disk image was slightly smaller at 937.9 MB on disk and offered exactly the same amount of free space when mounted.
Compacting the empty sparse bundle was reported as reclaiming 52.3 MB, and reduced the disk space taken by the band files slightly to 1.06 GB while retaining the same number of bands, and still offering 124.66 GB of free space when mounted.
In this case, following writing 800 files totalling over 250 GB, and deleting a total of 160 of them, compaction made a remarkably small difference to the free space returned following removal of all files. There is also very little difference in the space efficiency of sparse bundles and modern read-write disk images.
One consistent difference observed throughout was in write speeds: they remained constant at 3.2 GB/s for the sparse bundle, and 1.0 GB/s for the read-write disk image, as reported here previously.
Conclusions
Sparse bundles are more complicated than read-write disk images (UDRW), with band size to be set, and compaction to be performed.
Default band size appears to work well, and manually setting band size should seldom be necessary.
Both types appear highly efficient in their use of disk space, with only small differences between them.
Although it might be important to compact sparse bundles in some cases, the amount of free space returned by compaction is unlikely to be significant in many circumstances.
I don’t recall seeing any Apple app promoted from the ranks as fast as Tips. It didn’t exist in Monterey, when it was still the neglected HelpViewer app lingering unloved in CoreServices. Then in Ventura it took on its new name, while staying hidden, as it remained in Sonoma too. Come Sequoia, it has leapt from version 10 to 15 and joined Apple’s first league apps between Time Machine and TV in the main Applications folder.
We can all recall the fall from grace of Network Utility and now Keychain Access, in leaving the main Utilities folder to go to their demise in CoreServices, but I don’t remember any other app making that trip in reverse, and so quickly. Of course, the app that now presents itself as Tips knows that it’s still actually com.apple.helpviewer inside, so is this real change, or merely cosplay?
When researching articles here, in most cases my first task is to consult Apple’s support documentation, in both Help pages and its extended support articles. The Tips app could now be a useful front end for that, although it doesn’t seem to offer anything more than a search in Safari does, and in some respects isn’t as helpful. Try searching Tips for terms like pin or pinning and you’ll see anything but its use in iCloud Drive, for which only the official phrase keep downloaded proves fruitful.
Tips currently appears stunted in other respects. Although it reluctantly gives access to the contents of third-party Help books, provided they’re in the traditional format and not PDFs, the only ones it offers for browsing are those for Apple’s apps. Disappointingly, it can’t find or open any man pages, where Apple now keeps a lot of more important information, for which you still have to open Terminal or resort to a third-party alternative.
It’s only when browsing Apple’s own documentation that you realise how much of it now lacks illustrations. Its model-specific hardware guides are the exception, and are accompanied by excellent labelled images, but Tips limits those to currently shipping models. If your Mac is Apple silicon but more than a year or two old, then you’ll just have to resort to the web. Sections aimed at the novice, such as Set your Apple Account picture, are well supported by cutouts from screenshots. But look at Store files in iCloud Drive, and there are five substantial topics with just a single lightbulb icon and no screenshots.
For me the biggest disappointment is that Tips only offers what’s readily available, in terms of technical content. If it isn’t in an existing Help book, then Tips draws a blank. Search for DFU mode, restore firmware, or even how to restore mac firmware, and Tips can’t find anything to suggest from among the main Help books. Extending its scope to include All Topics only finds useful results if you already have Apple Configurator installed, in which case it’s surely simpler just to open that app’s Help book directly, as Tips doesn’t include it in its list of User Guides either. This is all the more disappointing, as Apple has published an excellent support article entitled How to revive or restore Mac firmware, readily found by searching in Safari, but apparently beyond the scope of Tips.
This initial version feels clunky in other respects. Most annoyingly, it’s incapable of remembering what you were last reading. Every time you open the Tips app, it returns to its default home page, rather than restoring its contents to those shown when you last quit the app. As there’s no means of bookmarking items, you’ll find yourself wasting a lot of time navigating back to information you’ve already found. Neither does the app support more than its single window, and has no way of splitting that to allow you to refer to two or more pages at the same time.
In its current state, it’s hard to see how it justifies its place among Apple’s first league apps, which makes me wonder whether it’s destined to use AI in a future release of Sequoia. However, for that to be any improvement, Tips is going to have to broaden and deepen its knowledge, or it may as well crawl back to CoreServices and its former name of HelpViewer.
The most distinguishing feature of Doctor Who’s TARDIS is that it’s much smaller on the outside, only capable of accommodating a couple of people at a squeeze, but huge on the inside. Allow me to introduce you to a sparse bundle that’s similar, in that its bundle size is only 14 MB, but the size of its internal band files comes to a total of over 90 GB. Here’s how to make yourself one.
Method
To do this, you need an Intel or Apple silicon Mac running Sequoia 15.0 or 15.0.1, and an external SSD. A hard disk might do, but I haven’t tried it.
Connect and format the external SSD in APFS, using Disk Utility. Then create an empty sparse file with the following specifications:
Size: 10 GB or more
Format: APFS
Encryption: none
Partitions: Single partition – GUID Partition Map
Image Format: sparse bundle disk image.
Save that to the external SSD at its top (root) level. As Disk Utility leaves your new sparse bundle mounted, unmount it, wait a few seconds, then double-click it to mount it again. Copy one or more large files to the mounted sparse bundle. With a bundle size of 10 GB, something around 5-6 GB should be ideal, but the sky’s the limit if you created a large sparse bundle to begin with.
Once that’s complete, unmount the sparse bundle, then select it on your external SSD and press Command-I to see the Get Info dialog.
Although the size will vary slightly, in my case, with a nearly full 125 GB sparse bundle, the Finder reports its size as 13.8 MB, or 14.8 MB on disk.
So where did all that file vanish to? Has APFS somehow made it into a sparse file? Sadly not: view the contents of the sparse bundle, select its band folder containing all the band files with your data, and its size will be much bigger than that of the bundle. In my case, that’s 92.7 GB in more than 10,000 band files.
There’s your TARDIS: a 13.8 MB sparse bundle containing over 90 GB of band files.
Breaking the illusion
Once you’ve created your TARDIS sparse bundle, you can see its magic still works when you connect that external SSD to another Mac. But there’s one thing that will break the magic spell: create a folder in the same volume, and move the sparse bundle inside that. If you then mount it and add a file, you’ll see it suddenly expand to its full bundle size. Similarly, if you repeat the whole trick, but this time save the sparse bundle inside a folder, then it won’t work. This is why you can only do this on an external SSD, as you can’t write anything to the top level of your boot disk.
I don’t know whether this works with other bundles, such as VMs for Apple silicon Mac virtualisation, but it might be interesting to try it.
Act now, don’t wait
While this remarkable bug is present in macOS Sequoia 15.0 and 15.0.1, I’m afraid its days are numbered. If you want to experience the TARDIS sparse bundle, you’ve only got another week or two, as it appears to be fixed in the current beta of 15.1. For once, I’ll be a little sad to see a bug fixed, although when I discovered it, it couldn’t have been more infuriating.
Apple has just released an update to XProtect Remediator security software for Catalina or later, bringing it to version 147. The previous version was 145.
Apple doesn’t release information about what security issues this update might add or change. There are no changes in the number or names of its scanning modules, and Bastion rules also remain unchanged.
You can check whether this update has been installed by opening System Information via About This Mac, and selecting the Installations item under Software.
A full listing of security data file versions is given by SilentKnight, LockRattler and SystHist for El Capitan to Sequoia available from their product page. If your Mac has not yet installed this update, you can force it using SilentKnight, LockRattler, or at the command line.
If you want to install this as a named update in SilentKnight, its label is XProtectPayloads_10_15-147.
I have updated the reference pages here which are accessed directly from LockRattler 4.2 and later using its Check blog button.
Over the last few years, the performance of disk images has been keenly debated. In some cases, writing to a disk image proceeds at a snail’s pace, but this has appeared unpredictable. Over two years ago, I reported sometimes dismal write performance to disk images, summarised in the table below.
This article presents new results from tests performed using macOS 15.0.1 Sequoia, which should give a clearer picture of what performance to expect now.
Methods
Previous work highlighted discrepancies depending on how tests were performed, whether on freshly made disk images, or on those that had already been mounted and written to. The following protocol was used throughout these new tests:
A 100 GB APFS disk image was created using Disk Utility, which automatically mounts the disk image on completion.
A single folder was created on the mounted disk image, then it was unmounted.
After a few seconds, the disk image was mounted again by double-clicking it in the Finder, and was left mounted for at least 10 seconds before performing any tests. That should ensure read-write disk images are converted into sparse file format, and allows time for Trimming.
My utility Stibium then wrote 160 test files ranging in size from 2 MB to 2 GB in randomised order, a total of just over 53 GB, to the test folder.
Stibium then read those files back, in the same randomised order.
The disk image was then unmounted, its size checked, and it was trashed.
All tests were performed on a Mac Studio M1 Max, using its 2 TB internal SSD, and an external Samsung 980 Pro 2 TB SSD in an OWC Express 1M2 enclosure running in USB4 mode.
Results
These are summarised in the table below.
Read speeds for sparse bundles and read-write disk images were high, whether the container was encrypted or not. On the internal SSD, encryption resulted in read speeds about 1 GB/s lower than those for unencrypted tests, but differences for the external SSD were small and not consistent.
Write speeds were only high for sparse bundles, where they showed similar effects with encryption. Read-write disk images showed write speeds consistently about 1 GB/s, whether on the internal or external SSD, and regardless of encryption.
When unencrypted, read and write speeds for sparse (disk) images were also slower. Although faster than read-write disk images when writing to the internal SSD, read speed was around 2.2 GB/s for both. Results for encrypted sparse images were by far the worst of all the tests performed, and ranged between 0.08 to 0.5 GB/s.
Surprisingly good results were obtained from a new-style virtual machine with FileVault enabled in its disk image. Although previous tests had found read and write speeds of 4.4 and 0.7 GB/s respectively, the Sequoia VM achieved 5.9 and 4.5 GB/s.
Which disk image?
On grounds of performance only, the fastest and most consistent type of disk image is a sparse bundle (UDSB). Although on fast internal SSDs there is a reduction in read and write speeds of about 1 GB/s when encrypted using 256-bit AES, no such reduction should be seen on fast external SSDs.
On read speed alone, a read-write disk image is slightly faster than a sparse bundle, but its write speed is limited to 1 GB/s. For disk images that are more frequently read from than written to, read-write disk images should be almost as performant as sparse bundles.
However, sparse (disk) images delivered weakest performance, being particularly slow when encrypted. Compared with previous results from 2022, unencrypted write performance has improved, from 0.9 to 2.0 GB/s, but their use still can’t be recommended.
Performance range
It’s hard to explain how three different types of disk image can differ so widely in their performance. Using the same container encryption, write speeds ranged from 0.08 to 3.2 GB/s, for a sparse image and sparse bundle, on an external SSD with a native write speed of 3.2 GB/s. It’s almost as if sparse images are being deprecated by neglect.
Currently, excellent performance is also delivered by FileVault images used by Apple’s lightweight virtualisation on Apple silicon. The contrast is great between its 4.5 GB/s write speed and that of an encrypted sparse image at 0.1 GB/s, a factor of 45 when running on identical hardware.
Recommendations
For general use, sparse bundles (UDSB) are to be preferred for their consistently good read and write performance, whether encrypted or unencrypted.
When good write speed is less important, read-write disk images (UDRW) can be used, although their write performance is comparable to that of USB 3.2 Gen 2 at 10 Gb/s and no faster.
Sparse (disk) images (UDSP) are to be avoided, particularly when encrypted, as they’re likely to give disappointing performance.
Encrypting UDSB or UDRW disk images adds little if any performance overhead, and should be used whenever needed.
Apple has just released an update to XProtect for all supported versions of macOS, bringing it to version 5278. As usual, Apple doesn’t release information about what security issues this update might add or change.
Relative to the last version released for all supported versions of macOS (5277), this version adds three new definitions for MACOS.ADLOAD.I, MACOS.SOMA.G and MACOS.SOMA.H.
You can check whether this update has been installed by opening System Information via About This Mac, and selecting the Installations item under Software.
A full listing of security data file versions is given by SilentKnight, LockRattler and SystHist for El Capitan to Sequoia available from their product page. If your Mac hasn’t yet installed this update, you can force it using SilentKnight, LockRattler, or at the command line.
If you want to install this as a named update in SilentKnight, its label is XProtectPlistConfigData_10_15-5278.
For Sequoia only: there’s no sign of this update being made available in iCloud, which still returns an XProtect version of 5272. If you download and install it using Software Update, softwareupdate or SilentKnight, then once that’s complete you need to update the primary XProtect bundle in Terminal using the command sudo xprotect update
then entering your admin password. If you’re unsure what to do, this article explains it comprehensively and simply.
I have updated the reference pages here which are accessed directly from LockRattler 4.2 and later using its Check blog button.
Recent versions of macOS have added new ways to lay out windows, including Stage Manager and most recently tiling. This collection of screenshots demonstrates the repertoire available in macOS 15 Sequoia, with brief notes to suggest how to get the best from them. Controls and options for most of these are in Desktop & Dock settings, except where given otherwise. There are also many third-party enhancements available: those are intentionally omitted for the sake of simplicity and clarity. Similarly, there are often several different controls and actions that can achieve the same effect. Only the more obvious are given here. To enlarge any of the screenshots below, click on the image.
Full screen single window
This is the simplest of all, best suited to windows already containing multiple views that need full space, here with Xcode. Control whether the window is displayed with or without the menu bar in Control Centre settings, in Automatically hide and show the menu bar at its foot. Integrate this with access to the Finder and other apps by setting it in its own Space.
To put a window in full screen mode, click on its green traffic light control at its top left, or use the command in the View menu. To exit, use the same controls.
Freestyle overlapping
Probably the most traditional and popular layout, with multiple overlapping windows from different apps. Although dynamic and flexible, it’s easy to lose track of apps and windows in their multiple layers.
To find a lost window or app, enter Mission Control by pressing the F3 key, then click on the window to bring it into focus. Switch focus between apps by selecting the app in the Dock.
Use the Desktop to collect your working documents, or set it as wallpaper.
Tiled
Some apps, here BBEdit in its Find Differences feature, tile multiple views into a single window, here with linked scrollers. Sequoia provides additional tools to tile app windows so they don’t overlap. These are best suited to layouts with 2-4 windows occupying similar areas, but can be adapted for others.
Simplest are two windows arranged in a split view, either horizontally or vertically. Arrange them using the Move & Resize commands in the Window menu, from the window’s green traffic light control popup menu, or by moving the window with controls enabled in Desktop & Dock settings. Note the latter don’t work with windows dragged to the bottom edge of the screen, only the top and sides.
For those who like a clean, regular look with four open windows, tile them into quadrants with the same controls. This shows the small gaps between windows when Tiled windows have margins is enabled in Desktop & Dock settings.
Use a regular tiled layout adjusted to suit their contents better. Resizing windows manually loses the margins of the original tiling.
The alternative Full Screen Split View is quick to create. To put the currently focussed window into one half, use one of the Full Screen Tile options in the Window menu, or its green traffic light popup menu. That puts the other half of the display into Mission Control, where you select the window to place in the other half of the Split View.
Adjust the balance between the size of the two windows using the central bar. To return the windows to normal, click on their green traffic light control.
On their own, tiled layouts are constraining, but when used in their own Space can be valuable.
Stage Manager
Stage Manager puts a suite of windows from one or more apps into a single active group on stage, with quick switching between different suites. Enable and control it in its section in Desktop & Dock settings. Switch between different window groups simply by clicking on them in the icon list at the left.
Management of suites is dynamic, and detailed in the following articles:
For further flexibility, use Stage Manager in one or more Spaces.
Spaces
Spaces are complete Desktop layouts you can switch between, accessed through Mission Control. To create a second Space, press F3 for Mission Control and move your pointer to the top of the display, then click on the + at the right. Options are at the very bottom of Desktop & Dock settings. Any of the above layouts can be included as a Space.
Multiviews
One remaining problem with layouts is support for multiple views of the same document. Better text editors let you split a window into two views of the same document, useful when you need to refer back while editing. Browsers will open two tabs or windows on the same page. However, very few PDF viewers let you view more than one section at once of the same PDF in multiple windows. That’s a feature of my free Podofyllin.
Until about three years ago, most types of disk image had fixed size. When you created a read-write disk image (UDRW) of 10 GB, it occupied the same 10 GB on disk whether it was full or empty. The only two that could grow and shrink in size were sparse bundles and sparse disk images, with the former generally preferred for its better resizing and performance.
This changed silently in Monterey, since when read-write disk images have been automatically resized by macOS, and saved in APFS sparse file format. As this remains undocumented, this article explains how this works, and how and where you can use it to your advantage.
Sparse
In this context, the word sparse refers to two very different properties:
Sparse bundles and sparse (disk) images are types of disk image that can change in size, and can be stored on a wide range of file systems and storage, including on NAS and other networked storage.
Sparse files are stored in a highly efficient file format available in modern file systems, where only the data in a file is stored, and empty space within that file doesn’t waste storage space. It’s available in APFS, but not in HFS+.
Requirements
For read-write disk images to be sparse, the following are required:
The disk image must be saved to, and remain on, an APFS volume.
The file system within the disk image can be either APFS or HFS+, but not FAT or ExFAT.
The disk image must be created and unmounted first. For that initial mount, the disk image isn’t a sparse file, so occupies its full size on disk.
Whenever that disk image is mounted again, and has sufficient free space within its set limit, it will be saved in sparse file format, and occupy less than its full size on disk.
Demonstration
Create a new read-write disk image of at least 1 GB size with an internal file system of APFS on an APFS volume, using DropDMG, Disk Utility, or an alternative.
Once it has been created and mounted, unmount it in the Finder.
Select the disk image in the Finder and open the Get Info dialog for it. Confirm that its size on disk is the same as that set.
(Optional) Open the disk image using Precize, and confirm that it’s not a sparse file.
Double-click the disk image to mount it in the Finder, and wait at least 10 seconds before unmounting it.
Select the disk image in the Finder and open the Get Info dialog for it. Confirm that its size on disk is now significantly less than that set.
(Optional) Open the disk image using Precize, and confirm that it has now become a sparse file.
How it works
When you create the disk image, macOS creates and attaches its container, and creates and mounts the file system within that. This is then saved to disk as a regular file occupying the full size of the disk image, plus the overhead incurred by the disk image container itself. No sparse files are involved at this stage.
When that disk image is mounted next, its container is attached through diskarbitrationd, then its file system is mounted. If that’s APFS (or HFS+), it undergoes Trimming, as with other mounts. That coalesces free storage blocks within the image to form one contiguous free space. The disk image is then saved in APFS sparse file format, skipping that contiguous free space. When the file system has been unmounted and the container detached, the space used to store the disk image has shrunk to the space actually used within the disk image, plus the container overhead. Unless the disk image is almost full, the amount of space required to store it on disk will be smaller than the full size of the disk image.
This is summarised in the diagram below.
The size of read-write disk images is therefore variable depending on the contents, the effectiveness of Trimming in coalescing free space, and the efficiency of APFS sparse file format.
Conversion to sparse file
When mounting an APFS file system in a read-write disk image, APFS tests whether the container backing store is a sparse format, or a flat file. In the case of a newly created read-write disk image that hasn’t yet been converted into a sparse file, that’s detected prior to Spaceman (the APFS Space Manager) scanning for free blocks within its file system. When free blocks are found, APFS sets the type of backing store to sparse, gathers the sparse bytes and ‘punches a hole’ in the disk image’s file extents to convert the container file into sparse format. That appears in the log as: handle_apfs_set_backingstore:6172: disk5s1 Set backing store as sparse
handle_apfs_set_backingstore:6205: disk5 Backing storage is a raw file
_punch_hole_cb:37665: disk3s5 Accumulated 4294967296 sparse bytes for inode 30473932 in transaction 3246918, pausing hole punching
where disk5s1 is the disk image’s mounted volume, and disk3s5 is the volume in which the disk image container is stored.
Trimming for efficient use of space
That conversion to sparse format is normally only performed once, but from then on, each time that disk image is mounted it’s recognised as having a sparse backup store, and Spaceman performs a Trim to coalesce free blocks and optimise on-disk storage requirements. For an empty read-write disk image of 2,390,202 blocks of 4,096 bytes each, as created in a 10 GB disk image, log entries are: spaceman_scan_free_blocks:4106: disk5 scan took 0.000722 s, trims took 0.000643 s
spaceman_scan_free_blocks:4110: disk5 2382929 blocks free in 7 extents, avg 340418.42
spaceman_scan_free_blocks:4119: disk5 2382929 blocks trimmed in 7 extents (91 us/trim, 10886 trims/s)
spaceman_scan_free_blocks:4122: disk5 trim distribution 1:0 2+:0 4+:0 16+:0 64+:0 256+:7
accounting for a total of 9.8 GB.
Changes made to the contents of the disk image lead to a gradual reduction in Trim yield. For example, after adding files to the disk image and deleting them, instead of yielding the full 9.8 GB, only 2,319,074 blocks remain free, yielding a total of 9.5 GB.
For comparison, initial Trimming on a matching empty sparse bundle yields the same 9.8 GB. After file copying and deletion, and compaction of the sparse bundle, Trimming performs slightly better, yielding 2,382,929 free blocks for a total of 9.6 GB. Note that Trimming of sparse bundles is performed by APFS Spaceman separately from management of bands in backing storage, which isn’t a function of the file system.
Size efficiency
Although read-write disk images stored as sparse files are efficient in their use of disk space, they’re still not as compact as sparse bundles. For an empty 10 GB image, the read-write type requires 240 MB on disk, but a sparse bundle only needs 13.9 MB. After light use storing files, then deleting the whole contents, a 10 GB read-write disk image grows to occupy 501 MB, but following compaction a sparse bundle only takes 150 MB. That difference may not remain consistent over more prolonged use, though, and ultimately compacting sparse bundles may cease freeing any space at all.
It’s also important to remember that sparse bundles need to be compacted periodically, if any of their contents are deleted, or they may not reduce in size after deletions. Read-write disk images can’t be compacted, and reclaim disk space automatically.
Benefits and penalties
Read-write disk images saved as sparse files are different from sparse bundles in many ways. Like any sparse file, the disk image still has the same nominal size as its full size, and differs in the space taken on disk. Sparse bundles should normally only have band files sufficient to accommodate their current size, so their nominal size remains similar to the space they take on disk. The result is that, while read-write disk images in sparse file format will help increase free disk space, their major benefit is in reducing ‘wear’ in SSDs by not wasting erase-write cycles storing empty data.
Unlike the band file structure in sparse bundles, which can be stored on almost any disk, APFS sparse files have to be treated carefully if they are to remain compact. Moving them to another file system or over a network is likely to result in their being exploded to full size, and I have explained those limitations recently.
Both read-write disk images and sparse bundles deliver good read performance, but write performance is significantly impaired in read-write disk images but not sparse files. Encryption of disk images and sparse bundles also has significant effects on their performance, and in some cases write performance is badly affected by encryption. I have previously documented their performance in macOS Monterey, and will be updating those figures for Sequoia shortly.
Summary
Read-write disk images saved on APFS storage in Monterey and later are no longer of fixed size, and should use significantly less disk space unless full, provided the disk image has been unmounted at least once since creation, and the file system in the disk image is APFS or HFS+.
This is because macOS now saves the disk image in sparse file format.
To retain the disk image in sparse file format, it needs to remain in APFS volumes, and normal precautions are required to maintain its efficient use of space. The disk image can’t be manually compacted, in the way that sparse bundles require to be.
The main benefit of this strategy is to minimise erase-write cycles, so reducing ‘wear’ on SSDs.
Sparse bundles are still more efficient in their use of disk space, and have higher write speeds, but read-write disk images are now closer in their efficiency and performance.
Writing a coherent and comprehensive description of a painting is an excellent exercise not only for writing skills, but to remind yourself of the adage that a picture is worth a thousand words. Nowhere is this more appropriate than when describing graphical user interfaces such as macOS, as I reminded myself this week when trying to sort out a problem with Desktop behaviour.
That took me to Sequoia’s revised System Settings, where those for Desktop & Dock have become a farrago covering anything from the default web browser to the addition of margins to tiled windows. Although I’ve been one of System Settings’ few advocates, that section has become an example of what they shouldn’t be.
After the Finder, tools for configuring the appearance and behaviour of the interface are the most important. For many years, we had to do this through the keyhole of the fixed window size provided by System Preferences.
Many of its more complex panes had to resort to multiple tabs, here in the Network pane of 2002.
This was no easier for relatively simple panes, such as Sound, seen here in macOS 12 nearly 20 years later.
Ventura’s completely reworked System Settings were controversial, but at last provided the ability to expand panes vertically, although at that time many used floating windows to arrange discrete groups of settings in a hierarchy, such as those for Stage Manager.
Some of those used pictures, although most were illustrative rather than replacements for a thousand words, unlike the screenshot below.
Sequoia’s Desktop & Dock settings have been flattened out into a single view that’s so deep it even has to be scrolled on a Studio Display. Its sections include the Dock, Desktop, Stage Manager, Widgets, default web browser, Windows, tiling, Mission Control, keyboard shortcuts and Hot Corners, while further settings for the Desktop are controlled in Appearance and Wallpaper. Many of these are obscure unless you already use a feature, and many need further help, even for those who have been using macOS for years.
Like Desktop & Dock settings, its help page uses words exclusively rather than pictures, often doing little more than paraphrasing the text labels already shown in System Settings. Apple Support is one of many to provide a video tutorial for these settings, but those are of limited benefit when you just want to check how the options available for tabs in windows affect the appearance and behaviour of document windows, for example. Yet that would be so simple to demonstrate in three cutouts from screenshots, and so much more meaningful than the paraphrasing in Help: “Choose when you want documents to open in a tab (instead of in a new window): Never, Always or In Full Screen.”
In the last few years, Apple has devoted considerable engineering effort into enriching the macOS human interface, with Stage Manager and most recently Sequoia’s tiling feature. Even though you may not wish to use either, they do have their uses and some find them powerful.
Tiling is a case in point, as Apple seems to have decided that it should be enabled by default when you upgrade to Sequoia. That results in frustratingly changed behaviour when you drag a small window up to the top or another edge of the display, and it suddenly tiles out to cover the whole screen, without offering any Undo. Only by rummaging through System Settings will you discover that you can return that to normal by turning off Tile by dragging windows to screen edges in Desktop & Dock settings, although this has nothing to do with the Desktop or Dock.
In principle, I still believe System Settings is the right way forward, with its vertical flexibility and use of SwiftUI. But leading designs using those now come from third-party developers. It’s time for Apple to reassert its former mastery of the human interface, and realise the potential of SwiftUI by making System Settings a paragon rather than a pickle.
This is the time of year when macOS keeps offering you the upgrade to the new version of macOS, but you may not want to go there yet. This article explains how you can stay running your existing version of macOS, while keeping it up to date.
Skint and SkintM
By default, SilentKnight is intended to download all the latest updates from Apple’s software update servers. Although you can configure it to behave differently, as I explain below, you may like to look at Skint or its menu bar sibling SkintM. Those don’t check for updates, they only check installed versions. They will warn you when your Mac has fallen behind with updates, and let you decide what you want to do.
Skint and SkintM do check that your Mac is running the latest version of its major release of macOS, and is happy if you’re still running Monterey, Ventura or Sonoma, as well as Sequoia, but it will advise you when they fall behind with their security updates; after all, that’s what it’s for.
Switching SilentKnight to manual
For many, SilentKnight’s button to Install all updates is most worrying, as that might inadvertently upgrade macOS to Sequoia. In fact, it isn’t dangerous at all, but before I explain why, you can remove that button altogether. Open SilentKnight, and its Settings, and set them to look as below.
This will still download and install the updates you want, but you won’t be tempted to inappropriately install the lot of them.
Once you’ve done that, click the Set button, quit SilentKnight, and run it again. Now when it tells you there are updates to be installed, you can’t click on a button to bring upgrade disaster.
In fact, after testing SilentKnight with macOS updates and upgrades, they don’t work like that anyway. If SilentKnight were to download a macOS update or upgrade, it can’t complete its installation. macOS first tells you that the update couldn’t be installed, then offers to install it in Software Update. All you have to do is shut your Mac down at that point, then start it up in Safe mode and the update will be stopped.
However, for your comfort and safety, I recommend untickingAllow Install All Updates, just in case.
Installing only the updates you want
Having avoided the update you don’t want, you now need to download and install those that you do. Scroll the lower text view to the bottom, to reveal all the updates available. Each has an opening line that declares it’s a Label, like * Label: XProtectPayloads_10_15-142
It’s that label you use to identify each update.
In the File menu, select the Install Named Update… command to open the manual updating window. One by one, copy and paste the label from the main window into the Name of update box and click on the Install Named Update button. SilentKnight will then tell you that it has been downloaded and installed. It only takes a few seconds to work through a list of updates like XProtect that you do want, and bring your Mac up to date without inadvertently upgrading it to Sequoia.
Further information
SilentKnight has a wealth of additional information that will help you solve problems like these. The most common are explained in its short text SilentKnight Help, in the Help menu, and there’s also a detailed Help Reference in the same menu.
Key points
Open Settings, and untickAllow Install All Updates, click Set, then quit the app. Open it again, and install each named update one at a time using that command in the File menu, pasting the Label in for each wanted update and clicking the Install Named Update button for each.
This assumes that you are running the latest version of SilentKnight, 2.11. If you’re still running version 1 you need to update for Catalina or later.
Many of you still seem puzzled as to how XProtect installs and updates in macOS Sequoia. This article tries to make this clear, so you can keep XProtect up to date.
Sonoma and earlier
Until this changed in Sequoia, XProtect has been straightforward: its data is stored in a bundle named XProtect.bundle, in the path /Library/Apple/System/Library/CoreServices, which is on the Data volume so it can readily be updated. When you or macOS downloads an XProtect update, it simply replaces that bundle with the new one. This is shown in the diagram below.
Sequoia
XProtect in macOS 15 prefers not to use the XProtect.bundle in its old location of /Library/Apple/System/Library/CoreServices (although it can do if there’s no alternative). Instead, it looks for XProtect.bundle in its new location, /var/protected/xprotect.
However, when you or your Mac use the old update system, including Software Update, softwareupdate or SilentKnight, that still installs the update in the old location, where it won’t normally be used by XProtect when making its checks. What’s supposed to happen is that at least once a day, macOS checks whether there’s a newer update in the old location. If there is, then it should automatically prepare and move that to the new location in /var/protected/xprotect for XProtect to use.
If you want that to happen immediately, then you can run the following command in Terminal: sudo xprotect update
then enter your admin user’s password. The xprotect command tool will then complete the installation of that update from its old location in /Library/Apple/System/Library/CoreServices into its new location in /var/protected/xprotect.
There’s also a second way that XProtect in Sequoia can be updated, and that’s over a connection to iCloud. If that’s used, then the update is installed straight into its new location, and doesn’t change the XProtect bundle in the old location at all. Although Apple has used that earlier, all XProtect updates since the release of Sequoia have come using the old Software Update system, so have needed to be completed using the xprotect command in Terminal.
This is shown in the diagram below. The blue boxes show the old Software Update system, and the pink boxes are the new parts that ensure the update is installed in the new location.
SilentKnight
SilentKnight still works using softwareupdate, and can’t use the new xprotect command for updates yet, because that requires structural changes in the app that will be available in version 3. However, in Sequoia it reports the version of XProtect installed in the new location, as that’s the one that XProtect now uses.
When SilentKnight discovers a new version of XProtect via softwareupdate, it therefore installs that in the old location, in the path /Library/Apple/System/Library/CoreServices. It has no choice but to do that. Once that’s been installed to the old location, the version shown for XProtect won’t change, as that requires macOS to complete the second stage of the installation. You can then either:
leave macOS to complete the installation itself, which should happen over the next day or so, or
run sudo xprotect update in Terminal, which will complete that update immediately. SilentKnight will then show the updated version number correctly.
Key points
In Sequoia, when XProtect is updated by Software Update, softwareupdate or SilentKnight, you should either leave macOS to complete that installation, or run sudo xprotect update in Terminal if you want it to be updated immediately.
This only applies to macOS Sequoia: Sonoma and earlier still work as they always have done.
Apple has just released an update to XProtect for all supported versions of macOS, bringing it to version 5277. As usual, Apple doesn’t release information about what security issues this update might add or change.
Relative to the last version released for all supported versions of macOS (5276), this version contains extensive changes, largely of an editorial nature. It adds one new detection rule for MACOS.PIRRIT.CHU, and removes rules for OSX.Genieo.C, OSX.Genieo.B, OSX.Genieo.A and OSX.Leverage.a.
Many rules have changes to their detection hashes, where existing SHA1 hashes are replaced with SHA256. Among the rules changed by this are 36:
OSX.Proton.B
OSX.Vindinstaller.A
OSX.OpinionSpy.B
OSX.InstallImitator.C
OSX.Eleanor.A
OSX.InstallImitator.A
OSX.VSearch.A
OSX.Machook.A
OSX.Machook.B
OSX.iWorm.A
OSX.iWorm.B/C
OSX.NetWeird.ii
OSX.NetWeird.i
OSX.GetShell.A
OSX.Abk.A
OSX.CoinThief.A
OSX.CoinThief.B
OSX.CoinThief.C
OSX.HellRTS.A
OSX.MacDefender.B
OSX.QHostWB.A
OSX.Revir.A
OSX.Revir.ii
OSX.Flashback.A
OSX.Flashback.B
OSX.Flashback.C
OSX.FileSteal.ii
OSX.MaControl.i
OSX.Revir.iii
OSX.Revir.iv
OSX.SMSSend.i
OSX.SMSSend.ii
OSX.eicar.com.i
OSX.AdPlugin.i
OSX.AdPlugin2.i
OSX.Prxl.2
You can check whether this update has been installed by opening System Information via About This Mac, and selecting the Installations item under Software.
A full listing of security data file versions is given by SilentKnight, LockRattler and SystHist for El Capitan to Sequoia available from their product page. If your Mac hasn’t yet installed this update, you can force it using SilentKnight, LockRattler, or at the command line.
If you want to install this as a named update in SilentKnight, its label is XProtectPlistConfigData_10_15-5277.
For Sequoia only: so far, I have seen no sign of this update in iCloud, which still returns an XProtect version of 5272. If you download and install it using Software Update, softwareupdate or SilentKnight, then once that is complete you need to update the primary XProtect bundle in Terminal using the command sudo xprotect update
then entering your admin password.
I have updated the reference pages here which are accessed directly from LockRattler 4.2 and later using its Check blog button.
A disk image is a file or a bundle containing what could otherwise be the contents of a disk. It’s a common way to store and move complete file systems in a neat package, for items that need to be separated from the physical storage provided by a drive. macOS uses disk images for some tasks of great importance, including:
Recovery and Hardware Diagnostics systems,
additions to macOS such as Safari, its supporting frameworks, and dyld caches, in cryptexes,
networked storage for Time Machine backups, in sparse bundles,
lightweight virtual machines on Apple silicon Macs.
You could use them to store encrypted data on unencrypted volumes, and they’re often used for delivering Apple and third-party software.
Disk images are poorly documented for both users and developers, and have changed significantly over the last few years. Articles in this series explain how to choose between different types of disk image, how to create and use them, and what to do when they go wrong.
Containers and file systems
Disk images consist of two distinct components: the file or bundle itself functioning as a container, and the file system contained inside it. When referred to in this context, disk image containers are completely unrelated to the sandbox containers found in ~/Library/Containers.
This distinction is important in several respects, although it isn’t apparent when you use disk images in the Finder. Preparing a disk image for access involves two separate functions: attaching its container, and mounting any file systems found inside it. When that’s performed by the Finder, perhaps by double-clicking the disk image, those two actions appear fused into one. Similarly, removing the disk image requires all its mounted file systems to be unmounted first, then the container is detached.
One feature that’s widely confused is the encrypted disk image. This involves encryption of the whole container, rather than using an encrypted file system within it. Now that Disk Utility no longer supports the creation of encrypted HFS+ volumes, one remaining alternative is to use an encrypted disk image containing an HFS+ volume.
If you want an analogy for disk images, attaching the container is like connecting an external disk, and once that has been performed, the file systems contained by that disk have to be mounted before you can access their contents.
Types
There are many different types of disk image in use, of which the two this series is most concerned with are plain UDIF read-write disk images (UDRW), and sparse bundles (UDSB). Others you may encounter include:
plain UDIF read-only (UDRO),
various compressed versions of UDRW,
CD/DVD master for export (UDTO),
sparse disk image (UDSP), a single file rather than a bundle.
Those specify the container format; within each, there’s the usual choice of file systems, although throughout these articles it will normally be assumed that APFS will be used unless otherwise specified.
The word sparse in sparse bundle and sparse disk image doesn’t refer to APFS sparse files, but to the fact that those types of disk image can grow and diminish in size, and normally try to occupy the minimum amount of disk space. This is an unfortunate name collision.
Structure
With the exception of sparse bundles, all disk images are contained within a single file of opaque structure.
Sparse bundles consist of a single bundle folder containing:
bands, a folder containing the contents of the disk image in band files
info.plist and its backup copy info.bckup, containing settings including band size
lock, an empty file
mapped, a folder containing small data files to match all of the band files except the first
token, an empty file.
Container size
Until this changed in Monterey (or thereabouts), non-sparse disk images had fixed container sizes. Create a UDIF read-write disk image (UDRW) of 10 GB, and the space occupied by it on disk was approximately 10 GB, whether it was empty or full. Although it remains undocumented, when stored on APFS volumes, UDRW disks can now take advantage of APFS sparse file format, and will normally only occupy the disk space required for the contents of their file system.
This is only true once the disk image has been mounted for the first time after it has been created, mounted and unmounted. To see this, create a test read-write disk image (for example, using Disk Utility) of 10 GB size. Then unmount it, and use the Finder’s Get Info command to inspect its size on disk. That will be 10 GB. Then mount the disk image again, pause a couple of seconds, unmount it, and Get Info will show its size on disk is now much smaller.
As I’ll explain in detail in a later article, this is because each time that disk image is mounted, if its internal file system is HFS+ or APFS, its contents will be trimmed, and saved to disk in sparse file format, which omits all its empty space. This only applies to read-write disk images when they’re stored in APFS file systems; copy them to HFS+ and they’ll explode to full size, as HFS+ doesn’t support the sparse file format.
Considering just the two leading types, empty sizes for a 100 GB disk image are:
a sparse bundle is 35.4 MB empty, 53.3 GB when containing about 51 GB files, stored across 6,359 band files.
a read-write disk image (UDRW) shrinks to 333.8 MB once stored as an empty sparse file, 53.6 GB when containing about 51 GB files, in a single file container.
Performance
Some types of disk image perform poorly, and can be very slow to write to. Recent versions of macOS have brought improvements, although some options such as encryption can still impair performance significantly. For the two leading types, when their container is stored on the internal SSD of an Apple silicon Mac, with native read and write speeds of around 6-7 GB/s:
an initially empty unencrypted sparse bundle reads at 5.1 GB/s, and writes at 4.8 GB/s.
an initially empty unencrypted read-write disk image (UDRW) reads at 5.3 GB/s, and writes at only 1 GB/s.
Tests were performed using my utility Stibium, across a range of 160 files of 2 MB to 2 GB size in randomised order, with macOS 15.0.1.
Key points
Disk images consist of a file or bundle containing one or more file systems; the container and its contents are distinct.
To access the contents of a disk image, the container is first attached, then the file system(s) within it are mounted. In the Finder, those two processes appear as a single action.
Encrypted disk images encrypt the container, and don’t necessarily contain encrypted file systems.
Disk images come in many different types, and can contain different file systems.
Sparse bundles have a file and folder structure inside their bundle folder, with their data saved in band files; all other disk images are single files.
Sparse bundles grow and shrink according to the size of files stored within them.
In recent macOS, and on APFS, read-write disk images (UDRW) are stored in APFS sparse file format, enabling them to grow and shrink as well.
In recent macOS, unencrypted sparse bundles have read and write performance close to that of the disk they’re stored on. Read-write disk images read at similar speeds, but write more slowly, at about 20% of read speed.
Overnight European time, Apple released a small but urgent update to macOS Sequoia, bringing it to version 15.0.1. Although there are no updates to Sonoma or Ventura, there are updated versions of Safari for both.
This Sequoia update is about 1.42 GB for Apple silicon Macs, and around 500 MB for Intel models. It doesn’t bring any firmware updates, but Safari is updated to version 18.0.1 (20619.1.26.31.7). While Apple has released security notes for the concomitant updates for iOS and iPadOS, there are none reported for macOS.
Although Apple remains tight-lipped about exactly what has been fixed in this update, it does admit to fixing a bug in Messages that could crash the app in unusual circumstances, and to improving “compatibility with third-party security software”. It’s assumed the latter refers to the network problems that have been widely reported.
Changes seen in bundled apps include a single build increment in Messages, and Passwords with a new version of 1.0.1. Apart from those and Safari, the only other bundled app to see any change is Photos, with a small build increment. This suggests that there are undisclosed improvements to the new Passwords app.
Significant changes seen in /System/Library include:
Dock, small build increment
CFNetwork and Network frameworks, build increments
If you’ve already upgraded to macOS Sequoia, you’ll probably be well aware of the changes it has brought in updating the data used by XProtect. Although it’s now designed to get those updates from iCloud, so far most have been delivered through the traditional Software Update route, and have had to be installed in their new location either manually in Terminal, or by waiting for the new system to get round to it. So is it worth this extra hassle: how has the new XProtect improved?
We’ve now seen two updates only for Sequoia’s XProtect, one for all Macs, and one only for older macOS. Although still early days, the differences in the XProtect bundle have been obvious. Both versions 5273 and 5275, which were only released for macOS 15, contained large additions to their detection rules, which vanished just as suddenly in 5276, the latest version common to all versions of macOS. To see what has changed, we must examine their XProtect.yara files, the only one in its bundle that has seen any changes.
YARA rules
XProtect performs static checks on code before it’s run, to assess whether there’s evidence that it’s malicious. Its tests are based on YARA rules, explained in detail here. Each rule sets out conditions that must be met for XProtect to conclude that a file is known to be malicious. For example, the rule used by XProtect to detect the Eicar test, a standard non-malicious sample used in testing, is: rule EICAR
{
meta:
description = "OSX.eicar.com.i"
xprotect_rule = true
condition:
filesize <= 100000000 and hash.sha1(0, filesize) == "3395856ce81f2b7382dee72602f798b642f14140"
}
Its meta section names the detection and states that it’s an XProtect rule. The condition to be met for this rule requires a file size of less than 100 MB and the SHA1 hash of the whole file, from its start (0) to the end (filesize), matches the hash given. Some rules have far more complicated conditions, requiring combinations of tests.
Changing rules
New rules that have only appeared in YARA files used by Sequoia have followed a different pattern. Here’s an excerpt from an example: rule XProtect_MACOS_DOLLITLE_CT
{
meta:
description = "MACOS.DOLITTLE.CT"
uuid = "[UUID]"
condition:
hash.sha256(0, filesize) == "[SHA256 hash]" or
…
}
where [UUID] is the rule’s unique identifier, and [SHA256 hash] is the hash value for that part of the condition.
For the first time this rule’s metadata includes a UUID, and its conditions require the SHA256 hash of the file contents to match one of a number of specific values. In this case, the rule gives only six different hashes, but in the rule for MACOS.SOMA.CT there are 3,124 hashes to match against.
So the new YARA rules tested by Apple using Sequoia’s XProtect don’t yet reveal any evidence of new capabilities. What they do suggest is that it’s capable of handling even larger sets of rules, including single rules testing well over 3,000 file hashes. Over the last year, XProtect’s YARA files have increased considerably in their size and complexity. XProtect.yara from version 2173 a year ago had only 223 rules in just over 3,000 lines of code, while version 5275 for XProtect in Sequoia has more than 350 rules requiring nearly 17,000 lines of code.
XProtect future
It seems most unlikely that Apple will ever update Sonoma or earlier versions of macOS to use comparable code in XProtect to that in Sequoia. We have already seen how it has forked XProtect’s YARA rules to deliver different versions for Sequoia and previous macOS, with the newer XProtect receiving odd-numbered releases, and older ones even-numbered. Although those have been confusing for those users who track security data updates, Apple expects us to leave it to macOS to download and install the right updates promptly.
If the last couple of weeks have been chaotic, I fear that the future will be similar. I’ll continue to do my best to inform you of updates to XProtect’s data, and to help you keep your Macs up to date, whichever version of macOS they’re running.
[Thanks to Arnaud for correcting my original reference to file sizes, rather whole-file hashes.]
Over the last 17 years, Time Machine has backed up many millions of Macs, ranging from the last Power Macs to the latest M3 models, every hour. It has changed greatly over that time. This article uses my free utility T2M2 to explain how Time Machine now makes automatic backups to APFS storage, and how to check that. This account is based primarily on what happens in macOS Sonoma and Sequoia, when they’re backing up automatically every hour to local storage.
T2M2 offers three features to see what’s happening with Time Machine and backups:
Check Time Machine button, to analyse backups made over the last few hours.
Speed button, to view progress reports in the log during a long backup.
Browse log button, to show filtered log extracts in fullest detail.
These are each detailed in T2M2’s Help book. Here I’ll concentrate on the first and last, and defer speed checks to that documentation.
Browse log for detail
In practice, you’re most likely to view T2M2’s analysis using the Check Time Machine button first, but here I’ll walk through the greater detail available in log extracts, to build understanding of the sequence of events in each automatic backup.
To help you see which subsystems are involved in each stage, T2M2 displays their entries in different colours, and you can show or hide each of those.
Automatic backups are called off by the DAS-CTS dispatching mechanism, whose entries are shown in red (DAS) and blue (CTS). They schedule backups so they don’t occur when the Mac is heavily loaded with other tasks, and call a chain of services to start the backup itself. Dispatching is now reliable over long time periods, but can be delayed or become irregular in some situations. Inspecting those DAS-CTS entries usually reveals the cause.
From there on, most informative log entries are made by Time Machine itself.
Preparations are to:
Find each destination backup storage, decide whether any rotation scheme applies, so determine the destination for this backup.
Check write performance to the backup destination.
Find the machine store on the destination.
Determine which local snapshots should be removed, and delete them.
Create local snapshot(s) as ‘stable’ snapshot(s), and mount them.
Mount previous local snapshot(s) as ‘reference’ snapshot(s).
Determine how to compute what needs to be backed up from each source. This should normally use FSEvents to build the EventDatabase.
Scan the volumes to be backed up to determine what needs to be backed up.
Estimate the total size of the new backup to be created.
Once backing up starts, entries cover:
Copying designated items to the destination.
Posting periodic progress reports during longer backups.
When that’s complete, closing stages are to:
Report details of the backup just completed.
Set local snapshots ready for the next backup, with the ‘stable’ snapshot(s) marked as ‘reference’, and unmount local snapshots.
Delete working folder used during the previous backup as ‘incomplete’.
Create the destination backup snapshot.
Delete any old backups due for removal.
Report backup success or other outcome.
They’re summarised in this diagram. Although derived from Sonoma, Sequoia brings no substantial change.
T2M2 analyses all those log entries to produce a summary of how Time Machine has performed over the last few hours. That is broken down into sections as follows.
Backup destination. This is given with the free space currently available on that volume, followed by the results of write speed measurements made before each backup starts. Backing up 1 volumes to ThunderBay2 (/dev/disk10s1,TMBackupOptions(rawValue: 257)): /Volumes/ThunderBay2
Current free space on backup volumes:
✅ /Volumes/ThunderBay2 = 1.67 TB
Destination IO performance measured:
Wrote 1 50 MB file at 286.74 MB/s to "/Volumes/ThunderBay2" in 0.174 seconds
Concurrently wrote 500 4 KB files at 23.17 MB/s to "/Volumes/ThunderBay2" in 0.088 seconds
Backup summary. This should be self-evident. Started 4 auto backup cycles, and 0 manual backups;
completed 4 volume backups successfully,
last backup completed successfully 53.4 minutes ago,
Times taken for each auto backup were 0.3, 0.3, 0.2, 0.2 minutes,
intervals between the start of each auto backup were 61.4, 60.2, 60.3 minutes.
Backups created and deleted (or ‘thinned’). Created 4 new backups
Thinned:
Thinning 1 backups using age-based thinning, expected free space: 1.67 TB actual free space: 1.67 TB trigger 50 GB thin 83.33 GB dates: (
"2024-10-01-043437")
Local snapshots created and deleted. Created 4 new snapshots, and deleted 4 old snapshots.
How the items to be backed up were determined. This shows which methods Time Machine used to work out which items needed to backed up, and should normally be FSEvents, once a first full backup has been made. Of 4 volume backups:
0 were full first backups,
0 were deep scans,
4 used FSEvents,
0 used snapshot diffs,
0 used consistency scans,
0 used cached events.
Backup results. For each completed backup, Time Machine’s report on how many items were added, and their size. Finished copying from volume "External1"
1 Total Items Added (l: Zero KB p: Zero KB)
3 Total Items Propagated (shallow) (l: Zero KB p: Zero KB)
403274 Total Items Propagated (recursive) (l: 224.93 GB p: 219.31 GB)
403275 Total Items in Backup (l: 224.93 GB p: 219.31 GB)
1 Directories Copied (l: Zero KB p: Zero KB)
2 Files Move Skipped (l: Zero KB p: Zero KB) | 2 items propagated (l: 6 KB p: 12 KB)
1 Directories Move Skipped (l: Zero KB p: Zero KB) | 403269 items propagated (l: 224.93 GB p: 219.31 GB)
Error messages. ✅ No error messages found.
iCloud Drive and pinning
Backing up the contents of iCloud Drive is a longstanding problem for Time Machine, if Optimise Mac Storage is enabled. Those files in iCloud Drive that are stored locally should be backed up, but any that have been evicted from local storage could only be included if they were to be downloaded prior to the backup starting.
In Sonoma and earlier, the only way to ensure that an evicted file in iCloud Drive is included in a Time Machine backup is to manually download it. Sequoia now lets you ‘pin’ individual files and whole folders so that they aren’t evicted, so will always be included in Time Machine backups. This is explained here.
Discrepancies and glitches
T2M2 tries to make sense from log entries that don’t always behave as expected. As backups can be complex, there are situations when T2M2 may report something that doesn’t quite add up. This most commonly occurs when there have been manual backups, or a third-party app has been used to control the scheduling and dispatch of backups. If you do see figures that don’t appear quite right, don’t assume that there’s something wrong with Time Machine or your backups. Generally, these glitches disappear from later automatic backups, so you might like to leave it for a few hours before checking Time Machine again to see if the problem has persisted.
As I hinted on Monday, I’ve been updating my free iCloud Drive utility Cirrus to work with pinning files and folders in macOS Sequoia. It’s only when delving into the depths of iCloud Drive and pinning that you realise how bizarre it is, and how its interface can be intensely frustrating. This article starts with a brief practical demonstration of the oddities of this alien interface before trying to explain it, and ends with the new version of Cirrus.
Pinning frustration
To perform this brief demonstration, all you need is iCloud Drive with Optimise Mac Storage enabled, running in macOS Sequoia. Although it’s entirely non-destructive, you might like to try this using scrap files that you can trash in frustration at the end.
Create a new folder in iCloud Drive, give it a suitable name, and copy 12-18 files into it. Wait until they have all synced up to iCloud, then select just one of them and open the Finder’s contextual menu using Control-click, noting the new command Keep Downloaded that would pin that file to ensure it isn’t evicted from local storage.
Now press Command-A to select all those files in that folder, and open the contextual menu again. Notice that both the Keep Downloaded and Remove Download commands are now absent from the menu, and the only option offered is Download Now, even though although all those files are still downloaded.
Now select just a few of those files, ten or less, and the contextual menu offers Keep Downloaded and Remove Download commands again.
Rather than pinning these files piecemeal, pin their parent folder instead, using the contextual menu. A second or so after the folder shows the pinned icon, it’s shown against all the files inside that folder. So pinning a folder is simpler than trying to pin the files inside it, at least until you want to unpin any of them: select one of the files inside the pinned folder, open the contextual menu, and you’ll see there’s nothing there about Keep Downloaded, just another command to Show Downloaded Folder instead. Use that, and the Finder will then select the pinned folder for you (note, if you try this in Files in iOS, it doesn’t always work!).
The lesson is that, before you can unpin any of the files in that folder, you have to unpin the folder, which in turn unpins every file (and folder) within that folder, leaving you to manually pin the files you do want to pin, in batches of no more than ten at a time.
If you want further fun, with the parent folder still pinned, create a new folder inside that, and copy a file into that enclosed folder. You’ll see that, whatever you might want, everything that goes into that pinned folder is also pinned now, but can only be unpinned by unpinning everything inside the folder.
Pinning files and folders
The reason for this bizarre and annoying interface is the way that pinning is implemented.
When you pin files individually or in groups of up to ten, each file gains its own pinning extended attribute, of com.apple.fileprovider.pinned. But when you pin a folder, only that folder gains the extended attribute, none of the files or folders within it. The whole folder and the paths within it are designated as being pinned. And, as far as I can tell from the absence of any better information in Apple’s missing documentation, there’s no single method to determine whether a file in iCloud Drive is pinned.
Instead, you have to both
look for the extended attribute attached to the file, and
check all the folders in its path to determine if any of them has the extended attribute, which would then pin everything in their path.
Apple doesn’t document any file or URL attribute that can be used to determine whether a file or folder is pinned.
Cirrus 1.15
I had hoped that this new version of Cirrus, my free utility for exploring, working with and testing iCloud Drive, would be able to identify individual files and folders that are pinned in Sequoia. Because of the lack of any attribute to help this, I have temporarily abandoned my attempts.
What Cirrus now does, though, is performs deep scans of folders in iCloud Drive and reports which files and folders are pinned, giving individual file sizes, and a total file size and number of pinned files. This enables you to check where all your pinned files and folders are, and the pinning burden of iCloud Drive.
If you use pinning much, this is surely essential housekeeping.
Cirrus version 1.15 requires Big Sur 11.5, but its new feature requires macOS 15 Sequoia anyway, and is available from here: cirrus115
from Downloads above, from its Product Page, and through its auto-update mechanism. I hope it makes your pinning more painless.
Apple has just released an update to XProtect for all supported versions of macOS, bringing it to version 5276. As usual, Apple doesn’t release information about what security issues this update might add or change.
Relative to the last version released for Sequoia (5275), this version removes all the new-style rules that had been added to that and 5273. Relative to the general release version 5274, and 5275, it adds one new rule for MACOS.PIRRIT.BM.OBF.
You can check whether this update has been installed by opening System Information via About This Mac, and selecting the Installations item under Software.
A full listing of security data file versions is given by SilentKnight, LockRattler and SystHist for El Capitan to Sequoia available from their product page. If your Mac hasn’t yet installed this update, you can force it using SilentKnight, LockRattler, or at the command line.
If you want to install this as a named update in SilentKnight, its label is XProtectPlistConfigData_10_15-5276.
For Sequoia only: so far, I have seen no sign of this update in iCloud, which still returns an XProtect version of 5272. If you download and install it using Software Update, softwareupdate or SilentKnight, then you need to update the primary XProtect bundle in Terminal using the command sudo xprotect update
then entering your admin password.
I have updated the reference pages here which are accessed directly from LockRattler 4.2 and later using its Check blog button.
When Apple introduced notarization six years ago, it warned us that it would eventually become obligatory for developers, and reassured us that we would still be able to run our own apps that aren’t notarized. macOS Sequoia is the first version of macOS to expect apps delivered from outside the App Store to be notarized, but still to allow us to run apps that aren’t. This article explains how that’s accomplished, and what you should do to work within these new rules.
Sequoia’s rules
When an app is being launched for the first time on that Mac, if it has been put into quarantine with a quarantine extended attribute, Gatekeeper will check whether it has been notarized. If it has, then its launch will progress to further checks such as those of XProtect. If it hasn’t been notarized, then macOS will warn you of that, and halt its launch.
If you want to launch the app despite that warning, open Privacy & Security settings, where you can click the button to Open Anyway.
You will then be warned again, and given the advice “don’t open this unless you are certain it is from a trustworthy source.”
Clicking Open Anyway then takes you to the third dialog, where you’ll need to authenticate to launch the app.
Why all the dialogs?
The old Finder bypass for Gatekeeper was simple: use the contextual menu to Open the app, and there’s a single dialog to get through. That’s so simple that it has become widely used by malware.
These are the helpful instructions provided by one of the most widely encountered malicious apps, Atomic Stealer, when you mount its disk image. The invitation is hard to refuse unless you know the trouble it will bring. Detailing the new bypass system is going to prove a real challenge to those intent on delivering us their malicious apps.
How to run your own apps
The best way to avoid getting embroiled in this process is to build your own apps locally. All modern build systems such as Xcode and its tools now apply an ad-hoc signature to the app or command tool, but their products aren’t put into quarantine. Starting with the source code gives you the opportunity to satisfy yourself that the code hasn’t been tampered with, or is malicious. You may also be able to verify checksums or hashes to provide additional reassurance.
Using pre-built apps and tools brings significant risk. Supply-chain attacks are increasingly common, so even if you’re prepared to trust the source of the app, they may be unaware that it has been tampered with. Remember that, in security terms, there’s no robust way to tell the difference between a benign app and malware unless it has been notarized.
How to move apps between Macs
One of AirDrop’s annoyances is that everything transferred so conveniently also acquires a quarantine extended attribute, so putting it through full Gatekeeper first run checks on notarization. This isn’t just done to annoy, but allows for the fact that AirDropped apps can come from untrusted sources.
If, instead of using AirDrop, you use local file sharing, no quarantine extended attribute is added, and your unnotarized app won’t be put through a full first run check, so you won’t have to suffer the dance of the three dialogs before it will launch.
If you have access to a Developer ID and need to distribute an app more widely, you can notarize other developers’ apps. Apple has made it clear that submission of an app for notarization doesn’t have to be performed by its developer, the owner of the certificate used to sign the app with, but anyone with a current developer account with Apple can do so provided they follow the correct process. This is most appropriate for those in enterprise and organisations who need to use third-party products that aren’t yet notarized by their developer.
Tricks to avoid
You can of course strip the quarantine extended attribute before trying to run an app for the first time. This is a potentially dangerous workaround, as it bypasses first run checks that could spot malware. For those of us who often transfer our own apps for testing using AirDrop, it’s easy to drop them into a folder, Zip that, then on the recipient Mac strip the quarantine xattr before unZipping the archive. My utility Cormorant can make that even simpler. If you were to opt to do that, you must be absolutely certain of the provenance of the app you’re transferring.
For those who have test systems that frequently need to run unnotarized apps, another potential solution has been to disable Gatekeeper on them. Perhaps anticipating a move to do that, Apple has changed how this can be done, making it a two-step procedure, in which you first allow the disabling of checks, then in Privacy & Security settings control how that’s implemented. Apple encourages those who need to do this to use installed profiles instead, or handle it through MDM payloads. This has been explored in detail by Brandon Dalton in his account of these changes.
Is notarization worth it?
In terms of risk alone, only ever running code on your Mac that has been signed by Apple (all macOS code, all App Store apps) or has been notarized by Apple, is almost the safest policy (after running only those apps bundled with macOS). In the early years of notarization, a handful of malicious apps did manage to get notarized for a brief time, but it has been a good while since the last of those. The highest risk strategy is to ignore notarization completely, as it allows all malware to run freely.
There has been a lot of misinformation as to how notarization does or doesn’t work. It’s a key part of Apple’s switch from relying on the chain of trust in code signatures to assessing executable code using its CDHashes. Every notarized app is associated with CDHashes that can verify the integrity of its code, and can be compared against those obtained during the notarization process. This allows Apple to divide code into one of three classes:
known good, gathered from notarization,
known bad, discovered in malicious code in the wild,
unknown, found in code that hasn’t been notarized, or found in known malware.
If all the code that your Mac runs is in the first category, or signed by Apple, then it’s at lowest risk. What you should aim for is to ensure that any code in the third category can’t turn out to be malicious.
Login
