Normal view

There are new articles available, click to refresh the page.
Today — 17 September 2024Main stream

Apple has released macOS 15.0 Sequoia and security updates to 14.7 and 13.7

By: hoakley
17 September 2024 at 01:13

As promised last week, Apple has released the upgrade to macOS 15.0 Sequoia, together with security updates to bring Sonoma to version 14.7, and Ventura to 13.7. There should also be Safari updates to accompany the latter two.

The Sequoia update is around 6.6 GB for Apple silicon Macs, and 14.7 is around 1.6 GB. For Intel Macs, 15.0 is around 4.9 GB as an ‘update’, and 14.7 is around 860 MB.

Security release notes for Sequoia list around 77 vulnerabilities addressed, including two in the kernel, none of which Apple is aware may have been exploited in the wild. Release notes list 36 vulnerabilities addressed in Sonoma 14.7 here, and there are 30 listed for Ventura 13.7 here.

iBoot firmware is updated to version 11881.1.1, Intel T2 firmware to version 2069.0.0.0.0 (iBridge 22.16.10353.0.0,0), and Safari to 18.0 (20619.1.26.31.6).

After completing the upgrade to 15.0, you are likely to see that the installed XProtect version is 0, in other words that there is no XProtect data. You can leave your Mac to automatically download the required data from iCloud, or manually force it using the command
sudo xprotect update
then entering your admin password. That will normally ‘activate’ the XProtect data previously installed, and set the version to 5272, although that will then need to be updated to 5273 separately. Don’t be surprised if you end up repeating the trip to Terminal to get this to work.

Last updated 1900 GMT 16 September 2024.

Yesterday — 16 September 2024Main stream

Looking ahead to Sequoia’s updates

By: hoakley
16 September 2024 at 14:30

Later today, Apple is expected to release macOS Sequoia 15.0. For those interested in planning their immediate or delayed upgrade, these are my forecast dates for its minor versions over the coming year. Like all the best weather forecasts, this is most accurate for the next 5 days, and those for further into the future are likely to be decreasingly reliable.

Minor version release dates for Sonoma have been broadly similar to those of others since Big Sur:

  • 14.0 – 26 September,
  • 14.1 – 25 October,
  • 14.2 – 11 December,
  • 14.3 – 22 January,
  • 14.4 – 07 March,
  • 14.5 – 13 May,
  • 14.6 – 29 July,
  • 14.7 – 16 September.

Ventura differed mostly because it had a later start date to its cycle, in October, resulting in the delay of 13.1 until December. Subsequent versions thus trailed Sonoma by one, for example with 13.5 on 24 July, against 14.6 on 29 July. Although Apple is believed to have some flexibility in the release dates for minor updates, the timetable for the cycle appears to be fixed well in advance, and is probably already at least pencilled in for Sequoia.

Most minor updates bring new versions of firmware, the kernel and key kernel extensions such as APFS. In between those may be patch updates to fix serious bugs or security vulnerabilities that can’t wait for the next minor version, such as 14.3.1 on 8 February, two weeks after 14.3 and a month before 14.4.

According to Apple’s release notes, the current release candidate for 15.0 has no significant bugs that remain unfixed, and we hope that remains the case.

15.1: October 2024

Apple has already announced that this first ‘minor’ update will bring its AI features, including most significantly Writing Tools. Although those have been in beta-testing for almost as long as 15.0, in terms of changes, the step from 15.0 will in many ways be greater than that from 14.6 to 15.0. However, that only applies to Apple silicon Macs that support AI.

For all Macs, this is likely to bring fixes for some more substantial bugs, although because of the short interval between 15.0 and 15.1, few are likely to be addressed until 15.2.

This update is likely to coincide with new Mac products launched at an as-yet unannounced Mac event in October, where Apple is expected to promote its new M4 Macs as being ‘made for AI’, much in the way that it did last week with the iPhone 16 range.

15.2: December 2024

Turnaround time fixing even straightforward high priority bugs makes it likely that most in 15.0 will be addressed not in 15.1 but 15.2, before Christmas. This will also catch the first fixes and any additional enhancements required by AI, so may well be one of the more substantial updates this cycle. The aim is to give engineering teams a chance to catch up with the vacation without leaving too much to await their return in the New Year.

15.3: January 2025

This update is largely constrained by the effects of the Christmas vacation, but should enable most issues arising in 15.0 and 15.1 to be fixed, leaving Sequoia running sweetly.

15.4: March 2025

This is the major mid-cycle update, that is most likely to contain new and enhanced features, often making it the largest update of the cycle. Apple also seems to use this to introduce initial versions of new features intended to become fully functional before the end of the cycle. One example of this was XProtect Remediator, released on 14 March 2022 in Monterey 12.3, but not really functional until June that year.

Unfortunately, these enhancements can also cause problems, and this update in March has a track record of sporadic more serious bugs, including the occasional kernel panic.

15.5: May 2025

A month or so before the first beta-release of the next major version of macOS, this normally aims to fix as many remaining bugs as possible, and progress any enhancements introduced in the previous update. If you’ve reported a bug before April, then if it’s going to be fixed in this cycle, this is the most likely time; any new bugs reported after this update are most likely to be carried over to the next major release.

15.6: July 2025

This really is the last chance for fixes and feature-tweaks before the next major version is released in September. If all is working out well, this should be the most stable and bug-free release, although in some years late changes have turned this update into a nightmare, and Sonoma required a patch update in early August to address those.

When best to upgrade?

If third-party software, hardware and other compatibility requirements don’t apply, there’s no way to predict which is the best version to choose as an upgrade from previous macOS. Every version contains bugs, some of them may be serious, others may be infuriating and intrude into your workflows. But those aren’t predictable. If you’re unsure, wait a few days after a minor update, or even 15.0, check around with others, and decide then. If you’re really cautious and have an Apple silicon Mac, I suggest you might like to consider upgrading a week or two after the release of 15.1, by which time most of any major issues with 15.0 and AI should have come to the surface.

For myself, I already have my designated beta-testing Mac, a MacBook Pro M3 Pro, running 15.1 beta, and my other three Macs (iMac Pro, Mac Studio M1 Max and MacBook Pro 16-inch 2019) will all be running 15.0 by midnight tonight, I hope. I’ll let you know how I get on.

Before yesterdayMain stream

Updating macOS with an Installer and in Recovery

By: hoakley
5 September 2024 at 14:30

With macOS Sequoia fast approaching from the horizon comes the question as to how to upgrade and update, whether to Sequoia or one of its recent predecessors. If you’re happy to go with what Software Update offers, then that’s usually simplest and most efficient. This article considers what you should do if you want something different, from updating to any previous version, to using a single installer to update several different Macs.

Procedures given here should work with all versions of macOS from Monterey onwards. They may work too with Big Sur, but its installers weren’t always as reliable, so you should there be well-prepared to have to migrate from a backup in case the installation creates a fresh, empty Data volume instead of firmlinking up to your existing one.

Which installer?

As Apple discontinued standalone updater packages when it introduced Big Sur, the choice now is between downloading the full Installer app, and performing the process in Recovery mode. The latter severely limits your choice to what it’s prepared to offer, so you’re almost certainly going to need to obtain the full Installer for the version of macOS you want. Rather than use the Installer app provided in the App Store, download the Installer package from the links given by Mr. Macintosh. Those provide a package that’s easier to store and move around, unlike the Installer app itself. It will typically be a little over 13.5 GB, and works on both Intel and Apple silicon Macs.

Standard procedure

As with any update or upgrade, first ensure you have a full recent backup before starting. If anything does go wrong during the procedure you’ll then be able to perform a fresh install and migrate from that backup.

Unless you want to install everything afresh and migrate from your backup, don’t try erasing either your System or Data volume. You’d have to do that in Recovery mode anyway, limiting your options as to which version of macOS you can install unless you create a bootable installer first.

Double-click the installer package to launch it in the Installer utility. The default is to save the Installer app to your current Applications folder, which should work fine as long as you remember to delete it once you’ve finished. Once complete, launch that Installer app and follow its instructions.

sininstall2

When macOS restarts at the end of the process, check the version now running, confirm that your Data volume has survived intact, and run SilentKnight to ensure that all security data files are up-to-date.

Recovery

Intel Macs have a slight advantage when it comes to installing macOS in Recovery mode, as depending on the keys held during startup, you should be able to coax a choice of versions out of an Intel system. Unless you simply want to install or update to the current version, though, you’ll probably want to avoid doing so in Recovery.

sininstall3

There’s another good reason for not using Recovery, in that delivery of installers to Macs running in Recovery can be painfully slow, and you may well be in for a longer wait than if you downloaded the Installer direct.

However, if you want to erase the current boot volume group on your Mac’s internal storage so you can install a fresh copy of macOS and restore the contents of its Data volume from backups, Recovery is normally the best place to do that. Apple works through the process for Intel Macs, and Apple silicon models. The key step is to select the Macintosh HD boot volume group and click on the Erase tool to perform Erase Volume Group.

When the SSV was first introduced in Big Sur, there were many problems resulting from erasing just one volume in the boot volume group. If that happened to be the System volume, when macOS was installed it created a new firmlinked Data volume, leaving the existing Data volume as an orphan. That was usually done in a misguided attempt to have a fresh install of the System volume and SSV while keeping the existing contents of the Data volume, but doesn’t do that. Every installation of the SSV in any given version of macOS since Big Sur is identical, so it isn’t necessary to erase it, but simply to install or update macOS.

Bootable installer disk

Another traditional way to install macOS is using a bootable installer disk, normally a USB ‘thumb’ drive, although you can also create a small HFS+ volume for the purpose on an external SSD. Apple provides detailed instructions for doing this using a range of versions of macOS.

In many cases, installing a version of macOS older than the one that’s currently running requires this, as old Installers usually fail to run in newer macOS. Unfortunately, on Apple silicon Macs, this isn’t the powerful tool that it once was, as the Mac doesn’t boot fully from the external disk, and as a result it has no role in dealing with problems with internal storage.

Virtual Machines on Apple silicon

Installer apps and Recovery installs both work fine in virtual machines running on Apple silicon hosts. However, there’s one special circumstance you need to beware of. One of the major new features in virtualisation in Sequoia is support for iCloud and some other services dependent on Apple ID. If you want to use those, then the VM must be created new in Sequoia, using a Sequoia IPSW image. You can’t update or upgrade an existing VM from a previous version of macOS and use iCloud services in it.

Summary

  • If you can, use Software Update to update or upgrade macOS, as it minimises download size and is simplest.
  • If you want to perform a different update, or run one installer on several Macs, download and use the appropriate Installer package.
  • If you want to erase the existing system including all your data, use Recovery mode to erase the whole volume group, then install macOS and migrate from your backup.
  • Never erase only your Mac’s System volume, as that will orphan its current Data volume.
  • If you want to downgrade to an older version of macOS, you’ll probably need to do so from a bootable installer disk.
  • If you want a VM to use iCloud, then create a fresh VM using a Sequoia IPSW, as an upgraded VM can’t access iCloud.

Where has Safari gone, and why are macOS updates larger for Apple silicon?

By: hoakley
4 September 2024 at 14:30

My previous explanation of how recent versions of macOS merge their System and Data volumes into what appears to be a single volume, omitted a third component, including Safari. Look in the System/Applications folder where all the bundled apps are stored on the SSV, and there’s no Safari to be seen, yet it appears in the top-level Applications folder. This article explains how that now works using cryptexes, and how they differ between Intel and Apple silicon Macs.

Finding Safari

As the modern boot volume group evolved through Catalina to Big Sur, Safari and its supporting frameworks were stored in the Data volume. That stopped with the arrival of Ventura, and they’re now stored in the third components that complete the modern boot volume group. You can see when files are stored on a different volume using my free app Precize to reveal their full paths. Use that to examine three apps from the merged Applications folder, and you’ll understand what I mean:

  • Chess.app has a path of /System/Applications/Chess.app demonstrating that it’s one of the apps bundled in the SSV, where almost all of the System folder is stored.
  • Cirrus.app, like any other app you have installed, has a path of /Applications/Cirrus.app, making it clear that it’s stored on the writable Data volume.
  • Safari.app has the weird path of /System/Volumes/Preboot/Cryptexes/App/System/Applications/Safari.app that demands further explanation.

Note that the Finder’s Get Info dialogs aren’t as truthful, and don’t tell the full story.

Their volfs paths are also worth noting. On my Intel Mac, they are:

  • Chess.app is at /.vol/16777240/1152921500311883667; because all macOS 14.6.1 SSVs are identical, your Chess.app should have the same inode number too.
  • Cirrus.app is at /.vol/16777240/461665725
  • Safari.app is at /.vol/16777238/993517

The first two follow a familiar pattern you’ll see throughout the System and Data volumes: their volume ID 16777240 is common to both, and that assigned to the merged volumes, but their inode numbers are wildly different. Huge numbers like 1152921500311883667 come from the SSV, while smaller ones like 461665725 are from the Data volume. Then there’s a slightly lower volume ID of 16777238 and a small inode number of 993517 for Safari, demonstrating that it’s somewhere altogether different: that’s a cryptex, a cryptographically protected disk image with an interesting history.

Why a cryptex?

When the modern boot volume group was being designed and developed, it took into account Safari’s special needs by making it the only bundled app to be stored in the Data volume. This enables it to be updated without having to go through the whole process of building a new SSV, allowing Apple to deliver urgent security patches to Safari and its underlying WebKit and other frameworks. There could also have been political considerations in separating Apple’s bundled browser from the other apps included in macOS.

This changed in Ventura in the autumn/fall of 2022, when Apple applied technology it had originally developed for its customised iPhone, the Security Research Device, dubbed the cryptex, a name formed as a portmanteau for CRYPTographically sealed EXtension. This offers two advantages:

  • Safari, its supporting frameworks, and other components of macOS that Apple prefers not to build into the SSV, can be delivered in cryptexes. As I’ll explain later, this also enables tailoring of macOS to platform.
  • Some urgent security patches could be delivered in cryptexes, making them faster to release and simpler to install in a Rapid Security Response (RSR).

Since then, RSRs seem to have had their day, and appear to have fallen from favour. But, as a means of delivering Safari and other more changeable components of macOS, cryptexes have proved their worth.

How a cryptex works

Although a cryptex is at heart a read-only disk image that is mounted during the boot process, it has two properties of particular importance:

  • Its contents are cryptographically verified, in much the same way that the contents of the SSV are, using hashes of its entire contents.
  • Its internal file system is grafted into the root file system when it’s mounted, rather than being mounted as a separate volume.

APFSCryptexMount1

Mounting a cryptex starts with validation of the payload and its manifest. It then undergoes a sequence of processes similar to the mounting of an APFS volume, with a checkpoint search to establish stable checkpoint indices, and a check to discover whether there’s anything to recover, which seems unlikely. The graft is then performed in a series of opaque steps, with root hash authentication and validation. The object ID is found, and the graft completed.

Once this has been completed for each of the standard cryptexes and any installed RSRs, the contents of those are effectively part of the system, as a hybrid of the SSV and cryptexes. In the case of the Safari app, this process effectively places it in the main Applications folder, even though the original app is actually located in the System/Applications folder of the App cryptex in /System/Volumes/Preboot/Cryptexes.

As with the current boot System and Data volumes, grafted cryptexes aren’t unmounted or ungrafted until shutdown.

There are currently three main cryptexes in use, App containing Safari, its frameworks and other supporting files, and OS, with a range of other system items including additional frameworks, and several large dyld shared caches. You’ll also see an Incoming cryptex in /System/Volumes/Preboot/Cryptexes. As they’re outside the SSV, new and replacement cryptexes are installed without rebuilding the SSV, and in some cases don’t even need a soft restart of macOS.

Architecture-specific cryptexes

In addition to providing Safari and its related components, cryptexes also provide useful economy in shared caches, and explain why macOS updates for Apple silicon Macs are invariably larger than those for Intel models.

While the contents of the SSV appear to be identical on both Intel and Apple silicon, thus have a single signature, the two architectures differ in their cryptexes. Those for Apple silicon Macs contain dyld shared caches for both architectures, and a set of aot shared caches, presumably to support Rosetta 2, and amounting to 5.24 GB in total size; those for Intel Macs only contain Intel dyld shared caches of 1.68 GB total size.

Given their sizes, that’s a valuable efficiency both for updates and in storage required, and is the major reason for updates for Apple silicon Macs always being larger than those for Intel. Thankfully, because those shared caches are supplied compressed, the difference in update sizes is much smaller than the 3.56 GB difference when they’re decompressed and installed.

Apple has just released an update to XProtect Remediator

By: hoakley
4 September 2024 at 03:47

Apple has just released an update to XProtect Remediator security software for Catalina or later, bringing it to version 145. The previous version was 142.

Apple doesn’t release information about what security issues this update might add or change. There are no changes in the number or names of its scanning modules, and Bastion rules also remain unchanged.

You can check whether this update has been installed by opening System Information via About This Mac, and selecting the Installations item under Software.

A full listing of security data file versions is given by SilentKnight, LockRattler and SystHist for El Capitan to Sonoma available from their product page. If your Mac has not yet installed these updates, you can force them using SilentKnight, LockRattler, or at the command line.

If you want to install this as a named update in SilentKnight, its label is XProtectPayloads_10_15-145.

I have updated the reference pages here which are accessed directly from LockRattler 4.2 and later using its Check blog button.

I maintain lists of the current versions of security data files for Sonoma on this page, Ventura on this page, Monterey on this page, Big Sur on this page, Catalina on this page, Mojave on this page, High Sierra on this page, Sierra on this page, and El Capitan on this page.

Launching apps in Sonoma 14.6.1: Conclusions

By: hoakley
3 September 2024 at 14:30

Over a series of three articles last week, I pored over thousands of log entries to examine how macOS Sonoma 14.6.1 checks applications it’s launching, under normal full security settings, with reduced security, and for known malware. This article draws together my conclusions from those tests run in virtual machines on an Apple silicon Mac.

Layered security

Like other security functions in macOS, app launch security is built in layers, including checks of

  • code-signing certificates (multiple times);
  • CDHashes, including their consistency, and against Apple’s database for notarized apps, and their revocation;
  • quarantine extended attributes, which normally trigger a user consent dialog, and may result in app translocation;
  • previous launch, in the LaunchServices database;
  • matches with Yara rules in XProtect’s data;
  • user consent to a first launch prompt dialog;
  • launch and other constraints.

Additional data may also be collected and stored in the provenance database that first appeared in Ventura.

Not all checks are performed on every launch of an app. At a minimum, for a notarized app that has been run only recently, these might consist of only local checks against CDHashes and with the app’s existing entry in the LaunchServices database. Checks are also modified by reducing security settings:

  • Disabling Gatekeeper checks doesn’t stop those checks from taking place, but apparently ignores some results, notably those obtained by XProtect. It doesn’t affect checks of CDHashes against Apple’s database.
  • Disabling SIP has more pervasive effects in largely disabling the com.apple.syspolicy sub-system, affecting several layers, although checks of CDHashes against Apple’s database are unaffected.

com.apple.syspolicy

In full security conditions, one sub-system dominates log entries concerning app launch security, com.apple.syspolicy. This is clearest in Gatekeeper and XProtect checks. Although the log entries that follow may appear bewildering, they are the best illustration of this point.

When launching a notarized app that hasn’t previously been run on that Mac and has a quarantine xattr, Gatekeeper and XProtect scans are reported in the following sequence of entries:
com.apple.syspolicy.exec GK process assessment: <private> <-- (<private>, <private>)
com.apple.syspolicy.exec Gatekeeper assessment rooted at: <private>
com.apple.syspolicy.exec Skipping TCC check due to process: 692, 0, 692
com.apple.syspolicy.exec queueing up scan for code: PST: (vuid: 7C5C43BF-A338-4228-B61E-5038F1D93EDB), (objid: 62947), (team: (null)), (id: (null)), (bundle_id: (null))
com.apple.syspolicy.exec starting work for scan for code: PST: (vuid: 7C5C43BF-A338-4228-B61E-5038F1D93EDB), (objid: 62947), (team: (null)), (id: (null)), (bundle_id: (null))
com.apple.syspolicy.exec allowUI is YES, creating codeEval object: PST: (vuid: 7C5C43BF-A338-4228-B61E-5038F1D93EDB), (objid: 62947), (team: (null)), (id: (null)), (bundle_id: (null))
com.apple.syspolicy.exec Adding default exception for team: <private>
com.apple.syspolicy.exec Registered app bundle for protection: PST: (vuid: 7C5C43BF-A338-4228-B61E-5038F1D93EDB), (objid: 62947), (team: QWY4LRW926), (id: (null)), (bundle_id: (null))
com.apple.syspolicy.exec GK performScan: PST: (vuid: 7C5C43BF-A338-4228-B61E-5038F1D93EDB), (objid: 62947), (team: QWY4LRW926), (id: (null)), (bundle_id: (null))
com.apple.xprotect XProtectScan beginAnalysisWithResultsHandler continueOnError is set to 0
com.apple.xprotect XPAssessment performAnalysisOnFileImpl continueOnError set to 0
com.apple.xprotect Xprotect is performing a direct malware and dylib scan: <private>

Those checks later complete in entries such as:
com.apple.syspolicy.exec GK Xprotect results: PST: (vuid: 7C5C43BF-A338-4228-B61E-5038F1D93EDB), (objid: 62947), (team: QWY4LRW926), (id: (null)), (bundle_id: (null)), XPScan: 0,-7676743164328624005,2024-08-26 08:19:01 +0000,(null)
com.apple.syspolicy.exec GK scan complete: PST: (vuid: 7C5C43BF-A338-4228-B61E-5038F1D93EDB), (objid: 62947), (team: QWY4LRW926), (id: (null)), (bundle_id: (null)), 4, 4, 0
com.apple.syspolicy.exec scan finished, waking up any waiters: PST: (vuid: 7C5C43BF-A338-4228-B61E-5038F1D93EDB), (objid: 62947), (team: QWY4LRW926), (id: co.eclecticlight.SystHist), (bundle_id: co.eclecticlight.SystHist)
com.apple.syspolicy.exec App gets first launch prompt because responsibility: <private>, <private>
com.apple.syspolicy.exec GK evaluateScanResult: 0, PST: (vuid: 7C5C43BF-A338-4228-B61E-5038F1D93EDB), (objid: 62947), (team: QWY4LRW926), (id: co.eclecticlight.SystHist), (bundle_id: co.eclecticlight.SystHist), 1, 0, 1, 0, 4, 4, 0
com.apple.syspolicy.exec GK eval - was allowed: 1, show prompt: 1
com.apple.syspolicy.exec Skipping TCC check due to process: 692, 0, 692
com.apple.syspolicy Found console users: <private>
com.apple.syspolicy.exec Prompt shown (5, 0), waiting for response: PST: (vuid: 7C5C43BF-A338-4228-B61E-5038F1D93EDB), (objid: 62947), (team: QWY4LRW926), (id: co.eclecticlight.SystHist), (bundle_id: co.eclecticlight.SystHist)

When SIP has been disabled, there are precious few entries from com.apple.syspolicy or com.apple.syspolicy.exec. Instead, XProtect appears to be left to its own devices, and doesn’t fare well:
com.apple.xprotect XPAssessment performAnalysisOnFileImpl continueOnError set to 0
com.apple.xprotect XprotectService Calling SecAssessmentCreate with URL <private>, context <private>
XprotectService SecTrustEvaluateIfNecessary
com.apple.xprotect XprotectService Bundle is not apple signed
com.apple.xprotect XprotectService Bundle size result: 18388222 (YES)
com.apple.xprotect XprotectService Always scan: YES
com.apple.xprotect XprotectService Starting malware scan for: <private>
kernel XprotectService [697] crossed memory high watermark (15 MB); EXC_RESOURCE
kernel Full corpse enqueued for XprotectService
com.apple.xnu memorystatus kernel kernel EXC_RESOURCE -> XprotectService[697] exceeded mem limit: ActiveSoft 15 MB (non-fatal)
ReportCrash event condition bump 0 -> 1
ReportCrash post-exception thread qos drop 21 -> 17
ReportCrash PID 697 exceeded the memory high watermark; Invoking ReportMemoryException with corpse.

There are no other entries referring to Gatekeeper or those checks. The effects of disabling SIP appear extensive and pervasive throughout several of the layers of app launch security.

CDHashes are central

With the adoption of notarization, apps run in macOS should now fall into one of five categories:

  • signed by Apple, either its own apps or those delivered through its App Store;
  • notarized by Apple, with its CDHashes added to Apple’s database;
  • signed (either with a Developer certificate, or ad hoc) locally, and not distributed over the internet, with its own unique CDHashes;
  • unwanted or malicious, with revoked CDHashes,
  • unrecognised, and potentially malicious.

These emphasise the importance of the online ‘notarization’ checks of CDHashes performed in all circumstances where macOS doesn’t have previous records of saved CDHashes for that code. Their primary purpose isn’t to validate notarization, but to identify code as known good, known bad, or unknown. When Apple’s security engineers identify new malware, its CDHashes can quickly be added to the database as being revoked, so ensuring that all subsequent checks of the same CDHash will be classified as revoked, for malicious code. This is a rapid response that should have no false positives, in which benign code is mistakenly identified as being malicious.

Typically, the checking sequence is reported in the log with:
com.apple.syspolicy looking up ticket: <private>, 2, 1
com.apple.syspolicy cloudkit record fetch: <private>, <private>
com.apple.syspolicy cloudkit request cache info: <private>, max-age=300
com.apple.syspolicy CKTicketStore network reachability: 1, Mon Aug 26 09:15:45 2024
com.apple.syspolicy Inserting ticket: <private>
com.apple.syspolicy completing lookup: <private>, 0

[and so on with further lookups]
and those are among the only entries from com.apple.syspolicy seen when SIP is disabled.

When full security is enabled, those are completed with
com.apple.syspolicy.exec GK evaluateScanResult: 0, PST: (vuid: 7C5C43BF-A338-4228-B61E-5038F1D93EDB), (objid: 62947), (team: QWY4LRW926), (id: co.eclecticlight.SystHist), (bundle_id: co.eclecticlight.SystHist), 1, 0, 1, 0, 4, 4, 0
But when SIP is disabled, those don’t appear, and seem to be substituted by application of Security rule 11 instead.

The downside of CDHash checks is that their false negative rate can be alarmingly high. Change a single bit in the code being hashed, and the hash will amplify that change, and is completely different. Hence the importance of notarization to establish which CDHashes definitely aren’t from malicious code.

One threat to this system occurs when a user mistakenly blocks their Mac from connecting to Apple’s database using CloudKit, for example using a misconfigured software firewall. Without a suitable vulnerability, malicious software shouldn’t be able to use this approach to block a payload from being checked.

I don’t know whether any third-party security products use a similar checking mechanism with their own local or remote CDHash databases, but this appears to be a great advantage to the protection built into macOS.

Performance

Two of the checks performed with full security enabled are dependent on the size of the app being checked. Fully validating an app’s CDHashes against those in its signature or notarization ticket should benefit from hardware acceleration, particularly on Apple silicon, and can be tackled hierarchically. It appears unlikely to result in significant delays to launching an app.

XProtect scans are more likely to be responsible for observable delays in app launch times, though. With the recent growth in the number of Yara rules, and their length, scans performed after an app’s first launch are the most probable cause of large and complex app bundles requiring several seconds before the app can be run.

Summary

I have updated the flow chart I first proposed as a result of observations made of app launches in Sonoma 14.4.1:

launchsonomaapp2

This is also available as a tear-out PDF here: launchsonomaapp2

I welcome any evidence that will refine and improve that, please.

Previous articles

Launching apps in Sonoma 14.6.1: Full security
Launching apps in Sonoma 14.6.1: Reduced security
Launching apps in Sonoma 14.6.1: Known malware
How does Sonoma check an app before launch? (Sonoma 14.4.1)

What is Macintosh HD now?

By: hoakley
2 September 2024 at 14:30

Perhaps you just tried to save a document, only to be told you don’t have sufficient permissions to do so, or attempted to make another change to what’s on your Mac’s internal storage, with similar results. You then select the Macintosh HD disk in the Finder and Get Info. No wonder that didn’t work, as you only have read-only access to that disk. But if you unlock it and try to make any changes to permissions, you see

xpermserror

What’s going on?

Between macOS Mojave, with its single system volume, and Big Sur, the structure of the Mac system or boot volume has changed, with Catalina as an intermediate. Instead of Macintosh HD (or whatever you might have renamed it to) being one volume on your boot disk, it’s now two intertwined and joined together. What you see now as Macintosh HD isn’t even a regular APFS volume, but a read-only snapshot containing the current macOS. No wonder you can’t change it.

Root

Select the boot disk Macintosh HD in the Finder, and it appears to have four visible folders, Applications, Library, System and Users, just like it always did. Press Command-Shift-. to reveal hidden folders and all the usual suspects like bin, opt and usr are still where they should be. That’s the root of the combined System and Data volumes, and what’s shown there is a combination of folders on both volumes, with the top level or root on the Sealed System Volume (SSV).

The contents of those folders are also the result of both volumes being merged together using what Apple terms firmlinks:

  • Applications contains apps installed in your own Applications folder on the Data volume, and those bundled in macOS on the SSV. You can see just the latter in the path System/Applications, where they appear to be duplicated, but aren’t really.
  • Library comes only from the Data volume, and all its contents are on that volume. But inside it, in the path Library/Apple/System/Library are some components that should appear in the main System/Library.
  • System comes only from the SSV, although it has some contents merged into it using firmlinks, such as those folders in Library.
  • Users also comes only from the Data volume, and includes all Home folders for users.

So while the root of Macintosh HD might be in the SSV, much of its contents are on the Data volume, and can be written to, even though the root is a read-only snapshot, thanks to those firmlinks.

Data volume

There are two places that mounted volumes are listed in the Finder: the hidden top-level folder Volumes, where Macintosh HD is just a link back to the root complete with its merged volumes, and in System/Volumes, where what’s shown as Macintosh HD is in fact not the merged volumes, but only the Data volume. You can confirm that by looking at what’s in System/Volumes/Macintosh HD/System, where you only see the parts of the System folder that are stored on the Data volume, and not those stored on the SSV.

What is more confusing there is that System/Volumes/Macintosh HD/Applications is the same merged folder containing both user and bundled apps as in the top-level Applications folder. That’s an artefact resulting from the way that its firmlink works.

But if you open the Get Info dialog on System/Volumes/Macintosh HD, you’ll see the same as with the root Macintosh HD disk, information about the root and not the Data volume.

Mounted in System/Volumes are several other volumes like VM and Preboot, and (depending on whether this is an Intel or Apple silicon Mac) folders such as Recovery and xarts, that you really don’t want to mess with.

Permissions problems

Tackling problems that appear to be the result of incorrect permissions is best done at the lowest folder level. If you’re trying to save a document to the Documents folder inside your Home folder, select that and Get Info on it. Chances are that you are the owner and have both Read & Write permissions as you should. In that case, the problem most likely rests with privacy protection as in Privacy & Security settings. You then suffer Catch-22, as you can only effect changes to those by closing and opening the app, and as you can’t save your document before closing the app, you’re at risk of losing its contents. You may have better luck trying a different folder, creating a new one inside your Home folder, or using the Save As… command instead (which may be revealed by holding the Option key when opening the File menu).

Full layout

In case you’re wondering exactly which folders are merged into the hybrid Macintosh HD ‘volume’, those are shown below in increasing levels of detail, starting with the broad layout.

BootVolGpVentapfs

Then to a simplified version of the full layout.

BigSurIntSimple

Finally, in complete detail.

BigSurIntegrated

Happy navigating!

Advanced SilentKnight: updating macOS and avoiding updates

By: hoakley
30 August 2024 at 14:30

Although we won’t know for sure until Apple releases the upgrade to macOS Sequoia next month, once again it will probably be presented as an update rather than a macOS upgrade. This means that, instead of Software Update downloading a complete Sequoia installer app, if you do choose to upgrade, it will be run through Software Update the same way that it might have updated from 14.5 to 14.6.

Although this is more efficient, resulting in a smaller update and faster completion, it also opens up the possibility of human error: what if you accidentally opt to upgrade, or click on SilentKnight’s Install all updates button? This article explains how you can stop that upgrade from completing, and how you could upgrade using SilentKnight instead of Software Update.

Updating or upgrading macOS

When SilentKnight has completed checking for updates, if there’s a macOS update or upgrade available you can install it if you wish, from within SilentKnight. Although my personal preference is to hand macOS updates over to Software Update in Settings, SilentKnight should also work fine, up to a point.

skupdate1

To test this, I opened a VM running Sonoma 14.6, with XProtect 5270 and XPR 140. When it had found all three updates available, I clicked on the Install all updates button, just as I have always advised you not to! SilentKnight proceeded to download the macOS 14.6.1 update, but once that was complete it failed to install it. It then proceeded to download the XProtect and XPR updates, which it did successfully install on its own.

skupdate2

There was a vague notification that “Some updates could not be installed”, and the VM was left in 14.6, with XProtect and XPR correctly updated.

skupdate3

skupdate4

At that stage, Software Update stated the 14.6.1 update was available, offering a Restart Now button. When I clicked on that, the VM restarted and installed the 14.6.1 update successfully.

SilentKnight doesn’t provide the handy progress indicator that Software Update does, but just turns its busy spinner until the updates have finished. So you may still prefer to install macOS updates using Software Update. However, the end result should be just the same if you let SilentKnight do it, and finish off the installation using Software Update.

Downloading updates

skupdate5

Using another copy of the same VM running Sonoma 14.6 with outdated XProtect and XPR, I set SilentKnight’s settings to download but not install updates, then clicked the Download all updates button.

This left all the updates uninstalled, but there was no sign of them in the standard /Library/Updates folder as documented for softwareupdate. I looked high and low for those updates, but was unable to find them anywhere. I therefore recommend that you don’t use this option until someone has worked out where those downloaded updates are kept.

Unwanted macOS updates

If you click either the Install all updates or Download all updates button and one of the updates is for macOS, that will leave Software Update poised to complete the installation. If you didn’t want to install that macOS update, there is a way that you can now persuade Software Update to forget that it has been downloaded and is waiting, ready to install. This is most useful if you didn’t intend updating macOS, and now want to undo the process.

Shut your Mac down, then start it up in Safe mode. Leave it there for a minute or so, then restart it back into normal mode. Those uninstalled updates should now have been flushed, and Software Update is back to where it started.

Summary

  • You should now be able to install macOS updates using SilentKnight if you wish. When warned that some updates weren’t installed, open Software Update settings and complete the installation there using the Restart Now button.
  • Don’t use SilentKnight’s setting to only download but not install updates, as the downloaded updates can’t be found and used.
  • If you inadvertently click the Install all updates button and want to reverse that for a macOS update, let the download complete, shut down, start up in Safe mode, wait a minute, then restart in normal mode.
  • These apply to Apple silicon Macs, and are untested in Intel Macs, although there’s no reason to believe they should differ there.

Launching apps in Sonoma 14.6.1: Known malware

By: hoakley
29 August 2024 at 14:30

Previous articles in this series described how macOS 14.6.1 security systems check the launch of apps when full security is in force on an Apple silicon Mac, and how those are changed by disabling SIP and Gatekeeper checks. Those have shown how checks are layered in accordance with the Security architecture of macOS, how different layers are invoked according to the status of an app (whether it’s quarantined, notarized, or has been run previously), and how extensive are the effects of disabling SIP. But no account of app security can be complete without examining how it protects against real malware, the aim of this article.

Methods

In these tests, I have again run four variants of the same 14.6.1 VM:

  • Full Security, with SIP and Gatekeeper/XProtect enabled;
  • Full Security, with Gatekeeper/XProtect disabled;
  • Permissive Security, with SIP disabled;
  • Permissive Security, with both SIP and Gatekeeper/XProtect disabled.

Samples of malicious software were obtained from the Objective-See Foundation’s collection. Three were chosen:

  • Atomic Stealer (AMOS, or Soma)
  • Genieo (InstallMac)
  • XCSSET

These were downloaded directly to each of the four VMs, when they were running in isolation in ViableS. Each was then unZipped and the contents moved to the Documents folder to try to ensure that their code wouldn’t be subjected to app translocation. Full log extracts were obtained from the Full Security VM for the first 5 seconds after launching Atomic Stealer and XCSSET; as the Genieo sample only installed its payload and didn’t launch its code, no log record was obtained for that. Log records weren’t obtained for the other three VMs, although the results of running the malicious payloads were observed for comparison against those of the Full Security VM.

Atomic Stealer

zamosshot

This was presented in a disk image that hadn’t been signed by a Developer certificate, and encouraged the user to try to bypass full Gatekeeper checks by opening the malicious payload CardGame.app using the Open command in the Finder’s contextual menu, a common strategy adopted by malware developers. This ruse was spotted early as a security exception with the code -67062, indicating that the disk image was unsigned, and that resulted in the app being translocated in its disk
SecTranslocateCreateSecureDirectoryForURL: created /private/var/folders/s0/[…]/CardGame.app
This appears to be a less usual cause of translocation, although strictly within its rules.

AMFI quickly found a code signature issue, as reported by the kernel
AMFI: '/private/var/folders/s0/[…]/CardGame.app/Contents/MacOS/My Go Application.app' has no CMS blob?
AMFI: '/private/var/folders/s0/[…]/CardGame.app/Contents/MacOS/My Go Application.app': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.

Gatekeeper and XProtect scans followed, and the CDHashes were checked with Apple’s database over CloudKit. This discovered that one of the hashes had been revoked
Notarization daemon found revoked hash: {length = 20, bytes = 0xe430ea6d59a70ac00c1b8552092f4de0bbb80232}
resulting in another security exception, this time of -66992, confirming that this code has been revoked. That check was then repeated with the same result.

Shortly after that, the XProtect scan was completed, finding a match for Atomic Stealer A
GK Xprotect results: PST: (vuid: 11F66D42-5827-3465-A741-F434860C2862), (objid: 20), (team: (null)), (id: (null)), (bundle_id: (null)), XPScan: 11,-7676743164328624005,2024-08-27 07:31:10 +0000,MACOS.SOMA.A
and the decision was made to present the malware warning prompt
present prompt: uid=501, conn=yes, type=Malware, op.ident=2F90B5EF-D483-43C7-BBD1-77E8EABF4D62, info.ident=8D56578B-833F-4629-86F0-4E0A8EDD7D49, info={<private>}
indicating that it’s game over for the CardGame app and its disk image.

This sample of Atomic Stealer was thus detected by two different and independent methods: its CDHash ‘notarization’ check revealing its revocation, and the XProtect scan matching it to the known signature of MACOS.SOMA.A. As the first of those is unaffected by disabling SIP or Gatekeeper, it’s not surprising that the sample was detected and blocked in each of the four VMs.

Genieo

zgenieshot

This was presented in an Installer package as an Intel binary. This claimed to install “Apple software” and triggered a request to download and install Rosetta 2 if that wasn’t already available. The installer appeared to complete without eliciting any warnings, and it’s presumed that the malware would either have been detected later when there was an attempt to launch it, or in an XProtect Remediator scan.

All four VMs behaved identically, and there was no sign of recognition that the software installed might be malicious. This raises questions about the security inherent to Installer packages and whether there might be exploits available using Intel binaries in Rosetta 2, given that it resigns translated executable code.

XCSSET

zxcsshot

This was presented in a bogus app named Xcode. Attempting to run that resulted in its detection, and the invitation to remove it, in this case without it being positively identified.

As this was presented as an app containing unsigned code, that came under suspicion early during its assessment, and it was translocated even though it had been moved from its original location
SecTranslocateCreateSecureDirectoryForURL: created /private/var/folders/s0/[…]/Xcode.app
That appears to have occurred beyond previous rules for translocation.

Gatekeeper and XProtect scans followed, and it was confirmed that the code was unsigned
Error Domain=NSOSStatusErrorDomain Code=-67062
Unsigned code in: PST: (vuid: 7C5C43BF-A338-4228-B61E-5038F1D93EDB), (objid: 81906), (team: (null)), (id: (null)), (bundle_id: (null))

CDHash checks using CloudKit didn’t find a match, and were simply reported as
ticket not available: <private>
Gatekeeper’s scan reported that the app didn’t contain a bundle, but XProtect found no match with current Yara rules. The decision was made to present the malware warning prompt
present prompt: uid=501, conn=yes, type=Malware, op.ident=A66F9ED6-EDE7-48E9-B1F8-74CB77C43C9E, info.ident=39D1FBB5-2620-483B-AD3C-6FC5118A406F, info={<private>}
and the attempt to launch the app was blocked.

As these traits would still be detected with SIP and Gatekeeper disabled, all four VMs blocked the code and displayed the same alert to the user.

Limitations

In reality, it’s common for attacks to consist of the initial download of a small dropper, which in turn downloads the main payload. One of the disadvantages of testing malware samples is that this presentation of the payload can’t be taken into account. Payloads are often downloaded using methods that escape quarantine. Another significant difference is that samples often lack code signatures that may be present in the originals, and may change frequently as Developer certificates are revoked and replaced.

Detection information

There appears to be almost no information on how macOS detects different groups of malicious software. Inevitably, Apple provides none at all, and few in-depth analyses of malware give any details about its presentation, in terms of any signatures used, and whether they or CDHashes have since been revoked by Apple. This is a difficult area, given that many of those who analyse and report on malware work for vendors of security products. There appears to be a valuable role for independent assessment of whether and how detection takes place in macOS, major factors in any risk assessment.

I’d like to express my gratitude to the Objective-See Foundation for collecting and making available its extensive library of malware samples, without which none of these tests would have been possible.

Launching apps in Sonoma 14.6.1: Reduced security

By: hoakley
28 August 2024 at 14:30

In the first of these articles, I examined security aspects of the process of launching various app configurations in macOS Sonoma 14.6.1, on an Apple silicon Mac with full boot security and other security settings. This article moves on to discover how those change when boot security and security settings are reduced. Full details of how this was done are given in the previous article.

To remind you, the apps used were:

  • SystHist – notarized, quarantined, moved from its landing folder to avoid app translocation;
  • SilentKnight – notarized, not quarantined, previously run;
  • Sparsity – notarized, not quarantined, not previously run;
  • DelightEd3 – not notarized, signed with a Developer certificate, not quarantined, not previously run;
  • DelightEd3resigned – not notarized, ad hoc signed, not quarantined, not previously run.

None of the apps run in an app sandbox, and those notarized use a hardened runtime.

This article covers these three variants of the same 14.6.1 VM:

  • Full Security, with Gatekeeper/XProtect disabled;
  • Permissive Security, with SIP disabled;
  • Permissive Security, with both SIP and Gatekeeper/XProtect disabled.

In each VM, settings were confirmed using SilentKnight, which in turn calls standard system tools to determine current security settings, such as those when both SIP and Gatekeeper were disabled.

sksipoff

Gatekeeper disabled

Surprisingly, with Gatekeeper assessments disabled, com.apple.syspolicy.exec still reported that Gatekeeper assessments were made
GK process assessment: <private> <-- (<private>, <private>)
Gatekeeper assessment rooted at: <private>

and later
queueing up scan for code: PST: (vuid: 7C5C43BF-A338-4228-B61E-5038F1D93EDB), (objid: 69229), (team: (null)), (id: (null)), (bundle_id: (null))
GK performScan: PST: (vuid: 7C5C43BF-A338-4228-B61E-5038F1D93EDB), (objid: 69229), (team: QWY4LRW926), (id: (null)), (bundle_id: (null))

Following that, XProtect scanned
XPAssessment performAnalysisOnFileImpl continueOnError set to 0
Xprotect is performing a direct malware and dylib scan: <private>

using its standard Yara rules.

CloudKit ticket lookup also proceeded as normal. After a while, though, XProtect announced
Xprotect is skipping executable assessment: <private>

This concluded with
GK scan complete: PST: (vuid: 7C5C43BF-A338-4228-B61E-5038F1D93EDB), (objid: 69229), (team: QWY4LRW926), (id: (null)), (bundle_id: (null)), 4, 4, 0
and
GK evaluateScanResult: 0, PST: (vuid: 7C5C43BF-A338-4228-B61E-5038F1D93EDB), (objid: 69229), (team: QWY4LRW926), (id: co.eclecticlight.SystHist), (bundle_id: co.eclecticlight.SystHist), 1, 0, 1, 0, 4, 4, 0
GK eval - was allowed: 1, show prompt: 1

The normal prompt for user consent was displayed, and handled as expected. Following that, launch proceeded normally.

Similar entries appeared in the checks made on all apps that had undergone Gatekeeper and XProtect assessment when full security was in force. There is nothing in the log entries to indicate that disabling Gatekeeper had any effect on the checks that were made, although as none of these apps failed assessment, it’s possible that any failures would have been ignored.

SIP disabled

When SIP was disabled, the structure of pre-launch assessments changed, and appeared disordered in comparison to those performed under full security and with only Gatekeeper disabled. Most notable, perhaps, was the almost complete absence of log entries from the com.apple.syspolicy subsystem, which in full security is so prominent, although its service syspolicyd did appear in entries.

Although quarantine was recognised, no entry reported the start or conclusion of any GK (Gatekeeper) assessment, nor subsequent XProtect scans. Instead, the XProtect service wrote
Bundle is not apple signed
Bundle size result: 18388222 (YES)
Always scan: YES

Normal ticket checks were made via CloudKit, but shortly after those were completed, XProtect tried to use its standard Yara rules, and ran out of memory doing so, with the kernel reporting
process XprotectService [697] crossed memory high watermark (15 MB); EXC_RESOURCE
XProtectService therefore ran into trouble before it had even started to scan the app. While some entries suggested prompting the user for their consent, that doesn’t appear to have happened. Eventually the app launched in spite of the disorder that had preceded.

When launching a notarized app that wasn’t quarantined, neither Gatekeeper nor XProtect appear to have had any involvement in the approval of the launch.

SIP and Gatekeeper disabled

Results were essentially identical to those obtained with SIP alone disabled, even down to XProtectService exceeding its memory high watermark, and the almost complete absence of log entries from the com.apple.syspolicy subsystem.

SIP and Gatekeeper settings

Prior to examining these log records, I thought I had a clear idea as to what these two controls do. In fact, neither of them does what you’d expect.

Disabling Gatekeeper or XProtect checks doesn’t stop them from occurring, although it might result in macOS ignoring any errors they might find. That would be consistent with the statement in the spctl man page: “Operations that would be denied by system policy will be allowed to proceed; assessment APIs always report success.”

On the other hand, disabling SIP almost completely stops the whole com.apple.syspolicy subsystem, which ordinarily plays a major role in pre-launch checking of apps. This effectively kills both Gatekeeper and XProtect, leaving those checks in disarray. When the XProtectService tries to lend a hand, its attempt to ingest the current Yara rules runs it out of memory, and it appears unable to render any useful assistance to the pre-launch checks.

This may explain why disabling SIP has the effect of shortening the time to launch an app, most noticeably with larger and more complex apps. In return for launching in a shorter time, the app probably isn’t checked against XProtect’s Yara definitions, so could still contain malicious code that would pass undetected.

In the next article I’ll show what does happen when this system encounters live malware.

Launching apps in Sonoma 14.6.1: Full security

By: hoakley
27 August 2024 at 14:30

This is the first of a series of three articles that look in detail at the launch process of apps in macOS Sonoma 14.6.1, with the emphasis on security checks. This follows my earlier look in 14.4.1, and covers a wider range of situations, including the effects of disabling SIP and Gatekeeper, and how known malicious software is handled.

Methods

All tests were performed in a series of Sonoma 14.6.1 virtual machines (VMs) running on a Mac Studio M1 Max host, also running 14.6.1. VMs are preferred as they enable a consistent environment and easy control of boot security and security settings, together with relatively low rates of log entries. Log extracts were obtained using Ulbow and analysed in their entirety for the first 5-7 seconds after launching apps in the Finder.

Apps used were:

  • SystHist – notarized, quarantined, moved from its landing folder to avoid app translocation;
  • SilentKnight – notarized, not quarantined, previously run;
  • Sparsity – notarized, not quarantined, not previously run;
  • DelightEd3 – not notarized, signed with a Developer certificate, not quarantined, not previously run;
  • DelightEd3resigned – not notarized, ad hoc signed, not quarantined, not previously run.

None of the apps run in an app sandbox, and those notarized use a hardened runtime.

Four variants of the same 14.6.1 VM were run:

  • Full Security, with SIP and Gatekeeper/XProtect enabled;
  • Full Security, with Gatekeeper/XProtect disabled;
  • Permissive Security, with SIP disabled;
  • Permissive Security, with both SIP and Gatekeeper/XProtect disabled.

All had bridged network access to the network and internet, and shared folders with the host, when running these non-malicious apps.

This article describes what happens in the log in the first of those conditions, full security with both SIP and Gatekeeper/XProtect enabled.

Quarantined notarized app

This underwent the fullest checks of these tests. Once LaunchServices announces that it’s opening the app, the following sequence of events is recorded.

CDHashes from the app are copied, here only those for the Arm architecture. As the app is unknown, it’s next registered with LaunchServices. Gatekeeper assessment is then started just 0.07 seconds after announcement of the launch, in the log entry
GK process assessment: <private> <-- (<private>, <private>)
com.apple.syspolicy.exec then starts work on scanning for code, followed by the first mention by LaunchServices that the app is quarantined.

The Gatekeeper scan is announced in
GK performScan: PST: (vuid: 7C5C43BF-A338-4228-B61E-5038F1D93EDB), (objid: 62947), (team: QWY4LRW926), (id: (null)), (bundle_id: (null))
followed by the XProtect scan in
Xprotect is performing a direct malware and dylib scan: <private>
and assignment of the risk category according to its quarantine
QUARANTINE: Setting risk category to LSRiskCategoryUnsafeExecutable
XProtect states the Yara rules it’s using
Using XProtect rules location: /Library/Apple/System/Library/CoreServices/XProtect.bundle/Contents/Resources/XProtect.yara

com.apple.syspolicy next processes the app’s notarization ticket
looking up ticket: <private>, 2, 1
by trying to fetch its record using CloudKit. That’s followed by entries indicating the network access required to connect with iCloud and check the ticket. Success is reported by com.apple.syspolicy in
CKTicketStore network reachability: 1, Mon Aug 26 09:15:45 2024
looking up ticket: <private>, 2, 0

and further lookups.

A little later, Gatekeeper announces the XProtect results
GK Xprotect results: PST: (vuid: 7C5C43BF-A338-4228-B61E-5038F1D93EDB), (objid: 62947), (team: QWY4LRW926), (id: (null)), (bundle_id: (null)), XPScan: 0,-7676743164328624005,2024-08-26 08:19:01 +0000,(null)
and its scan is complete
GK scan complete: PST: (vuid: 7C5C43BF-A338-4228-B61E-5038F1D93EDB), (objid: 62947), (team: QWY4LRW926), (id: (null)), (bundle_id: (null)), 4, 4, 0

Because this is the first launch of a quarantined app, com.apple.syspolicy.exec decides it gets a first launch or “code-evaluation” prompt “because responsibility”. If the user gives approval, the app is allowed to proceed. Its quarantine flag is updated, and the bundle record registered as trusted. The final step is then to create and save its provenance data
Created provenance data for target: TA(e8217440d9326f59, 2), PST: (vuid: 7C5C43BF-A338-4228-B61E-5038F1D93EDB), (objid: 62947), (team: QWY4LRW926), (id: co.eclecticlight.SystHist), (bundle_id: co.eclecticlight.SystHist)
Handling provenance root: TA(e8217440d9326f59, 2)
Wrote provenance data on target: TA(e8217440d9326f59, 2), PST: (vuid: 7C5C43BF-A338-4228-B61E-5038F1D93EDB), (objid: 62947), (team: QWY4LRW926), (id: co.eclecticlight.SystHist), (bundle_id: co.eclecticlight.SystHist)
Putting executable into provenance with metadata: TA(e8217440d9326f59, 2)
Putting process into provenance tracking with metadata: 692, TA(e8217440d9326f59, 2)
Tracking process with attributes: 692, TA(e8217440d9326f59, 2)

Without quarantine

A notarized app that hasn’t been run previously on that system and isn’t quarantined undergoes a similar sequence, but without the first launch or “code-evaluation” prompt. Its bundle record is registered as trusted, rather than being classified as an Unsafe Executable, but it still gets a full XProtect scan and ticket lookup using CloudKit.

Subsequent launches

The briefest launch process is that for an app that has only recently been run. That appears to skip Gatekeeper and XProtect assessments, and there’s no ticket lookup either. Pre-launch processes can then take less than 0.1 second.

Launching a known app following a cold boot can be as quick, although in this case there is a brief Gatekeeper assessment reported in the log. The key entry here comes from com.apple.syspolicy.exec:
Code already evaluated, using results.
Those are checked by Gatekeeper before launch proceeds, with the kernel reporting
evaluation result: 2, exec, allowed, cache, 1724654056, 4, c0a2e35c20a69dfd, /Applications/SilentKnight.app

Signed with developer certificate

An unquarantined app that isn’t notarized but is correctly signed using a Developer certificate is similar to its notarized equivalent, except that looking up the ticket using CloudKit is of course unsuccessful. Repeated attempts are made to find it, though, before going on to check “the legacy list” and check “legacy policy”. This results in the decision
Match downgraded from DevID to None based on legacy policy for: PST: (vuid: 7C5C43BF-A338-4228-B61E-5038F1D93EDB), (objid: 60118), (team: QWY4LRW926), (id: (null)), (bundle_id: (null))
but the kernel decides to allow launch to proceed
evaluation result: 6, exec, allowed, cache, 1724660700, 0, 9576bac3e248c07b, /Applications/DelightEd3.app

Ad hoc signature

This is detected early during pre-launch checks by AMFI (Apple Mobile File Integrity), despite the bundle record being registered as trusted. The kernel reports
AMFI: '/Applications/DelightEd3resigned.app/Contents/MacOS/DelightEd' is adhoc signed.
AMFI then records
No certificate chain found
Failure getting cert chain
Basic requirement validation failed, error: Error Domain=NSOSStatusErrorDomain Code=-67050 UserInfo={SecCSArchitecture=<private>}

and an error code of -423, given as “The file is adhoc signed or signed by an unknown certificate chain”.

Despite that, Gatekeeper assessment continues, with an XProtect scan. Attempts to look up the app’s ticket inevitably fail despite many attempts, and an error code of -67018 “Code did not match any currently allowed policy” is awarded. Launch then proceeds.

In the next article I’ll show how those are affected by disabling SIP and Gatekeeper assessments.

Why launch constraints can crash apps

By: hoakley
23 August 2024 at 14:30

Some apps may crash when launched because there’s something wrong in the app. In Ventura and later, that might occur because macOS is refusing to run them because of security rules, specifically launch constraints. These were extended in Sonoma to allow any app to limit the code it runs to what should be there, in launch environment and library constraints. This article explains what these are, and how you can recognise when constraints are applied.

Code without constraints

Launching an app without constraints isn’t as unconstrained as that might suggest. It’s still given an environment to run in, with settings such as the user’s Home folder and some standard paths including a temporary folder buried in /var/folders. If you’re interested to see what those can include, Mints has a button to show you its own launch environment.

On top of those, the app is limited by standard permissions as to what it can access without obtaining elevated privileges, and everything is subject to the privacy restrictions imposed by TCC according to the app’s Privacy & Security Settings.

But the app can be run from pretty well anywhere, and can run code from libraries, frameworks and other places as it wishes.

Launch constraints

The first set of launch constraints became obvious if you tried to copy and run from a different location one of the apps bundled in Ventura. This has had its purposes in the past, for example to run Network Utility after Apple first gutted then removed it. Try that with one of Ventura’s bundled apps, and the copy can’t be run from any location apart from the SSV it’s installed in, as it crashes immediately. Look in its crash report and you’ll see something like
Exception Type: EXC_CRASH (SIGKILL (Code Signature Invalid))
Exception Codes: 0x0000000000000000, 0x0000000000000000
Termination Reason: CODESIGNING 4 Launch Constraint Violation

That’s given in a bit more detail in the main log, for Terminal as
AMFI: Launch Constraint Violation (enforcing), error info: c[1]p[1]m[1]e[2], (Constraint not matched) launching proc[vc: 1 pid: 2440]: /Users/hoakley/Documents/00crypt/Terminal.app/Contents/MacOS/Terminal, launch type 0, failure proc [vc: 1 pid: 2440]: /Users/hoakley/Documents/00crypt/Terminal.app/Contents/MacOS/Terminal
ASP: Security policy would not allow process: 2440, /Users/hoakley/Documents/00crypt/Terminal.app/Contents/MacOS/Terminal
xpcproxy exited due to OS_REASON_CODESIGNING | Launch Constraint Violation, error info: c[1]p[1]m[1]e[2], (Constraint not matched) launch type 0, failure proc [vc: 1]: /Users/hoakley/Documents/00crypt/Terminal.app/Contents/MacOS/Terminal

The same happens if you try running a forbidden command, such as /usr/libexec/periodic-wrapper.

Open the app using Apparency and view its Launch Information, and you’ll see the launch constraints that caused this.

apparency2

For the Chess app, those read
(on-authorized-authapfs-volume or on-system-volume ) and launch-type = 3 /* CS_LAUNCH_TYPE APPLICATION / and validation-category = 1 /* CS_VALIDATION_CATEGORY_PLATFORM */
which should give you a good idea that app can only be run from its standard location in the SSV or System volume. To make this even harder, Sonoma’s Finder tries to stop you from even copying bundled apps to other locations, and you now have to be ingenious to try launch constraints out.

Launch constraints were first described by Csaba Fitzl, and he has since compiled a listing of all those known. Those shown for Chess.app are Category 14, and common to other bundled apps. Their effect is to prevent all copies of that app from being launched from elsewhere.

Trust caches

Instead of macOS looking up each binary’s launch constraints from the binary itself, all those constraints are assembled into Trust Caches, where they’re listed by the code directory’s hash (cdhash). To look up the launch constraints for the Terminal app, the system first calculates the cdhash for its code directory, then looks in the Trust Cache for the launch constraints given for that cdhash.

The System volume contains a static Trust Cache that covers all the executable binaries that come as part of the system. That’s locked into read-only storage during the early kernel boot phase of startup. Additional Trust Caches are authenticated to ensure they haven’t been tampered with, and loaded when required. Apple cites the example of the Trust Cache required by the code within macOS software updates (known as the update brain) that runs the process, allowing it to run with platform privileges, as it requires to perform the update. Apple gives further details on Trust Caches in its Platform Security Guide.

Disabling launch constraints

What if you need to ignore those launch constraints imposed by macOS? Because system executables are laid out in the static Trust Cache, there’s no way to modify that, and no way to override it. All you can do is disable System Integrity Protection (SIP), which is required for launch constraints to operate.

Environment constraints

Launch constraints and the Trust Cache system are complete and fully enforced as of Ventura 13.3, and have been extended for use by third-parties in Sonoma. Developers can build dictionaries containing facts and applying operations to them to improve the security of their apps. Constraint dictionaries are either saved in property lists for launchd, or in those used for signing code. These too are associated with cdhashes, use some categories common to other trust caches, and work similarly to protect third-party code such as helper apps.

While they might appear overkill, they can be used to address known security problems, of which the most prominent must be maintaining trust with privileged helper apps and XPC services, which have often proved weak points in app security. Apple provides two detailed articles, one explaining how to define these constraints, the other how to apply them. I suspect that we’ll be seeing more of these in the future.

Controlling System Integrity Protection using csrutil: a reference

By: hoakley
21 August 2024 at 14:30

System Integrity Protection (SIP) is more than one of the frontline security protections in macOS, it’s a whole family of them, including controls over kernel extensions, protection for NVRAM, and more. This article explains how you can manage different features in SIP using the csrutil command tool, primarily on Apple silicon Macs running Sonoma or Sequoia.

SIP fully enabled

Unless you have very good reason to do otherwise, your Mac should have SIP fully enabled at all times. Utilities like SilentKnight check whether this is so, and if SIP isn’t enabled, you need to fix that as soon as possible. Start your Mac up in Recovery mode, pass through to Options, and open Terminal from the menu there. Enter the following command:
csrutil enable
and then authenticate when prompted. When you have quit Terminal, open Startup Security Utility from the same menu. You should there be able to set your Mac’s security back to full, unless you need to run it at reduced security to load a third-party kernel extension. Once that is set, restart your Mac and it should have returned to normal with SIP fully active, as SilentKnight should confirm.

SIP’s features

Its original and core feature is to protect much of the system from being changed, even by those with root privileges, referred to here as Filesystem Protections, and introduced in El Capitan, in 2015. This is now extensive, and ranges from whole directory trees in the System volume down to individual extended attributes such as com.apple.macl on otherwise unprotected files.

SIP is also responsible for enforcing requirements for the signing and notarization of non-system kernel extensions, and protection of the integrity of the kernel, respectively termed Kext Signing and Kernel Integrity Protections. It imposes restrictions on the use of kernel debuggers in Debugging Restrictions and the DTrace tool in DTrace Restrictions.

Restrictions over changes that can be made to the NVRAM and boot arguments it can apply to the kernel are controlled by SIP’s NVRAM Protections and Boot-arg Restrictions, while the Authenticated Root Requirement should be self-explanatory. Other features of SIP, including the enforcement of the hardened runtime required of notarized apps, are most probably part of its Apple Internal features.

SIP partially disabled

Although there may be occasions when you need to disable SIP completely, those should be extremely rare. In most cases, you should aim to disable only the features that require it, then return SIP to fully enabled as soon afterwards as possible. One good way to remind yourself that you have temporarily disabled SIP is to write that on a sticky note and put that on your Mac’s display until you’ve enabled SIP again. Don’t leave it to memory, as you’re likely to procrastinate and forget.

To disable SIP completely, start up in Recovery, set Startup Security Utility to Reduced Security, and open Terminal. There enter the command
csrutil disable
confirm that’s what you really want to do, and authenticate. When you restart your Mac, SIP should then be disabled, as you can confirm in Terminal using
csrutil status

Finer control is separated out into its individual features:

  • Apple Internal, disabled in XNU by CSR_ALLOW_APPLE_INTERNAL, and only disabled when SIP is fully disabled
  • Kext Signing, disabled by CSR_ALLOW_UNAPPROVED_KEXTS, abbreviated in csrutil to kext
  • Filesystem Protections, disabled by CSR_ALLOW_UNRESTRICTED_FS, abbreviated to fs
  • Debugging Restrictions, disabled by CSR_ALLOW_KERNEL_DEBUGGER and CSR_ALLOW_TASK_FOR_PID, abbreviated to debug
  • DTrace Restrictions, disabled by CSR_ALLOW_UNRESTRICTED_DTRACE, abbreviated to dtrace
  • NVRAM Protections, disabled by CSR_ALLOW_UNRESTRICTED_NVRAM, abbreviated to nvram
  • BaseSystem Verification, abbreviated to basesystem
  • Boot-arg Restrictions, disabled with nvram
  • Kernel Integrity Protections, disabled with kext
  • Authenticated Root Requirement, disabled by CSR_ALLOW_UNAUTHENTICATED_ROOT, managed separately using csrutil authenticated-root disable and enable.

To enable SIP with one of these features disabled, use a command of the form
csrutil enable --without [feature]
where [feature] is the abbreviated name of the feature to be disabled, such as kext. Chain those options to disable multiple features, such as
csrutil enable --without kext --without nvram
which enables SIP with the exception of Kext Signing and Kernel Integrity Protections, and NVRAM Protections and Boot-arg Restrictions.

Other configuration flags in XNU that don’t appear to be directly supported by csrutil include: CSR_ALLOW_TASK_FOR_PID, CSR_ALLOW_DEVICE_CONFIGURATION, CSR_ALLOW_ANY_RECOVERY_OS and CSR_ALLOW_EXECUTABLE_POLICY_OVERRIDE. Those should be disabled when SIP is fully disabled, though.

Authenticated Root Requirement is managed separately from other SIP controls. To enable it, use the command
csrutil authenticated-root enable
To disable it,
csrutil authenticated-root disable
and to check its status
csrutil authenticated-root status
Note that fully enabling SIP also enables Authenticated Root Requirement.

The following summary table should help you use these complicated options for control of SIP features using csrutil.

csrutilopts

To disable a specific feature, use the --without option for that abbreviated feature name, or the authenticated-root disable option. Note that running csrutil enable also enables the Authenticated Root Requirement. If you need to disable that, use the command to do so after setting other SIP features using csrutil enable --without.

These commands should be good to use on Apple silicon (and Intel) Macs running Sonoma, and don’t appear to have changed in Sequoia.

Although you can check SIP status in regular user mode, making changes to it should be performed in Recovery mode, and in Apple silicon Macs any disabling of its features is likely to require Reduced Security to be set in Startup Security Utility. Status reports for most combinations of disabled features will warn that the combination being used may not be supported in the future; the only common exceptions to that are when SIP is fully enabled or fully disabled.

Thanks to Ryan for adding CSR_ALLOW_TASK_FOR_PID to the control list given above.

Sonoma’s unfinished business: exclaves, conclaves and the kernel

By: hoakley
20 August 2024 at 14:30

It’s not uncommon for mid-cycle releases of macOS to gain new features in preparation for the next major version. Perhaps the most fundamental and significant added to Sonoma 14.4, together with iOS 17.4, iPadOS 17.4 and watchOS 10.4, are exclaves. As far as I’m aware, this is the first time this term has been used in computing, so before trying to explain what they do for our Macs, I’ll demystify its established use in geography.

Definitions

An enclave is a country that’s entirely surrounded by another country, and it’s easy to see how that matches its meaning in the Secure Enclave, which handles our most important secrets, and provides FileVault encryption. Those are highly secure services provided by a separate processing unit within the main Apple silicon chip, or the T2 in recent Intel Macs.

An exclave is a fragment or part of a country that’s separated from the main part, and lies within one or more countries other than its original country. Although I haven’t come across an explanation of what this represents in computing, for these purposes I assume that an exclave is a resource that’s separated from its parent, for example a small portion of Secure Enclave that’s outside the main part, supporting a virtual machine, perhaps.

exclaves1

Although I hadn’t previously come across the term conclave in computing, Apple is now using it in recent versions of the XNU kernel. This isn’t derived from geography, but is normally used for a private meeting or close assembly, most specifically the meeting of cardinals that elects the next Pope. While there’s scant evidence as to its meaning in this context, I’ll assume that it’s based on that general concept.

Secure Enclaves

Apple introduced its first true Secure Enclave for Macs in the T2 chip in December 2017. This chip consists of a four-core main CPU derived from the A10 used in the iPhone 7 and contemporaneous iPads and iPod Touch, with a 32-bit Arm CPU running a completely different operating system, sepOS (a custom version of the L4 microkernel), dedicated to handling and working with the secrets protected by its Secure Enclave. That has its own secure EEPROM storage, an AES engine to perform hardware-accelerated encryption and decryption for the internal SSD, and more.

The first M1 models released in November 2020 came with an integral Secure Enclave, as have all subsequent M-series chips. These are a significant improvement on their predecessors in the T2, adding replay prevention, a second-generation Secure Storage Component, and more. They continue to run their own operating system, sepOS. Support for biometric authentication using Face ID is implemented as a secure mode in the main Neural Engine (ANE), but apparently remains unused in M-series chips. That secure ANE mode is an exclave of the Secure Enclave.

Exclaves

Prior to macOS Sequoia, macOS virtual machines (VMs) running in lightweight virtualisation on Apple silicon are unable to use Apple ID or to access iCloud, because they’re unable to access any protected secrets from the host’s Secure Enclave. In macOS 15 and later, creation of a VM running macOS 15 or later can configure an identity derived from the host Secure Enclave, enabling access to resources requiring Apple ID including iCloud. This is accomplished using an exclave of the Secure Enclave.

As of macOS 14.6.1, exclaves and conclaves are implemented in and supported by three kernel extensions:

  • ExclaveKextClient.kext (introduced in macOS 14.0)
  • ExclaveSEPManagerProxy.kext
  • ExclavesAudioKext.kext

and two Private Frameworks:

  • CoreSpeechExclave.framework
  • libmalloc_exclaves_introspector.framework

These are currently all in version 1.0. If you’re running a beta-release of Sequoia, you might find it interesting to check the version numbers of those components in their respective /System/Library folders.

According to the few comments in the XNU source code, exclaves provide a fixed static set of resources to the XNU kernel. Examples given include conclave managers, services like Apple ID for VMs, named buffers and audio buffers. These resources are named and have a corresponding identifier shared between XNU and the enclave. Those are discovered during the boot process, and made available in a table at two levels. The root table assembles resources by their scope, while secondary tables list actual processes.

For example, a root table entry for the domain com.apple.conclave.a might link to a secondary table listing an audio buffer, an audio service as a conclave manager, and additional buffering. In the case of audio exclaves, they might be used to connect a system extension running with user privileges to privileged access in the kernel and its extensions.

Information about exclaves is hard to come by. Currently, it appears that any log entries originating from exclaves are shown as coming from the kernel. The only time you’re likely to come across more specific information is in the panic log generated following a kernel panic involving an exclave, when the normal content should be supplemented by information from the exclave. These might have improved in Sequoia.

From the evidence in XNU source code and elsewhere, exclaves are intended to support three subsystems:

  • audio system extensions, allowing them to replace third-party kernel extensions, via ExclavesAudioKext.kext
  • Apple ID in macOS VMs via ExclaveSEPManagerProxy.kext
  • secure ANE via ExclavesAudioKext.kext.

Although support for each of these first appeared in macOS 14.4, the second of them, at least, is already functional in Sequoia, but hasn’t been back-ported to Sonoma, restricting its use to Mac hosts and VMs running Sequoia or later.

There are also multiple references to Tightbeam, which doesn’t appear to have been released in open source. This refers to highly focussed lasers used in communications in The Expanse sci-fi. In macOS, it first appeared as the name of a Private Framework released in macOS 13.0, and supported since then. There’s still a great deal more to be discovered about exclaves, conclaves and XNU.

References

Apple’s Platform Security Guide, which explains Secure Enclaves in detail, but doesn’t mention exclaves or conclaves.
Using iCloud with macOS VMs (Apple).
XNU open source on GitHub, see osfmk/kern/exclaves* for instance.

A proposed new interface for SilentKnight 3

By: hoakley
17 August 2024 at 15:00

At the end of this year, my free utility SilentKnight will celebrate its eighth birthday (including its time as LockRattler), hopefully by then in its third app and third version. It originated after a gaffe when a whole batch of brand new MacBook Pros were delivered with their System Integrity Protection (SIP) disabled. At that time the only way to test that was at the command line.

scripting54

By December 2016, LockRattler version 2.0 was already running a suite of nine checks including SIP status. In those days, XProtect had little to do with checking for known malware, and was mainly concerned with ensuring the latest version of Adobe Flash Player was installed.

lockrattler4rel01

With version 4.0 a year later, LockRattler had assumed its familiar layout, and had begun checking settings for Software Update, although it couldn’t run them itself.

lockrattler414

Another year had passed when LockRattler reached version 4.1, and could both check for and install all pending updates.

lockrattler4181

Version 4.18 of December 2018 added the use of red text to draw attention to changes and discrepancies, and LockRattler hasn’t changed a great deal since.

The following year, I pursued goals to automate checking for updates and firmware versions, and simplify LockRattler’s interface in a new app, initially named EFIcienC.

eficienc06

By the time it had reached version 1.0b2 in July 2019, it was recognisable as SilentKnight, its name when it was released later that month as version 1.0.

silentknight01d

silentknight11401

Support for Apple silicon Macs was incorporated early, and finalised in version 1.14 in November 2020.

sk204

SilentKnight 2.0 was released in October 2022, and continues to support macOS up to Sequoia.

As I have explained, macOS 15 changes the management of XProtect and its updates, requiring substantial internal changes in SilentKnight, so I’m now working on version 3. I think the time is right to revisit and redesign its interface, and if possible implement it using SwiftUI so that it becomes more dynamic and adaptive. I have already been experimenting with that, and offer these ideas for your debate.

Both LockRattler and SilentKnight overwhelm with information that is often duplicated or redundant. Their fixed view layouts operate the same for all models of Mac, with the sole exception of information about Studio Displays. I’m keen to move to a more dynamic interface that delivers important information cleanly and clearly, without repetition.

silentknight31

When you first open SilentKnight 3, it should inform you what is right, and what isn’t, so categories in which all entries are as expected are left unexpanded, and only the one that has an unexpected result is automatically expanded, for you to see the abnormal result, shown with the result that SilentKnight expected. These functional groupings correspond closely to those in Skint.

silentknight32

Expanding all the entries shows a complete summary of information about that Mac at the time that SilentKnight last ran its checks, as given in the footer. These are set in a scrolling view and expand within the window as you prefer.

It’s my intention that this new version will list available updates by their label, and allow you to select which you want to download and install, with an easy option to install all of them. That should work around the problems of updating security data updates without updating or upgrading macOS itself. It should also do away with the Install Named Update feature in previous versions, without risking confusion.

Version 3 will also come with a refreshed app icon. The current one identifies EFI and MRT, both of which are increasingly becoming historical remnants now.

silentknight33

Finally, despite what I said earlier, I hope to make this available with support for late versions of Sonoma, as well as Sequoia.

I welcome your comments and suggestions, please.

What changed in macOS Sonoma? End of term report

By: hoakley
15 August 2024 at 14:30

Earlier this year, I wrote about the changes that had taken place in the System between Mojave and Sonoma 14.2.1. Now that 14.6 is out of the door and Sonoma is about to enter its two years of security-only maintenance, this article looks at how it progressed through its year as the current version of macOS. Those still running Monterey or earlier and wondering whether to upgrade now might like to take notes.

Over the last year, Sonoma has continued in the tradition of expanding the contents of the macOS System. Compared to late versions of Mojave in 2019, the number of bundles in the System library folder at /System/Library has increased by 175%, from 4,774 to 8,392, as shown in the chart below.

macossystemdata

Steep rises in the number of bundles took place with the appearance of the boot volume group in Catalina, then again shortly before the release of the first M1 Macs, and their M3 successors in later 2023, but less so prior to the M2.

macoskextdata

This is shown more clearly in the numbers of kernel extension bundles. New hardware often requires new kernel extensions for support, as well as their delivering architecture-specific features such as lightweight virtualisation on Apple silicon Macs. The total number of kernel extension bundles has risen from 680 in 10.15.6 to 772 in 14.6. Ironically, this is the same period over which Apple has been deprecating third-party kernel extensions, which are now decidedly uncommon on Apple silicon Macs.

Comparisons between folders within the /System/Library folder, between the last full releases of Catalina (10.15.6) and Sonoma (14.6), provide deeper insights as to just where that growth has taken place.

macossystembreakdown

Greatest growth has been in Private Frameworks, which have almost doubled in number, from 2,055 to 3,803, now accounting for 45% of bundles in /System/Library. Those form the private API in macOS; public Frameworks have only risen from 600 to 772, now accounting for less than a tenth of all bundles. Thus the major increase in macOS has come in code that isn’t accessible to third-parties, only to Apple.

If you’re running a Sequoia beta, you might like to compare the figures for Extensions and the total number of bundles in /System/Library to see whether there’s any sharp increase to support the introduction of the M4 chip, or for Apple Intelligence in 15.1. I’m putting my money on the next Macs being from the M3 family, and expecting sudden rapid growth in bundles coming slightly later to support the introduction of AI.

There’s finer detail that you might have missed. During Sonoma’s year, Contact Key Verification was introduced for Messages, to ensure that your messages only reach those intended, although it requires all devices using the same Apple ID to be running recent versions of macOS/iOS/iPadOS. In lightweight virtualization, a bug was fixed that had prevented the VMs of older versions of macOS from updating with their security releases.

Among the new kernel extensions are some apparently supporting a new kernel architecture feature of Exclaves. Those include ExclaveSEPManagerProxy and ExclavesAudioKext, which might be concerned with those enhancements to virtualisation supporting the use of Apple ID in Sequoia.

Sonoma 14.4 in particular brought many new Private Frameworks, some with intriguing names include CascadeEngine, CascadeSets, Dendrite, DendriteIngest, several Poirot frameworks, SemanticPerception, and SonicKit. In other updates there were Cosmo and Tabi.

Since its release, updates to Sonoma appear to have brought mostly minor enhancements and new features, together with clearing a substantial backlog of bugs. This seems to have been a good opportunity to evolve and consolidate rather than rushing ahead with changes.

macOS updates are shrinking, but what happened to RSRs?

By: hoakley
14 August 2024 at 14:30

When Apple introduced the Signed System Volume (SSV) in Big Sur, the size of macOS updates rose to their highest ever, as the overhead required for each update reaching 2.2 GB for Intel models and 3.1 GB for Apple silicon. Since then, they have steadily improved in efficiency. This article shows how they have performed in Sonoma, and asks whether Apple has abandoned its Rapid Security Responses (RSRs).

To keep my charts clearer, I here show the two architectures separately.

Total update size

macosupdatesizes6intel

This chart shows cumulative sizes of updates to macOS on Intel Macs from 10.14 Mojave, with its traditional single boot volume, through to macOS Sonoma 14.6. Each point represents the cumulative sum of all updates to that major version of macOS required to reach that minor version. Thus the point for 14.2 is the total of update sizes for 14.1, 14.1.1, 14.1.2 and 14.2. Sizes used aren’t those reported by Software Update, but those of the download itself, as reported in articles here, indexed on this page. That has made a significant difference for some updates for Apple silicon Macs, where the first reported update size has excluded additional architecture-specific overhead. RSRs have been excluded, as they’ve been duplicated in subsequent patch or minor updates. Lines shown are best fits by linear regression.

Update sizes rose markedly from Mojave, with its single boot volume, to Catalina, with its boot volume group, and again to a peak in Big Sur, with the SSV. They fell again as Monterey introduced greater efficiency, and Ventura and Sonoma have been almost identical, and smaller than Mojave.

macosupdatesizes6as

Apple silicon Macs started with the huge updates of Big Sur, which were even larger than those for Intel models, and benefitted from the improved efficiency of Monterey and Ventura. Unlike Intel Macs, though, Sonoma has seen further reduction in update sizes, although in each update they remain significantly larger than those for Intel models.

Big Sur took a total of 36.7 GB on Intel, and a remarkable 50.07 GB on M1; Sonoma has taken 14.1 GB on Intel, and 21.2 on M1. On Intel, Sonoma required 38% of the update size as Big Sur; on M1, that proportion was slightly higher at 42%. Apple silicon updates were 136% the size of those for Intel models in Big Sur; in Sonoma that ratio has risen to 150%.

Minimum update sizes were about 400 MB for Intel, and 820 MB for Apple silicon, which must be close to the overhead required for macOS updates for the two architectures. These reflect the size of the ‘Update Brain’ required to install each update, and all bundled firmware updates. The latter have been steadily reducing as Intel updates have supported fewer models without T2 chips, but have remained the same if not grown for the substantially larger firmware updates for Apple silicon’s Secure Boot.

Unscheduled updates

Sonoma reached version 14.6 without an excessive number of patch updates. The number of patch versions released between the x.0 and x.6 scheduled versions ranges from 4 (Monterey) to 6 (Bug Sur, Ventura), with Sonoma taking just 5, although 14.6 has been closely followed by 14.6.1 in what appears to be a bug fix rather than the first security update.

What has been surprising is that Apple has released no RSRs for Sonoma, not one. The last RSR was the second released for 13.4.1 more than a year ago, on 12 July 2023. RSRs were introduced to Ventura in May 2023 as a method of replacing some components of macOS that aren’t installed in the SSV but in separate cryptexes, cryptographically verified disk images stored on the hidden Preboot volume. They were intended to let Apple promulgate important fixes to vulnerabilities in Safari, WebKit, and some other bundled components without waiting for a full macOS update. Unfortunately, Apple’s second RSR in July last year was fumbled badly when it broke Safari access to many popular websites including Facebook, and had to be replaced three days later.

Although it’s never clear when a patch update could have been issued as an RSR, Sonoma 14.1.2 should have been a candidate as it’s only reported to have fixed two vulnerabilities in WebKit, that appeared to have been addressed in Safari-only updates for macOS 12 and 13. It looks now as if Apple’s initial enthusiasm for RSRs has cooled, and they’re unlikely to be significant in the future.

Standalone updaters

Updates to macOS Catalina and earlier were also available in downloadable standalone installer packages. Apple discontinued those, amid much protest from users, when it introduced Big Sur. Although there were some suggestions that they might return in the future, as the new update mechanism matured, there has still been no sign of them. Currently the only form of updater available is the full macOS Installer app, typically over 13 GB in size. This remains a shortcoming compared with Catalina and earlier, but it appears that standalone updaters aren’t missed by most Mac users, or at least that Feedback to Apple has been insufficient to produce a response.

Major updates

Prior to macOS Ventura, upgrading to the first release of the next major version of macOS required downloading its full installer app. Although larger than necessary to perform the upgrade, it was relatively error-proof, as the user had to intentionally start the app’s installation process, which could also be cancelled once it was in progress.

Ventura was the first major version for which Software Update used a mechanism similar to that for minor updates, and didn’t download a discrete macOS installer app. As Apple hadn’t warned of that, it caught some users by surprise, when they found their Mac was being upgraded almost automatically. This was repeated with the release of Sonoma, and the problem compounded when some users appear to have been upgraded without their consent.

As Tom Bridge has pointed out, this requires smaller downloads that install considerably faster, but, on Intel Macs at least, don’t require user consent or authentication.

Apple is likely to employ the same update mechanism when it releases Sequoia and takes us on to the next cycle.

Apple has released Sonoma 14.6.1 and 13.6.9 patch updates

By: hoakley
8 August 2024 at 02:09

Apple has just released patch updates to bring macOS Sonoma to version 14.6.1, and Ventura to 13.6.9. There’s no update for Monterey, though.

There are no entries for these updates in Apple’s security release notes, which simply report “no CVE entries” for both. However, the matching updates for iOS apparently refer to important bug fixes, including one that prevented changing Advanced Data Protection settings. Whether that’s the same for Sonoma and Ventura is anyone’s guess.

There are no updates to firmware on T2 Intel models, or on Apple silicon, and Safari’s version and build numbers haven’t changed either.

The 14.6.1 update is just over 650 MB to download on Intel systems, and around 1.4 GB on Apple silicon.

Looking through the contents of the /System folder shows small build increments in the Security framework and some keychain-related files such as Keychain Circle Notification app. There are no other changes in /System or the bundled apps at all.

Last updated 1845 GMT 7 August 2024.

What has changed in macOS Sonoma 14.6

By: hoakley
30 July 2024 at 06:38

As I forecast only yesterday, Apple has today released macOS Sonoma version 14.6. According to its previous calendar, this should be the last release of the cycle featuring general bug fixes and any remaining enhancements. In previous years this has been released in September, just before the first release of the new version of macOS. This year, in September we could see either 14.7 or 14.6.1, Sonoma’s first security-only update.

Apple hasn’t provided any release notes at all for 14.6, although it did for the release candidate, where it stated that this fixes app crashes when running iPhone and iOS apps on Apple silicon Macs, and a complex bug resulting in the hardware video decoder not being used when it should have been.

Security vulnerabilities addressed total 54, of which several are bugs in open source code. These are detailed in Apple’s security release notes.

There are firmware updates for all Macs with T2 or Apple silicon chips, and probably several other recent Intel models. T2 Macs should now be running version 2022.140.5.0.0 (iBridge: 21.16.6074.0.0,0), and Apple silicon Macs iBoot version 10151.140.19. I will analyse other models and update the firmware tables here, and SilentKnight’s expectations, later in the week, to allow time to update.

Sonoma 14.6 has a build number of 23G80.

Looking at changes in the System volume, there are many minor increments in build numbers, including both public and private frameworks. Of the bundled apps, in addition to several minor build increments, the following appear more substantial:

  • Music updated to version 1.4.6
  • News to version 9.5
  • Photos a large build increment
  • Safari to version 17.6 (19618.3.11.11.5)
  • Stocks to version 6.2.3
  • TV to version 1.4.6.

In the /System/Library folder, APFS is updated to version 2236.141.1 and the FileProvider framework has a significant build increment.

These confirm that many small general bug fixes are included with the long list of security fixes, although there’s no sign of any new kernel extensions or private frameworks with exotic names.

Apple has just released Sonoma 14.6, Ventura 13.6.8 and Monterey 12.7.6

By: hoakley
30 July 2024 at 04:50

Apple has just released the update to bring Sonoma to version 14.6, together with security updates to Ventura (13.6.8) and Monterey (12.7.6). I expect that there will also be a separate update to Safari for Ventura and Monterey.

The 14.6 update is around 2.2 GB for Apple silicon Macs, and 1.8 GB for Intel models.

iBoot firmware for Apple silicon Macs is updated to version 10151.140.19, firmware of Intel Macs with T2 chips is updated to version 2022.140.5.0.0 (iBridge: 21.16.6074.0.0,0), and Safari to version 17.6 (19618.3.11.11.5).

Significant bugs fixed include app crashes when running iPhone and iOS apps on Apple silicon Macs, and a complex bug in video decoding in which the hardware decoder wasn’t used when it should have been.

Security release notes for Sonoma reveal a total of 54 vulnerabilities have been addressed, including several in open source code, but none is reported as having been known to be exploited yet. Those for Ventura list 36, and for Monterey 32.

I’ll update this article with further details as they arrive.

Last updated 2142 GMT 29 July 2024.

Last Week on My Mac: Are you ready for Sonoma 14.6?

By: hoakley
28 July 2024 at 15:00

Last Tuesday, as the uproar over the CrowdStrike catastrophe was still subsiding, and alongside the fourth developer beta of Sequoia, Apple quietly provided its first release candidate for macOS 14.6. Don’t be surprised if that ships early next week, alongside Ventura 13.6.8 and Monterey 12.7.6, making it the earliest release of the sixth minor version in the annual macOS cycle since Mojave. For the last four years, from Catalina to Ventura, the last full update marking the start of two years of security-only maintenance has taken place in September.

To put this into context, it’s worth revisiting how Apple numbers macOS versions and how that fits into its annual release cycle:

  • The first digits in the version number indicate the major version, currently 14 for Sonoma, which changes around September-October of each year with the start of each new cycle.
  • The second digit indicates the minor version, currently 5 for Sonoma, and starts from 0 with the first major release in September. Minor updates are scheduled well in advance, and culminate in the sixth as the last regular update (Catalina was an exception in running to 7), and the start of that version’s first year of security-only updates.
  • The third digit indicates the patch version, used for urgent unscheduled fixes between those minor versions. The previous release of Sonoma before 14.5 was 14.4.1, which fixed a few urgent bugs and security vulnerabilities. For security-only updates after the x.6 release, this marks each security update.

Thus the first year of Sonoma is expected to run from its first release as 14.0 on 26 September last year, through 14.1 to 14.6, its last full update, shortly before the release of Sequoia. Sonoma then enters its first year of security-only updates in 14.6.1, and its second year in the autumn/fall of 2025 with versions 14.7, 14.7.1, and so on, until it becomes unsupported after a total of three years.

Sonoma has been running early throughout its release cycle, starting with its first release, and squeezed in 14.2 well before Christmas. While this could indicate that Apple doesn’t intend putting it into security-only maintenance until after version 14.6.1 or even 14.7 in September, around the time of Sequoia’s initial release, that would mark a significant change in the annual cycle. You also have to wonder how many non-security fixes could be prepared for release during August, when most of Apple’s software engineers are fully extended in finishing off the major new versions of macOS, iOS, iPadOS, watchOS, tvOS and visionOS for release the following month.

I expect next week will bring the release of Sonoma 14.6, the last general update and start of its two years in security-only maintenance, together with Ventura 13.6.8 as the end of its first year of security-only fixes, and Monterey 12.7.6, its swan song as it becomes unsupported. Those should pave the way for Sequoia 15.0 in September, bringing support for a batch of new Macs featuring M3 and M4 chips, to include an updated Mac Studio, Mac mini, and MacBook Pros.

If your Mac is still running Ventura or Monterey and is supported by a more recent version of macOS, now is the time to make a decision about whether and when to upgrade. Even if Monterey does get one more final security update, it’s almost certain to fall well short of that provided for Ventura and Sonoma. If you were intending to upgrade to Sonoma, then it’s not likely to have any further general fixes after this forthcoming update, but only to receive security updates from August onwards.

Appendix: Previous last general updates to macOS

  • Ventura 13.6 – 21 September 2023
  • Monterey 12.6 – 12 September 2022
  • Big Sur 11.6 – 13 September 2021
  • Catalina 10.15.7 – 24 September 2020
  • Mojave 10.14.6 – 22 July 2019
  • High Sierra 10.13.6 – 9 July 2018
  • Sierra 10.12.6 – 19 July 2017

Data from System Updates

Excluding folders and files from Time Machine, Spotlight, and iCloud Drive

By: hoakley
9 July 2024 at 14:30

Backups, indexing for search, and cloud storage are each very useful when and where you want them. But there are occasions when you want to exclude folders and files from their reach. This article is a short reference to how you can exclude items from being backed up, indexed and searched, and stored in iCloud Drive.

Time Machine backups

Time Machine has several mechanisms for excluding items from its backups. The first of those used to be its Standard Exclusions list, at /System/Library/CoreServices/backupd.bundle/Contents/Resources/StdExclusions.plist, but that has long since disappeared from there. Currently the best way to inspect that list is by viewing the hidden file .exclusions.plist at the top level of any backup.

This lists several categories of exclusions, of which the first are apiExclusionPaths, added by individual third-party apps. The equivalent of what used to appear in StdExclusions.plist is now given under the standardExclusionPaths key, containing:

  • .DocumentRevisions-V100 – the version database on each volume, added in Big Sur,
  • .Spotlight-V100 – Spotlight metadata,
  • .Trashes – contents of Trash folders,
  • .fseventsd – the File System Events database,
  • /Library/Logs – traditional text log files,
  • /Users/Guest – guest user files,
  • /private/var (partial) – various transient files,

among many other ephemeral items.

Of those, only one results in any significant loss of data, the version database. Although this was dutifully copied by Time Machine into backups for several years, the current structure of that database appears to make it impossible to restore successfully, even when restoring a complete volume. By the middle of the Catalina cycle, it had become a frequent cause of Time Machine choking, so was added to the standardExclusionPaths for Big Sur. As far as I’m aware this was never fixed in Catalina.

Following those are stickyExclusionPaths, excluding various Photos Library contents. The key systemFilesExcluded is set to true to ensure that the whole System volume is always excluded, following which are any userExclusionPaths you have set using tmutil or in Time Machine settings, using the Options… button.

xclusions1

By default, volumes on external storage are automatically added to this exclusion list; if you want an external volume to be backed up by Time Machine then you will need to remove it from the exclusion list manually.

In addition to the rules given in the Standard Exclusion list, any file or folder can have an extended attribute of type com.apple.metadata:com_apple_backup_excludeItem attached to it as a ‘sticky’ exclusion, which should add it to the stickyExclusionPaths list and prevent it from being backed up by Time Machine. If you then remove that extended attribute, it should be backed up normally again.

Mike Bombich gives a thorough and detailed account of what CCC doesn’t copy on this page. Other backup utilities should also provide full lists on their support site.

Note that no folders or files can be excluded from APFS snapshots, whether made by Time Machine, CCC or any other app, as snapshots always capture the entire volume.

Spotlight search

There have been three general methods of excluding folders from Spotlight’s indexing and search, although only two of them still work reliably:

  • appending the extension .noindex to the folder name (this previously worked using .no_index instead);
  • making the folder invisible to the Finder by prefixing a dot ‘.’ to its name;
  • putting an empty file named .metadata_never_index inside the folder; that no longer works in recent macOS.

System Settings offers Spotlight Privacy settings in two sections. Search results won’t normally prevent indexing of those items, but does block them from appearing in search results. Spotlight’s indexing exclusion list is accessed from the Spotlight Privacy… button, where you can add items that you don’t want indexed.

xclusions2

iCloud Drive storage

This works primarily by inclusion, in that you copy or move items to the iCloud Drive folder when you want them to be copied up to iCloud Drive. There are occasions, though, when you want to exclude specific folders and files that need to be in a folder in iCloud Drive, but you don’t want those items to be stored in the cloud. iCloud won’t sync items with the extensions .nosync or .tmp, so putting files inside folders whose name ends in either of those extensions will ensure that they’re not stored in iCloud Drive.

Other items intended for local housekeeping and similar purposes that are excluded from syncing include:

  • .DS_Store files
  • files whose name starts (A Document Being Saved
  • .ubd files
  • items with names containing .weakpkg
  • desktop.ini files
  • items with names starting with ~$
  • items named $RECYCLE.BIN, which are trash folders
  • items named icon\r, which are custom folder icons.

Files and folders with purposes that make iCloud syncing inappropriate include those:

  • named iPhoto Library, and presumably now Photos Library too
  • named Dropbox, .dropbox, or .dropbox.attr
  • with the extension .photoslibrary, .photolibrary, .aplibrary, .migratedaplibrary, .migratedphotolibrary, or .migratedaperturelibrary
  • named Microsoft User Data
  • named ~ with an extension of 3 or more characters.

The only setting in System Settings gives control over Desktop & Documents folders, and those of individual apps; there’s no exclusion list available.

xclusions3

Simple reference to folder controls

  • Time Machine: add to settings with the Options… button, or using tmutil;
  • Spotlight indexing: add to settings with the Spotlight Privacy… button, or append the extension .noindex to the folder name;
  • iCloud Drive: append the extension .nosync or .tmp to the folder name.

Last Week on My Mac: Buried treasure

By: hoakley
7 July 2024 at 15:00

Much of our literature is based on a few legends. When one pirate, William Kidd, was believed to have buried his ill-gotten gains on Gardiners Island, that formed the basis of many adventure stories, including Washington Irving’s Wolfert Webber, Edgar Allan Poe’s The Gold-Bug, and Robert Louis Stevenson’s Treasure Island, and dozens of derivatives. Last week I too was digging for buried treasure, this time in the deep soil of the macOS log.

Following up on my previous observations of Battery Centre (com.apple.BatteryCenter) and its log entries recording checks on wireless input devices and UPS, my first task was to build myself a shovel using SwiftUI, a utility I’ve named Unhidden for several good reasons. Although I’ll be releasing and explaining that tomorrow (Monday) morning, in this article I’ll explain why it, and other log analysis tools, are so valuable.

For the last eight years, since its introduction in Sierra, macOS has written copious entries into its Unified log. Although supported from the start by the log command tool, for the first three of those years, third-party developers were denied direct access to the new log. Even now, reading log entries using Swift is rudimentary to say the least: for example, the API still can’t deliver a collection of entries written during an arbitrary period, which can only be achieved by calling the log command to deliver them in JSON format, and parsing that. Given those limitations, all my previous utilities that retrieve log entries have relied on the log command; Unhidden is the first of them to use the Swift API of OSLog.

If Apple intended the Unified log and its bad habits, like censoring so much key information with <private>, as a means of obfuscating macOS internals, it was hugely successful. Seasoned Mac experts who had previously gained much from browsing the log found themselves submerged in so much log chatter that most abandoned all further attempts. So many developers gave up trying to make use of the log that Apple is still trying to win them back. In High Sierra, using the newfs_apfs command to encrypt a previously unencrypted APFS volume leaked passwords in plain text to the Unified log for six months until Sarah Edwards discovered the bug, and Apple fixed it in 10.13.4.

Those who use my utility T2M2 will be aware of how its analysis of log entries makes them aware of serious problems occurring in their Time Machine backups that are never reported to the user. Two years ago, log entries confirmed the function of a new form of XProtect, Remediator, and how it runs periodic scans looking for malware. The only way to tell if APFS trims volumes is to read the entries it writes in the log, and for many other systems in macOS log entries are the only evidence of their completing key tasks successfully.

This journey into com.apple.BatteryCenter started with chance observations that revealed how macOS may periodically interrogate what Apple terms Human Interface Devices, or HID (hence Unhidden), to discover key information including their battery charge status. As far as I’m aware, there’s no public API to perform that, and the only approach is to dig through the I/O Registry, a task akin to searching for treasure using the clues to a cryptic crossword puzzle instead of a map.

After drawing a complete blank in Apple’s documentation I discovered that BatteryCenter is a Private Framework new to macOS Sonoma 14.0, and didn’t exist in Ventura, where similar information came from com.apple.iohid. Alongside are two related user interface frameworks, BatteryCenterUI and BatteryUIKit, making a total of three of the over 1,600 private frameworks in Sonoma, and presumably destined for Sequoia. Whether they’ll ever make the leap to being part of the public API appears doubtful, so for the time being, the only way to access them is through their entries in the log.

When it comes to searching for buried treasure in macOS, there’s nowhere as rewarding as the Unified log. There may be no maps, and when we need shovels all Apple provides are trowels, but there’s more silver and gold here than there ever were on Gardiners Island.

How Time Machine backs up iCloud Drive in Sonoma

By: hoakley
5 July 2024 at 14:30

macOS Sonoma brought major internal changes to iCloud Drive, and coupled with the opacity of Time Machine, you may now be wondering whether it still backs up what you have stored in iCloud Drive, and if so, where to find those backups. This article explains how this now works, and where those backups can be found.

What does Time Machine back up?

By default, Time Machine backs up almost everything on volumes it’s set to back up. It doesn’t, of course, try to back anything up from the Signed System Volume (SSV) containing macOS itself, and there are a few folders it automatically excludes, such as the hidden folder at the root of each volume containing the versions database (as that can’t be restored in any case). But unless you add iCloud Drive to its exclusion list from your Home Library folder, Time Machine will back up all the files and folders in iCloud Drive as long as they’re stored locally at the time it makes that backup.

Dataless files

It’s that last condition that’s the most important consideration. If your Mac has Optimise Mac Storage turned off, then iCloud Drive is run as a Replicating FileProvider, where everything stored in iCloud Drive is also stored locally. In that case, all those items will be backed up by Time Machine just as they would be if kept in purely local storage.

If Optimise Mac Storage is turned on, iCloud Drive behaves as a Non-Replicating FileProvider, and some or many of the files you store in iCloud Drive may be evicted from local storage, and are then no longer available for backup. This is explained in the diagram.

iCloudDriveFileSummary1

Files in local storage consist of attributes, metadata (extended attributes) and data. Move them to iCloud Drive and their data is then copied up to iCloud. When a file is evicted to save local storage space, the local file’s data is removed, making that copy of the file dataless. In previous versions of macOS, that was accomplished using a special stub file, but in Sonoma it simply loses its data, and the distinctive iCloud icon is displayed next to the file in the Finder.

icloudnos1

Dataless files can’t be backed up by Time Machine, as without their data there’s nothing to back up. If you want that file to be backed up, then it has to be downloaded or materialised by downloading its data from iCloud. Some third-party backup utilities can manage that automatically for you, but Time Machine doesn’t. If you want to ensure that a file in iCloud Drive is backed up when Optimise Mac Storage is turned on you’ll thus have to download it manually if it has been evicted to iCloud.

Restoring from backup

The simplest way to restore a file or folder in iCloud Drive from a Time Machine backup is through the Time Machine app. Before opening that app, select iCloud Drive in the sidebar of the front Finder window, then the app will take you back through its backups of iCloud Drive.

tmicloud

The main drawback of that is that items you restore will replace any existing items. You may also prefer to use the Finder’s or a command line interface to your backups, both of which allow you to copy restored items where you prefer. The difficulty here is that macOS deliberately hides the path to iCloud Drive: in your active Home folder, they’re located in ~/Library/iCloud Drive, but that path doesn’t work in Time Machine backups.

To access them in a backup, follow the Finder path
[backupname]/[backup].backup/Data/Users/[username]/Library/Mobile Documents/com~apple~CloudDocs/

From Terminal, you’ll have more of a struggle, but need to look for
/Volumes/.timemachine/[UUID]/[backup].backup/[backup].backup/Data/Users/[username]/Library/Mobile\ Documents/com\~apple\~CloudDocs/
where the path component [backup].backup is repeated. If you locate an item first in the Finder, you can always drag and drop that into the command line for it to automatically paste the correct path.

Summary

  • Unless you manually exclude iCloud Drive from Time Machine backups, the contents of iCloud Drive will be backed up provided that items haven’t been evicted from local storage.
  • If Optimise Mac Storage is turned off, the whole of iCloud Drive will be backed up by Time Machine, as eviction doesn’t occur.
  • If Optimise Mac Storage is turned on, only those files and folders that are stored locally will be backed up by Time Machine. To ensure an item is backed up, manually download it before the backup is made.
  • Restoring iCloud Drive items from a backup is simplest in the Time Machine app, but also available in the Finder, from a path equivalent to ~/Library/Mobile Documents/com~apple~CloudDocs/ in that backup.

What do XProtect BehaviourService and Bastion rules do?

By: hoakley
28 June 2024 at 14:30

Not content with two different XProtects, Apple added a third to macOS Ventura, XProtect BehaviorService (XBS), part of the new Bastion behavioural-based malware detection system. Rather than performing on-demand or periodic scans of static code, this watches for potentially malicious behaviours, such as attempts to access folders used by browsers such as Safari and Google Chrome. This article summarises what XBS is doing as we prepare to upgrade from Sonoma to Sequoia.

What they do

Apple tells us precious little about XBS and Bastion, mentioning them in its Platform Security Guide: “In addition, XProtect contains an advanced engine to detect unknown malware based on behavioral analysis. Information about malware detected by this engine, including what software was ultimately responsible for downloading it, is used to improve XProtect signatures and macOS security.”

At present, XBS and Bastion only record suspicious events in the XBS database at /var/protected/xprotect/XPdb, report them to Apple, but don’t attempt to intervene in any way. They determine what to report according to a set of rules applied by syspolicyd that are compiled from source files updated inside XProtect Remediator update bundles. Changes in those, in XPR’s scanning modules, and in XProtect’s detection signatures, are reported on this blog for each update released by Apple.

Development

Over the period since its introduction, Bastion rules have grown steadily, from four to 12:

  • In macOS 13.5 (24 July 2023) there were 4 rules, increasing to 5 in September 2023.
  • XProtect Remediator (XPR) 108 (8 August 2023) brought the first separate Bastion rule update.
  • XPR 112 added rules 6 and 7.
  • XPR 123 added rules 8 and 9, and adjusted rule 7.
  • XPR 130 added rule 10.
  • XPR 131 added rule 11.
  • XPR 137 added rule 12, and amended rules 6 and 7.

Updates provided in XProtect Remediator contain two files for XBS and Bastion:

  • bastion.sb, a text file containing the latest Bastion SystemPolicyConfiguration, its rules;
  • BastionMeta.plist, a property list defining behaviour dictionaries for XBS and Bastion.

Bastion rules

The Bastion SystemPolicyConfiguration file bastion.sb is prefaced with the line (version 3), which hasn’t changed since the first update.

This first defines four groups of processes: usual-offenders, common exceptions to several rules, and separate groups of exceptions to each of Bastion rules 1, 2, 3 and 12. For example, com.apple.mds and other Spotlight indexing processes are usual-offenders, while com.apple.Finder is only a rule-one-offender. Interestingly, three of the XProtect Remediator scanning modules (MRTv3, Pirrit and WaterNet) are included in the list of usual-offenders.

Using those lists of exceptions, Bastion rules are then built as filters:

  1. excludes other processes from accessing private data for Google Chrome, Firefox and Safari;
  2. excludes other processes from accessing private data for Messages, Microsoft Teams, Slack and WhatsApp;
  3. excludes other processes from accessing the QuarantineEvents database;
  4. controls access to two socket ioctl commands SIOCIFCREATE and SIOCGIFDESC;
  5. controls access to writing files with a period/stop at the start of their name within Library/PrivilegedHelperTools/ directories.
  6. controls creating or writing to files with a name starting with com within /Library/Application Support/
  7. controls creating or writing to files with a name starting with com within /Library/Application Support/ and user /Library/Application Support/ directories
  8. controls creating or writing to files with a name starting with a period/stop, other than .DS_Store, in user /Library/Application Support/ directories
  9. excludes other processes from creating or writing to files in user /Library/Containers/com.apple.Safari/Data/Library/Safari/AppExtensions/ directories
  10. controls creating or writing to files with a name starting with a period/stop, other than .DS_Store, .betamigrated and .localized, in the /Users/Shared/ directory
  11. controls execution of processes from files with a name starting with a period/stop in the /Users/Shared/ directory
  12. excludes other processes from accessing private data for Notes, Safari Cookies, Chrome, Brave, Microsoft Edge, Opera, Vivaldi, Firefox, Arc, other cookies, Electrum and Coinomi wallets, Exodus, atomic, Binance, Filezilla, Steam and Discord.

The updated bastion.sb file supplied in XPR updates is explicitly referenced by syspolicyd to replace the version embedded in its own code.

BastionMeta.plist

This property list contains a metadata dictionary of 12 behaviours, each correlating with a Bastion rule. Each has a Signature Name, such as macOS.NetworkSniffer.Generic, a Boolean value indicating the need for immediate reporting, and a binary flag ranging from 1 to 2048. The behaviours are named:

  1. Browser
  2. Messages
  3. QntDb
  4. NetworkSniffer
  5. HiddenPrivilegedHelpers
  6. ADLOAD NumericPath
  7. ADLOAD PersistenceSearch
  8. Persistence HiddenAppSupport
  9. Safari ExtensionModification
  10. Persistence HiddenShared Generic
  11. Persistence HiddenShared Exec
  12. InfoStealers.

Behaviours detected

Individual rules currently detect:

  1. attempts to access private browser data
  2. attempts to access private messaging data
  3. attempts to access quarantine records
  4. attempts to perform network packet sniffing
  5. attempts to write to hidden privileged helper apps
  6. Adload behaviours
  7. Adload persistence behaviours
  8. persistence behaviour using hidden files in user /Library/Application Support/ directories
  9. attempts to create and use Safari extensions
  10. persistence behaviour using hidden files in /Users/Shared/
  11. persistence behaviour running hidden files in /Users/Shared/
  12. attempts by an InfoStealer to access a wide range of private data.

Summary

  • In macOS Ventura and later, XProtect BehaviorService (XBS) and its Bastion rules detect suspicious behaviours that might reflect malicious activity.
  • Bastion rules are updated within XProtect Remediator updates, using two files bastion.sb and BastionMeta.plist.
  • There are currently 12 Bastion rules, covering generic behaviours such as accessing private data, to those indicative of Adload and InfoStealer malware.
  • Suspicious behaviour is recorded locally to the XBS database and reported to Apple, but isn’t notified to the user.
  • Currently the primary purpose of XBS and Bastion is to provide Apple’s security team with intelligence to improve protection provided by XProtect and XProtect Remediator.

Reference

Chris Long, Leveraging Osquery To Examine The XProtect Behavioral Service DB

❌
❌