Reading view

There are new articles available, click to refresh the page.

How iCloud can be simpler than a server

Apple provides so many services for different parts of macOS that it’s hard to keep track of them. If you want to see a short summary, this article lists all service connections for enterprise network administrators, although it doesn’t detail which services use which servers, for example referring to “macOS updates” in many entries.

Many of you seem surprised to learn that Sequoia’s new XProtect updates come from iCloud, although Apple has been using iCloud for similar purposes for at least the last five years.

One good example that’s used every day on your Mac are the notarization checks sometimes run by Gatekeeper when macOS launches executable code, such as an app. In that case, com.apple.syspolicy processes the app’s notarization ticket
looking up ticket: <private>, 2, 1
by trying to fetch its record from iCloud using CloudKit. That’s followed by log entries indicating the network access required to connect with iCloud and check the ticket. Success is reported by com.apple.syspolicy in
CKTicketStore network reachability: 1, Mon Aug 26 09:15:45 2024
looking up ticket: <private>, 2, 0

and further lookups. I first reported those checks with iCloud back in Catalina, in 2019.

A simple way to illustrate the differences between this and using the general softwareupdated service is to compare what happens in the log when you ask if there are any updates available.

softwareupdate

When SilentKnight does this, it uses the only supported method, the softwareupdate tool, as used to keep XProtect up to date in all versions of macOS prior to Sequoia. That command hands over to the softwareupdated service to run the check. That in turn uses components of com.apple.SoftwareUpdateController to summarise the update state of that Mac, connect to the Software Update Server, check all the current versions and build numbers of macOS and its ancillaries, and arrive at a list of updates required. This is even more complex than it sounds, as com.apple.SoftwareUpdateController has to check key settings such as whether the root volume is sealed or not.

You can trace this through several thousand log entries, and after around 4.4 seconds and multiple network connections, softwareupdate finally informs SilentKnight that there are no updates available.

xprotect

Running the command
sudo xprotect check
in Sequoia is far simpler and quicker, as it checks for just one component’s updates through iCloud. The command connects to XProtectUpdateService in the XprotectFramework private framework in macOS, which in turn fires up CloudKit to connect to iCloud. That fetches a database record and returns the result to XProtectUpdateService, and so back to the xprotect tool as its result. Total time taken is 0.5 second.

As Apple’s intent in changing the management of XProtect and its data appears to be to facilitate more frequent and macOS-specific updates, iCloud is an ideal platform to host this on.

Pinniped with tusks

There is, though, one last thing: what is the walrus? As that might seem an odd question, read these two log entries encountered when browsing what happened with the xprotect check command:

12:08:00.919841 com.apple.cdp XPC Error while fetching walrus status: Error Domain=NSCocoaErrorDomain Code=4099 "The connection to service named com.apple.cdp.daemon was invalidated: failed at lookup with error 3 - No such process." UserInfo={NSDebugDescription=The connection to service named com.apple.cdp.daemon was invalidated: failed at lookup with error 3 - No such process.}
12:08:00.919845 com.apple.cloudkit CoreCDP reports that walrus is undetermined for the logged in account. Error: Error Domain=NSCocoaErrorDomain Code=4099 UserInfo={NSDebugDescription=<private>}

The prospect of an undetermined walrus that can’t be fetched from inside my Mac might seem worrying 🤭

XProtect has changed again in macOS Sequoia 15.2

If your Mac is running macOS Sequoia version 15.2 and you keep an eye on its security data updates, you may have noticed that those have changed again. Perhaps the only way to make sense of what has happened is to understand how XProtect updates work in recent versions of macOS.

Before doing that, there are two important issues to make clear:

  • Despite their names, XProtect and XProtect Remediator are different, and in Sequoia are updated using different mechanisms. XProtect is run on-demand when macOS is about to execute code, while XProtect Remediator runs scans for malware in the background about once a day. This article is only about how XProtect updates, as XProtect Remediator is still updated using the softwareupdate mechanism, and that hasn’t changed.
  • Sequoia obtains XProtect updates through a connection to iCloud that is independent of whether your Mac is logged into your Apple Account in iCloud. So long as your Mac is connected to the internet and nothing blocks its iCloud connections, those updates will be available to it, regardless of whether it’s connected to your Apple Account or iCloud Drive. This is in common with other services that macOS relies on, such as making notarization checks on apps, and updating linguistics data and those used by AI.

macOS Sonoma and earlier

Like other security data updates including XProtect Remediator, XProtect is updated through Software Update, or the command tool softwareupdate, which is how SilentKnight obtains its updates too. Shortly after starting up, and at least daily after that, your Mac’s softwareupdated service contacts Apple’s software update servers and checks whether there are any updates available. If there are, they’ll be automatically downloaded and installed, provided that’s enabled in System Settings.

In this case, XProtect is installed in its traditional place, in /Library/Apple/System/Library/CoreServices as XProtect.bundle.

mcOS Sequoia 15.0-15.1.1

For this brief period, XProtect updates have been available using either of two methods.

The traditional method using Software Update or softwareupdate continues to install the XProtect bundle in its old location, where it could still be used, but the XProtect service in Sequoia expects to find it in a different location, in /private/var/protected/xprotect, where it’s still known as XProtect.bundle.

The new method installs the XProtect bundle directly into its new location, and doesn’t obtain it using Software Update or softwareupdated, but an undocumented service that connects to iCloud instead. That appears to activate at least once a day, independent of softwareupdated, and silently checks for XProtect updates in iCloud. When it finds one, that’s installed directly to /private/var/protected/xprotect but not the traditional location in /Library/Apple/System/Library/CoreServices. It’s therefore perfectly possible for the two XProtects to get out of sync, although the only one that Sequoia uses is that stored in the new location.

Sequoia also introduces a new command tool xprotect to help manage these new updates. If you run
sudo xprotect update
and the version of the XProtect bundle in the traditional location is greater than that in the new location, then the newer version will be copied to the new location to install it there as an update, instead of having to wait for the update through the new iCloud mechanism. This also provides redundancy, but relies on the two sources providing identical updates.

To add a final twist of confusion, if xprotect copies the XProtect bundle from the traditional to the new location, it does so by copying its contents, so its version number changes but the bundle itself retains its previous date of creation and modification, which can prove thoroughly confusing. The lesson here is to check the bundle by its version number, not by its dates.

macOS Sequoia 15.2 and later

In the latest release of Sequoia, the traditional method of updating XProtect is no longer used. If softwareupdate were to download and install an update, then it will only end up in the traditional location, and xprotect update can’t use that to update the new location.

In normal use, this means that the user can’t update XProtect until that new version is made available from iCloud. This ensures that the only versions provided to Macs running 15.2 and later are those intended to be used in Sequoia, but it also means that any delay in providing those via iCloud will leave Macs without the latest update.

Apple has modified the xprotect command to provide one let-out, though: use
sudo xprotect update --prerelease
and it “will attempt to use a prerelease update, if available.” Note that still can’t use a traditional XProtect update installed in the traditional location, and still has to be able to obtain the update from iCloud, but it might work when xprotect check and xprotect update can’t offer a new update yet. Note that Apple describes that early-bird version as prerelease, and it shouldn’t therefore be used as if it’s a regular release, and only for good reasons.

SilentKnight

Because of these changes, as of Sequoia 15.2, SilentKnight is no longer able to provide any assistance in updating XProtect, and its next version 3.0 will also be unable to do anything more useful than inform you that your current version of XProtect is up to date, or out of date. If it is out of date, then it’s up to you what you decide to do about it. In most cases, that should be to leave well alone and let macOS handle the update as it’s intended to.

Because time of release of XProtect updates through softwareupdate and iCloud differ, and vary around the world, availability from one source is no guarantee that the other will also be offering that update. For example, on 17 December, the softwareupdate version became available by about 1830 GMT. A prerelease version for 15.2 was available several hours before that, but the regular release wasn’t available from iCloud until early the following day. If Apple is intending to fork XProtect data for 15.x from those for older versions of macOS, the two update services will be offering quite different bundles anyway, as we experienced briefly in the early days of Sequoia.

Although I do periodically poll traditional softwareupdate servers for new updates, repeatedly doing so for iCloud updates doesn’t appear a good way to proceed, and could in any case be misleading. On the other hand I don’t think it’s intended that apps like SilentKnight should check for prerelease updates as a matter of course, and abuse of that option could lead to its removal. In other words, Apple wants full control over when your Mac can and will update to new versions of XProtect, presumably because Sequoia will be getting different updates in the future.

Summary

  • If your Mac hasn’t been upgraded to Sequoia, XProtect updates continue as usual, and SilentKnight will continue to be able to find and install them as it has done in the past.
  • If your Mac is running Sequoia 15.0 to 15.1.1, then it will continue to be offered XProtect updates both ways, and should continue to do so until you update it to 15.2.
  • Macs running 15.2 and later now get XProtect updates differently. Neither you nor SilentKnight can alter that, unless you want to try obtaining prerelease updates through the xprotect command in Terminal. While SilentKnight and Skint will warn you when an XProtect update is expected, you should rely on macOS to handle those updates for you.

A simple guide to how XProtect installs and updates in Sequoia

Many of you still seem puzzled as to how XProtect installs and updates in macOS Sequoia. This article tries to make this clear, so you can keep XProtect up to date.

Sonoma and earlier

Until this changed in Sequoia, XProtect has been straightforward: its data is stored in a bundle named XProtect.bundle, in the path /Library/Apple/System/Library/CoreServices, which is on the Data volume so it can readily be updated. When you or macOS downloads an XProtect update, it simply replaces that bundle with the new one. This is shown in the diagram below.

xprotectupd3

Sequoia

XProtect in macOS 15 prefers not to use the XProtect.bundle in its old location of /Library/Apple/System/Library/CoreServices (although it can do if there’s no alternative). Instead, it looks for XProtect.bundle in its new location, /var/protected/xprotect.

However, when you or your Mac use the old update system, including Software Update, softwareupdate or SilentKnight, that still installs the update in the old location, where it won’t normally be used by XProtect when making its checks. What’s supposed to happen is that at least once a day, macOS checks whether there’s a newer update in the old location. If there is, then it should automatically prepare and move that to the new location in /var/protected/xprotect for XProtect to use.

If you want that to happen immediately, then you can run the following command in Terminal:
sudo xprotect update
then enter your admin user’s password. The xprotect command tool will then complete the installation of that update from its old location in /Library/Apple/System/Library/CoreServices into its new location in /var/protected/xprotect.

There’s also a second way that XProtect in Sequoia can be updated, and that’s over a connection to iCloud. If that’s used, then the update is installed straight into its new location, and doesn’t change the XProtect bundle in the old location at all. Although Apple has used that earlier, all XProtect updates since the release of Sequoia have come using the old Software Update system, so have needed to be completed using the xprotect command in Terminal.

This is shown in the diagram below. The blue boxes show the old Software Update system, and the pink boxes are the new parts that ensure the update is installed in the new location.

xprotectupd4

SilentKnight

SilentKnight still works using softwareupdate, and can’t use the new xprotect command for updates yet, because that requires structural changes in the app that will be available in version 3. However, in Sequoia it reports the version of XProtect installed in the new location, as that’s the one that XProtect now uses.

When SilentKnight discovers a new version of XProtect via softwareupdate, it therefore installs that in the old location, in the path /Library/Apple/System/Library/CoreServices. It has no choice but to do that. Once that’s been installed to the old location, the version shown for XProtect won’t change, as that requires macOS to complete the second stage of the installation. You can then either:

  • leave macOS to complete the installation itself, which should happen over the next day or so, or
  • run sudo xprotect update in Terminal, which will complete that update immediately. SilentKnight will then show the updated version number correctly.

Key points

In Sequoia, when XProtect is updated by Software Update, softwareupdate or SilentKnight, you should either leave macOS to complete that installation, or run sudo xprotect update in Terminal if you want it to be updated immediately.

This only applies to macOS Sequoia: Sonoma and earlier still work as they always have done.

❌