Many of you still seem puzzled as to how XProtect installs and updates in macOS Sequoia. This article tries to make this clear, so you can keep XProtect up to date.
Sonoma and earlier
Until this changed in Sequoia, XProtect has been straightforward: its data is stored in a bundle named XProtect.bundle, in the path /Library/Apple/System/Library/CoreServices, which is on the Data volume so it can readily be updated. When you or macOS downloads an XProtect update, it simply replaces that bundle with the new one. This is shown in the diagram below.
Sequoia
XProtect in macOS 15 prefers not to use the XProtect.bundle in its old location of /Library/Apple/System/Library/CoreServices (although it can do if there’s no alternative). Instead, it looks for XProtect.bundle in its new location, /var/protected/xprotect.
However, when you or your Mac use the old update system, including Software Update, softwareupdate or SilentKnight, that still installs the update in the old location, where it won’t normally be used by XProtect when making its checks. What’s supposed to happen is that at least once a day, macOS checks whether there’s a newer update in the old location. If there is, then it should automatically prepare and move that to the new location in /var/protected/xprotect for XProtect to use.
If you want that to happen immediately, then you can run the following command in Terminal: sudo xprotect update
then enter your admin user’s password. The xprotect command tool will then complete the installation of that update from its old location in /Library/Apple/System/Library/CoreServices into its new location in /var/protected/xprotect.
There’s also a second way that XProtect in Sequoia can be updated, and that’s over a connection to iCloud. If that’s used, then the update is installed straight into its new location, and doesn’t change the XProtect bundle in the old location at all. Although Apple has used that earlier, all XProtect updates since the release of Sequoia have come using the old Software Update system, so have needed to be completed using the xprotect command in Terminal.
This is shown in the diagram below. The blue boxes show the old Software Update system, and the pink boxes are the new parts that ensure the update is installed in the new location.
SilentKnight
SilentKnight still works using softwareupdate, and can’t use the new xprotect command for updates yet, because that requires structural changes in the app that will be available in version 3. However, in Sequoia it reports the version of XProtect installed in the new location, as that’s the one that XProtect now uses.
When SilentKnight discovers a new version of XProtect via softwareupdate, it therefore installs that in the old location, in the path /Library/Apple/System/Library/CoreServices. It has no choice but to do that. Once that’s been installed to the old location, the version shown for XProtect won’t change, as that requires macOS to complete the second stage of the installation. You can then either:
leave macOS to complete the installation itself, which should happen over the next day or so, or
run sudo xprotect update in Terminal, which will complete that update immediately. SilentKnight will then show the updated version number correctly.
Key points
In Sequoia, when XProtect is updated by Software Update, softwareupdate or SilentKnight, you should either leave macOS to complete that installation, or run sudo xprotect update in Terminal if you want it to be updated immediately.
This only applies to macOS Sequoia: Sonoma and earlier still work as they always have done.
By now you should have gathered that macOS Sequoia 15.0 has brought changes to XProtect and the way that it’s updated. This article attempts to explain what has changed, and how that affects both macOS and you.
Sonoma and earlier
XProtect’s data is contained in a bundle in the path /Library/Apple/System/Library/CoreServices/XProtect.bundle, on the Data volume. When XProtect scans code to check it for known malware, it uses the Yara rules in XProtect.bundle/Contents/Resources/XProtect.yara, although it first checks unsuccessfully for rules in a file named XProtect2.yara within the bundle that, as far as I know, has never existed.
Updates to the XProtect bundle are provided through the normal Software Update mechanism, and summarised in the diagram below. Each Mac periodically checks Apple’s software update service for new updates. When it finds a version with a higher version number than that installed, the same system downloads and installs the updated bundle, so updating the Yara rules used. That may be performed directly, or through a Content Caching server, when that’s available. Checks for updates and their installation can also be performed manually, as in the command tool softwareupdate, as used by SilentKnight.
Sequoia: locations
macOS Sequoia adds a new location for XProtect’s data bundle, in the path /var/protected/xprotect/XProtect.bundle, also on the Data volume, which contains the primary copy of the Yara rules used by XProtect on that Mac. When XProtect scans code to check it for known malware, it looks for Yara rules in the following order:
XProtect.yara in /var/protected/xprotect/XProtect.bundle
XProtect2.yara in /Library/Apple/System/Library/CoreServices/XProtect.bundle (never found, as that file doesn’t exist)
XProtect.yara in /Library/Apple/System/Library/CoreServices/XProtect.bundle.
Thus, an XProtect bundle in CoreServices acts as a secondary source of Yara rules, and is only used if there’s no primary XProtect bundle in /var/protected/xprotect.
XProtect records in the log which set of Yara rules it uses for each of its ‘direct malware and dylib scans’.
Alongside the XProtect bundle in /var/protected/xprotect is XProtect’s database XPdb, which is also present in previous versions of macOS. This is likely to contain data from XProtect scans.
Sequoia: updates
At least once every 24 hours, XProtect’s Update Service automatically looks for updates to its bundle. This may include a check through softwareupdated, which may check Apple’s software update service for updates available there. If it finds any there, it appears that they are downloaded and installed in the secondary location in CoreServices.
In Sequoia, the primary check is made with iCloud using CloudKit, and is announced in the log in the entry CloudKit update source coming online
This uses a container with the ID com.apple.sear.xprotect-updates, SEAR being the in-house name for Apple’s security teams. Any update found there is ‘considered’ by comparison with the version currently installed in /var/protected/xprotect, and the decision whether to download and install that update is recorded in the log, for example Keeping local update 5273, installed <private>
XProtect’s Update Service appears to apply the usual rule that it will only download and install an XProtect bundle if its version number is greater than that already installed, and won’t attempt to install a bundle with a lower version number.
If the version number of the XProtect bundle in the secondary location in CoreServices is greater than that in the primary location in /var/protected/xprotect, then XProtect’s Update Service may copy the bundle from the secondary location to replace that in the primary location. However, immediately after upgrading to Sequoia, when the primary location has no XProtect bundle, several hours can pass before a primary bundle is installed by XProtect’s Update Service. Over that period, XProtect scans use the Yara rules installed in the secondary XProtect bundle in CoreServices. This is summarised in the diagram below.
Sequoia: xprotect command
Apple has added a command tool to Sequoia, to help manage the new version of XProtect and its unusual habits. As its documentation isn’t complete, here are further details of some of its options.
xprotect version (which doesn’t require sudo) returns the current version of the XProtect bundle in the primary location in /var/protected/xprotect, but not the version of the secondary bundle in CoreServices. If it returns a version of 0, that means that no XProtect bundle is currently installed in /var/protected/xprotect, although XProtect will still be able to check against the Yara rules in the secondary bundle.
sudo xprotect check (requires sudo) checks and returns the current version of XProtect available from iCloud, not that available from Apple’s software update service. If XProtect has already copied a bundle from the secondary to primary locations, then the version reported from iCloud may be lower than that reported by xprotect version.
sudo xprotect update (requires sudo) requests the latest XProtect version from iCloud. If its version is greater than that in the primary bundle, then that update will be installed automatically. It may also check the version of the secondary bundle, and is likely to copy-install that to the primary location if that version is greater than the primary bundle.
Sequoia: version therapy
The release of Sequoia was accompanied by a series of changes in the availability of XProtect versions for different macOS:
13 Sep (approx) XProtect version 5273 available from Software Update Service for Sequoia only
16 Sep macOS 15.0 released, with version 5273 available from Software Update Service for Sequoia only; upgraded Macs updated to 5273 by copying from secondary to primary locations; 5273 not provided from iCloud, where 5272 remained the current version
18 Sep Software Update Service resumed delivery of 5272 to Sonoma and earlier
18 Sep Software Update Service started delivery of 5274 to Sonoma and earlier; 5273 no longer available for Sequoia, with 5272 still available from iCloud
24 Sep Software Update Service delivered 5275 for Sequoia; no change to Sonoma and earlier, and 5272 still available from iCloud.
Thus, those Macs upgraded to Sequoia early, or which had been running a beta-release, were likely to have a primary XProtect version of 5273. Those that upgraded after 18 September are likely to have done so after installing the 5274 update, which will then have been copied to the new primary XProtect location, and prevented them from being updated to 5273. This makes a big difference in terms of Yara rules: although the differences between versions 5272 and 5274 are largely cosmetic, 5273 contains well over 3,000 new lines to perform file size checks on potential malware.
Hopefully Apple will sort all this out in the next round of XProtect updates.
One of my greatest regrets in life is that I never invested in a stock of cards with just four words on them: I told you so. Although I’m often wrong, when I see an obvious problem on its way, it’s disappointing how often it’s ignored. Instead of last week being all about Sequoia, I ended up spending most of my time trying to discover what the hell was going on with XProtect and its updates.
My feeling of unease started before last weekend. My fourth and last-ditch fallback Mac is a lovely MacBook Pro 16-inch 2019. It doesn’t see much use, so in the days before the release of Sequoia I charged it up and updated it to 14.6.1. It was still running an old version of XProtect, 2195 from the end of May, so I was surprised when it didn’t update XProtect, and no update was offered. At roughly the same time, my MacBook Pro M3 Pro running 15.1 beta updated from 5272 to 5273, and other beta-testers reported the same.
Bearing in mind that Apple had told nobody what was going on with XProtect or its updates, when last Monday came and my three other Macs that had been running 14.6.1 were upgraded to 15.0, I was still none the wiser. As with everyone else upgrading to Sequoia, any existing XProtect bundle was ignored by the new version of XProtect, and when asked what version of its security data it was running, it replied 0 (zero), indicating that it had no such data. That was a surprise to many, and something not anticipated by any information from Apple.
If you weren’t using SilentKnight, you’d then have to guess either that macOS would somehow fix that before you ran any newly downloaded apps, or that there would be a new command tool named xprotect, whose man page also doesn’t provide any guidance as to what to do.
For those who checked their new Sequoia installation with SilentKnight may have been just as surprised to see the XProtect version of 0, but at least they were guided to run the xprotect command tool, and force an update. Even then, confusion reigned when some Macs were updated to 5273 while others were left at 5272. Either has got to be better than 0, though.
Meanwhile, those who had updated to 14.7 or 13.7 got no XProtect update at all. If those Macs had already installed 5272, then that’s where they stayed, with no sign of 5273, which appeared exclusive to those who had ventured up to Sequoia. But if those Macs were still using an older version of XProtect’s data, there was no update available to take them to 5272, which others had installed when it had been released at the end of last month (28 August).
By the end of the following day, 17 September, this was the state of play with XProtect data:
those running Sequoia could have no XProtect data at all (‘version 0’), 5272, or 5273;
those running Sonoma 14.7 or any earlier version of macOS could have 5272 if they had updated before 13 September, or any older version if they hadn’t.
Then late on 17th, or early on 18th depending on your time zone, everything changed again. Apple’s software update servers started offering 5272 to everyone, and as an added bonus, those running Sequoia might also be offered 5273, although the new iCloud update service seemed content just to offer 5272. Within 24 hours, Apple released yet another XProtect data update, to version 5274, only this time it wasn’t made available to Macs running Sequoia. By the end of 18 September, the state of play had changed to:
those running Sequoia could have no XProtect data at all (‘version 0’), 5272, or 5273, but not 5274;
those running Sonoma 14.7 or any earlier version of macOS could have 5274 if they had updated on 18 September, or any older version except 5273 if they hadn’t.
Does any of this make a difference, though: just what came in the updates to 5273 and 5274?
For Macs running macOS Sonoma or earlier, 5274 is just a renaming exercise, and doesn’t materially affect XProtect’s ability to detect malware. Perhaps it was a placebo after all.
The new Yara definitions added to 5273 are very different, though. As I described in my announcement here: “This adds Yara definitions for MACOS.DOLITTLE.CT, MACOS.SHEEPSWAP.CT and MACOS.SOMA.CT using a new format of rule, with each rule given a UUID and listing SHA256 hashes of file size.” Here’s a short excerpt: rule XProtect_MACOS_SOMA_CT
{
meta:
description = "MACOS.SOMA.CT"
uuid = "DA584E59-A152-4E5B-A906-D354144DCA69"
condition:
hash.sha256(0, filesize) == "59e01f6f925af0643f4751191d28eab11ffb014412cd66ece8e5c77ba082977a" or
following which are a further 3,123 hashes, whereas MACOS.DOLITTLE.CT has just six.
Clearly, Sequoia’s XProtect is quite a different beast from that in Sonoma. It looks like Apple has forked XProtect data files, the Yara definitions in particular. It’s now likely that each future XProtect data update will either be destined for Sonoma and earlier, or specifically for Sequoia. Keeping them in the same version numbering sequence is only going to lead to greater confusion, unless Apple only intends releasing further updates for macOS 15 and later.
As Apple continues to provide absolutely no information about this, I fear that we’re just going to be left stumbling on in the dark, wondering what’s going on with one of the primary security defences in macOS. It must be time to order a new batch of I told you so cards.
macOS and its smaller security updates are widely announced, here and across many other sites supporting Apple products. What should you do, though, when you know updates have been released by Apple but your Mac can’t find them, or when it tries to install them and fails?
Before Sequoia, almost all software updates are normally fetched by Software Update in System Settings/Preferences, or alternatively at the command line by softwareupdate, also used by my free SilentKnight. They work through the softwareupdated service that should be running in the background. If you run a local Content Caching server, then softwareupdated should automatically connect to that and ask it for the update; otherwise, it tries to connect to Apple’s software update servers over the internet. Although this chain is usually reliable, it has several points of weakness.
Sequoia brings greater complexity, in that one its most important security data updates for XProtect is intended to be delivered over a CloudKit connection with iCloud, although those updates can still arrive from Software Update as well. However, when delivered through softwareupdated the XProtect bundle is installed but not ‘activated’ for XProtect’s use. To do that, open Terminal and enter the command sudo xprotect update
then authenticate with your admin password when prompted.
Update not found
You open Software Update or SilentKnight, and are told that your Mac is up to date, although it’s still running the older version of macOS, or hasn’t installed a smaller security update.
The most likely reasons for this include:
Apple’s software update servers are in heavy demand, and are temporarily refusing new connections. As Apple tends to release a lot of updates at once, this isn’t uncommon, particularly in the autumn/fall with the new versions of macOS and others. The only solution is to try again later, although sometimes you can kickstart the process by running SilentKnight or softwareupdate. Apple provides a page showing the status of its many internet services, where these are listed as macOS Software Update, but transient problems due to load seldom get reported there.
Your Mac, or its Content Caching server if you’re running one, can’t connect to Apple’s servers because of a network fault. Again the only solution is to try again later, in the hope that the fault has been fixed.
softwareupdated or your Content Caching server aren’t working properly. This is normally rectified by restarting that Mac and trying again once it’s up and running. In some cases, it can require that the client Mac is started up in Safe mode before the update becomes available.
If an update has only just been announced, then the software update servers that your Mac connects to may not be offering that update yet. Availability around the world isn’t instant, and often you’ll find that an Apple silicon Mac can find an update and install it readily, while an Intel Mac on the same network may be unable to discover the same update for another hour or more.
Note that, unless an update is listed as being available, you can’t force it by trying to install the update using its label, either in softwareupdate or SilentKnight.
For updates to XProtect in Sequoia, try opening Terminal and entering the command sudo xprotect check
then authenticating with your admin password when prompted. This should force XProtect management to look for an update. If it finds one, then entering sudo xprotect update
should download it from iCloud and install it. Note that this command is only available in Sequoia. For further information, man xprotect tells you as much as Apple lets you know.
Update fails to install
This is easiest to detect when you use SilentKnight, which will report the update is available, then when you try to install it, you’ll see an error message in the scrolling text window reporting that installation of the update failed, and the component being updated won’t change to the new version number.
If the Software Update pane shows an error, that should provide similar information. Otherwise, to download and install waiting updates you can type softwareupdate -ia --include-config-data (or in El Capitan sudo softwareupdate -ia)
in Terminal, to see the same messages shown by SilentKnight, as that’s also the tool it uses to obtain waiting updates. If you know your way around the Unified Log, you should discover parallel entries there.
By far the most common cause for failure to install updates like this is that something has gone wrong with softwareupdate or softwareupdated, best corrected by restarting your Mac and trying again. If it still doesn’t work, start up in Safe mode and try from there. One of the primary purposes of Safe mode is to resolve problems with updates and updating, whether they’re full macOS updates or small security data updates like XProtect.
If you’re not running a local Content Caching server and still can’t get the update to install, all you can do is wait an hour or two and try again.
Content Caching problems
If you’re running a local Content Caching server, then the problem could now rest with the copy of the update stored in its cache. When the local server downloaded the update from Apple’s software update servers, it may have become damaged. Once that damaged copy has been put into your local server’s cache, that’s the update that it will serve to all your local Macs when they connect to it to obtain the update.
What can make this worse is that, even if you do manage to get the Mac running the Content Caching server to update successfully, that doesn’t mean that it will replace the damaged copy in its cache, which may continue to deliver that same damaged version to all the Macs that try connecting to it.
To confirm this, you can inspect the log, as I’ve described here.
The most immediate solution, which should allow all your local systems to update correctly, is to turn the Content Caching service off in Sharing, shut down the Content Caching server, or isolate that server from the rest of the network. Then update all your other systems, which should download fresh copies of the update directly from Apple’s servers. Once that’s done, you can bring the server back up in Safe mode and try updating it there.
For a period of over six months in 2022-23, updates for XProtect and XProtect Remediator obtained through Content Caching servers frequently failed to install correctly. In that time, the simplest solution was to disable the server before trying to download and install those updates, and to enable it again once all updates had been completed. It’s still not clear where that problem occurred, but it has since been fixed and updates should be reliable now.
I don’t know any way to remove individual updates from the Content Caching server. Apple’s command tool for its maintenance, AssetCacheManagerUtil, only knows how to flush whole caches, using sudo AssetCacheManagerUtil [flushCache|flushPersonalCache|flushSharedCache]
where the commands set the cache to be flushed:
flushCache flushes the entire content cache.
flushPersonalCache flushes all personal (iCloud) content.
flushSharedCache flushes all shared (non-iCloud) content.
Flushing a large cache may not be what you want to do. So long as there’s no storage problem and the update affected was most probably supplied broken, there shouldn’t be any harm in leaving it where it is.
In Sequoia, XProtect’s new updates delivered from iCloud are likely to bypass Content Caching servers altogether, although Apple hasn’t clarified that yet.
Nothing helps
If you’ve worked your way through to the end here but still haven’t solved the problem, contact Apple Support, who can escalate it to someone who can hopefully do something about the problem.
Although we won’t know for sure until Apple releases the upgrade to macOS Sequoia next month, once again it will probably be presented as an update rather than a macOS upgrade. This means that, instead of Software Update downloading a complete Sequoia installer app, if you do choose to upgrade, it will be run through Software Update the same way that it might have updated from 14.5 to 14.6.
Although this is more efficient, resulting in a smaller update and faster completion, it also opens up the possibility of human error: what if you accidentally opt to upgrade, or click on SilentKnight’s Install all updates button? This article explains how you can stop that upgrade from completing, and how you could upgrade using SilentKnight instead of Software Update.
Updating or upgrading macOS
When SilentKnight has completed checking for updates, if there’s a macOS update or upgrade available you can install it if you wish, from within SilentKnight. Although my personal preference is to hand macOS updates over to Software Update in Settings, SilentKnight should also work fine, up to a point.
To test this, I opened a VM running Sonoma 14.6, with XProtect 5270 and XPR 140. When it had found all three updates available, I clicked on the Install all updates button, just as I have always advised you not to! SilentKnight proceeded to download the macOS 14.6.1 update, but once that was complete it failed to install it. It then proceeded to download the XProtect and XPR updates, which it did successfully install on its own.
There was a vague notification that “Some updates could not be installed”, and the VM was left in 14.6, with XProtect and XPR correctly updated.
At that stage, Software Update stated the 14.6.1 update was available, offering a Restart Now button. When I clicked on that, the VM restarted and installed the 14.6.1 update successfully.
SilentKnight doesn’t provide the handy progress indicator that Software Update does, but just turns its busy spinner until the updates have finished. So you may still prefer to install macOS updates using Software Update. However, the end result should be just the same if you let SilentKnight do it, and finish off the installation using Software Update.
Downloading updates
Using another copy of the same VM running Sonoma 14.6 with outdated XProtect and XPR, I set SilentKnight’s settings to download but not install updates, then clicked the Download all updates button.
This left all the updates uninstalled, but there was no sign of them in the standard /Library/Updates folder as documented for softwareupdate. I looked high and low for those updates, but was unable to find them anywhere. I therefore recommend that you don’t use this option until someone has worked out where those downloaded updates are kept.
Unwanted macOS updates
If you click either the Install all updates or Download all updates button and one of the updates is for macOS, that will leave Software Update poised to complete the installation. If you didn’t want to install that macOS update, there is a way that you can now persuade Software Update to forget that it has been downloaded and is waiting, ready to install. This is most useful if you didn’t intend updating macOS, and now want to undo the process.
Shut your Mac down, then start it up in Safe mode. Leave it there for a minute or so, then restart it back into normal mode. Those uninstalled updates should now have been flushed, and Software Update is back to where it started.
Summary
You should now be able to install macOS updates using SilentKnight if you wish. When warned that some updates weren’t installed, open Software Update settings and complete the installation there using the Restart Now button.
Don’t use SilentKnight’s setting to only download but not install updates, as the downloaded updates can’t be found and used.
If you inadvertently click the Install all updates button and want to reverse that for a macOS update, let the download complete, shut down, start up in Safe mode, wait a minute, then restart in normal mode.
These apply to Apple silicon Macs, and are untested in Intel Macs, although there’s no reason to believe they should differ there.
Login
