Normal view

There are new articles available, click to refresh the page.
Today — 17 September 2024Main stream

Sequoia introduces pinning to iCloud Drive

By: hoakley
17 September 2024 at 04:00

One of the unannounced features in macOS Sequoia is, for many who use iCloud Drive, one of its most important, as this upgrade introduces the ability to pin files and folders to ensure they remain downloaded and don’t get evicted. This article explains how to use this feature, exactly what it does, and how it works.

Pin an item in iCloud Drive

Assuming that your Mac is connected to iCloud Drive and you have enabled Optimise Mac Storage, so that files in iCloud Drive can be evicted from local storage, you can use pinning to ensure that designated files aren’t evicted, and remain downloaded and synced with iCloud Drive. This is an important advance for anyone who needs certain files to be kept in local storage, so remaining accessible even when connection to iCloud isn’t possible, such as when you’re travelling.

pinning1

Select the file or folder in the Finder and Control-click to bring up the contextual menu. You’ll see a new item there, Keep Downloaded. Select that.

pinning2

That adds a new icon to the pinned files, with a white downward arrow on a circular grey background. This indicates that file is to be kept downloaded or pinned locally.

pinning3

To unpin an item, select it and bring up the contextual menu. The Keep Downloaded item will be ticked. Select that command, and that toggles the setting off, unpinning the item so it can be evicted from local storage once again.

Pinning only works if your Mac’s iCloud Drive has Optimise Mac Storage enabled, which puts it into non-replicated FileProvider mode, and allows you and macOS to Remove Download or evict files. Its effect is to prevent that from occurring. Pinned files will remain downloaded to your Mac all the time, just as if iCloud Drive were in replicated FileProvider mode, with Optimise Mac Storage disabled.

A file’s pinning setting travels with it if you move that file within the same volume, and may be preserved when it’s moved away, for example by transferring the file by AirDrop. It’s preserved when you make changes to that file, but not when it’s copied, and pinning settings are specific to a local Mac, and not transferred to other Macs or devices connected to the same iCloud Drive.

How it works

Pinning is set by macOS attaching an extended attribute (xattr) named com.apple.fileprovider.pinned to each file that’s pinned. That normally contains a single byte, the character 1 as the single byte 0x31, although changing that to 0 works just as well, so it appears it’s the xattr that’s important, rather than its contents.

pinning4

When a com.apple.fileprovider.pinned extended attribute is attached to a file, it ceases being eligible for eviction, and is kept downloaded, just as if it were in a replicated FileProvider. Unpinning the file strips the xattr.

Xattr flags

I have recently summarised the features of extended attributes, and explained the system of flags they use to determine how those are copied. In addition to adding the com.apple.fileprovider.pinned xattr to those encountered in Sequoia, Apple has changed the xattr flag system to cope with this new xattr. These flags are appended to the xattr name, after a # character, and pinning xattrs are actually named com.apple.fileprovider.pinned#PX to determine when they should be preserved during file copy operations. The X flag is new to Sequoia, and can be used for other xattrs.

This brings the table of available xattr flags to six. Flags can be upper or lower case letters C, N, P, S, B or X, and invariably follow the # separator, which is presumably otherwise forbidden from use in a xattr’s name. Upper case sets (enables) that property, while lower case clears (disables) that property. The properties are now (as of 15.0):

  • C: XATTR_FLAG_CONTENT_DEPENDENT, which ties the flag and the file contents, so the xattr is rewritten when the file data changes. This is normally used for checksums and hashes, text encoding, and position information. The xattr is preserved for copy and share, but not in a safe save.
  • P: XATTR_FLAG_NO_EXPORT, which doesn’t export or share the xattr, but normally preserves it during copying.
  • N: XATTR_FLAG_NEVER_PRESERVE, which ensures the xattr is never copied, even when copying the file.
  • S: XATTR_FLAG_SYNCABLE, which ensures the xattr is preserved during syncing with services such as iCloud Drive. Default behaviour is for xattrs to be stripped during syncing, to minimise the amount of data to be transferred, but this will override that.
  • B: XATTR_FLAG_ONLY_BACKUP, which keeps the xattr only in backups, including Time Machine.
  • X: XATTR_FLAG_ONLY_SAVING, which keeps the xattr only when saving and in backups, including Time Machine (introduced in macOS 15.0).

These must operate within another general restriction of xattrs: their name cannot exceed a maximum of 127 UTF-8 characters.

This new flag doesn’t change the macOS default flags set for different types of xattr, which remain as:

  • com.apple.quarantinePCS
  • com.apple.TextEncodingCS
  • com.apple.metadata:kMDItemCollaborationIdentifierB
  • com.apple.metadata:kMDItemIsSharedB
  • com.apple.metadata:kMDItemSharedItemCurrentUserRoleB
  • com.apple.metadata:kMDItemOwnerNameB
  • com.apple.metadata:kMDItemFavoriteRankB
  • com.apple.metadata:* (except those above) – PS
  • com.apple.security.*S
  • com.apple.ResourceForkPCS
  • com.apple.FinderInfoPCS
  • com.apple.root.installedPC

(* here represents the wild card.)

Until Apple releases the source code for Sequoia’s open source components, the table of copy intents remains unchanged at

  • XATTR_OPERATION_INTENT_COPY – a simple copy, preserves xattrs that don’t have flag N or B
  • XATTR_OPERATION_INTENT_SAVE – save, where the content may be changing, preserves xattrs that don’t have flag C or N or B
  • XATTR_OPERATION_INTENT_SHARE – share or export, preserves xattrs that don’t have flag P or N or B
  • XATTR_OPERATION_INTENT_SYNC – sync to a service such as iCloud Drive, preserves xattrs if they have flag S, or have neither N nor B
  • XATTR_OPERATION_INTENT_BACKUP – back up, e.g. using Time Machine, preserves xattrs that don’t have flag N

as documented in man xattr_name_with_flags. Once I have been able to check the new source code, I will confirm any changes to the intents table.

Summary

  • If you have Optimise Mac Storage turned on for iCloud Drive, you can now pin files so that they aren’t evicted from local storage.
  • To pin a file or folder, use the Finder’s contextual menu (Control-click) and select the Keep Downloaded item.
  • To unpin a file, toggle the Keep Downloaded item off (unticked).
  • Pinning works by adding an extended attribute to each pinned file.
  • A sixth xattr flag has been added, X, to keep that xattr only when saving and in backups.

Apple has released macOS 15.0 Sequoia and security updates to 14.7 and 13.7

By: hoakley
17 September 2024 at 01:13

As promised last week, Apple has released the upgrade to macOS 15.0 Sequoia, together with security updates to bring Sonoma to version 14.7, and Ventura to 13.7. There should also be Safari updates to accompany the latter two.

The Sequoia update is around 6.6 GB for Apple silicon Macs, and 14.7 is around 1.6 GB. For Intel Macs, 15.0 is around 4.9 GB as an ‘update’, and 14.7 is around 860 MB.

Security release notes for Sequoia list around 77 vulnerabilities addressed, including two in the kernel, none of which Apple is aware may have been exploited in the wild. Release notes list 36 vulnerabilities addressed in Sonoma 14.7 here, and there are 30 listed for Ventura 13.7 here.

iBoot firmware is updated to version 11881.1.1, Intel T2 firmware to version 2069.0.0.0.0 (iBridge 22.16.10353.0.0,0), and Safari to 18.0 (20619.1.26.31.6).

After completing the upgrade to 15.0, you are likely to see that the installed XProtect version is 0, in other words that there is no XProtect data. You can leave your Mac to automatically download the required data from iCloud, or manually force it using the command
sudo xprotect update
then entering your admin password. That will normally ‘activate’ the XProtect data previously installed, and set the version to 5272, although that will then need to be updated to 5273 separately. Don’t be surprised if you end up repeating the trip to Terminal to get this to work.

Last updated 1900 GMT 16 September 2024.

Yesterday — 16 September 2024Main stream

Looking ahead to Sequoia’s updates

By: hoakley
16 September 2024 at 14:30

Later today, Apple is expected to release macOS Sequoia 15.0. For those interested in planning their immediate or delayed upgrade, these are my forecast dates for its minor versions over the coming year. Like all the best weather forecasts, this is most accurate for the next 5 days, and those for further into the future are likely to be decreasingly reliable.

Minor version release dates for Sonoma have been broadly similar to those of others since Big Sur:

  • 14.0 – 26 September,
  • 14.1 – 25 October,
  • 14.2 – 11 December,
  • 14.3 – 22 January,
  • 14.4 – 07 March,
  • 14.5 – 13 May,
  • 14.6 – 29 July,
  • 14.7 – 16 September.

Ventura differed mostly because it had a later start date to its cycle, in October, resulting in the delay of 13.1 until December. Subsequent versions thus trailed Sonoma by one, for example with 13.5 on 24 July, against 14.6 on 29 July. Although Apple is believed to have some flexibility in the release dates for minor updates, the timetable for the cycle appears to be fixed well in advance, and is probably already at least pencilled in for Sequoia.

Most minor updates bring new versions of firmware, the kernel and key kernel extensions such as APFS. In between those may be patch updates to fix serious bugs or security vulnerabilities that can’t wait for the next minor version, such as 14.3.1 on 8 February, two weeks after 14.3 and a month before 14.4.

According to Apple’s release notes, the current release candidate for 15.0 has no significant bugs that remain unfixed, and we hope that remains the case.

15.1: October 2024

Apple has already announced that this first ‘minor’ update will bring its AI features, including most significantly Writing Tools. Although those have been in beta-testing for almost as long as 15.0, in terms of changes, the step from 15.0 will in many ways be greater than that from 14.6 to 15.0. However, that only applies to Apple silicon Macs that support AI.

For all Macs, this is likely to bring fixes for some more substantial bugs, although because of the short interval between 15.0 and 15.1, few are likely to be addressed until 15.2.

This update is likely to coincide with new Mac products launched at an as-yet unannounced Mac event in October, where Apple is expected to promote its new M4 Macs as being ‘made for AI’, much in the way that it did last week with the iPhone 16 range.

15.2: December 2024

Turnaround time fixing even straightforward high priority bugs makes it likely that most in 15.0 will be addressed not in 15.1 but 15.2, before Christmas. This will also catch the first fixes and any additional enhancements required by AI, so may well be one of the more substantial updates this cycle. The aim is to give engineering teams a chance to catch up with the vacation without leaving too much to await their return in the New Year.

15.3: January 2025

This update is largely constrained by the effects of the Christmas vacation, but should enable most issues arising in 15.0 and 15.1 to be fixed, leaving Sequoia running sweetly.

15.4: March 2025

This is the major mid-cycle update, that is most likely to contain new and enhanced features, often making it the largest update of the cycle. Apple also seems to use this to introduce initial versions of new features intended to become fully functional before the end of the cycle. One example of this was XProtect Remediator, released on 14 March 2022 in Monterey 12.3, but not really functional until June that year.

Unfortunately, these enhancements can also cause problems, and this update in March has a track record of sporadic more serious bugs, including the occasional kernel panic.

15.5: May 2025

A month or so before the first beta-release of the next major version of macOS, this normally aims to fix as many remaining bugs as possible, and progress any enhancements introduced in the previous update. If you’ve reported a bug before April, then if it’s going to be fixed in this cycle, this is the most likely time; any new bugs reported after this update are most likely to be carried over to the next major release.

15.6: July 2025

This really is the last chance for fixes and feature-tweaks before the next major version is released in September. If all is working out well, this should be the most stable and bug-free release, although in some years late changes have turned this update into a nightmare, and Sonoma required a patch update in early August to address those.

When best to upgrade?

If third-party software, hardware and other compatibility requirements don’t apply, there’s no way to predict which is the best version to choose as an upgrade from previous macOS. Every version contains bugs, some of them may be serious, others may be infuriating and intrude into your workflows. But those aren’t predictable. If you’re unsure, wait a few days after a minor update, or even 15.0, check around with others, and decide then. If you’re really cautious and have an Apple silicon Mac, I suggest you might like to consider upgrading a week or two after the release of 15.1, by which time most of any major issues with 15.0 and AI should have come to the surface.

For myself, I already have my designated beta-testing Mac, a MacBook Pro M3 Pro, running 15.1 beta, and my other three Macs (iMac Pro, Mac Studio M1 Max and MacBook Pro 16-inch 2019) will all be running 15.0 by midnight tonight, I hope. I’ll let you know how I get on.

Before yesterdayMain stream

Last Week on my Mac: 15.0 or wait for 15.1?

By: hoakley
15 September 2024 at 15:00

It’s strange to think that, as we’re wondering whether and when to upgrade to Sequoia, Apple’s engineering teams are already at work on macOS 16. While they’re thinking out what we’ll chew over next summer, you may well be asking if you should upgrade to 15.0 next week, wait for the AI features coming in 15.1 next month, or leave your decision until 2025?

For those with Macs and iPhones that can both be upgraded, iPhone Mirroring is probably the most obviously attractive new feature. It completes the integration of Continuity, and could transform your workflows. Fortunately for such a key feature, it should work with all supported Macs, not just Apple silicon models. There’s one small and temporary disappointment, though, as drag and drop between Mac and iPhone isn’t expected in 15.0, but in an update “later this year”.

The new Passwords app should spare you from wanting to pay for a third-party password manager. This is much more than just shelling out the existing Passwords feature from Safari and System Settings, and at last gives full control over passkeys and other shared secrets in your Keychain in iCloud.

Although some see Sequoia’s new dislike for apps that aren’t notarized (or from the App Store) as an unnecessary burden, for most of us this will raise the bar against running malware and increase our margin of safety. It has been some time since any malicious software has been successfully notarized, and most of the current epidemic of stealers aren’t even signed with a Developer certificate. Instead, they usually prompt the user to open them using the existing Finder bypass, something that no longer works in Sequoia without explicitly and individually giving permission to that app in Privacy & Security settings.

It will be interesting to see how malware developers respond to this challenge, as trying to give the user detailed instructions as to how they can be run without being blocked by Gatekeeper should now arouse the suspicion of even the most careless and inattentive.

While we’re on the subject of security, remember that Sequoia is now the only version of macOS that gets full security updates over the coming year. While Sonoma and Ventura will still get some, if you want the lot then you’ll need to upgrade. Monterey, of course, now gets none at all. This gets more brutal when considering other bugs that aren’t relevant to security: those will only be fixed in Sequoia, not even in Sonoma.

For those who virtualise macOS on Apple silicon, support for Apple ID gives VMs access to iCloud Drive at last, although it stops short of enabling the App Store or its apps, so isn’t as useful as it should have been. There are two important restrictions to this:

  • Apple ID can only be used in a Sequoia guest running on a Sequoia host, and
  • the Sequoia VM has to be built from a Sequoia IPSW file, and can’t be upgraded from a Sonoma or earlier VM.

As long as your Mac stays with Sonoma, you won’t be able to use Apple ID in any of its VMs, including Sequoia. This still leaves us with the paradox that Apple wants us to buy and run apps from its App Store, but VMs are the one place where you can’t use them.

Among the less prominent improvements that have caught my attention are a timed messaging feature of Send Later in Messages, and a batch of improvements in Freeform. If you’ve come to like that relatively new app, you should find Sequoia worth the effort. I’ve also been impressed to see one of the oldest bugs remaining in the Finder has finally been addressed in macOS 15. I’ll be putting the bunting out in celebration after I’ve upgraded on Monday.

As with Sonoma, some of the most important new features haven’t been documented even for developers. Among those are changes to XProtect in terms of its updating and management, and speculation as to how that might affect its function. As I have explained, XProtect’s detection rules have grown enormously over the last few months, and it’s likely that Apple intends improving how XProtect can apply its Yara rules, and making their updating more efficient.

Finally, Sequoia is almost certainly going to be delivered as if it were an update, and won’t download its installer app unless you’re upgrading from a significantly older version of macOS, just as has happened in all recent macOS upgrades. Remember that upgrading macOS these days comes with a one-way ticket: changing your mind afterwards will cost you a lot of time and messing about to step back to Sonoma. However, accidental upgrades shouldn’t be feared. For instance, if you inadvertently click the Install all updates button in SilentKnight and want to reverse that for a macOS update, let the download complete, shut down, start up in Safe mode, wait a minute, then restart in normal mode.

Whatever you choose tomorrow, I hope it works well for you. And in case you’re wondering, if you’ve got an Apple silicon Mac, you’re going to love 15.1.

macOS Sequoia ships next week; here’s a SilentKnight update for it

By: hoakley
11 September 2024 at 14:30

Apple will release macOS 15.0 Sequoia on 16 September, that’s next Monday, alongside iOS and iPadOS 18.0, and upgrades and updates for lesser mortals. Among the latter are Sonoma 14.7 and Ventura 13.7, as I’ll explain later. Sequoia introduces two important changes to security data checked and updated by SilentKnight, for which I have built and notarized another new version of that app, 2.11, which is essential for anyone intending to upgrade to Sequoia, and worthwhile for all running Catalina or later.

What’s coming next week

Apple has just provided release candidates for the following three new versions of macOS:

  • Sequoia 15.0, its first full release,
  • Sonoma 14.7, its first security-only update,
  • Ventura 13.7, the first of its security-only updates for its final year of support.

There’s not expected to be any update to Monterey 12.7.6, which is no longer supported, even with security updates.

The minor version numbers of Sonoma and Ventura will then be the same, the first time this has happened. In previous release cycles, the start of the first year of security-only updates has been with x.6, as it was with Ventura, and proceeded through the year with versions x.6.1, x.6.2, and so on. Over the coming year, we can expect 14.7.1 and 13.7.1, then 14.7.2 and 13.7.2, continuing until Ventura reaches the end of its third and final year of support in a year’s time.

Sequoia 15.1, the first release with AI support, is now expected in October, and continues in beta-testing, alongside AI-enhanced versions of iOS and iPadOS in versions 18.1.

TCC in Sequoia

The TCC database in /Library/Apple/Library/Bundles/TCC_Compatibility.bundle was introduced in Mojave (when it had a different location, of course), and has been updated with each new major version of macOS since. That has now vanished, and I can find no trace of it, nor any apparent substitute. If you run SilentKnight 2.10 in Sequoia, that will be reported as an error, so version 2.11 addresses that by omitting that result both from its display box and the text report below.

silentknight11

XProtect in Sequoia

Since it was first introduced many moons and versions of macOS ago, there has been a bundle named XProtect.bundle in CoreServices, most recently in the path /Library/Apple/System/Library/CoreServices/XProtect.bundle, that has provided data for XProtect scans of executable code and other security services. That bundle has been updated frequently in downloads labelled XProtectPlistConfigData. Although that can still be present in Sequoia, XProtect now uses a completely different source for its data, that is normally updated through iCloud’s CloudKit rather than Software Update.

The result is that your Mac can have an up-to-date XProtect.bundle in the normal location, but XProtect itself may not be up-to-date at all. For example, in fresh installs of Sequoia, XProtect.bundle is usually absent, and the new tool to check its version may report a number of 0.

SilentKnight versions 2.10 and 2.11 have been updated to cope with this major change, which Apple has apparently not seen fit to document (yet). They check the correct current version using a new command tool, and report that version number faithfully. At present, though, SilentKnight isn’t able to update this new form of XProtect. You can either leave macOS to do that itself in its own time, or you can run a command in Terminal to force the update immediately:
sudo xprotect update
following which you’ll need to authenticate with your admin user password.

I intend to address this more completely in SilentKnight version 3, but for the time being this is fully documented in SilentKnight’s Help book and Help Reference, in these latest versions.

SilentKnight, Skint, SystHist, LockRattler

SilentKnight version 2.11 is strongly recommended for anyone intending to update to Sequoia this year, and, as it also fixes a bug in reporting Studio Display firmware in VMs, is worthwhile for those remaining with Sonoma for longer. It’s available from here: silentknight211
from Downloads above, on its Product Page, and through its auto-update mechanism.

Thankfully, as Skint doesn’t check TCC, the current version 1.08 remains fully compatible with Sequoia. The current release of SystHist, 1.20, works well with Sequoia too, and usefully distinguishes between the two different types of XProtect update, XProtectPlistConfigData delivered through Software Update, and XProtectCloudKitUpdate the new one obtained through iCloud instead.

I don’t intend to update LockRattler for the time being. It won’t report the true version of XProtect, but does report that it can’t find TCC or the GKE data. Otherwise it should continue to function as expected in Sequoia.

More to come in Sequoia 15.0

These changes to XProtect are but one of the significant changes that Apple hasn’t yet mentioned. Once 15.0 has been released, I’ll be delighted to provide fuller details of others.

Summary

  • On Monday 16 September, Apple will release macOS 15.0, and security updates 14.7 and 13.7.
  • Monterey is no longer supported.
  • Download and install SilentKnight 2.11 if you’re intending to upgrade to Sequoia this year.
  • Skint and SystHist remain fully compatible with Sequoia.
  • Watch here for further news on Sequoia once it has been released next week.
  • Sequoia 15.1 with AI will be released next month (October).

What is Macintosh HD now?

By: hoakley
2 September 2024 at 14:30

Perhaps you just tried to save a document, only to be told you don’t have sufficient permissions to do so, or attempted to make another change to what’s on your Mac’s internal storage, with similar results. You then select the Macintosh HD disk in the Finder and Get Info. No wonder that didn’t work, as you only have read-only access to that disk. But if you unlock it and try to make any changes to permissions, you see

xpermserror

What’s going on?

Between macOS Mojave, with its single system volume, and Big Sur, the structure of the Mac system or boot volume has changed, with Catalina as an intermediate. Instead of Macintosh HD (or whatever you might have renamed it to) being one volume on your boot disk, it’s now two intertwined and joined together. What you see now as Macintosh HD isn’t even a regular APFS volume, but a read-only snapshot containing the current macOS. No wonder you can’t change it.

Root

Select the boot disk Macintosh HD in the Finder, and it appears to have four visible folders, Applications, Library, System and Users, just like it always did. Press Command-Shift-. to reveal hidden folders and all the usual suspects like bin, opt and usr are still where they should be. That’s the root of the combined System and Data volumes, and what’s shown there is a combination of folders on both volumes, with the top level or root on the Sealed System Volume (SSV).

The contents of those folders are also the result of both volumes being merged together using what Apple terms firmlinks:

  • Applications contains apps installed in your own Applications folder on the Data volume, and those bundled in macOS on the SSV. You can see just the latter in the path System/Applications, where they appear to be duplicated, but aren’t really.
  • Library comes only from the Data volume, and all its contents are on that volume. But inside it, in the path Library/Apple/System/Library are some components that should appear in the main System/Library.
  • System comes only from the SSV, although it has some contents merged into it using firmlinks, such as those folders in Library.
  • Users also comes only from the Data volume, and includes all Home folders for users.

So while the root of Macintosh HD might be in the SSV, much of its contents are on the Data volume, and can be written to, even though the root is a read-only snapshot, thanks to those firmlinks.

Data volume

There are two places that mounted volumes are listed in the Finder: the hidden top-level folder Volumes, where Macintosh HD is just a link back to the root complete with its merged volumes, and in System/Volumes, where what’s shown as Macintosh HD is in fact not the merged volumes, but only the Data volume. You can confirm that by looking at what’s in System/Volumes/Macintosh HD/System, where you only see the parts of the System folder that are stored on the Data volume, and not those stored on the SSV.

What is more confusing there is that System/Volumes/Macintosh HD/Applications is the same merged folder containing both user and bundled apps as in the top-level Applications folder. That’s an artefact resulting from the way that its firmlink works.

But if you open the Get Info dialog on System/Volumes/Macintosh HD, you’ll see the same as with the root Macintosh HD disk, information about the root and not the Data volume.

Mounted in System/Volumes are several other volumes like VM and Preboot, and (depending on whether this is an Intel or Apple silicon Mac) folders such as Recovery and xarts, that you really don’t want to mess with.

Permissions problems

Tackling problems that appear to be the result of incorrect permissions is best done at the lowest folder level. If you’re trying to save a document to the Documents folder inside your Home folder, select that and Get Info on it. Chances are that you are the owner and have both Read & Write permissions as you should. In that case, the problem most likely rests with privacy protection as in Privacy & Security settings. You then suffer Catch-22, as you can only effect changes to those by closing and opening the app, and as you can’t save your document before closing the app, you’re at risk of losing its contents. You may have better luck trying a different folder, creating a new one inside your Home folder, or using the Save As… command instead (which may be revealed by holding the Option key when opening the File menu).

Full layout

In case you’re wondering exactly which folders are merged into the hybrid Macintosh HD ‘volume’, those are shown below in increasing levels of detail, starting with the broad layout.

BootVolGpVentapfs

Then to a simplified version of the full layout.

BigSurIntSimple

Finally, in complete detail.

BigSurIntegrated

Happy navigating!

Last Week on my Mac: XProtect tormentor

By: hoakley
1 September 2024 at 15:00

If XProtect Remediator came of age in macOS Ventura, then it has been XProtect’s turn in Sonoma. Starting from version 2171 with 216 rules in under 3,000 lines in its Yara definitions, it emerged a year later in version 5272 with 347 rules in over 13,000 lines, although mercifully not after 3,100 versions.

I had always assumed that those Yara rules were compiled straightaway into something more tractable for checking executable code, but it seems that each time XProtect performs one of its ‘direct malware and dylib scans’, it first looks for a non-existent Yara file, then uses the rules in the XProtect.bundle, as it reports in the log:
com.apple.xprotect Xprotect is performing a direct malware and dylib scan: <private>
com.apple.xprotect Rule path is not accessible: /Library/Apple/System/Library/CoreServices/XProtect.bundle/Contents/Resources/XProtect2.yara
com.apple.xprotect Using XProtect rules location: /Library/Apple/System/Library/CoreServices/XProtect.bundle/Contents/Resources/XProtect.yara

Apparently, to cope with this explosive growth, and potentially support more frequent tweaks to its growing horde of Yara rules, macOS Sequoia is changing the way that XProtect’s data is updated and managed. A chance find by @L0Psec revealed how this has moved beyond those updates delivered by softwareupdate, and a new command tool xprotect handles this separately in CloudKit.

Last week’s update to XProtect’s Yara file was an experience those beta-testing Sequoia 15.0 or 15.1 must have found profoundly confusing, and I quickly became aware of reports that were changing by the minute.

When XProtect 5272 was first made available through softwareupdate, Sonoma and earlier systems found and installed it as usual, as did some running Sequoia betas. That updated the visible XProtect.bundle in CoreServices, but didn’t update XProtect according to its new xprotect command tool, which still reported the local version of XProtect as 5271. Without knowing how XProtect has changed, the user would most likely see this as a bug.

A little later, I saw reports of Sequoia installations apparently updating spontaneously via CloudKit, using its new mechanism, which did change the version reported by xprotect version.

At this stage, I had a 15.0 virtual machine that had updated ‘correctly’ via CloudKit, and its host 15.1 system that had updated its bundle via softwareupdate, but still wasn’t apparently running the new version afterwards. Those of us who didn’t experience a spontaneous CloudKit update were left in limbo. I had originally changed the version databases used by SilentKnight and Skint to show a correct version of 5272 for Sequoia, and hurriedly had to revert that to 5271 before I became inundated with complaints from those whose Macs hadn’t been able to update.

It then occurred to me to try using the xprotect command to force a CloudKit update on my 15.1 system. I first entered
sudo xprotect check
only to be told that the version available was still 5271. But when I ran
sudo xprotect update
a miracle happened, with the response
Update succeeded: Activated update LocalUpdate[5272]

That command had convinced macOS to ‘activate’ the updated bundle in /Library/Apple/System/Library/CoreServices rather than waiting for it to become available from CloudKit, a feature not mentioned in its man page or usage info. I returned to my version databases to change them a third time, back to 5272.

Previous XProtect updates such as 5271 that were obtained through CloudKit are now identified by SystHist as XProtectCloudKitUpdate, while those obtained by softwareupdate and activated using the xprotect command appear as standard XProtectPlistConfigData, as they do in Sonoma and earlier.

With the release of Sequoia due later this month, the xprotect command tool and XProtect’s new CloudKit updates have already encountered troubled water. If Apple stays true to form and doesn’t mention a word about this change, or its effect on XProtect updates, many of the millions of new Sequoia users could end up falling behind. But as we’re not supposed to know what the latest version is, nor which is currently active on our Macs without taking to Terminal’s command line, maybe most won’t be allowed to notice.

I’d like to think that Apple will explain these changes to users, document its new command tool properly, and ensure that users know the current version of XProtect data, and can check whether their Mac is up to date without having to resort to Terminal or third-party products, perhaps in System Information. Will I be disappointed?

Advanced SilentKnight: updating macOS and avoiding updates

By: hoakley
30 August 2024 at 14:30

Although we won’t know for sure until Apple releases the upgrade to macOS Sequoia next month, once again it will probably be presented as an update rather than a macOS upgrade. This means that, instead of Software Update downloading a complete Sequoia installer app, if you do choose to upgrade, it will be run through Software Update the same way that it might have updated from 14.5 to 14.6.

Although this is more efficient, resulting in a smaller update and faster completion, it also opens up the possibility of human error: what if you accidentally opt to upgrade, or click on SilentKnight’s Install all updates button? This article explains how you can stop that upgrade from completing, and how you could upgrade using SilentKnight instead of Software Update.

Updating or upgrading macOS

When SilentKnight has completed checking for updates, if there’s a macOS update or upgrade available you can install it if you wish, from within SilentKnight. Although my personal preference is to hand macOS updates over to Software Update in Settings, SilentKnight should also work fine, up to a point.

skupdate1

To test this, I opened a VM running Sonoma 14.6, with XProtect 5270 and XPR 140. When it had found all three updates available, I clicked on the Install all updates button, just as I have always advised you not to! SilentKnight proceeded to download the macOS 14.6.1 update, but once that was complete it failed to install it. It then proceeded to download the XProtect and XPR updates, which it did successfully install on its own.

skupdate2

There was a vague notification that “Some updates could not be installed”, and the VM was left in 14.6, with XProtect and XPR correctly updated.

skupdate3

skupdate4

At that stage, Software Update stated the 14.6.1 update was available, offering a Restart Now button. When I clicked on that, the VM restarted and installed the 14.6.1 update successfully.

SilentKnight doesn’t provide the handy progress indicator that Software Update does, but just turns its busy spinner until the updates have finished. So you may still prefer to install macOS updates using Software Update. However, the end result should be just the same if you let SilentKnight do it, and finish off the installation using Software Update.

Downloading updates

skupdate5

Using another copy of the same VM running Sonoma 14.6 with outdated XProtect and XPR, I set SilentKnight’s settings to download but not install updates, then clicked the Download all updates button.

This left all the updates uninstalled, but there was no sign of them in the standard /Library/Updates folder as documented for softwareupdate. I looked high and low for those updates, but was unable to find them anywhere. I therefore recommend that you don’t use this option until someone has worked out where those downloaded updates are kept.

Unwanted macOS updates

If you click either the Install all updates or Download all updates button and one of the updates is for macOS, that will leave Software Update poised to complete the installation. If you didn’t want to install that macOS update, there is a way that you can now persuade Software Update to forget that it has been downloaded and is waiting, ready to install. This is most useful if you didn’t intend updating macOS, and now want to undo the process.

Shut your Mac down, then start it up in Safe mode. Leave it there for a minute or so, then restart it back into normal mode. Those uninstalled updates should now have been flushed, and Software Update is back to where it started.

Summary

  • You should now be able to install macOS updates using SilentKnight if you wish. When warned that some updates weren’t installed, open Software Update settings and complete the installation there using the Restart Now button.
  • Don’t use SilentKnight’s setting to only download but not install updates, as the downloaded updates can’t be found and used.
  • If you inadvertently click the Install all updates button and want to reverse that for a macOS update, let the download complete, shut down, start up in Safe mode, wait a minute, then restart in normal mode.
  • These apply to Apple silicon Macs, and are untested in Intel Macs, although there’s no reason to believe they should differ there.

Why launch constraints can crash apps

By: hoakley
23 August 2024 at 14:30

Some apps may crash when launched because there’s something wrong in the app. In Ventura and later, that might occur because macOS is refusing to run them because of security rules, specifically launch constraints. These were extended in Sonoma to allow any app to limit the code it runs to what should be there, in launch environment and library constraints. This article explains what these are, and how you can recognise when constraints are applied.

Code without constraints

Launching an app without constraints isn’t as unconstrained as that might suggest. It’s still given an environment to run in, with settings such as the user’s Home folder and some standard paths including a temporary folder buried in /var/folders. If you’re interested to see what those can include, Mints has a button to show you its own launch environment.

On top of those, the app is limited by standard permissions as to what it can access without obtaining elevated privileges, and everything is subject to the privacy restrictions imposed by TCC according to the app’s Privacy & Security Settings.

But the app can be run from pretty well anywhere, and can run code from libraries, frameworks and other places as it wishes.

Launch constraints

The first set of launch constraints became obvious if you tried to copy and run from a different location one of the apps bundled in Ventura. This has had its purposes in the past, for example to run Network Utility after Apple first gutted then removed it. Try that with one of Ventura’s bundled apps, and the copy can’t be run from any location apart from the SSV it’s installed in, as it crashes immediately. Look in its crash report and you’ll see something like
Exception Type: EXC_CRASH (SIGKILL (Code Signature Invalid))
Exception Codes: 0x0000000000000000, 0x0000000000000000
Termination Reason: CODESIGNING 4 Launch Constraint Violation

That’s given in a bit more detail in the main log, for Terminal as
AMFI: Launch Constraint Violation (enforcing), error info: c[1]p[1]m[1]e[2], (Constraint not matched) launching proc[vc: 1 pid: 2440]: /Users/hoakley/Documents/00crypt/Terminal.app/Contents/MacOS/Terminal, launch type 0, failure proc [vc: 1 pid: 2440]: /Users/hoakley/Documents/00crypt/Terminal.app/Contents/MacOS/Terminal
ASP: Security policy would not allow process: 2440, /Users/hoakley/Documents/00crypt/Terminal.app/Contents/MacOS/Terminal
xpcproxy exited due to OS_REASON_CODESIGNING | Launch Constraint Violation, error info: c[1]p[1]m[1]e[2], (Constraint not matched) launch type 0, failure proc [vc: 1]: /Users/hoakley/Documents/00crypt/Terminal.app/Contents/MacOS/Terminal

The same happens if you try running a forbidden command, such as /usr/libexec/periodic-wrapper.

Open the app using Apparency and view its Launch Information, and you’ll see the launch constraints that caused this.

apparency2

For the Chess app, those read
(on-authorized-authapfs-volume or on-system-volume ) and launch-type = 3 /* CS_LAUNCH_TYPE APPLICATION / and validation-category = 1 /* CS_VALIDATION_CATEGORY_PLATFORM */
which should give you a good idea that app can only be run from its standard location in the SSV or System volume. To make this even harder, Sonoma’s Finder tries to stop you from even copying bundled apps to other locations, and you now have to be ingenious to try launch constraints out.

Launch constraints were first described by Csaba Fitzl, and he has since compiled a listing of all those known. Those shown for Chess.app are Category 14, and common to other bundled apps. Their effect is to prevent all copies of that app from being launched from elsewhere.

Trust caches

Instead of macOS looking up each binary’s launch constraints from the binary itself, all those constraints are assembled into Trust Caches, where they’re listed by the code directory’s hash (cdhash). To look up the launch constraints for the Terminal app, the system first calculates the cdhash for its code directory, then looks in the Trust Cache for the launch constraints given for that cdhash.

The System volume contains a static Trust Cache that covers all the executable binaries that come as part of the system. That’s locked into read-only storage during the early kernel boot phase of startup. Additional Trust Caches are authenticated to ensure they haven’t been tampered with, and loaded when required. Apple cites the example of the Trust Cache required by the code within macOS software updates (known as the update brain) that runs the process, allowing it to run with platform privileges, as it requires to perform the update. Apple gives further details on Trust Caches in its Platform Security Guide.

Disabling launch constraints

What if you need to ignore those launch constraints imposed by macOS? Because system executables are laid out in the static Trust Cache, there’s no way to modify that, and no way to override it. All you can do is disable System Integrity Protection (SIP), which is required for launch constraints to operate.

Environment constraints

Launch constraints and the Trust Cache system are complete and fully enforced as of Ventura 13.3, and have been extended for use by third-parties in Sonoma. Developers can build dictionaries containing facts and applying operations to them to improve the security of their apps. Constraint dictionaries are either saved in property lists for launchd, or in those used for signing code. These too are associated with cdhashes, use some categories common to other trust caches, and work similarly to protect third-party code such as helper apps.

While they might appear overkill, they can be used to address known security problems, of which the most prominent must be maintaining trust with privileged helper apps and XPC services, which have often proved weak points in app security. Apple provides two detailed articles, one explaining how to define these constraints, the other how to apply them. I suspect that we’ll be seeing more of these in the future.

Controlling System Integrity Protection using csrutil: a reference

By: hoakley
21 August 2024 at 14:30

System Integrity Protection (SIP) is more than one of the frontline security protections in macOS, it’s a whole family of them, including controls over kernel extensions, protection for NVRAM, and more. This article explains how you can manage different features in SIP using the csrutil command tool, primarily on Apple silicon Macs running Sonoma or Sequoia.

SIP fully enabled

Unless you have very good reason to do otherwise, your Mac should have SIP fully enabled at all times. Utilities like SilentKnight check whether this is so, and if SIP isn’t enabled, you need to fix that as soon as possible. Start your Mac up in Recovery mode, pass through to Options, and open Terminal from the menu there. Enter the following command:
csrutil enable
and then authenticate when prompted. When you have quit Terminal, open Startup Security Utility from the same menu. You should there be able to set your Mac’s security back to full, unless you need to run it at reduced security to load a third-party kernel extension. Once that is set, restart your Mac and it should have returned to normal with SIP fully active, as SilentKnight should confirm.

SIP’s features

Its original and core feature is to protect much of the system from being changed, even by those with root privileges, referred to here as Filesystem Protections, and introduced in El Capitan, in 2015. This is now extensive, and ranges from whole directory trees in the System volume down to individual extended attributes such as com.apple.macl on otherwise unprotected files.

SIP is also responsible for enforcing requirements for the signing and notarization of non-system kernel extensions, and protection of the integrity of the kernel, respectively termed Kext Signing and Kernel Integrity Protections. It imposes restrictions on the use of kernel debuggers in Debugging Restrictions and the DTrace tool in DTrace Restrictions.

Restrictions over changes that can be made to the NVRAM and boot arguments it can apply to the kernel are controlled by SIP’s NVRAM Protections and Boot-arg Restrictions, while the Authenticated Root Requirement should be self-explanatory. Other features of SIP, including the enforcement of the hardened runtime required of notarized apps, are most probably part of its Apple Internal features.

SIP partially disabled

Although there may be occasions when you need to disable SIP completely, those should be extremely rare. In most cases, you should aim to disable only the features that require it, then return SIP to fully enabled as soon afterwards as possible. One good way to remind yourself that you have temporarily disabled SIP is to write that on a sticky note and put that on your Mac’s display until you’ve enabled SIP again. Don’t leave it to memory, as you’re likely to procrastinate and forget.

To disable SIP completely, start up in Recovery, set Startup Security Utility to Reduced Security, and open Terminal. There enter the command
csrutil disable
confirm that’s what you really want to do, and authenticate. When you restart your Mac, SIP should then be disabled, as you can confirm in Terminal using
csrutil status

Finer control is separated out into its individual features:

  • Apple Internal, disabled in XNU by CSR_ALLOW_APPLE_INTERNAL, and only disabled when SIP is fully disabled
  • Kext Signing, disabled by CSR_ALLOW_UNAPPROVED_KEXTS, abbreviated in csrutil to kext
  • Filesystem Protections, disabled by CSR_ALLOW_UNRESTRICTED_FS, abbreviated to fs
  • Debugging Restrictions, disabled by CSR_ALLOW_KERNEL_DEBUGGER and CSR_ALLOW_TASK_FOR_PID, abbreviated to debug
  • DTrace Restrictions, disabled by CSR_ALLOW_UNRESTRICTED_DTRACE, abbreviated to dtrace
  • NVRAM Protections, disabled by CSR_ALLOW_UNRESTRICTED_NVRAM, abbreviated to nvram
  • BaseSystem Verification, abbreviated to basesystem
  • Boot-arg Restrictions, disabled with nvram
  • Kernel Integrity Protections, disabled with kext
  • Authenticated Root Requirement, disabled by CSR_ALLOW_UNAUTHENTICATED_ROOT, managed separately using csrutil authenticated-root disable and enable.

To enable SIP with one of these features disabled, use a command of the form
csrutil enable --without [feature]
where [feature] is the abbreviated name of the feature to be disabled, such as kext. Chain those options to disable multiple features, such as
csrutil enable --without kext --without nvram
which enables SIP with the exception of Kext Signing and Kernel Integrity Protections, and NVRAM Protections and Boot-arg Restrictions.

Other configuration flags in XNU that don’t appear to be directly supported by csrutil include: CSR_ALLOW_TASK_FOR_PID, CSR_ALLOW_DEVICE_CONFIGURATION, CSR_ALLOW_ANY_RECOVERY_OS and CSR_ALLOW_EXECUTABLE_POLICY_OVERRIDE. Those should be disabled when SIP is fully disabled, though.

Authenticated Root Requirement is managed separately from other SIP controls. To enable it, use the command
csrutil authenticated-root enable
To disable it,
csrutil authenticated-root disable
and to check its status
csrutil authenticated-root status
Note that fully enabling SIP also enables Authenticated Root Requirement.

The following summary table should help you use these complicated options for control of SIP features using csrutil.

csrutilopts

To disable a specific feature, use the --without option for that abbreviated feature name, or the authenticated-root disable option. Note that running csrutil enable also enables the Authenticated Root Requirement. If you need to disable that, use the command to do so after setting other SIP features using csrutil enable --without.

These commands should be good to use on Apple silicon (and Intel) Macs running Sonoma, and don’t appear to have changed in Sequoia.

Although you can check SIP status in regular user mode, making changes to it should be performed in Recovery mode, and in Apple silicon Macs any disabling of its features is likely to require Reduced Security to be set in Startup Security Utility. Status reports for most combinations of disabled features will warn that the combination being used may not be supported in the future; the only common exceptions to that are when SIP is fully enabled or fully disabled.

Thanks to Ryan for adding CSR_ALLOW_TASK_FOR_PID to the control list given above.

Sonoma’s unfinished business: exclaves, conclaves and the kernel

By: hoakley
20 August 2024 at 14:30

It’s not uncommon for mid-cycle releases of macOS to gain new features in preparation for the next major version. Perhaps the most fundamental and significant added to Sonoma 14.4, together with iOS 17.4, iPadOS 17.4 and watchOS 10.4, are exclaves. As far as I’m aware, this is the first time this term has been used in computing, so before trying to explain what they do for our Macs, I’ll demystify its established use in geography.

Definitions

An enclave is a country that’s entirely surrounded by another country, and it’s easy to see how that matches its meaning in the Secure Enclave, which handles our most important secrets, and provides FileVault encryption. Those are highly secure services provided by a separate processing unit within the main Apple silicon chip, or the T2 in recent Intel Macs.

An exclave is a fragment or part of a country that’s separated from the main part, and lies within one or more countries other than its original country. Although I haven’t come across an explanation of what this represents in computing, for these purposes I assume that an exclave is a resource that’s separated from its parent, for example a small portion of Secure Enclave that’s outside the main part, supporting a virtual machine, perhaps.

exclaves1

Although I hadn’t previously come across the term conclave in computing, Apple is now using it in recent versions of the XNU kernel. This isn’t derived from geography, but is normally used for a private meeting or close assembly, most specifically the meeting of cardinals that elects the next Pope. While there’s scant evidence as to its meaning in this context, I’ll assume that it’s based on that general concept.

Secure Enclaves

Apple introduced its first true Secure Enclave for Macs in the T2 chip in December 2017. This chip consists of a four-core main CPU derived from the A10 used in the iPhone 7 and contemporaneous iPads and iPod Touch, with a 32-bit Arm CPU running a completely different operating system, sepOS (a custom version of the L4 microkernel), dedicated to handling and working with the secrets protected by its Secure Enclave. That has its own secure EEPROM storage, an AES engine to perform hardware-accelerated encryption and decryption for the internal SSD, and more.

The first M1 models released in November 2020 came with an integral Secure Enclave, as have all subsequent M-series chips. These are a significant improvement on their predecessors in the T2, adding replay prevention, a second-generation Secure Storage Component, and more. They continue to run their own operating system, sepOS. Support for biometric authentication using Face ID is implemented as a secure mode in the main Neural Engine (ANE), but apparently remains unused in M-series chips. That secure ANE mode is an exclave of the Secure Enclave.

Exclaves

Prior to macOS Sequoia, macOS virtual machines (VMs) running in lightweight virtualisation on Apple silicon are unable to use Apple ID or to access iCloud, because they’re unable to access any protected secrets from the host’s Secure Enclave. In macOS 15 and later, creation of a VM running macOS 15 or later can configure an identity derived from the host Secure Enclave, enabling access to resources requiring Apple ID including iCloud. This is accomplished using an exclave of the Secure Enclave.

As of macOS 14.6.1, exclaves and conclaves are implemented in and supported by three kernel extensions:

  • ExclaveKextClient.kext (introduced in macOS 14.0)
  • ExclaveSEPManagerProxy.kext
  • ExclavesAudioKext.kext

and two Private Frameworks:

  • CoreSpeechExclave.framework
  • libmalloc_exclaves_introspector.framework

These are currently all in version 1.0. If you’re running a beta-release of Sequoia, you might find it interesting to check the version numbers of those components in their respective /System/Library folders.

According to the few comments in the XNU source code, exclaves provide a fixed static set of resources to the XNU kernel. Examples given include conclave managers, services like Apple ID for VMs, named buffers and audio buffers. These resources are named and have a corresponding identifier shared between XNU and the enclave. Those are discovered during the boot process, and made available in a table at two levels. The root table assembles resources by their scope, while secondary tables list actual processes.

For example, a root table entry for the domain com.apple.conclave.a might link to a secondary table listing an audio buffer, an audio service as a conclave manager, and additional buffering. In the case of audio exclaves, they might be used to connect a system extension running with user privileges to privileged access in the kernel and its extensions.

Information about exclaves is hard to come by. Currently, it appears that any log entries originating from exclaves are shown as coming from the kernel. The only time you’re likely to come across more specific information is in the panic log generated following a kernel panic involving an exclave, when the normal content should be supplemented by information from the exclave. These might have improved in Sequoia.

From the evidence in XNU source code and elsewhere, exclaves are intended to support three subsystems:

  • audio system extensions, allowing them to replace third-party kernel extensions, via ExclavesAudioKext.kext
  • Apple ID in macOS VMs via ExclaveSEPManagerProxy.kext
  • secure ANE via ExclavesAudioKext.kext.

Although support for each of these first appeared in macOS 14.4, the second of them, at least, is already functional in Sequoia, but hasn’t been back-ported to Sonoma, restricting its use to Mac hosts and VMs running Sequoia or later.

There are also multiple references to Tightbeam, which doesn’t appear to have been released in open source. This refers to highly focussed lasers used in communications in The Expanse sci-fi. In macOS, it first appeared as the name of a Private Framework released in macOS 13.0, and supported since then. There’s still a great deal more to be discovered about exclaves, conclaves and XNU.

References

Apple’s Platform Security Guide, which explains Secure Enclaves in detail, but doesn’t mention exclaves or conclaves.
Using iCloud with macOS VMs (Apple).
XNU open source on GitHub, see osfmk/kern/exclaves* for instance.

A proposed new interface for SilentKnight 3

By: hoakley
17 August 2024 at 15:00

At the end of this year, my free utility SilentKnight will celebrate its eighth birthday (including its time as LockRattler), hopefully by then in its third app and third version. It originated after a gaffe when a whole batch of brand new MacBook Pros were delivered with their System Integrity Protection (SIP) disabled. At that time the only way to test that was at the command line.

scripting54

By December 2016, LockRattler version 2.0 was already running a suite of nine checks including SIP status. In those days, XProtect had little to do with checking for known malware, and was mainly concerned with ensuring the latest version of Adobe Flash Player was installed.

lockrattler4rel01

With version 4.0 a year later, LockRattler had assumed its familiar layout, and had begun checking settings for Software Update, although it couldn’t run them itself.

lockrattler414

Another year had passed when LockRattler reached version 4.1, and could both check for and install all pending updates.

lockrattler4181

Version 4.18 of December 2018 added the use of red text to draw attention to changes and discrepancies, and LockRattler hasn’t changed a great deal since.

The following year, I pursued goals to automate checking for updates and firmware versions, and simplify LockRattler’s interface in a new app, initially named EFIcienC.

eficienc06

By the time it had reached version 1.0b2 in July 2019, it was recognisable as SilentKnight, its name when it was released later that month as version 1.0.

silentknight01d

silentknight11401

Support for Apple silicon Macs was incorporated early, and finalised in version 1.14 in November 2020.

sk204

SilentKnight 2.0 was released in October 2022, and continues to support macOS up to Sequoia.

As I have explained, macOS 15 changes the management of XProtect and its updates, requiring substantial internal changes in SilentKnight, so I’m now working on version 3. I think the time is right to revisit and redesign its interface, and if possible implement it using SwiftUI so that it becomes more dynamic and adaptive. I have already been experimenting with that, and offer these ideas for your debate.

Both LockRattler and SilentKnight overwhelm with information that is often duplicated or redundant. Their fixed view layouts operate the same for all models of Mac, with the sole exception of information about Studio Displays. I’m keen to move to a more dynamic interface that delivers important information cleanly and clearly, without repetition.

silentknight31

When you first open SilentKnight 3, it should inform you what is right, and what isn’t, so categories in which all entries are as expected are left unexpanded, and only the one that has an unexpected result is automatically expanded, for you to see the abnormal result, shown with the result that SilentKnight expected. These functional groupings correspond closely to those in Skint.

silentknight32

Expanding all the entries shows a complete summary of information about that Mac at the time that SilentKnight last ran its checks, as given in the footer. These are set in a scrolling view and expand within the window as you prefer.

It’s my intention that this new version will list available updates by their label, and allow you to select which you want to download and install, with an easy option to install all of them. That should work around the problems of updating security data updates without updating or upgrading macOS itself. It should also do away with the Install Named Update feature in previous versions, without risking confusion.

Version 3 will also come with a refreshed app icon. The current one identifies EFI and MRT, both of which are increasingly becoming historical remnants now.

silentknight33

Finally, despite what I said earlier, I hope to make this available with support for late versions of Sonoma, as well as Sequoia.

I welcome your comments and suggestions, please.

Last Week on My Mac: What is happening with XProtect updates?

By: hoakley
11 August 2024 at 15:00

Despite the worst efforts of Elon Musk to destroy everything good that remains of Twitter, it came to my rescue yet again when I recalled a tweet from @L0Psec from 12 June. That had announced a new command tool he discovered in early macOS Sequoia betas that also solved the mystery of what had gone wrong last week in my MacBook Pro’s security data updates.

I hadn’t had time to investigate the new command back in June, but when I noticed that my beta-test system had failed to update to XProtect version 5271 and wouldn’t even offer that update, it occurred to me a tool named xprotect might cast light on this mystery. Sure enough, when it assured me that the update had been installed after all, I could piece together what had happened and search Apple’s release notes unsuccessfully for an explanation. At some time between 23 July when version 5270 was released, and 6 August when it was replaced by 5271, Sequoia’s mechanism for updating XProtect’s data had changed completely without so much as a brief warning.

In that period of a fortnight, XProtect stopped behaving as it had for the last 15 years, since it was introduced in Mac OS X 10.6 Snow Leopard in August 2009. The version number of XProtect.bundle in CoreServices became a relic of the past, and didn’t show that actually installed. Software Update and softwareupdate, which had happily delivered hundreds of previous updates, had now fallen silent about XProtect.

Macs running older versions of macOS still found and installed the latest version, but not those running Sequoia. When it did appear, among the many listed by System Information in Installations, it was named as XProtectCloudKitUpdate, implying it had been downloaded from iCloud using CloudKit. I have later confirmed that obtaining this update doesn’t require the user to be signed in to iCloud, and it has joined the army of maintenance services using iCloud servers.

Updating XProtect in Sequoia

If you’re not using the latest version of SilentKnight, which now does this automatically, you can check the version of XProtect data installed on your Mac using the command
xprotect version

If that or SilentKnight reports an older version than expected, and you want to manually check and install any available update, rather than using softwareupdate, use
sudo xprotect check
Then, if there is an update available, obtain and install it with
sudo xprotect update
both of which will require you to authenticate, as they must be run with elevated privileges.

I am working on a new version of SilentKnight that will save you the trouble of using Terminal to do that. Details of this new command tool xprotect are available in its man page, or from
xprotect -h
although neither explains how this has changed, or why.

Why change?

The early years of XProtect saw emphasis on blocking vulnerable and exploited versions of Adobe Flash Player, and its Yara detection rules developed only slowly. When Adobe finally killed Flash Player at the end of 2020, attention turned to XProtect’s role of detecting and blocking other malicious software when it was first launched.

Its Yara rules grew steadily, as did the size of its XProtect bundle. Version 2109 from 27 September 2019 was 228 KB, and had risen to 2.5 MB by the update to 2178 of 4 January this year. This growth has accelerated lately, and version 5271 released on 2 August reached a total of 3.2 MB. The number of Yara rules contained in that bundle has exploded since the start of this year, with version 2192 on 23 April adding no less than 74 new rules tackling Adload malware, all in a single update.

Releasing new versions through Software Update is a slow and complex process, geared better to low frequencies. Before 2020, XProtect had usually been updated every month, but this year alone there have been 20 updates in less than eight months. Updating XProtects’s Yara rules using iCloud should be quicker, more efficient, and capable of promulgating changes more frequently. Apple could issue new rules as they’re developed and tested, then provide summary updates for Sonoma and older macOS to catch up every 2-4 weeks.

Presumably, these new iCloud updates transfer their payload as binary data rather than verbose Yara text definitions. If they do use CloudKit, then they could directly update a Mac’s security database, much as apps using CloudKit already do, and as used to update notarization data.

Older macOS

Alongside its sibling XProtect Remediator (XPR), XProtect is the front line of Apple’s campaign against malware. XPR was introduced in macOS Monterey two years ago and backported to Catalina and Big Sur. As it’s more of a standalone service, that doesn’t appear to have required much change in those older versions.

This new delivery mechanism for Sequoia is more likely to require internal surgery to security sub-systems, and appears less likely to be offered in Sonoma or Monterey. There are still many Macs running older versions of macOS no longer receiving macOS security updates, and I expect Apple will want to continue offering them more traditional updates for the foreseeable future. Those will also enable security researchers to keep a watch on which malware XProtect can detect using the rules in its Yara file.

Informing beta-testers

I’d like to thank @L0Psec for being the only person to draw attention to what would otherwise have appeared a worrying situation, and to remind Apple of the need to keep beta-testers informed. We’re all keen to keep our test systems well-protected, and should have been warned of this change, and told of the new command tool, rather than hearing about it in the scarred remains of what used to be Twitter. I hope this will be rectified for the next public beta-release, or there could be an avalanche of Feedback reports.

It’s no good telling us that XProtect “uses YARA signatures, a tool used to conduct signature-based detection of malware, which Apple updates regularly”, then those updates are obfuscated.

Gatekeeper and notarization in Sequoia

By: hoakley
10 August 2024 at 15:00

There has been a recent outcry over one quietly mentioned change coming in macOS Sequoia, making it harder to run apps that haven’t been notarized by Apple. As there’s some confusion as to exactly what’s going on, this article explains how this should work, and what benefits notarization brings in return for this added inconvenience.

How it used to work

Although Apple has required developers to notarize their macOS apps for several years, starting back in Mojave 10.14.5, it has so far done little to enforce this requirement, except in special cases such as kernel extensions. The first of my free apps to be notarized was LockRattler, back at the end of July 2018, over six years ago, when Apple referred to making notarization obligatory at some time in the future.

In Sonoma and earlier, any app that hasn’t been notarized, but has been quarantined because it has been downloaded from the internet (or a similar cause), won’t open first time from a normal double-click. Instead the user has to select the Open command from the Finder’s contextual menu, resulting in a dialog giving the user the choice of opening the app in spite of it not being notarized and ‘checked for malicious software by Apple’. Once that first run has been completed successfully, no further prompts result, and the app runs normally thereafter.

How Sequoia will work

Changes are only enforced on apps that have been quarantined because of the way they arrived on that Mac. If an app doesn’t have a quarantine extended attribute with the quarantine flag set, although notarization is still checked by Gatekeeper, the app is allowed to run without any additional action by the user. This also allows you to continue to build your own apps locally, sign them using ad hoc certificates, and run them as before. Where this could get more inconvenient is with modes of transfer such as AirDrop that do put apps into quarantine, although they may not have been outside your local network.

Gatekeeper will then refuse to run for the first time those apps that aren’t notarized, if they’re in quarantine. The only way that you’ll then be able to launch an unnotarized app is by adding it to a list of exemptions using Privacy & Security settings. That’s performed on an individual basis, per app, as Apple has now announced to developers.

None of this applies to apps supplied by Apple, or third-party apps supplied through the App Store, which are signed by Apple, not notarized, and aren’t quarantined.

There are two main ways to circumvent this process.

You could strip the quarantine extended attribute before trying to run the app for the first time. This is a potentially dangerous workaround, as it bypasses first run checks that could spot malware. For those of us who often transfer our own apps for testing using AirDrop, it’s easy to drop them into a folder, Zip that, then on the recipient Mac strip the quarantine xattr before unZipping the archive. My utility Cormorant can make that even simpler. If you were to opt to do that, you must be absolutely certain of the provenance of the app you’re transferring.

For those who have test systems that frequently need to run unnotarized apps, another potential solution is to disable Gatekeeper on them. Perhaps anticipating a move to do that, Apple has changed how this can be done, making it a two-step procedure, in which you first allow the disabling of checks, then in Privacy & Security settings control how that is implemented. Apple encourages those who need to do this to use installed profiles instead, or handle it through MDM payloads. This has been explored in detail by Brandon Dalton in his account of these changes.

I’m frequently surprised to learn how many Mac users have disabled Gatekeeper checks in the past and forgotten to reinstate them, only to be reminded when they later run SilentKnight for the first time.

There’s also a third method that’s little-known, although in the early years of notarization it was practised widely, and that’s notarizing other developers’ apps. Apple has made it clear that submission of an app for notarization doesn’t have to be performed by its developer, the owner of the certificate used to sign the app with, but anyone with a current developer account with Apple can do so provided they follow the correct process. This is most appropriate for those in enterprise and organisations who need to use third-party products that aren’t yet notarized by their developer.

Why notarization?

Superficially, notarization brings two obvious benefits, in apps being checked for malware by Apple before they’re issued a notarization ticket, and in the ‘hardened’ runtime they’re required to adopt to qualify for notarization. Those remain controversial; in the past some malware was apparently notarized, although that seems to have resulted in more stringent checks being applied. But those overlook its greatest benefit.

Code-signing used to be considered an effective way of ensuring the integrity and provenance of executable code, but has now become too low a bar to be effective. It’s all too easy to strip a signature and resign code, and in any case acquiring developer certificates is neither expensive nor difficult. Although Apple does revoke certificates promptly once their abuse is detected, the interval between first abuse and revocation is often sufficient for most who develop malware. Once one certificate has been revoked, it’s then easy to move on and use another, always keeping one step ahead of Apple.

In recent versions of macOS, executable code is recognised and managed using its cdhashes, a tree of hashes of its components within a bundle, for instance. A notarization ticket contains cdhashes that have been accepted by Apple as a record of that app when it underwent notarization. To verify the integrity and authenticity of any notarized code, all macOS has to do is compare freshly computed cdhashes against those provided in the app and its ticket, and against those in Apple’s notarization records. This presents a much higher bar to those trying to masquerade as or tamper with notarized executable code.

By making it harder for a user to run unnotarized code, the chances of that user inadvertently running malicious code are considerably reduced.

Further reading

Notarization for developers (Apple)
How does Sonoma check an app before launch?

SilentKnight version 2.10 works better with XProtect updates

By: hoakley
9 August 2024 at 14:30

Imagine my surprise earlier this week when I went to ensure my MacBook Pro had successfully installed new versions of XProtect and XProtect Remediator. SilentKnight informed me that XProtect was still at version 5270, and there was no update available to take it to 5271. I then manually checked the version of XProtect.bundle in CoreServices, and that confirmed it hadn’t been updated, but SilentKnight still couldn’t find any update, despite XProtect Remediator updating as normal.

My next visit was to the list of Installations in System Information, where it informed me that XProtect had been updated to 5271, thanks to XProtectCloudKitUpdate, instead of the usual XProtectPlistConfigData installation downloaded through softwareupdate. After a little jiggling with the new xprotect command tool I realised what had happened, and why SilentKnight was awry.

After an unfortunate delay, described later, I’m at last able to offer a new version of SilentKnight that won’t make this same mistake when run in recent beta-releases of Sequoia. Version 2.10 now detects whether it’s running on Sequoia; if it is, instead of inspecting the version of the XProtect.bundle in CoreServices, it now uses the official method of
xprotect version
instead.

Sadly, things get more difficult if XProtect is out of date. SilentKnight can’t use its normal call for update installation to softwareupdate, as that doesn’t handle XProtect updates any more in Sequoia. The official method is first to check whether an update is available using
xprotect check
then perform the update with
xprotect update
Although that might seem simple enough for SilentKnight to handle, both of those calls need to be made with elevated privileges. In Terminal, you’d just preface them with sudo, but apps can’t do that, and should normally use a privileged helper app, running with root privileges, something none of my apps do yet.

There’s another problem for SilentKnight in that it now needs to use two separate methods of checking for updates, and that in turn requires its code to be completely rewritten, a task I’m deferring to a whole new version, SilentKnight 3, which will only run on Sequoia. Until that’s available, bear with me and use the xprotect commands above as you need (remembering to sudo them); at least SilentKnight will now indicate when that might be necessary.

SilentKnight version 2.10 is available from here: silentknight210
from Downloads above, from its Product Page, and through its auto-update mechanism. Apart from fixing any remaining bugs, I intend this to be the final release of SilentKnight 2 with support up to and including macOS Sonoma, when updating remained so simple.

You can see the effect during one of my tests in a freshly made Sequoia VM.

silentknight1001

At first, there’s apparently no XProtect installed at all, and the command tool returns a version of 0. Although there are updates offered for MRT (really?), Gatekeeper Compatibility Data and XProtectPayloads (XProtect Remediator), none is offered for XProtect.

I then manually updated XProtect using the command tool, and ran SilentKnight 2.10 again.

silentknight1002

Immediately afterwards, XProtect is at the current version number of 5271 even though there’s no XProtect.bundle in CoreServices, and there’s no record of that update in the list of latest updates. That’s listed in the new version of SystHist, though, giving its mysterious XProtectCloudKitUpdate origin.

This update has been released a day later than I intended, because of a disorientating bug that caused the app to crash early whenever it tried to start up, complaining of an unreachable file path when it was starting to run its main code. I suspected a problem in the structure of the app and chased many red herrings before I realised this was the result of overenthusiastic code.

I have put the call to the xprotect command tool into conditional code, with an if to check whether it was running on Sequoia or earlier macOS. Apparently the path to the command was being checked before the instructions discovered whether that code would be run. As the path to the command tool doesn’t exist on earlier macOS, when it was being checked before determining which macOS was running, that was failing in Sonoma, as expected. Once I had worked out where this was occurring, without any clue from the errors, I had to change the type of conditional test used so the code didn’t check a non-existent path on older macOS.

Sequoia changes security data updates: updated utilities

By: hoakley
8 August 2024 at 14:30

Soon after the first beta-release of macOS 15.0 Sequoia, I noticed a post on X (formerly Twitter) reporting a new command tool xprotect, but never got the chance to take a look at it. With recent updates to the beta versions, its purpose has become clear: Sequoia uses a different mechanism to update its XProtect data. And that’s not compatible with any of my security update utilities, LockRattler, SilentKnight, Skint, or even SystHist.

The reason for this is that, in all previous versions of macOS going back many years, XProtect data are stored in a bundle named XProtect.bundle, where you (and my apps) can check its installed version number. But in Sequoia that bundle is neither used nor updated when XProtect data are updated. Indeed, Software Update and its command tool equivalent softwareupdate, used by my apps, can’t even see XProtect updates any more, as they’re checked and installed using a different mechanism, which is where the xprotect command tool comes into play.

Run any of those apps on a recent version of Sequoia, and they’ll dutifully report the version of XProtect found in that bundle, which will now be out of date, but the update is apparently not available, although in many cases it has already been installed and protection is up to date!

Unfortunately, as is so often the case, searching Sequoia’s release notes and other Apple sources fails to discover any information about this change, or the xprotect command tool.

syshist901

SystHist was due its annual update anyway, so its new version 1.20 now handles Sequoia updates and (as well as it can) reports those it can find to XProtect. This new version is available from here: systhist120
from Downloads above, from its Product Page, and through its auto-update mechanism.

Skint (and its menu bar companion SkintM) version 1.08 now checks correctly for the XProtect version in Sequoia, and is available from here: skint108
from Downloads above, from its Product Page, and through its auto-update mechanism.

I’m not proposing updating LockRattler to address this issue, as SilentKnight and Skint are now capable of handling most needs. If this distresses you, please let me know and I’ll see what I can do in the coming weeks. However, without major internal surgery it’s never going to work fully with Sequoia, I’m afraid.

silentknight15prob

I was hoping to release a new version of SilentKnight today too, but Xcode gremlins are currently blowing that away, and I fear I have some reconstruction needed before it will do anything other than crash! As soon as I have a stable release, I will make it available.

These changes are sufficient to make it unwise to continue with the current major version of SilentKnight, as all its update code needs to be reworked so that it handles other updates through softwareupdate, as it does at present, and has a privileged helper app to obtain updates using the new mechanism for XProtect alone. My intention therefore is to provide one final update to SilentKnight 2 that will work without being able to update XProtect in macOS 15. That will be the final release to support macOS Catalina to Sonoma. The new version will be SilentKnight 3, will require Sequoia, and I hope will be released before Sequoia 15.0 hits the streets.

I apologise for this, and for the mess it’s causing for SilentKnight in Sequoia, but as you’ll appreciate this is all way beyond my control. I will also be writing further about what appears to be happening with XProtect for the weekend.

Last Week on My Mac: Two betas and one giant leap

By: hoakley
4 August 2024 at 15:00

As I promised here last Sunday, this was a week of updates. The same day that we were busy updating Sonoma, Ventura and/or Monterey, many developers were puzzling as to why they were being offered a beta-release of Sequoia 15.1, long before 15.0 was even ready for baking. There has been speculation as to whether Apple might skip past 15.0 and go straight to 15.1 when it comes to the autumn/fall. This isn’t the first time that Apple has done this during beta-testing, though, so let me hypothesise a little to explain what’s going on, as Apple seems to forgotten to do that yet again.

From September, there are three appointments that Apple can’t miss, two of which already have dates fixed: the release of 15.0 and next major versions of Apple’s other operating systems, and the release of the next batch of new Macs. The third is more flexible, and depends on when Apple Intelligence is ready to go public.

For the next new Macs, Apple wouldn’t dream of shipping them with Sonoma, and there may well be hardware support ready in Sequoia to handle their new chips, whether M3 Ultra or the first of the M4 family in Macs. That means having a decent release of Sequoia to pre-install on the stocks that will shortly be rolling off production lines. That sets the timetable for the release of 15.0, which looks like September.

For many Mac users, 15.0 brings the most important new feature in iPhone mirroring; as they don’t yet have access to Apple silicon, its forthcoming AI means little, at least until Apple has persuaded them to move up to a new Mac. Apple thus needs the completion of 15.0 to meet its timetable for general release and for pre-installation on those shiny new Macs. It has already informed us, though, that 15.0 won’t bring AI in final release form, but most probably as a public beta.

AI needs as much beta-testing as possible to ensure that, when released, it doesn’t become the laughing stock that ChatGPT and other AI have been in their time. While many may marvel at the good things in your AI, it’s the silly and dangerous things that rapidly become most prominent. It’s no good protesting that those are edge cases, as the stigma will have stuck.

That leaves Apple with a problem: it needs two releases of Sequoia to be beta-tested, so its only solution is to fork them at this stage, in the hope that it will continue to get bug reports on the more significant issues remaining in 15.0, and find those rough edges in AI that could give it a bad name when the public first gets its teeth into it.

For developers with just one Mac they can dedicate to beta-testing, so long as it’s Apple silicon, there’s a convenient solution: install the 15.1 beta as the primary system, and then test 15.0 in a virtual machine, which will now at least do iCloud even if other Apple ID services aren’t available yet.

For those of us who want our apps to support AI features like Writing Tools, this is a good way ahead, despite the extra testing and work that it creates. Although only three of my apps do much with text that would benefit from access to Writing Tools, I am keen that those work well when 15.1 is released, maybe in October or November, depending on progress with beta-testing. But I found what was said at WWDC about the requirements was confusing: in places, it seemed that most apps working with text should work fine, but there was also the spectre that apps have to support TextKit 2, which remains an unknown quantity for many, I suspect.

In his presentation at WWDC, Liu Dongyuan stated: “The good news is, if you are using a UITextView, NSTextView or WKWebView, it just works. Keep in mind that UITextView or NSTextView has to use TextKit 2 to support the full Writing Tools experience. If you are using TextKit 1, you will get a limited experience that just shows rewritten results in a panel.” Would I then have to convert DelightEd, Podofyllin and Nalaprop to use TextKit 2, and if so, how would I go about that?

I tried to assemble source code to do this job, but the snippets from WWDC got me nowhere. I downloaded example code from a previous WWDC, and predictably that failed to run correctly in Sonoma and crashed. It looked as if getting my apps to work with Writing Tools was going to take me until the release of 15.1 to get right.

So, like all lazy programmers, I cheated. My three apps had largely been developed well before TextKit 2 was released, and I hadn’t fiddled with the settings for their views since. I was delighted to find a new popup menu at the foot of the Attributes Inspector for NSTextView that could set the view’s Text Layout to one of three options: Default, TextKit 1, and TextKit 2. By default that had of course been set to Default, so my first check was to set it to TextKit 2, build DelightEd, and see how well that worked. As I should have expected, it rendered the view unusable because no content could be selected, although the Select All command still worked, and that enabled Writing Tools to work as well.

As my attempt to cheat macOS had failed, I went back to the Text Layout popup, abandoned the pretence and set it to TextKit 1, as it should have been. macOS duly rewarded me for my honesty, allowing both my view to work correctly, and Writing Tools to function. Not only did they work, but they worked fully in the way that they do for other apps, without any apparent penalty for being TextKit 1, apart from losing the wizzy visual effects seen in TextEdit, for example.

All it took to make three text-handling apps get full benefit from Writing Tools was setting a single popup menu and rebuilding each. In the end it took me longer to update their documentation than it had taken to fix the apps.

I’m hoping now that those three apps will prove good enough to use in macOS 15.0 and 15.1, giving me time to see where else AI might take them. That will most probably require a visit to the alien land of App Intents. Each time I have attempted to read my way into them I have run out of neurones before they make any sense to me, so grasping the basics could take me a little while. There’s now a small suite of intents aimed at word processor apps that look worth investigating.

For text-based apps, support for Writing Tools is going to be critical. If well executed they should prove the biggest thing for Macs since Mac OS X. For anyone who writes, from school kids to eminent professors, this year should be truly transformative.

Last Week on My Mac: Are you ready for Sonoma 14.6?

By: hoakley
28 July 2024 at 15:00

Last Tuesday, as the uproar over the CrowdStrike catastrophe was still subsiding, and alongside the fourth developer beta of Sequoia, Apple quietly provided its first release candidate for macOS 14.6. Don’t be surprised if that ships early next week, alongside Ventura 13.6.8 and Monterey 12.7.6, making it the earliest release of the sixth minor version in the annual macOS cycle since Mojave. For the last four years, from Catalina to Ventura, the last full update marking the start of two years of security-only maintenance has taken place in September.

To put this into context, it’s worth revisiting how Apple numbers macOS versions and how that fits into its annual release cycle:

  • The first digits in the version number indicate the major version, currently 14 for Sonoma, which changes around September-October of each year with the start of each new cycle.
  • The second digit indicates the minor version, currently 5 for Sonoma, and starts from 0 with the first major release in September. Minor updates are scheduled well in advance, and culminate in the sixth as the last regular update (Catalina was an exception in running to 7), and the start of that version’s first year of security-only updates.
  • The third digit indicates the patch version, used for urgent unscheduled fixes between those minor versions. The previous release of Sonoma before 14.5 was 14.4.1, which fixed a few urgent bugs and security vulnerabilities. For security-only updates after the x.6 release, this marks each security update.

Thus the first year of Sonoma is expected to run from its first release as 14.0 on 26 September last year, through 14.1 to 14.6, its last full update, shortly before the release of Sequoia. Sonoma then enters its first year of security-only updates in 14.6.1, and its second year in the autumn/fall of 2025 with versions 14.7, 14.7.1, and so on, until it becomes unsupported after a total of three years.

Sonoma has been running early throughout its release cycle, starting with its first release, and squeezed in 14.2 well before Christmas. While this could indicate that Apple doesn’t intend putting it into security-only maintenance until after version 14.6.1 or even 14.7 in September, around the time of Sequoia’s initial release, that would mark a significant change in the annual cycle. You also have to wonder how many non-security fixes could be prepared for release during August, when most of Apple’s software engineers are fully extended in finishing off the major new versions of macOS, iOS, iPadOS, watchOS, tvOS and visionOS for release the following month.

I expect next week will bring the release of Sonoma 14.6, the last general update and start of its two years in security-only maintenance, together with Ventura 13.6.8 as the end of its first year of security-only fixes, and Monterey 12.7.6, its swan song as it becomes unsupported. Those should pave the way for Sequoia 15.0 in September, bringing support for a batch of new Macs featuring M3 and M4 chips, to include an updated Mac Studio, Mac mini, and MacBook Pros.

If your Mac is still running Ventura or Monterey and is supported by a more recent version of macOS, now is the time to make a decision about whether and when to upgrade. Even if Monterey does get one more final security update, it’s almost certain to fall well short of that provided for Ventura and Sonoma. If you were intending to upgrade to Sonoma, then it’s not likely to have any further general fixes after this forthcoming update, but only to receive security updates from August onwards.

Appendix: Previous last general updates to macOS

  • Ventura 13.6 – 21 September 2023
  • Monterey 12.6 – 12 September 2022
  • Big Sur 11.6 – 13 September 2021
  • Catalina 10.15.7 – 24 September 2020
  • Mojave 10.14.6 – 22 July 2019
  • High Sierra 10.13.6 – 9 July 2018
  • Sierra 10.12.6 – 19 July 2017

Data from System Updates

Will updated apps still run on older macOS?

By: hoakley
25 July 2024 at 14:30

It may seem strange, but each new version of macOS brings a hidden feature that determines which older versions of macOS are supported by software. That’s largely set by those targeted by the new version of Xcode that supports the new macOS, in its minimum deployment target. This article explains how that works, and why you’ll see an increasing number of updates that require Big Sur or later over the coming year.

The largest developers like Microsoft and Adobe operate their own policy for supporting older versions of macOS, and in most cases follow Apple’s unwritten rules, in supporting the current macOS release, and the two previous versions. So, once their apps are fully compatible with Sequoia, you should expect them to support macOS 13 Ventura and later.

Smaller and independent developers usually try to support a wider range of macOS versions, but as most use Xcode to build their software, they’re dependent on the deployment targets supported by Xcode. This is where it gets more complicated.

Xcode support

Alongside Sequoia in beta-testing is Xcode 16, due for release at about the same time. That runs on late versions of Sonoma (14.5 or later) and Sequoia, and is the first version of Xcode that can build apps that make use of new features in Sequoia, the latest SDK (software development kit) supported by Xcode 16. Apple doesn’t update older versions of Xcode to support newer macOS, so a developer who’s still using Xcode 15 generally can’t use features introduced in Sequoia in their apps.

To make this a bit more complicated, that isn’t always so. For example, my macOS virtualisation apps Viable and Vimy are built using Xcode 15, but when they’re used in Sequoia they can now create VMs that support Apple ID, a new feature in Sequoia. However, other new virtualisation features such as support for USB devices will only be available to virtualisers built using Xcode 16, as they require the Sequoia SDK, which isn’t available in Xcode 15. Similarly, we don’t yet know whether Sequoia’s new AI Writing Tools will work with existing apps built for Sonoma rather than Sequoia.

Each version of Xcode has four compatibilities:

  • The minimum macOS required to run that version of Xcode; for Xcode 16, that’s macOS 14.5 or later.
  • The latest SDK it supports; for Sequoia support, that’s Xcode 16.
  • The hardware architectures it supports; Apple silicon was first supported by Xcode 12 (ish), so that’s not important here.
  • The minimum deployment target, or the oldest version of macOS for which it has an SDK; for Xcode 14 and later, including version 16, that’s macOS 10.13 High Sierra.

Of those, it’s the last that determines which older versions of macOS can be supported by apps built with that version of Xcode.

Minimum deployment target

Xcode versions 12 and 13 have a minimum deployment target of macOS 10.9, so apps built with them could support macOS as old as Mavericks from 2013. When Apple introduced Xcode 14 there were many complaints that its minimum deployment target rose to macOS 10.13 High Sierra. Apple appears to have listened, and the new Xcode for Sequoia, version 16, keeps the same minimum deployment target of 10.13.

However, it’s still not that simple. Apple recommends that developers using Xcode 16 build for macOS 11 Big Sur, the oldest version provided in the popup menu used to set the minimum deployment target. To build for older versions of macOS, the developer has to type the version in. Surprisingly, at present at least, Xcode 16 is happy to build for macOS 10.12 Sierra as a target, although whether that would work in reality isn’t clear.

Thus, when a developer is picking the minimum deployment target, the oldest version of macOS supported by a new app or update, the new Xcode can still support 10.13, but nudges the choice to Big Sur.

Code in an app also has to be compatible with all versions of macOS from its minimum deployment target. If a developer sets that too low, then they may well have to write conditional code to cater for differences in features across the whole range of target versions of macOS. That’s determined by what the app does, and how it does it: an app working with a newer feature such as Live Text can’t use that in macOS older than Monterey, as that’s when it was introduced. Feature support becomes most critical with SwiftUI, which wasn’t introduced until Catalina, and has changed greatly since.

SwiftUI

Like all precocious youngsters, SwiftUI has seen extensive changes since its introduction in 2019. Prior to macOS 13, support for key features in macOS apps is sufficiently limited to make it quite a challenge for a substantial app to rely largely or completely on SwiftUI. Even more recent versions of macOS use different calls to the API. For example, adding colour to Text in macOS 13 and earlier is performed by setting its foregroundColor; from macOS 14 onwards, that has to be its foregroundStyle instead. That can be accommodated with conditional code if the developer wishes.

Don’t be in the least bit surprised if apps that have switched to using SwiftUI for their interface require a minimum macOS version of 13 or 14, and it may not be long after the release of Sequoia that you start coming across the first relying on macOS 15.

Summary

  • Apps from major vendors like Microsoft and Adobe are likely to limit support to macOS 13, 14 and 15 once they’re fully compatible with Sequoia.
  • Other developers should still be able to support back to High Sierra, although Apple is encouraging all to build for Big Sur and later.
  • Support for older macOS also depends on whether an app requires features only available in more recent versions of macOS.
  • SwiftUI apps are likely to be the exception, and require Ventura or Sonoma.

Reference

Details of all versions of Xcode

Sequoia VMs can cause kernel panics

By: hoakley
17 July 2024 at 23:01

If you are beta-testing macOS 15 Sequoia in a lightweight virtual machine on an Apple silicon Mac, beware that it can cause the host to suffer a kernel panic. Although I haven’t tested this with other virtualisers, my understanding is that the danger applies to them all, and not just my own Viable and Vimy.

Sequoia developer beta 2, both versions of developer beta 3, and I expect the first public beta, are already hungry for memory. In Sonoma and earlier VMs, if you give the guest 16 GB of memory, it’s likely to use considerably less than that. Those betas of Sequoia will probably use a little more than is allocated to them. But that will double if you restart the VM, and if your host Mac has insufficient memory for twice that VM’s original allocation, it’s likely to suffer a kernel panic with the VM still open.

Previously, in Sonoma and earlier, restarting the VM results in the VM Service for the virtualiser (as shown in Activity Monitor) yielding almost all its memory when restarting, and that rising during booting of the VM. In Sequoia, memory is fully retained during the restart, and then rises until it reaches twice that allocated to the VM, or your Mac panics if it runs out of physical memory first.

I’m very grateful to Joe for reporting this. He has sent a Feedback report to Apple, and I hope this is fixed in the next beta release. In the meantime, don’t restart any Sequoia VM unless your Mac has more than twice the free memory allocated to the VM.

I’d be grateful if those using other virtualisers could confirm whether those are also affected. I suspect they are, as the virtualiser doesn’t normally handle restarting of the VM, but leaves that to the host and guest macOS.

Which apps for which macOS, from El Capitan to Sequoia?

By: hoakley
17 July 2024 at 14:30

Now that the first public beta of Sequoia is available, I thought it might be helpful to detail which of my most popular apps are compatible with versions of macOS from El Capitan to that Sequoia beta.

Update and security utilities

Skint runs daily check on key security systems. Skint 1.07 runs on Monterey and later, and appears fully compatible with Sequoia.

sk221

SilentKnight runs automatic checks of firmware and security systems. For El Capitan to Mojave you should use SilentKnight 1.21, but for all versions of macOS from Catalina onwards use SilentKnight 2.9. This appears fully compatible with Sequoia, although at present it will report TCC Not found, which you can safely ignore. Apple doesn’t release new versions of some of its security databases until late in the beta phase, when I expect that will be put right.

LockRattler provides manual checks of firmware and security systems, similar to those in SilentKnight. For El Capitan and Sierra, use LockRattler 4.35, but for all later versions of macOS from Mojave onwards, use LockRattler 4.37 instead. This too appears fully compatible with Sequoia, although it reports TCC Not found for current betas.

SysHist lists full system and security update installation history. In El Capitan and Sierra, use SystHist 1.17, but for High Sierra and later, use SystHist 1.19 instead. This currently doesn’t show any Sequoia updates, but I will release a revised version in the coming weeks to address that. This is because it has to be able to recognise macOS updates by name, and that isn’t stable until later during the beta-testing phase.

XProCheck checks on XProtect Remediator scans completed and reported in the log. Use XProCheck 1.5 in all versions of macOS from Catalina onwards, that support this new variant of XProtect. I intend to release an updated version in the coming weeks, but this version appears fully compatible with Sequoia.

Rich text and PDF

DelightEd4

DelightEd is a Rich Text (RTF) editor with special Dark Mode features and support for interlinear text. The latest version to run on Sierra is DelightEd 2.0b4, but for all more recent versions of macOS I recommend using DelightEd 2.2. This appears fully compatible with Sequoia, although I don’t know yet whether it will support Writing Tools, because of conflicting documentation.

podofyllin20

Podofyllin is a lightweight PDF viewer and analysis utility. Podofyllin 1.2 is compatible with High Sierra and later, and appears to be fully compatible with Sequoia too.

Log and technical utilities

T2M2 provides quick but thorough checks of Time Machine backing up. When backing up to HFS+ backup stores, or to a NAS, use T2M2 1.19, but when backing up to APFS stores or NAS in Big Sur or later, use T2M2 2.02 instead. Although I haven’t tested this yet with Sequoia, I believe that it should work well.

Mints is a multi-purpose utility that produces custom log extracts, including iCloud, extensive system info and more. Sierra is still supported by Mints 1.9, but for all more recent macOS from High Sierra onwards, use Mints 1.19. This also appears fully compatible with Sequoia.

xattred is a full-featured extended attribute editor, which can also add quarantine xattrs. For El Capitan and Sierra, use xattred 1.2, and for High Sierra and later, use xattred 1.5 instead. This appears fully compatible with Sequoia.

purgeable1

Precize looks deep into files, bundles and folders to show their full size including extended attributes, provides macOS Bookmarks and volfs paths as enduring file references, and detailed information contained in Bookmarks and Aliases. El Capitan and Sierra are still supported by Precize 1.12, but for High Sierra and later use Precize 1.14, which also appears fully compatible with Sequoia.

ulbow101

Ulbow is a log browser designed for ease of use. Ulbow 1.3 runs on Sierra, the first version of macOS to support the new Unified log. Use Ulbow 1.10 on High Sierra and later versions, and it also appears fully compatible with Sequoia, except that it currently can’t create logarchives, a shortcoming in all more recent versions of macOS.

viable12n13

Virtualisation

Viable creates and runs macOS VMs on Apple silicon Macs. Viable beta 12 (1.0.12) works on all versions from Monterey on, and on Sequoia it will now create VMs with Apple ID support. However, it doesn’t yet support the suspension and quitting of VMs (you still have to shut down the guest macOS), and doesn’t give access to external USB devices, a feature new to Sequoia. This article explains how to virtualise Sequoia on a host running either Sonoma or Sequoia.

Vimy runs macOS VMs on Apple silicon Macs from a double-click, and is the runtime companion to Viable. Vimy 0.7 (fourth beta) runs on all versions from Monterey on, including Sequoia, and supports all the features in VMs created using Viable.

Other apps of mine that remain available, many supporting versions of macOS back to El Capitan, are detailed here.

Enjoy, and please report any issues, particularly those with Sequoia betas, so that I can fix them before Apple releases macOS 15 in a couple of months.

Last Week on My Mac: Did Apple forget its own App Store?

By: hoakley
14 July 2024 at 15:00

I was sorely tempted to pre-order an Apple Vision Pro, but it wasn’t the cost that was the decider. When I checked, I realised that Apple has locked in its most exciting new technology to running only what’s provided through its App Store. Not that I don’t buy through Apple’s App Stores, but if there’s one thing that stultifies innovation, it’s a bureaucracy that obsesses with its rules.

It took Steve Jobs a while to accept that iPhones needed third-party apps, and Apple launched its iTunes App Store for iOS in 2008, just over a year after the first iPhone had been released. Early in 2011, the Mac followed suit, but as an addition to well-established direct distribution. At first it provided a convenient central platform for Apple’s own products.

appstore1

Although promoted for its curation, security and trustworthiness, over the last 13 years each has been profoundly undermined. You don’t have to spend long looking in the App Store app to appreciate that it’s as well curated as a painting exhibition requiring all frames to be gilded and more than two inches wide, let alone the prevalence of scam apps on iOS App Stores.

appstore2

Its track record of security nearly came to grief in 2015, when hundreds of apps on the China store were discovered to have been victims of a supply-chain attack by XcodeGhost. Just a couple of months later the macOS App Store suffered major problems with its security certificates, causing most of its apps to be unusable and erroneously reported as damaged.

appstore173

Nevertheless it has continued to attract important apps from major developers, as shown below in 2015, when it was far more navigable.

Apple's Mac App Store now offers many quite expensive - and highly reputable - apps.

Despite being mired in controversy since they were unleashed, Apple’s App Stores have prospered, both for Apple and for the precious few developers who achieve success on them. The one growth area that they have so far missed out on has been virtual machines running on Apple silicon Macs, which have been unable to access the macOS App Store, or to run the great majority of apps purchased from it.

Shortly after Apple released lightweight virtualisation for Apple silicon Macs in 2022, those who had started to experiment with them discovered what appeared to be a major blind spot in their design: as they didn’t support signing in with an Apple ID, they could neither access iCloud services, nor run third-party apps supplied through the App Store. Obvious though this shortcoming was to users, it apparently hadn’t occurred to Apple, who hadn’t even started to build in support for Apple ID.

This was completed in time to be included among the new features announced for macOS Sequoia last month, when Apple promised that it “supports access to iCloud accounts and resources when running macOS in a virtual machine (VM) on Apple silicon”. With issues of virtualising what was needed from the host’s Secure Enclave apparently solved, some of us had come to expect that would include App Store access, which is also controlled by Apple ID. It’s now clear that Apple didn’t intend to include its App Store as a “related application”, which was implicitly excluded.

However little you might love the App Store, support in macOS VMs is essential if they are to be of any general use. VMs that can’t run all App Store apps as part of the benefits of signing in with an Apple ID are so stunted as to be of little use. Would it be that difficult to implement, now that those VMs can be signed in to all the other services that depend on an Apple ID? Did Apple really forget its own App Store when deciding what apps should be allowed to run in a VM?

If you consider this to be a showstopper for virtualising macOS on Apple silicon Macs, then please make it clear to Apple through Feedback.

Sequoia, virtualisation and Apple ID

By: hoakley
12 July 2024 at 14:30

The third developer beta of macOS 15 Sequoia finally brings support for Apple ID in macOS virtual machines (VM). As this is likely to form the first public beta-release next week, here’s a short guide to how to install a Sequoia VM, and what you can do with it. I’m delighted to report that my own free virtualisation apps Viable and Vimy already support Sequoia VMs on Sonoma 14.5 and Sequoia hosts, and I expect that will be true of other virtualisers for Apple silicon Macs.

Installing Sequoia as a VM

When running Sequoia developer beta 3, or the first public beta, download an IPSW image from Apple’s beta support site, or via Mr Macintosh’s compilation. Ensure that you download developer beta 3 or public beta 1 or later, depending on which programme you’ve joined. Then install that IPSW using Viable in the normal way, as detailed here.

If you’re virtualising Sequoia on a Sonoma 14.5 host, you may need to install additional software before installing the Sequoia IPSW using Viable. One way to discover that is to proceed normally using the IPSW you’ve just downloaded. You’ll then be prompted to install a software update.

sequoiavm1

At present, this will fail, but I expect that Apple will provide that additional software for the public beta.

sequoiavm2

If it doesn’t, and you’re unsuccessful in installing the additional software, trash that VM (but not the IPSW inside it), install and run the latest beta-release of Xcode 16 from Apple’s beta support site. Once that has been run, you should be able to install Sequoia without any problems.

First run

Open the VM using Viable, and work through its configuration as normal.

If the VM is hosted on Sequoia developer beta 3 or later, you should be able to enter your Apple ID and password, and opt for FileVault on its Data volume during that initial configuration. If it’s hosted on any older version of macOS, then you shouldn’t try entering your Apple ID and password, as that will fail. This is because the minimum requirements for Apple ID support in a VM are:

  • the host running Sequoia developer beta 3 or later, and
  • the VM running Sequoia developer beta 3 or later.

If your Mac and VM meet those, the VM should then trigger normal 2FA confirmation over iCloud, and then activate iCloud, iCloud Drive, and support for related applications such as passwords, calendar and file sharing via iCloud.

App Store support

In Apple’s release notes for Sequoia developer beta 3, it states that the following issue has been resolved: “Users will not be able to sign-in to iCloud and related applications”. Apple has previously stated that Sequoia “supports access to iCloud accounts and resources when running macOS in a virtual machine (VM) on Apple silicon”. However, that currently doesn’t include access to the App Store or use of apps purchased from it.

At present, “access to iCloud accounts and resources” does include:

  • iCloud Drive
  • Keychain in iCloud, fully supported in Passwords.app, including passkeys
  • syncing shared iCloud databases such as calendars and address book
  • shared Photos using iCloud
  • third-party apps sharing data using CloudKit.

It doesn’t include connecting to the App Store, and as a result apps obtained from the App Store that check the current user is entitled to run them will fail to open. There appears to be no workaround for this, although some apps including several of Apple’s will run because they don’t appear to perform those user checks. In those cases, copying the app from the host enables you to run the app in that VM, but that doesn’t apply to the great majority of paid-for App Store apps.

If you’re disappointed that Apple still hasn’t opened access to its own App Store in VMs, please request this feature using Feedback.

Nesting virtualisation

Apple has also announced that Sequoia will support nesting on models with M3 chips, where you can run a macOS VM inside a macOS VM. Although Viable is no longer blocked from running in a VM, this feature doesn’t appear to work yet, at least not using Viable on an M3 Pro.

Downloads

Viable version 1.0.12 (beta 12) and Vimy 0.7 (beta 4) are available from their Product Page, and appear fully compatible with Sequoia, although they don’t yet support the suspend/resume feature for closing VMs. I believe that ViableS 1.0.12 is also compatible.

Should you try the public beta-release of Sequoia?

By: hoakley
1 July 2024 at 14:30

Some time in the next week or two, Apple is likely to release its first public beta of macOS 15 Sequoia. This article is intended to help you decide whether to risk or resist that tempting offer.

As with Sonoma a year ago, to install the public beta-release you no longer have to download a special enabler from a closed website. This is now done through an extra option in Software Update. All you need to do is sign up here, and once the public beta is released you should see it offered in Software Update, when your Mac is signed in using the Apple ID you signed up with. There’s also an option there that caters for those who wish to use a different Apple ID for betas.

sequoiabeta

Can your Mac run Sequoia?

Sequoia is officially supported on most Intel Macs with T2 chips, all Apple silicon Macs, and just one Intel iMac without a T2 chip, the iMac 2019 (iMac19,x). If you have an older Mac, then it might be able to run Sequoia using OCLP, but won’t do so until that has been updated to support that, later this year.

The detailed list of supported models is:

  • iMac19,x, all T2 iMacs including iMac Pro from 2017
  • MacBook Air 2020 and later, but not 2018 or 2019
  • MacBook Pro 2018 and later (all T2 models)
  • Mac mini 2018 and later
  • Mac Pro 2019 and later
  • all Apple silicon Macs.

Note that the one Intel model without a T2 chip doesn’t support iPhone mirroring because that relies on hardware device attestation for pairing. Apple Intelligence will only be available on Apple silicon models, and initially only those using US English as their primary language.

What do you get in the beta?

Apple’s official account of new features is fairly comprehensive. I have drawn attention to some additional features in the following articles:
First details of macOS 15 Sequoia
Does Sequoia’s Password app change keychains?
What’s going on with AI in Sequoia?
How Sequoia changes virtualisation on Apple silicon
How file systems can change in Sequoia with FSKit.

Apple provides extensive release notes for betas. Those for Sequoia are worth reading as they explain some of the current problems you could encounter.

Can you lose that Mac?

The next question you should ask is whether you could afford to completely lose your Mac for a while, as a result of a problem with the beta. Although that’s most unlikely to happen, it’s a risk you’ve got to be prepared for when you install any pre-release version of macOS.

Never, under any circumstances, install a beta of macOS on any Mac you rely on for production. Betas invariably involve firmware updates, so even if you install the beta on an external disk, it will change your Mac’s firmware. Undoing that is hard enough for an Apple silicon model, and it’s simply not possible on Intel Macs. All you can then do is wait for another beta, or maybe the final release in the autumn/fall, which does update the firmware to something more compatible.

Betas also normally come with updated versions of key components such as iCloud, the APFS file system and Time Machine. Consider carefully what havoc they could produce if there’s a bug affecting other storage used by that Mac, and its backups.

If the worst comes to the worst, you could end up having to restore that Mac to an older version of macOS. Apple explains how to do that, and you should read that account carefully before making any decision. If you’re thinking of installing betas on an Apple silicon model, beware that process requires another Mac running Apple Configurator 2 or macOS Sonoma, and restoring it in DFU mode.

Internal or external SSD?

One way to reduce the risk posed by beta versions of macOS is to install them on external storage. While that can enforce some degree of separation and protection, it still means that firmware is updated, and still brings significant risk of disaster. Don’t try this with a production Mac, even from an external disk.

If you’re going to install the beta on an external disk, you’ll need to be comfortable with the procedure for Apple silicon Macs. Although it does become straightforward with practice, some seem unable to get it to work at all. Intel Macs are far simpler, of course, although one important catch with T2 models is that you have to downgrade their security using Startup Security Utility in Recovery mode, if you haven’t already done so, or they can’t boot from an external disk.

Multiple systems on the same disk

You can also install multiple boot volume groups on the same disk, letting you choose which version of macOS to start up from. This provides even less separation or protection than installing them on separate disks, so should never be attempted on any production Mac.

Apple recommends that you do this into separate boot volume groups within the same APFS container, which has the great advantage that they share the same free space within that container. However, there are times when that can work against you, and I’ve explained why and how to opt for separate containers instead. The choice is yours.

Virtual machine

Some consider that the best way of keeping out of trouble when running beta versions of macOS is to install them into a Virtual Machine (VM), for example using Parallels, UTM or my own Viable. This can’t alter the firmware of the Mac hosting the VM, and that alone makes it far safer. On an Intel Mac, provided that your virtualiser can run Sonoma, it should be capable of running Sequoia betas too, and you should be able to find full instructions as to how to do that from its support site.

This is even simpler on Apple silicon Macs, with their extensive built-in support for running virtualised macOS. Use any of the virtualisers, including Parallels, UTM, and my own Viable. Full instructions for Viable are given here. While Sequoia in a VM works best when on a host also running Sequoia, it still runs very nicely indeed in Sonoma.

iCloud

Some betas bring substantial changes to iCloud, and in the past that has caused lasting havoc to accounts and on iCloud storage. I’m not aware of any particular issues that have been reported in this respect with Sequoia betas, but many testers prefer to use a different iCloud account for Macs when running beta-releases of macOS.

Kernel panics

If you do decide to install the Sequoia beta, or have already done so, I have a big favour to ask on behalf of tens of millions of users, and most of Apple’s engineers. By all means take a good look at its new features, and give Apple plenty of feedback on what you think of them. But please pay careful attention to the basics, exercising your Mac with peripherals such as external displays and hubs. Where you discover problems, please work with Apple to ensure that it knows what they are. If you can, test out features such as Time Machine (being careful not to put your existing backups at risk), which seldom get much attention from other beta-testers.

In particular, send Feedback reports on any kernel panic which your Mac encounters when running a beta. The normal system report, sent after your Mac has restarted, is helpful, but further details are much better still. Even betas should never suffer kernel panics; if yours does, please help Apple’s engineers fix that problem before Sequoia is released.

For those who do beta-test Sequoia, I wish us success, and hope you enjoy testing, and helping Apple make Sequoia even better for all of us.

What do XProtect BehaviourService and Bastion rules do?

By: hoakley
28 June 2024 at 14:30

Not content with two different XProtects, Apple added a third to macOS Ventura, XProtect BehaviorService (XBS), part of the new Bastion behavioural-based malware detection system. Rather than performing on-demand or periodic scans of static code, this watches for potentially malicious behaviours, such as attempts to access folders used by browsers such as Safari and Google Chrome. This article summarises what XBS is doing as we prepare to upgrade from Sonoma to Sequoia.

What they do

Apple tells us precious little about XBS and Bastion, mentioning them in its Platform Security Guide: “In addition, XProtect contains an advanced engine to detect unknown malware based on behavioral analysis. Information about malware detected by this engine, including what software was ultimately responsible for downloading it, is used to improve XProtect signatures and macOS security.”

At present, XBS and Bastion only record suspicious events in the XBS database at /var/protected/xprotect/XPdb, report them to Apple, but don’t attempt to intervene in any way. They determine what to report according to a set of rules applied by syspolicyd that are compiled from source files updated inside XProtect Remediator update bundles. Changes in those, in XPR’s scanning modules, and in XProtect’s detection signatures, are reported on this blog for each update released by Apple.

Development

Over the period since its introduction, Bastion rules have grown steadily, from four to 12:

  • In macOS 13.5 (24 July 2023) there were 4 rules, increasing to 5 in September 2023.
  • XProtect Remediator (XPR) 108 (8 August 2023) brought the first separate Bastion rule update.
  • XPR 112 added rules 6 and 7.
  • XPR 123 added rules 8 and 9, and adjusted rule 7.
  • XPR 130 added rule 10.
  • XPR 131 added rule 11.
  • XPR 137 added rule 12, and amended rules 6 and 7.

Updates provided in XProtect Remediator contain two files for XBS and Bastion:

  • bastion.sb, a text file containing the latest Bastion SystemPolicyConfiguration, its rules;
  • BastionMeta.plist, a property list defining behaviour dictionaries for XBS and Bastion.

Bastion rules

The Bastion SystemPolicyConfiguration file bastion.sb is prefaced with the line (version 3), which hasn’t changed since the first update.

This first defines four groups of processes: usual-offenders, common exceptions to several rules, and separate groups of exceptions to each of Bastion rules 1, 2, 3 and 12. For example, com.apple.mds and other Spotlight indexing processes are usual-offenders, while com.apple.Finder is only a rule-one-offender. Interestingly, three of the XProtect Remediator scanning modules (MRTv3, Pirrit and WaterNet) are included in the list of usual-offenders.

Using those lists of exceptions, Bastion rules are then built as filters:

  1. excludes other processes from accessing private data for Google Chrome, Firefox and Safari;
  2. excludes other processes from accessing private data for Messages, Microsoft Teams, Slack and WhatsApp;
  3. excludes other processes from accessing the QuarantineEvents database;
  4. controls access to two socket ioctl commands SIOCIFCREATE and SIOCGIFDESC;
  5. controls access to writing files with a period/stop at the start of their name within Library/PrivilegedHelperTools/ directories.
  6. controls creating or writing to files with a name starting with com within /Library/Application Support/
  7. controls creating or writing to files with a name starting with com within /Library/Application Support/ and user /Library/Application Support/ directories
  8. controls creating or writing to files with a name starting with a period/stop, other than .DS_Store, in user /Library/Application Support/ directories
  9. excludes other processes from creating or writing to files in user /Library/Containers/com.apple.Safari/Data/Library/Safari/AppExtensions/ directories
  10. controls creating or writing to files with a name starting with a period/stop, other than .DS_Store, .betamigrated and .localized, in the /Users/Shared/ directory
  11. controls execution of processes from files with a name starting with a period/stop in the /Users/Shared/ directory
  12. excludes other processes from accessing private data for Notes, Safari Cookies, Chrome, Brave, Microsoft Edge, Opera, Vivaldi, Firefox, Arc, other cookies, Electrum and Coinomi wallets, Exodus, atomic, Binance, Filezilla, Steam and Discord.

The updated bastion.sb file supplied in XPR updates is explicitly referenced by syspolicyd to replace the version embedded in its own code.

BastionMeta.plist

This property list contains a metadata dictionary of 12 behaviours, each correlating with a Bastion rule. Each has a Signature Name, such as macOS.NetworkSniffer.Generic, a Boolean value indicating the need for immediate reporting, and a binary flag ranging from 1 to 2048. The behaviours are named:

  1. Browser
  2. Messages
  3. QntDb
  4. NetworkSniffer
  5. HiddenPrivilegedHelpers
  6. ADLOAD NumericPath
  7. ADLOAD PersistenceSearch
  8. Persistence HiddenAppSupport
  9. Safari ExtensionModification
  10. Persistence HiddenShared Generic
  11. Persistence HiddenShared Exec
  12. InfoStealers.

Behaviours detected

Individual rules currently detect:

  1. attempts to access private browser data
  2. attempts to access private messaging data
  3. attempts to access quarantine records
  4. attempts to perform network packet sniffing
  5. attempts to write to hidden privileged helper apps
  6. Adload behaviours
  7. Adload persistence behaviours
  8. persistence behaviour using hidden files in user /Library/Application Support/ directories
  9. attempts to create and use Safari extensions
  10. persistence behaviour using hidden files in /Users/Shared/
  11. persistence behaviour running hidden files in /Users/Shared/
  12. attempts by an InfoStealer to access a wide range of private data.

Summary

  • In macOS Ventura and later, XProtect BehaviorService (XBS) and its Bastion rules detect suspicious behaviours that might reflect malicious activity.
  • Bastion rules are updated within XProtect Remediator updates, using two files bastion.sb and BastionMeta.plist.
  • There are currently 12 Bastion rules, covering generic behaviours such as accessing private data, to those indicative of Adload and InfoStealer malware.
  • Suspicious behaviour is recorded locally to the XBS database and reported to Apple, but isn’t notified to the user.
  • Currently the primary purpose of XBS and Bastion is to provide Apple’s security team with intelligence to improve protection provided by XProtect and XProtect Remediator.

Reference

Chris Long, Leveraging Osquery To Examine The XProtect Behavioral Service DB

❌
❌