macOS 15.0 Sequoia brings several new features for lightweight virtualisation on Apple silicon Macs, including most importantly support for iCloud at last. This article explains how to build a macOS VM on an Apple silicon Mac that can access iCloud and iCloud Drive, and its quirks and limitations.

Why so late?

Whether support for Apple Account (formerly Apple ID) had been intended for macOS lightweight virtualisation, isn’t known. However, it wasn’t until after its release, when many users asked for it, that Apple appears to have started work to implement support. Because those secrets are normally protected by the Secure Enclave, a mechanism had to be found to provide access to those from within the VM, using an exclave. That wasn’t ready to include in virtualisation for Sonoma, therefore was only released in Sequoia 15.0 on 16 September.

How to build a macOS VM with Apple Account support

For this to work, there are two essential requirements:

• The host is running macOS 15.0. That can be the result of upgrading an older version of macOS.
• The VM is built from an IPSW image file for macOS 15.0. It cannot work on a VM that has been upgraded to 15.0 from an older version of macOS.

It’s possible to build a VM running 15.0 on a recent version of Sonoma, but that can’t support Apple Account. It may be possible to build a VM running 15.1 beta on a 15.0 or 15.1 beta host; however, that hasn’t worked reliably on hosts running 15.1 beta. For best results, use the release 15.0 on a release 15.0 host.

Building the VM is performed normally. In Viable, set the size of the Virtual Disk and the remainder of the settings to those you want for the VM when you start it. Then click the Install… button, select the 15.0 IPSW file, then save the VM using the name and location that you want. I normally do this using a duplicate of the IPSW file, so the original remains in place. Being an APFS clone, it takes no real disk space to do that.

Once installation has succeeded, check the settings again ready for the VM’s first run, then click on Start VM… button and select the VM you just made. You will be taken through its personalisation and configuration in the normal way. Ensure that you there enter your Apple Account name, then its password.

At the end of that process, you should see this summary, including FileVault enabled if that’s the host configuration. The VM will then open, and sync with iCloud Drive. If you take a look in Privacy & Security settings, you’ll see that FileVault is disabled. If you try to enable it, whether you opt for iCloud recovery or a Recovery Key, you’ll see that it can’t be turned on there.

Apple Account settings warn you that “some features are unavailable”, most obviously the Media & Purchases item, which is greyed out.

However, many iCloud features are active, including Passwords and iCloud Drive.

The official list of unsupported features reads:

• Apple Media Services, most importantly the App Store. Apart from some of Apple’s free apps like Pages, Numbers and Keynote, no App Store apps will run in a VM. Still.
• iCloud Mail.
• Apple Wallet.
• Find My.

Apple also includes iCloud Backup, although as far as I’m aware, that still isn’t a feature of macOS.

Installing XProtect data

One of my first tasks with a fresh VM is to bring it up to date with security data updates, and that’s now more complicated. I copy SilentKnight across from the host, then run it in the VM.

This is typical of a fresh Sequoia system, with the version of XProtect shown as 0, indicating that XProtect has no installed data. In this case, an update isn’t offered in SilentKnight, so I open Terminal and type in
sudo xprotect update

The response here indicates that Software Update had already downloaded and installed the XProtect bundle, but it hadn’t been installed into the new XProtect and activated, which was accomplished by that command.

With all greens, apart from FileVault, that VM is now ready for use.

One final tip: I often use VMs to test what’s potentially destructive or damaging to them. To save me from having to build each individually, I set one up, with all the apps I need, then shut it down. When I want to use that as a disposable VM, I simply duplicate it and run the copy, leaving the original unharmed. Because duplication is performed as much as possible by APFS cloning, this is really quite economical on disk space.

One of the unannounced features in macOS Sequoia is, for many who use iCloud Drive, one of its most important, as this upgrade introduces the ability to pin files and folders to ensure they remain downloaded and don’t get evicted. This article explains how to use this feature, exactly what it does, and how it works.

Pin an item in iCloud Drive

Assuming that your Mac is connected to iCloud Drive and you have enabled Optimise Mac Storage, so that files in iCloud Drive can be evicted from local storage, you can use pinning to ensure that designated files aren’t evicted, and remain downloaded and synced with iCloud Drive. This is an important advance for anyone who needs certain files to be kept in local storage, so remaining accessible even when connection to iCloud isn’t possible, such as when you’re travelling.

Select the file or folder in the Finder and Control-click to bring up the contextual menu. You’ll see a new item there, Keep Downloaded. Select that.

That adds a new icon to the pinned files, with a white downward arrow on a circular grey background. This indicates that file is to be kept downloaded or pinned locally.

To unpin an item, select it and bring up the contextual menu. The Keep Downloaded item will be ticked. Select that command, and that toggles the setting off, unpinning the item so it can be evicted from local storage once again.

Pinning only works if your Mac’s iCloud Drive has Optimise Mac Storage enabled, which puts it into non-replicated FileProvider mode, and allows you and macOS to Remove Download or evict files. Its effect is to prevent that from occurring. Pinned files will remain downloaded to your Mac all the time, just as if iCloud Drive were in replicated FileProvider mode, with Optimise Mac Storage disabled.

A file’s pinning setting travels with it if you move that file within the same volume, and may be preserved when it’s moved away, for example by transferring the file by AirDrop. It’s preserved when you make changes to that file, but not when it’s copied, and pinning settings are specific to a local Mac, and not transferred to other Macs or devices connected to the same iCloud Drive.

How it works

Pinning is set by macOS attaching an extended attribute (xattr) named com.apple.fileprovider.pinned to each file that’s pinned. That normally contains a single byte, the character 1 as the single byte 0x31, although changing that to 0 works just as well, so it appears it’s the xattr that’s important, rather than its contents.

When a com.apple.fileprovider.pinned extended attribute is attached to a file, it ceases being eligible for eviction, and is kept downloaded, just as if it were in a replicated FileProvider. Unpinning the file strips the xattr.

Xattr flags

I have recently summarised the features of extended attributes, and explained the system of flags they use to determine how those are copied. In addition to adding the com.apple.fileprovider.pinned xattr to those encountered in Sequoia, Apple has changed the xattr flag system to cope with this new xattr. These flags are appended to the xattr name, after a # character, and pinning xattrs are actually named com.apple.fileprovider.pinned#PX to determine when they should be preserved during file copy operations. The X flag is new to Sequoia, and can be used for other xattrs.

This brings the table of available xattr flags to six. Flags can be upper or lower case letters C, N, P, S, B or X, and invariably follow the # separator, which is presumably otherwise forbidden from use in a xattr’s name. Upper case sets (enables) that property, while lower case clears (disables) that property. The properties are now (as of 15.0):

• C: XATTR_FLAG_CONTENT_DEPENDENT, which ties the flag and the file contents, so the xattr is rewritten when the file data changes. This is normally used for checksums and hashes, text encoding, and position information. The xattr is preserved for copy and share, but not in a safe save.
• P: XATTR_FLAG_NO_EXPORT, which doesn’t export or share the xattr, but normally preserves it during copying.
• N: XATTR_FLAG_NEVER_PRESERVE, which ensures the xattr is never copied, even when copying the file.
• S: XATTR_FLAG_SYNCABLE, which ensures the xattr is preserved during syncing with services such as iCloud Drive. Default behaviour is for xattrs to be stripped during syncing, to minimise the amount of data to be transferred, but this will override that.
• B: XATTR_FLAG_ONLY_BACKUP, which keeps the xattr only in backups, including Time Machine.
• X: XATTR_FLAG_ONLY_SAVING, which keeps the xattr only when saving and in backups, including Time Machine (introduced in macOS 15.0).

These must operate within another general restriction of xattrs: their name cannot exceed a maximum of 127 UTF-8 characters.

This new flag doesn’t change the macOS default flags set for different types of xattr, which remain as:

• com.apple.quarantine – PCS
• com.apple.TextEncoding – CS
• com.apple.metadata:kMDItemCollaborationIdentifier – B
• com.apple.metadata:kMDItemIsShared – B
• com.apple.metadata:kMDItemSharedItemCurrentUserRole – B
• com.apple.metadata:kMDItemOwnerName – B
• com.apple.metadata:kMDItemFavoriteRank – B
• com.apple.metadata:* (except those above) – PS
• com.apple.security.* – S
• com.apple.ResourceFork – PCS
• com.apple.FinderInfo – PCS
• com.apple.root.installed – PC

(* here represents the wild card.)

Until Apple releases the source code for Sequoia’s open source components, the table of copy intents remains unchanged at

• XATTR_OPERATION_INTENT_COPY – a simple copy, preserves xattrs that don’t have flag N or B
• XATTR_OPERATION_INTENT_SAVE – save, where the content may be changing, preserves xattrs that don’t have flag C or N or B
• XATTR_OPERATION_INTENT_SHARE – share or export, preserves xattrs that don’t have flag P or N or B
• XATTR_OPERATION_INTENT_SYNC – sync to a service such as iCloud Drive, preserves xattrs if they have flag S, or have neither N nor B
• XATTR_OPERATION_INTENT_BACKUP – back up, e.g. using Time Machine, preserves xattrs that don’t have flag N

as documented in man xattr_name_with_flags. Once I have been able to check the new source code, I will confirm any changes to the intents table.

Summary

• If you have Optimise Mac Storage turned on for iCloud Drive, you can now pin files so that they aren’t evicted from local storage.
• To pin a file or folder, use the Finder’s contextual menu (Control-click) and select the Keep Downloaded item.
• To unpin a file, toggle the Keep Downloaded item off (unticked).
• Pinning works by adding an extended attribute to each pinned file.
• A sixth xattr flag has been added, X, to keep that xattr only when saving and in backups.

In the previous article, I outlined what extended attributes do, and how they work in macOS. I also started to explain how some are considered ephemeral, while others are persistent. This article continues from there, by documenting how macOS decides what to do with them when a file containing xattrs is copied.

Although Apple does now explain a little about this in the context of the FileProvider framework and syncing with cloud services, the only useful documentation is provided in man xattr_name_with_flags, and two source code files that are part of the open source copyfile component.

In 2013, as part of its enhancements for iCloud in particular, Apple added support for flags on xattrs to indicate how those xattrs should be handled when the file is copied in various ways. Rather than change the file system, Apple opted for what’s perhaps best seen as an elegant kludge: appending characters to the end of the xattr’s name.

If you work with xattrs, you’ve probably already seen this in those whose name ends with a hash # then one or more characters: that’s actually the flags, not part of the name, what Apple refers to as a ‘property list’. To avoid confusion I won’t use that term here, but refer to them as xattr flags. A common example of this is com.apple.lastuseddate#PS, which is seen quite widely. In recent years, Apple has added one flag, B, and there’s another to come with Sequoia.

Xattr flags

Flags can be upper or lower case letters C, N, P, S or B, and invariably follow the # separator, which is presumably otherwise forbidden from use in a xattr’s name. Upper case sets or enables that property, while lower case clears or disables that property. There are currently (macOS 14.6.1) five properties:

• C: XATTR_FLAG_CONTENT_DEPENDENT, which ties the flag and the file contents, so the xattr is rewritten when the file data changes. This is normally used for checksums and hashes, text encoding, and position information. The xattr is preserved for copy and share, but not in a safe save.
• P: XATTR_FLAG_NO_EXPORT, which doesn’t export or share the xattr, but normally preserves it during copying.
• N: XATTR_FLAG_NEVER_PRESERVE, which ensures the xattr is never copied, even when copying the file.
• S: XATTR_FLAG_SYNCABLE, which ensures the xattr is preserved during syncing with services such as iCloud Drive. Default behaviour is for xattrs to be stripped during syncing, to minimise the amount of data to be transferred, but this will override that.
• B: XATTR_FLAG_ONLY_BACKUP, which keeps the xattr only in backups, including Time Machine (added recently).

These operate within another general restriction of xattrs: their name cannot exceed a maximum of 127 UTF-8 characters.

Defaults

macOS provides a standard ‘whitelist’ of default flag settings for different types of xattr. These aren’t contained in a configuration file, but are baked into the xattr flag code, where as of macOS 14.6.1 the following default flags are set for different types of xattr (* here represents the wild card):

• com.apple.quarantine – PCS
• com.apple.TextEncoding – CS
• com.apple.metadata:kMDItemCollaborationIdentifier – B
• com.apple.metadata:kMDItemIsShared – B
• com.apple.metadata:kMDItemSharedItemCurrentUserRole – B
• com.apple.metadata:kMDItemOwnerName – B
• com.apple.metadata:kMDItemFavoriteRank – B
• com.apple.metadata:* (except those above) – PS
• com.apple.security.* – S
• com.apple.ResourceFork – PCS
• com.apple.FinderInfo – PCS
• com.apple.root.installed – PC

Copy intents

Also contained in the source code is a table of intents, that explains how different types of copy are affected by different combinations of xattr flag. Currently, those are:

• XATTR_OPERATION_INTENT_COPY – a simple copy, preserves xattrs that don’t have flag N or B
• XATTR_OPERATION_INTENT_SAVE – save, where the content may be changing, preserves xattrs that don’t have flag C or N or B
• XATTR_OPERATION_INTENT_SHARE – share or export, preserves xattrs that don’t have flag P or N or B
• XATTR_OPERATION_INTENT_SYNC – sync to a service such as iCloud Drive, preserves xattrs if they have flag S, or have neither N nor B
• XATTR_OPERATION_INTENT_BACKUP – back up, e.g. using Time Machine, preserves xattrs that don’t have flag N

Use

If you want a xattr preserved when it passes through iCloud, you therefore need to give it a name ending in the xattr flag S, such as co.eclecticlight.MyTest#S. Sure enough, when xattrs with that flag are passed through iCloud Drive, those xattrs are preserved even if the default rule would treat them differently. Similarly, to have a xattr that is stripped even when you just make a local copy of that file, append #N to its name.

There’s a further limit imposed on xattrs synced by FileProvider, including those for iCloud Drive, that strips all individual xattrs that are larger than a certain size. Apple gives that as “about 32KiB total for each item”, and my measurements performed in the recent past put that at about 32,650 bytes, slightly less than 32,767.

In itself, this information is valuable if you ever use any metadata stored in xattrs. It’s used in my intergrity-checking utilities Dintch, Fintch and cintch to ensure the xattr containing a file’s hash isn’t stripped by passage through iCloud Drive, for instance. On Tuesday morning next week, once Sequoia has been released, I’ll explain how Apple has extended this system to achieve something that many have been wishing for.

Backups, indexing for search, and cloud storage are each very useful when and where you want them. But there are occasions when you want to exclude folders and files from their reach. This article is a short reference to how you can exclude items from being backed up, indexed and searched, and stored in iCloud Drive.

Time Machine backups

Time Machine has several mechanisms for excluding items from its backups. The first of those used to be its Standard Exclusions list, at /System/Library/CoreServices/backupd.bundle/Contents/Resources/StdExclusions.plist, but that has long since disappeared from there. Currently the best way to inspect that list is by viewing the hidden file .exclusions.plist at the top level of any backup.

This lists several categories of exclusions, of which the first are apiExclusionPaths, added by individual third-party apps. The equivalent of what used to appear in StdExclusions.plist is now given under the standardExclusionPaths key, containing:

• .DocumentRevisions-V100 – the version database on each volume, added in Big Sur,
• .Spotlight-V100 – Spotlight metadata,
• .Trashes – contents of Trash folders,
• .fseventsd – the File System Events database,
• /Library/Logs – traditional text log files,
• /Users/Guest – guest user files,
• /private/var (partial) – various transient files,

among many other ephemeral items.

Of those, only one results in any significant loss of data, the version database. Although this was dutifully copied by Time Machine into backups for several years, the current structure of that database appears to make it impossible to restore successfully, even when restoring a complete volume. By the middle of the Catalina cycle, it had become a frequent cause of Time Machine choking, so was added to the standardExclusionPaths for Big Sur. As far as I’m aware this was never fixed in Catalina.

Following those are stickyExclusionPaths, excluding various Photos Library contents. The key systemFilesExcluded is set to true to ensure that the whole System volume is always excluded, following which are any userExclusionPaths you have set using tmutil or in Time Machine settings, using the Options… button.

By default, volumes on external storage are automatically added to this exclusion list; if you want an external volume to be backed up by Time Machine then you will need to remove it from the exclusion list manually.

In addition to the rules given in the Standard Exclusion list, any file or folder can have an extended attribute of type com.apple.metadata:com_apple_backup_excludeItem attached to it as a ‘sticky’ exclusion, which should add it to the stickyExclusionPaths list and prevent it from being backed up by Time Machine. If you then remove that extended attribute, it should be backed up normally again.

Mike Bombich gives a thorough and detailed account of what CCC doesn’t copy on this page. Other backup utilities should also provide full lists on their support site.

Note that no folders or files can be excluded from APFS snapshots, whether made by Time Machine, CCC or any other app, as snapshots always capture the entire volume.

Spotlight search

There have been three general methods of excluding folders from Spotlight’s indexing and search, although only two of them still work reliably:

• appending the extension .noindex to the folder name (this previously worked using .no_index instead);
• making the folder invisible to the Finder by prefixing a dot ‘.’ to its name;
• putting an empty file named .metadata_never_index inside the folder; that no longer works in recent macOS.

System Settings offers Spotlight Privacy settings in two sections. Search results won’t normally prevent indexing of those items, but does block them from appearing in search results. Spotlight’s indexing exclusion list is accessed from the Spotlight Privacy… button, where you can add items that you don’t want indexed.

iCloud Drive storage

This works primarily by inclusion, in that you copy or move items to the iCloud Drive folder when you want them to be copied up to iCloud Drive. There are occasions, though, when you want to exclude specific folders and files that need to be in a folder in iCloud Drive, but you don’t want those items to be stored in the cloud. iCloud won’t sync items with the extensions .nosync or .tmp, so putting files inside folders whose name ends in either of those extensions will ensure that they’re not stored in iCloud Drive.

Other items intended for local housekeeping and similar purposes that are excluded from syncing include:

• .DS_Store files
• files whose name starts (A Document Being Saved
• .ubd files
• items with names containing .weakpkg
• desktop.ini files
• items with names starting with ~$
• items named $RECYCLE.BIN, which are trash folders
• items named icon\r, which are custom folder icons.

Files and folders with purposes that make iCloud syncing inappropriate include those:

• named iPhoto Library, and presumably now Photos Library too
• named Dropbox, .dropbox, or .dropbox.attr
• with the extension .photoslibrary, .photolibrary, .aplibrary, .migratedaplibrary, .migratedphotolibrary, or .migratedaperturelibrary
• named Microsoft User Data
• named ~ with an extension of 3 or more characters.

The only setting in System Settings gives control over Desktop & Documents folders, and those of individual apps; there’s no exclusion list available.

Simple reference to folder controls

• Time Machine: add to settings with the Options… button, or using tmutil;
• Spotlight indexing: add to settings with the Spotlight Privacy… button, or append the extension .noindex to the folder name;
• iCloud Drive: append the extension .nosync or .tmp to the folder name.

macOS Sonoma brought major internal changes to iCloud Drive, and coupled with the opacity of Time Machine, you may now be wondering whether it still backs up what you have stored in iCloud Drive, and if so, where to find those backups. This article explains how this now works, and where those backups can be found.

What does Time Machine back up?

By default, Time Machine backs up almost everything on volumes it’s set to back up. It doesn’t, of course, try to back anything up from the Signed System Volume (SSV) containing macOS itself, and there are a few folders it automatically excludes, such as the hidden folder at the root of each volume containing the versions database (as that can’t be restored in any case). But unless you add iCloud Drive to its exclusion list from your Home Library folder, Time Machine will back up all the files and folders in iCloud Drive as long as they’re stored locally at the time it makes that backup.

Dataless files

It’s that last condition that’s the most important consideration. If your Mac has Optimise Mac Storage turned off, then iCloud Drive is run as a Replicating FileProvider, where everything stored in iCloud Drive is also stored locally. In that case, all those items will be backed up by Time Machine just as they would be if kept in purely local storage.

If Optimise Mac Storage is turned on, iCloud Drive behaves as a Non-Replicating FileProvider, and some or many of the files you store in iCloud Drive may be evicted from local storage, and are then no longer available for backup. This is explained in the diagram.

Files in local storage consist of attributes, metadata (extended attributes) and data. Move them to iCloud Drive and their data is then copied up to iCloud. When a file is evicted to save local storage space, the local file’s data is removed, making that copy of the file dataless. In previous versions of macOS, that was accomplished using a special stub file, but in Sonoma it simply loses its data, and the distinctive iCloud icon is displayed next to the file in the Finder.

Dataless files can’t be backed up by Time Machine, as without their data there’s nothing to back up. If you want that file to be backed up, then it has to be downloaded or materialised by downloading its data from iCloud. Some third-party backup utilities can manage that automatically for you, but Time Machine doesn’t. If you want to ensure that a file in iCloud Drive is backed up when Optimise Mac Storage is turned on you’ll thus have to download it manually if it has been evicted to iCloud.

Restoring from backup

The simplest way to restore a file or folder in iCloud Drive from a Time Machine backup is through the Time Machine app. Before opening that app, select iCloud Drive in the sidebar of the front Finder window, then the app will take you back through its backups of iCloud Drive.

The main drawback of that is that items you restore will replace any existing items. You may also prefer to use the Finder’s or a command line interface to your backups, both of which allow you to copy restored items where you prefer. The difficulty here is that macOS deliberately hides the path to iCloud Drive: in your active Home folder, they’re located in ~/Library/iCloud Drive, but that path doesn’t work in Time Machine backups.

To access them in a backup, follow the Finder path
[backupname]/[backup].backup/Data/Users/[username]/Library/Mobile Documents/com~apple~CloudDocs/

From Terminal, you’ll have more of a struggle, but need to look for
/Volumes/.timemachine/[UUID]/[backup].backup/[backup].backup/Data/Users/[username]/Library/Mobile\ Documents/com\~apple\~CloudDocs/
where the path component [backup].backup is repeated. If you locate an item first in the Finder, you can always drag and drop that into the command line for it to automatically paste the correct path.

Summary

• Unless you manually exclude iCloud Drive from Time Machine backups, the contents of iCloud Drive will be backed up provided that items haven’t been evicted from local storage.
• If Optimise Mac Storage is turned off, the whole of iCloud Drive will be backed up by Time Machine, as eviction doesn’t occur.
• If Optimise Mac Storage is turned on, only those files and folders that are stored locally will be backed up by Time Machine. To ensure an item is backed up, manually download it before the backup is made.
• Restoring iCloud Drive items from a backup is simplest in the Time Machine app, but also available in the Finder, from a path equivalent to ~/Library/Mobile Documents/com~apple~CloudDocs/ in that backup.