In the previous article, I outlined what extended attributes do, and how they work in macOS. I also started to explain how some are considered ephemeral, while others are persistent. This article continues from there, by documenting how macOS decides what to do with them when a file containing xattrs is copied.

Although Apple does now explain a little about this in the context of the FileProvider framework and syncing with cloud services, the only useful documentation is provided in man xattr_name_with_flags, and two source code files that are part of the open source copyfile component.

In 2013, as part of its enhancements for iCloud in particular, Apple added support for flags on xattrs to indicate how those xattrs should be handled when the file is copied in various ways. Rather than change the file system, Apple opted for what’s perhaps best seen as an elegant kludge: appending characters to the end of the xattr’s name.

If you work with xattrs, you’ve probably already seen this in those whose name ends with a hash # then one or more characters: that’s actually the flags, not part of the name, what Apple refers to as a ‘property list’. To avoid confusion I won’t use that term here, but refer to them as xattr flags. A common example of this is com.apple.lastuseddate#PS, which is seen quite widely. In recent years, Apple has added one flag, B, and there’s another to come with Sequoia.

Xattr flags

Flags can be upper or lower case letters C, N, P, S or B, and invariably follow the # separator, which is presumably otherwise forbidden from use in a xattr’s name. Upper case sets or enables that property, while lower case clears or disables that property. There are currently (macOS 14.6.1) five properties:

• C: XATTR_FLAG_CONTENT_DEPENDENT, which ties the flag and the file contents, so the xattr is rewritten when the file data changes. This is normally used for checksums and hashes, text encoding, and position information. The xattr is preserved for copy and share, but not in a safe save.
• P: XATTR_FLAG_NO_EXPORT, which doesn’t export or share the xattr, but normally preserves it during copying.
• N: XATTR_FLAG_NEVER_PRESERVE, which ensures the xattr is never copied, even when copying the file.
• S: XATTR_FLAG_SYNCABLE, which ensures the xattr is preserved during syncing with services such as iCloud Drive. Default behaviour is for xattrs to be stripped during syncing, to minimise the amount of data to be transferred, but this will override that.
• B: XATTR_FLAG_ONLY_BACKUP, which keeps the xattr only in backups, including Time Machine (added recently).

These operate within another general restriction of xattrs: their name cannot exceed a maximum of 127 UTF-8 characters.

Defaults

macOS provides a standard ‘whitelist’ of default flag settings for different types of xattr. These aren’t contained in a configuration file, but are baked into the xattr flag code, where as of macOS 14.6.1 the following default flags are set for different types of xattr (* here represents the wild card):

• com.apple.quarantine – PCS
• com.apple.TextEncoding – CS
• com.apple.metadata:kMDItemCollaborationIdentifier – B
• com.apple.metadata:kMDItemIsShared – B
• com.apple.metadata:kMDItemSharedItemCurrentUserRole – B
• com.apple.metadata:kMDItemOwnerName – B
• com.apple.metadata:kMDItemFavoriteRank – B
• com.apple.metadata:* (except those above) – PS
• com.apple.security.* – S
• com.apple.ResourceFork – PCS
• com.apple.FinderInfo – PCS
• com.apple.root.installed – PC

Copy intents

Also contained in the source code is a table of intents, that explains how different types of copy are affected by different combinations of xattr flag. Currently, those are:

• XATTR_OPERATION_INTENT_COPY – a simple copy, preserves xattrs that don’t have flag N or B
• XATTR_OPERATION_INTENT_SAVE – save, where the content may be changing, preserves xattrs that don’t have flag C or N or B
• XATTR_OPERATION_INTENT_SHARE – share or export, preserves xattrs that don’t have flag P or N or B
• XATTR_OPERATION_INTENT_SYNC – sync to a service such as iCloud Drive, preserves xattrs if they have flag S, or have neither N nor B
• XATTR_OPERATION_INTENT_BACKUP – back up, e.g. using Time Machine, preserves xattrs that don’t have flag N

Use

If you want a xattr preserved when it passes through iCloud, you therefore need to give it a name ending in the xattr flag S, such as co.eclecticlight.MyTest#S. Sure enough, when xattrs with that flag are passed through iCloud Drive, those xattrs are preserved even if the default rule would treat them differently. Similarly, to have a xattr that is stripped even when you just make a local copy of that file, append #N to its name.

There’s a further limit imposed on xattrs synced by FileProvider, including those for iCloud Drive, that strips all individual xattrs that are larger than a certain size. Apple gives that as “about 32KiB total for each item”, and my measurements performed in the recent past put that at about 32,650 bytes, slightly less than 32,767.

In itself, this information is valuable if you ever use any metadata stored in xattrs. It’s used in my intergrity-checking utilities Dintch, Fintch and cintch to ensure the xattr containing a file’s hash isn’t stripped by passage through iCloud Drive, for instance. On Tuesday morning next week, once Sequoia has been released, I’ll explain how Apple has extended this system to achieve something that many have been wishing for.

One of the innovative features in classic Mac OS was its use of resource forks, allowing structured metadata to be attached to any file. When Mac OS X merged that with the more traditional Unix approach adopted by NeXTSTEP, those were nearly lost. Classic Mac apps were restructured from storing most of their components, including their executable code, in their resource fork, when Mac OS X flattened those into an app bundle consisting of a hierarchy of separate files in folders, without any resources.

For the first four years of Mac OS X resource forks were reluctantly tolerated, until the solution came in 10.4 with the introduction of extended attributes, including one to contain what had previously been stored in the resource fork, which became an extended attribute or xattr with the name com.apple.ResourceFork.

All files in HFS+ and APFS (and other file systems) contain a fairly standard set of metadata known as attributes, information about a file such as its name, datestamps and permissions. Xattrs are extensions to those that contain almost any other type of metadata, the first notable xattr coming in Mac OS X 10.5, named com.apple.quarantine. That contains quarantine information for apps and other files downloaded from the internet, in a format so ancient that the quarantine flag is stored not in binary but as text.

The quarantine xattr provides a good demonstration of some of the valuable properties of xattrs: it can be attached to any file (or folder) without changing its data, and isn’t included when calculating CDHashes for code signatures. It can thus be added safely without any danger of altering the app or its code, although it does change the way in which macOS handles the code, by triggering security checks used to verify it isn’t malicious. Once those have been run, the flag inside the quarantine xattr can be changed to indicate it has been checked successfully.

Far from being a passing phase, or dying out as some had expected, xattrs have flourished since those early days. This has happened largely unseen by the user: few change anything revealed in the Finder’s Get Info dialog, although they’re used to store some forms of visible metadata such as Finder tags, and the URL used to download items from the internet. Editing xattrs is normally performed silently: you’re not made aware of changes in the quarantine xattr, and in most cases the only way to manage xattrs is to use the xattr command tool, or one of very few apps like xattred that can edit and manage them.

Examples

Among the well-known and important xattrs you can encounter are:

• com.apple.quarantine the quarantine xattr, containing a quarantine flag
• com.apple.rootless marks items individually protected by System Integrity Protection (SIP)
• com.apple.provenance contains data about the origin of apps that have been quarantined
• com.apple.metadata:kMDItemCopyright records copyright info
• com.apple.metadata:kMDItemWhereFroms the origin of downloaded file as a URL
• com.apple.metadata:_kMDItemUserTags Finder tags
• com.apple.TextEncoding reveals text file encoding
• com.apple.ResourceFork a classic Mac resource fork

Storage

In APFS and HFS+, xattrs aren’t stored with file data, nor with a file or folder’s normal attributes.

For smaller extended attributes up to 3,804 bytes, their data is stored with the xattr in the file system metadata. Larger extended attributes are stored as data streams, with separate records, but still separately from the file data. Apple doesn’t give a limit on the maximum size of xattrs, but they can certainly exceed 200 KB, and each file and folder can have an effectively unlimited number of them.

Persistence

Most file systems to which macOS can write either handle xattrs natively (HFS+, APFS), or macOS uses a scheme to preserve them, as in the hidden files written to FAT and ExFAT volumes. NFS is an important exception, and files copied to NFS will have all their xattrs stripped. Neither are extended attributes unique to Macs: most file systems used by Linux support them, and even Windows can at a push.

Because xattrs contain a wide range of metadata, some are treated as being ephemeral, others as persistent. Moving files with xattrs around within the same volume shouldn’t affect their xattrs, as that takes place within the same file system. Copying files to another volume, even if both use APFS, may leave some xattrs behind if they’re considered to be ephemeral.

The most complex situation is when a file with xattrs is moved to iCloud Drive. The Mac that originated that file is likely to retain most if not all of its xattrs, because the local copy remains within the same volume and file system. However, not all xattrs are copied up to iCloud storage, so other Macs accessing that file may only see a small selection of them. The rules for which xattrs are to be preserved during file copying, including in iCloud Drive, are baked into macOS, and the subject of the next article.