Chinese whispers is an old children’s game where everyone sits in a circle, and one child whispers into the ear of the next on their right a sentence like Send reinforcements, we’re going to advance. That child then whispers the message they heard to the child on their right, until it reaches the one who started it, who says out loud what they heard, classically Send three-and-fourpence, we’re going to a dance, as a demonstration of how messages can so easily become corrupted. What this has to do with China remains one of childhood’s mysteries. I should also explain that three-and-fourpence was idiomatic British English in the days before our currency was ‘decimalised’, and meant three shillings and four (old) pence, about 17 (new) pence, sufficient at one time to enjoy a good night out.
In this article I’m going to do much the same with metadata for a PDF document, tracing what gets indexed by Spotlight, so becoming discoverable by search, and what is displayed in the Finder. This relies on several of my utilities, most of which are available from this page.
Source PDF
I prepared a completely unrelated PDF using my favourite PDF editor, PDF Expert, by adding metadata to be saved in the file’s data. As you might expect, there are several ways that could be stored in the PDF format, including XMP metadata, but in this case for simplicity they were added in the document information dictionary.
I inspected that in a source view in Podofyllin, which found the following fields: /Author (Author name in pdf)
/Creator (Pages)
/Keywords (keyword1 pdf)
/Subject (Subject in pdf)
/Title (0PDFtest1accessdefault)
When rendered in macOS, those are ‘flattened’ by its Quartz PDF engine, to /Author (Author name in pdf)
/Creator (Pages)
/Keywords (keyword1 pdf)
/AAPL:Keywords [(keyword1 pdf)]
/Subject (Subject in pdf)
/Title (0PDFtest1accessdefault)
Note the copying of keywords into a new attribute AAPL:Keywords.
Extended attributes
I then added seven extended attributes using Metamer, with names such as com.apple.metadata:kMDItemAuthors, as shown below in xattred.
Spotlight import
I then inspected the file in SpotTest’s new Drop Window, which reported the following attributes found by mdimport: ":EA:kMDItemAuthors" = "author in xattr";
":EA:kMDItemComment" = "xattr comment";
":EA:kMDItemCreator" = "xattr creator";
":EA:kMDItemDescription" = "xattr description";
":EA:kMDItemKeywords" = "keyword1,xattr";
":EA:kMDItemSubject" = "xattr subject";
":EA:kMDItemTitle" = "xattr title";
all from the extended attributes, while those derived from the PDF data were kMDItemAuthors = (Pages);
kMDItemCreator = Pages;
kMDItemDescription = "Subject in pdf";
kMDItemKeywords = ("keyword1 pdf");
kMDItemTitle = 0PDFtest1accessdefault;
Those attributes have already changed, with PDF Subject becoming kMDItemDescription, Creator being duplicated into kMDItemAuthors, and the loss of PDF Author.
Spotlight indexes
Attributes reported by mdls changed again to kMDItemAuthors = (Pages)
kMDItemComment = "xattr comment"
kMDItemCreator = "Pages"
kMDItemDescription = "Subject in pdf"
kMDItemKeywords = ("keyword1,xattr")
kMDItemSubject = "xattr subject"
kMDItemTitle = "0PDFtest1accessdefault"
This has lost the xattr attributes kMDItemAuthors, kMDItemCreator, kMDItemDescription and kMDItemTitle, and the PDF kMDItemKeywords. That list of 7 attributes should then be searchable using Spotlight.
The Finder
The final step was to discover which of those could be displayed in the Finder, either in its Get Info dialog, or in the Preview panel of a Finder window.
Only 5 of those attributes survived in the Finder, and were given as Authors: Pages
Content Creator: Pages
Description: Subject in pdf
Keywords: keyword1,xattr
Title: 0PDFtest1accessdefault
Of those, 4 are taken from the metadata in the PDF file, and only the Keywords were taken from its extended attribute. The attribute named as Authors contains a duplicate of what had originally been in the PDF Creator field, but neither of the PDF Author or xattr kMDItemAuthors fields. Those paths are traced in the diagram below.
Conclusions
Of the total of 12 distinct metadata attributes added in the PDF data and extended attributes, only 6 different items were indexed by Spotlight, and 4 were displayed in the Finder (allowing for the duplication of Authors and Content Creator).
Before relying on metadata for search and access in the Finder, it’s essential to verify that the attributes you intend using are successfully indexed and displayed. Choose the wrong attributes and you’ll never find anything.
Cloud services have long differed in how they handle metadata, particularly that stored in extended attributes (xattrs). The one service you would expect to get this right is Apple’s own iCloud Drive, but it seems to have silently changed policy without rhyme or reason. It has been almost three years since I last tested iCloud Drive to assess which metadata it does preserve, and I’ve been shocked to discover that it now preserves almost no xattrs at all.
Testing
This used a Mac mini M4 Pro and a MacBook Pro M3 Pro, both running macOS 26.4.1 Tahoe, and connected to the same iCloud+ account, with a local Content Caching server active. On both Macs, iCloud Drive was configured in nonreplicating mode, with Optimise Mac Storage turned on. All files were downloaded locally when necessary before use.
A test suite of files with extensive metadata content in xattrs was moved to a folder in iCloud Drive on the Mac mini, and ample time was allowed for them to sync up to iCloud. Each was inspected on the Mac mini, then on the MBP, and their metadata was compared using xattred.
Results
All metadata was preserved in the original files that were stored locally on the Mac mini, even after evicting them and downloading their data to reconstitute them. There was no evidence that large xattrs were removed by eviction, as they have been in the past.
Only the following metadata was synced down from iCloud Drive to the MBP:
com.apple.metadata:_kMDItemUserTags (Finder Tag), which transferred complete with its custom label
com.apple.lastuseddate#PS
com.apple.quarantine
com.apple.TextEncoding
co.eclecticlight.dintch.hash#S and co.eclecticlight.dintch.time#S, custom xattrs used by Dintch and its relatives to check file integrity.
Metadata stored within a file’s data such as EXIF was also preserved, as would be expected.
Finder Comments were not synced to the MBP, either in their primary .DS_Store location, or as a xattr.
Two of the more obvious examples viewed in xattred are:
This is one of the test files I used recently when looking at xattrs suitable for metadata, with a Finder Tag added. Above is the original on the Mac mini, and below is the synced copy on the MBP. Every single com.apple.metadata: xattr has been stripped, except for com.apple.metadata:_kMDItemUserTags.
This is a crafted test suite that I have used for some years, above in its original on the Mac mini, and below as synced to the MBP, where just three xattrs have survived passage through iCloud Drive.
What should happen?
Policy for iCloud Drive appears to have changed over the last few years. When I first tested this between Sierra and High Sierra in 2018, few xattrs were synced, and even Finder Tags were stripped. Just over a year later, we realised this should be controlled by xattr flags, and when I tested this formally in a late version of macOS Sonoma in July 2023, com.apple.metadata: xattrs were preserved if they weren’t too large. However, eviction caused all large xattrs to be stripped even from the original source Mac.
Since then, iCloud Drive has continued to adopt the new File Provider framework, and Apple has further extended xattr flags, now described here. At no stage, though, does Apple appear to have documented which metadata and extended attributes should be preserved in iCloud Drive, so we can only speculate what it intends should happen.
Why has this changed?
Apple doesn’t appear to have made any relevant changes to the xattr flag system as detailed in the source of copyfile. Under that, almost all com.apple.metadata: xattrs should be treated as if they have PS flags attached, so should be “preserved during syncing with services such as iCloud Drive”.
One possible explanation is that the File Provider framework doesn’t respect xattr flags as laid out in copyfile, possibly because it doesn’t use copyfile but has its own independent mechanism. That would be in accord with Dropbox, which only appears to support syncing a few types of xattr. Information for Microsoft OneDrive is too vague for comparison, though.
Consequences
You should expect all metadata stored in xattrs to be stripped when synced via iCloud Drive. The exceptions to that include Finder Tags and those explicitly assigned an S flag, such as the custom integrity xattrs used here. Unfortunately, appending #S to well-known xattr types is likely to cause problems, as few apps handle those flags correctly, so won’t recognise the xattr type.
If you want to preserve that metadata, then you’ll either need to archive files using a method that preserves xattrs, or transfer it using a more conventional network method such as AirDrop, which does preserve almost all xattrs.
Currently, as far as metadata is concerned, iCloud Drive has no advantage over third-party cloud storage such as Dropbox.
Conclusions
When synced via iCloud Drive, expect all metadata stored in xattrs, with the exception of Finder Tags and those explicitly assigned an S flag, to be stripped when accessed from another Mac.
Xattrs and other metadata are faithfully preserved when moved between Macs using AirDrop. If you want to preserve their metadata, use that instead of iCloud Drive.
When I was researching yesterday’s article about storing and managing metadata, I encountered some unexpected behaviour that merited deeper investigation. This article explains how that can lead to macOS ignoring and hiding metadata.
Spotlight is about more than just search, as it runs the metadata services for macOS. Those ingest information about files, contents of their extended attributes, metadata extracted from file data such as EXIF in images, and indexed text content. When the Finder needs to populate the fields of a Get Info dialog, or those shown in a Preview pane, it calls on Spotlight’s volume indexes to provide the data. The process of analysing and extracting metadata is thus important to a wider range of features beyond search, and it’s concerning when metadata known to be present is unexpectedly missing.
To determine what might be going wrong, I took six test files in macOS 26.4.1:
two plain text files, indexed by /System/Library/Spotlight/RichText.mdimporter
one RTF file, indexed by /System/Library/Spotlight/RichText.mdimporter
one PDF file, indexed by /System/Library/Spotlight/PDF.mdimporter
one JPEG and one PNG screenshot, indexed by /System/Library/Spotlight/Image.mdimporter.
To each I attached five xattrs containing distinctive test text:
com.apple.metadata:kMDItemAuthors, known in the Finder’s Find search terms as Authors;
com.apple.metadata:kMDItemComment, known as Comment, and distinct from Finder or Spotlight Comment;
com.apple.metadata:kMDItemDescription, known as Description;
com.apple.metadata:kMDItemKeywords, known as Keywords;
com.apple.metadata:kMDItemSubject, known as Subject of this item.
I then used the Finder’s Find window to search for part of the contents of each of the extended attributes for each of the test files. I also viewed each file in a Preview pane, with maximal display of information, as described here, and in a Get Info dialog.
For each file, I ran the following two commands: mdimport -t -d2 filename
to list all its known metadata recognised by mdimporter. That crashed for both the image files, a bug that has been long present. I also ran mdls filename
to list indexed metadata, which didn’t crash.
Search results
Search worked as expected and proved successful for all files with Description, Keywords and Subject. However, I was unable to find Authors metadata for the RTF file, and Comment metadata couldn’t be found for the JPEG or PNG images.
Finder preview pane
The two image formats displayed none of the metadata used. Otherwise Keywords was shown in each, Subject and Comment were omitted from PDF, and Authors and Description omitted from RTF.
These are summarised in the table below.
There are two serious failures shown here, failure to index Authors metadata correctly in the RTF file, and failure to index Comment metadata correctly in the JPG and PNG files. Otherwise all failures appear to be the result of design decisions over which types of metadata should be displayed in the Finder.
Failure to index Authors in RTF
From the mdimport listings, all five xattrs were recognised correctly and should have been indexed. However, there were additional [null] listings for four of them, including Authors, presumably derived from the absent RTF Info section, an optional feature in the format and often absent. Separate testing revealed that, for the Authors metadata alone, the content extracted from the Info section of the RTF overrode that obtained from the xattr, even when it was null. Adding the authors to the Author (note the singular) entry in Info restored its indexing and search, but using the text in RTF Info rather than the xattr.
There are thus two related bugs here:
The Author field from RTF Info metadata should be indexed into its own field, to avoid conflict with any com.apple.metadata:kMDItemAuthors xattr present.
Only com.apple.metadata:kMDItemAuthors xattr content should be indexed into Authors.
Failure to index Comment in JPG/PNG
No mdimport listing was available for these image files because of the tool crashing persistently. However, it’s clear that the contents of Comment was taken not from the xattr, but from the EXIF UserComment, which had been set by macOS to Screenshot when that screenshot had been made. That metadata should surely have been indexed into a separate category such as a new kMDItemUserComment to avoid such collisions.
There are again two related bugs:
The UserComment field from EXIF metadata shouldn’t be indexed into Comment, but into its own field.
Only com.apple.metadata:kMDItemComment xattr content should be indexed into Comment.
Conclusions
There appear to be indexing bugs in /System/Library/Spotlight/RichText.mdimporter and /System/Library/Spotlight/Image.mdimporter. This is the third bug I have investigated in RichText.mdimporter.
Those can lead to incorrect indexing in kMDItemAuthors (Authors) and kMDItemComment (Comment) respectively.
They limit significantly the usefulness of those xattrs as metadata in macOS.
Of the five metadata xattrs tested here, only one, kMDItemKeywords (Keywords), appears both reliable for search and displayed generally in the Finder.
kMDItemDescription or Description, and kMDItemSubject or Subject are reliable for search, but aren’t displayed well in the Finder.
Aside from these bugs, poor display coverage in the Finder limits the usefulness of these xattrs.
Investigating bugs in Spotlight mdimporters can be challenging, particularly when mdimport is so prone to crash.
One of the design features of macOS is that a file’s metadata can be stored separately from that file’s data. This is normally achieved by extended attributes, xattrs, and to aid that there’s a rich and extendable range of them. This article explains what you can and can’t do with them, as of macOS 26.4.1.
More generally, most metadata are stored within a file’s data, to accommodate operating systems and file systems that don’t have such rich features. Most image formats, for example, incorporate standard collections of metadata such as EXIF information, which are saved with the image data. PDF and Word documents have similar features. These have the disadvantage that changing the metadata results in the file data being altered, and that makes it difficult to track and to guarantee the data’s integrity. This can result in damage or corruption to the data in a PDF file during editing of comment metadata, for example. When possible it’s far better to separate metadata.
Distinguishing between data and metadata can also be tricky at times. This is easiest with non-verbal data like images and audio, where text is clearly separate from data, but it can appear more arbitrary with written documents. Some content such as copyright information or an index is universally accepted as metadata, but abstracts and appendixes may vary. Some types of document, such as official standards, draw explicit distinctions, using qualifiers like normative and informative, to assist.
Finder Comments and Tags
These are the two standard types of metadata currently best supported in macOS, both being readily accessible in the Finder and applicable to any file or folder.
The main problem with Finder Comments is that they are primarily stored separately in the hidden .DS_Store file in the same directory as the item, although a secondary copy is written to a xattr attached to that file/folder. They’re easily accessed in the Finder’s Get Info dialog, and can be shown in List view windows, although that doesn’t work for multi-line comments. On balance, their strange storage makes them fragile and unsuitable for many uses.
Finder Tags are stored properly in a xattr, and are most widely distinguished by the coloured tag displayed. Although they can be repurposed to store text, their main value remains in categorisation. There’s a practical limit of 20-25 characters before their text label is cropped in most views, and labels for multiple items can only be shown in the Finder’s List view layout. For the majority, they are best used for allocating items to a limited number of categories, distinguished foremost by their tag colour, and aren’t suitable for more substantial text like even a brief summary.
Properties
To be generally suitable for storing text metadata for a file or folder in macOS, these should:
be attached to the file or folder as a xattr;
be capable of storing and displaying up to 3,804 bytes of UTF-8 text, the upper limit of data stored alongside the xattr;
be indexed by Spotlight so their contents can be searched;
be preserved in iCloud Drive, and when copied to a different volume;
be displayed and edited easily, ideally in the Finder, without the need for third-party software.
Which xattrs?
Of the dozens of xattrs available, I’m aware of just three that come closest to meeting all those requirements:
com.apple.metadata:kMDItemComment, known in Spotlight search menu as Comment, and different from Spotlight Comment, which is synonymous with Finder Comment;
These are sufficiently persistent as to be preserved in iCloud Drive, and when transferred between Macs using AirDrop. Although they can be displayed in the Finder, they each require third-party software for creation and management. As explained below, this doesn’t apply to image files, which are expected to store their metadata in EXIF information within the file, and not in xattrs.
Management
All three can be created and managed using my free Metamer, as well as with the more extensive features of xattred.
Metamer is a lightweight drag-and-drop utility to create, edit and view many different types of xattr, including the three recommended. It uses a Combo box offering 16 of the most commonly used xattrs, and you can enter the full name of any other if you need. Although it can be used to edit multi-line text, it’s designed to work best with single lines.
xattred is a general xattr editor with more extensive capabilities, and ideal for checking all the xattrs attached to a file or folder. It doesn’t offer the conveniences of Metamer, though.
Display
Each of these three xattrs is displayed in the More Info section of the Get Info dialog, and depending on settings, they can also be shown in the Finder’s preview pane when enabled in Preview Options for that file type. The latter is explained in more detail here, and can appear counter-intuitive at times. However, they can only be added and displayed for supported file types. For example, JPEG and PNG image files can’t display these three xattrs in Get Info dialogs or in the preview pane, but PDF, RTF, text and many other file types can.
None of the three can be listed for multiple files in any of the Finder’s view layouts.
Spotlight
All three are available in the list of search terms available in the Other item at the foot of the search term menu in a Finder Find window, as listed above.
But
When researching this article, I discovered some odd behaviours that render some xattrs both invisible and undiscoverable by Spotlight search. I hope to describe those fully tomorrow if I can get my head around them.
Finder tags are a simple and accessible way of categorising files and folders by attaching one or more of seven colours to them, together with a text label. This article explains how they work, and how to get the best from them.
Tag metadata
Finder tags are stored in extended attributes of type com.apple.metadata:_kMDItemUserTags. They consist of a binary property list containing a little UTF-8 text. This is an array of strings, each containing a tag name, followed by the newline character, followed by the colour number (0-7). The array can be empty, which usually indicates a tag had been attached, but has now been removed.
Colour numbers used are:
none, 0
grey, 1
green, 2
purple, 3
blue, 4
yellow, 5
red, 6
orange, 7.
Thus a tag name might read Red\n6, or Orange\n7 Green\n2 for two colours, where \n represents the newline character 0a. As the user can assign (almost) arbitrary text to labels, those could equally be Urgent\n6, or Important\n7 Project\n2. When in a location indexed by Spotlight, the text content prior to the newline character (excluding the digit) in each tag is indexed.
When the Finder writes a tag extended attribute to an item, it also adds a null com.apple.FinderInfo xattr of 32 bytes length, if that isn’t already present.
Tagging
Tags are easily added to a file or folder in the Finder by selecting the item and using the command in the contextual menu. The menu lists the seven favourite tags, as selected in the Finder’s settings, by colour.
A separate command opens a floating window offering all tags instead, giving access to others in addition to the seven favourites.
Tags are removed from items using the same tools, either to remove one of the seven favourite tags, or any other that has been attached.
Once attached to a file or folder, the colours of its tags are displayed in the Name section of each type of Finder window. List view can be set to display the text label as well, although that’s not available in other window types. In macOS Tahoe, tagged folders are also coloured according to the last of the tags attached to them, when that option is enabled in the Finder’s settings.
Tags are widely supported in the Finder, macOS tools, and apps. When you show Tags in the Finder’s sidebar, you can view items with only a specific tag by selecting that tag in the sidebar. The same feature works in file Open and Save dialogs in all apps using the standard macOS dialog too. You can further refine tag-based listings by opting to sort those files in date or other orders, and you can group by tags as well.
Editing and management
Tags and their text labels are managed in the Finder’s settings, in the Tags tool.
This allows you to add and remove tags from the main list, to edit the label of each (either in the same way you rename files in the Finder, or using the contextual menu), to delete a tag, and to set which tags appear in the list of seven favourites at the foot. If you customise one of the standard tags by changing its name, say from Red to Urgent, then the Finder automatically changes all items marked with that tag to show that new label.
Similarly, deleting an existing tag from this list removes it and its label from all files and folders tagged with them, throughout that Mac’s storage. This behaviour can sometimes catch users unaware of those extensive consequences.
Each tag and label remains associated with that file or folder: when you copy it across to another Mac, the tag remains attached, and on that second system should continue to show the same colour and label. This also applies to any custom tags you add. The important exception to this is if the item is moved using a method that doesn’t preserve its extended attributes, which occurs on some file systems such as NFS, and when using some command line tools. Some backup methods may also not preserve extended attributes, and will strip all your tags; Time Machine preserves them all.
Categories and labels
There are only seven different tag colours, and although many different labels can be associated with each colour, in practice Finder tags work best when tags and labels are matched one-to-one. Note that you can’t associate two different coloured tags with the same label.
Tagging is simplest when each colour only has a single label associated with it, as colours and labels are then fully interchangeable. Labels can also work well when used to qualify categories, for example using red tags for those tasks due this week, with text specifying the day. Broadening the range of labels for a given tag can quickly lead to confusion: imagine the consequences of using a red tag with emails that are labelled Important, and those marked as Junk.
For example, if you want to classify your photos into no more than seven categories, such as social, travel, parties, and so on, they should work well with tags. If you want to add the names of people in each image, or their locations, then tags are likely to be the wrong type of metadata for that. Another example is tagging projects: if you need no more than seven different project tags, then Finder tags should serve well. If you need a dozen or more, then they’ll quickly prove inadequate.
Labels aren’t intended to contain longer text, as they’re not displayed in a way that would make that easy to read. If you want to attach structured or free text to files and folders, use other extended attributes including Keywords, Comment (not Finder comment, though), Description, Subject and Information. Although these are also indexed by Spotlight and can be searched for, there is still no option to display them in Finder windows.
Spotlight search
Finder tags can also be used as a criterion in Spotlight searches, although those are based not on the tag colour but its label, such as Important for a red tag labelled Important. You’ll need to add the Tags item to the Finder’s list of search criteria before you can use it to find items in the Finder’s Find window.
Parts of macOS go back a long way, and have scars to prove it. Among those are Finder comments: text you can easily add to a file’s metadata in the Finder’s Get Info dialog. Like all good metadata, it’s also searchable in Spotlight, so can be used to organise and categorise your documents. However, there are some catches that can make that unreliable.
Finder comment
To see what I mean, you’ll need an app like my free Metamer, a simple metadata editor, or xattred, to display extended attributes. Pick an old file you can sacrifice for this purpose. Select it in the Finder, and Get Info for it. In the Comments section, type in a comment.
Open that file in Metamer, and select the FinderComment item in its combo box menu.
Your freshly written Finder Comment has already been added as an extended attribute.
Open the file in xattred, and you’ll see that text as a String in a property list for that extended attribute.
Now change the name of that file, perhaps by adding a short suffix. Get Info and the Finder Comment is still there.
Look at the extended attribute using Metamer or xattred, and you’ll see that the String in the property list has been deleted, and the extended attribute no longer contains the text still displayed in the Get Info dialog.
This is because
the primary copy of the Finder Comment goes into the hidden .DS_Store file in the same folder as the document, and that’s the one used by the Finder;
a secondary copy is saved in a xattr of type com.apple.metadata:kMDItemFinderComment for the file, which the Finder knows nothing about.
You can experience other strange effects from this dissociation between those two copies, and their different behaviours. As a result I don’t recommend use of Finder comments, because of their unreliability.
Steganography
Finder comments and other metadata provide a scheme for steganography in macOS. This was first used by Wdef, one of the original viruses that affected Classic Mac OS in about 1990, and was resurrected 30 years later in 2020, when Phil Stokes of Sentinel Labs reported a Bundlore variant abusing the Resource fork, then as now an extended attribute of type com.apple.ResourceFork.
Just recently, William Charles Gibson and Ryan Conry of Cisco Talos have described how a similar technique could be used “to stage payloads in a way that evades static file analysis”. They propose using a payload script, encoded using Base64, and written to Finder comment metadata.
Writing the malicious Finder comment is simple to achieve using the AppleScript code set comment of newFile to "$PAYLOAD"
and retrieval, decoding and execution by the command mdls -name kMDItemFinderComment -raw ~/Desktop/remote_test.txt | base64 -D | bash
also run from within AppleScript.
There are three disadvantages in using a Finder comment for this purpose, its fragility as shown above, its visibility to the user in the Finder’s Get Info dialog, and reliance on Spotlight indexing and search. If you want to use file metadata for steganography, then you’re better off using an extended attribute directly, accessing it with the xattr command tool, which is robust, remains with the file rather than in a hidden .DS_Store file, and doesn’t need to be indexed by Spotlight or found using mdls.
That also gives you a wider choice of extended attribute. If traceability isn’t important, a custom xattr can be used with the PS flags to ensure its persistence in copies, saves, syncs and backups. There’s a long list of those supported in macOS, including kMDItemProjects, which is stored in the com.apple.metadata:kMDItemProjects xattr and treated as having PS flags, and most importantly isn’t exposed to the user in the Finder’s Get Info dialog.
At this stage you might want to check how thoroughly your malware protection checks extended attributes for malicious content. As a rough guess, I suspect the answer is not at all.
Extended attributes aren’t unique to macOS and Apple’s other OSes, but they have the longest and richest history, going right back to the first versions of Mac OS. Until the advent of Mac OS X they were resource forks, structured metadata that stored everything from executable code to keyboard layouts. After a shaky first four years in early versions of Mac OS X, 10.4 introduced extended attributes, xattrs, that steadily became used more extensively.
All files in HFS+ and APFS (and other file systems) contain a fairly standard set of metadata known as attributes, information about a file such as its name, datestamps and permissions. Xattrs are extensions to those that contain almost any other type of metadata, the first notable xattr coming in Mac OS X 10.5, named com.apple.quarantine. That contains quarantine information for apps and other files downloaded from the internet, in a format so ancient that the quarantine flag is stored not in binary but as text.
The quarantine xattr demonstrates some of the valuable properties of xattrs: it can be attached to any file or folder without changing its data, and isn’t included when calculating CDHashes for code signatures. It can thus be added safely without any danger of altering the app or its code, although it does change the way that macOS handles the code, by triggering security checks used to verify it isn’t malicious. Once those have been run, the flag inside the quarantine xattr can be changed to indicate it has been checked successfully.
Far from being a passing phase, or dying out as some had expected, xattrs have flourished since those early days. This has happened largely unseen by the user: few alter anything revealed in the Finder’s Get Info dialog, although they’re used to store some forms of visible metadata such as Finder tags, and the URL used to download items from the internet.
Xattrs are named using a reverse-URL format like com.mycorp.setting. There’s no official name registry, as that should suffice to ensure each type remains unique. Editing xattrs is normally performed silently: you’re not made aware of changes in the quarantine xattr, and in most cases the only way to manage xattrs is to use the xattr command tool, or one of very few apps like xattred that can edit and manage them.
Storage
Xattrs aren’t stored with a file’s data, nor with its normal attributes.
For smaller extended attributes up to 3,804 bytes, their data is stored alongside the xattr in the file system metadata. Larger extended attributes are stored as data streams, with separate records, but still separate from the file data. Apple doesn’t give a limit on the maximum size of xattrs, but they can certainly exceed 200 KB, and each file and folder can have an effectively unlimited number of them.
Persistence
Metadata varies in its useful lifetime. Some xattrs are intended to have a short life and then either be rewritten or discarded, while others are expected to be more or less permanent. Files that are copied to non-native file systems including FAT and ExFAT volumes can preserve xattrs in hidden shadow files, but those copied to NFS will have all their xattrs stripped.
Moving a file with xattrs within the same volume shouldn’t affect those xattrs, as they remain within the same file system, but copying them to another volume, even if both use APFS, may leave some of them behind if they’re considered to be ephemeral.
At least two xattr types may be protected by System Integrity Protection, SIP, and thus can’t be directly removed by the user. This is most widespread in the MACL xattr, com.apple.macl, but can also be seen sometimes in provenance, com.apple.provenance. The way that works is quite distinctive, in that cutting or deleting the xattr is successful for an instant before macOS adds the xattr back. As SIP protection doesn’t survive copying to another volume, when you need to remove a MACL xattr you can do so on a copy of the file on a different volume.
The most complex situation is when a file with xattrs is moved to iCloud Drive. The Mac that originated that file is likely to retain most if not all of its xattrs, because the local copy remains within the same volume and file system. However, not all xattrs are copied up to iCloud storage, so other Macs accessing that file may only see a small selection of them. The rules for which xattrs are to be preserved during file copying, including in iCloud Drive, are baked into macOS, and are outlined in the Appendix at the end.
In 2013, as part of its enhancements for iCloud in particular, Apple added support for flags on xattrs to indicate how those xattrs should be handled when the file is copied in various ways. Rather than change the file system, Apple opted to append characters to the end of the xattr’s name.
If you work with xattrs, you’ve probably already seen this in those whose name ends with a hash # then one or more characters: those are actually the flags, not part of the name, what Apple refers to as a ‘property list’. To avoid confusion I refer to them as xattr flags. A common example of this is com.apple.lastuseddate#PS, which is seen quite widely.
Appendix: Xattr flags
When first introduced in Mac OS X, no provision was made for xattrs to have type-specific preservation, and that was added later using flags suffixed to the xattr’s name. For example, the com.apple.lastuseddate xattr found commonly on edited files is shown with a full name of com.apple.lastuseddate#PS to assign the two flags P and S to it, and the most recent xattr com.apple.fileprovider.pinned, used to mark files in iCloud Drive that have been pinned, has the two flags P and X assigned to it for a the full name of com.apple.fileprovider.pinned#PX.
This is a kludge, as you normally have to refer to the xattr name including its flags, although the flags aren’t really part of its name. It’s further complicated by a set of system tables for some standard xattr types that don’t have flags suffixed, but are treated as if they do. One notable example of those is the quarantine xattr com.apple.quarantine, which is handled by macOS as if it has the PCS flags attached, although those are never used when referring to it by name.
There are also lower case flags that can be used to override those set in system tables, although those appear to be used exceedingly rarely, and I don’t recall ever coming across them. In theory, if you were using a new type based on the standard com.apple.metadata: family, com.apple.metadata:kMDItemNew, you could alter its behaviour to some similar types with the flags psB, as in com.apple.metadata:kMDItemNew#psB. I have no idea whether that would be respected in practice. For the rest of this article, I will ignore the existence of those lower case flags.
Intents
File operations involving decisions about the preservation of xattrs are simplified into the following intents:
copy – simply copying a file from a source to a destination and preserving its data, such as using cp, is labelled XATTR_OPERATION_INTENT_COPY
save – saving a file when probably changing its content, including performing a ‘safe save’; this may over-write or replace the source with the saved file. Some xattrs shouldn’t be preserved in this process of XATTR_OPERATION_INTENT_SAVE
share – sharing or exporting this file, perhaps as an attachment to email, or placing the file in a public folder. Some sensitive metadata shouldn’t be preserved in XATTR_OPERATION_INTENT_SHARE
sync – syncing the file to a service such as iCloud Drive, in XATTR_OPERATION_INTENT_SYNC
backup – backing the file up, perhaps using Time Machine, in XATTR_OPERATION_INTENT_BACKUP.
Flags
As of macOS 15.0 (including 26.0), the following flags are supported:
C: XATTR_FLAG_CONTENT_DEPENDENT ties the flag with the file contents, so the xattr has to be recreated when the file data changes. This may be appropriate for checksums and hashes, text encoding, and position information. The xattr is then preserved for copy and share, but not in a safe save.
P: XATTR_FLAG_NO_EXPORT doesn’t export or share the xattr, but preserves it during copying.
N: XATTR_FLAG_NEVER_PRESERVE ensures the xattr is never preserved, even when copying the file.
S: XATTR_FLAG_SYNCABLE ensures the xattr is preserved during syncing with services such as iCloud Drive. Default behaviour is for xattrs to be stripped during syncing, to minimise the amount of data to be transferred, but this flag overrides that.
B: XATTR_FLAG_ONLY_BACKUP keeps the xattr only in backups, including Time Machine, where there’s no desire to minimise what’s backed up.
X: XATTR_FLAG_ONLY_SAVING keeps the xattr only when saving and in backups, including Time Machine (macOS 15.0 and later only).
There’s another system limit that must be adhered to: total length of the xattr name including any # and flags cannot exceed a maximum of 127 UTF-8 characters.
System tables
These are hard-coded in source, where * represents a ‘wild card’:
com.apple.quarantine – PCS preserved in copy, sync, backup
com.apple.metadata:kMDItemCollaborationIdentifier – B backup
com.apple.metadata:kMDItemIsShared – B backup
com.apple.metadata:kMDItemSharedItemCurrentUserRole – B backup
com.apple.metadata:kMDItemOwnerName – B backup
com.apple.metadata:kMDItemFavoriteRank – B backup
com.apple.metadata:* (except those above) – PS copy, save, sync, backup
com.apple.security.* – S or N depending on sandboxing, see below
com.apple.ResourceFork – PCS copy, sync, backup
com.apple.FinderInfo – PCS copy, sync, backup
com.apple.root.installed – PC copy, backup.
System defaults for com.apple.security.* depend on whether the app performing the file operation is running in an app sandbox. Non-sandboxed apps apply S to preserve the xattr for copy, save, share, sync, backup; for sandboxed apps N is applied so the xattr is never preserved, even when copying the file.
Flags and intents
We can now revisit the list of intents, and establish the effects of xattr flags on each, as:
XATTR_OPERATION_INTENT_COPY preserves xattrs that don’t have flag N or B or X
XATTR_OPERATION_INTENT_SAVE preserves xattrs that don’t have flag C or N or B
XATTR_OPERATION_INTENT_SHARE preserves xattrs that don’t have flag P or N or B or X
XATTR_OPERATION_INTENT_SYNC preserves xattrs if they have flag S, or have neither N nor B
XATTR_OPERATION_INTENT_BACKUP preserves xattrs that don’t have flag N.
Finally, Apple provides separate information on how xattrs are synced by FileProvider, for iCloud Drive and third-party cloud services using that API. This confirms that the S flag should sync a xattr, but is vague on other flags, simply stating “some older attributes are also synced”. However, a cap is applied on the maximum size of xattrs that are syncable, at “about 32KiB total for each item”. If the xattrs exceed that limit “the system automatically makes some of the attributes nonsyncable.” More puzzlingly, it states “the resource fork is content and isn’t included in the extended attributes dictionary.”
Sources
xattr_flags.h, xattr_flags.c, xattr_properties.h in copyfile source, e.g. at Apple’s OSS Distributions Github
man xattr_name_with_flags(3), included in copyfile source FileProvider (Apple).
Apple silently introduced the com.apple.macl extended attribute in macOS Catalina, as part of its new ‘privacy by user intent’ protection, as detailed here. Its first public mention came in October 2019, when Quinn “The Eskimo!” revealed its purpose and the fact that it contained a two-byte header of 01 00 and a UUID associated with the app that obtained access ‘by intent’.
It next caught the eye of Jeff Johnson, who described it in detail on 18 December 2019. It was he who first reported its most obvious problem: “It turns out that the com.apple.macl extended attribute is governed by System Integrity Protection, so the only way to delete it is to disable SIP, or boot into another volume and delete it from there. Thus, once you implicitly grant special access to a file, you can’t easily revoke that special access.”
It was soon associated with other problems: “open a PDF in Preview without saving it, and it will be given a quarantine flag but no com.apple.macl xattr. If you try to Save that to overwrite the original document, Preview promptly refuses, and may not even be able to write that file out under another name.”
Further details of how this MACL xattr are involved in privacy protection were given by Adam Chester in October 2020. He again described the contents as consisting of a two-byte header, incremented by now to 02 00, followed by a UUID representing the app with permission to access the file, and pointed out that UUID is “unique for each system, user and application”.
Armed with a way of generating an endless supply of MACL xattrs using my test app Insent, and a little crawling through xattrs using xattred, I set out to get further details of what they now contain and their life cycle.
Header
Their two-byte header follows one of two patterns:
the first byte incrementing from 01 (2019) to 08 (2025) over time, as if a version number, followed by either 00 or 40.
the first byte 00, followed by 43, 81 or C1.
These appear to determine the effect of the MACL, for example 08 00 granting access to a protected directory for listing its contents, and read access to files within it.
UUID
All UUIDs seen in MACL xattrs are version 4, so have largely random content, and are saved in binary form. As far as I’m aware, no one has yet discovered how to identify the app identified by these UUIDs, other than by finding another MACL xattr with the same, known app association. This is made more complex by the fact that the same app is identified by different UUIDs in different sessions, as explained below. As a result, once a Mac has restarted, there’s no way to match previous UUIDs with the apps they identified.
Presentation
Each header-UUID pair is packed consecutively in time order, the most recent last in the xattr, into blocks of four, totalling 72 bytes, with trailing space padded by zeroes, making the size of each MACL xattr a multiple of 72. The great majority of them consist of only 72 bytes.
Life cycle
Each new header and UUID is written to the xattr at the time that the kernel grants access to that directory or file, and apparently remains there indefinitely, with more recent header-UUID pairs being appended at the end.
Currently, in macOS 26.4, each app’s UUID is unique to that session, and restarting that Mac ensures the app will subsequently be identified using a different UUID. This may be the result of tokens used by the sandbox being invalidated on rebooting, as described by Adam Chester.
For example, here are the header-UUID pairs associated with Insent in three consecutive sessions, taken from the same MACL xattr on the ~/Desktop directory:
0800 BF6F283B-2179-4155-AA30-FAA4C4B7ACBE
0800 03475E10-B904-447A-87F0-641B3F61B377
0800 86522100-DF0A-4AD2-BE42-F98A28374ECC
Within each session, each MACL xattr written for Insent to three different directories was the same.
MACL xattrs are still protected by SIP, in that any attempt to remove the xattr results in it being restored immediately.
The effect is that grants of access by intent were invalidated by restarting the Mac, as the previous UUID was no longer recognised as corresponding to Insent. Previous criticism that this special access persisted and couldn’t be revoked has been addressed, albeit crudely, as that does require restarting.
Summary
The com.apple.macl extended attribute was introduced in macOS Catalina to enable ‘privacy by user intent’.
Each record in the xattr consists of a two-byte header followed by a UUID.
Records are stored consecutively, most recent last, in 72-byte blocks with trailing zero padding in the last block, when necessary.
The header can have either of two forms, one starting with a version number from 01 to 08, followed by 00 or 40. The other form starts with 00, followed by 43, 81 or C1.
The UUID identifies the app granted access via the MACL xattr, but an app’s UUID changes after a restart, limiting tracking.
Currently, because restarting results in changed UUIDs, access granted by user intent should also be reset after a restart.
MACL xatrrs can’t be removed directly by the user as they’re protected by SIP.
Login
