© Bryan Anselm for The New York Times
Over the last few years, files and apps on our Macs have started to bristle with unfamiliar extended attributes (xattr). The oldest is the quarantine xattr, containing the quarantine flag, dating back to the introduction of Gatekeeper in 2012. Although its primary purpose is to determine which apps should undergo first run checks, it’s also to be found on many files. Then in macOS Catalina, the MACL xattr appeared and now seems to get attached to pretty well everything, no matter where it has come from. It was joined by the provenance xattr in macOS Ventura, and that too is spreading like wildfire on both apps and files. This article reviews why they’re there, and what you can do about them.
Since its introduction, Gatekeeper has drawn a distinction between apps that originated outside the Mac, and those that can be fully trusted, when performing security assessments on the first occasion. To enable that, apps that download items from the internet, or transfer them from another system on the same network, attach a quarantine xattr to every file that arrives on your Mac. When archives are decompressed, for example, the quarantine xattr is propagated to every file they contain. Gatekeeper then performs full first run checks on those apps, and in the right circumstances they may be run in translocation.
This com.apple.quarantine xattr is also attached to non-executable files, where its role isn’t clear, as they aren’t checked by Gatekeeper, and their quarantine flag isn’t cleared after they have been opened for the first time. However, you’ll find them on all items that have been downloaded by an app or tool that attaches them. As this xattr isn’t protected in any way, it’s straightforward to remove, although you should avoid doing so for apps whose origins could be suspicious, as that would prevent Gatekeeper from running its additional checks.
This is thought to be an abbreviation for Mandatory Access Control List, and might be intended to preserve privacy while allowing the user to open files. The com.apple.macl xattr is now probably the most common of all, as these get attached to any and every file, including apps, even if they were created on that Mac and never left its local storage.
This xattr contains 72 bytes of what could be two UUIDs, or just binary data. However, it’s protected by SIP, preventing any user from stripping it. This can be responsible for problems, for example files that can’t be opened in their default editor app, and some that can’t be saved. In the past one way of triggering this blocking behaviour was to set a document to be opened by default using an app other than its normal app, then saving it from that app before trying to open it again.
Perhaps the simplest way to remove this xattr is to copy the file to another volume, where the xattr is no longer protected by SIP, stripping it using my free editor xattred or the xattr command tool, then copying that back to its original location. Although it’s likely to be given another MACL xattr shortly, that should be less prone to cause problems.
Most recent versions of macOS have what’s known as a Provenance Sandbox that enables the security system to track the origins of files, and trace which app has altered them. This has recently been detailed in full in Koh M. Nakagawa’s account of XProtect Remediator. It operates quite differently from the regular app sandbox, and doesn’t appear to impose any restrictions.
Apps that aren’t signed by Apple are assigned an 11-byte integer when they first clear Gatekeeper’s checks, and those are entered into the Provenance Tracking table in the ExecPolicy database, and attached to the app in the com.apple.provenance xattr. When that app performs operations like opening a file in write mode, or creating a new one, the same xattr with that app’s provenance ID is attached to the file. Thus, by checking the provenance ID on any file with the xattr, the app that last wrote to the file can be identified.
Provenance IDs and xattrs aren’t assigned to Apple’s own apps, or those installed from the App Store, but they are to apps that are signed using certificates other than Apple’s, and those that are notarised. When a file is created or changed by an app without a provenance ID, no xattr is attached to that file, and any existing xattr is left unchanged.
This is a powerful tool in gathering security intelligence. For example, suppose a Mac has just installed previously unknown malware that started to write files in one of the locations watched by behavioural XProtect under one of its Bastion rules. Those could be inspected, perhaps by one of the scanning modules in XProtect Remediator, the provenance ID checked against details in the Provenance Tracking table, and information forwarded to Apple for further investigation.
Evidence so far suggests that you don’t want to try to tamper with the provenance xattr, as it doesn’t appear to have any role in blocking access to files, and is working on our side. Like the MACL xattr, it’s now normally protected by SIP, so can’t be removed directly.
Logincom.apple.quarantine is likely to be found on any app or file downloaded or transferred from another system, but appears harmless.com.apple.macl is likely to be attached to most apps and files, even those that have remained local at all times. It can sometimes cause problems including blocking the file from being opened or saved, but is hard to remove as it’s protected by SIP.com.apple.provenance is used to track which app has created or modified files. This can be important in security intelligence, so shouldn’t be removed, although it appears harmless and is working for our benefit.Loose words confuse, and one of the words used most loosely in macOS is permissions. This explainer tries to separate them from other access controls.
The access permissions of files and folders are set in their attributes in the file system, stay with that item, and are applied universally for all apps and processes that try to access them.
The simplest and most basic of access controls, these can be inspected and changed in the Finder’s Get Info dialog for all accessible files and folders. They control the ability of apps and other code to read from and write to each file and folder. Normally, if you’re the named owner of a file or folder, you expect to have both read and write access, and that ensures the apps you run with user privileges can open, edit and save changed files.
Permissions are relatively crude controls, so Access Control Lists (ACLs) can refine those permissions with more specific restrictions. They were introduced in Mac OS X 10.4 Tiger in 2005, and are now applied as standard to some widely used folders including the Home folder. The presence of ACLs is normally indicated in the Get Info dialog by the words You have custom access.
No matter what security controls and privacy protection might give you access to, they can’t override the fundamental limits imposed by permissions, and can only limit access further.
The macOS security system imposes its own controls according to its rules. Most obvious among those are System Integrity Protection (SIP) and sandboxing.
SIP was introduced in 2015 with El Capitan, and primarily puts system folders, files and some components, including certain extended attributes, beyond the reach of even the root user. The only way to get past SIP is to disable it, a serious undertaking as it has more general effects on security.
Sandboxing is security protection that limits the files individual apps can access by imposing a sandbox as set by their entitlements. It’s therefore determined by the app trying to access files, and doesn’t apply to apps that don’t run in the sandbox. As an app’s entitlements are baked into its signature, there’s nothing a user can do to alter them.
macOS designates certain locations and resources as being private, and protects them using its Transparency, Consent and Control (TCC) system. Although it has a longer history, this was first implemented in its current form in macOS 10.14 Mojave in 2018, and since then has grown with every new major version of macOS. Privacy protection operates outside of file attributes, using a rule-based system applied to each app, and applies to command tools and other processes as well.
Among the folders this protects are the Desktop, Documents, Downloads, and those on removable storage. While access to individual folders is controlled, if you do encounter problems it’s usually simplest to add that app to the list of those with Full Disk Access, in Privacy & Security settings, in the first instance. That can leave a lot of apps with unnecessary access to private data, so you should periodically check through the list of apps with Full Disk Access to ensure they all still require it. Remember that Full Disk Access can’t override restrictions imposed by permissions or ACLs.
In Tahoe, privacy-protected folders include:
Unlike permissions and security controls, there’s no command line interface to these controls, which can only be accessed by the user in Privacy & Security settings. As a result, TCC uses an attribution chain that traces up through the call chain to an app that is responsible for the privacy settings to be applied. For example, when you run commands in Terminal, the privacy settings used by TCC are those of the Terminal app, while helper apps are normally the responsibility of their parent app.
Privacy protection is built around the user’s consent and intent. When a process tries to save a file to a protected folder that it doesn’t already have access to, you should be prompted to give your consent before TCC allows that. Alternatively, when an app tries that, it should display the File Save dialog, where you can express your intent to save the file to that folder. Without consent or intent, TCC should block that file from being saved there unless that app has been given Full Disk Access in Privacy & Security settings.
Privacy protection is the most complex and opaque of these, and can present the toughest problems to solve.
For many years, I believed what the log command told me, that log entries could contain any of the 26 or more fields available, although they didn’t. It was only as I was developing utilities like LogUI that I discovered that log entries come in (at least) four types, each with its own format and set of fields. This article is a guide to those, and how they are displayed in LogUI.
Those available in the OSLog API are fewer in number than appear in log command output, and include some of limited interest. Those not exposed in LogUI include:
LogUI displays each entry using the same order of fields and colour-coding. Although they can’t all be seen together in any single log entry, the overall sequence is:
Standard levels are given as text, and chosen from one of the following: fault, error, notice, info, debug, undefined. These are only used with Regular log entries, not other types. LogUI always includes debug level entries when they’re available.
The four types of entry are:
These are normally by far the most common, and are seen throughout the processes and subsystems writing log entries. They normally contain up to the following fields:
datestamp, [1], activity ID, category, level, sender, process, process ID, subsystem, thread ID, message
These examples include:
datestamp, 1, activity ID, category, level, sender, process, process ID, subsystem, thread ID, message
which is fairly normal.
These mark specific activities, and vary according to the process and subsystem. Some are valuable waypoints, and all are recognisably shorter than most regular entries, containing only the following fields:
datestamp, [2], activity ID, parent activity ID, sender, process, process ID, thread ID, message
These are examples of one of the most valuable activities reported, indicating a mouse-click or tap. They’re also complete, as they consist of:
datestamp, 2, activity ID, parent activity ID, sender, process, process ID, thread ID, message
These are rare but instantly recognisable because of their brevity. They consist of just three fields:
datestamp,[3], message
Two of the most important are the announcement of the start of kernel boot:
and adjustment of the internal clock:
These are also easy to search for, as their message contains the distinctive === string.
By default, LogUI extracts omit Signposts, but if you tick the Show Signposts box before loading log entries, they will also be displayed. In parts, they can outnumber regular entries, and are unlikely to provide you with meaningful information unless you know internal details of their process. Each Signpost adds three type-specific fields, shown after the process ID:
datestamp, [4], activity ID, category, sender, process, process ID, signpost ID, signpost name, signpost type, subsystem, thread ID
These examples come from the most prolific user of Signposts, SkyLight for WindowServer, and contain
datestamp, 4, activity ID, category, sender, process, process ID, signpost ID, signpost name, signpost type, subsystem, thread ID
Note there’s no message field, as their meaning is determined by the signpost fields.
The OSLog API doesn’t currently give access to Mach times, only to opaque Date variables. From those, LogUI’s datestamps give:
year–month–day hour:minute:second.microsecond+timezone
All times and time zones given are those current when LogUI obtains that extract, not when that entry was written. This can become extremely confusing when clocks change to and from {summer time, daylight time, daylight saving time, daylight savings time, DST}, and when passing through time zones, and great care is required when reading logs containing such corrections. LogUI tries to make this easier, but care is still required.
If you have used the log command, you may be aware that can return Mach times as ticks, offering precision down to the nanosecond. This currently isn’t possible when using the OSLog API, but given the relatively slow clock of Apple silicon Macs, it’s no longer as useful as it used to be with Intel’s nanosecond Mach ticks.
By default, all data in message fields is censored to protect the privacy of the user. This replaces chunks of text with the dreaded <private>, and can render many entries devoid of any useful information. In recent versions of macOS log privacy can be removed by installing a profile. However, that doesn’t apply retrospectively, only to log entries collected after the profile has been installed.
Removing privacy also increases the size in storage of log entries, and may reveal sensitive information. As soon as you have completed acquisition of the logs you need with privacy removed, you should therefore remove the profile, to allow normal censorship to resume.
A suitably signed profile is here: enablelogprivatedata
To install the profile, unzip the archive into a convenient folder and double-click it. You’ll be informed by a notification that you need to review the profile to install it. Open System Settings and select Device Management in its View menu, or search for it. Follow the instructions given there to install and activate it.
Sadly, this doesn’t cover all censorship. Some systems require additional configuration changes, for example CFNetwork diagnostic logging. Jeff Johnson explained how to enable that, but Apple has since blocked that, and I don’t know of a practical alternative. In some cases, additional detail can be obtained by adjusting log settings.
A short practical summary of LogUI’s log entries is provided in its Help book.
