Last week, in a quiet village nestling between golf courses in the Green Belt of Surrey, to the south of London, a huge void opened up in its High Street. Some of the older locals recalled there were old mine workings in the area, making it plausible that sinkhole may be due to the collapse of those abandoned mines or tunnels. What has shocked the residents of Godstone is unfortunately not uncommon, the result of failure to restore the land and what’s under it before developing on top.

Last week, after a long period of deliberate abstinence, I returned to the subject of permissions, privacy and security protections, and how they conspire to prevent us from accessing our own documents and files. In this case, there’s a warren of underground tunnels that can collapse when you’re least expecting it, although most aren’t disused but still active and poised to turn into voids of misunderstanding.

Privacy controls over locations started in macOS Mojave back in 2018, and ~/Desktop, ~/Documents, ~/Downloads, removable volumes and others were added in Catalina the following year. These have become so “transparent”, to use Apple’s well-worn euphemism, that developers have criticised them relentlessly since, and most users are still completely at sea with them, over five years later.

Apple’s Mac User Guide for Sequoia now lists 32 protection categories from Location Services to Lockdown Mode, but explains remarkably little. It passes over in silence the distinction between settings for Files & Folders and those for Full Disk Access. Presumably it leaves each individual app with the task of explaining those to the user, in the context of that app’s potential access.

This is the extent of its current explanations for users:

• Files & Folders “allow apps to access files and folders in different locations on this Mac. The listed apps have requested access.”
• Full Disk Access “allows apps to access all files on your computer, including data from other apps (for example, Mail, Messages, Safari and Home), data from Time Machine backups and certain administrative settings for all users on this Mac. To add an app, click the Add button, select the app in the list, then click Open.”

For once, information given in Apple’s Platform Security Guide is briefer, and it does come a bit closer to making that distinction, even if it avoids using the term Full Disk Access, and muddies the waters by referring instead to “full internal storage access”, which isn’t accurate.

Nowhere does Apple explain how those privacy settings interact with permissions, or any of the unexpected behaviours that we’ve become used to since Mojave. For example, some still report that an app that has been able to open a document without problems is unable to save that document, even though that file and its folder have appropriate permissions set. This has been associated with documents whose default app has been changed from that set for that document type, and undocumented extended attributes such as com.apple.macl, which is even protected by SIP to prevent the user from trying to rectify the behaviour.

For developers, there’s a long series of WWDC presentations reporting the many changes that have been made to extend privacy protection without addressing its user interface, and Apple’s Developer Forums. But if your app wants to discover whether it has been given Full Disk Access by the user, “Except in very limited circumstances, there’s no good way to:

• tell if you have the Full Disk Access privilege
• explicitly ask for the privilege.”

What if alongside its concerted effort to deliver us Apple Intelligence, it were to devote a little time to design and implement a consistent and integrated interface to permissions, privacy constraints and other limitations to what we can open and save on our Macs, and deliver it in the Finder rather than by trial and error? If Apple doesn’t address this soon, these cracks could open up like that sinkhole in Godstone.

Accessing files and folders isn’t as simple as it used to be. There was a time when, if a file was in your Home folder, you had full rights over it, and apart from those parts controlled by root, you had free roam. Now macOS plays evil tricks: you try to save a file in ~/Documents only to be told that you can’t, and even when you’ve assumed root privileges much remains out of your control. This article explains five barriers that you may have to negotiate.

Those are:

• Permissions were part of Mac OS X when it was introduced nearly 25 years ago, and provides basic controls over which users and groups can read or write items, in a system common to Unix and many other operating systems.
• Access Control Lists (ACLs) refine those permissions with more specific restrictions on access to items, and were introduced in Mac OS X 10.4 Tiger in 2005.
• Privacy controls enforced by the Transparency, Consent and Control (TCC) system have included restrictions on access to specific folders since macOS 10.14 Mojave in 2018.
• System Integrity Protection (SIP) was introduced in 2015 with El Capitan, and primarily puts system folders beyond the reach of root.
• App sandboxing is security protection that limits the files individual apps can access by imposing a sandbox as determined by their entitlements.

For any app including the Finder to have access to a file or folder, all five of those must give approval.

Permissions

The simplest and most basic controls, these can be inspected and changed in the Finder’s Get Info dialog for all accessible files and folders. They control the ability of apps and other code to read from and write to each file and folder. Normally, if you’re the named owner of a file or folder, you expect to have both read and write access, and that ensures that the apps you run with user privileges can open, edit and save changed files. There’s also the Locked checkbox that might seem obvious, although it often gets overlooked as a cause of problems when saving a file.

Some locations are particularly sensitive to errors in permissions, such as all those property lists containing settings for apps and services stored in ~/Library/Preferences. They are mostly handled by a background service cfprefsd, but if permissions have become changed so that it can’t access a property list it needs to use, all sorts of strange errors can result. In the past measures have been used to correct or repair those permissions, but those problems should now be long since past, and repairs are generally unnecessary and potentially dangerous.

ACLs

Access Control Lists, ACLs, add a more sophisticated system for enforcing more specific restrictions that cover a wide range of features. Their only common use in modern macOS is to preserve standard structures of folders in the Home folder. Regular Posix permissions don’t do that well, so a common ACL used is
group:everyone deny delete
to prevent anyone from deleting the folder it’s applied to. You should find that ACL on each named user folder and the Guest folder in /Users. Within each user folder, it’s normally applied to all the standard folders, including Desktop, Documents, Downloads, Library, Movies, Music, Pictures, Public and Sites, and on some folders within ~/Library.

The result of that ACL is that any attempt to remove that folder fails silently, so preserving the standard layout. But it doesn’t affect access given to the contents of those folders, nor can it be held responsible for errors when trying to read or write files within a protected folder. Unfortunately, the Finder’s Get Info dialog isn’t exactly forthcoming about the presence or effects of ACLs, merely stating that the user has “custom access”.

Thankfully, some GUI tools give good access to ACLs, and my favourite remains TinkerTool System, which has excellent support for them, and much more besides. It’s most valuable in displaying Effective Permissions.

Built-in tools are provided in Terminal. Show all ACLs using the command ls -le to return entries such as
drwx------@ 5 hoakley staff 160 8 Oct 12:09 Desktop
0: group:everyone deny delete
drwx------@ 81 hoakley staff 2592 17 Feb 15:42 Documents
0: group:everyone deny delete

You can add an ACL using chmod, such as
chmod +a "everyone deny delete" MyFolder

Privacy protection

macOS designates certain locations and resources as being private, and protects them using its Transparency, Consent and Control (TCC) system. Among the folders that this protects are the Desktop, Documents, Downloads, and removable storage. While access to individual folders has fine control, if you do encounter problems it’s usually simplest to add that app to the list of those with Full Disk Access, in Privacy & Security settings, in the first instance. That can leave a lot of apps with unnecessary access to private data, so you should periodically check through the list of apps with Full Disk Access to ensure they all still require it. Note that Full Disk Access can’t override permissions or ACLs.

In Sequoia, privacy-protected folders now include:

• ~/Desktop
• ~/Documents
• ~/Downloads
• iCloud Drive
• third-party cloud storage, if used
• removable volumes
• network volumes
• Time Machine backups.

SIP and security

In principle, System Integrity Protection (SIP) should only apply to system files, but it’s also used to lock down some other components, including com.apple.macl extended attributes, which can be attached to user files. Those have been implicated in some problems in which regular user documents become locked down, and typically won’t save any changes. Although the exact role and use of this extended attribute isn’t clear, it’s generally accepted as designating an app that’s approved to edit that file, so providing document-specific privacy control.

SIP is easy to spot in the Finder’s Get Info dialog, as only the System has write access, and permissions are again described as “custom access”.

com.apple.macl extended attributes are normally protected by the same mechanism that applies SIP, so any attempt to remove them is likely to fail. One workaround for that is to pass the file through another volume, so that when it’s copied back the extended attribute is no longer protected, and can be stripped.

These problems appear most likely when apps other than that set as the default app to open a document type are used to edit files. Another approach to their solution is to select an affected document in the Finder, Get Info, set it to Open with the default app, then click on Change All… to reset all others to that default app.

Sandboxes

Sandboxing is a security feature that individual apps can opt into. As it’s a requirement of those distributed through Apple’s App Store, it’s now common in third-party software. The starting point is to restrict a sandboxed app to the folders created in its sandbox, saved in its folder in ~/Containers. As this severely limits the usefulness of most apps, it’s common for them to have entitlements that extend their ability to access files and folders beyond that basic sandbox.

As this is controlled by the entitlements system and baked into the app’s signature, it’s not something that the user has any control over. If a sandboxed app is unable to access files or folders that you need to use, contact its developer.

What can you change?

• Permissions are generally simple and easy to control. Beware of changing those in Library folders, but your documents and other files should be straightforward to manage.
• ACLs are more obscure and complex. Unless you understand what you’re doing, leave them well alone.
• Privacy controls are extensive and fraught. The only control you have here is giving apps Full Disk Access when needed. Check that list periodically to ensure you haven’t been overgenerous.
• SIP shouldn’t be tampered with unless you fully understand what you’re doing. On Apple silicon Macs disabling it reduces boot security to Permissive, with other consequences. Details of fine control using csrutil are here.
• App sandboxing isn’t within user control.

Whether you’ve become familiar with the basics in Recovery mode on Apple silicon Macs, or you just need deeper information, this article is intended to supplement my illustrated guide.

Which Recovery mode?

Unless your Mac has two or more systems installed, there are only two Recovery modes:

• Paired Recovery, the default, run from a mounted disk image stored in the Recovery volume in the active boot volume group;
• Fallback Recovery, engaged using a short press, then a held press of the Power button, in a ‘di-dah’ rhythm. This isn’t always available, but when it is it runs from a dedicated partition/container on the internal SSD. Although it can do everything else, you can’t use its Startup Security Utility, which is only available in paired Recovery.

This is a bit more complicated when your Mac has more than one bootable system available. The rule then is that it will enter paired Recovery for the current boot volume group. If you have Sonoma and Sequoia installed and want to do anything with the Sonoma boot volume group, then you should make Sonoma the active Startup Disk, and ideally restart into that, before shutting down to start up in Recovery mode. Although the initial Options screen allows you to choose boot volume groups, that procedure ensures that you’ll always use the correct tools for the job.

Change keyboard

Throughout Recovery mode, you can change the default keyboard, which is set to that in NVRAM. At the top right, the keyboard menu offers those standard in macOS. If you’re experiencing problems entering your password, check that you’re using the right keyboard here.

Installer Log

This is useful for the checks that it reports, including statements that the network is reachable, and details of the current user, whether they have admin privileges, and have a secure token to be able to unlock FileVault. If you need to get into the weeds, its popup menu can be set to Show All Logs, and you might even be able to save the log to somewhere that’s accessible when back in normal user mode.

Utilities: Share Disk…

This menu command is used to put that Mac into the modern equivalent of Target Disk mode. That requires it to be connected to another Mac using a Thunderbolt or USB data cable, and will then mount its Data volume on that Mac, connected using SMB over the cable. Although not as fast as a direct Thunderbolt connection, it’s valuable if you need to retrieve files from a Mac that can’t boot in normal user mode.

Utilities: Terminal

This opens a bash shell rather than the zsh we’re now used to, as root user. There are a great many useful tools included in the Recovery System, including: fsck, fsck_apfs, mount, mount_apfs, csrutil, asr, nvram, spctl, sysctl, and /usr/bin and /usr/sbin contain many more. The biggest snag with them is that, in 15.3.1 at least, there’s no man command, although many will show their standard usage information with an -h option.

One command tool peculiar to Recovery mode is its equivalent to sysdiagnose, recoverydiagnose, in case you need to file a bug report about Recovery mode.

There’s also repairHomePermissions, which launches a GUI app to repair permissions on a selected Home folder on the Data volume. I strongly recommend that you don’t try using this, as you’ll end up with the user being locked out of every folder in their Home folder. This appears to be a historical remnant that has somehow been left behind, long after it outlived its usefulness.

Utilities: Startup Security Utility

I have already explained at length how to use this to change boot security, in conjunction with adjusting SIP, and more, in this article.

Main Window: Disk Utility

Although Disk Utility invariably opens with its default settings, the version supplied in Recovery is every bit as capable as that in normal user mode. Once it has opened, set its View tool to Show All Devices, and if you want to work with snapshots, set its View menu to Show APFS Snapshots.

Main Window: Safari

As with Disk Utility, the version of Safari provided is almost as fully equipped as normal, and apparently runs from a cryptex. It opens with a local page that explains features in Recovery mode, but you can access Google search engine and pretty well anything else you want, including articles in this blog.