Normal view

There are new articles available, click to refresh the page.
Before yesterdayMain stream

What is Macintosh HD now?

By: hoakley
2 September 2024 at 14:30

Perhaps you just tried to save a document, only to be told you don’t have sufficient permissions to do so, or attempted to make another change to what’s on your Mac’s internal storage, with similar results. You then select the Macintosh HD disk in the Finder and Get Info. No wonder that didn’t work, as you only have read-only access to that disk. But if you unlock it and try to make any changes to permissions, you see

xpermserror

What’s going on?

Between macOS Mojave, with its single system volume, and Big Sur, the structure of the Mac system or boot volume has changed, with Catalina as an intermediate. Instead of Macintosh HD (or whatever you might have renamed it to) being one volume on your boot disk, it’s now two intertwined and joined together. What you see now as Macintosh HD isn’t even a regular APFS volume, but a read-only snapshot containing the current macOS. No wonder you can’t change it.

Root

Select the boot disk Macintosh HD in the Finder, and it appears to have four visible folders, Applications, Library, System and Users, just like it always did. Press Command-Shift-. to reveal hidden folders and all the usual suspects like bin, opt and usr are still where they should be. That’s the root of the combined System and Data volumes, and what’s shown there is a combination of folders on both volumes, with the top level or root on the Sealed System Volume (SSV).

The contents of those folders are also the result of both volumes being merged together using what Apple terms firmlinks:

  • Applications contains apps installed in your own Applications folder on the Data volume, and those bundled in macOS on the SSV. You can see just the latter in the path System/Applications, where they appear to be duplicated, but aren’t really.
  • Library comes only from the Data volume, and all its contents are on that volume. But inside it, in the path Library/Apple/System/Library are some components that should appear in the main System/Library.
  • System comes only from the SSV, although it has some contents merged into it using firmlinks, such as those folders in Library.
  • Users also comes only from the Data volume, and includes all Home folders for users.

So while the root of Macintosh HD might be in the SSV, much of its contents are on the Data volume, and can be written to, even though the root is a read-only snapshot, thanks to those firmlinks.

Data volume

There are two places that mounted volumes are listed in the Finder: the hidden top-level folder Volumes, where Macintosh HD is just a link back to the root complete with its merged volumes, and in System/Volumes, where what’s shown as Macintosh HD is in fact not the merged volumes, but only the Data volume. You can confirm that by looking at what’s in System/Volumes/Macintosh HD/System, where you only see the parts of the System folder that are stored on the Data volume, and not those stored on the SSV.

What is more confusing there is that System/Volumes/Macintosh HD/Applications is the same merged folder containing both user and bundled apps as in the top-level Applications folder. That’s an artefact resulting from the way that its firmlink works.

But if you open the Get Info dialog on System/Volumes/Macintosh HD, you’ll see the same as with the root Macintosh HD disk, information about the root and not the Data volume.

Mounted in System/Volumes are several other volumes like VM and Preboot, and (depending on whether this is an Intel or Apple silicon Mac) folders such as Recovery and xarts, that you really don’t want to mess with.

Permissions problems

Tackling problems that appear to be the result of incorrect permissions is best done at the lowest folder level. If you’re trying to save a document to the Documents folder inside your Home folder, select that and Get Info on it. Chances are that you are the owner and have both Read & Write permissions as you should. In that case, the problem most likely rests with privacy protection as in Privacy & Security settings. You then suffer Catch-22, as you can only effect changes to those by closing and opening the app, and as you can’t save your document before closing the app, you’re at risk of losing its contents. You may have better luck trying a different folder, creating a new one inside your Home folder, or using the Save As… command instead (which may be revealed by holding the Option key when opening the File menu).

Full layout

In case you’re wondering exactly which folders are merged into the hybrid Macintosh HD ‘volume’, those are shown below in increasing levels of detail, starting with the broad layout.

BootVolGpVentapfs

Then to a simplified version of the full layout.

BigSurIntSimple

Finally, in complete detail.

BigSurIntegrated

Happy navigating!

Last Week on My Mac: Layered security and herd immunity

By: hoakley
25 August 2024 at 15:00

Reviewing security products intended to detect and remove malicious software is far harder than it used to be. There was a time when all you had to do was set your virtual machine to Reduced Security, disable SIP, and maybe Gatekeeper too, then strip any quarantine extended attributes from samples of recent malware. With nothing to trigger checks by macOS security, the malware was fully exposed to the security software under test, and you could assess how successful the product was at detection.

Yet in Sonoma it seems that you also need to disable AMFI (Apple Mobile File Integrity), or macOS will intervene to detect the malware before the product under test, which then won’t get a look-in.

LaunchSonomaApp1

Back in Sonoma 14.4.1 I summarised those layers of checks in this diagram.

Although it’s encouraging that malware detection is now so pervasive, this makes it almost impossible to assess the other side of an anti-malware product, how well it can remove the malicious software it discovers. The same applies to Apple’s own XProtect Remediator. Stunting and tricking macOS to the point where malware can deploy fully isn’t anywhere near as easy as it was.

xprmensis01

You know you have failed again when macOS pops up an alert like this, telling you that it has intercepted your attempt to bypass its protection.

From a user’s point of view, this can only be good. macOS security protection is designed and applied in multiple layers so that, even if something manages to trick its way past one check, the next layer is there to stop it in its tracks.

The strangest thing about XProtect Remediator isn’t the fact that, left to its own devices, it doesn’t inform the user when it detects malware, but that it will happily remove the malware it detects and let you carry on using your Mac as if nothing had happened. If you’re as old school as me, you might wonder why you shouldn’t have to wipe your Mac completely, restore it in DFU mode if an Apple silicon model, and rebuild it from scratch. Surely the slightest suspicion of anything malicious demands such a scorched earth approach?

That too depends on what security protection your Mac has active. If that includes SIP and the SSV, then there’s no known malware that can alter anything on the SSV, and what it can do on your Data and other volumes is also limited. The days of viruses wreaking havoc throughout your entire Mac thankfully seem long since past. Viruses, of course, were designed to replicate themselves throughout a Mac’s storage, but today’s trojans and stealers are after two things, and those are your secrets and money.

But that depends on the behaviour of potential victims of malware. Like any business, those who try to profit from theft of our secrets and other malicious behaviour have to consider the size of their market of victims. For many years, this was one of the Mac’s defences against malware: why would anyone want to develop for such a small proportion of potential victims, when PCs readily provided far more? More recently, some have unfortunately recognised our potential, and now we’re suffering the consequences.

This is where standard macOS security protection comes into play. At present, the great majority of Macs running Big Sur or later have SIP enabled and their SSV fully protected. The victim market for malware requiring write access to the System volume is tiny. But what if it became more usual for Macs to be unprotected, and all that malware had to do was gain root privileges to be able to write to the System volume?

This is a phenomenon akin to vaccination and resulting herd immunity in pandemics like Covid. States and nations with low protection by vaccination and little herd immunity suffered the highest rates of infection and consequent death rates. But so few realised that exercising their personal choice to remain unvaccinated had that effect, despite the fact that humans throughout the world had eliminated the deadly disease of smallpox by building herd immunity in the late twentieth century.

Security protection in macOS works in layers. Each layer you disable opens wider the window of opportunity for attackers. The more Macs that have any given layer of protection disabled, the lower the herd immunity to attack, and the more likely that malware will try to take advantage of that.

Dual-booting your Mac with multiple versions of APFS

By: hoakley
22 August 2024 at 14:30

Since its first public release seven years ago in High Sierra, APFS has changed greatly. Connect a bootable external disk with Sonoma installed to a Mac running macOS 10.13 and it won’t know what to make of it. That’s because many of the features used by APFS today didn’t exist until Catalina and Big Sur. This article explains how your Mac can cope with running such different versions of APFS, and how to avoid the problems that can arise when a Mac can start up in two or more different versions of macOS.

Fences

Disk structures, APFS and macOS fall into three main phases:

  • High Sierra and Mojave, which can run 32-bit code, boot from a single integrated volume, and don’t understand Signed System Volumes (SSVs); they run APFS versions 748.x.x to 945.x.x.
  • Catalina (macOS 10.15), which can’t run any 32-bit code, and boots from a group of volumes where the system is contained in its own read-only volume; it runs APFS version 1412.x.x.
  • Big Sur and later, which can’t run any 32-bit code, and boot from an SSV that’s a specially sealed snapshot on the System volume; they run APFS version 1677.x.x and later.

Fuller and finer details of the changes with different versions of macOS are given in the Appendix at the end.

If you need your Mac to run more than one version of macOS, it’s easiest and most compatible if the versions installed come from only one of those phases. Two phases are more of a challenge, and all three needs great care to ensure that one version doesn’t damage the file systems of another.

When you do need to run macOS from two or more phases, the cleanest and safest way is to only mount disks and volumes from one phase at a time. For example, when running Mojave, if you unmount disks and volumes for all later versions of macOS, then you won’t see any notifications warning you of Incompatible Disk. This disk uses features that are not supported on this version of macOS. Unfortunately, unless all your boot systems are on external disks, this isn’t easy to achieve.

Warnings

Generally speaking, APFS tools including those run by Disk Utility (including fsck_apfs, used by First Aid) are backward-compatible, but may not be as reliable when a tool from an older version of macOS is run on a newer version.

First Aid and fsck_apfs normally report versions of APFS tools used on the volumes and containers they’re checking. These draw attention to any potential for incompatibilities, such as:
The volume [volume name] was formatted by diskmanagementd (1412.141.1) and last modified by apfs_kext (945.250.134).
telling you that volume was created by macOS 10.15, and last changed by macOS 10.14.

Other messages you might see include warnings like: warning: container has been mounted by APFS version 2236.141.1, which is newer than 1412.141.3.7.2, which is less helpful.

If you’re going to run multiple versions of macOS on the same Mac, you’ll have to get used to those. For reference, APFS versions decode to macOS versions as:

  • 249.x.x is the beta release from macOS 10.12 Sierra
  • 748.x.x is 10.13 High Sierra
  • 945.x.x is 10.14 Mojave
  • 1412.x.x is 10.15 Catalina
  • 1677.x.x is 11 Big Sur
  • 1933.x.x is 12 Monterey up to 12.2
  • 1934.x.x is 12.3 Monterey and later
  • 2142.x.x is 13 Ventura
  • 2235.x.x is 14 Sonoma up to 14.3
  • 2236.x.x is 14.4 Sonoma and later
  • 2311.x.x and later is macOS 15 Sequoia beta.

Dangers

To minimise the risk of any problems arising, any manipulation of the file system should be performed by Disk Utility, the diskutil command tool or others, for the same or later version of macOS. Thus, if you have Mojave and Big Sur installed, you could use tools from either to maintain Mojave file systems, but should only use those for Big Sur when maintaining Big Sur’s file systems.

This becomes more difficult when file system maintenance needs to be performed in Recovery mode. Once the Mac has started up in Recovery, check the version of macOS before opening Disk Utility, using fsck_apfs, diskutil or anything similar.

This is one situation where versions of macOS with an SSV can become tricky, because of their different associations with Recovery systems. On Apple silicon Macs, Big Sur doesn’t use a paired Recovery volume, whereas Monterey and later do. To be confident that the Mac starts up in the correct version of Recovery, it should have been running that version normally before you start it up in Recovery. Once in Recovery, check the version of macOS before proceeding to work with its file systems.

Appendix: APFS and macOS version details

APFS major version numbers change with major version of macOS:

  • macOS 10.12 has APFS version 0.3 or 249.x.x, which shouldn’t be used at all.
  • 10.13 has 748.x.x, which doesn’t support Fusion Drives, but has basic support for volume roles.
  • 10.14 has 945.x.x, the first version to support Fusion Drives.
  • 10.15 has 1412.x.x, the first version to support the multi-volume boot group, and introduces extended support for volume roles, including Data, Backup and Prelogin.
  • 11 has 1677.x.x, the first version to support the SSV, and Apple silicon. On M1 Macs, it doesn’t support the paired Recovery volume, though.
  • 12 has 1933.x.x until 12.2, thereafter 1934.x.x, which support the paired Recovery volume on Apple silicon.
  • 13 has 2142.x.x, and is probably the first to support trimming of UDRW disk images and their storage as sparse files.
  • 14 has 2235.x.x, until 14.3, thereafter 2236.x.x.

Minor version numbers increment according to the minor version of macOS, and patch numbers wander without pattern.

How to install a Combo update of recent macOS?

By: hoakley
19 August 2024 at 14:30

When a solution works once, we tend to keep using it even though it may no longer be necessary, or may no longer be possible in the same way. This article is about one example, how we used to update macOS using Combo updaters in the days before Big Sur. Now those are no longer available, should we perform a fresh install of macOS whenever there’s an update?

How macOS updates worked

Before we upgraded to Big Sur, macOS updates were relatively simple, and based on installer packages, structured collections of files that are copied to the boot volume in accordance with scripts. Once that copying is complete and the boot volume contains the files forming the new version of macOS, the Mac can be rebooted from those.

Take the last of those major versions of macOS, Catalina. In the diagram below, I show in miniature what happened during the course of its first two minor updates, as an example.

comboupdate1

Updating from Mojave to Catalina 10.15.0 was performed using installer packages that completely replaced everything in its system, then created in a new System volume, as shown at the top. This was referred to as an upgrade, as the whole system was installed afresh.

Then (skipping a Supplemental Update) came the first minor update to 10.15.1 on 29 October 2019. For the sake of this example, consider that includes a new version of the cat command tool. Rather than that update containing the whole system all over again, it only contains and installs what has changed, including the new cat binary.

Then, on 10 December 2019, Apple released the update to 10.15.2, containing another set of changed files, this time perhaps replacing the cp command tool. There are two ways to install that update, either as a delta update or a Combo. The delta update contains the changes from 10.15.1, here cp; the Combo update contains everything that has changed since 10.15.0, both that in 10.15.1, here cat, and in 10.15.2 with cp.

Combo updates tackled what was then a common problem, when a previous update hadn’t been installed correctly. If the update to 10.15.1 didn’t actually install the new cat and we then installed the delta update for 10.15.2, the latter wouldn’t have worked properly. As there was limited error checking in those old updaters, we would then be using a flawed installation of macOS. By installing the 10.15.2 Combo updater instead of the delta, the chance of errors like that were significantly reduced, and we were thoroughly impressed by updating using the Combo rather than delta update.

In fact, it was seldom necessary to install the Combo update every time, as most updates went well and errors weren’t as common as it seemed. What I have always recommended has been to install the delta update first, and only to resort to the Combo version if that didn’t work properly.

Big Sur and the SSV

Catalina’s novel boot volume group, with its separate System and Data volumes, was only an intermediate step to Big Sur, which brought the greatest structural change in Mac operating systems since the introduction of Mac OS X twenty years earlier. Here, the great majority of macOS runs from a mounted snapshot with a read-only file system separate from that of user files on the paired Data volume, and requires a fundamentally different method of installation and updating.

In addition to updating the contents of the system, from Big Sur onwards installers and updaters have considerably more to do. Those tasks include creating the new snapshot to contain the bootable copy of the system, firmlinking that System snapshot with its paired Data volume, building a tree of cryptographic hashes so that the snapshot can be sealed, and verifying the signature of its seal against Apple’s requirement.

comboupdate2

If we follow Big Sur’s first couple of updates, you’ll see how they have changed. On 14 December 2020, the update to 11.1.0 might have gone through similar changes to Catalina, with the replacement of its cat command tool. Once that had been installed on the System volume, a snapshot was made of that System volume, hashes were computed for all its contents, and assembled into a tree. At the top of the tree, a hash of all the hashes in the next layer down forms its seal, which is then hashed again to form the signature of the Signed System Volume (SSV). The signature is compared against that set by Apple for the whole volume, and only if they’re identical is the System snapshot accepted for use.

There’s no room here for the slightest error if the signature on the Signed System Volume (SSV) is going to match its requirement. Within the SSV are additional changes, such as large dyld caches, which serve to make this more complicated, and more recently this includes cryptexes containing system components like Safari that are kept outside the SSV so they can be updated outside full macOS updates.

When that was updated again to 11.2.0 on 1 February 2021, perhaps with a new version of the cp tool, the new installer repeats this process of creating a new snapshot, building its tree of hashes, sealing them, creating the signature, and comparing that with Apple’s signature. This means that each and every update is verified to be perfect, and no errors can be tolerated.

No Combo updates

In the new system, from Big Sur onwards, Apple only provides Combo updates for those updating from versions of macOS older than the previous version, and only when required. If you were to update straight from 11.0 to 11.2, then your Mac downloads all that it requires for that update. If your Mac is already running 11.1.0, there’s no option to install the changes in 11.1.0 again, nor could you need to, as its installation on 11.1.0 has already been verified as being identical to Apple’s reference.

If you felt that wasn’t adequate, then the only option is to install the whole of 11.2.0 from scratch, and running the Big Sur Installer app will do that for you. However, there’s no point in doing that, as the outcome will be identical, as verified by its signature match against Apple’s. All that would do is take longer, and increase wear on your Mac’s internal SSD.

That’s why Apple doesn’t provide the user with Combo updates any more, because they’re now superfluous.

❌
❌