Normal view

There are new articles available, click to refresh the page.
Before yesterdayMain stream

What is Macintosh HD now?

By: hoakley
2 September 2024 at 14:30

Perhaps you just tried to save a document, only to be told you don’t have sufficient permissions to do so, or attempted to make another change to what’s on your Mac’s internal storage, with similar results. You then select the Macintosh HD disk in the Finder and Get Info. No wonder that didn’t work, as you only have read-only access to that disk. But if you unlock it and try to make any changes to permissions, you see

xpermserror

What’s going on?

Between macOS Mojave, with its single system volume, and Big Sur, the structure of the Mac system or boot volume has changed, with Catalina as an intermediate. Instead of Macintosh HD (or whatever you might have renamed it to) being one volume on your boot disk, it’s now two intertwined and joined together. What you see now as Macintosh HD isn’t even a regular APFS volume, but a read-only snapshot containing the current macOS. No wonder you can’t change it.

Root

Select the boot disk Macintosh HD in the Finder, and it appears to have four visible folders, Applications, Library, System and Users, just like it always did. Press Command-Shift-. to reveal hidden folders and all the usual suspects like bin, opt and usr are still where they should be. That’s the root of the combined System and Data volumes, and what’s shown there is a combination of folders on both volumes, with the top level or root on the Sealed System Volume (SSV).

The contents of those folders are also the result of both volumes being merged together using what Apple terms firmlinks:

  • Applications contains apps installed in your own Applications folder on the Data volume, and those bundled in macOS on the SSV. You can see just the latter in the path System/Applications, where they appear to be duplicated, but aren’t really.
  • Library comes only from the Data volume, and all its contents are on that volume. But inside it, in the path Library/Apple/System/Library are some components that should appear in the main System/Library.
  • System comes only from the SSV, although it has some contents merged into it using firmlinks, such as those folders in Library.
  • Users also comes only from the Data volume, and includes all Home folders for users.

So while the root of Macintosh HD might be in the SSV, much of its contents are on the Data volume, and can be written to, even though the root is a read-only snapshot, thanks to those firmlinks.

Data volume

There are two places that mounted volumes are listed in the Finder: the hidden top-level folder Volumes, where Macintosh HD is just a link back to the root complete with its merged volumes, and in System/Volumes, where what’s shown as Macintosh HD is in fact not the merged volumes, but only the Data volume. You can confirm that by looking at what’s in System/Volumes/Macintosh HD/System, where you only see the parts of the System folder that are stored on the Data volume, and not those stored on the SSV.

What is more confusing there is that System/Volumes/Macintosh HD/Applications is the same merged folder containing both user and bundled apps as in the top-level Applications folder. That’s an artefact resulting from the way that its firmlink works.

But if you open the Get Info dialog on System/Volumes/Macintosh HD, you’ll see the same as with the root Macintosh HD disk, information about the root and not the Data volume.

Mounted in System/Volumes are several other volumes like VM and Preboot, and (depending on whether this is an Intel or Apple silicon Mac) folders such as Recovery and xarts, that you really don’t want to mess with.

Permissions problems

Tackling problems that appear to be the result of incorrect permissions is best done at the lowest folder level. If you’re trying to save a document to the Documents folder inside your Home folder, select that and Get Info on it. Chances are that you are the owner and have both Read & Write permissions as you should. In that case, the problem most likely rests with privacy protection as in Privacy & Security settings. You then suffer Catch-22, as you can only effect changes to those by closing and opening the app, and as you can’t save your document before closing the app, you’re at risk of losing its contents. You may have better luck trying a different folder, creating a new one inside your Home folder, or using the Save As… command instead (which may be revealed by holding the Option key when opening the File menu).

Full layout

In case you’re wondering exactly which folders are merged into the hybrid Macintosh HD ‘volume’, those are shown below in increasing levels of detail, starting with the broad layout.

BootVolGpVentapfs

Then to a simplified version of the full layout.

BigSurIntSimple

Finally, in complete detail.

BigSurIntegrated

Happy navigating!

How to install a Combo update of recent macOS?

By: hoakley
19 August 2024 at 14:30

When a solution works once, we tend to keep using it even though it may no longer be necessary, or may no longer be possible in the same way. This article is about one example, how we used to update macOS using Combo updaters in the days before Big Sur. Now those are no longer available, should we perform a fresh install of macOS whenever there’s an update?

How macOS updates worked

Before we upgraded to Big Sur, macOS updates were relatively simple, and based on installer packages, structured collections of files that are copied to the boot volume in accordance with scripts. Once that copying is complete and the boot volume contains the files forming the new version of macOS, the Mac can be rebooted from those.

Take the last of those major versions of macOS, Catalina. In the diagram below, I show in miniature what happened during the course of its first two minor updates, as an example.

comboupdate1

Updating from Mojave to Catalina 10.15.0 was performed using installer packages that completely replaced everything in its system, then created in a new System volume, as shown at the top. This was referred to as an upgrade, as the whole system was installed afresh.

Then (skipping a Supplemental Update) came the first minor update to 10.15.1 on 29 October 2019. For the sake of this example, consider that includes a new version of the cat command tool. Rather than that update containing the whole system all over again, it only contains and installs what has changed, including the new cat binary.

Then, on 10 December 2019, Apple released the update to 10.15.2, containing another set of changed files, this time perhaps replacing the cp command tool. There are two ways to install that update, either as a delta update or a Combo. The delta update contains the changes from 10.15.1, here cp; the Combo update contains everything that has changed since 10.15.0, both that in 10.15.1, here cat, and in 10.15.2 with cp.

Combo updates tackled what was then a common problem, when a previous update hadn’t been installed correctly. If the update to 10.15.1 didn’t actually install the new cat and we then installed the delta update for 10.15.2, the latter wouldn’t have worked properly. As there was limited error checking in those old updaters, we would then be using a flawed installation of macOS. By installing the 10.15.2 Combo updater instead of the delta, the chance of errors like that were significantly reduced, and we were thoroughly impressed by updating using the Combo rather than delta update.

In fact, it was seldom necessary to install the Combo update every time, as most updates went well and errors weren’t as common as it seemed. What I have always recommended has been to install the delta update first, and only to resort to the Combo version if that didn’t work properly.

Big Sur and the SSV

Catalina’s novel boot volume group, with its separate System and Data volumes, was only an intermediate step to Big Sur, which brought the greatest structural change in Mac operating systems since the introduction of Mac OS X twenty years earlier. Here, the great majority of macOS runs from a mounted snapshot with a read-only file system separate from that of user files on the paired Data volume, and requires a fundamentally different method of installation and updating.

In addition to updating the contents of the system, from Big Sur onwards installers and updaters have considerably more to do. Those tasks include creating the new snapshot to contain the bootable copy of the system, firmlinking that System snapshot with its paired Data volume, building a tree of cryptographic hashes so that the snapshot can be sealed, and verifying the signature of its seal against Apple’s requirement.

comboupdate2

If we follow Big Sur’s first couple of updates, you’ll see how they have changed. On 14 December 2020, the update to 11.1.0 might have gone through similar changes to Catalina, with the replacement of its cat command tool. Once that had been installed on the System volume, a snapshot was made of that System volume, hashes were computed for all its contents, and assembled into a tree. At the top of the tree, a hash of all the hashes in the next layer down forms its seal, which is then hashed again to form the signature of the Signed System Volume (SSV). The signature is compared against that set by Apple for the whole volume, and only if they’re identical is the System snapshot accepted for use.

There’s no room here for the slightest error if the signature on the Signed System Volume (SSV) is going to match its requirement. Within the SSV are additional changes, such as large dyld caches, which serve to make this more complicated, and more recently this includes cryptexes containing system components like Safari that are kept outside the SSV so they can be updated outside full macOS updates.

When that was updated again to 11.2.0 on 1 February 2021, perhaps with a new version of the cp tool, the new installer repeats this process of creating a new snapshot, building its tree of hashes, sealing them, creating the signature, and comparing that with Apple’s signature. This means that each and every update is verified to be perfect, and no errors can be tolerated.

No Combo updates

In the new system, from Big Sur onwards, Apple only provides Combo updates for those updating from versions of macOS older than the previous version, and only when required. If you were to update straight from 11.0 to 11.2, then your Mac downloads all that it requires for that update. If your Mac is already running 11.1.0, there’s no option to install the changes in 11.1.0 again, nor could you need to, as its installation on 11.1.0 has already been verified as being identical to Apple’s reference.

If you felt that wasn’t adequate, then the only option is to install the whole of 11.2.0 from scratch, and running the Big Sur Installer app will do that for you. However, there’s no point in doing that, as the outcome will be identical, as verified by its signature match against Apple’s. All that would do is take longer, and increase wear on your Mac’s internal SSD.

That’s why Apple doesn’t provide the user with Combo updates any more, because they’re now superfluous.

macOS updates are shrinking, but what happened to RSRs?

By: hoakley
14 August 2024 at 14:30

When Apple introduced the Signed System Volume (SSV) in Big Sur, the size of macOS updates rose to their highest ever, as the overhead required for each update reaching 2.2 GB for Intel models and 3.1 GB for Apple silicon. Since then, they have steadily improved in efficiency. This article shows how they have performed in Sonoma, and asks whether Apple has abandoned its Rapid Security Responses (RSRs).

To keep my charts clearer, I here show the two architectures separately.

Total update size

macosupdatesizes6intel

This chart shows cumulative sizes of updates to macOS on Intel Macs from 10.14 Mojave, with its traditional single boot volume, through to macOS Sonoma 14.6. Each point represents the cumulative sum of all updates to that major version of macOS required to reach that minor version. Thus the point for 14.2 is the total of update sizes for 14.1, 14.1.1, 14.1.2 and 14.2. Sizes used aren’t those reported by Software Update, but those of the download itself, as reported in articles here, indexed on this page. That has made a significant difference for some updates for Apple silicon Macs, where the first reported update size has excluded additional architecture-specific overhead. RSRs have been excluded, as they’ve been duplicated in subsequent patch or minor updates. Lines shown are best fits by linear regression.

Update sizes rose markedly from Mojave, with its single boot volume, to Catalina, with its boot volume group, and again to a peak in Big Sur, with the SSV. They fell again as Monterey introduced greater efficiency, and Ventura and Sonoma have been almost identical, and smaller than Mojave.

macosupdatesizes6as

Apple silicon Macs started with the huge updates of Big Sur, which were even larger than those for Intel models, and benefitted from the improved efficiency of Monterey and Ventura. Unlike Intel Macs, though, Sonoma has seen further reduction in update sizes, although in each update they remain significantly larger than those for Intel models.

Big Sur took a total of 36.7 GB on Intel, and a remarkable 50.07 GB on M1; Sonoma has taken 14.1 GB on Intel, and 21.2 on M1. On Intel, Sonoma required 38% of the update size as Big Sur; on M1, that proportion was slightly higher at 42%. Apple silicon updates were 136% the size of those for Intel models in Big Sur; in Sonoma that ratio has risen to 150%.

Minimum update sizes were about 400 MB for Intel, and 820 MB for Apple silicon, which must be close to the overhead required for macOS updates for the two architectures. These reflect the size of the ‘Update Brain’ required to install each update, and all bundled firmware updates. The latter have been steadily reducing as Intel updates have supported fewer models without T2 chips, but have remained the same if not grown for the substantially larger firmware updates for Apple silicon’s Secure Boot.

Unscheduled updates

Sonoma reached version 14.6 without an excessive number of patch updates. The number of patch versions released between the x.0 and x.6 scheduled versions ranges from 4 (Monterey) to 6 (Bug Sur, Ventura), with Sonoma taking just 5, although 14.6 has been closely followed by 14.6.1 in what appears to be a bug fix rather than the first security update.

What has been surprising is that Apple has released no RSRs for Sonoma, not one. The last RSR was the second released for 13.4.1 more than a year ago, on 12 July 2023. RSRs were introduced to Ventura in May 2023 as a method of replacing some components of macOS that aren’t installed in the SSV but in separate cryptexes, cryptographically verified disk images stored on the hidden Preboot volume. They were intended to let Apple promulgate important fixes to vulnerabilities in Safari, WebKit, and some other bundled components without waiting for a full macOS update. Unfortunately, Apple’s second RSR in July last year was fumbled badly when it broke Safari access to many popular websites including Facebook, and had to be replaced three days later.

Although it’s never clear when a patch update could have been issued as an RSR, Sonoma 14.1.2 should have been a candidate as it’s only reported to have fixed two vulnerabilities in WebKit, that appeared to have been addressed in Safari-only updates for macOS 12 and 13. It looks now as if Apple’s initial enthusiasm for RSRs has cooled, and they’re unlikely to be significant in the future.

Standalone updaters

Updates to macOS Catalina and earlier were also available in downloadable standalone installer packages. Apple discontinued those, amid much protest from users, when it introduced Big Sur. Although there were some suggestions that they might return in the future, as the new update mechanism matured, there has still been no sign of them. Currently the only form of updater available is the full macOS Installer app, typically over 13 GB in size. This remains a shortcoming compared with Catalina and earlier, but it appears that standalone updaters aren’t missed by most Mac users, or at least that Feedback to Apple has been insufficient to produce a response.

Major updates

Prior to macOS Ventura, upgrading to the first release of the next major version of macOS required downloading its full installer app. Although larger than necessary to perform the upgrade, it was relatively error-proof, as the user had to intentionally start the app’s installation process, which could also be cancelled once it was in progress.

Ventura was the first major version for which Software Update used a mechanism similar to that for minor updates, and didn’t download a discrete macOS installer app. As Apple hadn’t warned of that, it caught some users by surprise, when they found their Mac was being upgraded almost automatically. This was repeated with the release of Sonoma, and the problem compounded when some users appear to have been upgraded without their consent.

As Tom Bridge has pointed out, this requires smaller downloads that install considerably faster, but, on Intel Macs at least, don’t require user consent or authentication.

Apple is likely to employ the same update mechanism when it releases Sequoia and takes us on to the next cycle.

❌
❌