Normal view

There are new articles available, click to refresh the page.
Before yesterdayMain stream

External boot disks: structure and problems

By: hoakley
31 March 2025 at 14:30

Most modern Macs start up from their internal SSD, and have a single bootable system installed. Although its structure has changed considerably over recent years, and differs between Intel and Apple silicon Macs, they are generally reliable, and knowledge of their structure and function is seldom required.

Those who need to start their Mac up from two or more versions of macOS, or who do so from an external disk, may be able to get by without deeper understanding, but the moment there’s a problem can become confused, and end up having to install macOS from scratch.

Intel T2 and Apple silicon

Both types of Mac are designed to support Secure Boot, but because of the reliance of Intel Macs on UEFI firmware, they can only boot securely from their internal SSD. Enabling an Intel T2 Mac to start up from an external disk thus requires its boot security to be reduced by changing that in Startup Security Utility, in Recovery mode. Booting from an external system is then straightforward, and doesn’t require additional security measures as used in Apple silicon.

Apple silicon Macs have been designed from the outset to support Secure Boot when starting up from both internal and external disks, and don’t require any reduction in their boot security to be able to start up from multiple systems on external SSDs. It’s a common misunderstanding that trying to change Boot Security in Startup Security Utility can help solve Apple silicon boot problems, but if anything it only complicates them. Almost the only good reason for reducing boot security of an Apple silicon bootable system is when third-party kernel extensions are required. Otherwise don’t tamper with Startup Security Utility, as it will only confuse, as we’ll see later.

For an Apple silicon Mac to boot from a macOS system, that must have a LocalPolicy created and saved to the internal SSD. LocalPolicy is normally created during macOS installation, or when intending to start up from a suitably installed system. Problems with LocalPolicy have been common when using external disks, and are covered in these articles:

Boot disk architecture

Another significant difference between Intel and Apple silicon Macs is the architecture of their internal SSDs: Apple silicon has two additional partitions (APFS containers), and lacks the traditional EFI partition. However, there are no such differences in the structure of their bootable external disks, and its relevance here is limited, and mainly affects Big Sur as I explain below.

macOS 10.13-10.15

DiskStructureMojave

Bootable disks in versions of macOS between High Sierra and Mojave (above) are based on their single boot volume, supplemented by hidden Preboot, Recovery and VM volumes. macOS 10.15 Catalina (below) divided that boot volume into a Boot Volume Group, consisting of paired System and Data volumes, in addition to the same three hidden volumes.

DiskStructureCatalina

Basic support for volume roles was introduced in the first release version of APFS in macOS 10.13, and was extended to cover further roles in 10.15. Thus versions of APFS prior to 1412.x.x don’t understand volume roles used by subsequent systems. From Catalina onwards you can use the command
diskutil apfs listVolumeGroups
to see paired System and Data volumes, for example
+-- Container disk5 5BA1AAC8-3AD4-4594-AF01-7C0AA75CABAD
| |
| +-> Volume Group 68627CBB-D774-444E-97C6-F9511B5030F3
| =================================================
| APFS Volume Disk (Role): disk5s1 (Data)
| Name: Macintosh HD - Data
| Volume UUID: 68627CBB-D774-444E-97C6-F9511B5030F3
| Capacity Consumed: 580769214464 B (580.8 GB)
| -------------------------------------------------
| APFS Volume Disk (Role): disk5s5 (System)
| Name: Macintosh HD
| Volume UUID: 8B9FF440-75C8-4F7F-B09E-9222D44A2276
| Capacity Consumed: 11239186432 B (11.2 GB)

Note that each volume group has its own UUID, which is normally the same as that of the Data volume in the pair. Identification of volumes, containers and other structures in APFS is dependent on these UUIDs, which are an essential part of the GUID Partition Table scheme used to partition disks.

Catalina’s System volume is mounted read-only, but macOS is booted from that volume rather than the Signed System Volume snapshot introduced in Big Sur.

macOS 11

BootDiskStructureIntelBigSur

Although the structure of Big Sur internal SSDs in Apple silicon Macs has two additional partitions (APFS containers), Apple_APFS_ISC containing three volumes, and Apple_APFS_Recovery containing two volumes, bootable external disks have the same structure as the internal SSD in Intel Macs.

One important functional difference, which remains relevant to Big Sur boot disks, is that Apple silicon Macs don’t use the paired Recovery volume as their primary Recovery system: booting an Apple silicon Mac running Big Sur into Recovery should instead use the Recovery system installed in their internal SSD, in the Apple_APFS_Recovery partition. In subsequent versions of macOS, that’s used instead for secondary or Fallback Recovery. Thus Big Sur can be a problem when it comes to Recovery, and for this reason is best avoided on Apple silicon Macs. If it’s essential to install a copy of Big Sur, then be prepared for problems with Recovery mode.

macOS 11 also established the architecture for dual-boot partitions, with two or more Boot Volume Groups.

BootDualDiskStructureIntelBigSur

Although the Boot Volume Group has only ever referred to paired System and Data volumes, within each partition/container the group also requires three or four additional volumes, for Preboot, Recovery, VM and (apparently in Big Sur only) Update. How those work with multiple Boot Volume Groups is explained below. Once way to avoid the inevitable complexities is to install each Boot Volume Group into a separate partition/container, which provides each with its own suite of Preboot, Recovery and VM volumes.

As the Signed System Volume (SSV) was introduced in macOS 11, versions of APFS prior to 1677.x.x shouldn’t be expected to understand SSVs.

macOS 12-15

The next significant architectural changes in Apple silicon Macs were the introduction of paired Recovery volumes in macOS 12 Monterey, and of cryptexes in macOS 13 Ventura.

In macOS 11 and 12, Safari, its supporting components, and some other parts of macOS are installed to the Data volume for ease of maintenance. Apple replaced that with separate secure disk images termed cryptexes, loaded from the Preboot volume during the boot process and grafted into the root file system. As a single Preboot volume can be shared by more than one Boot Volume Group, bundled cryptexes must be provided to the correct group, and I suspect that’s accomplished by associating them with their Volume Group UUID. If that becomes confused in any way, cryptexes could be incorrectly associated or missing altogether, something that’s almost certainly fatal to the boot process.

Above is the current layout of a partition/container containing a single bootable system for a Mac running Sequoia. Device names given are illustrative, although their numbering is typical of that seen for external bootable volumes. In this instance, the base name for the Boot Volume Group is External, and the External System volume is unmounted as usual.

Below is a typical partition/container containing two bootable systems named ExternalA and ExternalB. This has two Boot Volume Groups, each consisting of a System volume with its SSV, and a Data volume, to which are added their three common volumes, Preboot, Recovery and VM.

Management

Creation

The most reliable tools for creating and working with Boot Volume Groups and other system components are those in macOS installer apps, and even they can have their moments and bugs. It’s usually possible to ‘clone’ groups using the asr command tool, a feature that’s offered in some third-party utilities including Carbon Copy Cloner and SuperDuper. However, Apple has made it clear that given a choice between supporting asr and addressing boot security, it gives the latter absolute priority. asr has suffered some serious bugs since the SSV was introduced in Big Sur, and shouldn’t be relied on.

asr is careful to ensure that, when cloning Boot Volume Groups, UUIDs are changed so that it doesn’t end up with volumes or groups with the same UUID. That may not be the case when using some other tools such as dd for duplication. If you prefer a simple and stress-free life, it’s better to put your trust in a macOS installer rather than try crafting your own Boot Volume Groups.

Apple’s own tools such as Disk Utility now try to steer the user away from making mistakes, such as deleting just one of a Boot Volume Group, leaving the other orphaned with its firmlinks broken.

Firmware

Firmware is another source of confusion. That installed in the Mac, using its internal storage, should always reflect the firmware that has been included in the most recent version of macOS installed or updated on that Mac, whether that was installed to the internal SSD or to an external disk. The only exception to this is when installing or updating macOS in a Virtual Machine, which can’t affect the host’s firmware.

What may appear more puzzling are the versions reported in System Information: that given as System Firmware should be the iBoot version for the main Mac, and the most recent. The OS Loader, though, varies with the Boot Volume Group, and in older macOS is likely to be earlier than the iBoot version of the main Mac.

Recovery

Recovery system versions are even more complex. When everything works as it should, the version installed in any paired Recovery volume should be the newest of the Boot Volume Groups that it’s paired with, in the same partition/container. If a Mac running 15.3 from its internal SSD has a bootable external disk with a single container with two Boot Volume Groups for 13.7 and 14.7, then the Recovery system for the internal SSD should be 15.3, while that shared by the two systems on the external disk should be 14.7. It’s likely that Fallback Recovery on the internal SSD will be from an earlier version of macOS 15, but that’s less predictable, and there’s no separate Fallback Recovery in external systems.

Switching between systems and their Boot Volume Groups is normally performed using Startup Disk in General settings, or its equivalent in System Preferences. Those record the chosen Boot Volume Group in NVRAM, from where it’s used during the next boot.

That setting is used to determine which Recovery system is used if the next boot is performed with the Power button pressed to enter Recovery mode. That in turn determines what can be performed in Recovery. This is most critical for determining how Startup Security Utility works, as that can only change boot security settings for the chosen Boot Volume Group saved in NVRAM.

For example, in the dual-boot container shown above, selecting ExternalB as the Startup Disk, shutting down, then starting up into Recovery mode should enable Startup Security Utility to control boot security for ExternalB but not ExternalA, nor the system installed on the internal SSD. This can be used to discover which Recovery system is running: open Startup Security Utility and the only boot system that can be changed there is the current default boot system.

APFS

Although APFS should be backward compatible, making it relatively safe to make changes to an older version of APFS from a newer system, forward compatibility is more limited. Using older versions of Disk Utility or tools like fsck on newer versions of APFS risks errors, failure and at worst damage. The Appendix at the end of this article summarises version numbering in APFS and major changes to beware of. Although not easy to discover the version of APFS used to create or modify any given volume, one way is to use
fsck_apfs -n -S [device]
giving the volume’s device name. The report should then be prefaced by a statement such as
The volume [volume name] was formatted by diskmanagementd (1412.141.1) and last modified by apfs_kext (945.250.134).
telling you that volume was created by macOS 10.15, and was last changed by macOS 10.14.

Troubleshooting

The first and fundamental step in trying to diagnose the cause of problems with multiple Boot Volume Groups or bootable external disks is to examine their container and volume structure using two diskutil commands:
diskutil apfs list
lists all APFS volumes by container and gives key information about each, including role and UUID, and
diskutil apfs listVolumeGroups
lists all recognised Boot Volume Groups, which you can tally against the first. Pipe these to text files so you can study and refer back to them.

Those should enable you to verify that the structure is as intended, and to establish the relationships between systems and paired Recovery volumes. From there you should be able to focus on the Boot Volume Group that’s misbehaving, ensuring that its Data volume has been backed up. If necessary, that can be used as a migration source if that group needs to be deleted and reinstalled.

Tips & tricks

  • The host Mac must be capable of running that version of macOS.
  • macOS installers are the most reliable means of creating and installing Boot Volume Groups.
  • During macOS installation, an external disk must be connected to a port other than the DFU port, as listed here and in Mactracker.
  • When installing an older major version of macOS, perform this from an external bootable HFS+ volume as detailed by Apple.
  • Use an HFS+J partition on an external SSD rather than a USB ‘thumb’ drive.
  • Boot from an installer volume through Recovery mode.
  • When using a laptop Mac, run it from mains power throughout macOS installation.
  • Apple silicon Macs boot from external disks in Full Security, and reducing that doesn’t solve any problems.
  • Each container with one or more Boot Volume Groups contains one set of Preboot, Recovery and VM volumes shared between them.
  • If you’re unsure which Recovery system you’re using, open Startup Security Utility, as the only group it can change settings for is the one it’s booted into.
  • Big Sur doesn’t support paired Recovery on Apple silicon, and that can cause problems.
  • The version of Recovery installed in any paired Recovery volume should be that of the newest of the Boot Volume Groups that it’s paired with.
  • Use fsck_apfs -n -S [device] to discover the APFS version.
  • Use diskutil apfs list and diskutil apfs listVolumeGroups to discover volume structure.
  • Try using a Virtual Machine instead, if you don’t need to be able to run software from the App Store.

Appendix: APFS version numbers

APFS major version numbers change with major version of macOS:

  • macOS 10.12 has APFS version 0.3 or 249.x.x, which shouldn’t be used at all.
  • 10.13 has 748.x.x, which doesn’t support Fusion Drives, but has basic support for volume roles.
  • 10.14 has 945.x.x, the first version to support Fusion Drives.
  • 10.15 has 1412.x.x, the first version to support the multi-volume boot group, and introduces extended support for volume roles, including Data, Backup and Prelogin.
  • 11 has 1677.x.x, the first version to support the SSV, and Apple silicon. On M1 Macs, it doesn’t support the paired Recovery volume.
  • 12 has 1933.x.x until 12.2, thereafter 1934.x.x, which support the paired Recovery volume on Apple silicon.
  • 13 has 2142.x.x, and is probably the first to support trimming of UDRW disk images and their storage as sparse files.
  • 14 has 2235.x.x, until 14.3, thereafter 2236.x.x.
  • 15 has 2313.x.x until 15.2, thereafter 2317.x.x until 15.4, thereafter 2332.x.x.

How external bootable disks work with Apple silicon Macs

By: hoakley
21 February 2025 at 15:30

Unlike Intel Macs (including those with T2 chips), all Apple silicon Macs always start their boot process from their internal SSD, even when they are set to start up from a bootable external disk. This ensures the security and integrity of that process and prevents an attacker from starting that Mac up without credentials. This article explains how this affects the use of Apple silicon Macs.

Bootable external disks

In addition to normal requirements for a macOS installation on an external disk to be able to boot a Mac, ownership of the boot volume group on that disk is required. This is normally performed when installing macOS on that disk, as explained here, and results in the ownership of that boot volume group by an authorised user of that Mac. This is incorporated into a LocalPolicy that is saved to the internal SSD of that Mac.

If that ownership and LocalPolicy haven’t been created at the time of installation, macOS should prompt for their creation when you select that external disk as a startup disk, in System Settings or Recovery mode. That enables an existing external disk to be made bootable for another Mac. If that fails, then macOS will refuse to start up from that external disk.

Disk structure

To accommodate the more advanced Secure Boot of Apple silicon Macs, their internal SSDs are divided into three partitions, with an extra six volumes beyond the boot volume group.

BootDiskStructureMSeq

The boot volume group itself is composed of:

  • The System volume, used to build the SSV, which is unmounted during normal running.
  • The Signed System Volume, SSV, a signed and sealed snapshot of the System volume, containing the immutable System.
  • The Data volume, by default named Data, mounted within the System directory tree, and writable by the user.
  • The small Preboot volume, used to store cryptexes.
  • The paired Recovery volume, containing a disk image of the Recovery system.
  • The VM volume as the backing store for virtual memory.
  • Possibly an Update volume that appears to have been used when upgrading to versions of Big Sur. This won’t be present on more recent Macs, and appears to have been disused since macOS 11.

The SSV is created from the System volume during macOS installation and update, and features a tree of cryptographic hashes verifying its integrity, and its signature to verify that its contents match those expected for that version of macOS.

Boot process

On Apple silicon Macs, this starts with the Boot ROM to validate the Low Level Bootloader (LLB), stage 1 of the boot process. The LLB in turn validates other firmware to be used in Stage 2, the LocalPolicy to be applied to the startup disk, and iBoot (Stage 2) itself, in accordance with the requirements of the applicable LocalPolicy. When starting up from a boot volume on an external disk, control is then transferred to that bootable system, which is stored in a single APFS container with the same layout as that for an Intel Mac.

Thus, when starting up from an external disk, M-series Macs require LLB and iBoot together with LocalPolicy all stored on the internal SSD. As LLB and iBoot are part of a macOS install, the best way to provide those is in a full installation of the current release of macOS.

LocalPolicy

The user controls LocalPolicy through Startup Security Utility, which is only accessible in Recovery Mode, and requires user authentication. There’s no LocalPolicy that applies to all users and all disks, though: each LocalPolicy is specific to a boot volume group and authorised user. For example, these can allow:

  • a single bootable external disk to be used to start up two or more Macs;
  • one Mac to be started up from any of several boot volume groups, which can be running older versions of macOS, or could load third-party kernel extensions.

Default LocalPolicy created for each bootable external disk provides Full Security, which among other things blocks the loading of third-party kernel extensions and requires SIP to be fully enabled. iBoot (Stage 2) validates kernel collections, signed System volumes, and other components to ensure their integrity, and that the kernel, extensions and macOS to be loaded have an acceptable version number.

LocalPolicy can be changed, in particular by changing Secure Boot settings using Startup Security Utility, first booting from that boot volume group, shutting down, and starting up in Recovery mode. That sequence ensures that the Mac boots into the Recovery volume paired with the System volume whose security settings are to be changed.

Fallback

The current startup disk setting is stored in NVRAM, which on Apple silicon Macs can’t readily be reset by the user. In the event that disk can’t be used to boot that Mac, it will normally fall back to starting up from the internal SSD. This is a more convoluted process in the event that the expected startup disk can’t be found at all: in that case, there may be a long delay in startup, during which the power light will remain on but the display remains black. Eventually, when all hope of finding the missing boot disk is abandoned, the Mac may boot from the hidden Recovery container on the internal SSD, normally used as Fallback Recovery, enabling the user to choose another startup disk and restart from that.

External boot volume groups

The boot volume group on external storage has the same structure and features as that in the Apple_APFS container of the internal SSD, and once it has started up is self-sufficient. macOS is contained in its SSV, with cryptexes saved to its Preboot volume. Writeable system data, and all user data, are saved to its Data volume, and it has its own paired Recovery volume, used when the external system is booted into Recovery mode.

Its macOS is updated independently of that installed on the internal SSD, although any firmware updates to be installed as part of a macOS update change the firmware on the internal SSD, as there’s no firmware on the external SSD. In the event that internal and external macOS versions differ, the installer or updater will only ever update the firmware to the more recent version.

DFU mode

Apple silicon Macs are the first Apple computers whose firmware can be both upgraded and downgraded by installation of an IPSW image file when the Mac is in DFU mode. This enables you to erase the whole of the internal SSD and install all three containers, with firmware, the SSV and Data volumes to factory-fresh condition, for any supported version of macOS. This is invaluable when there’s a problem with firmware, corruption of the internal SSD without physical damage, or simply to revert to an older macOS.

Because restoring in DFU mode erases the whole of the internal SSD, it also blows away all saved LocalPolicy for that Mac. Following the restore process, any bootable external disk used with that Mac will need to have its ownership re-established so that a new LocalPolicy can be created for it.

Key points

  • To be bootable from an Apple silicon Mac, an external disk needs ownership of its boot volume group, and LocalPolicy for it.
  • Ownership is normally assigned and LocalPolicy created during macOS installation.
  • Ownership and LocalPolicy can also be created when selecting an external disk as the startup disk. That allows an external disk to be bootable with more than one Apple silicon Mac.
  • A complete macOS boot volume group is still required on the internal SSD to be able to boot from external disks, as the early stages of the boot process, including validation of its LocalPolicy, can only be run from the internal SSD.
  • LocalPolicy can be changed using Startup Security Utility in the paired Recovery volume.
  • Fallback policy ensures that a Mac can still start up if the expected external boot disk fails to boot, or is missing.
  • External boot volume groups are self-sufficient and don’t rely on the internal SSD once they have booted successfully.
  • All stored LocalPolicy is destroyed when a Mac is restored in DFU mode, and ownership and LocalPolicy then have to be recreated for any external bootable disks.

Getting more from Recovery on Apple silicon Macs

By: hoakley
18 February 2025 at 15:30

Whether you’ve become familiar with the basics in Recovery mode on Apple silicon Macs, or you just need deeper information, this article is intended to supplement my illustrated guide.

Which Recovery mode?

Unless your Mac has two or more systems installed, there are only two Recovery modes:

  • Paired Recovery, the default, run from a mounted disk image stored in the Recovery volume in the active boot volume group;
  • Fallback Recovery, engaged using a short press, then a held press of the Power button, in a ‘di-dah’ rhythm. This isn’t always available, but when it is it runs from a dedicated partition/container on the internal SSD. Although it can do everything else, you can’t use its Startup Security Utility, which is only available in paired Recovery.

This is a bit more complicated when your Mac has more than one bootable system available. The rule then is that it will enter paired Recovery for the current boot volume group. If you have Sonoma and Sequoia installed and want to do anything with the Sonoma boot volume group, then you should make Sonoma the active Startup Disk, and ideally restart into that, before shutting down to start up in Recovery mode. Although the initial Options screen allows you to choose boot volume groups, that procedure ensures that you’ll always use the correct tools for the job.

Change keyboard

Throughout Recovery mode, you can change the default keyboard, which is set to that in NVRAM. At the top right, the keyboard menu offers those standard in macOS. If you’re experiencing problems entering your password, check that you’re using the right keyboard here.

Installer Log

This is useful for the checks that it reports, including statements that the network is reachable, and details of the current user, whether they have admin privileges, and have a secure token to be able to unlock FileVault. If you need to get into the weeds, its popup menu can be set to Show All Logs, and you might even be able to save the log to somewhere that’s accessible when back in normal user mode.

Utilities: Share Disk…

This menu command is used to put that Mac into the modern equivalent of Target Disk mode. That requires it to be connected to another Mac using a Thunderbolt or USB data cable, and will then mount its Data volume on that Mac, connected using SMB over the cable. Although not as fast as a direct Thunderbolt connection, it’s valuable if you need to retrieve files from a Mac that can’t boot in normal user mode.

Utilities: Terminal

This opens a bash shell rather than the zsh we’re now used to, as root user. There are a great many useful tools included in the Recovery System, including: fsck, fsck_apfs, mount, mount_apfs, csrutil, asr, nvram, spctl, sysctl, and /usr/bin and /usr/sbin contain many more. The biggest snag with them is that, in 15.3.1 at least, there’s no man command, although many will show their standard usage information with an -h option.

One command tool peculiar to Recovery mode is its equivalent to sysdiagnose, recoverydiagnose, in case you need to file a bug report about Recovery mode.

There’s also repairHomePermissions, which launches a GUI app to repair permissions on a selected Home folder on the Data volume. I strongly recommend that you don’t try using this, as you’ll end up with the user being locked out of every folder in their Home folder. This appears to be a historical remnant that has somehow been left behind, long after it outlived its usefulness.

Utilities: Startup Security Utility

bootsec6

I have already explained at length how to use this to change boot security, in conjunction with adjusting SIP, and more, in this article.

Main Window: Disk Utility

Although Disk Utility invariably opens with its default settings, the version supplied in Recovery is every bit as capable as that in normal user mode. Once it has opened, set its View tool to Show All Devices, and if you want to work with snapshots, set its View menu to Show APFS Snapshots.

Main Window: Safari

As with Disk Utility, the version of Safari provided is almost as fully equipped as normal, and apparently runs from a cryptex. It opens with a local page that explains features in Recovery mode, but you can access Google search engine and pretty well anything else you want, including articles in this blog.

❌
❌