Normal view

There are new articles available, click to refresh the page.
Before yesterdayMain stream

Where has Safari gone, and why are macOS updates larger for Apple silicon?

By: hoakley
4 September 2024 at 14:30

My previous explanation of how recent versions of macOS merge their System and Data volumes into what appears to be a single volume, omitted a third component, including Safari. Look in the System/Applications folder where all the bundled apps are stored on the SSV, and there’s no Safari to be seen, yet it appears in the top-level Applications folder. This article explains how that now works using cryptexes, and how they differ between Intel and Apple silicon Macs.

Finding Safari

As the modern boot volume group evolved through Catalina to Big Sur, Safari and its supporting frameworks were stored in the Data volume. That stopped with the arrival of Ventura, and they’re now stored in the third components that complete the modern boot volume group. You can see when files are stored on a different volume using my free app Precize to reveal their full paths. Use that to examine three apps from the merged Applications folder, and you’ll understand what I mean:

  • Chess.app has a path of /System/Applications/Chess.app demonstrating that it’s one of the apps bundled in the SSV, where almost all of the System folder is stored.
  • Cirrus.app, like any other app you have installed, has a path of /Applications/Cirrus.app, making it clear that it’s stored on the writable Data volume.
  • Safari.app has the weird path of /System/Volumes/Preboot/Cryptexes/App/System/Applications/Safari.app that demands further explanation.

Note that the Finder’s Get Info dialogs aren’t as truthful, and don’t tell the full story.

Their volfs paths are also worth noting. On my Intel Mac, they are:

  • Chess.app is at /.vol/16777240/1152921500311883667; because all macOS 14.6.1 SSVs are identical, your Chess.app should have the same inode number too.
  • Cirrus.app is at /.vol/16777240/461665725
  • Safari.app is at /.vol/16777238/993517

The first two follow a familiar pattern you’ll see throughout the System and Data volumes: their volume ID 16777240 is common to both, and that assigned to the merged volumes, but their inode numbers are wildly different. Huge numbers like 1152921500311883667 come from the SSV, while smaller ones like 461665725 are from the Data volume. Then there’s a slightly lower volume ID of 16777238 and a small inode number of 993517 for Safari, demonstrating that it’s somewhere altogether different: that’s a cryptex, a cryptographically protected disk image with an interesting history.

Why a cryptex?

When the modern boot volume group was being designed and developed, it took into account Safari’s special needs by making it the only bundled app to be stored in the Data volume. This enables it to be updated without having to go through the whole process of building a new SSV, allowing Apple to deliver urgent security patches to Safari and its underlying WebKit and other frameworks. There could also have been political considerations in separating Apple’s bundled browser from the other apps included in macOS.

This changed in Ventura in the autumn/fall of 2022, when Apple applied technology it had originally developed for its customised iPhone, the Security Research Device, dubbed the cryptex, a name formed as a portmanteau for CRYPTographically sealed EXtension. This offers two advantages:

  • Safari, its supporting frameworks, and other components of macOS that Apple prefers not to build into the SSV, can be delivered in cryptexes. As I’ll explain later, this also enables tailoring of macOS to platform.
  • Some urgent security patches could be delivered in cryptexes, making them faster to release and simpler to install in a Rapid Security Response (RSR).

Since then, RSRs seem to have had their day, and appear to have fallen from favour. But, as a means of delivering Safari and other more changeable components of macOS, cryptexes have proved their worth.

How a cryptex works

Although a cryptex is at heart a read-only disk image that is mounted during the boot process, it has two properties of particular importance:

  • Its contents are cryptographically verified, in much the same way that the contents of the SSV are, using hashes of its entire contents.
  • Its internal file system is grafted into the root file system when it’s mounted, rather than being mounted as a separate volume.

APFSCryptexMount1

Mounting a cryptex starts with validation of the payload and its manifest. It then undergoes a sequence of processes similar to the mounting of an APFS volume, with a checkpoint search to establish stable checkpoint indices, and a check to discover whether there’s anything to recover, which seems unlikely. The graft is then performed in a series of opaque steps, with root hash authentication and validation. The object ID is found, and the graft completed.

Once this has been completed for each of the standard cryptexes and any installed RSRs, the contents of those are effectively part of the system, as a hybrid of the SSV and cryptexes. In the case of the Safari app, this process effectively places it in the main Applications folder, even though the original app is actually located in the System/Applications folder of the App cryptex in /System/Volumes/Preboot/Cryptexes.

As with the current boot System and Data volumes, grafted cryptexes aren’t unmounted or ungrafted until shutdown.

There are currently three main cryptexes in use, App containing Safari, its frameworks and other supporting files, and OS, with a range of other system items including additional frameworks, and several large dyld shared caches. You’ll also see an Incoming cryptex in /System/Volumes/Preboot/Cryptexes. As they’re outside the SSV, new and replacement cryptexes are installed without rebuilding the SSV, and in some cases don’t even need a soft restart of macOS.

Architecture-specific cryptexes

In addition to providing Safari and its related components, cryptexes also provide useful economy in shared caches, and explain why macOS updates for Apple silicon Macs are invariably larger than those for Intel models.

While the contents of the SSV appear to be identical on both Intel and Apple silicon, thus have a single signature, the two architectures differ in their cryptexes. Those for Apple silicon Macs contain dyld shared caches for both architectures, and a set of aot shared caches, presumably to support Rosetta 2, and amounting to 5.24 GB in total size; those for Intel Macs only contain Intel dyld shared caches of 1.68 GB total size.

Given their sizes, that’s a valuable efficiency both for updates and in storage required, and is the major reason for updates for Apple silicon Macs always being larger than those for Intel. Thankfully, because those shared caches are supplied compressed, the difference in update sizes is much smaller than the 3.56 GB difference when they’re decompressed and installed.

Copy speeds of large and sparse files

By: hoakley
7 August 2024 at 14:30

I have recently seen reports of very low speeds when copying large files such as virtual machines, in some cases extending to more than a day, even when they should have been sparse files, so requiring less time than would be expected for their full size. This article teases out some tests and checks that you can use to investigate such unexpectedly poor performance.

Expected performance

Time taken to copy or duplicate files varies greatly in APFS. Copies and duplicates made within the same volume should, when performed correctly, be cloned, so should happen in the twinkling of an eye, and without any penalties for size. This is regardless of whether the original is a sparse file, or a reasonably sized bundle or folder, whose contents should normally be cloned too. If cloning doesn’t occur, then the method used to copy or duplicate should be suspected. Apple explains how this is accomplished using the Foundation API of FileManager, using a copyItem() method. This is also expected behaviour for the Finder’s Duplicate command.

Copying a file to a different volume, whether it’s in the same container, or even on a different disk, should proceed as expected, according to the full size of the file, unless the original is a sparse file and both source and destination use the APFS file system. When an appropriate method is used to perform the copy between APFS volumes, sparse file format should be preserved. This results in distinctive behaviour in the Finder: at first, its progress dialog reflects the full (non-sparse) size of the file to be copied, and the bar proceeds at the speed expected for that size. When the bar reaches a point equivalent to the actual (sparse) size of the file being copied, it suddenly shoots to 100% completion.

Copying a sparse file to a file system other than APFS will always result in it expanding to its full, non-sparse size, and the whole of that size will then be transferred during copying. There is no option to explode to full size on the destination, nor to convert format on the fly.

External SSDs

When copying very large files, external disk performance can depart substantially from that measured using relatively small transfer sizes. While some SSDs will achieve close to their benchmark write speed, others will slow greatly. Factors that can determine that include:

  • a full SLC cache,
  • failure to Trim,
  • small write caches/buffers in the SSD,
  • thermal throttling.

Many SSDs are designed to use fast single-level cell (SLC) write caching to deliver impressive benchmarks and perform well in everyday use. When very large files are written to them, they can exceed the capacity of the SLC cache, and write speed then collapses to less than a quarter of that seen in their benchmark performance. The only solution is to use a different SSD with a larger SLC cache.

Trimming is also an insidious problem, as macOS by default will only Trim HFS+ and APFS volumes when they’re mounted if the disk they’re stored on has an NVMe interface, and won’t Trim volumes on SSDs with a SATA interface. The trimforce command may be able to force Trimming on SATA disks, although that isn’t clear, and its man page is forbidding.

Trimming ensures that storage blocks no longer required by the file system are reported to the SSD firmware so they can be marked as unused, erased and returned for use. If Trimming isn’t performed by APFS at the time of mounting, those storage blocks are normally reclaimed during housekeeping performed by the SSD firmware, but that may be delayed or unreliable. If those blocks aren’t released, write speed will fall noticeably, and in the worst case blocks will need to be erased during writing.

For best performance, SATA SSDs should be avoided, and NVMe used instead. NVMe is standard for USB 3.2 Gen 2 10 Gb/s, USB4 and Thunderbolt SSDs, which should all Trim correctly by default.

Disk images

Since Monterey, disk images with internal APFS (or HFS+) file systems have benefitted from an ingenious combination of Trimming and sparse file format when stored on APFS volumes. This can result in great savings in disk space used by disk images, provided that they’re handled as sparse files throughout.

When a standard read-write (UDIF) disk image is first created, it occupies its full size in storage. When that disk image is mounted, APFS performs its usual Trim, which in the case of a disk image gathers all free space into contiguous storage. The disk image is then written out to storage in sparse file format, which normally requires far less than its full non-sparse size.

This behaviour can save GB of disk space in virtual machines, but like other sparse files, is dependent on the file remaining on APFS file systems, otherwise it will explode to its full non-sparse size. Any app that attempts to copy the disk image will also need to use the correct calls to preserve its format and avoid explosion.

Tools

clonesparse3

Precize reports whether a file is a clone, is sparse, and provides other useful information including full sizes and inode numbers.

Sparsity can create test sparse files of any size, and can scan for them in folders.
Mints can inspect APFS log entries to verify Trimming on mounting, as detailed here.
Stibium is my own storage performance benchmark tool that is far more flexible than others. Performing its Gold Standard test is detailed here and in its Help book.

❌
❌