A basic Mac system consists of the Mac itself and external storage for its backups, and is by far the most popular configuration. For many folk backing up the whole of its Data volume is wise, but that isn’t always the most economical. If the Data volume contains large items that don’t need to be backed up as often as its working folders, that can waste space. This article shows how you can make it more efficient without additional cost or hardware.

Backups and local snapshots

Most good backup utilities including Time Machine also make local snapshots of the volumes they back up. Let’s say your Data volume contains 100 GB of files that either change little or don’t need to be backed up as frequently as the rest. One proven strategy for minimising the time and storage required for backups is to add those to the exclusion list, and back them up separately, maybe only once a week. You can do that to another volume on external storage, provided you ensure there’s sufficient space for both that and your normal automatic backups.

What that doesn’t do is keep those 100 GB out of the frequent snapshots made of the Data volume. While you can exclude files and folders from backups, snapshots always include everything in that volume, without exclusions. The only way to save the space they add to snapshot size is to move them to another volume that doesn’t get snapshots made of it. But your Mac’s standard disk layout doesn’t provide any spare volume for that.

This could apply to all sorts of relatively static data that doesn’t need Time Machine’s automatic hourly backups, including Virtual Machines and some large media libraries, although you won’t then be able to share these in iCloud Drive, which would require them to be in your Data volume.

Boot disk layout

Standard layout of the internal SSD of an Apple silicon Mac running Sequoia or earlier is shown below.

Intel Macs have the same Apple APFS container with the Boot Volume Group in it, but the other two containers are replaced by a single small EFI partition.

Adding another partition or container is possible, but not recommended as it has a fixed size, and lacks the flexibility of a volume. It also risks disturbing the three existing partitions/containers. As they’re essential for the Mac to start up successfully, you don’t want to meddle with them.

In practice, the best place to add a new volume is inside the third container, the one already holding the System and Data volumes. Add that in Disk Utility once you’ve decided the next two steps.

Limit volume size

Your new volume is going to share space in its container with all the existing volumes, including both System and Data. It’s usually wise to impose a maximum limit on the size it can grow to, to avoid compromising any of those. When you add the new volume, put a sensible limit on its Quota Size.

Encryption

Although Apple’s documentation isn’t explicit, volumes added to the boot container aren’t protected by FileVault, unlike the Data volume. If you want your extra volume to be encrypted, you’ll have to format it in APFS (Encrypted). Whether that’s accelerated by the hardware in the Secure Enclave isn’t clear, and on Apple silicon Macs it’s hard to tell the difference, as you should get similar full speed performance from your extra volume to that of the Data volume.

Setting it up

Open Disk Utility, ensure its View options are set to Show All Devices, then select the Container holding the boot volumes. Click the + tool to add the new volume.

Give the volume a name, then click on the Size Options… button.

Enter your chosen Quota Size, as the maximum you want to allow the extra volume to use on the boot SSD, and click OK.

Then select whether you want it formatted in plain APFS, or encrypted, and click the Add button.

If you’ve opted for APFS (Encrypted) you’ll then be prompted to enter the encryption password. Unlike FileVault, there’s no option for a Recovery Key, or for iCloud Recovery.

When you first unlock the extra volume, you’ll be given the option to save its password to your keychain. That confirms this isn’t being performed by FileVault, as that protects its encryption keys in the Secure Enclave.

There are a couple of quirks:

• If you try unmounting the extra volume using the Finder’s contextual menu, macOS might try to unmount all volumes on the boot disk, and warn you that it can’t. Simply cancel those warnings, and the extra volume should unmount fine. If you’re worried by this, unmount the volume in Disk Utility, which isn’t as silly.
• You can use the Finder contextual menu to encrypt or decrypt the volume if you change your mind.

Summary

• To save space in local snapshots made for backups of your Data volume, move bulky items that you back up separately to an extra volume alongside the Data volume.
• Set a Quota Size on the extra volume to limit the maximum space it can take.
• Use plain APFS or APFS (Encrypted) as the extra volume can’t be protected by FileVault.
• If you encrypt the volume, safeguard its password as there’s no recovery option if you lose it.
• The extra volume performs as well as any other volume on the internal SSD, and is far faster than using external storage.

Disk images, files that contain the contents of a physical storage medium, go back long before the first Mac. Among other tasks, they were originally used to contain representations of floppy disks for replication in manufacture.

Today disk images are at the heart of macOS, and widely used by third-parties. They’re an essential part of macOS installers, home to Recovery mode, and the basis for cryptexes. They’ve been used to burn and replicate optical disks, to archive disk contents, extensively for network backups, and for the distribution of software.

Classic Mac OS

In Classic Mac OS there were two utilities that worked with different formats: Disk Copy used replicas later in DC42 format, after Disk Copy version 4.2, while compressed formats known as DART were handled by the Disk Archive/Retrieval Tool, hence their name.

Mac OS 9 brought Disk Copy 6.0 with added support for the New Disk Image Format (NDIF), which supported resource forks, and ended with its last release version 6.3.3. This also supported read-only Rdxx formats.

By this time, variants of formats had become complex. Here, Disk Copy is configured to create a read-only compressed .img file containing the contents of a standard 1.4 MB floppy disk. In the upper window, it has completed validating the checksum on a self-mounting .smi disk image that’s part of a DiskSet. These could also be signed, using certificates issued not by Apple but by DigiSign.

Here’s Disk Copy saving an image of a hard disk using a similar read-only compressed format, this time to accommodate 1.5 GB.

Mac OS X

The release of Mac OS X 10.1 Puma in 2001 brought Apple’s new Universal Disk Image Format (UDIF), used in DMG disk images, which only had a single fork as its resource fork was embedded in the data fork. Although pre-release versions of Disk Copy 6.4 and 6.5 were available with UDIF support for Mac OS 9, neither was ever released, leaving Classic Mac OS without access to UDIF images. Its support for compression options in Apple Data Compression (ADC) unified the two disk image types, and extended support for images larger than a floppy disk. This new format enabled disk images to represent whole storage devices, complete with a partition map and disk-based drivers.

Tools provided in Mac OS X for working with disk images include Disk Utility and the command tool hdiutil.

On 21 January 2002, the first version of DropDMG, a third-party substitute for creating disk images, was released by C-Command Software. This quickly enabled developers to create disk images with artwork, licences and other features that weren’t accessible from the tools bundled in Mac OS X. DropDMG has flourished over the last 23 years, and remains popular today.

DropDMG’s options for creating a new disk image far exceed those in Disk Utility. Particularly helpful are the compatible version hints shown on various options, to remind you of which file systems are available in different macOS versions, and which types of disk image container are supported. DropDMG will even convert old NDIF disk images last used in Mac OS 9 to more modern formats. It will also change the password of an encrypted disk image from a menu command.

In Mac OS X 10.2 (2002), UDIF and most other supported formats were served from a kernel extension without requiring a helper process. The following year, 10.3 Panther started using a faceless utility DiskImageMounter to mount disk images. Apple then dropped support for embedded resource forks in disk images in Mac OS X 10.4.7, and newly created disk images became less compatible with older Mac OS versions.

Sparse bundles

Until Mac OS X 10.5 Leopard in 2007, all disk images had used single-file formats, although some could be segmented across file sets. Leopard introduced the sparse bundle with its folder of smaller band files containing data. These enabled the image to grow and shrink in size, and became popular means of storing mountable Mac file systems on servers using different file systems.

This is another third-party tool that improved access to disk images from the GUI, DMG Packager, seen in 2009. Unlike DropDMG, this appears to have vanished without trace.

In 2011, with the release of Mac OS X 10.7 Lion, Apple removed more support for old disk image formats. DiskImageMounter no longer opened NDIF .img, .smi self-mounting, .dc42 and .dart compressed formats, although the hdiutil command tool still retained some access to them.

Disk Utility, seen here in 2011, has provided basic access to many disk image formats, but these are only a small selection of options available in the hdiutil command tool, or in DropDMG.

This shows the complex set of options available when creating a new disk image in Disk Utility in OS X 10.10 Yosemite, before the advent of APFS.

Support for compression was enhanced in OS X 10.11 El Capitan with the addition of lzfse in a new ULFO format, and macOS 10.15 Catalina added lzma in ULMO. In both cases, these new formats aren’t accessible in older versions of macOS.

APFS support

The arrival of a pre-release version of the new APFS file system in macOS 10.12 Sierra brought its support in disk images, although only for experimental purposes, and Apple cautioned users to ensure their contents were well backed up.

In addition to adding the more efficient ULMO compressed format, macOS 10.15 Catalina is the last to support many Classic Mac OS disk image formats, including those from DiskCopy42, DART and NDIF from Disk Copy 6.x. Support for AppleSingle and MacBinary encodings, and dual-fork file support, were also removed in macOS 11.0 Big Sur in 2020.

This ‘warning’ alert from 2020 illustrates one of the longstanding issues with disk images. Although integrity checking of disk images using checksums has been valuable, when an error is found there’s no possibility of repair or recovery as the image can’t be ‘attached’, so its file system can’t be mounted.

macOS 12 Monterey in 2021 brought multiple deprecations of older formats, including UDBZ using bzip2 compression, segmented UDIF images, and embedded resources. It’s also thought to be the first version of macOS in which UDIF read/write images (UDRW) have been stored in APFS sparse file format, although Apple has nowhere mentioned that. This has transformed what had previously been space-inefficient disk images that retained empty storage into a format that can prove almost as efficient as sparse bundles. This results from the Trim on mounting HFS+ and APFS file systems within the image freeing unused space, enabling that to be saved in the sparse file format.

Disk images have never been glamorous, but have remained at the heart of every Mac.

References

man hdiutil
Introduction
Tools
How read-write disk images have gone sparse
Performance
Bands, Compaction and Space Efficiency

Appendix: Disk image formats

Supported

• UDRW – UDIF read/write
• UDRO – UDIF read-only
• UDCO – UDIF ADC-compressed
• UDZO – UDIF zlib-compressed
• ULFO – UDIF lzfse-compressed (OS X 10.11)
• ULMO – UDIF lzma-compressed (macOS 10.15)
• UDTO – DVD/CD-R master for export
• UDSP – sparse image, grows with content
• UDSB – sparse bundle, grows with content, bundle-backed, Mac OS X 10.5
• UFBI – UDIF entire image with MD5 checksum.

Unsupported

• DC42 – Disk Copy 4.2 (Classic)
• DART – compressed, for Disk Archive/Retrieval Tool (Classic)
• Rdxx – read-only Disk Copy 6.0 formats
• NDIF – Disk Copy 6.0, including IMG and self-mounting SMI
• IDME – ‘Internet enabled’, on downloading post-processed to automatically copy visible contents into a folder, then move the image to the Trash. Now deemed highly insecure.
• UDBZ – UDIF bzip2-compressed image (deprecated).

Sometimes Macs behave in ways that you don’t forget. It was nearly six years ago, in 2019 when my iMac Pro was still running macOS Mojave, that it took over half an hour to start up in Safe mode. As became obvious, that was because it checked the integrity of every snapshot, including its long series of Time Machine backups. That behaviour stopped with the release of Catalina later that year, and revisiting the experience last week was interesting, when I examined what Safe mode does now in Sequoia.

Safe mode fsck

Checking disk integrity has long been a feature claimed of Safe mode, although Apple has only recently clarified that its checks are “similar to the more comprehensive check performed by the First Aid feature of Disk Utility,” unlike those in the days of Mojave.

Because these are run when starting up into Safe mode, macOS can’t report them directly to the user. There are only two places for their results to be recorded: in the fsck_apfs log at /var/log/fsck_apfs.log (and its error log at /var/log/fsck_apfs_error.log), and in the Unified log, neither of which are likely to be visited by users. It doesn’t seem possible that checks could be run before entries are made in the Unified log, and if they were their results would be inaccessible, and of no value to the user.

If you do look there, you might be surprised to discover that the checks run by fsck_apfs are ‘quick checks’, described as a “check whether the device is `clean’. If device is an APFS volume, fsck_apfs will quickly check the APFS container and the specified APFS volume. If device is an APFS container, fsck_apfs will quickly check the APFS container and all the APFS volumes in it. By default, no repairs are attempted during a quick check.” Thankfully they also no longer include Time Machine snapshots or backups.

For users, the most important of volumes to be checked is the Data volume in the current boot volume group. Yet careful examination of both log records reveals that such checks fail, returning error 65, because the device is mounted with write access. Although they could be performed using fsck_apfs‘s live verification, no attempt is made to do that. From the active boot volume group, only the Recovery and Update volumes are checked, together with all accessible external volumes.

Normal user fsck

Before you think that’s better than making no checks at all, the sequence of fsck_apfs checks run at the start of Safe mode appears to be identical to those made during startup into normal user mode, and reports the same failures. Thus, in this respect Safe mode is no different from normal, despite Apple listing this as a feature of Safe mode.

It’s worth reading the fsck_apfs man page to see the many options available in this command, which is the basis for all checks and repairs made to APFS volumes and containers. There are two in particular that you may prefer to invoke rather than using Disk Utility’s single option to both check and repair. My favourites that make it worth opening Terminal instead of Disk Utility are -n to prevent it from attempting to make any repairs, and -S to skip snapshots.

Given that fsck_apfs appears unable to repair any errors it finds in snapshots and admits to that in its man page where it states that “no repairs can be made”, it puzzles me that without the -S option it will still check all snapshots it comes across when running its checks. Perhaps it’s time to alter that default behaviour and make checking snapshots the explicit option.

First Aid

Disk Utility’s First Aid is descended from a succession of utilities starting with the Disk First Aid app in Classic Mac OS, which separated disk Verify and Repair features.

Mac OS X merged the two Classic apps Drive Setup and Disk First Aid, but kept them as distinct tools within its new Disk Utility app and retained separate Verify and Repair features.

Those two features were still present in Mountain Lion or Mavericks, shown here, but soon afterwards only Repair was available.

Now that APFS is mature and has proven itself more than a worthy successor to HFS+, the need to both check and repair its containers and volumes should be abating. This would be a good time to return to separate controls, allowing users to check APFS in the expectation that there are no errors to be found. This is more important now because of the absence of any alternatives.

Monopoly

Disk Utility is in a unique position. Apple has chosen to lock third-parties out of developing competing disk repair utilities, in particular those that can be used in Recovery mode. Users can’t opt for apps with more or better features, putting the onus on Apple to provide a comprehensive solution that meets users’ wishes, not its idea of essential requirements.

Conclusions

• Apple needs to be explicit about what disk checks are run in Safe mode, and where users can inspect their results.
• fsck_apfs is flexible and powerful, but has strange defaults that merit reconsideration.
• Disk Utility should offer more options beyond its single ‘check and repair’, in particular to check only and with snapshots excluded. Those are most important in Recovery mode, where there can be no alternative.