Following tests over two years ago, I recommended the use of sparse bundles (UDSB) rather than plain UDIF read-write (UDRW) disk images, because of their faster and more consistent performance. Since then, read-write disk images have adopted a sparse file format, making them more space-efficient. However, their write performance has remained substantially lower than sparse bundles, and the latter remain the preferred format.

More recent performance tests have confirmed this in Sequoia, as shown in the table below.

Most recently, tests of file compression showed relatively poor performance within a sparse bundle, with overall compression speed only 57% that of the SSD hosting the bundle despite apparently good write speeds. This article explores why, and whether that should alter choice of disk images.

Consistency in write speed

As previous articles note, inconsistent performance troubles disk images. Because of their complex bundle structure and storage across thousands of band files, it’s important to consider whether this poor performance is merely a reflection of that inconsistency.

The original Stibium write speed measurements, separated out from the collation shown in that recent article, show more dispersion about the regression line than other tests. As those are based on a limited number of file sizes, a total of 130 write speed measurements were performed using file sizes ranging from 4.4 MB to 3.0 GB, 50 of them being randomly chosen.

Those confirm that some individual write speeds are unexpectedly low, and dispersion isn’t as low as hoped for, but don’t make inconsistency a plausible explanation.

Band size

Unlike other types of disk image, data saved in sparse bundles is contained in many smaller files termed bands, and the maximum size of those band files can be set when a sparse bundle is created. By default, band size is set at 8.4 MB and there are only two ways to use a custom size, either when creating the sparse bundle using the command hdiutil create or in my free utility Spundle. Although those allow you to set a preferred band size, the size actually used may differ slightly from that.

Band size could have significant effects on the performance and stability of sparse bundles, as it determines how many band files are used to store data. Using a size that results in more than about 100,000 band files has in the past (with HFS+ at least) made sparse bundles prone to failure.

To assess the effect of different band sizes on compression rate, I therefore repeated exactly the same test file compression in sparse bundles with 12 band sizes ranging from 2.0 to 1020 MB, as measured rather than the value set.

In the graph above, time to complete the compression task is shown for each of the band sizes tested, and shows a minimum time (fastest compression) at a band size of 15.4 MB, of about 8.3 seconds, significantly lower than the 11.08 seconds at the default of 8.4 MB. Band sizes between 2.0-8.4 MB showed little differences, but increasing band size to 33.8 MB and greater resulted in much slower compression.

When expressed as compression rates, peak performance at a band size of 15.4 MB is even clearer, and approaches 1.9 GB/s, more than twice that of a read-write disk image, but still far below rates in excess of 2.6 GB/s found for the USB4 and internal SSDs. With band sizes above 33.8 MB, compression rates fall below those of a read-write disk image.

Another way of looking at optimum band size is to convert that into the number of band files required. Fastest rates were observed here in sparse bundles that would have about 6,000-7,000 bands in total when the sparse bundle is full, and in this case with 2,000-2,300 band files in use. Performance fell when the maximum number of band files fell below 3,000, or those in use fell below 1,000.

Streaming

Compression tests impose different demands on storage from conventional read or write benchmarks because file data to be compressed is streamed by reading data from storage, and writing compressed data out to the same storage device. Although source and destination are separate files, allowing simultaneous reading and writing, if those files are stored in common band files, it’s likely that access has to be limited and can’t be simultaneous.

As observed here, that’s likely to result in performance substantially below that expected from single-mode transfer tests. Unfortunately, it’s also extremely difficult to measure attempts to simultaneously read and write from different files in the same storage medium.

Conclusions

• Sparse bundles (UDSB) remain generally faster in use than read-write (UDRW) disk images.
• Sparse bundle performance is sensitive to band size.
• Default 8.4 MB bands aren’t fastest in all circumstances.
• When sparse bundle performance is critical, it may be worth optimising band file size instead of using the default.
• Tasks that stream data from and back to the same storage are likely to run more slowly in a sparse bundle than on its host storage.

Previous articles

Dismal write performance of Disk Images
Which disk image format?
Disk Images: Performance

The most common economy that many make when specifying their next Mac is to opt for just 512 GB internal storage, and save the $/€/£600 or so it would cost to increase that to 2 TB. After all, if you can buy a 2 TB Thunderbolt 5 SSD for around two-thirds of the price, why pay more? And who needs the blistering speed of that expensive internal storage when your apps work perfectly well with a far cheaper SSD? This article considers whether that choice matters in terms of performance.

To look at this, I’m going to use the same compression task that I used in this previous article, my app Cormorant relying on the AppleArchive framework in macOS to do all the heavy lifting. As results are going to differ considerably when using other apps and other tasks, I’d like to make it clear that this can’t reach general conclusions that apply to every task and every app: your mileage will vary. My purpose here is to show how you can work out whether using a slower disk will affect your app’s performance.

Methods

In that previous article, I concluded that compression speed was unlikely to be determined by disk performance, as even with all 10 P cores running compression threads, that rate only reached 2.28 GB/s, around the same as write speed to a good Thunderbolt 3 SSD. For today’s tests, I therefore set Cormorant to use the default number of threads (all 14 on my M4 Pro) so it would run as fast as the CPU cores would allow when compressing my standard 15.517 GB IPSW test file.

I’m fortunate to have a range of SSDs to test, and here use the 2 TB internal SSD of my Mac mini M4 Pro, an external USB4 enclosure (OWC Express 1M2) with a 2 TB Samsung 990 Pro SSD, and an external 2 TB Thunderbolt 3 SSD (OWC Envoy Pro SX). As few are likely to have access to such a range, I included two disk images stored on the internal SSD, a 100 GB sparse bundle, and a 50 GB read-write disk image, to see if they could be used to model external storage. All tests used unencrypted APFS file systems.

The first step with each was to measure its write speed using Stibium. Unlike more popular benchmarking apps for the Mac, Stibium measures the speed across a wide range of file sizes, from 2 MB to 2 GB, providing more insight into performance. After those measurements, those test files were removed, the large test file copied to the volume, and compressed by Cormorant at high QoS with the default number of threads.

Disk write speeds

Results in each test followed a familiar pattern, with rapidly increasing write speeds to a peak at a file size of about 200 MB, then a steady rate or slow decline up to the 2 GB tested. The read-write disk image was a bit more erratic, though, with high write speeds at 800 and 1000 MB.

At 2 GB file size, write speeds were:

• 7.69 GB/s for the internal SSD
• 7.35 GB/s for the sparse bundle
• 3.61 GB/s for the USB4 SSD
• 2.26 GB/s for the Thunderbolt 3 SSD
• 1.35 GB/s for the disk image.

Those are in accord with my many previous measurements of write speeds for those types of storage.

Compression rates

Times to compress the 15.517 GB test file ranked differently:

• 5.57 s for the internal SSD
• 5.84 s for the USB4 SSD
• 9.67 s for the sparse bundle
• 10.49 s for the Thunderbolt 3 SSD
• 16.87 s for the disk image.

When converted to compression rates, sparse bundle results are even more obviously an outlier, as shown in the chart below.

There’s a roughly linear relationship between measured write speed and compression rate in the disk image, Thunderbolt 3 SSD, and USB4 SSD, and little difference between the latter and the internal SSD. These suggest that disk write speed becomes the rate-limiting factor for compression when write speed falls below about 3 GB/s, but above that faster disks make little difference to compression rate.

Poor performance of the sparse bundle was a surprise, given how close its write speeds are to those of the host internal SSD. This is probably the result of compression writing a single very large file across its 14 threads; as the sparse bundle stores file data on a large number of band files, their overhead appears to have got the better of it. I will return to look at this in more detail in the near future, as sparse bundles have become popular largely because of their perceived superior performance.

The difference in compression rates between USB4 and Thunderbolt 3 SSDs is also surprisingly large. Of course, a Mac with fewer cores to run compression threads might not show any significant difference: a base M3 chip with 4 P and 4 E cores is unlikely to achieve a compression rate much in excess of 1.5 GB/s on its internal SSD because of its limited cores, so the restricted write speed of a Thunderbolt 3 SSD may not there become the rate-limiting factor.

Conclusions

• The rate-limiting step in task performance will change according to multiple factors, including the effective use of multiple threads on multiple cores, and disk performance.
• There’s no simple model you can apply to assess the effects of disk performance, and tests using disk images can be misleading.
• You can’t predict whether a task will be disk-bound from disk benchmark performance.
• Even expensive high-performance external SSDs can result in noticeably poor task performance. Maybe that money would be better spent on a larger internal SSD after all.

Of the two most dependable types of disk image, read-write (UDRW) disk images are the simpler. The only options you can set are its internal file system, whether the container is encrypted, and its maximum capacity. From there on, macOS will automatically Trim the disk image when it’s mounted, and adjust the size it takes on disk accordingly.

Sparse bundles can be more complex to configure in the first place, and may need routine maintenance to ensure their size on disk doesn’t grow steadily with use. This article considers how significant those complications are.

Band size

Most if not all storage divides data into blocks; in the case of SSDs, the default block size in APFS is 4,096 bytes, and that’s effectively the unit of storage in the single file of a read-write disk image.

Its equivalent in a sparse bundle is the maximum size that each of its band files can reach, normally set to 8.4 MB in macOS Sequoia and its predecessors, and equivalent to slightly more than 2,000 SSD blocks. Smaller band sizes are more efficient in the space required to store the contents of a sparse bundle, but require more band files for the same quantity of data stored. Experience with older versions of macOS suggests that it’s not a good idea for the total number of band files to exceed 100,000, but I’m not aware of any recent evidence to support that.

When creating very large sparse bundles, of the order of TB, macOS now may adjust the band size to limit their number. Some users report that those sparse bundles perform better as a result, for instance when being used to store backups on a network. There don’t appear to be any advantages to using band sizes less than 8.4 MB.

Neither DropDMG nor Disk Utility currently appear to allow any control over band size; Spundle and hdiutil not only let you specify custom band sizes when creating sparse bundles, but also allow you to change band size by copying an existing sparse bundle into a new one, which could be useful if you were changing the maximum size of a sparse bundle.

Compaction

Each time a writeable APFS or HFS+ file system is mounted in a disk image, including both sparse bundle and read-write types, storage blocks that are no longer in use should be returned for reuse by the process of Trimming. This is automatic, and in the case of single-file read-write disk images, is the only maintenance that occurs.

Because sparse bundles add another layer of band files over SSD storage, they too require periodic maintenance in the process of compaction. To compact a sparse bundle, macOS scans the band files and removes those that are no longer being used by the file system in the sparse bundle. This only applies to sparse bundles with APFS or HFS+ file systems, though, and isn’t guaranteed to free up any space. One small catch is that, by default, compaction won’t take place on a notebook running on battery power; hdiutil and Spundle provide an option to override that.

Space efficiency in use

I’ve been unable to find any objective comparison between space efficiency of modern read-write disk images and sparse bundles, so have performed my own on a USB4 external SSD connected to my Mac Studio M1 Max. I created two test cases of 125 GB images on a freshly-formatted APFS volume, one a sparse bundle with the default 8.4 MB band size, the other a read-write disk image (UDRW). In macOS Sequoia 15.0.1, the initial size they took on disk was 14.8 MB for the sparse bundle, and 336 MB for the disk image.

Using my utility Stibium, I then wrote and deleted a series of files in each, as follows:

1. 160 files of 2 MB to 2 GB written totalling 53 GB
2. 40 largest of those, sizes 600 MB to 2 GB, were then deleted, leaving 120
3. another 160 files written totalling 53 GB
4. 40 largest of those deleted, leaving a total of 240
5. another 160 files written totalling 53 GB
6. 40 largest of those deleted, leaving a total of 360
7. another 160 files written totalling 53 GB
8. 40 largest of those deleted, leaving a total of 480
9. another 160 files written totalling 53 GB, leaving a total of 640 files.

At that point, the sparse bundle had 104.08 GB stored in 12,408 band files, and the read-write disk image had 91.63 GB stored in its single file. I then deleted all files and folders in each of the images, and unmounted and remounted each image several times to ensure Trimming had completed. The sparse bundle then had 1.12 GB in 135 band files, and offered 124.66 GB of free space when mounted; the read-write disk image was slightly smaller at 937.9 MB on disk and offered exactly the same amount of free space when mounted.

Compacting the empty sparse bundle was reported as reclaiming 52.3 MB, and reduced the disk space taken by the band files slightly to 1.06 GB while retaining the same number of bands, and still offering 124.66 GB of free space when mounted.

In this case, following writing 800 files totalling over 250 GB, and deleting a total of 160 of them, compaction made a remarkably small difference to the free space returned following removal of all files. There is also very little difference in the space efficiency of sparse bundles and modern read-write disk images.

One consistent difference observed throughout was in write speeds: they remained constant at 3.2 GB/s for the sparse bundle, and 1.0 GB/s for the read-write disk image, as reported here previously.

Conclusions

• Sparse bundles are more complicated than read-write disk images (UDRW), with band size to be set, and compaction to be performed.
• Default band size appears to work well, and manually setting band size should seldom be necessary.
• Both types appear highly efficient in their use of disk space, with only small differences between them.
• Although it might be important to compact sparse bundles in some cases, the amount of free space returned by compaction is unlikely to be significant in many circumstances.

Previous articles

Introduction
Tools
How read-write disk images have gone sparse
Performance

Over the last few years, the performance of disk images has been keenly debated. In some cases, writing to a disk image proceeds at a snail’s pace, but this has appeared unpredictable. Over two years ago, I reported sometimes dismal write performance to disk images, summarised in the table below.

This article presents new results from tests performed using macOS 15.0.1 Sequoia, which should give a clearer picture of what performance to expect now.

Methods

Previous work highlighted discrepancies depending on how tests were performed, whether on freshly made disk images, or on those that had already been mounted and written to. The following protocol was used throughout these new tests:

1. A 100 GB APFS disk image was created using Disk Utility, which automatically mounts the disk image on completion.
2. A single folder was created on the mounted disk image, then it was unmounted.
3. After a few seconds, the disk image was mounted again by double-clicking it in the Finder, and was left mounted for at least 10 seconds before performing any tests. That should ensure read-write disk images are converted into sparse file format, and allows time for Trimming.
4. My utility Stibium then wrote 160 test files ranging in size from 2 MB to 2 GB in randomised order, a total of just over 53 GB, to the test folder.
5. Stibium then read those files back, in the same randomised order.
6. The disk image was then unmounted, its size checked, and it was trashed.

All tests were performed on a Mac Studio M1 Max, using its 2 TB internal SSD, and an external Samsung 980 Pro 2 TB SSD in an OWC Express 1M2 enclosure running in USB4 mode.

Results

These are summarised in the table below.

Read speeds for sparse bundles and read-write disk images were high, whether the container was encrypted or not. On the internal SSD, encryption resulted in read speeds about 1 GB/s lower than those for unencrypted tests, but differences for the external SSD were small and not consistent.

Write speeds were only high for sparse bundles, where they showed similar effects with encryption. Read-write disk images showed write speeds consistently about 1 GB/s, whether on the internal or external SSD, and regardless of encryption.

When unencrypted, read and write speeds for sparse (disk) images were also slower. Although faster than read-write disk images when writing to the internal SSD, read speed was around 2.2 GB/s for both. Results for encrypted sparse images were by far the worst of all the tests performed, and ranged between 0.08 to 0.5 GB/s.

Surprisingly good results were obtained from a new-style virtual machine with FileVault enabled in its disk image. Although previous tests had found read and write speeds of 4.4 and 0.7 GB/s respectively, the Sequoia VM achieved 5.9 and 4.5 GB/s.

Which disk image?

On grounds of performance only, the fastest and most consistent type of disk image is a sparse bundle (UDSB). Although on fast internal SSDs there is a reduction in read and write speeds of about 1 GB/s when encrypted using 256-bit AES, no such reduction should be seen on fast external SSDs.

On read speed alone, a read-write disk image is slightly faster than a sparse bundle, but its write speed is limited to 1 GB/s. For disk images that are more frequently read from than written to, read-write disk images should be almost as performant as sparse bundles.

However, sparse (disk) images delivered weakest performance, being particularly slow when encrypted. Compared with previous results from 2022, unencrypted write performance has improved, from 0.9 to 2.0 GB/s, but their use still can’t be recommended.

Performance range

It’s hard to explain how three different types of disk image can differ so widely in their performance. Using the same container encryption, write speeds ranged from 0.08 to 3.2 GB/s, for a sparse image and sparse bundle, on an external SSD with a native write speed of 3.2 GB/s. It’s almost as if sparse images are being deprecated by neglect.

Currently, excellent performance is also delivered by FileVault images used by Apple’s lightweight virtualisation on Apple silicon. The contrast is great between its 4.5 GB/s write speed and that of an encrypted sparse image at 0.1 GB/s, a factor of 45 when running on identical hardware.

Recommendations

• For general use, sparse bundles (UDSB) are to be preferred for their consistently good read and write performance, whether encrypted or unencrypted.
• When good write speed is less important, read-write disk images (UDRW) can be used, although their write performance is comparable to that of USB 3.2 Gen 2 at 10 Gb/s and no faster.
• Sparse (disk) images (UDSP) are to be avoided, particularly when encrypted, as they’re likely to give disappointing performance.
• Encrypting UDSB or UDRW disk images adds little if any performance overhead, and should be used whenever needed.

Previous articles

Introduction
Tools
How read-write disk images have gone sparse

Until about three years ago, most types of disk image had fixed size. When you created a read-write disk image (UDRW) of 10 GB, it occupied the same 10 GB on disk whether it was full or empty. The only two that could grow and shrink in size were sparse bundles and sparse disk images, with the former generally preferred for its better resizing and performance.

This changed silently in Monterey, since when read-write disk images have been automatically resized by macOS, and saved in APFS sparse file format. As this remains undocumented, this article explains how this works, and how and where you can use it to your advantage.

Sparse

In this context, the word sparse refers to two very different properties:

• Sparse bundles and sparse (disk) images are types of disk image that can change in size, and can be stored on a wide range of file systems and storage, including on NAS and other networked storage.
• Sparse files are stored in a highly efficient file format available in modern file systems, where only the data in a file is stored, and empty space within that file doesn’t waste storage space. It’s available in APFS, but not in HFS+.

Requirements

For read-write disk images to be sparse, the following are required:

• The disk image must be saved to, and remain on, an APFS volume.
• The file system within the disk image can be either APFS or HFS+, but not FAT or ExFAT.
• The disk image must be created and unmounted first. For that initial mount, the disk image isn’t a sparse file, so occupies its full size on disk.
• Whenever that disk image is mounted again, and has sufficient free space within its set limit, it will be saved in sparse file format, and occupy less than its full size on disk.

Demonstration

1. Create a new read-write disk image of at least 1 GB size with an internal file system of APFS on an APFS volume, using DropDMG, Disk Utility, or an alternative.
2. Once it has been created and mounted, unmount it in the Finder.
3. Select the disk image in the Finder and open the Get Info dialog for it. Confirm that its size on disk is the same as that set.
4. (Optional) Open the disk image using Precize, and confirm that it’s not a sparse file.
5. Double-click the disk image to mount it in the Finder, and wait at least 10 seconds before unmounting it.
6. Select the disk image in the Finder and open the Get Info dialog for it. Confirm that its size on disk is now significantly less than that set.
7. (Optional) Open the disk image using Precize, and confirm that it has now become a sparse file.

How it works

When you create the disk image, macOS creates and attaches its container, and creates and mounts the file system within that. This is then saved to disk as a regular file occupying the full size of the disk image, plus the overhead incurred by the disk image container itself. No sparse files are involved at this stage.

When that disk image is mounted next, its container is attached through diskarbitrationd, then its file system is mounted. If that’s APFS (or HFS+), it undergoes Trimming, as with other mounts. That coalesces free storage blocks within the image to form one contiguous free space. The disk image is then saved in APFS sparse file format, skipping that contiguous free space. When the file system has been unmounted and the container detached, the space used to store the disk image has shrunk to the space actually used within the disk image, plus the container overhead. Unless the disk image is almost full, the amount of space required to store it on disk will be smaller than the full size of the disk image.

This is summarised in the diagram below.

The size of read-write disk images is therefore variable depending on the contents, the effectiveness of Trimming in coalescing free space, and the efficiency of APFS sparse file format.

Conversion to sparse file

When mounting an APFS file system in a read-write disk image, APFS tests whether the container backing store is a sparse format, or a flat file. In the case of a newly created read-write disk image that hasn’t yet been converted into a sparse file, that’s detected prior to Spaceman (the APFS Space Manager) scanning for free blocks within its file system. When free blocks are found, APFS sets the type of backing store to sparse, gathers the sparse bytes and ‘punches a hole’ in the disk image’s file extents to convert the container file into sparse format. That appears in the log as:
handle_apfs_set_backingstore:6172: disk5s1 Set backing store as sparse
handle_apfs_set_backingstore:6205: disk5 Backing storage is a raw file
_punch_hole_cb:37665: disk3s5 Accumulated 4294967296 sparse bytes for inode 30473932 in transaction 3246918, pausing hole punching
where disk5s1 is the disk image’s mounted volume, and disk3s5 is the volume in which the disk image container is stored.

Trimming for efficient use of space

That conversion to sparse format is normally only performed once, but from then on, each time that disk image is mounted it’s recognised as having a sparse backup store, and Spaceman performs a Trim to coalesce free blocks and optimise on-disk storage requirements. For an empty read-write disk image of 2,390,202 blocks of 4,096 bytes each, as created in a 10 GB disk image, log entries are:
spaceman_scan_free_blocks:4106: disk5 scan took 0.000722 s, trims took 0.000643 s
spaceman_scan_free_blocks:4110: disk5 2382929 blocks free in 7 extents, avg 340418.42
spaceman_scan_free_blocks:4119: disk5 2382929 blocks trimmed in 7 extents (91 us/trim, 10886 trims/s)
spaceman_scan_free_blocks:4122: disk5 trim distribution 1:0 2+:0 4+:0 16+:0 64+:0 256+:7
accounting for a total of 9.8 GB.

Changes made to the contents of the disk image lead to a gradual reduction in Trim yield. For example, after adding files to the disk image and deleting them, instead of yielding the full 9.8 GB, only 2,319,074 blocks remain free, yielding a total of 9.5 GB.

For comparison, initial Trimming on a matching empty sparse bundle yields the same 9.8 GB. After file copying and deletion, and compaction of the sparse bundle, Trimming performs slightly better, yielding 2,382,929 free blocks for a total of 9.6 GB. Note that Trimming of sparse bundles is performed by APFS Spaceman separately from management of bands in backing storage, which isn’t a function of the file system.

Size efficiency

Although read-write disk images stored as sparse files are efficient in their use of disk space, they’re still not as compact as sparse bundles. For an empty 10 GB image, the read-write type requires 240 MB on disk, but a sparse bundle only needs 13.9 MB. After light use storing files, then deleting the whole contents, a 10 GB read-write disk image grows to occupy 501 MB, but following compaction a sparse bundle only takes 150 MB. That difference may not remain consistent over more prolonged use, though, and ultimately compacting sparse bundles may cease freeing any space at all.

It’s also important to remember that sparse bundles need to be compacted periodically, if any of their contents are deleted, or they may not reduce in size after deletions. Read-write disk images can’t be compacted, and reclaim disk space automatically.

Benefits and penalties

Read-write disk images saved as sparse files are different from sparse bundles in many ways. Like any sparse file, the disk image still has the same nominal size as its full size, and differs in the space taken on disk. Sparse bundles should normally only have band files sufficient to accommodate their current size, so their nominal size remains similar to the space they take on disk. The result is that, while read-write disk images in sparse file format will help increase free disk space, their major benefit is in reducing ‘wear’ in SSDs by not wasting erase-write cycles storing empty data.

Unlike the band file structure in sparse bundles, which can be stored on almost any disk, APFS sparse files have to be treated carefully if they are to remain compact. Moving them to another file system or over a network is likely to result in their being exploded to full size, and I have explained those limitations recently.

Both read-write disk images and sparse bundles deliver good read performance, but write performance is significantly impaired in read-write disk images but not sparse files. Encryption of disk images and sparse bundles also has significant effects on their performance, and in some cases write performance is badly affected by encryption. I have previously documented their performance in macOS Monterey, and will be updating those figures for Sequoia shortly.

Summary

• Read-write disk images saved on APFS storage in Monterey and later are no longer of fixed size, and should use significantly less disk space unless full, provided the disk image has been unmounted at least once since creation, and the file system in the disk image is APFS or HFS+.
• This is because macOS now saves the disk image in sparse file format.
• To retain the disk image in sparse file format, it needs to remain in APFS volumes, and normal precautions are required to maintain its efficient use of space. The disk image can’t be manually compacted, in the way that sparse bundles require to be.
• The main benefit of this strategy is to minimise erase-write cycles, so reducing ‘wear’ on SSDs.
• Sparse bundles are still more efficient in their use of disk space, and have higher write speeds, but read-write disk images are now closer in their efficiency and performance.

Previous articles

Introduction
Tools

If you’re going to use disk images of any type, then getting the right tool for the job is essential. This article considers the leading candidates:

• Disk Utility, bundled with macOS
• DropDMG, $24.99 from C-Command, or from the App Store
• Spundle, free from its Product Page here
• hdiutil, the command tool bundled with macOS.

Although I’m sure there are a great many others, IMHO those should be at the top of your list.

Disk Utility

Create a new disk image using the New Image command in its File menu and there’s a basic range of choices on offer.

This dialog has a longstanding bug, where it can reset the size you’ve entered if you change another setting, which can help you make mistakes. Otherwise, it gives limited access to some of the many options available, sufficient for the occasional and not too demanding user. Further options are available in its Images menu, including verification, adding checksums, conversions between types, and resizing. Notable by its absence is the ability to change the password of an encrypted disk image, which is unhelpfully deferred to the command line.

Documentation in Disk Utility’s Help book is also scant, and insufficient to serve as a reference. As Apple doesn’t provide any further technical information, apart from that in man hdiutil, you may find yourself searching websites such as this.

DropDMG

Since its release over 22 years ago, this has been the first choice for many who need to work with disk images, and is without doubt the best for those who distribute software in disk images. It has grown into the most comprehensive and capable utility for working with any type of disk image, and is backed up by a superb 123-page manual that goes a long way to filling the gap left by Apple. That manual is well-maintained, and contains links to recent technical articles and further information.

DropDMG’s options for creating a new disk image far exceed those in Disk Utility. Particularly helpful are the compatible version hints shown on various options, to remind you of which file systems are available in different macOS versions, and which types of disk image container are supported. DropDMG will even convert old NDIF disk images last used in Mac OS 9 to more modern formats. It will also change the password of an encrypted disk image from a menu command.

For those who need to work with standard configurations, perhaps for software distribution, it lets you save and reuse them with ease. Those can include signing with developer certificates, product licences, background images, custom volume icons, and more. Whichever type of disk image you want to create or maintain, DropDMG should be your first choice.

Spundle

There are a few options for sparse bundles that even DropDMG doesn’t expose, such as control over band size, the ability to resize a sparse bundle, and to change its band size. If you want access to those, Spundle is a useful adjunct.

Note that, unlike DropDMG, Spundle only works with sparse bundles.

hdiutil

Although this remains the definitive command tool that offers types of disk image and features you didn’t even know existed, it’s fiendishly complex to use, with a sprawling and overloaded grammar. Its man page runs to more than 11,000 words, but appears never to have been rewritten into any cohesive account of disk images, or command options. For example, change information is given in two sections, Compatibility and What’s New. Changes made in Catalina and later appear at the end of the Compatibility section, then the final What’s New section reverses time order and goes back from Catalina to Mac OS X 10.5.

I only recommend hdiutil for those who need to work with disk images in shell scripts, or for those few features that aren’t available in DropDMG or, for sparse bundles, in Spundle. It’s a command tool of last resort.

Previous article

Introduction

A disk image is a file or a bundle containing what could otherwise be the contents of a disk. It’s a common way to store and move complete file systems in a neat package, for items that need to be separated from the physical storage provided by a drive. macOS uses disk images for some tasks of great importance, including:

• Recovery and Hardware Diagnostics systems,
• additions to macOS such as Safari, its supporting frameworks, and dyld caches, in cryptexes,
• networked storage for Time Machine backups, in sparse bundles,
• lightweight virtual machines on Apple silicon Macs.

You could use them to store encrypted data on unencrypted volumes, and they’re often used for delivering Apple and third-party software.

Disk images are poorly documented for both users and developers, and have changed significantly over the last few years. Articles in this series explain how to choose between different types of disk image, how to create and use them, and what to do when they go wrong.

Containers and file systems

Disk images consist of two distinct components: the file or bundle itself functioning as a container, and the file system contained inside it. When referred to in this context, disk image containers are completely unrelated to the sandbox containers found in ~/Library/Containers.

This distinction is important in several respects, although it isn’t apparent when you use disk images in the Finder. Preparing a disk image for access involves two separate functions: attaching its container, and mounting any file systems found inside it. When that’s performed by the Finder, perhaps by double-clicking the disk image, those two actions appear fused into one. Similarly, removing the disk image requires all its mounted file systems to be unmounted first, then the container is detached.

One feature that’s widely confused is the encrypted disk image. This involves encryption of the whole container, rather than using an encrypted file system within it. Now that Disk Utility no longer supports the creation of encrypted HFS+ volumes, one remaining alternative is to use an encrypted disk image containing an HFS+ volume.

If you want an analogy for disk images, attaching the container is like connecting an external disk, and once that has been performed, the file systems contained by that disk have to be mounted before you can access their contents.

Types

There are many different types of disk image in use, of which the two this series is most concerned with are plain UDIF read-write disk images (UDRW), and sparse bundles (UDSB). Others you may encounter include:

• plain UDIF read-only (UDRO),
• various compressed versions of UDRW,
• CD/DVD master for export (UDTO),
• sparse disk image (UDSP), a single file rather than a bundle.

Those specify the container format; within each, there’s the usual choice of file systems, although throughout these articles it will normally be assumed that APFS will be used unless otherwise specified.

The word sparse in sparse bundle and sparse disk image doesn’t refer to APFS sparse files, but to the fact that those types of disk image can grow and diminish in size, and normally try to occupy the minimum amount of disk space. This is an unfortunate name collision.

Structure

With the exception of sparse bundles, all disk images are contained within a single file of opaque structure.

Sparse bundles consist of a single bundle folder containing:

• bands, a folder containing the contents of the disk image in band files
• info.plist and its backup copy info.bckup, containing settings including band size
• lock, an empty file
• mapped, a folder containing small data files to match all of the band files except the first
• token, an empty file.

Container size

Until this changed in Monterey (or thereabouts), non-sparse disk images had fixed container sizes. Create a UDIF read-write disk image (UDRW) of 10 GB, and the space occupied by it on disk was approximately 10 GB, whether it was empty or full. Although it remains undocumented, when stored on APFS volumes, UDRW disks can now take advantage of APFS sparse file format, and will normally only occupy the disk space required for the contents of their file system.

This is only true once the disk image has been mounted for the first time after it has been created, mounted and unmounted. To see this, create a test read-write disk image (for example, using Disk Utility) of 10 GB size. Then unmount it, and use the Finder’s Get Info command to inspect its size on disk. That will be 10 GB. Then mount the disk image again, pause a couple of seconds, unmount it, and Get Info will show its size on disk is now much smaller.

As I’ll explain in detail in a later article, this is because each time that disk image is mounted, if its internal file system is HFS+ or APFS, its contents will be trimmed, and saved to disk in sparse file format, which omits all its empty space. This only applies to read-write disk images when they’re stored in APFS file systems; copy them to HFS+ and they’ll explode to full size, as HFS+ doesn’t support the sparse file format.

Considering just the two leading types, empty sizes for a 100 GB disk image are:

• a sparse bundle is 35.4 MB empty, 53.3 GB when containing about 51 GB files, stored across 6,359 band files.
• a read-write disk image (UDRW) shrinks to 333.8 MB once stored as an empty sparse file, 53.6 GB when containing about 51 GB files, in a single file container.

Performance

Some types of disk image perform poorly, and can be very slow to write to. Recent versions of macOS have brought improvements, although some options such as encryption can still impair performance significantly. For the two leading types, when their container is stored on the internal SSD of an Apple silicon Mac, with native read and write speeds of around 6-7 GB/s:

• an initially empty unencrypted sparse bundle reads at 5.1 GB/s, and writes at 4.8 GB/s.
• an initially empty unencrypted read-write disk image (UDRW) reads at 5.3 GB/s, and writes at only 1 GB/s.

Tests were performed using my utility Stibium, across a range of 160 files of 2 MB to 2 GB size in randomised order, with macOS 15.0.1.

Key points

• Disk images consist of a file or bundle containing one or more file systems; the container and its contents are distinct.
• To access the contents of a disk image, the container is first attached, then the file system(s) within it are mounted. In the Finder, those two processes appear as a single action.
• Encrypted disk images encrypt the container, and don’t necessarily contain encrypted file systems.
• Disk images come in many different types, and can contain different file systems.
• Sparse bundles have a file and folder structure inside their bundle folder, with their data saved in band files; all other disk images are single files.
• Sparse bundles grow and shrink according to the size of files stored within them.
• In recent macOS, and on APFS, read-write disk images (UDRW) are stored in APFS sparse file format, enabling them to grow and shrink as well.
• In recent macOS, unencrypted sparse bundles have read and write performance close to that of the disk they’re stored on. Read-write disk images read at similar speeds, but write more slowly, at about 20% of read speed.