Following tests over two years ago, I recommended the use of sparse bundles (UDSB) rather than plain UDIF read-write (UDRW) disk images, because of their faster and more consistent performance. Since then, read-write disk images have adopted a sparse file format, making them more space-efficient. However, their write performance has remained substantially lower than sparse bundles, and the latter remain the preferred format.

More recent performance tests have confirmed this in Sequoia, as shown in the table below.

Most recently, tests of file compression showed relatively poor performance within a sparse bundle, with overall compression speed only 57% that of the SSD hosting the bundle despite apparently good write speeds. This article explores why, and whether that should alter choice of disk images.

Consistency in write speed

As previous articles note, inconsistent performance troubles disk images. Because of their complex bundle structure and storage across thousands of band files, it’s important to consider whether this poor performance is merely a reflection of that inconsistency.

The original Stibium write speed measurements, separated out from the collation shown in that recent article, show more dispersion about the regression line than other tests. As those are based on a limited number of file sizes, a total of 130 write speed measurements were performed using file sizes ranging from 4.4 MB to 3.0 GB, 50 of them being randomly chosen.

Those confirm that some individual write speeds are unexpectedly low, and dispersion isn’t as low as hoped for, but don’t make inconsistency a plausible explanation.

Band size

Unlike other types of disk image, data saved in sparse bundles is contained in many smaller files termed bands, and the maximum size of those band files can be set when a sparse bundle is created. By default, band size is set at 8.4 MB and there are only two ways to use a custom size, either when creating the sparse bundle using the command hdiutil create or in my free utility Spundle. Although those allow you to set a preferred band size, the size actually used may differ slightly from that.

Band size could have significant effects on the performance and stability of sparse bundles, as it determines how many band files are used to store data. Using a size that results in more than about 100,000 band files has in the past (with HFS+ at least) made sparse bundles prone to failure.

To assess the effect of different band sizes on compression rate, I therefore repeated exactly the same test file compression in sparse bundles with 12 band sizes ranging from 2.0 to 1020 MB, as measured rather than the value set.

In the graph above, time to complete the compression task is shown for each of the band sizes tested, and shows a minimum time (fastest compression) at a band size of 15.4 MB, of about 8.3 seconds, significantly lower than the 11.08 seconds at the default of 8.4 MB. Band sizes between 2.0-8.4 MB showed little differences, but increasing band size to 33.8 MB and greater resulted in much slower compression.

When expressed as compression rates, peak performance at a band size of 15.4 MB is even clearer, and approaches 1.9 GB/s, more than twice that of a read-write disk image, but still far below rates in excess of 2.6 GB/s found for the USB4 and internal SSDs. With band sizes above 33.8 MB, compression rates fall below those of a read-write disk image.

Another way of looking at optimum band size is to convert that into the number of band files required. Fastest rates were observed here in sparse bundles that would have about 6,000-7,000 bands in total when the sparse bundle is full, and in this case with 2,000-2,300 band files in use. Performance fell when the maximum number of band files fell below 3,000, or those in use fell below 1,000.

Streaming

Compression tests impose different demands on storage from conventional read or write benchmarks because file data to be compressed is streamed by reading data from storage, and writing compressed data out to the same storage device. Although source and destination are separate files, allowing simultaneous reading and writing, if those files are stored in common band files, it’s likely that access has to be limited and can’t be simultaneous.

As observed here, that’s likely to result in performance substantially below that expected from single-mode transfer tests. Unfortunately, it’s also extremely difficult to measure attempts to simultaneously read and write from different files in the same storage medium.

Conclusions

• Sparse bundles (UDSB) remain generally faster in use than read-write (UDRW) disk images.
• Sparse bundle performance is sensitive to band size.
• Default 8.4 MB bands aren’t fastest in all circumstances.
• When sparse bundle performance is critical, it may be worth optimising band file size instead of using the default.
• Tasks that stream data from and back to the same storage are likely to run more slowly in a sparse bundle than on its host storage.

Previous articles

Dismal write performance of Disk Images
Which disk image format?
Disk Images: Performance

If you’re going to use disk images of any type, then getting the right tool for the job is essential. This article considers the leading candidates:

• Disk Utility, bundled with macOS
• DropDMG, $24.99 from C-Command, or from the App Store
• Spundle, free from its Product Page here
• hdiutil, the command tool bundled with macOS.

Although I’m sure there are a great many others, IMHO those should be at the top of your list.

Disk Utility

Create a new disk image using the New Image command in its File menu and there’s a basic range of choices on offer.

This dialog has a longstanding bug, where it can reset the size you’ve entered if you change another setting, which can help you make mistakes. Otherwise, it gives limited access to some of the many options available, sufficient for the occasional and not too demanding user. Further options are available in its Images menu, including verification, adding checksums, conversions between types, and resizing. Notable by its absence is the ability to change the password of an encrypted disk image, which is unhelpfully deferred to the command line.

Documentation in Disk Utility’s Help book is also scant, and insufficient to serve as a reference. As Apple doesn’t provide any further technical information, apart from that in man hdiutil, you may find yourself searching websites such as this.

DropDMG

Since its release over 22 years ago, this has been the first choice for many who need to work with disk images, and is without doubt the best for those who distribute software in disk images. It has grown into the most comprehensive and capable utility for working with any type of disk image, and is backed up by a superb 123-page manual that goes a long way to filling the gap left by Apple. That manual is well-maintained, and contains links to recent technical articles and further information.

DropDMG’s options for creating a new disk image far exceed those in Disk Utility. Particularly helpful are the compatible version hints shown on various options, to remind you of which file systems are available in different macOS versions, and which types of disk image container are supported. DropDMG will even convert old NDIF disk images last used in Mac OS 9 to more modern formats. It will also change the password of an encrypted disk image from a menu command.

For those who need to work with standard configurations, perhaps for software distribution, it lets you save and reuse them with ease. Those can include signing with developer certificates, product licences, background images, custom volume icons, and more. Whichever type of disk image you want to create or maintain, DropDMG should be your first choice.

Spundle

There are a few options for sparse bundles that even DropDMG doesn’t expose, such as control over band size, the ability to resize a sparse bundle, and to change its band size. If you want access to those, Spundle is a useful adjunct.

Note that, unlike DropDMG, Spundle only works with sparse bundles.

hdiutil

Although this remains the definitive command tool that offers types of disk image and features you didn’t even know existed, it’s fiendishly complex to use, with a sprawling and overloaded grammar. Its man page runs to more than 11,000 words, but appears never to have been rewritten into any cohesive account of disk images, or command options. For example, change information is given in two sections, Compatibility and What’s New. Changes made in Catalina and later appear at the end of the Compatibility section, then the final What’s New section reverses time order and goes back from Catalina to Mac OS X 10.5.

I only recommend hdiutil for those who need to work with disk images in shell scripts, or for those few features that aren’t available in DropDMG or, for sparse bundles, in Spundle. It’s a command tool of last resort.

Previous article

Introduction