As the Trump administration pulls government websites and data offline, it is selectively stripping away the public record, letting the president declare his own version of history, archivists and historians said.
The head of the National Archives, which has been described as “the custodian of America’s collective memory,” was fired by President Trump in February.
Disk images, files that contain the contents of a physical storage medium, go back long before the first Mac. Among other tasks, they were originally used to contain representations of floppy disks for replication in manufacture.
Today disk images are at the heart of macOS, and widely used by third-parties. They’re an essential part of macOS installers, home to Recovery mode, and the basis for cryptexes. They’ve been used to burn and replicate optical disks, to archive disk contents, extensively for network backups, and for the distribution of software.
Classic Mac OS
In Classic Mac OS there were two utilities that worked with different formats: Disk Copy used replicas later in DC42 format, after Disk Copy version 4.2, while compressed formats known as DART were handled by the Disk Archive/Retrieval Tool, hence their name.
Mac OS 9 brought Disk Copy 6.0 with added support for the New Disk Image Format (NDIF), which supported resource forks, and ended with its last release version 6.3.3. This also supported read-only Rdxx formats.
By this time, variants of formats had become complex. Here, Disk Copy is configured to create a read-only compressed .img file containing the contents of a standard 1.4 MB floppy disk. In the upper window, it has completed validating the checksum on a self-mounting .smi disk image that’s part of a DiskSet. These could also be signed, using certificates issued not by Apple but by DigiSign.
Here’s Disk Copy saving an image of a hard disk using a similar read-only compressed format, this time to accommodate 1.5 GB.
Mac OS X
The release of Mac OS X 10.1 Puma in 2001 brought Apple’s new Universal Disk Image Format (UDIF), used in DMG disk images, which only had a single fork as its resource fork was embedded in the data fork. Although pre-release versions of Disk Copy 6.4 and 6.5 were available with UDIF support for Mac OS 9, neither was ever released, leaving Classic Mac OS without access to UDIF images. Its support for compression options in Apple Data Compression (ADC) unified the two disk image types, and extended support for images larger than a floppy disk. This new format enabled disk images to represent whole storage devices, complete with a partition map and disk-based drivers.
Tools provided in Mac OS X for working with disk images include Disk Utility and the command tool hdiutil.
On 21 January 2002, the first version of DropDMG, a third-party substitute for creating disk images, was released by C-Command Software. This quickly enabled developers to create disk images with artwork, licences and other features that weren’t accessible from the tools bundled in Mac OS X. DropDMG has flourished over the last 23 years, and remains popular today.
DropDMG’s options for creating a new disk image far exceed those in Disk Utility. Particularly helpful are the compatible version hints shown on various options, to remind you of which file systems are available in different macOS versions, and which types of disk image container are supported. DropDMG will even convert old NDIF disk images last used in Mac OS 9 to more modern formats. It will also change the password of an encrypted disk image from a menu command.
In Mac OS X 10.2 (2002), UDIF and most other supported formats were served from a kernel extension without requiring a helper process. The following year, 10.3 Panther started using a faceless utility DiskImageMounter to mount disk images. Apple then dropped support for embedded resource forks in disk images in Mac OS X 10.4.7, and newly created disk images became less compatible with older Mac OS versions.
Sparse bundles
Until Mac OS X 10.5 Leopard in 2007, all disk images had used single-file formats, although some could be segmented across file sets. Leopard introduced the sparse bundle with its folder of smaller band files containing data. These enabled the image to grow and shrink in size, and became popular means of storing mountable Mac file systems on servers using different file systems.
This is another third-party tool that improved access to disk images from the GUI, DMG Packager, seen in 2009. Unlike DropDMG, this appears to have vanished without trace.
In 2011, with the release of Mac OS X 10.7 Lion, Apple removed more support for old disk image formats. DiskImageMounter no longer opened NDIF .img, .smi self-mounting, .dc42 and .dart compressed formats, although the hdiutil command tool still retained some access to them.
Disk Utility, seen here in 2011, has provided basic access to many disk image formats, but these are only a small selection of options available in the hdiutil command tool, or in DropDMG.
This shows the complex set of options available when creating a new disk image in Disk Utility in OS X 10.10 Yosemite, before the advent of APFS.
Support for compression was enhanced in OS X 10.11 El Capitan with the addition of lzfse in a new ULFO format, and macOS 10.15 Catalina added lzma in ULMO. In both cases, these new formats aren’t accessible in older versions of macOS.
APFS support
The arrival of a pre-release version of the new APFS file system in macOS 10.12 Sierra brought its support in disk images, although only for experimental purposes, and Apple cautioned users to ensure their contents were well backed up.
In addition to adding the more efficient ULMO compressed format, macOS 10.15 Catalina is the last to support many Classic Mac OS disk image formats, including those from DiskCopy42, DART and NDIF from Disk Copy 6.x. Support for AppleSingle and MacBinary encodings, and dual-fork file support, were also removed in macOS 11.0 Big Sur in 2020.
This ‘warning’ alert from 2020 illustrates one of the longstanding issues with disk images. Although integrity checking of disk images using checksums has been valuable, when an error is found there’s no possibility of repair or recovery as the image can’t be ‘attached’, so its file system can’t be mounted.
macOS 12 Monterey in 2021 brought multiple deprecations of older formats, including UDBZ using bzip2 compression, segmented UDIF images, and embedded resources. It’s also thought to be the first version of macOS in which UDIF read/write images (UDRW) have been stored in APFS sparse file format, although Apple has nowhere mentioned that. This has transformed what had previously been space-inefficient disk images that retained empty storage into a format that can prove almost as efficient as sparse bundles. This results from the Trim on mounting HFS+ and APFS file systems within the image freeing unused space, enabling that to be saved in the sparse file format.
Disk images have never been glamorous, but have remained at the heart of every Mac.
UDSB – sparse bundle, grows with content, bundle-backed, Mac OS X 10.5
UFBI – UDIF entire image with MD5 checksum.
Unsupported
DC42 – Disk Copy 4.2 (Classic)
DART – compressed, for Disk Archive/Retrieval Tool (Classic)
Rdxx – read-only Disk Copy 6.0 formats
NDIF – Disk Copy 6.0, including IMG and self-mounting SMI
IDME – ‘Internet enabled’, on downloading post-processed to automatically copy visible contents into a folder, then move the image to the Trash. Now deemed highly insecure.
With Mac OS X came a new tool for installing and updating the system, quite different from what had been used in Classic Mac OS. The Mac OS X Installer app uses packages (.pkg), and metapackages (.mpkg) containing multiple packages to be installed together. Apple thus provided installations and updates as metapackages that could either be downloaded from its update servers using Software Update, or from the update’s web page. The same method was used until Big Sur, when updates changed again.
A system update in Mac OS X 10.1.3 in March 2002 was installed by the Installer app following authentication.
This update required just 148 MB of disk space, and could readily be accommodated by any of these volumes of 11.5 GB capacity.
Most updates were concluded by a period ‘optimising system performance’, as determined by the post-install scripts in the package.
Here, Software Update is delivering Security Update 2004-05-03 to Mac OS X 10.3.3 Panther. Some system components like QuickTime were still supplied and installed separately, but the great majority of Mac OS X was integrated into a whole, and there were no options to install separate components.
Over this period, the system and user data shared a single boot volume, and updating the system mainly involved replacement of updated files. Installer packages contained those replacements together with scripts that were run to update the contents of the boot volume. For much of this, firmware updates were still supplied and installed separately, although later they were integrated into macOS updates.
Until the introduction of System Integrity Protection (SIP) in El Capitan, the only protection provided to system files and folders was in their permissions. Incomplete or incorrect installations and updates were therefore not uncommon, as despite its name, SIP didn’t have any means of verifying the integrity of system files. A procedure was introduced to verify directory structures and permissions against those listed in the Bill of Materials (BoM) for macOS updates, by repairing permissions, but that was still unable to verify the integrity of the files themselves.
Installer metapackages are highly portable, and were commonly downloaded to be installed on multiple Macs. To keep updates as small as possible, they were provided in two forms: a Delta update converted only the previous release, while a Combo update contained everything required to update the last major version and all intermediate minor versions in a single step.
The High Sierra 10.13 upgrade in September 2017 brought greatest change, with its inclusion of Apple’s new file system APFS. Macs that didn’t have a Fusion Drive had their system volume converted to APFS during the upgrade, although it was another year before the same happened to Fusion Drives.
Updates didn’t always work out right for everyone. This was a common problem with High Sierra Supplemental Update of 29 November 2017, for example.
This all changed with the first version of macOS to boot from a signed snapshot, Big Sur, in November 2020, to support the improved Secure Boot of Apple silicon Macs. This abandoned the use of Installer packages, relying instead on an Update Brain integrated into each installer app and downloaded update.
From then onwards, Apple has provided several different presentations of macOS installers and updates:
Updates are only delivered through Software Update or its command tool equivalent softwareupdate, and have to be downloaded from Apple’s servers, or delivered through a local Content Caching Server.
Full macOS installer apps are offered in the App Store then delivered through Software Update.
Full macOS installer packages are available through softwareupdate or direct from Apple’s servers, and are named InstallAssistant. When installed, these create a full installer app.
Full macOS installers are offered in Recovery, from where they’re downloaded from Apple’s servers.
Full IPSW images containing firmware, Recovery and macOS partitions can be installed to restore Apple silicon Macs in DFU mode, using Apple Configurator 2 or later versions of the Finder. Those effectively reset that Mac to factory condition with that version of macOS pre-installed.
There are further complications to this. For instance, an older macOS Installer app can’t be run in a newer major version of macOS. The workaround for that is to create a bootable installer volume, and boot from that to run its older installer.
macOS updates are supplied compressed, and require up to 30 minutes preparation before they can be installed.
There are now no optional installations, as every copy of any given version of macOS is identical within its Signed System Volume (SSV).
The size of these new updates is considerably greater than those of older Installer packages, particularly in Big Sur, as engineering optimisations were being performed.
This chart shows cumulative sizes of updates to macOS on Intel Macs from 10.14 Mojave, with its traditional single boot volume, through to macOS Sonoma 14.6. Each point represents the cumulative sum of all updates to that major version of macOS required to reach that minor version. Thus the point for 14.2 is the total of update sizes for 14.1, 14.1.1, 14.1.2 and 14.2. Sizes used aren’t those reported by Software Update, but those of the download itself, as reported in articles here, indexed on this page. Lines shown are best fits by linear regression.
Update sizes rose markedly from Mojave, with its single boot volume, to Catalina, with its boot volume group, and again to a peak in Big Sur, with the SSV. They fell again as Monterey introduced greater efficiency, and Ventura and Sonoma have been almost identical, and smaller than Mojave.
Apple silicon Macs started with the huge updates of Big Sur, which were even larger than those for Intel models, and benefitted from the improved efficiency of Monterey and Ventura. Unlike Intel Macs, though, Sonoma has seen further reduction in update sizes, although in each update they remain significantly larger than those for Intel models.
macOS Ventura in 2022 experimented with Rapid Security Responses (RSRs), much smaller updates intended to provide urgent security fixes to Safari and some of its supporting components. These take advantage of cryptexes, cryptographically verified disk images stored on the hidden Preboot volume. Updating cryptexes alone is far quicker too, as the SSV is left untouched. Unfortunately, the second RSR resulted in serious problems with Safari so had to be replaced three days later. The last RSR was released on 12 July 2023, and they appear to have been abandoned since.
Upgrading to the first release of the next major version of macOS had required downloading its full installer app from the App Store. Apple broke from this in macOS Ventura in October 2022, when that new macOS was installed as an update instead. Although this reduced its size and installation time required, it caught many users on the hop, as Apple provided no warning of the change. This approach has since become standard with both Sonoma and Sequoia.
Installing and updating the Mac’s operating system has probably changed more over the last 41 years than any other feature.
With more new M4 Macs in the offing, one question that I’m asked repeatedly is whether you should save money by getting a Mac with the smallest internal SSD and extend that using cheaper external storage. This article considers the pros and cons.
Size and prices
In Apple’s current M4 models, the smallest internal storage on offer is 256 GB. For the great majority, that’s barely adequate if you don’t install any of your own apps. It might suffice in some circumstances, for example if you work largely from shared storage, but for a standalone Mac it won’t be sufficient in five years time. Your starting point should therefore be a minimum of 512 GB internal SSD. Apple’s typical charge for increasing that to 2 TB is around $/€/£ 600.
The alternative to 2 TB internally would be an external 2 TB SSD. Unless you’re prepared to throw it away after three years, you’ll want to choose the most versatile interface that’s also backward compatible. The only choice here is Thunderbolt 5, which currently comes at a small premium over USB4 or Thunderbolt 3. Two TB would currently cost you $/€/£ 380-400, although those prices are likely to reduce in the coming months as TB5 SSDs come into greater supply.
Don’t be tempted to skimp with a USB 3.2 Gen 2 external SSD if that’s going to be your main storage. While it might seem a reasonable economy now, in 3-5 years time you’ll regret it. Besides, it may well have severe limitations in not Trimming as standard, and most don’t support SMART health indicators.
Thus, your expected saving by buying a Mac with only 512 GB internal storage, and providing 2 TB main storage on an external SSD, is around $/€/£ 200-220, and that’s really the only advantage in not paying Apple’s high price for an internal 2 TB SSD.
Upgrading internal storage in an Apple silicon model currently isn’t feasible for most users. As Apple doesn’t support such upgrades, they’re almost certain to invalidate its warranty and any AppleCare+ cover. That could change in the future, at least for some models like the Mac mini and Studio, but I think it unlikely that Apple would ever make an upgrade cheaper than initial purchase.
External boot disk
One of the few compelling reasons for choosing a Mac with minimal internal storage is when it’s going to be started up from an external boot disk. Because Apple silicon Macs must always start their boot process from their internal storage, and that Mac still needs Recovery and other features on its internal SSD, you can’t run entirely from an external SSD, but you could probably get away with the smallest available for its other specifications, either 256 or 512 GB.
Apple silicon Macs are designed to start up and run from their internal storage. Unlike Intel Macs with T2 chips, they will still boot from an external disk with Full Security, but there are several disadvantages in them doing so. Among those are the fact that, on an external boot disk, FileVault encryption isn’t performed in hardware and is inherently less secure, and AI isn’t currently supported when booted from an external disk. Choosing to do that thus involves compromises that you might not want to be stuck with throughout the lifetime of that Mac.
External media libraries
Regardless of the capacity of a Mac’s internal storage, it’s popular to store large media libraries on external storage, and for many that’s essential. This needs to be planned carefully: some libraries are easier to relocate than others, and provision has to be made for their backups. If you use hourly Time Machine backups for your working folders, you’ll probably want to back up external media libraries less frequently, and to different external storage.
External Home folder
Although it remains possible to relocate a user’s entire Home folder to external storage, this seems to have become more tricky in recent versions of macOS. Home folders also contain some of the most active files, particularly those in ~/Library, so moving them to an external SSD is going to require its good performance.
A more flexible alternative is to extend some working folders to external storage, while retaining the Home folder on internal storage. This can fit well with backup schedules, but you will still need to ensure the whole Home folder is backed up sufficiently frequently. This does have an unfortunate side-effect in privacy protection: this may require most of your working apps to be given access to Removable Volumes in the Files & Folders item in Privacy & Security settings. Thankfully, that should only need to be performed once when first using an app with external storage.
How much free space do you need?
When you’re weighing up your options to minimise the size of your new Mac’s internal storage, you also need to allow sufficient free space on each disk. APFS is very different from HFS+ in this respect: on external disks, in particular, HFS+ continues to work happily with just a few MB free, and could be filled almost to capacity. APFS, modern macOS and SSDs don’t work like that.
Measuring how much free space is needed isn’t straightforward either, as macOS trims back on its usage in response to falling free space. Some key features, such as retaining log entries, are sacrificed to allow others to continue. Snapshots can be removed or not made. Perhaps the best measurements come from observing the space requirements of VMs, where total virtual disk space much below 50 GB impairs running of normal functions. That’s the total size of the virtual disk, not the amount of free space, and doesn’t apply when iCloud or AI are enabled.
The other indicator of minimum free space requirements is for successful upgrading of macOS, which appears to be somewhere between 30-40 GB. This makes it preferable to keep an absolute minimum of around 50 GB free at all times. When possible, 100 GB gives more room for comfort.
SSD wear and performance
When the first M1 Macs were released, base models with just 8 GB of memory and 128 GB internal SSDs were most readily available, with custom builds (BTO) following later. As a result, many of those who set out to assess Apple’s new Macs ended up stress-testing those with inadequate memory and storage for the tasks they ran.
Many noticed rapid changes in their SSD wear indicators, and some were getting worryingly close to the end of their expected working life after just three years. Users also reported that SSD performance was falling. The reasons for those are that SSDs work best, age slowest, and remain fastest when they have ample free space. One common rule of thumb is to keep at least 20-25% of SSD capacity as free space, although evidence is largely empirical, and in places confused.
The simplest factor to understand is the effect of SSD size on wear. As the memory in an SSD is expected to last a fixed number of erase-write cycles, all other things being equal, writing and rewriting the same amount of data to a smaller SSD will reach that number more quickly. Thus, in general terms and under the same write load, a 512 GB SSD will last about half as long as a 1 TB SSD.
All other things aren’t equal, though, and that’s where wear levelling and Trim come into play. Without levelling the number of erase-write cycles across all the memory in an SSD, some would reach their limit far sooner than others. To tackle that, SSDs incorporate mechanisms to even out the use of individual memory cells, as wear levelling. The less free space available on an SSD, the less effective wear levelling can be, giving larger SSDs a significant advantage if they also have more free space.
Trimming is performed periodically to allow storage that has already been made available for reuse, for example when a file has been deleted, to be erased and made ready. Both APFS and HFS+ will Trim compatible SSDs when mounting a volume, but Trim support for external SSDs is only provided by default for those with NVMe interfaces, not SATA, and isn’t available for other file systems including ExFAT. Some SSDs may still be able to process available storage in their routine housekeeping, but others won’t. Without Trimming, an SSD gradually fills with unused memory waiting to be erased, and will steadily grind to a halt, with write speeds falling to about 10% of new.
Thus, to ensure optimum performance and working life, SSDs should be as large as possible, with much of their storage kept free. Experience suggests that a healthy amount of free space is 20-50% of their capacity.
Striking the best compromise
Apple silicon Macs work best and fastest when largely running from their internal SSDs. By all means reduce the capacity required by moving more static media libraries, and possibly large working folders, to an external SSD. But there’s no escaping the evidence that your Mac will work best and longest when its internal storage has a minimum of 20% free at all times, and you must ensure that never falls below 50 GB free space. Finally, consider your needs not today, but when you intend replacing that Mac in 3-5 years time, or any savings made now will prove a false economy.
In the elegant simplicity of Classic Mac OS, applications were intended to be single files with much of their contents stored in their resource fork, where they kept all the paraphernalia they required, from code resources to windows, icons and dialogs.
This is QuarkXPress version 4.11 from around 2000, its resources displayed in the resource editor ResEdit. The app icons shown are stored in a resource of type BNDL, a ‘bundle’, but not in the later sense of the term.
In this respect, as in many others, Mac OS X abandoned its Mac past in favour of what had been developed in NeXTSTEP, here a structured directory termed a bundle. These were primarily used for constructs that the user was intended to see as a single item, usually containing executable code and its resources, forming applications, frameworks and plug-ins.
Early bundles
At first, there were two types of bundle in use. Versioned bundles consisted of a file containing executable code, together with a directory containing resources and more directories with further resources, without any Contents directory or top-level property list. New-style bundles contained a minimum of a top-level directory named Contents, and inside that a property list named Info.plist, as continues today. Some implementations of the new-style bundle also contained a file named PkgInfo containing type and creator information that had originated in Classic Mac OS; although apparently not mandatory, PkgInfo has survived to the present in app bundles.
The information property list Info.plist contained key-value pairs in XML format, for attributes such as:
bundle name,
name of bundle executable,
version,
type and creator codes, inherited from Classic Mac OS and duplicating those in PkgInfo,
app and document icons,
other metadata.
Current bundles are direct descendants of those new-style bundles.
Fat apps with support for both Classic Mac OS and Mac OS X had a more complex structure. Inside their Contents directory, they had the following:
MacOSClassic directory, containing the Classic version of their executable code,
MacOS directory, containing the Mac OS X version of their executable code,
Info.plist file,
Resources directory, containing resources such as icons, images, and directories of localised resources such as nibs and strings,
Other directories such as Frameworks, PlugIns, SharedFrameworks and SharedSupport as necessary.
To enable that to run in Mac OS 9, alongside the Contents directory was an alias to the executable code in Contents/MacOSClassic.
The Finder recognised a directory as a bundle and treated it as a single entity if:
the directory name had an extension for a standard bundle type, such as .app, .bundle, .framework, .plugin, .kext and others, or
the directory name had an extension identifying it as a type set as a bundle (or package), such as .rtfd, or
the directory had its Finder bundle bit set.
Frameworks are an exception to the general rule that the Finder displays bundles as single items, as they are still shown as folders. However, the original reason given by Apple for this, “so that you can browse their header files”, no longer applies.
Packages
By 2005, Apple was drawing a distinction between bundles with the formal structure given above, including the Info.plist inside a Contents directory, and other directories presented to the user as a single item, such as RTFD documents, that had different structures and are strictly termed packages. For example, the Rich Text Format Directory (RTFD) favoured by Apple consists of:
Contents, a directory containing a PkgInfo file
files containing graphics to be included within the documents
TXT.rtf, a Rich Text Format file containing the styled text.
This is closer to the versioned bundles of early Mac OS X. Currently, there are distinct UTI types for com.apple.bundle and com.apple.package, with bundles conforming to both, but packages only to the latter. Unfortunately, this distinction between bundles and packages has largely fallen into disuse because of the name collision with Installer packages.
More recently, in its iWork documents, Apple has moved away from plain document packages or bundles to a custom package in a Zip-compressed archive, for example in .pages documents, which still conform to com.apple.package although they can’t be opened in the Finder. Compression has also become popular with space-inefficient formats such as XML.
Signatures
The introduction of code signing in 2007 brought the requirement to store signature information including code directory hashes (CDHashes) inside the app bundle. This has been implemented using an additional directory, _CodeSignature, containing a CodeResources file. Single-file command tools can instead have their signature embedded in their binary.
Most recently, in 2018, notarization has added the requirement to accommodate its ‘ticket’, which is ‘stapled’ as another CodeResources file inside the Contents directory. However, stapling isn’t required, as notarization can be verified by looking up the CDHashes with Apple’s online service in iCloud. There’s also still no way to staple a notarization ticket to a single-file binary.
Platform-specific code
The transition from Classic Mac OS to Mac OS X used separate directories for executable code within app bundles, with that named MacOS containing code to be run in Mac OS X. When the Mac architecture changed from PowerPC to Intel, from 32-bit to 64-bit, and again from Intel to Arm, app bundles have continued to use the MacOS directory, with executable code for each supported architecture being saved within each file.
Key dates
1984 single-file application with resource fork, type and creator codes
2001 Mac OS X 10.0 Cheetah, with versioned and new-style bundles, PkgInfo for type and creator codes, Info.plist in new-style bundles
2005 bundles and packages
2007 Mac OS X 10.5 Leopard, _CodeSignature directory for code signatures
2018 macOS 10.14 Mojave, CodeResources for notarization ticket.
References
Apple’s Bundle Programming Guide, last updated to 2017
Apple, Inside Mac OS X, System Overview, July 2002, PDF no longer available online.
If you have an Apple device, you’ll be familiar with the idea behind Storage Settings in System Settings > General. What you might not be prepared for is what you’ll see there. Like its equivalent in the General section of the Settings app, the bar chart at the top often shows that much of your Mac’s storage is filled with System Data, and it neither breaks that down into anything more meaningful, nor does it offer a tool to do anything about it.
Storage Settings on macOS has a troubled history. It used to be part of About This Mac, and often took so long to complete its bar chart that folk gave up waiting. In some cases, it even crashed in the process. More often than not, the totals it gave for used and free space on the startup disk were substantially different from those reported by the Finder or Disk Utility.
In recent versions of macOS, Storage Settings has improved steadily. It does now complete its analysis in a reasonable period, and in most cases its figures aren’t too different from Disk Utility’s. But System Data remains a puzzle that confuses rather than enlightens.
What is System Data?
From the information given by Apple, it’s most likely that this isn’t really what we’d consider to be data used or required by macOS, but the often substantial difference between how much space is used, and how much can be attributed to other categories, like Books and Music. It’s not a real category, but an etcetera, a ragbag including all sorts of files and other data.
Calculating some of the other categories seems difficult enough. For example, my Music folder contains over 100 GB of individual audio files and my Music library, but Storage Settings recognises less than 50 GB of that in its Music category. The remaining 50 GB has to be accounted for elsewhere, and that’s likely to be buried in System Data. With Photos and TV it’s the other way around, with almost all my libraries stored on an external SSD, but Storage Settings still claims I have 36 GB in TV and doesn’t even mention Photos.
So the categories it does list don’t always match with what we know is on the disk. When it adds all those together and takes that total away from the amount of disk space used, that difference is little more than a guesstimate, and unlikely to contain much data required by macOS. No wonder Storage Settings can’t suggest any way to reduce that size.
Dynamic storage
Unlike more traditional file systems, APFS is dynamic in its use of storage space, and macOS now uses aggressive caching policies. One of those came to light when we thought the Finder had a large memory leak, only to be told that, in the right circumstances, it can retain GB of Quick Look thumbnails in memory, so that scrolling through them can be smooth. When there’s sufficient free disk space, macOS can maintain large caches without using any swap space on disk.
APFS features like snapshots can retain data we presumed had been deleted, but has to be kept until the snapshot referencing it has been deleted up to 24 hours later. That space has to be accounted for somewhere, and in many cases that too goes into the System Data category, although it has actually been created by Time Machine, which oddly doesn’t have its own category.
Although not given in the Storage bar chart, both the Finder and Disk Utility should state the amount of purgeable space in use. This could be freed when necessary, but that’s determined by macOS rather than the user. I have previously looked at how purgeable some features like snapshots are, and in practice you shouldn’t rely on their automatic removal when you think your Mac needs more free space.
What is Storage Settings good for?
Like its equivalent on iPhones and iPads, some of the tools it provides are of great value in housekeeping. Click on the ⓘ Info button for each of them to get further information and for actions you can take. Categories with the more useful tools include:
Applications, shows apps by size, and which are Intel-only, and duplicates;
Developer, if you have Xcode installed, can clean up build files and device support;
Documents, lists by size and type, 32-bit apps;
Messages, lists larger attachments and lets you delete them;
Music Creation, helps you remove sound libraries;
macOS, tells you how much space is used by Apple Intelligence.
Although Music might seem promising, it’s only interested in music video files you can remove.
How to check free space
Don’t trust Storage Settings or the Finder to provide an accurate estimate of free space available on disk. Instead, open Disk Utility (with Show All Devices selected in its view) and in the list at the left in its main window, pick the volume you’re interested in, then go up to its container in the list and select that. Free space shown there is the most accurate estimate of that available to all volumes within that container, as in APFS volumes within the same container all share the same free space.
Select one of those volumes, and you’ll see free space given as a higher value, with a figure for the space that’s purgeable in brackets. That represents the maximum space that could be freed if all purgeable contents were to be deleted.
For example, free space shown here for a container is 619.72 GB, which is the space available without any purging. One volume within that container is given as having 864.5 GB available, with 244.78 GB purgeable:
864.5 = 619.72 + 244.78 GB
Figures given by the Finder are only refreshed periodically, while Disk Utility should recalculate them whenever you select a volume or container, so should always be up to date.
Summary
System Data in Storage Settings isn’t just files and data for macOS, but everything not listed in another category, and even that is only approximate, a rough estimate.
Storage Settings has useful tools for managing the contents of your startup disk.
For an accurate estimate of free space, use Disk Utility, and the free space given there for the volume’s container.
Don’t expect macOS to free up any purgeable space for you.
Across much of the temperate parts of the northern hemisphere, this is the wettest part of the year. It’s when puddles are everywhere, and what used to be firm ground turns into soft deep mud. Footpaths and bridleways become deep tracts of mud, impassable in anything but high boots. Yet look through paintings of winter and you’ll notice that few artists before 1800 have depicted people, vehicles or animals in mud of any significant depth. This weekend I look at some of the more faithful accounts of this ingloriously muddy time of year.
Louis-Léopold Boilly (1761–1845), Passer Payez (Pay to Pass) (c 1803), further details not known. Wikimedia Commons.
In the early nineteenth century, streets in major cities in Europe including Paris spent much of the winter as muddy morasses. Enterprising poorer inhabitants took long planks to locations where the more affluent would try to cross those rivers of mud, and hired them out to enable the rich to stay cleaner.
This is shown well in Louis-Léopold Boilly’s Passer Payez, or Pay to Pass, from about 1803, where a whole family is taking advantage of one of these crossings. This spared their footwear and clothing the otherwise inevitable coating of mud. As you can see, their shoes, lower legs and clothing are amazingly clean, as if they might actually have been painted in Boilly’s studio rather than the muddy streets of Paris.
Adolph von Menzel (1815–1905), Hussars Rescue a Polish Family (1850), paper, 34.5 x 47 cm, Museum Georg Schäfer, Schweinfurt, Germany. Wikimedia Commons.
As realism and real-world scenes became more popular in the middle of the nineteenth century, Adolph von Menzel showed a more accurate view of the problem of muddy roads in his Hussars Rescue a Polish Family from 1850. It had clearly been a wet autumn, with the leaves still burning red and gold on the trees in the background. These mounted soldiers are helping the elderly women from their carriage across the muddy ruts of the road. The hussar in the foreground, with his back to the viewer, even has mud on his riding boots.
Jean-Léon Gérôme (1824–1904), The Death of Marshal Ney (1868), oil on canvas, 64.1 x 104.1 cm, Sheffield Gallery, Sheffield, England. Photo from Militärhistoria 4/2015, via Wikimedia Commons.
One of the first artists to have used mud in a more meaningful way is Jean-Léon Gérôme, in his 1868 painting of The Death of Marshal Ney. Michel Ney (1769-1815) was a leading military commander during the French Revolutionary and Napoleonic Wars, and was made a Marshal of France by Napoleon. Following Napoleon’s defeat and exile in the summer of 1815, Ney was arrested, and tried for treason by the Chamber of Peers. He was found guilty, and executed by firing squad near the Luxembourg Gardens in Paris on 7 December 1815.
Gérôme shows Ney’s body abandoned after the execution, slumped face down and lifeless in the mud, his top hat resting apart at the right edge of the canvas. The firing squad is being marched off, to the left and into the distance. The mud only reinforces Gérôme’s powerful image of a cold, bleak, heartless execution.
Ludwig Knaus (1829–1910), Mud Pies (1873), oil on canvas, 64.4 x 109.4 cm, Walters Art Museum, Baltimore, MD. Wikimedia Commons.
Mud also has its recreational uses, as children of all eras will attest. Ludwig Knaus’s painting of Mud Pies from 1873 shows a group of children in the evening, near Dusseldorf, Germany, who are enjoying play in and with the mud, which is less fun for the swineherd behind them.
Giuseppe De Nittis (1846–1884), The Victoria Embankment, London (1875), other details not known. Wikimedia Commons.
While other Impressionists had been exploring the effects of transient light on the River Thames, in 1875, Giuseppe De Nittis examined the city’s muddy and rutted streets, in his painting of The Victoria Embankment, London. This wasn’t one of the older roads in the city either: the Victoria Embankment wasn’t constructed until 1865, and had only opened to traffic five years before De Nittis painted it.
John Atkinson Grimshaw (1836–1893), At The Park Gate (1878), oil on canvas, 51 x 61 cm, Private collection. Wikimedia Commons.
Muddy roads in northern British cities like Leeds were one of the favourite settings for the nocturnes of John Atkinson Grimshaw. At The Park Gate from 1878 (above) and November from 1879 (below) are glistening examples.
John Atkinson Grimshaw (1836–1893), November (1879), oil on canvas, 76.2 x 62.9 cm, location not known. Wikimedia Commons.Benjamin Williams Leader (1831–1923), February Fill Dyke (1881), oil on canvas, dimensions not known, Birmingham Museums Trust, Birmingham, England. Wikimedia Commons.
There’s an old English proverb “February fill dyke, be it black or be it white”, referring to the rain (black) or snow (white) that usually falls heavily during the month and fills all the ditches. Benjamin Williams Leader borrows that in his February Fill Dyke showing the waterlogged countryside near Worcester in 1881.
Mud became a favourite effect in the Naturalist paintings made so popular in France by Jules Bastien-Lepage.
Jules Bastien-Lepage (1848–1884), Pas Mèche (Nothing Doing) (1882), oil on canvas, 132.1 x 89.5 cm, Scottish National Gallery, Edinburgh, Scotland. Wikimedia Commons.
Pas Mèche (Nothing Doing) (1882) shows a cheeky ploughboy equipped with his whip and horn, on his way out to work in the fields. His face is grubby, his clothing frayed, patched, and dirty, and his boots caked in mud.
But for real mud, deep enough for wheels and legs to sink in and cake clothing, I turn to central and eastern Europe.
Jakub Schikaneder (1855–1924), The Sad Way (1886), oil on canvas, 141 × 217 cm, Národní galerie v Praze, Prague, The Czech Republic. Image by Ophelia2, via Wikimedia Commons.
Jakub Schikaneder’s The Sad Way from 1886 shows a single weary horse towing a cart on which a coffin rests. The woman, presumably widowed before her time, stares emptily at the rutted mud track, as a man walks beside them. It’s late autumn in a world that is barren, bleak, muddy and forlorn.
Józef Marian Chełmoński (1849–1914), Market (date not known), oil on canvas, 57.5 x 67.5 cm, Kościuszko Foundation, New York, NY. Wikimedia Commons.
Józef Marian Chełmoński’s undated Market is one of the most vivid insights into country life in Poland during the late nineteenth and early twentieth centuries. To reach this street market, carts are being drawn through a deep ditch full of muddy water. Market stalls are mounted on tables set in the mud, which forms the basis for everything.
Alfred Wierusz-Kowalski (1849–1915), Meeting the Train (date not known), oil on canvas, 19 x 23.5 cm, Private collection. Wikimedia Commons.
Also undated is contemporary and fellow Polish artist Alfred Wierusz-Kowalski’s Meeting the Train. A couple of horse-drawn carts have gone to a rural railway station to meet a train. The winter snow still covers much of the ground, except where it has been turned into rutted mud on the road.
Had there been opinion pollsters during the eighteenth and nineteenth centuries, no doubt we would have a good idea of the three most popular reasons for moving from the country to towns and cities. Evidence from paintings isn’t always reliable, and depends on the artist’s opinion. However, there are several good indications that I’ll consider here.
Perhaps the most detailed account appears in the many clues in William Hogarth’s opening image in his series A Harlot’s Progress. Sadly, some may have been lost when his original paintings were destroyed by fire in 1755, forcing us to rely on the engravings made of them.
William Hogarth (1697–1764), A Harlot’s Progress: 1 Ensnared by a Procuress (engraving 1732 after painting c 1731), engraving, 30.8 x 38.1 cm, British Museum, London. Wikimedia Commons.
Moll Hackabout, who is about to start her downfall as a harlot, is dressed in her Sunday best, with a fine bonnet and white dress to signify that she’s an innocent country girl. In Moll’s luggage is a symbolic dead goose, suggesting her death from gullibility. The address on a label attached to the dead goose reads “My lofing cosen in Tems Stret in London” (‘My loving cousin in Thames Street in London’), implying that Moll’s move to London has been arranged through intermediaries, who will have profited from her being trafficked into the hands of Elizabeth Needham, a notorious brothel-keeper and madame.
Hogarth thus presents Moll as a gullible victim of human trafficking to supply country-fresh prostitutes for London. Although he did paint a companion series to A Harlot’s Progress involving a man, he didn’t come from the country, so Hogarth sheds no light on the reasons for men and families moving from the country into cities, a theme that doesn’t appear to have been tackled in well-known paintings.
John Roddam Spencer Stanhope (1829–1908), Robin of Modern Times (1860), oil on canvas, 48.3 x 85.7 cm, Private collection. Wikimedia Commons.
John Roddam Spencer Stanhope’s Robin of Modern Times (1860) is a wide-angle composition set in the rolling countryside of southern England, during the summer. The foreground is filled with a young woman, who is asleep on a grassy bank, her legs akimbo. She wears cheap, bright red beads strung on a necklace, and a floral crown fashioned from daisies is in her right hand. She wears a deep blue dress, with a black cape over it, and the white lace of her petticoat appears just above her left knee. On her feet are bright red socks and black working/walking boots. A couple of small birds are by her, one a red-breasted robin, and there are two rosy apples near her face. In the middle distance, behind the woman’s head, white washing hangs to dry in a small copse. A farm labourer is working with horses in a field, and at the right is a distant farmhouse. This most probably refers to the continuing account of how girls and young women from the country around London found their way to the city to become its prostitutes.
There were other more prosaic reasons that younger people migrated. Among them was a long period of harsh weather, the Little Ice Age, that lasted until the late nineteenth century.
George Morland (1763–1804), Winter Landscape with Figures (c 1785), oil on canvas, 72.4 x 92.7 cm, Yale Center for British Art, New Haven, CT. Wikimedia Commons.
Many British winters featured deep and prolonged snow, even in the south of the country, as shown so well in George Morland’s Winter Landscape with Figures from about 1785. This period was hard enough in towns and cities, but many farms and villages in the country remained isolated for weeks at a time. Even when there wasn’t snow on the ground, there were prolonged droughts and widespread crop failure.
Over this period, staple crops were also changing, with the rising popularity of the potato. Just as rural populations were becoming dependent on the potato, they were struck by the mould causing blight, in 1845. Within a year, much of Europe was suffering failure of the potato crop, leading to death from starvation in about one million in the Great Irish Famine, together with fewer deaths in Scotland and the rest of Europe.
Jules Bastien-Lepage (1848–1884), October: Potato Gatherers (1878), oil on canvas, 180.7 x 196 cm, National Gallery of Victoria, Melbourne. Wikimedia Commons.
Jules Bastien-Lepage shows the autumn harvest in a more successful year in his October: Potato Gatherers from 1878.
Another sustained pressure on those living in the country was the effect of land enclosure from 1500-1900.
John Crome’s Mousehold Heath, Norwich (c 1818-20) shows the low rolling land to the north-east of the city that had remained open heath and common land until the late eighteenth century. By 1810, much of it had been enclosed and ploughed up for agriculture. Crome opposed the enclosure of common land, and here shows the rich flora and free grazing provided. In the right distance some of the newly created farmland is visible as a contrast.
Enclosure concentrated productive land in the hands of those who owned it, and locked out the poorer labourers who had been reliant on common grazing.
While Britain was spared, across much of Europe the eighteenth and nineteenth centuries saw a succession of wars fought largely in the country. Advancing and retreating armies seized what food they could from farms, and often burned and destroyed what they couldn’t remove.
Horace Vernet (1789–1863), The Campaign in France (1814), oil on canvas, dimensions not known, location not known. Wikimedia Commons.
Once Napoleon’s armies had been defeated at Leipzig in 1813, those of the other countries in Europe pursued them into north-east France, where there was a series of smaller battles and skirmishes fought in the French countryside, amid the terrified farmers and their families. Here Horace Vernet shows soldiers fighting around a burning farmhouse, as its occupants try to escape with little more than their lives. Their cattle are panicking, and it appears that the farmer himself has been shot, perhaps when trying to defend his family. A small boy buries his head into his mother’s apron.
As a result of those pressures, whole families abandoned their relatives, often those who were older and less likely to make a successful living in the city, and went to live in the growing towns and cities.
Erik Henningsen (1855–1930), Farmers in the Capital (1887), oil, further details not known. Wikimedia Commons.
Erik Henningsen’s painted record of Farmers in the Capital from 1887 is one of few contemporary accounts. This family group consists of an older man, the head of household, two younger women, and a young boy. Everyone else is wearing smart leather shoes or boots, but these four are still wearing filthy wooden clogs, with tattered and patched clothing. The two men are carrying a large chest containing the family’s worldly goods, and beside them is their farm dog. The father is speaking to a mounted policeman, presumably asking him for directions to their lodgings.
The large brick building in the background is the second version of Copenhagen’s main railway station, opened in 1864, and replaced by the modern station in 1911. This demonstrates another significant factor in the attraction of people to towns and cities: the spread of railways across Europe.
It might seem strange today, but for much of the early history of the Mac we used printed manuals and books. For developers the reference was Apple’s Inside Macintosh series, lovingly crafted by its technical authors using Microsoft Word and published by Addison-Wesley. Even Aperture (2005) and Apple’s Pro apps like DVD Studio Pro 3 (2004) came with beautifully prepared and printed manuals.
It was the introduction of Apple’s first CD-ROM drive in 1988 and its subsequent release of the first developer CD, Phil and Dave’s Excellent CD the following year that started change. At first there was a battle between nascent common document formats, including Adobe’s Acrobat, Farallon Replica, DjVu and others. In the early 1990s this led to HTML-based help books, and printed manuals became gradually replaced by electronic documentation.
Unix had a longer evolution, with the first of many man pages written for formatting using nroff in 1971. Although printed references were published later, deep piles of fanfold printout remained popular long after the rest of computing had discovered the value of desktop publishing. While much reference material was still confined to source code files, brief glimpses into how to use commands are still revealed in lightly formatted text man pages.
The release of Mac OS X 10.0 brought with it a primary help system, Apple Help, as part of Cocoa. Those creating help books wrote them in HTML saved in a folder structure like a local website, and a cut-down browser Help Viewer displayed them for the user. The only detailed account of this, the Apple Help Programming Guide, was first published in 2003, and last revised a decade later, 12 years ago.
Help Viewer started with a spartan interface, here in 2004, and relied primarily on pages stored in local help books.
Help books can contain localised versions to support their use worldwide.
Although seldom used, you could build your own custom help book, as shown in this demonstration I created for the UK magazine MacUser in 2006.
Apple reserved /Library/Documentation/Help and its sibling in the user’s Home library for its own use, and from OS X 10.6 third-party help books were expected to be stored as a .help bundle in the Resources folder in an app’s bundle. HelpViewer.app was hidden away in /System/Library/CoreServices, and helpd was an on-demand service to watch Applications and Utilities folders for newly-added apps containing help bundles, and register them with macOS.
By 2017, HelpViewer’s toolbar offered Back and Forward buttons, a button to hide or show the sidebar, and one to share the current page.
Searching help was aided by an index built by Help Indexer during authoring (below, from 2006), and used Spotlight’s local search powers.
Unfortunately, help became neglected and started to deteriorate. During macOS High Sierra 10.13 to 10.13.4, it was revised internally, and the way that helpd worked was improved.
The macOS help system had previously relied on the user (or an installer) adding an app bundle to one of its two watched folders, /Applications and /Applications/Utilities, to trigger the process of adding any new or updated help book to those available. The helpd service was activated by File System Events when such an event occurred, and responded by checking the app bundle for a help book to register it. If an app was installed in another folder, even ~/Applications, then its help book might not get registered properly.
Identification of help books also changed. Previously, that for the Dictionary app was known as com.apple.Dictionary.help, making it difficult to accommodate more than one version of an app and its help book at a time. The new system incorporated a version number, such as com.apple.Dictionary.help*2.2.2.
When originally developed, the great majority of help books relied on static content built into them. All that had to happen for most was the launch of HelpViewer, and for it to open the supplied Help book.
Since then, help books had increasingly relied on online content, which needed to be refreshed before each access. Thus HelpViewer and helpd had to work together to deliver the latest content, with helpd doing the refreshing and updating to the databases in ~/Library/Caches/com.apple.helpd, which HelpViewer used with the WebCore rendering engine to display pages and interact with the user.
Since then further bugs have come and gone; among the more troublesome were those in Big Sur and Monterey, that could prevent a Help book from opening, or, if it did eventually open, the book displayed completely blank pages.
It also had some more amusing moments, including a phase in which the help system decided that I wanted to read Apple Configurator’s help book in French.
For Ventura, Apple revised the help system further, and renamed HelpViewer to Tips while keeping its formal identifier as com.apple.helpviewer, and it remained hidden away in CoreServices. In Sequoia, it leapt from version 10 to 15 and joined Apple’s first league apps between Time Machine and TV in the main Applications folder. But if you want to read the documentation for any command tool, you still have to refer to its man page.
Login
