The first file system for Macintosh computers wasn’t HFS+ or even its predecessor HFS, but Macintosh File System, MFS. This was introduced in System 1 on the 128K Mac just over 41 years ago, to support its 400 KB floppy disks. Although it was fairly primitive, it incorporated some visionary features, including forks. Each file had two sets of data: a data fork as in other file systems, and a resource fork for storing structured blobs of data or resources.

File naming was liberal compared with MS-DOS, allowing names up to 255 characters long, although that was restricted to 63 by the Finder. Names could consist of any printable character except the colon :, a limitation that persists in the Finder today. As there was no directory hierarchy, folders were an illusion and couldn’t be created directly by the user. Instead there was always an Empty Folder available, and when that was used, a fresh Empty Folder was created. As this was a single-user file system, there were no permissions.

MFS was still supported until it was finally discontinued 13 years later in System 8, in 1997.

Hierarchical File System, HFS

MFS had been designed for the low-capacity floppy disks of the time, and not for use on hard disks, where its limitations would have been only too apparent. For the release of the Macintosh Hard Disk 20 in September 1985, and in anticipation of the Macintosh SE 18 months later, a new Hierarchical File System had to be released to replace MFS in System 2.1. HFS remained fully supported until the arrival of Mac OS X 10.6 Snow Leopard in 2009, and finally dropped altogether in macOS Catalina a decade later.

Developed by Patrick Dirks and Bill Bruffey, HFS maintained many of the novel features in MFS, with resource forks, long file names up to a maximum of 31 characters, still excluding the colon, and in its standard single-user version didn’t support permissions. The latter were incorporated into AppleShare later. File and folder names were case-preserving but case-insensitive.

Larger storage capacities brought the need for a hierarchical directory structure, implemented using B-trees in a Catalog File that made the display of even large directories very quick. Although much of HFS used 32-bit integers, that didn’t apply to the number of files in a logical disk, which was limited to 65,535, which must have seemed sufficient at the time, and given the maximum volume size of 2 TB. With early hard disks being measured in tens of MB, that may have seemed in the distant future.

Mac OS Extended, HFS+

With the growth in capacity of hard disks, HFS had to be updated to address its limitations, in a project with the internal name of Sequoia, delivering HFS+ in Mac OS 8.1 in 1998. Switching to 32-bit fields to identify allocation blocks allowed more efficient use to be made of storage and a larger number of files in each volume. File names were increased in maximum length to 255 characters, and changed from MacRoman encoding to Unicode UTF-16 to accommodate a broader range of languages. Support for additional forks beyond data and resource paved the way for the switch to extended attributes, and OS startup support was improved to allow alien operating systems to boot from HFS+ volumes.

This screenshot shows a set of custom icons in a BNDL resource, in the QuarkXPress app in about 2000.

This shows file information available in HFS+ in Classic Mac OS in 2002.

HFS+ and its predecessors were prone to develop errors as a result of operating system crashes and other unexpected events, and those could be cumulative, leading to data loss. This was addressed with the introduction of journalling, designed and implemented by Dominic Giampaolo, who came to Apple from implementing the file system for BeOS. This was tentatively introduced as an option in Mac OS X 10.2.2 in late 2002, and made a standard feature in 10.3 the following October. Alongside that came an optional wrapper for case-sensitivity in what was dubbed HFSX, and a change in Unicode decomposition to Normalisation Form D (NFD).

Mac OS X 10.4 augmented Posix permissions with Access Control Lists (ACLs), although they were little-used outside server environments for some years. Prior to 10.5, as with most other file systems, HFS+ supported file but not directory hard-links. With the introduction of Time Machine in 10.5 Leopard, directory hard-links were added to support the structure and illusions of Time Machine backup stores.

File system support for encryption was a bit more troubled. The original FileVault, introduced in 2003 with Mac OS X 10.3 Panther, located user Home folders in an encrypted sparse disk image, which was improved in 10.5 by moving to sparse bundles. This suffered several shortcomings and vulnerabilities, and was replaced by whole-volume encryption in FileVault 2 in Mac OS X 10.7 Lion. That required the addition of a logical volume manager, Core Storage, which was then used for Fusion Drives introduced in 2012.

Apple File System, APFS

HFS+ had been designed for computers with hard disks. It lacks some of the features of more modern file systems such as snapshots, special files such as sparse files, and concurrent access. It’s also not well-suited to use with SSDs and storage in smaller, mobile devices, although when the first iPhone shipped with iOS 1.0 in 2007, it used HFSX, the case-sensitive variant of HFS+. That was until the release of iOS 10.3 on 27 March 2017, which silently converted its file system to APFS.

In 2014, Apple had decided to write its own file system from scratch, and Dominic Giampaolo, responsible for journalling in HFS+, and Mike Mackovitch became its lead engineers. APFS was announced two years later at WWDC in 2016, when it was expected to be released in another 18 months if development and testing went smoothly. Those who had hoped for ZFS were disappointed and many remain so today. macOS Sierra already had a pre-release version for those who wanted to preview it, but as we discovered when we upgraded to High Sierra, that was a far cry from what was to come.

After a promising period in beta, Apple discovered fundamental problems between APFS and its popular Fusion Drives. The first release of macOS 10.13 shipped with APFS version 748.1.46, but abruptly dropped support for those, so converted only those startup volumes on SSDs and hard disks. Snapshots were wobbly at first, and it quickly became clear that APFS was never going to perform well on rotating disks.

High Sierra had a stormy early release history, marred by a series of security gaffes. Vulnerabilities were fixed in the Supplemental Update released less than two weeks after 10.13, leaving snapshots to be improved in 10.13.1 on 31 October. Many expected problems with Fusion Drives would be fixed quickly, but those weren’t ready for release until the following September. Another problem that troubled the introduction of APFS to all platforms was the refusal during beta-testing to incorporate Unicode normalisation; this had to be resolved in later versions of macOS 10.13 and iOS 10, as explained here and here.

In September 2018, Apple at last released Mojave 10.14 with support for Fusion Drives, accompanied by the first version of the Apple File System Reference. Although a long and detailed document, developers soon realised how incomplete it was, in spite of the long delay in its publication. At last third-party file system developers had some hard information to work with, and users started assuming that third-party disk maintenance and repair tools were imminent.

Catalina brought major changes to APFS, with the use of expanded volume roles to form System Volume Groups, with their separate but firmlinked System and Data volumes. macOS 10.15.5 fixed a serious bug preventing the transfer of very large amounts of data to RAID volumes. At that time, Apple released an updated version of the Apple File System Reference, building expectations that third-party tools were just round the corner, at least among those who weren’t aware of how much information was still missing. Nearly five years later, it’s still that same edition dated 22 June 2020 that remains the latest information released by Apple about APFS.

Further major changes came with Big Sur 11.0.1 when it was released in November 2020, introducing the sealed and signed snapshot now used to boot macOS. This was also the first release to support making Time Machine backups to APFS volumes, and to support Apple silicon Macs.

Although Apple dropped early hints that APFS might be released as open source, unlike its predecessors, after eight years, information about its internals released by Apple still appears to be insufficient to allow third-party developers to create maintenance tools independent of those bundled in macOS. This reluctance may stem from the deep involvement between the file system and macOS security.

Summary timeline

• MFS Jan 1984 – Sep 1985, end of support 1997
• HFS Sep 1985 – Jan 1988, end of support 2019
• HFS+ Jan 1988 – present, still supported
• APFS Sep 2017 (iOS March) – present, still supported

References

Inside Macintosh: Files, Chapter 2 – File Manager, on HFS
MacCyclopedia critical account of HFS and HFS+. 25 May 2003
Apple TN1150 HFS Plus Volume Format, 5 March 2004
Apple’s APFS Reference (PDF), last revised 22 June 2020.

Wikipedia on
MFS
HFS
HFS+

In the last couple of weeks I’ve been asked to help recover data lost when files have been accidentally deleted, and an internal SSD has been wiped remotely using Find My Mac. What we perhaps haven’t fully appreciated is how improved security protection in our Macs has made it far harder, if not impossible, to recover such lost data. Allow me to explain in three scenarios.

Lost files on a hard disk

When files are deleted from a hard disk, the file system marks them as no longer being in use, and they’re left in place on the hard disk until they need to be overwritten with fresh data. If the hard disk has ample free space, that could occur days, weeks or even months later. Data recovery software and services can be used to scan each storage block and try to reconstruct the original files. If the file system and its data are encrypted, the encryption key is required to enable the contents to be decrypted.

There’s extensive experience in such data recovery, and provided the disk isn’t physically damaged or malfunctioning, results can be surprisingly good. As services charge according to the amount of data they recover, there are also strong incentives.

This works both ways, of course, in that someone who gets access to that hard disk could also recover files from it if they’re unencrypted. For this reason, when you’re passing on or disposing of a hard disk, you should perform a secure erase to overwrite its entire contents. If it’s going for recycling, once that has been done, you should also render the disk unusable by physically damaging its platters.

Deleted files on an SSD

What happens on an SSD depends on whether there’s already a snapshot of that volume. If there is, and that snapshot includes the deleted files, the file system metadata for them is retained in that snapshot, and the storage containing their data is also retained. The files can then be recovered by mounting that snapshot and either reverting the whole volume to that earlier state, or copying those files to a different volume.

If there’s no prior snapshot containing the files, the file system marks their extents as being free for reuse. At some time after their deletion, that information is sent to the SSD in a Trim command. When the SSD next has a moment to perform its routine housekeeping, the physical storage used will then be erased ready to be written to again.

Although there’s some uncertainty as to when that Trim command will be sent to the SSD, one time that we know that supported SSDs are Trimmed is during mounting, in the case of an internal SSD when that Mac starts up. So if your Mac has started up since the files were deleted, those files are most likely to have been completely erased from its internal SSD. With their erasure, chances of ever recovering those files have gone.

Wiped Data volume

Macs with T2 or Apple silicon chips have an ingenious method of ‘wiping’ the entire contents of the Data volume when it’s encrypted on the internal SSD. This can be triggered using the Erase All Content and Settings (EACAS) feature in the Transfer or Reset item in General settings, or remotely via Find My Mac. Either way, this destroys the ‘effaceable key’ and the ability to decrypt the contents of the Data volume, even if it’s not additionally protected by FileVault. As Apple states: “Erasing the key in this manner renders all files cryptographically inaccessible.”

This is to ensure that if your Mac is stolen, no one can recover the contents of its internal SSD once it has been wiped in this way. Nearly a year ago there were claims that old data could re-appear afterwards, but those turned out to be false.

I’m afraid that the only way to recover the data from a volume wiped using EACAS or Find My Mac is to restore it from a backup.

Backups are more important

For Intel Macs with T2 chips, and Apple silicon Macs, the chances of being able to recover files from their internal SSDs have become diminishingly small. This makes it all the more important that you make and keep good and comprehensive backups of everything in your Mac’s Data volume.

I’m always sad to hear of those who have suffered data loss, and shocked to learn of how many still don’t keep backups.