We back up to ensure that we can recover files, whole volumes, our complete Mac if needed. When that crucial document you were working on earlier has vanished, or becomes damaged, or disaster strikes a disk, backups are essential. But how do you preserve all those documents that used to come on paper, records, correspondence and certificates? How will you or your successors be able to retrieve them in ten or thirty years time? This brief article considers how you should archive them safely, which isn’t the same as backing them up.

By archiving, I mean putting precious files somewhere they can be retrieved in at least ten years time. They may include financial, business, employment and personal records, as well as all finished work that you want to record for posterity. For most, they’ll also include a careful selection of still images, movies, and the more important documents you might create, such as books, theses and papers. They’re what you and the law want you to keep in perpetuity, and to be able to retrieve even after you’re gone.

To see how this can be achieved, I consider: the storage medium to be used, file formats that will be retrievable, how to index them for access, physical storage conditions, and the checks of their integrity that are needed.

Storage medium

While backups are most likely to be kept on hard disks or SSDs, neither of those is in the least suitable for archives, as they have relatively short lifetimes and are too sensitive to storage conditions. Instead, you need a removable medium, today probably Blu-ray disks intended for archival use, such as M-DISC.

For those with copious archives of importance beyond their family, Sony used to offer Optical Disk Archive systems, but those products were discontinued last year and don’t appear to have a suitable replacement. This illustrates one of the problems with planning for the more distant future: today’s technology can all too easily become orphaned.

Businesses are increasingly turning to cloud services to store their archives, but for the great majority of us the recurring cost makes this impractical. In any case, best practice should be to use cloud services as a supplement to a physical archive. iCloud is more affordable for the storage of most important documents, but requires a Legacy Contact to be appointed.

File formats

While it’s fine to archive documents in their original format, as you do in your backups, it’s also important to extract their contents into more permanent formats. Among those most likely to prove durable for the next 50-100 years are:

• UTF-8 (and formerly ASCII) for text files,
• JPEG and PNG for still images,
• audio, video and rich media using one of the widely-used compression standards and file formats,
• XML-based open document standards,
• CSV for data,
• PDF provided that it complies with one of the archival standards PDF/A-1 to /A-4.

You may find it worthwhile tarring together large collections of smaller files, but don’t use an unusual compression or ‘archive’ format, which might prove inaccessible in the future.

Indexing and access

For larger collections, even when structured carefully, a thorough list of contents in UTF-8 text format is essential. While there are index and search tools that could help, in this respect too archives are different from backups. If you’re going to be gathering TB of files, look at some of the commercial solutions. Although some are free to use, like the long-established Greenstone, they aren’t intended for casual users and might prove demanding.

Physical storage conditions

Never print on the disk itself, which can result in its degradation, and keep paper records alongside disks in the same container, but not inside the cases themselves, where they could damage them.

Archive optical disks should be stored in cases with centre hub security, not in sleeves. They must be kept in a cool, dry and dark container, in which there is no mould or fungus. They also need to be protected from physical threats such as flood and fire. Firesafes are popular furniture for this, but you must then ensure that their combination or keys are readily available and not separated from the safe.

There used to be a vogue for commercial data repositories, often underground storage sites that had been repurposed. Not only were those expensive, but many failed to take the care that they promised, and plenty went bankrupt and put their contents at risk. If you can arrange it, store one copy with you, and another at a friend’s or relative’s at least a few miles away.

Integrity checks

If you’re serious about maintaining your archives, some form of integrity checking, such as that provided by my free utilities Dintch, Fintch and cintch, is essential. Check a sample on each disk once a year, to ensure that none has started to deteriorate. If you do detect errors, that’s the time to burn a replacement before the original is lost to decay.

Conclusion

Backups are for recovery, while archives are for posterity. Start building your archives now, and keep them safe for the future.

Further reading

How to burn a Blu-ray disc in Monterey
Wikipedia point of entry

Postscript

Some of you are reporting widespread claims that some Blu-ray burners no longer work in Sequoia. I have therefore repeated the process that I described in Monterey, using exactly the same Pioneer burner connected to a Mac Studio M1 Max running macOS 15.1. I’m delighted to report that it still works perfectly, and I see no reason that any other recent Pioneer optical drive should prove incompatible. All you need to do is follow the instructions.

Happy archiving!

Time Machine (TM) has evolved to be a good general-purpose backup utility that makes best use of APFS backup storage. However, it does have some quirks, and offers limited controls, that can make it tricky to use with more complex setups. Over the last few weeks I’ve had several questions from those trying to use TM in more demanding circumstances. This article explains how you can design volume layout and backup exclusions for the most efficient backups in such cases.

How TM backs up

To decide how to solve these problems, it’s essential to understand how TM makes an automatic backup. In other articles here I have provided full details, so here I’ll outline the major steps and how they link to efficiency.

At the start of each automatic backup, TM checks to see if it’s rotating backups across more than one backup store. This is an unusual but potentially invaluable feature that can be used when you make backups in multiple locations, or want added redundancy with two or more backup stores.

Having selected the backup destination, it removes any local snapshots from the volumes to be backed up that were made more than 24 hours ago. It then creates a fresh snapshot on each of those volumes. I’ll consider these later.

Current versions of TM normally don’t use those local snapshots to work out what needs to be backed up from each volume, but (after the initial full backup) should rely on that volume’s record of changes to its file system, FSEvents. These observe two lists of exclusions: those fixed by TM and macOS, including the hidden version database on each volume and recognised temporary files, and those set by the user in TM settings. Among the latter should be any very large bundles and folders containing huge numbers of small files, such as the Xcode app, as they will back up exceedingly slowly even to fast local backup storage, and can tie up a network backup for many hours. It’s faster to reinstall Xcode rather than restore it from a backup.

Current TM backups are highly efficient, as TM can copy just the blocks that have changed; older versions of TM backing up to HFS+ could only copy whole files. However, that can be impaired by apps that rewrite the whole of each large file when saving. Because the backup is being made to APFS, TM ensures that any sparse files are preserved, and handles clone files as efficiently as possible.

Once the backup has been written, TM then maintains old backups, to retain:

• hourly backups for the last 24 hours, to accompany hourly local snapshots,
• daily backups over the previous month,
• weekly backups stretching back to the start of the current backup series.

These are summarised in the diagram below.

Local snapshots

TM makes two types of snapshot: on each volume it’s set to back up, it makes a local snapshot immediately before each backup, then deletes that after 24 hours; on the backup storage, it turns each backup into a snapshot from which you can restore backed up files, and those are retained as stated above.

APFS snapshots, including TM local snapshots, include the whole of a volume, without any exceptions or exclusions, which can have surprising effects. For example, a TM exclusion list might block backing up of large virtual machine files resulting in typical backups only requiring 1-2 GB of backup storage, but because those VMs change a lot, each local snapshot could require 25 GB or more of space on the volume being backed up. One way to assess this is to check through each volume’s TM exclusion list and assess whether items being excluded are likely to change much. If they are, then they should be moved to a separate volume that isn’t backed up by TM, thus won’t have hourly snapshots.

Some workflows and apps generate very large working files that you may not want to clutter up either TM backups or local snapshots. Many apps designed to work with such large files provide options to relocate the folders used to store static libraries and working files. If necessary, create a new volume that’s excluded completely from TM backups to ensure those libraries and working files aren’t included in snapshots or backups.

TM can’t run multiple backup configurations with different sets of exclusions, though. If you need to do that, for instance to make a single nightly backup of working files, then do so using a third-party utility in addition to your hourly TM backups.

This can make a huge difference to free space on volumes being backed up, as the size of each snapshot can be multiplied by 24 as TM will try to retain each hourly snapshot for the last 24 hours.

Macs that aren’t able to make backups every hour can also accrue large snapshots, as they may retain older snapshots, that will only grow larger over time as that volume changes from the time that snapshot was made.

While snapshots are a useful feature of TM, the user has no control over them, and can’t shorten their period of retention or turn them off altogether. Third-party backup utilities like Carbon Copy Cloner can, and may be more suitable when local snapshots can’t be managed more efficiently.

iCloud Drive

Like all backup utilities, TM can only back up files that are in iCloud Drive when they’re downloaded to local storage. Although some third-party utilities can work through your iCloud Drive files downloading them automatically as needed, TM can’t do that, and will only back up files that are downloaded at the time that it makes a backup.

There are two ways to ensure files stored in iCloud Drive will be backed up: either turn Optimise Mac Storage off (in Sonoma and later), or download the files you want backed up and ‘pin’ them to ensure they can’t be removed from local storage (in Sequoia). You can pin individual files or whole folders and their entire contents by selecting the item, Control-click for the contextual menu, and selecting the Keep Downloaded menu command.

Key points

• Rotate through 2 or more backup stores to handle different locations, or for redundancy.
• Back up APFS volumes to APFS backup storage.
• Exclude all non-essential files, and bundles containing large numbers of small files, such as Xcode.
• Watch for apps that make whole-file changes, thus increasing snapshot and backup size.
• Store large files on volumes not being backed up to minimise local snapshot size.
• If you need multiple backup settings, use a third-party utility in addition to TM.
• To ensure iCloud Drive files are backed up, either turn off Optimise Mac Storage (Sonoma and later), or pin essential files (Sequoia).

Further reading

Time Machine in Sonoma: strengths and weaknesses
Time Machine in Sonoma: how to work around its weaknesses
Understand and check Time Machine backups to APFS
Excluding folders and files from Time Machine, Spotlight, and iCloud Drive

APFS has many of the features of modern file systems that make most efficient use of space, and others that aren’t found in more traditional file systems like HFS+. While those should all work well when used locally with other APFS volumes, they can prove incompatible with other file systems, and may have untoward side effects. This article looks at those features that you could encounter problems with, and how best to work around them.

Sparse files

Most modern file systems store files that contain significant amounts of blank or absent data in a special sparse file format. In APFS, this is implemented by only allocating file extents to those blocks that do contain data. In many cases, this can save significant disk space. Initially, sparse files were unusual in APFS, but over time they have become increasingly common, and can now be found in some types of disk image, virtual machine storage, and in some databases. Note that disk image types whose name includes the word sparse, sparse bundles and sparse disk images, don’t use sparse file format at all, but are the victims of an unfortunate name collision.

Neither macOS nor APFS can simply convert regular files to sparse format; for a file to be written in sparse format, the code writing it must explicitly skip the empty space. As a result, sparse files are prone to explode to full size when they’re copied or moved unless that’s between two local volumes both using APFS. Examples of where you should expect them to explode include:

• copy or move to HFS+, as that has no sparse file format
• copy between Macs using AirDrop or file sharing
• back up to network storage, although local Time Machine backups to APFS should preserve them
• copy or move to any other local file system other than APFS, even if that file system has its own support for sparse files.

In each of those cases, sparse files explode in size as they’re copied from the source volume. If you have a 100 GB sparse file that only takes 20 GB of local disk space, when it’s copied over the network, or to a local HFS+ volume, the full 100 GB has to be transferred.

One potential workaround is to compress the sparse file before transferring it. All good compression algorithms will work efficiently on the blank space in the file, so when compressed its size could be as small as the original sparse file. However, when it’s decompressed, even on another APFS volume, it will explode to full size. For disk images, that can be corrected by mounting them, as APFS will then trim their contents, and the disk image should be saved back into sparse format.

Clone files

These are two distinct files that share common data, normally the result of duplicating the original within the same APFS volume. Those two cloned files then only require the storage of the whole file, plus those data blocks that differ between them. This only works within the same volume, and the moment that either of the clones is moved or copied to any other volume, it assumes full size, as it can no longer share data with the other clone.

However, most other file systems don’t support file cloning in this way. When you duplicate a file in an HFS+ volume, there’s no shared data between them, and the two require twice the amount of space as one does.

Snapshots

Snapshots consist of a complete copy of the volume at an instant in time, so require a copy of the file system metadata for that volume, and retain copies of storage blocks for each file as they are changed subsequent to the snapshot being made, so you could roll back that volume to its state at the moment of the snapshot.

Although Time Machine backups contain snapshots of the volume they back up, snapshots can’t normally be copied to another disk or volume. Some have been able to make a complete copy of a disk including its snapshots using the dd command tool, but that should be considered experimental. In all other circumstances, snapshots stay where they were made, but you can always copy from an existing snapshot to reconstitute a volume.

Directory hard links

These aren’t available in APFS, but are supported in HFS+, where they’re used extensively in Time Machine backups. They work like regular hard links, but act on directories rather than individual files. They can’t be copied in any way to an APFS volume, but can be used to reconstitute a volume.

Extended attributes

These are additional metadata associated with files and folders, and are fully supported in both APFS and HFS+. However, many are treated as being ephemeral, and may not be preserved during copying or other actions. The system of flags used to determine which are preserved is detailed in this article.

Several other file systems also support extended attributes, but copying between them is unlikely to transfer them between file systems. In some cases, extended attributes are preserved using AppleDouble format, in which each file can have a hidden shadow with its name prefixed with ._ (dot-underscore) characters. These are most often seen in FAT and ExFAT volumes, but are prone to confuse users of other computers.

Key points

• APFS sparse files are only preserved when copying or moving files between local APFS volumes. In other circumstances they explode to full size.
• Good compression methods can keep a sparse file to a similar size, but decompression explodes them to full size. Disk images may then be restored as sparse files after they have been mounted again from APFS.
• Clone files aren’t preserved when copied or moved to any other volume.
• Snapshots can’t be copied at all, although they can be reconstituted as volumes.
• APFS doesn’t support the directory hard links used in Time Machine backups to HFS+, but a backup can be reconstituted as a volume.
• Extended attributes are preserved between APFS and HFS+, but not with other file systems, except in the shadow files of AppleDouble format, seen in FAT and ExFAT volumes.

Over the last 17 years, Time Machine has backed up many millions of Macs, ranging from the last Power Macs to the latest M3 models, every hour. It has changed greatly over that time. This article uses my free utility T2M2 to explain how Time Machine now makes automatic backups to APFS storage, and how to check that. This account is based primarily on what happens in macOS Sonoma and Sequoia, when they’re backing up automatically every hour to local storage.

T2M2 offers three features to see what’s happening with Time Machine and backups:

• Check Time Machine button, to analyse backups made over the last few hours.
• Speed button, to view progress reports in the log during a long backup.
• Browse log button, to show filtered log extracts in fullest detail.

These are each detailed in T2M2’s Help book. Here I’ll concentrate on the first and last, and defer speed checks to that documentation.

Browse log for detail

In practice, you’re most likely to view T2M2’s analysis using the Check Time Machine button first, but here I’ll walk through the greater detail available in log extracts, to build understanding of the sequence of events in each automatic backup.

To help you see which subsystems are involved in each stage, T2M2 displays their entries in different colours, and you can show or hide each of those.

Automatic backups are called off by the DAS-CTS dispatching mechanism, whose entries are shown in red (DAS) and blue (CTS). They schedule backups so they don’t occur when the Mac is heavily loaded with other tasks, and call a chain of services to start the backup itself. Dispatching is now reliable over long time periods, but can be delayed or become irregular in some situations. Inspecting those DAS-CTS entries usually reveals the cause.

From there on, most informative log entries are made by Time Machine itself.

Preparations are to:

• Find each destination backup storage, decide whether any rotation scheme applies, so determine the destination for this backup.
• Check write performance to the backup destination.
• Find the machine store on the destination.
• Determine which local snapshots should be removed, and delete them.
• Create local snapshot(s) as ‘stable’ snapshot(s), and mount them.
• Mount previous local snapshot(s) as ‘reference’ snapshot(s).
• Determine how to compute what needs to be backed up from each source. This should normally use FSEvents to build the EventDatabase.
• Scan the volumes to be backed up to determine what needs to be backed up.
• Estimate the total size of the new backup to be created.

Once backing up starts, entries cover:

• Copying designated items to the destination.
• Posting periodic progress reports during longer backups.

When that’s complete, closing stages are to:

• Report details of the backup just completed.
• Set local snapshots ready for the next backup, with the ‘stable’ snapshot(s) marked as ‘reference’, and unmount local snapshots.
• Delete working folder used during the previous backup as ‘incomplete’.
• Create the destination backup snapshot.
• Delete any old backups due for removal.
• Report backup success or other outcome.

They’re summarised in this diagram. Although derived from Sonoma, Sequoia brings no substantial change.

Or its PDF tear-out version: tmbackup14b

Check Time Machine summaries

T2M2 analyses all those log entries to produce a summary of how Time Machine has performed over the last few hours. That is broken down into sections as follows.

Backup destination. This is given with the free space currently available on that volume, followed by the results of write speed measurements made before each backup starts.
Backing up 1 volumes to ThunderBay2 (/dev/disk10s1,TMBackupOptions(rawValue: 257)): /Volumes/ThunderBay2
Current free space on backup volumes:
✅ /Volumes/ThunderBay2 = 1.67 TB
Destination IO performance measured:
Wrote 1 50 MB file at 286.74 MB/s to "/Volumes/ThunderBay2" in 0.174 seconds
Concurrently wrote 500 4 KB files at 23.17 MB/s to "/Volumes/ThunderBay2" in 0.088 seconds

Backup summary. This should be self-evident.
Started 4 auto backup cycles, and 0 manual backups;
completed 4 volume backups successfully,
last backup completed successfully 53.4 minutes ago,
Times taken for each auto backup were 0.3, 0.3, 0.2, 0.2 minutes,
intervals between the start of each auto backup were 61.4, 60.2, 60.3 minutes.

Backups created and deleted (or ‘thinned’).
Created 4 new backups
Thinned:
Thinning 1 backups using age-based thinning, expected free space: 1.67 TB actual free space: 1.67 TB trigger 50 GB thin 83.33 GB dates: (
"2024-10-01-043437")

Local snapshots created and deleted.
Created 4 new snapshots, and deleted 4 old snapshots.

How the items to be backed up were determined. This shows which methods Time Machine used to work out which items needed to backed up, and should normally be FSEvents, once a first full backup has been made.
Of 4 volume backups:
0 were full first backups,
0 were deep scans,
4 used FSEvents,
0 used snapshot diffs,
0 used consistency scans,
0 used cached events.

Backup results. For each completed backup, Time Machine’s report on how many items were added, and their size.
Finished copying from volume "External1"
1 Total Items Added (l: Zero KB p: Zero KB)
3 Total Items Propagated (shallow) (l: Zero KB p: Zero KB)
403274 Total Items Propagated (recursive) (l: 224.93 GB p: 219.31 GB)
403275 Total Items in Backup (l: 224.93 GB p: 219.31 GB)
1 Directories Copied (l: Zero KB p: Zero KB)
2 Files Move Skipped (l: Zero KB p: Zero KB) | 2 items propagated (l: 6 KB p: 12 KB)
1 Directories Move Skipped (l: Zero KB p: Zero KB) | 403269 items propagated (l: 224.93 GB p: 219.31 GB)

Error messages.
✅ No error messages found.

iCloud Drive and pinning

Backing up the contents of iCloud Drive is a longstanding problem for Time Machine, if Optimise Mac Storage is enabled. Those files in iCloud Drive that are stored locally should be backed up, but any that have been evicted from local storage could only be included if they were to be downloaded prior to the backup starting.

In Sonoma and earlier, the only way to ensure that an evicted file in iCloud Drive is included in a Time Machine backup is to manually download it. Sequoia now lets you ‘pin’ individual files and whole folders so that they aren’t evicted, so will always be included in Time Machine backups. This is explained here.

Discrepancies and glitches

T2M2 tries to make sense from log entries that don’t always behave as expected. As backups can be complex, there are situations when T2M2 may report something that doesn’t quite add up. This most commonly occurs when there have been manual backups, or a third-party app has been used to control the scheduling and dispatch of backups. If you do see figures that don’t appear quite right, don’t assume that there’s something wrong with Time Machine or your backups. Generally, these glitches disappear from later automatic backups, so you might like to leave it for a few hours before checking Time Machine again to see if the problem has persisted.

Further reading

What performance should you get from different types of storage?
Where should you back up to?
How big a backup store do you need?
How Time Machine backs up iCloud Drive in Sonoma
Time Machine backing up different file systems
Excluding folders and files from Time Machine, Spotlight, and iCloud Drive
Snapshots aren’t backups
Time Machine in Sonoma: strengths and weaknesses
Time Machine in Sonoma: how to work around its weaknesses
Time Machine in Sonoma: Rotating backups and NAS
A brief history of Time Machine

macOS supports several different disk formats, each of which has its purposes. This article explains which to choose from those you can create by formatting a disk in Sequoia. macOS also supports some other formats like NTFS for reading only, which I won’t cover here.

Disk structure

Before any file system can be formatted on a disk, the storage in the disk must be partitioned using one of three standard schemes:

• GUID Partition Map (GPT), standard for most disks and file systems in macOS;
• Master Boot Record (MBR), formerly used for MS-DOS and Windows;
• Apple Partition Map, an old format used by Macs, and worth avoiding unless you know it’s required.

Even if the whole of the disk is going have just one file system or volume, one of those is required, as it stores information about how the space on the disk is allocated.

Volumes and containers

Apart from APFS, most file systems you’re likely to use require a whole partition on the disk for each volume. For example, an HFS+ disk with two volumes is divided into two partitions, each of which contains one HFS+ volume. Because partitioning is intended to be static, that means those two volumes have fixed size, and don’t share free space between them. Once created, it’s possible to change partitioning, and Disk Utility will try to do so non-destructively, without losing any data in the volumes, but that isn’t always possible.

APFS volumes are different, and share free space within a partition, termed a container in APFS terminology. APFS containers are essentially similar to HFS+ volumes, as they’re static partitions, but APFS volumes are sized dynamically within their static container.

Thus, if you have a disk with two HFS+ volumes and two APFS volumes, that disk will have at least three partitions:

• one for each of the two HFS+ volumes,
• one as the container for the two APFS volumes, although they could instead each be given their own container.

You will see some claims that APFS volumes are more like directories or folders in HFS+, but those are confusing and should be ignored. Each APFS volume has its own file system, and dragging a file from one volume to another results in two completely separate copies of that file, using twice the storage space of one – that’s not how folders behave!

Available formats

Disk Utility version 22.7 in macOS Sequoia 15.0 can format the following file systems using a GUID Partition Map:

• APFS, not encrypted and case-insensitive
• APFS, encrypted and case-insensitive
• APFS, unencrypted and case-sensitive
• APFS, encrypted and case-sensitive
• HFS+ journalled and case-insensitive (JHFS+)
• HFS+ journalled and case-sensitive
• ExFAT
• MS-DOS (FAT32).

Using a Master Boot Record or Apple Partition Map, the following formats are available:

• HFS+ journalled and case-insensitive (JHFS+)
• HFS+ journalled and case-sensitive
• ExFAT
• MS-DOS (FAT32).

APFS is not compatible with Master Boot Record or Apple Partition Map.

The command tool diskutil additionally offers FAT, FAT12, FAT16, Free Space, and HFS+ without journalling, although those aren’t available in Disk Utility.

Note that, contrary to Disk Utility’s Help book, encrypted HFS+ formats are no longer available in Disk Utility, although with the availability of encrypted APFS formats, I can’t think why anyone would want to use encrypted HFS+. That’s because support for encryption was added later to HFS+, whereas it has been designed in from the start of APFS, and is far superior.

Which format for Macs?

The default format for disks to be accessed by Macs is now APFS, not encrypted and case-insensitive, and should be used with all Macs running Mojave and later.

Case-sensitivity can in some circumstances be important. The two situations where this is likely to be required is for the storage of Time Machine backups, and in volumes that might need to store native files from iOS or iPadOS, which use case-sensitive APFS.

Encrypted APFS is different from FileVault used on internal SSDs. However, if you enable FileVault on a macOS installation on external storage, that’s implemented as encrypted APFS. On the internal SSDs of Macs with T2 or Apple silicon chips, FileVault provides additional protection to the keys used for its hardware encryption performed in the Secure Enclave, and should always be enabled. Encrypted APFS is strongly recommended for volumes on external disks that contain private or sensitive data, such as Time Machine backups.

Thus, Time Machine backups should be made to case-sensitive encrypted or unencrypted APFS, and that’s what Time Machine will create for you.

APFS or HFS+?

APFS was designed for use primarily on SSDs rather than hard disks, and doesn’t have features intended to maintain performance when used on hard disks. Unless a volume on a hard disk is intended to store Time Machine backups, it may therefore still be preferable to use either of the supported HFS+ formats. Deciding which is better depends on how that volume will be used.

Because SSDs are largely unaffected by fragmentation of used or free space, APFS isn’t designed to minimise fragmentation, indeed some of its best features inevitably increase fragmentation. In particular, the file system metadata in APFS volumes may become badly fragmented, leading to poor performance on hard disks. Two extreme examples are boot disks and those used to store largely static media libraries.

Recent versions of macOS will happily boot from external hard disks, and despite their relatively poor transfer rates, are far from unusable. Problems come as you use that hard disk, and perform file operations in your Home folder. Over the course of a few weeks or months, fragmentation increases, particularly in file system metadata, and its performance declines.

Media libraries that are most frequently read from, and whose files undergo relatively little change, have fewer changes in their file system metadata, and may well never suffer from noticeable performance impairment. This is also true for Time Machine backups, although some users report deteriorating performance after many months or a year or two.

Time Machine in Sequoia will no longer start a new backup series on HFS+; if you try adding a disk with a single HFS+ volume on it as backup storage, it automatically converts the disk to an APFS container with a single APFS case-sensitive volume, and uses that to store its backups. However, you can still use third-party backup utilities including Carbon Copy Cloner to make backups to HFS+ volumes, although APFS is recommended now. That can have several significant disadvantages, among them:

• Snapshots can’t be made of backup storage.
• APFS special file types aren’t supported. For sparse files, this can lead to backups being substantially larger than the source files. Worse still, if you restore from a backed-up sparse file, it won’t be automatically converted back to sparse file format on APFS.
• Block copying isn’t normally supported, again making backups larger, and increasing the time required to back up.
• Where available, incremental backups may become unmanageable because of the number of files and folders.

As a general principle, both APFS and HFS+ can be backed up to APFS backup storage, while HFS+ storage is only suitable for backups from HFS+.

Which format for Windows compatibility?

Although other computers can read HFS+ and sometimes even APFS volumes, few of their users know how to access the Mac’s native file systems. If you need your storage to be accessible from other computers, one of the two supported MS-DOS/Windows formats is best.

Information given in the Disk Utility Help book is slightly misleading over the choice between FAT32 and ExFAT. FAT32 offers a maximum volume size of 2 TB, with files up to almost 4 GB each, but was primarily intended for magnetic media.

Unless the system intending to read the volume is limited to FAT32, it’s generally preferable to use ExFAT instead. That supports massive volumes and file sizes, and was optimised for use in flash memory. ExFAT is thus the more commonly encountered format for USB flash drives (thumb drives, memory sticks) and SD cards, where it’s the default format for SDXC and SDUC cards larger than 32 GB.

FAT32 and ExFAT are supported with both GUID Partition Map and Master Boot Record schemes. Unless the other computer or system is very old, prefer GUID Partition Map. Neither volume format supports encryption, so if you want to protect files you should encrypt them separately, or store them in an accessible encrypted archive format.

Summary

• For general purposes, default disk format should use a GUID Partition Map with either plain or encrypted APFS.
• Encrypted APFS should be used for volumes on external disks containing private or sensitive data. File Vault should be enabled on internal SSDs.
• New Time Machine backups can now only be made to case-sensitive APFS, either plain or encrypted.
• Hard disks with active file systems may suffer poor performance with APFS, and HFS+ with journalling may still be preferable, but it has significant limitations and disadvantages.
• Hard disks for more static use, such as for media libraries and backups, should be safe with APFS, but may eventually suffer poor performance.
• Unless there are good reasons for using FAT32, format USB flash drives, SDXC and SDUC cards that need to be used with non-Mac systems using ExFAT in a GUID Partition Map scheme.

In the days before Mac OS X, Apple didn’t provide a serious backup utility, and by the time we were starting to move up from Classic Mac OS the standard choice was normally Dantz Development’s Retrospect, first released in 1989 and still available today in version 19.

Time Machine wasn’t the first utility in Mac OS to back up local storage. In 2004, Apple’s first cloud subscription service .Mac included a Backup app that backed up local files to iDisk in the cloud, something that still isn’t supported today with iCloud.

In the following years, AirPort Wi-Fi systems flourished, and Apple decided to launch a consumer NAS incorporating an AirPort Extreme Base Station with a 500 GB or 1 TB hard disk. Software to support that was dubbed Time Machine, and was released in Mac OS X 10.5 Leopard on 26 October 2007, 17 years ago. The first Time Capsule was announced in January 2008, and shipped a month later.

Time Machine’s pane in System Preferences changed little until Ventura’s System Settings replaced it.

The application’s restore interface featured a single Finder-like window, much like today’s. Internally, Time Machine scheduled its backups using a system timer and launchd, making backups every hour regardless of what else the Mac might have been doing at the time.

The initial version of Time Machine was both praised and slated. Unlike Mike Bombich’s rival Carbon Copy Cloner, it couldn’t create bootable backups, and there were problems with FileVault encryption, which at that time could only encrypt Home folders, rather than whole volumes. Despite those, its introduction transformed the way that many used their Macs, and made it more usual for users to have backups.

From its release, Time Machine was dependent on features of the HFS+ file system to create its Finder illusion. Every hour the backup service examined the record of changes made to the file system since the last backup was made, using its FSEvents database. It thus worked out what had changed and needed to be copied into the backup. During the backup phase itself, it only copied across those files that had been created or changed since the last backup was made.

It did this by using hard links in the backup, and Apple added a new feature to its HFS+ file system to support this, directory hard links. Where an entire folder had remained unchanged since the last backup, Time Machine simply created a hard link to the existing folder in that backup. Where an existing file had been changed, though, the new file was written to the backup inside a changed folder, which in turn could contain hard links to its unchanged contents.

This preserved the illusion that each backup consisted of the complete contents of the source, while only requiring the copying of changed files, and creation of a great many hard links to files and folders. It was also completely dependent on the backup volume using the HFS+ file system, to support those directory hard links.

Without directory hard links, backups would quickly have become overwhelmed by hard links to files. If you had a million files and folders on the backup source volume, every hourly backup would have had to create a total of a million copied files or hard links. Directory hard links thus enabled the efficiency needed for this novel scheme to work.

Apple later introduced what it termed Mobile Time Machine, intended for notebooks that could be away from their normal backup destination for some time. In around 10,000 lines of code, Mac OS X came to create something like a primitive snapshot, only on HFS+.

When macOS introduced its new DAS-CTS scheduling and dispatch system for background activities, in (about) Sierra, Time Machine’s backups were added to that. That proved unfortunate at the time because of a bug in that system, which failed on Macs left running continuously for several days, when backups could become infrequent and irregular.

When Apple released the first version of APFS on Mac OS X in High Sierra, its new snapshot feature was immediately incorporated into Time Machine to replace the earlier Mobile variant. Initially, APFS snapshots were also used instead of the FSEvents database to determine what should be backed up. Since then, making each backup of an APFS volume has involved creating a snapshot that’s stored locally on the APFS volume being backed up. In High Sierra and Mojave, the structure of backups themselves didn’t change, so they still required an HFS+ volume and relied on directory hard links.

Catalina introduced a more complicated scheme to replace snapshots as the usual means for determining what to back up. This was presumably because computing a snapshot delta had proved slow. As the backup destination remained in HFS+ format that could’t use snapshots, it continued to rely on directory hard links.

Big Sur and its successors with Signed System Volumes (SSVs) retained the option to continue backing up to HFS+ volumes, but added the ability to back up APFS volumes to APFS backup storage at last.

When backing up to APFS, Time Machine reverses the design used in High Sierra: instead of using snapshots to determine what needed to be backed up before creating a backup using traditional hard links, most of the time Time Machine determines what has changed using the original method with FSEvents, then creates each backup as a synthetic snapshot on the backup store. Unlike earlier versions, in Big Sur and later Time Machine can’t back up the System volume.

Once Time Machine has made a detailed assessment of the items to be backed up, it forecasts the total size to be copied. The local snapshot is copied to an .inprogress folder on the backup volume, and backup copying proceeds. Where possible, only changed blocks of files are copied, rather than having to copy the whole of every file’s data, an option termed delta-copying that can result in significant savings. Old backups are removed both according to age, and to maintain sufficient free space on the backup volume, in what Time Machine refers to as age-based and space-based thinning.

Data copied to assemble the backup on the backup volume is formed into a synthetic snapshot used to present the contents of that backup both in the Time Machine app and the Finder. Those snapshots are presented in /Volumes/.timemachine/ although they’re still stored on the backup volume.

Although modern Time Machine backups to APFS are both quicker and more space-efficient, the structure of backup storage poses problems. Copying backup stores on HFS+ was never easy, but there are currently no tools that can transfer those on APFS to another disk.

Behind the familiar interfaces of its app and settings, Time Machine has come a long way over the last few years, from building an illusion using huge numbers of hard links to creating synthetic snapshots.