Reading view

There are new articles available, click to refresh the page.

How to preserve a document’s versions no matter what

In recent discussions here about the version system built into macOS, two potential problems were raised: first that a file’s versions don’t go with it wherever the file goes, and second that versions wouldn’t preserve datestamps. This article demonstrates how you can easily work around the first, and how the second isn’t correct.

The tools I use here are admittedly my own, but they’re free, and available from this Product Page. They’re also compatible with all versions of macOS from Big Sur (or earlier), and should work perfectly next week when Tahoe is released.

To pose my apps a challenge, I deliberately picked one of my source code files (for LogUI) that I can’t afford to move out of its current folder. This has a total of 230 versions tracking that file’s changing contents, and the development of that app, from its inception last year. Of course, as this is in Xcode I could have used a Git server for full version control, but versioning can do everything I want without going to that trouble.

These are the last few versions shown in Revisionist, simply by dragging and dropping the original file onto it. For each saved version, this displays its datestamp and size in bytes. To preview any of those, just double-click its entry and it will be opened as a QuickLook preview.

With the file open in Revisionist, I could click on the Archive button to save all its 230 versions to their own folder. However, if you just want to archive or move the file with all those versions, that’s quickest using the drag and drop feature in Versatility. Drop the original file on its window, then find the right location to save its versions in their own folder.

This is the result: the top of that folder of 230 versions, each numbered in sequence. QuickLook any of those and you’ll not only see their contents, but the datestamps on each version are those for that original saved (version) file. This original version was created on 7 July 2024, and last saved the following day.

You can now compress that folder of versions and move it wherever you want. I dropped mine onto my drag-and-drop compression utility Cormorant to turn the folder into a 16 KB Apple Archive, just 1.6 KB larger than the current version of that document. I have thought about building that option into Versatility, but you’ll probably prefer to use your own compressor like Keka.

If you wished, you could discard unwanted versions from that folder, because Revisionist and Versatility will reassemble the versions in order, and don’t notice if some of the numbers are missing. If you really wanted to, you could reorder the numbers, but that would be extremely confusing. The version system doesn’t rely on differences between versions, as it stores each version complete.

Once you’ve got your version archive where you want it, decompress it back to a folder, and drag and drop that onto Versatility. Save the file where you want it, and every one of those 230 versions is back inside it, just as they were in the original.

Here’s the copy I made earlier, seen in Revisionist, with the very first version previewed in QuickLook.

There are some relevant notes:

  • Versions do still work in iCloud Drive, but each Mac only sees the versions created and saved on that Mac, not those created by others. I try to avoid mixing versions with iCloud Drive because of that added complication.
  • If you select a version in Revisionist and click the Save button, the copy of that version is faithful, but is created as a new file, so has fresh datestamps.
  • Versions only work on HFS+ and APFS volumes, although you can pass archived versions across any file system.

Managing macOS versioning and the .DocumentRevisions-V100 folder

Whether you make use of it or not, each APFS or HFS+ volume contains data to support macOS versioning, a feature in many Mac apps now, including most of Apple’s own, such as Preview, TextEdit, Pages and Numbers. Unless you block the database from being created, every time you save a document in an app that offers a Revert To command in its File menu, the current version will be saved to that database. There is no way to disable this feature in individual apps. In some cases, that database can cause problems.

Each volume used to store normal working files has at least three hidden folders at its root level:

  • .fseventsd with file system events for that volume recorded in a series of files,
  • .Spotlight-V100 containing the volume Spotlight indexes, detailed here,
  • .DocumentRevisions-V100 containing the volume’s version database.

The last of those is locked away securely to prevent user access. As far as I’m aware, unlike the other two, it hasn’t been described in detail recently, and doesn’t appear in the otherwise comprehensive accounts of macOS by Amit Singh or Jonathan Levin. This article tries to rectify that.

Structure and function

The .DocumentRevisions-V100 folder contains:

  • .cs, containing ChunkStorage, with the ChunkStoreDatabase, and deeply nested folders of numbered storage chunks
  • AllUIDs, containing a nested com.apple.documentVersion folder with each version stored in a dataless file named by its UUID
  • db-V1, containing db.sqlite, the database of versions
  • LibraryStatus, a property list indicating whether the database state is trustable
  • metadata, a property list containing DISK_UUID and tookThinningOver values
  • purgatory, most commonly an empty folder
  • staging, also commonly empty.

When an app that supports versioning saves a file, the current version is added as a dataless file to a folder in AllUIDs, with its UUID as its name, its data are added to the ChunkStoreDatabase, and its details are added to the database in db-V1. Chunk sizes typically range up to just over 20 MB. The service responsible for versioning is revisiond, and the subsystems you’ll encounter in the log are com.apple.foundation.filecoordination and com.apple.chunkinglibrary.

Retrieving a version thus consists of looking it up in the db.sqlite database, and reconstituting that version as a file, using the dataless file with its attributes and metadata in the file UUID, and its data restored from the ChunkStore.

Apps other than the original used to create those versions can also access all the saved versions of any file, but to do so they must be able to open the current version, so having appropriate privileges and privacy settings. This effectively protects access to the contents of the versions database according to the restrictions imposed on its current version.

Backing up

For many years, until macOS Catalina, Time Machine dutifully backed up .DocumentRevisions-V100 folders, although it has never been able to restore them successfully. This is because of its dependence on reuniting chunks of data with a dataless file, and apparent reliance on inode numbers. Copying a .DocumentRevisions-V100 folder to another volume produces a weird result, as the dataless files in the com.apple.documentVersion folder may then point to completely different files.

Third-party backup utilities like Carbon Copy Cloner also don’t back up the folder, as they too are unable to restore it successfully. As a consequence, restoring from a backup, or migration from another volume or Mac, always loses all saved versions.

Housekeeping

The most common cause of problems with the version database is excessive size. Although its size isn’t readily discoverable, it can be a major contributor to that attributed to System Data in Storage settings and third-party utilities, and in some cases can exceed 100 GB.

In the past I had suspected versioning of using hard links or, in APFS, clone files, to achieve economic use of storage, but that appears to be the role of the ChunkStore. Multiple versions of some file types such as JPEG images appear to require a similar amount of storage as would individual files, but others such as RTF can appear significantly more efficient.

ChunkStore size is also dependent on its housekeeping. When a single large JPEG image with 12 versions was deleted, the size taken by chunks didn’t change until the Mac was shut down and started up again. ChunkStore housekeeping routines are clearly performed following start up, but may also occur at periodic intervals later.

Thus the best way to reduce the size of the ChunkStore is:

  • Identify documents whose saved versions are likely to be taking a lot of space in the version database.
  • For each of those documents, to remove unwanted versions.
  • Restart the Mac and leave it to perform ChunkStore housekeeping.

How to identify documents with many versions

My free utility Revisionist has a version crawler that will list all files in a volume or folder with the number of versions they currently have stored in that volume’s .DocumentRevisions-V100 folder. This is accessed in its Open Crawler command in the Window menu. That doesn’t attempt to estimate their size, though.

It’s not uncommon to discover some files with more than 100 versions. On one of my Macs, the current record is 230 versions, and it has several with over 100. Some apps such as Pages often generate large files, where even 20 saved versions can use substantial storage.

How to remove versions

To remove all versions of a document quickly and simply:

  • Select the document in the Finder and Duplicate it using Command-D.
  • Delete the original file and rename the duplicate.

This is because versions aren’t retained by clone files.

To remove some versions of a document, open that document in Revisionist, select the version(s) you want to delete, then click the Delete button. Note that, because of the way the version database works, there is no undo. Deleted versions cannot be recovered, and because they’re not backed up, you can’t restore that document and recover the versions from there.

Preview’s saving grace and how to use it

There are many more powerful PDF and image editors, yet plenty of us use Preview for those basic little tasks, where we don’t need the complications of a heavyweight. Preview isn’t without its faults, though. It can mutilate PDF annotations made by other apps, and sometimes unexpected things happen. To deal with those it has one feature that’s almost unique among PDF and image editors: it automatically saves versions using the macOS versioning system. This article explains how to make use of that.

Ask to save changes when closing documents

One important control over the behaviour of Preview and many other apps that you may not be aware of is this in the Windows section of Desktop & Dock settings. When combined with apps that use the macOS versioning system its effects are significant.

Ignoring versioning for the moment, when this setting is turned on, if you go to close an open document that has unsaved changes, before it’s closed you’ll be asked whether you want to save that changed document. This is a long-standing safety net that continues to protect us from losing lots of work by accident.

When this setting is turned off, apps may still ask you whether you want to save unsaved changes before closing a document unless the app uses the macOS versioning system. In that case, the app automatically saves a new version without offering any option. This might appear inappropriate, but as that’s non-destructive, it avoids interrupting your workflow: you can always revert to the previous version of that document, provided that you’re aware that it has been saved automatically.

If you’re not aware of what’s going on here, and how the versioning system works, this can cause odd effects you can demonstrate using Preview.

Preview’s saving grace

To see how Preview handles versions and the effect of that setting, find a copy of a suitable PDF or image file and duplicate it in the Finder. Name one copy something like testSavesOff, and the other testSavesOn. Set Ask to save changes when closing documents off to begin with, and open testSavesOff in Preview.

Now perform some destructive editing on that document without saving it, here a radical crop.

Leave it a couple of minutes before closing the document. Following that setting, Preview shouldn’t ask you whether to save the changed document, but will simply close it. Then quit Preview.

Now set Ask to save changes when closing documents on, and repeat the same sequence with testSavesOn instead. When you try to close that document, Preview should now ask you whether you want to save the changed document, to which you should click on Revert Changes to set the document back to its previous state instead.

Inspect the two documents using Quick Look and you’ll see that testSavesOn hasn’t changed, but testSavesOff has, although in neither case did you save those changes yourself.

All in the versions

Although you can see what has happened using Preview’s Revert To menu command, the clearest way to see what has happened to those two documents is to open them using my free Revisionist, which shows their saved versions.

testSavesOn has three saved versions. The current one is the same as the original, but the second version shows the destructive edit that you didn’t save.

testSavesOff has two saved versions, the original, and the current version is that after the destructive edit, which you also didn’t save.

Thanks to Preview’s use of the macOS versioning system, either way you’ve still got access to both versions of that document.

One point to note, though, is that the versioning system doesn’t automatically clean up old versions for you. When you’ve finished editing a document in Preview and don’t want to retain its old versions, delete them either using Revert To in Preview, or with Revisionist. Versions are only retained for the original document as long as it’s stored on the same volume. So you can also wipe old versions by duplicating a document and trashing the original, or copying it to another volume. Once they’re gone, you can’t restore them, as not even Time Machine can back up versions.

Inside PDF annotations

In addition to reading laid-out documents, the most popular purposes for PDFs are forms and annotation. As far as filling in PDF forms are concerned, I have just one word to say: Fillably, Joel Norvell’s outstanding app available from the App Store, which transforms Preview into the ideal platform for tax and other forms. Rather than struggling with the tangle of tools in a general PDF editor, Fillably provides the perfect suite for creating PDF forms.

Annotating PDFs is more complicated, though.

Encoding annotations

Annotations aren’t an afterthought, but a central part of PDF. All PDF documents consist of a list of hundreds or thousands of objects of different types, including annotations of the Annot type, and those are listed in one of the file’s standard dictionaries, its Annotations Dictionary, Annots.

There are at least 27 sub-types of Annot, including Caret, Highlight and Stamp, which are reflected in the annotation tools provided by apps from Acrobat to Preview. Seemingly complex annotations like popup notes are straightforward to code in PDF, requiring just two linked objects, one for the popup and its text, the other to specify its placement on the page. Others are more involved, as they can extend to include file attachments, sound, movie and other multimedia.

PDF versions

Despite their original simplicity, there are multiple problems that can arise with annotations.

With more recent versions of PDF, the ways in which they can be coded has increased. Mark up a PDF using the latest versions of Adobe Acrobat Reader or its ‘Pro’ CC colleague and they’ll cast it in PDF-1.6 and you’re unlikely to see a single Annot in their source. Most apps built on the Quartz PDF engine should write their files in PDF-1.3 so they can be accessed more widely, and should use regular Annot sub-types throughout. However, Preview likes to use opaque AAPL:AKAnnotationObjects that you won’t encounter anywhere else.

What Quartz does is to ‘flatten’ each PDF into a common 1.3 format for rendering, and that can be saved to disk. At present, that seems to work faithfully, but might give the impression that macOS can’t render more recent versions of PDF, which isn’t true. You can demonstrate that by opening an Acrobat PDF-1.6 document using an app that relies on the Quartz engine, such as PDF Expert, Preview or my Podofyllin, and comparing that with the original in Acrobat.

Podofyllin has a convenient feature for doing just that, in its source window. The uppermost of its three views displays Quartz ‘flattened’ code in PDF-1.3, the middle shows the original, here in PDF-1.6, and the lowermost is a summary of the latter.

Hidden annotations

The biggest dangers with annotations arise because of PDF’s ancient origins and a file format that doesn’t make sufficiently clear distinction between data and metadata. All annotations are metadata added to the underlying document, but in PDF, objects for each are mixed freely within the source. When they’re clearly distinguished with the Annot type, they should be easy to remove as a group, and PDF Expert offers that as a convenient command. That’s ideal when a document has been developed with the aid of reviewers’ annotations, to prepare the finished version for release.

Unfortunately this can cause its own problems, as PDF source is notorious for retaining old content that’s no longer visible in the rendered document, but can be read by anyone with a little knowledge about PDF. Like incomplete redactions, such hidden annotations have caused many embarrassments in the past, and will continue to catch folk out.

Preview’s bugs

Finally, Preview has had more than its fair share of bugs in handling PDF annotations. During my research for this article, Preview 11.0 (1069.7.1) in macOS 15.6 was generally well behaved, but did mangle comments added to a test document by PDF Expert and Adobe Acrobat. Preview has two behaviours that can appear disconcerting: that of its Highlights and Notes tool, and its use of versioning.

All Preview’s tools are single-shot apart from Highlights and Notes, the drawing pencil icon to the left of its popup menu. Click this once to apply highlighting to selected blocks of text, and to remove existing highlighted sections. Unfortunately when this tool is turned on, its own highlighting is so weak that it’s hard to see.

Overwritten files

Preview has a habit of saving PDF documents automatically when closing them, without any warning. If it has just mutilated an annotation, for example, you might assume the original file has just been overwritten and lost. However, Preview saves PDFs using the macOS document versioning system, so you can always recover the previous version.

This might at first seem an impossible task: use Preview to restore that old version and it will repeat its mutilation, defeating the purpose. Yet the original PDF editor won’t have access to previous versions, as it doesn’t use the versioning system. The solution is to use Revisionist, or Versatility, which can save the original as a separate document.

Key points

  • Annotations are a central feature of PDF, come in many sub-types, and can be complicated as they can be expressed in different ways.
  • The macOS Quartz PDF engine transforms them into PDF-1.3, which makes them simpler and more explicit, so they can be saved ‘flattened’.
  • PDF format mixes document data with metadata and annotations.
  • Few PDF editors offer to remove all annotations, and there’s a risk of some remaining hidden from view, but still remaining in the PDF source, potentially causing embarrassment when they’re discovered.
  • Preview’s earlier bugs in annotations have improved, but it can still mutilate those made by other PDF editors.
  • If Preview saves a mutilated PDF, you should be able to recover the previous version of that file using Revisionist or Versatility.

Updates to Cirrus (iCloud), Revisionist (versions), Spundle (sparse bundles) and T2M2 (Time Machine)

This next batch of updates to my apps includes more popular tools, covering iCloud, document versions, sparse bundles, and Time Machine backups.

iCloud

Cirrus gives you detailed insight into what’s stored in iCloud Drive, provides a ready-made log browser for checking what’s going on, and a simple test for syncing. Version 1.16 has an overhauled interface, and has been rebuilt with a new app icon ready for macOS 26 Tahoe. This version supports macOS from Big Sur onwards.

Cirrus 1.16 is now available from here: cirrus116
from its Product Page, and via its auto-update mechanism.

Document versions

Revisionist gives you direct access to versions of documents saved automatically by macOS, and a powerful suite of tools to work with them. You can run checks to discover which documents have saved versions, then browse those, previewing them with Quick Look. It can save individual versions as new files, and create archive folders containing all versions, that can be reconstituted into the original with those versions preserved. Version 1.10 has an overhauled interface, and has been rebuilt with a new app icon ready for macOS 26 Tahoe. This version supports macOS from Big Sur onwards.

Revisionist 1.10 is now available from here: revisionist110
from its Product Page, and via its auto-update mechanism.

Sparse bundle disk images

Spundle creates and maintains sparse bundle disk images, offering a range of supported file systems, and features such as compaction to maintain their efficiency. Version 1.9 has an overhauled window, and has been rebuilt with a new app icon ready for macOS 26 Tahoe. This version supports macOS from Big Sur onwards.

Spundle 1.9 is now available from here: spundle19
from its Product Page, and via its auto-update mechanism.

Time Machine backups

The Time Machine Mechanic, T2M2, is the standard utility for checking your Mac’s Time Machine backups. It checks and reports on their performance, free space on backup storage, how much has been transferred in each backup, and much more. Version 2.03 has an overhauled interface, and has been rebuilt with a new app icon ready for macOS 26 Tahoe. This version supports macOS from Big Sur onwards, backing up to APFS.

Depending on any changes finalised in the full public release of Tahoe later this year, I may need to make further adjustments to its code.

T2M2 2.03 is now available from here: t2m2203
from its Product Page, and via its auto-update mechanism.

Enjoy!

❌