Normal view

There are new articles available, click to refresh the page.
Before yesterdayMain stream

Solving Finder tag problems

By: hoakley
24 December 2024 at 15:30

Finder tags are a popular way to add accessible metadata to files and folders in macOS, with attractive properties:

  • Their associated colour tags are easily seen and instantly recognisable.
  • Each colour can be associated with one or more text labels.
  • The Finder can display tagged items conveniently.
  • Tags propagate through iCloud Drive, and can be used fully in iOS and iPadOS.
  • Tags are generally preserved within macOS, for example when copying or moving items between volumes.
  • Tags are searchable using Spotlight, as their text labels are indexed.

I have explained their use in this article.

Their principle limitation is that they’re limited to 7 colours, together with no colour. That allows you to assign multiple text labels to each colour, but the results of that can be confusing given that Finder display and search are based on their text labels not colours.

Implementation

Finder tags are implemented as extended attributes (xattr), currently of type com.apple.metadata:_kMDItemUserTags. However, historically they can also be implemented alongside other Finder information in com.apple.FinderInfo xattrs.

com.apple.metadata:_kMDItemUserTags xattrs consist of a binary property list containing brief UTF-8 text. This is an NSArray consisting of Strings, each containing a text label, followed by the newline character, followed by the colour number (0-7). The array can be empty. Obviously, labels can’t include the newline character; although they can include the colon character :, that has been reported as causing problems in some versions of macOS and is worth avoiding.

tags04

Colour numbers used are:

  • none, 0
  • grey, 1
  • green, 2
  • purple, 3
  • blue, 4
  • yellow, 5
  • red, 6
  • orange, 7.

Thus a tag name might read Red\n6, or Orange\n7 Green\n2 for two colours, where \n represents the newline character 0a.

When the Finder writes a tag xattr to an item, it also adds a null com.apple.FinderInfo xattr of 32 bytes length, if a xattr of that type isn’t already present.

com.apple.FinderInfo xattrs can be used to store a single colour tag without a text label, although this shouldn’t be encountered any more. When they do, a single byte is set in their fixed length of 32 bytes, the kColor flag just to the right of the Hide extension flag. For instance, <00000000 00000000 00040000 00000000 00000000 00000000 00000000 00000000> sets the Green tag. This scheme uses a different encoding for colour flag values:

  • none = 00, 01
  • grey = 02, 03
  • green = 04, 05
  • purple = 06, 07
  • blue = 08, 09
  • yellow = 0A, 0B
  • red = 0C, 0D
  • orange = 0E, 0F.

Basic checks

With a selection of tagged items, first verify the colours and text labels are shown correctly in the Finder’s Get Info dialog, and match those against com.apple.metadata:_kMDItemUserTags xattrs inspected using xattred‘s drag and drop interface. Also check com.apple.FinderInfo xattrs, although they shouldn’t have their kColor flag set.

Check tagged items both locally and in iCloud Drive. Note that tags should be preserved even when that file has been evicted from local storage to iCloud, as xattrs are stored locally and retained on dataless files. Tags should also be displayed in apps that support them, and in File Open and Save dialogs.

Problems with these basic checks should make you suspect file system errors in volumes affected. Run Disk Utility’s First Aid on that volume and perform necessary repairs. Although they should be shown in some other file systems, Finder tags are best-supported by HFS+ and APFS volumes and may have limitations in other file systems.

iCloud Drive syncing

Test this with:

  • In the Finder, open a user folder in iCloud Drive. Select a file there, and add a Finder tag to it using the contextual menu.
  • That should result in an immediate and brief sync up to iCloud, to copy that tag up.
  • Check that this change syncs across other Macs and devices connected to the same Apple Account.
  • If necessary, add the tag at a known clock time and use that to inspect iCloud systems in the log, using Mints or Cirrus.

Failure to sync the changed tag information with iCloud indicates a problem in syncing with iCloud, and requires separate diagnosis.

Spotlight search

Before looking at search for tags, first confirm normal Spotlight indexing and search function using Mints. If those tests don’t work correctly for search of file contents, address those problems first before assessing tags.

The single most common reason for search failures is that the item being searched for is in a location excluded either from indexing or from returning search results. Check Spotlight or Siri & Spotlight settings for the following:

  • That item’s category isn’t excluded from appearing in Spotlight search results. For example, if the Images item in the list there isn’t ticked, then images will still be indexed, but Spotlight won’t return any images in its search results. Unless you have a good reason, the simplest setting here is for all boxes to be ticked.
  • That item’s path isn’t within any of the locations listed in Search Privacy…, as those aren’t indexed at all.

There have been additional methods for excluding specific items from being indexed by Spotlight, of which two are currently effective:

  • appending the extension .noindex to the folder name (this previously worked using .no_index instead);
  • making the folder invisible to the Finder by prefixing a dot ‘.’ to its name;
  • putting an empty file named .metadata_never_index inside the folder; that no longer works in recent macOS.

Check that the items you expect Spotlight to find aren’t subject to any of those. Details of those items not synced by iCloud Drive are given in this article.

In addition to those excluded locations, Sequoia (and possibly other recent versions of macOS) generally excludes folders and files within either of the two user-writable Library folders, /Library and ~/Library. Limited indexing is performed within the Application Support folder, but that doesn’t appear to include tags. Although iCloud Drive as a whole is shown as being inside ~/Library, it’s indexed differently, as given above, and that should also include app-specific folders in iCloud Drive.

Before even considering rebuilding a volume’s Spotlight indexes, check whether Spotlight correctly indexes test files created using Mints, when you add tags to them. When tags are added, you may be able to see a short burst of mdworker activity in Activity Monitor, and it should be recorded in the log, as checked using Mints. Full details of diagnosing and fixing problems with Spotlight search are given in this article.

Step summary

  1. Verify tags as xattrs both locally and in iCloud Drive, using xattred.
  2. Verify syncing in iCloud Drive using Mints or Cirrus.
  3. Check Spotlight search is working correctly for other contents, using Mints.
  4. Verify tagged items are in a path that isn’t excluded from indexing, and their category is set to return search results, in System Settings.
  5. Diagnose any underlying Spotlight problems using Mints.

Last Week on My Mac: Health & Efficiency

By: hoakley
17 November 2024 at 16:00

macOS Sequoia has had its fair share of problems, particularly with networking, and by all accounts its bundled firewall still doesn’t play nice with third-party software firewalls. But most worrying are those whose Macs find Sequoia overwhelming, recent models that should take the upgrade in their stride, but apparently don’t.

Typically, from moments after logging in, these Macs remain incessantly busy with what we expect to be brief background tasks. Several hours later, Activity Monitor still shows long periods of frenetic activity on Apple silicon Efficiency (E) cores. These appear to keep building endless Spotlight indexes and rummaging through images with photoanalysisd. Surely something must be seriously amiss in Sequoia.

handecpuhistory

When you start up an Apple silicon Mac in Sequoia 15.1, it might well appear so. There’s a long procession working through background services such as Spotlight index maintainers (mdsync, mds, mds_stores, mdwrite and lots of mdworker_ processes), Time Machine (backupd), security (XProtectRemediator, lsd, syspolicyd), and others like CGPDFService, spotlightknowledged and photoanalysisd that might look vaguely familiar.

handecputotals

If this procession does come to an end before the user’s patience, CPU time for some of those is amazing. Here mdwrite ran for almost ten minutes, mds for nearly five, and that’s after a normal startup, long after the last update to 15.1. How can this be right? To understand why there’s probably nothing wrong, we should consider what Activity Monitor isn’t telling us, what those E cores are up to, and why this can be normal behaviour.

E cores and % CPU

In an Apple silicon Mac, its E cores are intended to run background threads as efficiently as possible, and that often means at low clock speeds (frequencies). In an M1 Mac, that could be as slow as 600 MHz, and even in an M4 at a mere 1,020 MHz. Those are less than a quarter of the maximum frequency of the P cores, and sip power at just a few milliwatts. Unfortunately, as I’ve explained in detail, % CPU and the CPU History window in Activity Monitor take no account of core frequency. What they portray as 100% could be anywhere between the minimum and maximum, making for a huge spread of core throughput.

Frequency of both E and P cores varies greatly as they’re running different threads, as controlled by macOS. In an M4 chip, P cores have 19 set frequency steps, and even E cores have seven. What might appear in the CPU History chart to be a steady 100% load on all four E cores is probably varying widely with core frequency, with much of it running slowly to eke out power.

There’s a little irony here, in that Apple silicon Macs working their E cores with background tasks may have greater visual impact than in an Intel Mac, where they will be spread more evenly across all the cores, and appear far less despite their real impact being greater.

What is being analysed and indexed?

Over the last couple of years, macOS has gained new abilities to analyse the content of images, including those embedded in PDF documents. It all started with the introduction of Live Text back in Monterey, and has blossomed since to include other types of analysis including categorising objects discovered within images.

Consider a PDF consisting of pages scanned from a book with diagrams and illustrations. macOS can now run Live Text on each page, and within each of its pictures, then add its text and graphics content to its Spotlight indexes. You may already have noticed that searching for words that commonly appear in certain types of image like screenshots returns many hits where that text is embedded in a menu or dialog within that screenshot. Performing text recognition and image analysis on those thousands of pages takes time, particularly when it’s mostly being run efficiently, in the background on E cores.

Other background features have also changed their behaviour. For example, XProtect Remediator scans have become less leisurely and prone to omission unless your Mac is left running but inactive for significant periods. It’s now common for scans to be run shortly after the first Time Machine backup is made, five or ten minutes after starting up, rather than waiting a few hours for an appropriate moment. These now take the opportunity to get their work done sooner, running in the background on the E cores, which is, after all, what they are designed to do.

Efficient workers

Apple’s M-series chips don’t come cheap. We pay for the P cores that run our apps fast and keep them responsive so we don’t have to wait as we so often did in the past. We also pay for the E cores that beaver away in the background to ensure that when we search for photos of Aunt Sue we don’t have to sort through duds of Uncle Dave. Like all the best workers, those E cores get on with their jobs unobtrusively, work anti-social hours without complaint, and don’t crave power. So please don’t kill them, as their efficiency keeps your Mac healthy and productive.

Irreverent Background Note

Health & Efficiency plays on the name of a magazine now known as H&E naturist that was popular among young adolescent males, as for many decades from the 1920s it was one of very few newsstand publications in the UK that featured photos of naked women and men. For curious schoolboys who were too short to reach its usual lofty position on the shelves, tall friends were a great asset. I also suspect that the title is itself a play on the term self-sufficiency, which has surprisingly early origins in sixteenth century English.

❌
❌