Although the great majority use GUI search tools provided in the Global Spotlight menu, Finder Find windows, and in-app Core Spotlight, macOS also provides access using query languages. This article takes a brief tour of those available in macOS. As with previous coverage, this doesn’t include third-party utilities such as HoudahSpot that provide their own interface to Spotlight searching, nor alternative search methods.

Search boxes

Search boxes provided by both Global Spotlight and Local Spotlight can accept a simple form of query language, for example
name:"target"*cdw
which also works with filename:, and performs the equivalent of the Matches operator in a Find window. These use English terms for the attributes to be used, like name and filename, including some of those listed here for Core Spotlight. However, limited information is available and this doesn’t appear to be extensive enough to use at scale. Operators available are also limited within those listed for Core Spotlight.

Modifiers available in current macOS include

• c for case-insensitivity,
• d to ignore diacritics such as accents,
• w to match on word boundaries, as marked by space, underscore _, hyphen – and changes of case used in CamelCase.

The asterisk * can be used as a wildcard to match substrings, and the backslash \ acts as an escape character, for example \" meaning a ” literal. In theory, simple predicates can be combined using && as AND, and || as OR.

In practice, getting these to work is tricky, and rarely worth the effort of trying.

Raw queries

One of the least-used attributes available in search bars in the Find window enables the use of what are termed raw queries. Confusingly, these use different names for attributes, such as kMDItemDisplayName instead of name. Otherwise these are more reliable than those used in search boxes. For example, when searching for the string target,
kMDItemDisplayName = "*target*"
is the equivalent of Contains, and
kMDItemDisplayName = "target*"w
appears functionally identical to Matches.

These appear to be an option of last resort, and need documentation.

mdfind

This command tool provides the most complete access to the central Spotlight Query Language, which defies abbreviation to SQL. In addition, it also supports a direct form that searches for matching file names only, using
mdfind -name "target"
to find the word target in filenames.

Unfortunately, although Spotlight Query Strings are predicates, they aren’t the same as NSPredicates used elsewhere within macOS. One of the most obvious differences is that Spotlight’s modifiers are appended to the value, not the operator, as they are when using search predicates in the log command, for example.

Query strings take the general form
attribute operator value[modifiers]
where

• attribute is a kMD… name defined as a metadata attribute key,
• operator can take a formal version such as ==, or may be abbreviated to just =, which appear to be identical in effect,
• value can be a string containing wildcards or escapes, or those detailed for special cases such as numbers and dates,
• modifiers include those given above.

Simple examples are
mdfind "kMDItemDisplayName = '*target*'"
or
mdfind "kMDItemDisplayName == '*target*'"

Apple’s current list of common metadata attribute keys is given here. Otherwise, documentation is old if not ancient, and there are obvious differences from current Spotlight and mdfind, such as the expanded list of modifiers.

File metadata queries are explained here, with an apparently duplicated version here. File metadata attributes are documented here, and a general account of predicates is here.

Saved Search Queries

On the face of it, one way to become familiar with and to develop query strings for use in mdfind might be to set them up in a Find window, save that and use the query string within it for your own. Although this can be helpful, queries in saved search files often use attributes not accessible to mdfind. For example, entering the string target in the search box setting a search bar to Kind is Image All is represented by the query
(((** = "target*"cdw)) && (_kMDItemGroupId = 13))
where the second attribute is an internal form of kMDItemKind.

However, this quickly runs into difficulties, as values of _kMDItemGroupId don’t appear to be documented, and substituting that with an alternative such as
kMDItemKind = "public.image"
fails silently.

Conclusions

• Spotlight query strings take several forms, none of them well-documented.
• Queries provided in Saved Search are of limited use, and are only likely to confuse.
• For occasional use, they are usually frustrating.
• For frequent use, third-party alternatives are more consistent and much better documented.

How everything grows over time. Twenty years ago a hard disk of 100 GB was often ample, now twenty times that can be insufficient, and some have even larger media libraries. Finding files from among tens of thousands used to be straightforward, but now we’re working with millions we’re often struggling. Last week’s discussions of Spotlight search and its alternatives highlighted how important search strategies have become.

Strategy

Perhaps the most common strategy we use to search quickly and effectively is to apply a series of properties or attributes narrowing from the general to the specific: a dog, a small dog, a small grey-and-white dog, a small grey-and-white Havanese dog. In just a few adjectives we have narrowed the field to a description applying to a small number of domestic pets.

This strategy has two essential requirements: the target of your search must be included in the list of items being searched, and each of the attributes or criteria you apply in succession must include the search target. The first is obvious and critical to Spotlight’s success, and the second is the basis of how attributes are chosen. If the dog’s colour had been specified as red, then that search would have failed.

One of many skills in successful searching is judging how exclusive each criterion should be, and being more inclusive to ensure none of the criteria might inadvertently exclude the target.

Although you can combine attributes in this way when searching using the general Spotlight search window accessed through the menu bar, that’s a global search including websites and everything searchable from Wikipedia to Photos albums and Messages. When looking for a file, searching in the Finder immediately narrows the scope, and saves you wading through many irrelevant results. You can then add a search bar for each criterion, perhaps specifying that you’re looking for an image in your ~/Documents folder, each time reducing the number of hits until your choice becomes sufficiently limited.

Incremental search

Spotlight offers another technique that has become popular in search engines as their performance has improved, in what’s known as live or incremental search. As you type letters into one of its search boxes, it shows results as it gets them. This isn’t much use when entering common combinations of letters, but as they become more specific this can save time and accommodate any uncertainty you might have over spelling or the rest of the word. I use this frequently in MarsEdit when looking for old articles I have written: for example, typing wrestl will find wrestler, wrestlers, wrestling, wrestled, etc.

This works well with most languages including English, where roots and meanings are concentrated in the first parts of words, and declension and conjugation are usually found in their endings. Not all languages work like that, though, and this may not perform as well in Georgian or even German due to their morphology.

Predicates

For those who prefer to use the command line, mdfind can use predicates to express combinations of attributes, but those aren’t readily used in the same incremental way to narrow results down interactively. Another situation where predicates often come into play is when searching log entries and using the log show command, and that brings me on to LogUI, my other concern last week.

Searching the log

Let’s say you want to discover all the information RunningBoard gathers about an app, something you know is written in a log entry by the com.apple.runningboard subsystem shortly after that app starts its launch sequence. While you could search for all entries for that subsystem in the minute or so around the time you launched the app, there are likely to be thousands of hits.

To narrow down that search you have several options, including:

• launch the app at a known time, and set that as the Start time, with a Period of just a couple of seconds;
• set a one-off predicate to subsystem == "com.apple.launchservices" OR subsystem == "com.apple.runningboard";
• search subsystems for com.apple.launchservices to identify the time that LaunchServices announces the app will be launched through RunningBoard;
• search messages for constructed job description, RunningBoard’s log entry giving the details you’re looking for.

Those are ordered in increasing specificity, reducing numbers of hits, and increasing requirement for prior knowledge. That’s a general association, in that the more prior knowledge you have, the more specific you can make your criteria, and the fewer irrelevant hits you will see. As with Spotlight search, the more of these criteria you apply, the greater your chance of success, provided they all match the entry you’re looking for.

LogUI

To make LogUI more amenable to incremental search strategies, two additional features are needed. Instead of only exporting whole log extracts to Rich Text, the app needs to save and read formatted extracts. It also needs the ability to eliminate entries that don’t meet search criteria. Together those will enable use of a predicate to save an extract of reduced size, then application of search criteria, maybe saving an even smaller extract.

One way to combine multiple searches is to use multiple search bars, in a similar way to the Finder’s Find window. However, that tends to become overcomplicated, and I suspect is relatively little-used. If you do need a series of search criteria, then you also need different ways of combining them, including OR as well as AND, and that becomes a GUI predicate editor. I have yet to see any successful GUI predicate editor.

Next week, in the days prior to WWDC, I’m going to be focussing on search strategies for Spotlight, before turning to LogUI to implement these changes. This is an ideal time to let me know what you’d like to see, and how LogUI can support more successful search.