The greatest challenge in using the Unified log is how to navigate its many thousands of entries, to find those you want to read. Success depends on the combination of two aids: time and waypoints (or landmarks).

Time

No matter how you obtain log extracts, you need to know when to look for those entries. The more precisely you can work out the time of interest, the quicker and easier it will be to locate the entries you’re interested in. While the log command offers alternatives, LogUI works throughout using the local time applicable when you access the log, allowing for your current time zone and any seasonal adjustment to it, when accessing the live log in that Mac.

However, the underlying times given in log extracts are those recorded by the Mac or device whose log you’re accessing. If its system clock was five minutes slow when those entries were written to its log, then you need to allow for that. For example, when I first started my Mac yesterday its clock might have been 1 minute slow. An event that occurred at 10:56 yesterday by the room clock would therefore appear in the log entries for 10:55.

One important time you can discover is the boot time of the Mac. Mints offers a Boot button to retrieve boot times over the last 24 hours. If the logs were written by a different Mac or device, then you’ll need to search for the time of that last boot. Fortunately the first two log entries are easily recognised:
11:41:37.562774+0100 === system boot: D3CEA9B4-F045-434D-8D12-C6E794A02F14
11:41:42.758780+0100 kprintf initialized
The long gap between the first two entries is accounted for by the firmware phase of the boot process. If necessary you can search for a message containing === (three equals signs). Mints provides the time of the first of those for each boot, and its UUID.

There are two occasions when time can become confusing, when clock corrections are applied, and when clocks are moved forward or back to add or remove summer or seasonal time changes. Fortunately the latter only change twice each year, although when they do, you really don’t want to see what happened in the log, and those changes aren’t even applied at a predictable time.

Clock corrections, like kernel boot, are readily found by the === text in their message. They normally happen in pairs, with the first correction the larger, and the second often far smaller. Here’s an example seen in consecutive log entries:
08:26:16.140474+0100 /usr/libexec/sandboxd[80] ==> com.apple.sandboxd
08:26:10.043353+0100 === system wallclock time adjusted
08:26:10.044335+0100 Sandbox: distnoted(72) deny(1) file-read-metadata /private
08:26:10.044601+0100 2 duplicate reports for Sandbox: distnoted(72) deny(1) file-read-metadata /private
08:26:10.044606+0100 Sandbox: distnoted(72) deny(1) file-read-metadata /Library
08:26:10.089204+0100 === system wallclock time adjusted
08:26:10.091850+0100 started normally

The first adjustment dropped the clock back by 6.1 seconds, from 08:26:16.140474 to 08:26:10.043353. This means that you’ll see times of 08:26:12 both before the correction and afterwards. The second adjustment, from 08:26:10.044606 to 08:26:10.089204, was far smaller at 0.045 seconds, and at least went in the right direction.

The most substantial clock corrections are made shortly after booting. Although macOS does make them later, the size of those should be smaller.

Waypoints

Even working with times resolved to the second, those can still leave you browsing thousands of log entries. To locate more precisely you need details of one or more entries that will be sufficiently distinctive to focus in on a few dozen. These are waypoints for navigation.

LogUI provides three methods for locating these waypoints:

• using a search predicate to determine which log entries are extracted from the log;
• applying search text to filter out all entries that don’t contain a term;
• searching a rich text export of the log extract.

Predicates

These are best used when the time period of your extract needs to be relatively long, so would return a large number of entries. For example, if you can only narrow the time down to several minutes, and are looking for the time that a specific app was launched, you can look for that app’s job description when it’s created and written to the log by RunningBoard.*

Over a period of two minutes, RunningBoard might write thousands of entries in the log, so looking for your app’s job description among them would be time-consuming. Set the start time and period to cover the whole of the time you want to search, then set a predicate for the subsystem com.apple.runningboard.

When LogUI fetches that log extract, there might still be over 2,000 entries, so now is the time to apply search text to filter those further.

Search filter

To filter those 2,000 entries and show only those containing job descriptions created by RunningBoard, enter the text constructed job in LogUI’s search box, with its menu set to Messages, and press Return. You’ll now see that list reduced to just a handful, and looking through them you can discover exactly when your waypoint occurred.

My example for this article starts with a period of just 2 minutes, in which there were more than 100,000 log entries.

Using the com.apple.runningboard predicate whittled those down to 13,443 entries.

Searching within those for constructed job left me with just 8 entries to look through.

Search rich text

Sometimes you can’t devise the right combination of predicate and search filter to discover what you’re looking for, which might be an error reported in a subsystem or a process that you can’t identify. One good way forward is to narrow your log extract as much as you can, then save the extract as Rich Text, open that in a suitable editor, and search through it for the word error. That will discover every log entry containing the word error anywhere, rather than confining it to the message text.

Using time and waypoints

Armed with your waypoint and the exact time of its entry in the log, you can now set that as the start time, set a period of a couple of seconds, and get a full log extract containing all the detail you might need. This should give you further clues to allow you to move through time using predicates and search filters to discover what happened. This is much quicker and less frustrating than trying to scan through thousands of log entries in search of vague clues.

Key points

• Use time and waypoints to find log entries.
• Mints’ Boot button gives times of each boot in the last 24 hours.
• Reduce the number of log entries returned using a predicate.
• Narrow those down using a search filter.
• Search all text by exporting the log extract as Rich Text.

* Sadly, the days of being able to access freely RunningBoard’s informative job descriptions in the log are over. As of macOS Tahoe, all you’ll see is the dreaded <private> of censorship. If you want to examine these now, you’ll have to remove log privacy protection first. Thanks, Apple, for providing such useful tools then rendering them next to useless.

Over the last few weeks, as I’ve been digging deeper into Spotlight local search, what seemed at first to be fairly straightforward has become far more complex. This article draws together some lessons that I have learned.

Apple’s search patents have been abandoned

Having tracked down a batch of Apple’s patents relating to search technologies that are likely to have been used in Spotlight, I was surprised to see those have been abandoned. For example, US Patent 2011/0113052 A1, Query result iteration for multiple queries, filed by John Hörnkvist on 14 January 2011, was abandoned five years later. I therefore have no insights to offer based on Apple’s extant patents.

Search is determined by index structure

Spotlight indexes metadata separately from contents, and both types of index point to files, apparently through their paths and names, rather than their inodes. You can demonstrate this using test file H in SpotTest. Once Spotlight has indexed objects in images discovered by mediaanalysisd, moving that file to a different folder breaks that association immediately, and the same applies to file I whose text is recognised by Live Text.

Extracted text, that recovered using optical character recognition (Live Text), and object labels obtained using image classification (Visual Look Up) are all treated as content rather than metadata. Thus you can search for content, but you can’t obtain a list of objects that have been indexed from images, any more than you can obtain Spotlight’s lexicon of words extracted as text.

Language

Spotlight’s indexes are multilingual, as demonstrated by one of Apple’s earliest patents for search technology. Extracted text can thus contain words in several languages, but isn’t translated. Object labels are likely to be in the primary language set at the time, for example using the German word weide instead of the English cattle, if German was set when mediaanalysisd extracted object types from that image. You can verify this in SpotTest using test file H and custom search terms.

If you change your Mac’s primary language frequently, this could make it very hard to search for objects recognised in images.

Search method makes a difference

The Finder’s Find feature can be effective, but has a limited syntax lacking OR and NOT unless you resort to using Raw Query predicates (available from the first popup menu).* This means it can’t be used to construct a search for text containing the word cattle OR cow. This has a beneficial side-effect, in that each term used should reduce the number of hits, but it’s a significant constraint.

The Finder does support some search types not available in other methods such as mdfind. Of the image-related types, John reports that kMDItemPhotosSceneClassificationLabels can be used in the Finder’s Find and will return files with matching objects that have been identified, but that doesn’t work in mdfind, either in Terminal or when called by an app. Other promising candidates that have proved unsuccessful include:

• kMDItemPhotosSceneClassificationIdentifiers
• kMDItemPhotosSceneClassificationMediaTypes
• kMDItemPhotosSceneClassificationSynonyms
• kMDItemPhotosSceneClassificationTypes.

One huge advantage of mdfind is that it can perform a general search for content using wildcards, in the form
(** == '[searchTerm]*'cdw)

Using NSMetadataQuery from compiled code is probably not worth the effort. Not only does it use predicates of different form from mdfind, but it’s unable to make use of wildcards in the same way that mdfind can, a lesson again demonstrated in SpotTest. mdfind can also be significantly quicker.

For example, you might use the form
mdfind "(kMDItemKeywords == '[searchTerm]*'cdw)"
in Terminal, or from within a compiled app. The equivalent predicate for NSMetadataQuery would read
(kMDItemKeywords CONTAINS[cdw] \"cattle\")

Another caution when using NSMetadataQuery is that apps appear to have their own single NSMetadataQuery instance on their main thread. That can lead to new queries leaking into the results from previous queries.

Key points

• Spotlight indexes metadata separately from contents.
• Text recovered from images, and objects recognised in images, appear to be indexed as contents. As a result you can’t obtain a lexicon of object types.
• Indexed data appear to be associated with the file’s path, and will be lost if a file is moved within the same volume.
• Text contents aren’t translated for indexing, so need to be searched for in their original language.
• Object types obtained from images appear to be indexed using terms from the primary language at the time they are indexed. If the primary language is changed, that will make it harder to search for images by contents.
• The Finder’s Find is constrained in its logic, and doesn’t support OR or NOT, although using Raw Queries can work around that.
• mdfind is most powerful, including wildcard search for content.
• NSMetadataQuery called from code uses a different predicate format and has limitations.

I’m very grateful to Jürgen for drawing my attention to the effects of language, and to John for reporting his discovery of kMDItemPhotosSceneClassificationLabels.

• I’m grateful to Jozef (see his comment below) for reminding me that there is a way to construct OR and NOT operations in the Finder’s Find, although it’s not obvious, and even when you know about it you may well find it difficult to use. I will explain more about this in another article next week.

Spotlight has been able to find text extracted from images using Live Text, and the names of objects recognised using Visual Look Up, for some years now. This article considers how you can and cannot search for those. Although this might seem obvious, it’s more confusing than it appears and could mislead you into thinking that Spotlight indexing or search isn’t working.

As detailed in yesterday’s account of Live Text, text recognition in images uses a lexicon to match words rather than proceeding in single characters. Terms assigned to recognised objects are also words. Thus, when searching for either type you should use words as much as possible to increase the chances of success.

Global Spotlight

Type a word like cattle into Spotlight’s search window and you can expect to see a full selection of documents and images containing the term or cattle objects. Those include images containing the word, and images containing objects identified as cattle, but don’t include images in PDF files, as they’re not analysed by mediaanalysisd, so don’t undergo character or object recognition in the same way that regular images like JPEGs do.

The Finder’s Find

Open a new window in the Finder and turn it into a Spotlight search using the Finder’s File menu Find command. In its search box at the top right, type in a word like cattle and in the popup menu select the lower option, Content contains. Press Return and the search box will now display ANY cattle. Then set the Kind to Image in the toolbar above search results, to narrow results down to image files. You should then see a full listing of image files that either contain the word cattle, or contain objects identified by image analysis as being cattle. Note how many of those appear.

Reconfigure the search so the search box is empty, and there are two rows of search settings: the first can remain the same as Kind is Image, but set the second to Contents contains cattle. Those images containing objects identified as cattle will now vanish, leaving images containing the word cattle still listed.

To understand the difference between these, you can save those two Find windows and read the underlying terms used for each search. The search that returned text obtained by both Live Text and Visual Look Up used
(((** = "cattle*"cdw)) && (_kMDItemGroupId = 13))
while the one excluding Visual Look Up used
(((kMDItemTextContent = "cattle*"cdw)) && (_kMDItemGroupId = 13))
instead. We can transfer those to the mdfind command tool to explore further.

mdfind

To use those as search predicates with mdfind we’ll translate them into more general form,
mdfind "(** == 'cattle*'cdw) && (kMDItemContentTypeTree == 'public.image'cd)"
should return both Live Text and Visual Look Up, while
mdfind "(kMDItemTextContent == "cattle*"cdw) && (kMDItemContentTypeTree == 'public.image'cd)"
only returns Live Text results.

The term (** == 'cattle*'cdw) has a special meaning because of its wild card **, and will return any match found in the metadata and contents of files. kMDItemTextContent is similar, but confined to text content, which doesn’t include the names of objects recognised in an image, and Apple doesn’t reveal whether there’s an equivalent that does.

Code search

Although apps can call mdfind to perform searches, they normally use NSMetadataQuery with an NSPredicate instead. That isn’t allowed to use a predicate like
** ==[cdw] "cattle"
so can’t search for objects identified using Visual Look Up. When it uses the substitute of
kMDItemTextContent ==[cdw] "cattle"
it also fails to find text obtained using Live Text. So the only way to search for recovered text in a compiled app is to call mdfind.

Timing

Searching for text obtained using Live Text, or object labels obtained using Visual Look Up, depends entirely on those being added to the Spotlight indexes on the volume. Observations of the log demonstrate just how quickly normal indexing by mdworker processes takes place. Here’s an example for a screenshot:
06.148292 com.apple.screencapture Write screenshot to temporary location
06.151242 [0x6000009bc4b0] activating connection: mach=true listener=false peer=false name=com.apple.metadata.mds
06.162302 com.apple.screencapture Moving screenshot to final location
06.169565 user/501/com.apple.mdworker.shared.1E000000-0600-0000-0000-000000000000 internal event: WILL_SPAWN, code = 0
06.198868 com.apple.DiskArbitration.diskarbitrationd mdworker_shared [7266]:118071 -> diskarbitrationd [380]
06.226997 kernel Sandbox apply: mdworker_shared[7266]

In less than 0.1 second an mdworker process has been launched and granted the sandbox it uses to generate metadata and content for Spotlight’s indexes. Unfortunately, that doesn’t include any Live Text or Visual Look Up content, which are generated separately by mediaanalysisd later. It’s hard to estimate how much later, although you shouldn’t expect to find such recovered text for several hours or days, depending on the opportunities for mediaanalysisd to perform background image analysis for this purpose.

Summary

• Words recovered from images (not those in PDF files, though) and objects recognised in them can be found by Spotlight search. For best results, choose words rather than letter fragments for search terms.
• Global Spotlight gives full access to both types.
• The Finder’s Find gives full access to both only when the search term is entered in the search box as Content contains, otherwise it may exclude objects recognised.
• mdfind can give full access to both types only when using a wildcard term such as (** == 'cattle*'cdw).
• NSMetadataQuery currently appears unable to access either type. Call mdfind instead.
• The delay before either type is added to the volume Spotlight indexes can be hours or days.