A surge in high quality research and patent applications has cemented China’s dominance in the industry.

One thing we humans are good at is searching. It’s a task we engage in from a few moments after birth until the time we slip away in death, we search everything around us. Locating and identifying that bird of prey wheeling high, finding the house keys, and that book we mislaid some time last week, meeting the perfect partner, discovering the right job, choosing the best education, looking through a Where’s Wally? or Where’s Waldo? book, and so on. Searching has transformed some into explorers like Christopher Columbus, and was the purpose of the chivalric quest. It’s what researchers in every field do, and thanks to Douglas Adams can be answered by the number 42.

Last week my searching took two new turns.

Spotlight

The first was more of a meta-search, in trying to discover more about the internals of Spotlight. Following the example of Maynard Handley, who has used them so successfully in understanding how M-series CPUs work, I looked through patents that have been awarded to Apple for the work of its search engineers. Yesterday’s slightly fuller history of Spotlight search is one result, and there are more to come in the future as I digest those patents concerned with performing search.

There’s a tinge of irony here, as many of my searches have been conducted using Google Patents, alongside Google Scholar one of the remaining search engines that doesn’t yet use AI and attempt to provide its own answers.

Logs

The other marks a new phase in my quest to get more information from the Unified log. Looking back to my first comment here, I realise how wildly over-optimistic I was when I wrote that it “should make my life a lot easier”, and that “a new version of Console will provide improved features to help us wade through logs.” Nine years later, I look wistfully at what remains of Console and realise how wrong I was on both counts.

When RunningBoard arrived in macOS Catalina, I soon noticed how “its log entries are profuse, detailed, and largely uncensored for privacy.” Since then it has proved garrulous to the point where its apparently ceaseless log chatter is a distraction, and can overwhelm attempts to read other log entries. I suspect it has contributed significantly to those advanced Mac users who now refuse to even try to make sense of the log.

One answer might be to tweak log preferences to shut out this noise, but given the purpose of RunningBoard in monitoring the life cycle of apps, why not try to use the information it provides? To do that, it’s first necessary to understand RunningBoard’s idiosyncratic language of assertions, and its protocols under which they’re acquired. The only way to do that without documentation is by observation: catalogue over 30 of those assertions for an interesting example like Apple’s Developer app, and see what they reveal.

By far the most informative entries from RunningBoard are those announcing that it’s acquiring an assertion, such as
Acquiring assertion targeting [app<application.developer.apple.wwdc-Release.9312198.9312203(501)>:2946] from originator [osservice<com.apple.uikitsystemapp(501)>:748] with description <RBSAssertionDescriptor| "com.apple.frontboard.after-life.subordinate" ID:424-748-2228 target:2946 attributes:[
<RBSDomainAttribute| domain:"com.apple.frontboard" name:"AfterLife-Subordinate" sourceEnvironment:"(null)">
]>

In a log often censored to the point of being unintelligible, this contains frank and explicit detail. The app is identified clearly, with the user ID of 501 and process ID of 2946. The originator is similarly identified as com.apple.uikitsystemapp with its PID of 748, which is confirmed in the middle digits in the Assertion ID. This is explicitly related to FrontBoard and an attribute named AfterLife-Subordinate. There’s not a single <private> to blight this entry, although further knowledge is needed to decode it fully.

Normally to get such information from a running process would require its source code to be instrumented with calls to write log entries, many of which would be lost to <private>, yet RunningBoard seems happy, for the moment, to provide that information freely. You can see what I mean by applying the predicate
subsystem == "com.apple.runningboard" AND message CONTAINS "Acquiring assertion t"
in LogUI, to obtain a running commentary on active apps and processes. Once you’ve identified a relevant assertion, you can focus attention on other log entries immediately prior to that. I will be following this up in the coming week, with fuller instructions and some demonstrations.

Although neither patents nor assertions have the significance of the number 42, in their own ways they show how the art and science of search aren’t dead yet, nor have they succumbed to AI.

Since writing A brief history of local search, I have come across numerous patents awarded to Apple and its engineers for the innovations that have led to Spotlight. This more detailed account of the origins and history of Spotlight uses those primary sources to reconstruct as much as I can at present.

1990

ON Technology, Inc. released On Location, the first local search utility for Macs, a Desk Accessory anticipating many of the features to come in Spotlight 15 years later. This indexed text found in the data fork of files, using format-specific importer modules to access those written by Microsoft Word, WordPerfect, MacWrite and other apps of the day. Those files and their indexed contents were then fully searchable. This required System Software 6.0 or later, and a Mac with a hard disk and at least 1 MB of RAM. It was developed by Roy Groth, Rob Tsuk, Nancy Benovich, Paul Moody and Bill Woods.

1991

Version 2 of On Location was released. ON Technology was later acquired by Network Corporation, then by Symantec in 2003.

1994

AppleSearch was released, and bundled in Workgroup Servers. This was based on a client-server system running over AppleShare networks. September’s release of System Software 7.5 introduced a local app Find File, written by Bill Monk.

1998

Sherlock was released in Mac OS 8.5. This adopted a similar architecture to AppleSearch, using a local service that maintained indexes of file metadata and content, and a client app that passed queries to it. This included remote search of the web through plug-ins working with web search engines, as they became available.

Early patent applications were filed by Apple’s leading engineers who were working on Sherlock, including US Patent 6,466,901 B1 filed 30 November 1998 by Wayne Loofbourrow and David Cásseres, for a Multi-language document search and retrieval system.

1999

Sherlock 2 was released in Mac OS 9.0. This apparently inspired developers at Karelia Software to produce Watson, ‘envisioned as Sherlock’s “companion” application, focusing on Web “services” rather than being a “search” tool like Sherlock.’

2000

On 5 January, Yan Arrouye and Keith Mortensen filed what became Apple’s US Patent 6,847,959 B1 for a Universal Interface for Retrieval of Information in a Computer System. This describes the use of multiple plug-in modules for different kinds of search, in the way that was already being used in Sherlock. Drawings show that it was intended to be opened using an item on the right of the menu bar, there titled [GO-TO] rather than using the magnifying glass icon of Sherlock or Spotlight. This opened a search dialog resembling a prototype for Spotlight, and appears to have included ‘live’ search conducted as letters were typed in.

2001

Karelia Software released Watson.

2002

Mac OS X Jaguar brought Sherlock 3, which many considered had an uncanny resemblance to Watson. That resulted in acrimonious debate.

2005

In preparation for the first Intel Macs, Mac OS X 10.4 Tiger, released in April 2005, introduced Spotlight as a replacement for Sherlock, which never ran on Intel Macs.

Initially, the Spotlight menu command dropped down a search panel as shown here, rather than opening a window as it does now.

2006

On 4 August, John M Hörnkvist and others filed what became US Patent 7,783,589 B2 for Inverted Index Processing, for Apple. This was one of a series of related patents concerning Spotlight indexing. Just a week later, on 11 August, Matthew G Sachs and Jonathan A Sagotsky filed what became US Patent 7,698,328 B2 for User-Directed search refinement.

A Finder search window, precursor to the modern Find window, is shown in the lower left of this screenshot taken from Tiger in 2006.

2007

Spotlight was improved in Mac OS 10.5 Leopard, in October. This extended its query language, and brought support for networked Macs that were using file sharing.

This shows a rather grander Finder search window from Mac OS X 10.5 Leopard in 2009.

2014

Search attributes available for use in the search window are shown here in OS X 10.9 Mavericks, in 2014.

In OS X 10.10 Yosemite, released in October, web and local search were merged into ‘global’ Spotlight, the search window that opens using the Spotlight icon at the right end of the menu bar, accompanied by Spotlight Suggestions.

2015

John M Hörnkvist and Gaurav Kapoor filed what was to become US Patent 10,885,039 B2 for Machine learning based search improvement, which appears to have been the foundation for Spotlight Suggestions, in turn becoming Siri Suggestions in macOS Sierra. Those were accompanied by remote data collection designed to preserve the relative anonymity of the user.

This shows a search in Global Spotlight in macOS 10.12 Sierra, in 2017.

c 2019

Apple acquired Laserlike, Inc, whose technology (and further patents) has most probably been used to enhance Siri Suggestions. Laserlike had already filed for patents on query pattern matching in 2018.

I’m sure there’s a great deal more detail to add to this outline, and welcome any additional information, please.

4 August 2025: I’m very grateful to Joel for providing me with info and links for On Location, which I have incorporated above.