Normal view

There are new articles available, click to refresh the page.
Before yesterdayMain stream

A more detailed history of Spotlight

By: hoakley
2 August 2025 at 15:00

Since writing A brief history of local search, I have come across numerous patents awarded to Apple and its engineers for the innovations that have led to Spotlight. This more detailed account of the origins and history of Spotlight uses those primary sources to reconstruct as much as I can at present.

1990

ON Technology, Inc. released On Location, the first local search utility for Macs, a Desk Accessory anticipating many of the features to come in Spotlight 15 years later. This indexed text found in the data fork of files, using format-specific importer modules to access those written by Microsoft Word, WordPerfect, MacWrite and other apps of the day. Those files and their indexed contents were then fully searchable. This required System Software 6.0 or later, and a Mac with a hard disk and at least 1 MB of RAM. It was developed by Roy Groth, Rob Tsuk, Nancy Benovich, Paul Moody and Bill Woods.

1991

Version 2 of On Location was released. ON Technology was later acquired by Network Corporation, then by Symantec in 2003.

1994

AppleSearch was released, and bundled in Workgroup Servers. This was based on a client-server system running over AppleShare networks. September’s release of System Software 7.5 introduced a local app Find File, written by Bill Monk.

1998

Sherlock was released in Mac OS 8.5. This adopted a similar architecture to AppleSearch, using a local service that maintained indexes of file metadata and content, and a client app that passed queries to it. This included remote search of the web through plug-ins working with web search engines, as they became available.

Early patent applications were filed by Apple’s leading engineers who were working on Sherlock, including US Patent 6,466,901 B1 filed 30 November 1998 by Wayne Loofbourrow and David Cásseres, for a Multi-language document search and retrieval system.

1999

Sherlock 2 was released in Mac OS 9.0. This apparently inspired developers at Karelia Software to produce Watson, ‘envisioned as Sherlock’s “companion” application, focusing on Web “services” rather than being a “search” tool like Sherlock.’

2000

On 5 January, Yan Arrouye and Keith Mortensen filed what became Apple’s US Patent 6,847,959 B1 for a Universal Interface for Retrieval of Information in a Computer System. This describes the use of multiple plug-in modules for different kinds of search, in the way that was already being used in Sherlock. Drawings show that it was intended to be opened using an item on the right of the menu bar, there titled [GO-TO] rather than using the magnifying glass icon of Sherlock or Spotlight. This opened a search dialog resembling a prototype for Spotlight, and appears to have included ‘live’ search conducted as letters were typed in.

2001

Karelia Software released Watson.

2002

Mac OS X Jaguar brought Sherlock 3, which many considered had an uncanny resemblance to Watson. That resulted in acrimonious debate.

2005

In preparation for the first Intel Macs, Mac OS X 10.4 Tiger, released in April 2005, introduced Spotlight as a replacement for Sherlock, which never ran on Intel Macs.

Initially, the Spotlight menu command dropped down a search panel as shown here, rather than opening a window as it does now.

2006

On 4 August, John M Hörnkvist and others filed what became US Patent 7,783,589 B2 for Inverted Index Processing, for Apple. This was one of a series of related patents concerning Spotlight indexing. Just a week later, on 11 August, Matthew G Sachs and Jonathan A Sagotsky filed what became US Patent 7,698,328 B2 for User-Directed search refinement.

A Finder search window, precursor to the modern Find window, is shown in the lower left of this screenshot taken from Tiger in 2006.

2007

Spotlight was improved in Mac OS 10.5 Leopard, in October. This extended its query language, and brought support for networked Macs that were using file sharing.

This shows a rather grander Finder search window from Mac OS X 10.5 Leopard in 2009.

2014

Search attributes available for use in the search window are shown here in OS X 10.9 Mavericks, in 2014.

In OS X 10.10 Yosemite, released in October, web and local search were merged into ‘global’ Spotlight, the search window that opens using the Spotlight icon at the right end of the menu bar, accompanied by Spotlight Suggestions.

2015

John M Hörnkvist and Gaurav Kapoor filed what was to become US Patent 10,885,039 B2 for Machine learning based search improvement, which appears to have been the foundation for Spotlight Suggestions, in turn becoming Siri Suggestions in macOS Sierra. Those were accompanied by remote data collection designed to preserve the relative anonymity of the user.

spotlighticloud

This shows a search in Global Spotlight in macOS 10.12 Sierra, in 2017.

c 2019

Apple acquired Laserlike, Inc, whose technology (and further patents) has most probably been used to enhance Siri Suggestions. Laserlike had already filed for patents on query pattern matching in 2018.

I’m sure there’s a great deal more detail to add to this outline, and welcome any additional information, please.

4 August 2025: I’m very grateful to Joel for providing me with info and links for On Location, which I have incorporated above.

A brief history of local search

By: hoakley
7 June 2025 at 15:00

Spotlight, the current search feature in macOS, does far more than find locally stored files, but in this brief history I focus on that function, and how it has evolved as Macs have come to keep increasingly large numbers of files.

Until early Macs had enough storage to make this worthwhile, there seemed little need. Although in 1994 there were precious few Macs with hard disks as large as 1 GB, networks could provide considerably more. That year Apple offered its first product in AppleSearch, based on a client-server system running over AppleShare networks, and in its Workgroup Servers in particular. This was a pioneering product that was soon accompanied by a local app, Find File, written by Bill Monk and introduced in System 7.5 that September.

Sherlock

The next step was to implement a similar architecture to AppleSearch on each Mac, with a service that maintained indexes of file metadata and contents, and a client that passed queries to it. This became Sherlock, first released in Mac OS 8.5 in 1998. As access to the web grew, this came to encompass remote search through plug-ins that worked with web search engines.

Those were expanded in Sherlock 2, part of Mac OS 9.0 from 1999 and shown above, and version 3 that came in Mac OS X 10.2 Jaguar in 2002. The latter brought one of the more unseemly conflicts in Apple’s history, when developers at Karelia claimed Sherlock 3 had plagiarised its own product, Watson, which in turn had been modelled on Sherlock. Apple denied that, but the phrase being Sherlocked has passed into the language as a result.

Spotlight

Sherlock remained popular with the introduction of Mac OS X, but was never ported to run native on Intel processors. Instead, Apple replaced it with Spotlight in Mac OS X 10.4 Tiger, in April 2005.

Initially, the Spotlight menu command dropped down a search panel as shown here, rather than opening a window as it does now.

A Finder search window, precursor to the modern Find window, is shown in the lower left of this screenshot taken from Tiger in 2006.

Spotlight was improved again in Mac OS 10.5 Leopard, in 2007. This extended its query language, and brought support for networked Macs that were using file sharing.

This shows a rather grander Finder search window from Mac OS X 10.5 Leopard in 2009.

Search attributes available for use in the search window are shown here in OS X 10.9 Mavericks, in 2014.

Spotlight’s last major redesign came in OS X 10.10 Yosemite, in 2014, when web and local search were merged into Global Spotlight, the search window that opens using the Spotlight icon at the right end of the menu bar. With Global Spotlight came Spotlight (then Siri from macOS Sierra) Suggestions, and they have been accompanied by remote data collection designed to preserve the relative anonymity of the user.

This Finder window in OS X 10.10 Yosemite, in 2015, shows a more complex search in progress.

spotlighticloud

This shows a search in Global Spotlight in macOS 10.12 Sierra, in 2017.

searchkey24

Local Search in the Finder’s Find window can now use a wide variety of attributes, some of which are shown here, in macOS 10.13 High Sierra, in 2018. Below are search bars for several different classes of metadata.

searchkey25

Over the years, Spotlight’s features have become more divided, in part to safeguard privacy, and to deliver similar features from databases. Core Spotlight now provides search features within apps such as Mail and Notes, where local searches lack access.

spotlightsteps1

Spotlight’s indexes are located at the root level of each indexed volume, in the hidden .Spotlight-V100 folder. Those are maintained by mdworker processes relying on mdimporter plugins to provide tailored access for different file types. If an mdimporter fails to provide content data on some or all of the file types it supports, those are missing from that volume’s indexes, and Spotlight search will be unsuccessful. This happened most probably in macOS Catalina 10.15.6, breaking the indexing of content from Rich Text files. That wasn’t fixed until macOS Big Sur 11.3 in April 2021.

Over the last few years, macOS has gained the ability to perform optical character recognition using Live Text, and to analyse and classify images. Text and metadata retrieved by the various services responsible are now included in Spotlight’s indexes. From macOS 13 Ventura in 2022, those services can take prolonged periods working through images and file types like PDF that include images they can process to generate additional content and metadata for indexing.

Those with large collections of eligible files have noticed sustained workloads as a result. Fortunately for those with Apple silicon Macs, those services, like Spotlight’s indexing, run almost exclusively on their Mac’s E cores, so have little or no effect on its ability to run apps. For those with Intel processors, though, this may continue to be troubling.

In less than 30 years, searching Macs has progressed from the basic Find File to Spotlight finding search terms in text recognised in photos, in almost complete silence. Even Spotlight’s 20th birthday passed just over a month ago, on 29 April, without so much as an acknowledgment of its impact.

❌
❌