When you search but Spotlight doesn’t find what you’re looking for, you might wonder whether that’s because the file containing it hasn’t been indexed correctly. This article suggests some methods you can use to investigate this.

Spotlight relies on hidden indexes it maintains at the top level of each volume, in the .Spotlight-V100 folder. When an app or process creates a new file or changes the contents of an existing file, that’s recorded in the volume’s FSEvents database. If that file is in a location that isn’t excluded from Spotlight’s index, an mdworker process should then start adding its contents to the volume indexes. To do this, it first checks what type of file it is, by its UTI. Spotlight then looks up the correct mdimporter for that type, and uses that to generate data that Spotlight adds to that volume’s indexes as a set of attributes.

Recent versions of macOS can extract additional information from certain types of files such as images and PDFs. This includes text recognised within images using Live Text, and object recognition and classification, performed in the background by other services such as photoanalysisd. Those are slower and take significantly more processing, and are normally run after traditional mdimporter extraction.

mdimporter plugins

For many file types, these are provided as part of macOS and stored in the system, in /System/Library/Spotlight on the SSV. Importers for third-party apps may be in /Library/Spotlight or ~/Library/Spotlight, or in the /Library/Spotlight folder inside the app bundle. To discover all mdimporter plugins currently installed, use the command
mdimport -L
although that doesn’t inform you which file types they support.

To get more information, identify a file of the correct document type, normally with that type’s standard filename extension, and use mdimport to discover which mdimporter is used for that type, and what it can extract from that specific file, in a command of the form
mdimport -t -d3 [filepath]
where [filepath] is the path of that file. You can vary the digit used in the -d3 option from 1-3, with larger numbers delivering more detail. For -d3 you’ll first be given the file type and mdimporter used, following which all the data extracted is given according to its Spotlight attributes:
Imported '/Users/hoakley/Documents/SpotTestA.rtf' of type 'public.rtf' with plugIn /System/Library/Spotlight/RichText.mdimporter.
37 attributes returned
followed by a long list.

If you’re still in the dark, the nuclear option is to perform a diagnostic dump similar to sysdiagnose but concentrating on Spotlight, using the command
mddiagnose -f [path]
to write the diagnostic output to the specified path. Further details are given in man mddiagnose.

Mints

My free utility Mints has a whole section devoted to testing and observing Spotlight. This includes creation of a folder of nine test files in the user’s Home folder, carrying out a test search that should find those test files, custom log analysis, and cleaning up test files afterwards. This is all fully documented in the Spotlight check and test section of its Help book.

To start the test, click on the Create Test button at a known clock time, such as 12:30:00. Wait at least ten seconds, say until 12:30:10, then click the Check Search button. Give your Mac a good 20 seconds to complete that, then open the Log Window and view entries from 12:30:00 for a period of 30 seconds, to cover both the indexing of the test files and the search. Once you’re happy that all is well, you can delete the test files.

Test files include:

• RTF rich text
• PDF with embedded text
• plain UTF-8 text
• HTML
• Microsoft Word DOCX
• JPEG image with the search term in its EXIF metadata
• Numbers spreadsheet
• Pages document
• plain UTF-8 text with the search term in a kMDItemKeywords extended attribute.

You can also customise the test files with other formats, provided that they contain the search text syzygy999 that Mints will look for.

Exclusions

Perhaps the most common reason for Spotlight not indexing a file’s contents is that it has been excluded from doing so. This is always worth checking before making the assumption the problem is in Spotlight.

There have been three general methods of excluding folders from indexing and search, although only two of them still work reliably:

• appending the extension .noindex to the folder name (this previously worked using .no_index instead);
• making the folder invisible to the Finder by prefixing a dot ‘.’ to its name;
• putting an empty file named .metadata_never_index inside the folder; that no longer works in recent macOS.

System Settings offers Spotlight Privacy settings in two sections. Items listed under Search results won’t normally prevent indexing of those items, but will block them from appearing in search results. Spotlight’s indexing exclusion list is accessed from the Search Privacy… button, where listed items won’t be indexed at all.

Summary of useful commands

• mdimport -L lists known Spotlight mdimporter plugins.
• mdimport -t -d3 [filepath] reports the mdimporter used for a file, and data extracted from that file.
• mddiagnose -f [path] runs full Spotlight diagnostics.

Like most computers, Apple’s Macs have had five main types of port to allow them to be connected to external devices: low speed intended commonly for keyboards and other input devices, high speed often for connection to external storage, display, network, and audio. This brief history concentrates on the first of those.

The original 128K Macintosh launched in 1984 came with a good range of ports, including two RS-422 serial ports and a DB-19 supporting external floppy disk drives. It had separate mouse and keyboard ports, DE-8 and RJ-11 respectively. In 1986, Apple’s IIGS brought the first implementation of a new type of port, Apple Desktop Bus or ADB, that was introduced to the Mac SE and II the following year, replacing the original mouse and keyboard ports.

ADB

At the time, ADB was unique to Apple’s computers, although it was later adopted by NeXT. This uses a 4-pin mini-DIN connector to hook up a daisy-chain of peripherals. The theoretical maximum speed of ADB is 125 Kb/s, although around 62 Kb/s was closer to that experienced in practice. Among these ADB devices came dongles, used to enforce software copy-protection. Devices on an ADB chain each have an address, defaulting to $2 for keyboards, and $3 for mice.

This is ADB Parser, analysing its traffic. Note both Keyboard and Ext. Keyboard initially shared the original address of 02, and the Mouse has 03.

Perhaps the greatest shortcoming of ADB was that it wasn’t intended to be hot-swappable, and the Mac was supposed to be shut down before any changes were made to its connected ADB devices. Although many of us ignored that, it was at our peril. Occasionally, hot-swapping ADB devices fried a fuse that was soldered into the motherboard; although there was a workaround, for most that required a trip to the local authorised Apple dealer and a replacement motherboard, at considerable expense.

ADB and serial ports underwent the transition from Motorola 68K processors to the PowerPC in 1994, and it was another four years before their replacement came, in USB 1.

USB

The first iMac, released in 1998, was also the first Mac without an ADB port, and came with two USB ports for its mouse, keyboard and other peripherals. The first version of USB had only been introduced in 1996, and brought with it new USB-A connectors, although those were initially complicated by a different USB-B format on printers.

The move to USB opened up many new possibilities for external devices including storage, and hubs quickly became needed to support the profusion. Performance was also greatly improved over ADB, leaping from 62 Kb/s to 12 Mb/s at what was appropriately known as Full Speed. That improved again when USB 2.0 (released in 2000) appeared in the 15-inch iMac, iBook and PowerBook G4, and Power Mac G5 of 2003. These reached 480 Mb/s in what was justifiably called High Speed.

Apple System Profiler here shows five USB devices, each with its own driver, connected to a hub. These include a third-party mouse, and a USB-to-serial adaptor.

Third-party mice and other input devices thrived with USB, although driver support was variable. This is a Microsoft IntelliPoint mouse that was popular.

USB was far more complex than ADB, as seen here in ADB Prober. This is a selection of the data in a basic powered USB hub. One of the lasting mysteries was why some peripherals worked more reliably when connected to powered hubs, while others didn’t work at all unless connected direct to a port on the Mac.

USB continued to develop, and with it a profusion of different plug and socket formats. Several different Mini and Micro versions came to fill our drawers and boxes with their cables and adaptors. By the time of USB 3.0 and its 5 Gb/s SuperSpeed and USB-C format, we were all hoping for a reprieve and the return of simplicity. Those hopes were dashed with USB 3.1 in 2013, and its Gen 1 and Gen 2 with their confusion of different terms.

Bluetooth

The last transition for input devices was to do away with cables (almost) altogether, and in 2017 Apple dropped its last models of wired mice and keyboards. At first their wireless replacements required USB-A ports for their charging cables, and have most recently switched to USB-C. Bluetooth had its origins in 1999-2001 in headsets and mobile phones, and by the time our input devices had gone wireless they were achieving data rates upwards of 1 Mb/s over a short range. Few of us now rely on ports and cables to support our keyboards, mice and trackpads.

Wikipedia

ADB
USB