Login
Check and diagnose Spotlight problems with SpotTest 1.1
Local Spotlight search problems appear common, and are all too often tackled blindly by forcing a volume’s indexes to be rebuilt. Although that can sometimes resolve the problem, without knowing what its cause is, it can just waste time and effort. Indeed, in some cases rebuilding the indexes can worsen the problem, at least temporarily. This article explains how you can use SpotTest 1.1 to perform systematic testing and arrive at a diagnosis before hazarding a guess at what the treatment should be.
1. Spotlight settings
Before opening SpotTest, check that Spotlight settings are in order, and don’t exclude the volume or folder you’re trying to search, or the document type you’re looking for. Then open Activity Monitor and watch its CPU % listing to verify that Spotlight isn’t currently in the process of reindexing or performing other maintenance to its indexes. If it is, delay testing until those have completed. Searching using Spotlight while it’s actively working on its indexes will give odd results, or none at all.
If you’re going to use SpotTest on any location with privacy control, such as your Home Documents folder, or an external disk, consider adding the app to the Full Disk Access list in Privacy & Security settings, before opening it.
2. Home folder test
Even if your interest is in a different volume, you should perform a basic test of a new test folder in your current Home folder, to establish a baseline and confirm that Spotlight is working there.
Open SpotTest and set it to defaults, with a Term of cattle, a Scope of [Home], and both Search Keywords and Search EXIF ticked.
Click the Create Tests tool (the leftmost of the tools) to create the folder of test files at the top level of your Home folder. Make a note of the time you do this to the second.
About 10-15 seconds after that, click either the Run NSMetadata test or Run mdfind test tool. You should then see a list of files found, including those in the test folder ~/0_SpotTestFiles, including A, B, C, D, E, F, G, K, L, M
If you don’t see those listed, open Mints and use the Log Window button in its Spotlight section to obtain a log extract from the time the test files were created, or use LogUI to do the same. You’ll then need to look at that log extract to see if there are clues as to why indexing didn’t take place in the period.
Leave the test folder where it is, and anything from 1 hour to 2 days later, repeat the search using either or both of those tools. Once additional indexing has been undertaken:
- NSMetadata should now find A, B, C, D, E, F, G, I, K, L, M but not H
- mdfind should now find A, B, C, D, E, F, G, H, I, K, L, M.
I is found by the Live Text process, and H by Visual Look Up, the latter only being found by the mdfind search.
These tests have demonstrated:
mdworkerandmdsindexing of files supported by system mdimporters;- delayed background
mediaanalysisdimage analysis andmdsindexing of Live Text and Visual Look Up content.
To match test files with their importers, click the Check importers tool. Note that file L doesn’t use a plugin, and N uses a plugin but can’t be found because the search term is inside an image in the PDF document, which currently isn’t recoverable content.
3. Custom mdimporter
Many apps come with their own custom mdimporter that provides Spotlight indexing support for document types not supported by macOS. In the past, these were normally installed in /Library/Spotlight, but more recent apps typically keep them inside the app bundle in Library/Spotlight. These can be tested easily.
Create and save one of those custom document types, so that it contains the word cattle in a way that should be searchable by Spotlight. Copy that document to the ~/0_SpotTestFiles folder, wait 10-15 seconds, then repeat the test search. You may well notice that NSMetadata search doesn’t find your custom test document, but mdfind does. This is because of the difference in the search criteria they use.
You should also click the Check importers tool to check that the correct mdimporter was recognised and used for the custom document type.
4. Volume test
If Spotlight works correctly with the test folder in your Home folder, you may wish to progress to testing a different volume or location. Having created its test folder in ~/0_SpotTestFiles, copy that to the other location. Before you try to change the Scope of the search, click on the 
button to list available volumes, then select the volume containing the copied test folder in the Scope menu.
When you perform the two types of search on that volume, the same rules should apply as to which will be found. Note though that finding files I and H can take much longer, or they may not appear at all.
5. Search term test
When you’re confident that a search term of cattle can be found reliably, you may wish to extend your testing to other terms. Take care when choosing custom terms, as you want to be confident that they should be found, but not in such numbers that the results are overwhelming. You will also need to create your own test files containing the custom term.
Diagnosis
SpotTest can thus provide key information on:
- delay or absence of find following creation of test files. If no indexing activity is seen in the log, that suggests indexing failure. If the test files are indexed promptly, it suggests search failure;
- delay or absence in finding files H and I, indicating an indexing failure;
- failure of a custom mdimporter to index a custom document type;
- failure to index another volume.
Those should fit in with the overall scheme used by Spotlight, as shown below.
Happy hunting!
