Local Spotlight search problems appear common, and are all too often tackled blindly by forcing a volume’s indexes to be rebuilt. Although that can sometimes resolve the problem, without knowing what its cause is, it can just waste time and effort. Indeed, in some cases rebuilding the indexes can worsen the problem, at least temporarily. This article explains how you can use SpotTest 1.1 to perform systematic testing and arrive at a diagnosis before hazarding a guess at what the treatment should be.

1. Spotlight settings

Before opening SpotTest, check that Spotlight settings are in order, and don’t exclude the volume or folder you’re trying to search, or the document type you’re looking for. Then open Activity Monitor and watch its CPU % listing to verify that Spotlight isn’t currently in the process of reindexing or performing other maintenance to its indexes. If it is, delay testing until those have completed. Searching using Spotlight while it’s actively working on its indexes will give odd results, or none at all.

If you’re going to use SpotTest on any location with privacy control, such as your Home Documents folder, or an external disk, consider adding the app to the Full Disk Access list in Privacy & Security settings, before opening it.

2. Home folder test

Even if your interest is in a different volume, you should perform a basic test of a new test folder in your current Home folder, to establish a baseline and confirm that Spotlight is working there.

Open SpotTest and set it to defaults, with a Term of cattle, a Scope of [Home], and both Search Keywords and Search EXIF ticked.

Click the Create Tests tool (the leftmost of the tools) to create the folder of test files at the top level of your Home folder. Make a note of the time you do this to the second.

About 10-15 seconds after that, click either the Run NSMetadata test or Run mdfind test tool. You should then see a list of files found, including those in the test folder ~/0_SpotTestFiles, including A, B, C, D, E, F, G, K, L, M

If you don’t see those listed, open Mints and use the Log Window button in its Spotlight section to obtain a log extract from the time the test files were created, or use LogUI to do the same. You’ll then need to look at that log extract to see if there are clues as to why indexing didn’t take place in the period.

Leave the test folder where it is, and anything from 1 hour to 2 days later, repeat the search using either or both of those tools. Once additional indexing has been undertaken:

• NSMetadata should now find A, B, C, D, E, F, G, I, K, L, M but not H
• mdfind should now find A, B, C, D, E, F, G, H, I, K, L, M.

I is found by the Live Text process, and H by Visual Look Up, the latter only being found by the mdfind search.

These tests have demonstrated:

• mdworker and mds indexing of files supported by system mdimporters;
• delayed background mediaanalysisd image analysis and mds indexing of Live Text and Visual Look Up content.

To match test files with their importers, click the Check importers tool. Note that file L doesn’t use a plugin, and N uses a plugin but can’t be found because the search term is inside an image in the PDF document, which currently isn’t recoverable content.

3. Custom mdimporter

Many apps come with their own custom mdimporter that provides Spotlight indexing support for document types not supported by macOS. In the past, these were normally installed in /Library/Spotlight, but more recent apps typically keep them inside the app bundle in Library/Spotlight. These can be tested easily.

Create and save one of those custom document types, so that it contains the word cattle in a way that should be searchable by Spotlight. Copy that document to the ~/0_SpotTestFiles folder, wait 10-15 seconds, then repeat the test search. You may well notice that NSMetadata search doesn’t find your custom test document, but mdfind does. This is because of the difference in the search criteria they use.

You should also click the Check importers tool to check that the correct mdimporter was recognised and used for the custom document type.

4. Volume test

If Spotlight works correctly with the test folder in your Home folder, you may wish to progress to testing a different volume or location. Having created its test folder in ~/0_SpotTestFiles, copy that to the other location. Before you try to change the Scope of the search, click on the button to list available volumes, then select the volume containing the copied test folder in the Scope menu.

When you perform the two types of search on that volume, the same rules should apply as to which will be found. Note though that finding files I and H can take much longer, or they may not appear at all.

5. Search term test

When you’re confident that a search term of cattle can be found reliably, you may wish to extend your testing to other terms. Take care when choosing custom terms, as you want to be confident that they should be found, but not in such numbers that the results are overwhelming. You will also need to create your own test files containing the custom term.

Diagnosis

SpotTest can thus provide key information on:

• delay or absence of find following creation of test files. If no indexing activity is seen in the log, that suggests indexing failure. If the test files are indexed promptly, it suggests search failure;
• delay or absence in finding files H and I, indicating an indexing failure;
• failure of a custom mdimporter to index a custom document type;
• failure to index another volume.

Those should fit in with the overall scheme used by Spotlight, as shown below.

Happy hunting!

Sometimes even the best-kept Macs start acting strangely, and no matter what you try, you can’t put your finger on the problem, and can’t make it go away. This article suggests some potentially radical solutions that should address the most intransigent of problems in Apple silicon Macs.

Hardware?

If the problem lies in a peripheral, or the Mac’s hardware, then everything else is doomed to fail. Start by disconnecting all non-essential peripherals, and if that doesn’t help, run hardware diagnostics from Recovery mode. Those don’t always catch problems, particularly in their early stages, so if you’re not convinced that your Mac is sound and healthy, book it in for your nearest Apple store or authorised service provider to run their more extensive tests.

To run hardware diagnostics in an Apple silicon Mac, start it up in Recovery by pressing its Power button until it displays that it’s loading options, then in the initial Options screen, hold Command-D until the Diagnostics Loader starts. This may require download of the disk image from Apple’s servers before testing can proceed, so a good Wi-Fi connection is important. Once loaded, there’s a hidden option for extended diagnostics that can be triggered by holding the Command-E key combination.

What to reinstall?

At this stage with an Intel Mac, you’d be considering performing a clean reinstall of macOS, maybe even trying to revert to an older version that didn’t show the problem. Although you can still try that in Recovery mode, Apple silicon Macs have a better and more thorough option, to Restore the whole of your Mac’s firmware and macOS. This is performed by putting it into DFU mode, connected to another Mac (either architecture) running a recent version of macOS, and performing the Restore from there.

Apple provides detailed instructions for you to do this yourself, provided you have the necessary second Mac and cable. If you don’t have those, you should be able to get this performed free of charge at an Apple store, or by an authorised service provider.

The cable used mustn’t be Thunderbolt, but plain USB-C. That’s because DFU mode doesn’t support Thunderbolt or its cable. Connect that to the designated DFU port on the Mac you’re going to Restore. That can be found in Apple’s note, or in Mactracker.

You used to have to run Apple Configurator on the second Mac, but this can now be handled through the Finder, where it’s usually the more reliable. Follow Apple’s instructions to Restore the current version of the firmware and macOS, or you can download an IPSW image file for most previous versions through the links on Mr. Macintosh’s site.

Before performing this, you must make a full backup of your Mac’s internal storage, as the Restore process wipes it clean, and you’ll want to restore from that backup afterwards. As this process is going to wipe your Mac, you’ll also want to check through third-party apps and subscriptions that need to be signed out or transferred. Check carefully through the Applications folder to ensure that you haven’t forgotten any that are still valid. Among those is the need to deauthorise your old Mac for Apple media, something you should do using one of its media apps such as Music or TV.

Revive or Restore?

Apple advises trying to revive your Mac first, as it’s a briefer procedure and doesn’t wipe the whole of the Mac’s internal storage. However, if you’re trying to fix a deep-seated problem, only a full Restore will do.

What does a Restore do?

Internal storage in Apple silicon Macs contains additional partitions/containers to those found in Intel Macs or on external boot disks. These store the firmware and other components used early during the boot process, as part of Secure Boot. A ‘clean’ reinstall only replaces the boot volume group, the Signed System Volume (SSV) and Data volume, while a Restore in DFU mode wipes everything including the firmware, and replaces it with fresh copies from the IPSW file.

The end result is that your Mac is running the firmware to match the version of macOS installed, just as it would have been from the factory. It then has to be personalised and reconfigured from scratch once it’s started up. Nothing from its old firmware, macOS or Data volume is left, even the NVRAM, stored in NOR Flash memory, is reset.

Summary

• Hardware diagnostics in Recovery
• Consider extended diagnostics in Apple store
• Back up and prepare Mac
• Restore in DFU mode, using chosen IPSW if desired
• Restore from backup.