Normal view

There are new articles available, click to refresh the page.
Before yesterdayMain stream

Last Week on My Mac: Losing confidence

By: hoakley
30 November 2025 at 16:00

Cast your mind back to when you learned to drive, ride a bike, speak a foreign language, perform a tracheostomy, or acquire any other skill. Wasn’t confidence the key to your success? Whatever we do in life, confidence is always critical. If you run a business, one of the metrics that are likely to be collected is confidence in your business, as that’s such an important economic indicator. Confidence is every bit as important in computing.

Over the last few weeks I’ve been discovering problems that have been eroding confidence in macOS. From text files that simply won’t show up in Spotlight search, to Clock timers that are blank and don’t function, there’s one common feature: macOS encounters an error or fault, but doesn’t report that to the user, instead just burying it deep in the log.

When you can spare the time, the next step is to contact Apple Support, who seem equally puzzled. You’re eventually advised to reinstall macOS or, in the worst case, to wipe a fairly new Apple silicon Mac and restore it in DFU mode, but have no reason to believe that will stop the problem from recurring. You know that Apple Support doesn’t understand what’s going wrong, and despite the involvement of support engineers, they seem as perplexed as you.

One reason for this is that macOS so seldom reports errors, and when it does, it’s uninformative if not downright misleading. Here’s a small gallery of examples I’ve encountered over the last few years, to bring back unhappy memories.

docprivacy06

recursivertfd01

recursivertfd02

lastweekquar03

sharedfold3

Maybe you saved an important webpage in Safari 26.1 using its Web Archive format, then a couple of days later discovered you couldn’t open it. There’s no error message, just a blank window, so you try again with the same result. Another site shows the same problem, forcing you to conclude that it’s a bug in Safari. Are you now going to devote your time to obtaining sufficient information to report that to Apple using Feedback? Or to contact Apple Support and pursue its escalation to an engineer who might fortuitously discover the cause?

Silent failures like these are least likely to be reported to Apple. In most cases, we find ourselves a workaround, here to abandon Web Archives and switch to saving webpages as PDF instead. When someone else mentions they too have the same problem, we advise them that Web Archives are broken, and our loss of confidence spreads by contagion.

Honest and understandable error reporting is essential to confidence. It enables us to tackle problems rather than just giving up in frustration, assuming that it’s yet another feature we used to rely on that has succumbed in the rush to get the next version of macOS out of the door.

Eroding confidence is also a problem that the vendors of AI appear to have overlooked, or at least seriously underestimated. It’s all very well using the euphemism of hallucination to play down the severity of errors generated by LLMs. But those can only cause users to lose confidence, no matter how ‘intelligent’ you might think your AI is becoming. Go talk to the lawyers who have been caught out by courts submitting AI fabrications whether they still have full confidence in your product.

Last Week on My Mac: The mystery of Safari’s Web Archives

By: hoakley
23 November 2025 at 16:00

It’s both a joy and a curse that so many tell me of bugs they encounter. The joy is that it enables me to investigate and report them here, but the curse is when I can’t reproduce the problem. This week’s curse has been Safari’s webarchives, a topic that I had wisely avoided for several years. Search this blog using the 🔍 tool at the top right of any page and you’ll see just four articles here that mention webarchives, and this is now the second in the last ten days.

While I’m writing about searching this blog, I should point out that tool doesn’t take you out to Google or any general search engine, but confines its scope to articles published here. Although precious few seem to use it, I find it invaluable when preparing articles, and strongly recommend it.

Not only had I avoided tackling this topic, but I see from my own local search that I have seldom used webarchives myself, although not as a result of any unreliability.

In principle, Safari’s webarchives should rarely cause a problem. They’re written by converting what Safari already holds in memory for a webpage into an XML property list, a process termed serialisation, and used effectively by a great many apps in more challenging circumstances. There may be occasions when this doesn’t quite work right, and it does require Safari to retain backward compatibility to ensure it can load and display property lists written some years ago. But by and large it should prove robust.

In practice, there are quite a few who appear unable to get this to work with many versions of Safari, yet I can’t repeat that here. For one reader, the most recent version of Safari that can reliably open their webarchives is 18.6, which is the only version I have experienced problems with. Running in macOS Ventura 13.7.8 here, that version appears unable to open the webarchives it creates, or those from later versions of Safari. Meanwhile Safari 26.1 running in macOS 26.1 has no trouble opening any webarchive I’ve tried from 2009 onwards.

For the last three years, Safari and its supporting libraries including WebKit have been provided to macOS in a cryptex, where they can’t be modified. The only way the user can go beyond Safari’s settings to change its behaviour is using Safari Extensions, which are controlled by Apple. There doesn’t appear to be any way for the user to prevent WebKit and Safari from loading webarchives correctly, intentionally or inadvertently.

Cursed by my inability to reproduce the problems reported, I have immersed myself in a couple of lengthy log extracts. One documents Safari 18.6 failing to open a webarchive it created, the other shows Safari 26.1 successfully opening the same webarchive.

Safari 18.6 seems to have been making good progress opening the webarchive until it came to loading the main frame. It then needed PolicyForNavigationAction before it could go any further:
01.154639 com.apple.WebKit Loading Safari WebKit 0x14c19b818 - [pageProxyID=21, webPageID=22, PID=596] WebPageProxy::decidePolicyForNavigationAction: listener called: frameID=24, isMainFrame=1, navigationID=26, policyAction=0, safeBrowsingWarning=0, isAppBoundDomain=0, wasNavigationIntercepted=0
01.154642 com.apple.WebKit Loading Safari WebKit 0x14c19b818 - [pageProxyID=21, webPageID=22, PID=596] WebPageProxy::receivedNavigationActionPolicyDecision: frameID=24, isMainFrame=1, navigationID=26, policyAction=0
01.154666 com.apple.WebKit Loading Safari WebKit 0x14c19b818 - [pageProxyID=21, webPageID=22, PID=596] WebPageProxy::isQuarantinedAndNotUserApproved: failed to initialize quarantine file with path.
01.154666 com.apple.WebKit Loading Safari WebKit 0x14c19b818 - [pageProxyID=21, webPageID=22, PID=596] WebPageProxy::receivedNavigationActionPolicyDecision: file cannot be opened because it is from an unidentified developer.
01.154799 Error Safari Safari Web view (pid: 596) did fail provisional navigation (Error Domain=NSURLErrorDomain Code=-999 "(null)")

So loading the main frame was halted with those chilling words “file cannot be opened because it is from an unidentified developer”, with which we’re only too familiar. The webarchive was in quarantine, it seems, and that put a stop to its loading. Only that isn’t quite accurate: there was no com.apple.quarantine xattr present, but one of those ubiquitous com.apple.macl xattrs instead. Safari had been stopped by its own security, didn’t even have the courtesy to inform us, and just sat there with an empty window going nowhere.

Safari 26.1 shows how it should have been done:
00.740168 com.apple.WebKit 0xa4bda0718 - [pageProxyID=19, webPageID=20, PID=1035] WebPageProxy::decidePolicyForNavigationAction: listener called: frameID=4294967298, isMainFrame=1, navigationID=25, policyAction=Use, isAppBoundDomain=0, wasNavigationIntercepted=0
00.740172 com.apple.WebKit 0xa4bda0718 - [pageProxyID=19, webPageID=20, PID=1035] WebPageProxy::receivedNavigationActionPolicyDecision: frameID=4294967298, isMainFrame=1, navigationID=25, policyAction=Use
00.740233 com.apple.WebKit 0xa4bda0718 - [pageProxyID=19, webPageID=20, PID=1035] WebPageProxy::receivedNavigationActionPolicyDecision: Swapping in non-persistent websiteDataStore for web archive.

From then, WebKit moves apace and the archived webpage is soon displayed.

This doesn’t of course mean that Safari’s failures to open and display webarchives successfully are all the result of NavigationActionPolicyDecisions that the webarchive can’t be opened because of this security problem, but I suspect this isn’t the only time this has occurred. The vagaries of com.apple.macl xattrs are well known, and their propensity to cause other innocent actions to be blocked is only too familiar. Unfortunately, the only reliable workaround is to knock a hole through macOS security by disabling SIP. But for this to happen without any information being displayed to the user is unforgivable.

Other apps that access Safari’s webarchives don’t appear tainted by this behaviour. Michael Tsai of C-Command Software tells me that his EagleFiler app hasn’t had such problems since its introduction in 2006. If you’ve been struggling to open webarchives in Safari, you might like to consider whether that could address those problems. In the meantime, I can see what I’ll be doing over Christmas.

I’m very grateful to Michael Tsai of C-Command Software for information and discussion.

Last Week on My Mac: Five Tahoe bugs

By: hoakley
16 November 2025 at 16:00

In the early years of this blog, I used to keep track of some of the more serious bugs in macOS. As that developed into what would have occupied me full-time, I’ve cut back to try to cover some of the most significant. What has surprised me with macOS 26.1 is the sudden rush of new bugs in an update that’s normally expected to fix more than it creates. To consider what might have gone wrong, here’s an overview of those I’ve been investigating so far.

macOS virtualisation (new in 26.1)

A macOS 26.1 guest assigns itself a serial number of zero for the VM, whether the VM has been installed from the 26.1 IPSW image file, or updated from a previous version of macOS. This results in features that rely on the VM’s serial number to fail, the most important being access to iCloud.

Further details.

Virtualisation is exceedingly complicated, and has suffered some previous accidents, such as the inability of M4 hosts running macOS 15.1 to virtualise guests with macOS earlier than 13.4. Although it’s easy to claim that better testing should detect these problems, the number of combinations of host Mac and macOS, and guest macOS increases their risk. Perhaps Apple should actively encourage third-party beta-testing in VMs.

Accessibility (new in 26.1)

macOS 26.1 introduces a new Appearance setting for Liquid Glass, but Apple hasn’t mentioned any change to the existing Reduce Transparency setting in Accessibility. However, that setting in 26.1 no longer disables Liquid Glass effects in sidebars and toolbars as it does in 26.0. User documentation for 26.1 is identical to that in macOS 15:
Make transparent items solid
Some windows and areas of the desktop, such as the Dock and menu bar, appear transparent by default. You can turn these transparent areas a solid grey to make it easier to distinguish them from the background.

This can be seen in the following screenshots.

This is 26.0 without Reduce Transparency.

This is 26.0 with Reduce Transparency turned on. Both the navigation sidebar and the window toolbar are completely opaque, and their contents are fully readable as a result.

This is 26.1 with Reduce Transparency turned on. Although the tools themselves are on opaque backgrounds, other areas remain partially transparent, and the toolbar in particular is visually cluttered and impairs accessibility.

Although this could be claimed to be intentional on Apple’s part, one visual feature that now appears when Reduce Transparency is turned on is the unreadable mess at the top of the System Settings window, where its search box overlays scrolling content in that sidebar.

If that’s intentional on Apple’s part, then macOS 26.1 is unsuitable for users with most forms of visual impairment, and many without.

Finder (new in 26.1)

In some Finder views, such as Column View, selecting an item at the left displays that item’s thumbnail and associated metadata. Below those are a selection of tools offering Finder services, such as Rotate Left, Markup, and more. Those are non-functional in 26.1, and if you want to use any of those services, you’ll have to use an alternative method, such as the contextual menu.

Further details.

This is a strange bug, as it doesn’t occur in macOS VMs, suggesting there’s something more complicated going on. However, it’s also obvious, easy to test, and should never have survived into a release version of macOS.

Clock (macOS 15 and 26)

In macOS 15 and 26, including 26.1, the Clock app offers Timers that are implemented using the mobiletimerd service. The latter appears to hoard every past timer in its property list until that grows too large for the service to run, following which the feature fails to function.

Further details.

According to Apple Support, an earlier bug in the mobiletimerd property list was fixed in macOS Sequoia. However, Apple is apparently unaware of the current problem. The current behaviour of mobiletimerd appears to be the result of poor design: if a service keeps adding more items to its property list, that will grow unconstrained, and sooner or later will cause this problem. It’s possible that fixing the previous bug may have resulted in the introduction of a new bug. Either way, this should have been detected long before it was released to the public.

Spotlight indexing (macOS 10.14 and later)

Since macOS Mojave, plain text files starting with certain characters don’t have their content indexed. Those files are correctly assigned to have their contents indexed by the macOS RichText mdimporter, according to their UTI. However, at the start of content indexing the text is checked for its ‘magic’ content. Those files that aren’t indexed because their opening bytes are recognised as being those of other types, and indexing is abandoned because of an error in the mdimporter. Examples of opening UTF-8 characters that can trigger this include the uncommon LG and HPA, and more common Draw.

Further details.

This is the strangest bug among these, as the Rich Text mdimporter is supposed to index content according to the UTI of the file being indexed, which is being recognised correctly. There should be no need to perform another less reliable method of file type recognition using the ‘magic’ rules that is then causing content indexing to fail. That appears to have been introduced over seven years ago, but never tested adequately against a suitable search corpus.

The same mdimporter had suffered another bug that failed to index the content of any Rich Text file that was also undetected for over six months in 2020-21. Without thorough testing of mdimporters, further bugs are likely to occur in release code and remain undetected for long periods.

Conclusions

  • Of these five serious bugs in macOS 26.1, three are new to 26.1, one inherited from macOS 15, and one dates back seven years to macOS 10.14.
  • At least two of the five appear to have been introduced when trying to fix earlier bugs.
  • All five should have become obvious during testing, and none should have remained in any public release of macOS.
  • Both of the bugs that were inherited from macOS 15 appear to reflect flawed design.
  • Only one of the bugs, that in virtualisation, is noted in Apple’s developer release notes for 26.1, and that wasn’t carried forward to its release notes for users.

Acknowledgements

I’m very grateful to Rich Trouton, Michele, Paul, Jürgen, Drew, aldous and others who have provided invaluable information about these bugs.

How the Clock hoards timers until it breaks

By: hoakley
14 November 2025 at 15:30

Sometimes known as Diogenes or Havisham Syndrome, pathological hoarding is not uncommon. Where you wouldn’t expect to see it is in the Clock app bundled in macOS, where it can block its features from working. This article describes this bug that can affect macOS Sequoia and Tahoe. I’m very grateful for the persistent work of Michele, who has contributed much of this information.

Timer failure

Michele uses the Timers feature in the bundled Clock app frequently. Recently it has become temperamental, and now won’t display the contents of that view. He has spent a lot of time working with Apple Support, and trying various fixes, but the only way he has found to restore normal function is to use timers from a different user account.

He sent me two long log extracts collected from the moment he launched the Clock app, one with over 6,000 entries, and the other with more than 25,000. Despite Claude’s imaginary problems, I had been unable to discover anything wrong in either of them. Comparing them against a normal log extract there were no obvious differences or abnormalities.

Then someone suggested that he looked at com.apple.mobiletimerd.plist in ~/Library/Preferences, and removed the whole file. That immediately restored normal timer function, and now his Clock app is working perfectly again.

Service crash

Fortunately, one of the two log extracts he sent me contains the explanation. It transpires the Timers feature in the Clock app relies on mobiletimerd, and just over three seconds into that log record, the Clock app tried to fire up mobiletimerd to help it do its job.

mobiletimerd is a background process that relies on Mach IPC, so was launched by launchd to handle the user’s timers:
03.008036 gui/501/com.apple.mobiletimerd [19118] Successfully spawned mobiletimerd[19118] because ipc (mach)
03.062723 com.apple.mobiletimer.logging mobiletimerd starting...

About 0.03 seconds later, mobiletimerd had exceeded its 15 MB memory allowance. It was therefore terminated, leaving that service inactive, and the Timers view empty:
03.099138 kernel process mobiletimerd [19118] crossed memory high watermark (15 MB); EXC_RESOURCE
03.099148 kernel memorystatus: mobiletimerd [19118] exceeded mem limit: InactiveHard 15 MB (fatal)
03.100180 kernel mobiletimerd[19118] Corpse allowed 1 of 5
03.100567 kernel 54578.846 memorystatus: killing_specific_process pid 19118 [mobiletimerd] (per-process-limit 0 0s rf:- type:daemon) 15360KB - memorystatus_available_pages: 1327431
03.100665 com.apple.opendirectoryd PID: 19118, Client: 'mobiletimerd', exited with 0 session(s), 0 node(s) and 0 active request(s)
03.100679 gui/501/com.apple.mobiletimerd [19118] exited with exit reason (namespace: 1 code: 0x7) - JETSAM_REASON_MEMORY_PERPROCESSLIMIT, ran for 110ms
03.100708 gui/501 [100015] service inactive: com.apple.mobiletimerd

A later attempt to get mobiletimerd running again was delayed for 10 seconds, so occurred after the end of that log extract.

Hoarding

Michele had already discovered the cause of this excessive memory use, as its com.apple.mobiletimerd.plist file was nearly 7 MB. By the time that had been expanded into XML text, that could easily have accounted for 15 MB of memory. At first it looked as if this might have been damage or corruption of that property list, but it turns out that the file is fine, just far too big. So how could its preference settings become so large?

Each time you create and run a timer in the Clock app, mobiletimerd seems to append its details to the com.apple.mobiletimerd.plist file. In addition to arrays of MTAlarms and MTStopwatches, it collects MTTimers for every timer you create and run, but never seems to remove any. Thus the MTTimers list continues growing until mobiletimerd exceeds its memory limit and can no longer be run.

It’s not clear why this property list should store all these MTTimers. They’re not accessible to the user, who is only able to run the tiny subset still displayed in the window. Although none of the information in the property list is particularly sensitive, it does provide a full record of the times that each timer has been run, at least until the file occupies too much memory for the timer to function. It’s possible that mobiletimerd also hoards old MTAlarms and MTStopwatches that could result in similar problems.

Solution

The only workaround for those who use timers often is to periodically remove ~/Library/Preferences/com.apple.mobiletimerd.plist and so restore normal timer function. Although some of the solutions recommended to Michele would unintentionally have achieved that, they would also have involved a lot of wasted effort, and none can bring a permanent solution, so would have to be repeated every time that property list had grown too large.

Thus the only way to address this problem is for Apple to fix the bug. Michele has been told that Apple did fix a bug with that property list in Sequoia, although by the observations above it might have introduced a different bug.

Conclusion

If any part of the Clock app becomes dysfunctional, delete ~/Library/Preferences/com.apple.mobiletimerd.plist to see if that fixes it.

Early bugs in macOS Tahoe 26.1: VMs and Finder Services

By: hoakley
5 November 2025 at 03:47

Here are two reports on bugs affecting macOS Tahoe 26.1 that might make you wait before updating.

macOS VMs can’t access iCloud

If you update a macOS virtual machine to 26.1, or create a new VM in 26.1, it loses its Serial number, which is apparently set as 0. Although most things you might do with a VM aren’t affected by that, it will prevent that VM from being able to access iCloud (if that has been enabled for the VM) and related features.

There’s no workaround for this, as serial numbers are created when the VM is first made, and the user can’t add or change them after that. If your VM needs access to iCloud, keep it running 26.0.1 instead.

I’m very grateful to Rich Trouton of Der Flounder for this information, which is also given in Apple’s developer release notes.

Finder Services

In some Finder views, such as Column View, selecting an item at the left displays the item’s thumbnail and associated metadata. Below those are a selection of tools offering Finder services, such as Rotate Left, Markup, and more. Those are non-functional in 26.1, and if you want to use any of those services, you’ll have to use an alternative method, such as the contextual menu.

I’m very grateful to Michele for letting me know of this.

If you know of any serious bug that is new to 26.1, please let me know and I’ll add it to this list after confirmation.

Last Week on My Mac: Why Spotlight can’t find some files

By: hoakley
26 October 2025 at 16:00

For the last seven years or so there have been many folk complaining that Spotlight local search hasn’t been finding the files they know are there. Many have resorted to repeatedly rebuilding its indexes, usually without success. Last week, thanks to Jürgen, Drew, aldous and others who have contributed, we have discovered one cause. A bug that appears to have been introduced in macOS Mojave, and is still present in Tahoe 26.0.1, that prevents Spotlight from indexing any of the contents of plain text files that start with certain characters.

Jürgen stumbled across the first example, with files starting with the two capital letters LG. At the time, that seemed extremely unusual and unlikely to affect many files. Then Drew added HPA and Draw to the list of forbidden characters. What looked like a rare event was becoming increasingly commonplace, and that list can only grow. How many indexing failures it could account for is impossible to guess.

Piecing together the evidence, it looks like this bug is inside the standard macOS RichText.mdimporter, now embedded in the Signed System Volume in /System/Library/Spotlight and at version 6.9 (350), as it has been since Sonoma (Ventura 13.7.8 has build 345.60.106, although that also suffers this bug). What happens is that saving a text file starting with forbidden characters correctly triggers Spotlight’s indexing service. That identifies the file as having the UTI public.plain-text and hands it over for its contents to be indexed. But the indexer inspects those first few characters, decides it’s a different type of file altogether, and promptly returns an error 4864 for an NSCoderReadCorruptError without going any further.

Apart from the text content not being added to Spotlight’s indexes, and a few lines buried in the Unified log, there are no indications of anything going wrong. If you test the importer using
mdimport -t -d3 filename
the file appears to import correctly, but that command doesn’t give any insight into the import of its contents, only standard attributes such as the filename that are indexed separately.

It was Drew who first suggested a plausible reason for this failure, confirmed by aldous: prior to attempting to index the text contents, Spotlight’s service was using a completely different method to check the type of the contents, the ‘magic’ database used by the file(1) command.

file(1) is an old Unix utility dating back to 1973 or earlier, operating independently of UTIs that were adopted in Mac OS X 10.4 Tiger 20 years ago. Rather than relying on a type assigned to a file, it ‘sniffs’ the contents, particularly the first few bytes of data, and uses a sprawling set of ad hoc rules to guess the file type. It turns out that files starting with the characters Draw were characteristic of a binary vector graphics format used by the !Draw app for RISC OS 2 in 1989. Rather than believing the file’s UTI for one of the most common types of files in macOS, Spotlight’s indexer therefore decided that it was trying to import file data that must now be as rare as hens’ teeth, and wouldn’t go any further.

If you’re sceptical about this coincidence, open the acorn magic data in /usr/share/file/magic in a text editor, and you’ll see the file opening string of Draw identified as RISC OS Draw file data. There are 332 other magic data files containing similar rules for identifying file types. I leave it as an exercise to the Unix wizard to build a list of all those that could cause similar problems with Spotlight indexing.

When this bug hunt started and it affected just LG and HPA, it was fairly esoteric and faintly amusing, at least as long as you didn’t write about your LG TV, high pressure air or Horizontal Pod Autoscaling. When Draw was added, and all those 333 magic files piled in, I realised how extensive this could be, and how little testing can be performed on Spotlight indexing and search.

Given that about eight years ago an Apple engineer wrote code for the RichText.mdimporter in macOS that introduced testing against some or all of the magic database, wouldn’t you have thought they’d test and debug that against test cases, such as text files starting with characters (mis)recognised by magic rules? And maybe occasionally over subsequent years and new versions of macOS, wouldn’t revised versions of the importer be tested again?

Apple likes Spotlight to be opaque to the user, for it to ‘just work’. There’s almost no documentation even for developers, and tools provided are strictly limited in what they can do, as demonstrated here in the case of mdimport. That’s all very well until Spotlight doesn’t work and no one outside Apple can do anything about it. Third-parties can’t even write custom mdimporters to do the job properly, as those bundled in macOS take priority.

If this was the first time that Spotlight indexing had let us down, I might feel more charitable. But between macOS Catalina 10.15.6 in July 2020 and Big Sur 11.3 in April 2021 macOS was incapable of indexing the content of any Rich Text files. There are still many documents that haven’t been indexed as a result. Those whose contents haven’t been indexed as a result of this bug will similarly be excluded from search until they too are reindexed by a fixed mdimporter. For Intel Macs that won’t be supported by macOS 27, that could well be forever.

A Spotlight bug affecting all recent macOS: the LG error

By: hoakley
23 October 2025 at 14:30

There’s a bug in Spotlight that can prevent it from indexing any of the contents of susceptible text files. This has been present since macOS 13 Ventura if not before, and is still present in Tahoe 26.0.1. I didn’t discover this myself, though: it was reported to me by Jürgen, to whom full credit is due. It’s also one of the strangest bugs I’ve come across, and all depends on two letters.

Demonstration

To demonstrate this bug, all you need is a single UTF-8 plain text file, created by TextEdit or any other app capable of saving plain text. Start the text with the two characters L and G, both in capitals. Then add one or more distinctive words, such as
LG syzygy

Save that file to a folder that you know is indexed and searched by Spotlight, then a few seconds later try searching for the word syzygy in its contents. Extend this as much as you want, maybe appending the whole of one of Charles Dickens’ novels, but no matter how you search for its contents, that file will never be found. If you want to get more serious, use that text file in my Spotlight test app SpotTest, and it will also be unable to find that file.

This only works with plain text files, not Rich Text, PDF or HTML. It’s also sensitive to those two letters. Set one of them in lowercase, preface them with a space, or substitute a different letter, and the contents of that file will then be indexed correctly and searchable as normal.

Affected macOS

I have tested this in virtual machines going back as far as macOS 13 Ventura, and it’s present in them all. If you have access to an earlier version of macOS, I’d be interested to know whether it affects that as well.

Cause

The two UTF-8 characters concerned, 4c 47, don’t appear to be anything special that could be misinterpreted.

Although it’s not easy to distinguish failure to index from search errors, saving a test file does result in repeated reports of an error that could cause Spotlight to fail when trying to index the file, for example the log entries
30.946740 mdwrite Decoding error: Error Domain=NSCocoaErrorDomain Code=4864 UserInfo={NSDebugDescription=[private]} for [private]
30.951004 mds Decoding error: Error Domain=NSCocoaErrorDomain Code=4864 UserInfo={NSDebugDescription=[private]} for [private]

Error code 4864 is NSCoderReadCorruptError, implying that the presence of those two characters at the start of a text file may be triggering a bug in RichText.mdimporter, the importer module shipped in macOS that’s responsible for indexing plain text files.

My current hypothesis is therefore that text files starting with the characters LG are failing to have their contents indexed correctly because of a bug in RichText.mdimporter.

History

This isn’t the first bug in the RichText.mdimporter. In macOS Catalina 10.15.6, the same mdimporter (then build 319.60.100) introduced a bug that broke indexing of Rich Text (RTF) files. That was perpetuated through early releases of Big Sur until it was finally fixed in RichText.mdimporter build 326.11 in Big Sur 11.3.

Because text files starting with the characters LG are exceedingly unusual, this bug appears to have been left in RichText.mdimporter for a great deal longer.

I will be reporting this to Apple in Feedback later this month. Please feel free to file your own Feedback if you can spare the time.

Summary

  • In macOS 13 to 26, plain text files starting with the characters LG cannot be searched for their contents.
  • This appears to be the result of a longstanding bug in RichText.mdimporter in macOS.
  • If those characters are altered, or prefixed by a space, indexing and search behave normally.

I’m very grateful to Jürgen for drawing this to my attention.

Emscripten Fetch 接口的一个潜在内存泄漏问题

7 May 2025 at 18:04

近日发现了一个非常刁钻的可能引起基于 Emscripten 编译的 WASM 程序内存泄漏的问题。Emscripten 工具链提供了 Fetch 功能模块,这个模块允许我们调用浏览器的 fetch 接口来进行网络访问。

一个使用 fetch 接口的简单例子是:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
#include <stdio.h>
#include <string.h>
#include <emscripten/fetch.h>

void downloadSucceeded(emscripten_fetch_t *fetch) {
printf("Finished downloading %llu bytes from URL %s.\n", fetch->numBytes, fetch->url);
// The data is now available at fetch->data[0] through fetch->data[fetch->numBytes-1];
emscripten_fetch_close(fetch); // Free data associated with the fetch.
}

void downloadFailed(emscripten_fetch_t *fetch) {
printf("Downloading %s failed, HTTP failure status code: %d.\n", fetch->url, fetch->status);
emscripten_fetch_close(fetch); // Also free data on failure.
}

int main() {
emscripten_fetch_attr_t attr;
emscripten_fetch_attr_init(&attr);
strcpy(attr.requestMethod, "GET");
attr.attributes = EMSCRIPTEN_FETCH_LOAD_TO_MEMORY;
attr.onsuccess = downloadSucceeded;
attr.onerror = downloadFailed;
emscripten_fetch(&attr, "myfile.dat");
}

Fetch API 提供了一些比较高阶的功能,一种一个比较重要的功能是,他可以将下载的内容缓存到 IndexDB 中,这个缓存机制能够突破浏览器自身的缓存大小的限制(一般超过 50MB 的文件浏览器的自动缓存机制会拒绝缓存)。但是这个缓存机制会导致内存泄漏。

1 泄漏产生的过程

在开头的例子中,我们需要再 onerror 和 onsuccess 回调中调用 emscripten_fetch_close 接口来关闭 fetch 指针代表的请求。在关闭过程中,fetch 使用的数据缓存区将会被回收。这个过程如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
EMSCRIPTEN_RESULT emscripten_fetch_close(emscripten_fetch_t* fetch) {
if (!fetch) {
return EMSCRIPTEN_RESULT_SUCCESS; // Closing null pointer is ok, same as with free().
}

// This function frees the fetch pointer so that it is invalid to access it anymore.
// Use a few key fields as an integrity check that we are being passed a good pointer to a valid
// fetch structure, which has not been yet closed. (double close is an error)
if (fetch->id == 0 || fetch->readyState > STATE_MAX) {
return EMSCRIPTEN_RESULT_INVALID_PARAM;
}

// This fetch is aborted. Call the error handler if the fetch was still in progress and was
// canceled in flight.
if (fetch->readyState != STATE_DONE && fetch->__attributes.onerror) {
fetch->status = (unsigned short)-1;
strcpy(fetch->statusText, "aborted with emscripten_fetch_close()");
fetch->__attributes.onerror(fetch);
}

fetch_free(fetch);
return EMSCRIPTEN_RESULT_SUCCESS;
}

可以看到,回收并非总会发生, emscripten_fetch_close 函数会对 fetch 的部分状态进行检查,如果检查失败,则会返回一个 EMSCRIPTEN_RESULT_INVALID_PARAM 的错误码,并且不会执行后续的清理过程(`fetch_free)。被检查的两属性中,fetch->id 是我们需要关注的对象。fetch->id 这个属性作为 fetch 的唯一标识符,是用来建立起 C++ 端的请求对象和 JS 端的请求对象的映射的。id 的值在 JS 端分配。查看源码中的 Fetch.js 文件,

1
2
3
4
5
6
7
8
9
10
11
12
function fetchXHR(fetch, onsuccess, onerror, onprogress, onreadystatechange) {
// ...

var id = Fetch.xhrs.allocate(xhr);
#if FETCH_DEBUG
dbg(`fetch: id=${id}`);
#endif
{{{ makeSetValue('fetch', C_STRUCTS.emscripten_fetch_t.id, 'id', 'u32') }}};

// ...

}

这是唯一的一处 id 复制。这段代码位于 fetchXHR 函数中,这意味着只有发起了 XHR 请求时,id 才会被分配。那么,如果缓存存在呢?这时不会调用 fetchXHR 函数(而是调用 fetchLoadCachedData 函数)。这意味着回调函数中我们试图调用 emscripten_fetch_close 函数来关闭请求并回收资源时,这个回收过程无法进行,这导致了内存泄漏。

2 怎么解决这个问题

要解决这个问题我们只需要强行让 fetch->id == 0 的检查无法通过即可,我们可以在 emscripten_fetch_close 调用前,强行设置 fetch->id 为一个非零值。那么什么值合适呢?如果我们取值和已有的请求的 id 相同,那么 emscripten_fetch_close 可能将那个请求关闭。研究 id 分配的过程(即 Fetch.xhrs.allocate 的实现)

1
2
3
4
5
6
// libcore.js
allocate(handle) {
var id = this.freelist.pop() || this.allocated.length;
this.allocated[id] = handle;
return id;
}

可以看到,id 是顺序分配的,且使用过的 id 会被回收使用(freelist)。因此我们可以设置一个较大的值,只要同一时间最大的并发请求数量不超过这个值,那就是安全的。我一般选择设置为 0xffff。 那么,正确的关闭请求的方式是:

1
2
3
4
if (fetch->id == 0) {
fetch->id = 0xffff;
}
emscripten_fetch_close(fetch);
❌
❌