Last Week on My Mac: How good is AI at solving Mac problems?
I suppose it had to happen that search engines and AI were exploited to deliver malware to the unsuspecting. As that article prompted a brief discussion of the usefulness and reliability of AI-based troubleshooting, I’ve been doing a little checking.
To examine this, I’ve posed Google’s AI some test questions. Rather than run through a long list, I’ve focussed on five that are reasonably frequent but have catches in them. Some are embedded in the question itself, others are inherent in the solution. My aim here isn’t to focus on the strengths of AI, but to understand its weaknesses better, just as you might with a human expert. After all, it doesn’t take much expertise to get the straightforward answers right.
1. How to reduce system data on mac
This question is framed carefully to reveal that the questioner has already used Storage settings, and has been confronted with a great deal of space being used by System Data, an unhelpful category and a situation that’s all too common, as I’ve considered here and elsewhere.
Google’s overview started well, telling me that “System Data includes caches, logs, and temporary files that build up over time and aren’t easily removable like regular documents.” Once it progressed to suggesting actions, though, it repeated a formula it seems to like, but is sadly well out of date: “Use Built-in Storage Management. Click the Apple menu () > About This Mac > Storage > Manage. Use recommendations like ‘Store in iCloud,’ ‘Optimize Storage,’ and ‘Empty Trash Automatically’.” If you try to follow those, you’ll immediately discover that Storage has moved elsewhere. Furthermore, those recommendations won’t tackle the problem framed in the question.
It continues on safer ground, with procedures to clear caches and logs, but those are conducted in Terminal, and there’s no mention of booting in Safe mode. It also directs the user to Terminal to remove snapshots using tmutil rather than Disk Utility. After that, it loses the direction in the question, recommending the user to “remove unused applications and files”, despite the fact that they aren’t included in System Data. Finally, and still off the subject, it mentions using DaisyDisk.
This demonstrates how Google’s AI can’t maintain a logical sequence in troubleshooting, and prefers to direct the user to command tools rather than familiar bundled utilities like Disk Utility, or one of the primary purposes of Safe mode.
2. How to reset home permissions on mac
This has long been a contentious issue, but for the last few years has been fairly settled, as explained here. We no longer reset or repair permissions.
This was Google’s most obvious disaster, as it advised: “To reset home folder permissions on a Mac, boot into Recovery Mode (Command-R), open Terminal, type resetpassword , and use the ‘Reset Home Directory Permissions and ACLs’ button in the utility to fix ownership and permissions for your user account, then restart. For newer macOS versions, Apple also suggests using the repairHomePermissions tool in Recovery, followed by a macOS reinstallation for a full fix, but the resetpassword utility is the primary way to reset the main permissions.”
Of course, Apple silicon Macs don’t use Command-R to enter Recovery Mode, and as you might expect, the resetpassword command opens Recovery Assistant where you can reset your password in the event that you forgot it. That has absolutely nothing to do with permissions, and demonstrates that Google hasn’t understood the question. There’s no such button in that utility, so it’s making things up. It’s also worth nothing that it later recommends that after running repairHomePermissions, you reinstall macOS.
Perhaps the fundamental problem here is the linked support content dating back to 2011, and a failure to recognise how this has changed in the years since. This suggests that its LLM doesn’t take time and change into account, which is deeply concerning when deriving advice on macOS.
3. How to identify clone files in macos
This has been a longstanding problem since the introduction of APFS. Note, though, that question isn’t posed to test whether two or more files are clones of one another, simply how to identify whether files are clones.
Google’s AI Overview is pretty good, and points out that “you need specialized tools or command-line tricks because Finder just sees copies”. However, the next section is titled “Using Finder (for general duplicates)” and gives a facile answer that’s completely inappropriate to that question. This demonstrates how AI always tries to answer, even when it doesn’t know an answer. After that it offers a Terminal solution that again finds duplicates but not clone files, as it doesn’t even check whether the files found have been cloned. It then suggests using specialised apps, including Precize and Sparsity, but lacks useful detail. It ends with pointing out the differences between hard links and clone files, but clearly hasn’t understood a word.
Humans are far more willing to admit they don’t know, and to ask follow-up questions to help them understand exactly what you’re asking.
4. How to run an unsigned app in macos
One of the well-known features of Apple silicon Macs is that, from their first release five years ago, they have only ever run code that has been signed, even if using just an ad-hoc signature, while Intel Macs remain able to run apps and code that has no signature at all. There’s also an important distinction between unsigned code, and code that has been signed by an ad-hoc signature rather than a developer signature.
Those are missed entirely by Google’s AI, as a result of which its answer is riddled with misunderstandings. It recommends what it terms ‘The Standard “Open Anyway”‘ method, which still can’t run unsigned code on Apple silicon. Its final recommendation is to use sudo spctl --master-disable, which disables Gatekeeper and XProtect checks but still doesn’t allow unsigned code to run on Apple silicon.
Given that LLMs are all about language rather than facts or knowledge, it’s surprising that it failed to see the distinction here. This topic was also widely discussed when Apple silicon Macs were introduced, so it’s puzzling that Google was unable to recall any discussion from that time.
5. How to remove com.apple.macl in macos
I’ve only recently revisited this topic, although it dates back to Catalina. This particular extended attribute is frequently added to files, and can have unpleasant consequences when opening or saving them is blocked. Unlike the ordinary quarantine xattr, when macOS applies this one it’s usually protected by SIP, which makes its removal fraught unless you know the trick.
Google AI’s answer made a promising start, writing that “you can use the xattr command in the Terminal, but you might need to use a specific approach depending on your macOS version and file location, as this attribute is often protected by System Integrity Protection (SIP) or file access permissions.” It then ignores the problems posed by SIP protection, and recommends trying the xattr command. As an alternative for “stubborn cases”, it recommends booting into Recovery, and using xattr from there, which should work if you can locate and access the file, which can be quite an achievement in Recovery.
In a bid to remain helpful, it next suggests granting the Terminal app Full Disk Access, although that’s irrelevant. It tries again with: “A common workaround involves moving the file using an application that doesn’t propagate the com.apple.macl attribute, or transferring it to a non-Mac file system.” It finally gets lost when trying to use iCloud Sync.
In common with other answers, Google’s AI started off well, as if it understood the heart of the problem, but quickly demonstrated that it was unable to recall a solution, and stopped making any sense.
Reproducibility
Before you rush off and try the same questions in your favourite AI, a word of warning: the answers you’ll be given will be different from mine, even if you use exactly the same words with Google. This is because randomisation is at the heart of AI, and each time you elicit a response from an LLM, it will differ. Sometimes those differences can be subtle and linguistic, others can manipulate different ‘facts’, or fabricate conflicting answers. This is, apparently, intentional, and hopefully never affects any human expert you consult.
Conclusions
These five questions have demonstrated that Google’s AI can produce some surprisingly accurate information that appears insightful and can match human expertise. In some cases, recommended solutions are sound and well-explained, but in others they appear based on outdated information that may conflict with the opening Overview. Where there aren’t readymade solutions it can quote, it will always try to be helpful in providing an answer, no matter how illogical or flawed that might be. In some cases those could lead an unsuspecting user into danger, and often ignore what was seeded in the original question.
The only way to use Google AI safely is to double-check everything carefully with authoritative sources before trying any of its suggestions, which surely removes much or all of its value.















