If you’ve read any of my articles here about the inner workings of CPU cores in Apple silicon chips, you’ll know I’m no stranger to using the command tool powermetrics to discover what they’re up to. Last week I attempted something more adventurous when trying to estimate how much power and energy are used in a single Visual Look Up (VLU).

My previous tests have been far simpler: start powermetrics collecting sample periods using Terminal, then run a set number of core-intensive threads in my app AsmAttic, knowing those would complete before that sampling stopped. Analysing dozens of sets of measurements of core active residency, frequency and power use is pedestrian, but there’s no doubt as to when the tests were running, nor which cores they were using.

VLU was more intricate, in that once powermetrics had started sampling, I had to double-click an image to open it in Preview, wait until its Info tool showed stars to indicate that stage was complete, open the Info window, spot the buttons that appeared on recognised objects, select one and click on it to open the Look Up window. All steps had to be completed within the 10 seconds of sampling collections, leaving me with the task of matching nearly 11,000 log entries for that interval against sampling periods in powermetrics' hundred samples.

The first problem is syncing time between the log, which gives each entry down to the microsecond, and the sampling periods. Although the latter are supposed to be 100 ms duration, in practice powermetrics is slightly slower, and most ranged between about 116 and 129 ms. As the start time of each period is only given to the nearest second, it’s impossible to know exactly when each sample was obtained.

Correlating log entries with events apparent in the time-course of power use is also tricky. Some are obvious, and the start of sampling was perhaps the easiest giveaway as powermetrics has to be run using sudo to obtain elevated privileges, which leaves unmistakeable evidence in the log. Clicks made on Preview’s tools are readily missed, though, even when you have a good estimate of the time they occurred.

Thus, the sequence of events is known with confidence, and it’s not hard to establish when VLU was occurring. As a result, estimating overall power and energy use for the whole VLU also has good confidence, although establishing finer detail is more challenging.

The final caution applies to all power measurements made using powermetrics, that those are approximate and uncalibrated. What may be reported as 40 mW could be more like 10 or 100 mW.

In the midst of this abundance of caution, one fact stands clear: VLU hardly stresses any part of an Apple silicon chip. Power used during the peak of CPU core, GPU and neural engine (ANE) activity was a small fraction of the values measured during my previous core-intensive testing. At no time did the ten P cores in my M4 Pro come close to the power used when running more than one thread of intensive floating-point arithmetic, and the GPU and ANE spent much of time twiddling their thumbs.

Yet when Apple released VLU in macOS Monterey, it hadn’t been expecting to be able to implement it at all in Intel chips because of its computational demand. What still looks like magic can now be accomplished with ease even in a base M1 model. And when we care to leave our Macs running, mediaanalysisd will plod steadily through recently saved images performing object recognition and classification to add them to Spotlight’s indexes, enabling us to search images by labels describing their contents. Further digging in Apple’s documentation reveals that VLU and indexing of discovered object types is currently limited by language to English, French, German, Italian, Spanish and Japanese.

Some time in the next week or three, when Apple releases macOS Tahoe, we’ll start seeing Apple silicon Macs stretch their wings with the first apps to use its Foundation Models. These are based on the same Large Language Models (LLMs) already used in Writing Tools, and run entirely on-device, unlike ChatGPT. This has unfortunately been eclipsed by Tahoe’s controversial redesign, but as more developers get to grips with these new AI capabilities, you should start to see increasingly novel features appearing.

What developers will do with them is currently less certain. These LLMs are capable of working with text including dialogue, thus are likely to appear early in games, and should provide specialist variants of more generic Writing Tools. They can also return numbers rather than text, and suggest and execute commands and actions that could be used in predictive automation. Unlike previous support for AI techniques such as neural networks, Foundation Models present a simple, high-level interface that can require just a few lines of code.

If you’ve got an Apple silicon Mac, there’s a lot of potential coming in Tahoe, once you’ve jiggled its settings to accommodate its new style.

If there’s one thing you can rely on about the UK weather, it’s rain. Unless you live in that narrow belt of East Anglia officially classed as semi-arid, you’ll be used to rain whatever the season or forecast.

The last time we had a long dry summer was 1976, when much of Northern Europe basked in sunshine from late May until the end of August. This year has proved similar, so here we are again, dry as a bone, banned from using hosepipes except to wash down horses, wondering when the inevitable floods will start. In 1976, dry weather broke but a couple of weeks after the appointment of a Minister for Drought, whose brief was promptly extended to cover the ensuing inundation.

With this shortage of water, it might seem surprising that over the next five years around a hundred new data centres are expected to be built in the UK. These are the data centres we all want to support our AI chatbots and cloud services, but nobody wants in their neighbourhood. No one has explained where all their power and water supplies will come from, although apparently ten new reservoirs are already being built in anticipation.

The best piece of advice we have been given to help our shortage of water is to delete all our old emails and photos. Apparently by reducing what we have stored in the cloud, those data centres won’t get so hot, and will consume less water. Really?

Meanwhile back on planet Earth, last week I was studying the log entries made on behalf of the Apple Neural Engine, ANE, inside my Mac mini’s M4 Pro chip, when it was running local models to support Live Text and Visual Look Up. We now take these features for granted, and maybe aren’t even aware of using them, or of what our Mac’s ANE is doing. Yet every Apple silicon Mac sold over the last five years has the dedicated hardware possessed by only a small minority of PCs. They can, of course, use other hardware including GPUs, well known for their excessive power and cooling demands. For many the only solution is to go off-device and call on some of those data centres, as you do with ChatGPT, Google’s answer engine, and even Elon Musk’s Grok if you really must.

Live Text is a particularly good example of a task that can, given the right hardware, be performed entirely on-device, and at relatively low energy cost. It’s also one that many of us would rather not farm out to someone’s data centre, but keep to the privacy of our own Mac. While it does work surprisingly well on recent Intel Macs, it’s just what the ANE was intended to make sufficiently performant that it can be commonplace. Just over three years ago, before WWDC 2022, I wrote: “But if I had to put my money anywhere, it would be on the ANE working harder in the coming months and years, to our advantage.”

With so many Macs now capable of what seemed miraculous in the recent past, we’re only going to see more apps taking advantage of those millions of ANEs. Developers are already starting to use Apple’s new Foundation Models supported by macOS 26 Tahoe, all of which run on-device rather than in those data centres. In case you’re concerned about the ethics of what this might be unleashing, Apple has already anticipated that in a stringent set of acceptable use requirements, that also apply to apps provided outside the App Store.

Obtaining reliable estimates of the performance and power consumption of the ANE is fraught, but I have measured them during Visual Look Up on an M1 Max (with an H11ANE), and found peak power used was 30-50 mW. According to mot’s comment to that article, when running an inference task intended to push that in an M1 Pro to the maximum, its ANE drew a maximum of 2 W. That’s frugal compared to running equivalent intensive tasks on Performance CPU cores or an Apple silicon GPU, which can readily use more than 1 W per P core.

Can someone suggest that, instead of deleting old emails and photos, we’d be better off running our favourite AI on-device using an Apple Neural Engine? I still don’t think it would do anything to help our current drought, but it could spare us a few of those projected data centres.