Get better performance from network storage
The performance of local drives is complicated enough, but networked storage is even worse. Just over three months ago I published a brief guide to what you should expect from several types of storage, including a little information from network-attached storage (NAS) systems. This article adds more data points to that, to help you decide which NAS to use.
Performance will vary across different NAS manufacturers and models, and figures here are based on those measured on a range of products, with a single system providing those for 10 GbE. However, I believe these should be realistic targets that all better models should be able to achieve if not exceed. All connections were made using recent versions of macOS with their respective SMB support, and no special features like iSCSI were used, to ensure these should be readily achievable during backup and file sharing.
This table compares performance of two types of local storage, hard disks and SSDs, connected via USB 3.2 Gen 2 at 10 Gb/s, with those achieved with five different NAS configurations, all using SMB. Hard disks in NAS systems were configured in RAID 1 (mirror) arrays, while those for SSDs are individual or as JBOD. Write speeds are given for:
- the single 50 MB write test performed by Time Machine before each backup;
- 500 multiple concurrent writes of 4 KB each, performed in those same Time Machine tests;
- calculated net write speed over a first full backup to APFS of at least 180 GB;
- general write speed measurement using my app Stibium, which gives broadly similar results to other leading benchmarking apps.
General read speeds are also obtained using Stibium, and similar to other apps. All speeds are given as MB/s for consistency.
Storage medium
As you’d expect, hard disks are substantially slower than SSDs, although their difference isn’t as clear-cut when the effects of connection speed are taken into account. For example, overall backup speed to SSDs over 2.5 GbE was significantly slower than that to hard disks over 10 GbE.
Connection speed
The other dominant effect is that of connection speed, and the use of SMB over Ethernet. These are clearest when comparing NVMe SSD performance.
In general read/write testing, a local SSD achieves close to the maximum expected performance of 10 Gb/s at just under 1 GB/s. With 10GbE and SMB, write performance is about 60% of that, but 115% for reading. General performance of 2.5GbE is roughly a quarter of that of 10GbE and a local SSD.
Thus, overall performance is determined by a combination of medium, connection speed and type, each of which is rate-limiting. To get best performance improvement, you need to use both a fast storage medium (SSD) and fast networking (10GbE), in a NAS capable of delivering such good performance (processor, memory).
I/O Throttling
Behind all these performance figures lurks the mystery of how much they are affected by throttling by macOS. There would seem little point in spending good money for a higher-end NAS with 10GbE support, buying expensive network switches, and filling that NAS with costly SSDs, if macOS were then to throttle away those gains.
As I’ve explained, prior to changes in the documentation in 2019, I/O policy on throttling explicitly excluded remote volumes mounted through networks, but they’re now explicitly included in currently policy. However, throttling only comes into play when higher-priority I/O is competing for a share of the same disk bandwidth, and it isn’t a general restriction in transfer rate. There’s also evidence, presented there, that macOS runs first full backups at higher Quality of Service (QoS) to allow them to complete faster than scheduled automatic backups.
Unless a client Mac is simultaneously accessing a share on the same NAS or network, it therefore appears unlikely that I/O throttling should significantly reduce the speed of backups.
Recommendations
To improve performance of networked storage, particularly in Time Machine backups to a NAS:
- Replace the NAS with a system designed for high performance, with processor, memory, 10GbE network connections and SSD slots capable of delivering that.
- Upgrade your network to support 10GbE between clients and NAS.
- Add SSDs initially to support caching for hard disk arrays, and eventually as primary backup storage.
Although suitable NAS systems will cost upwards of $/€/£ 1000, they should provide support for ample clients for years to come, and prove a worthwhile investment. Potential for improvement is greater than an order of magnitude: backups that previously had to be left running overnight, taking several hours, could take minutes, such as 180 GB in 8 minutes.
My sincere apologies – the table was omitted from the original version of this article. I have now restored that and hope it makes better sense as a result.