Explaining performance irregularities when unpacking .tar.gz archives
Solution 1:
It seems that Windows Defender will scan each new file whenever the program issues a "close" call – which happens synchronously, i.e. the call will not return and the thread will not continue until Defender completes the scan. This was discussed last year in a Linux.conf.au talk:
- "NTFS really isn't that bad" - Robert Collins (LCA 2020)
File system and OS kernel design may also be a factor; since each new file needs its metadata to be written and directory metadata to be updated, operation latency matters more than just the raw disk throughput. (For example, when writing a large file the OS can just issue a bunch of writes at once and collect the results later; but metadata updates might involve reading some data and waiting for the read to complete before it can be written back.)