Explaining performance irregularities when unpacking .tar.gz archives

Solution 1:

It seems that Windows Defender will scan each new file whenever the program issues a "close" call – which happens synchronously, i.e. the call will not return and the thread will not continue until Defender completes the scan. This was discussed last year in a Linux.conf.au talk:

  • "NTFS really isn't that bad" - Robert Collins (LCA 2020)

File system and OS kernel design may also be a factor; since each new file needs its metadata to be written and directory metadata to be updated, operation latency matters more than just the raw disk throughput. (For example, when writing a large file the OS can just issue a bunch of writes at once and collect the results later; but metadata updates might involve reading some data and waiting for the read to complete before it can be written back.)