How to achieve oflag=direct in dd on MacOS?

I have installed gdd on MacOS.

I figured out both dd and gdd don't support oflag=direct option.

 ‘direct’
      Use direct I/O for data, avoiding the buffer cache.  Note that
      the kernel may impose restrictions on read or write buffer
      sizes.  For example, with an ext4 destination file system and
      a Linux-based kernel, using ‘oflag=direct’ will cause writes
      to fail with ‘EINVAL’ if the output buffer size is not a
      multiple of 512.

^{from dd(1) on Linux}

And the built-in dd doesn't even support conv=fsync.

I am wondering how to achieve oflag=direct in dd or gdd on MacOS?

Solution 1:

How to achieve it depends on what you are trying to do:

If you're just trying to copy data from one disk to another, which is a very common use case for dd, you would typically achieve something similar to oflag=direct simply by using the right block device.

I.e. for example if you're copying like this:

dd if=/dev/disk0 of=/dev/disk1 bs=1m

You can speed it up by using raw disk access like this:

dd if=/dev/rdisk0 of=/dev/rdisk1 bs=1m

Note that the names of the block devices have been prefixed with "r", which stands for a "raw" device.

If you're not copying between block devices like that, there's still a way to get something similar to oflag=direct on Linux:

The reason why it's not implemented the same way as Linux, and thus not directly ported over when porting GNU dd to macOS, is that the macOS kernel simply does not support the O_DIRECT flag for fcntl.

The flag came to the Linux kernel about about 20 years ago as an "inspiration" from the similarly named flag in SGI IRIX. You set it on an file descriptor in order to bypass the normal caching subsystem.

However macOS originally did not have its cache configuration per file descriptor - instead it had it per vnode. Think of this as the difference between having the cache configuration set for a file per se, and having the cache configuration set for a particular instance of an open file. I.e. in the latter case, two different programs could be working on the same file with different cache configurations.

This meant that back then (i.e. 20+ years ago) macOS could not adopt O_DIRECT from SGI IRIX like Linux did, but they instead opted to create an option named F_NOCACHE. This also turns the ordinary caching subsystem off, but does per file instead of per file descriptor. Later on, macOS actually changed the workings of the operating system to have the cache configuration per file descriptor instead of per vnode, so now it's more or less the same thing.

You might ask - what did Linux users do before they got O_DIRECT? - They actually used the same type of "raw" devices as macOS also offers, and that I described above. Instead of prefixing them with "r" they were typically prefixed by "raw", but otherwise it is the same concept.

In practice, if you're just copying disks with dd, it is convenient to just use the raw devices. If you're trying to achieve something else, you probably belong to a niche use case, which isn't directly supported. To me it sounds like the easiest way forward there would be to change the gdd implementation and add a fcntl for F_NOCACHE, similarly to the O_DIRECT one that has been removed from the macOS port.

However, depending on what you're trying to do, you should be aware of slight differences in the semantics of the flags. For example if you want to use O_DIRECT in order to do benchmarking, you need to take additional care. F_NOCACHE is namely different from O_DIRECT in that while it does not add data to the cache, it will read data from the cache if it is already there for some reason. This will increase performance, but that might not be what you want for benchmarking.