Does NFS and SMB support sparse files?

This question was previously asked in stack overflow but the good folks there have recommended that i try the community over here instead.

I am researching on sparse files with regards to various filesystems and am trying to find something concrete that states that sparse files is supported by Network File Systems (NFS) or Server Message Block (SMB).

I understand that SMB is widely used in Windows and that according to this entry, an SMB server can support sparse file even if the underlying file system does not. However, if i am right, the file system that does not support sparse files would just fill the 'holes' with zeroes and this could lead to a performance problem.

With regards to NFS, i have not been able to find out anything about using NFS supporting sparse files.

Hence, my questions is,

Are sparse files supported in NFS and SMB ?

Solution 1:

NFS: it has a partial support for sparse file. Basically, it supports creating a sparse file but, when reading, the file is expanded to include zeroes. This means that, while you can create a sparse file via NFS, when reading back that very same file the in-transit network data will include any zeroes found on the original file. A simple test show that behavior:

cd /mnt/nfs
truncate test.img -s 1G
ls -lh test.img

-rw-r--r--. 1 root root 1.0G Oct 26 11:29 test.img

du -hs test.img

0 test.img

As you can see, the test.img file has an on-disk size of 0 bytes. However, reading back it using dd if=test.img of=/dev/null bs=1M iflag=direct shows

1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 10.2269 s, 105 MB/s

It is clear that when transferring the sparse file, it is expanded to include all zeros.

NFSv4.2 will expand by including special handling for network transfer of sparse file. In other word, with NFSv4.2 the above dd will complete almost instantly.

SMB: it has the same behavior as NFS, at least in my test environments, using a Samba v3.6.x server with CIFS v1 and a Linux client using mount.cifs. Maybe under Windows it behave differently...

Solution 2:

NFS

Yes, NFS 4.2 fully supports sparse files (see this canonical document and this presentation).

Prior to NFS 4.2, the NFS client/server model supported sparse files in the sense that the API supported all POSIX file operations. This meant that writing sparse files on a server which supported sparse files on the backing file system resulted in a sparse file being created (rather than storing lots of zeros). But reading the file would result in the transmission of a lot of zeroes for the sparse element. IE the answer is 'partially'.

NFS 4.2 adds the ability for the client to 'see' holes in the files, and therefore for the server to not have to transmit all those zeroes. From the ID:

1.4.3.  Sparse Files

Sparse files are ones which have unallocated or uninitialized data
blocks as holes in the file.  Such holes are typically transferred as
0s during I/O. READ_PLUS (see Section 15.10) allows a server to send
back to the client metadata describing the hole and DEALLOCATE (see
Section 15.4) allows the client to punch holes into a file.  In
addition, SEEK (see Section 15.11) is provided to scan for the next
hole or data from a given location.

Despite the fact the specification supports sparse files, it would be possible for a lazy implementor to avoid implementing support for sparse files in either the client or the server.

SMB

I know less about SMB, but I believe it does support sparse files too, if the relevant FS capability bit is set. See here for more info.