How to access files from a local server with infiniband without using IP

I want to start of by saying that I'm very inexperienced with everything I'm doing so please take it easy on me.

I have 4 computers each with Red hat Enterprise Linux server 7.8 installed, they are all connected to a Voltaire grid director 4036 infiniband switch thats running the subnet manager. For simplicity's sake I will call the computers s1-s4. I want s2, s3 and s4 to be able to access and download files from s1 without using IP since I've read that removing the use of IP's can improve performance by up to 20%(correct me on this if I'm wrong). Is it even possible to make file transfers without the use of IP?

Also, since I'm new to this site feel free to tell me if I should have given more information or was unclear about something. If you think I have misunderstood something then you can also feel free to correct me on it.

Thanks

tl;dr: look and stick with NFSoRDMA.

That's some points that needs to be clear. Considering that you're using RHEL 7 as basis you may want to serve files over NFS from s1 to s2, s3 and s4.

Your right about running IP over Infiniband is problem due to performance and latency issues, not to mention the high CPU load, since basically there's no IP offload on Infiniband, everything is done in software. When running pure IPoIB workload you'll probably have a server with 99% wasted CPU cycles on ksoftirq.

With that in mind, there's another issue: NFS, and also other file services, relies on IP networks to properly function.

What you want is an RDMA enabled protocol for file sharing. Usually they still depends on IP, but only for initial handshake, after the connection is stablished, they got upgraded to RDMA, and in your case, using Infiniband. There's also other fabrics that implements RDMA: RoCE and Omni-Path for instance.

For NFS specifically you should look for NFSoRDMA (NFS over RDMA). You didn't say which hardware do you have, but if it's Mellanox you'll need to install MLNX OFED (probably the LTS 4.9 version to support old hardware) and enable NFSoRDMA. I'm not sure if NFSoRDMA is available on the INBOX RHEL 7 OFED implementation. On EL8 is available AFAIK.

You'll still need an up and running IPoIB interface with an IP address, usually ib0, but with proper configuration NFS will only transfer control messages over IP while the bulk traffic will be over RDMA.

Other filesystems, for example, BeeGFS operates the same way. The initialize over IP and upgrades to RDMA. Gluster was an option in the past but RDMA has been removed from it.