NFS over soft iWARP setup

From Linux NFS

You do not have permission to edit this page, for the following reason:

You can view and copy the source of this page:

Soft iWARP is a software implementation of [https://datatracker.ietf.org/doc/html/rfc5040 RFC 5040], [https://datatracker.ietf.org/doc/html/rfc5041 RFC 5041], and [https://datatracker.ietf.org/doc/html/rfc5042 RFC 5042] that allows RDMA to be used on any Ethernet adapter using TCP. It is available in Linux kernels after 5.4. The following instructions are suitable for most of the latest Linux distributions. = Client and Server Common Setup = * Check for the <code>siw</code> kernel module. If you don't have it, then enable the following Kconfig options and rebuild your kernel CONFIG_INFINIBAND=y CONFIG_RDMA_SIW=m * Install the <code>iproute2</code> package and use the <code>rdma link add</code> command to load the module and start an RDMA interface. If "eth0" is your local Ethernet interface: [foo@linux-bar ~]$ sudo rdma link add siw0 type siw netdev eth0 [foo@linux-bar ~]$ sudo rdma link link siw0/1 state ACTIVE physical_state LINK_UP netdev eth0 The siw virtual device does not survive reboots. Add it to system startup scripts so that it is instantiated automatically after every system restart. == Ping Test == * Start an <code>rping</code> server on one machine [foo@linux-bar ~]$ sudo rping -s -v -C 3 server ping data: rdma-ping-0: ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqr server ping data: rdma-ping-1: BCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrs server ping data: rdma-ping-2: CDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrst server DISCONNECT EVENT... wait for RDMA_READ_ADV state 10 * Now check that the connection works from another machine [foo@linux-bor ~]$ sudo rping -c -a 192.168.122.40 -v -C 3 ping data: rdma-ping-0: ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqr ping data: rdma-ping-1: BCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrs ping data: rdma-ping-2: CDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrst client DISCONNECT EVENT... = NFS Setup = == Server == * Install the <code>nfs-utils</code> package and enable rdma from <code>/etc/nfs.conf</code> and restart service nfs-server [foo@linux-bar]$ sudo sed -i '/rdma/{s/^#//; s/rdma=n/rdma=y/}' /etc/nfs.conf [foo@linux-bar]$ grep -v ^# /etc/nfs.conf [general] [exportfs] [gssd] use-gss-proxy=1 [lockd] [mountd] [nfsdcld] [nfsdcltrack] [nfsd] rdma=y rdma-port=20049 [statd] [sm-notify] [foo@linux-bar]$ sudo mkdir -p /expdir [foo@linux-bar]$ sudo bash -c 'echo "/expdir *(rw,no_root_squash)" >/etc/exports' [foo@linux-bar]$ sudo systemctl restart nfs-server == Client == * Install the <code>nfs-utils</code> package and do nfs mount * Mount the server to check that it worked [foo@linux-bor]$ sudo mount -o rdma 192.168.122.40:/expdir /mnt/nfsmp [foo@linux-bor]$ mount | grep proto=rdma 192.168.122.40:/expdir on /mnt/nfsmp type nfs4 (rw,relatime,vers=4.2,rsize=262144,wsize=262144,namlen=255,hard,proto=rdma,port=20049,timeo=600,retrans=2,sec=sys,clientaddr=192.168.122.161,local_lock=none,addr=192.168.122.40) = See Also = * [https://github.com/linux-rdma/rdma-core/blob/master/Documentation/rxe.md Rdma-core's documentation] * [https://man7.org/linux/man-pages/man8/rdma.8.html The rdma man page]

Return to NFS over soft iWARP setup.