4.1 RDMA issues
From Linux NFS
Tom Tucker had a look at the spec and suggested:
- We should disable RDMA transports for 4.1 until we get the spec implemented (see below) - On the client side, I suggest we simply deny mount requests (with ENOTSUPP?) when RDMA + NFSv4.1 is requested -- PS: I don't think we should downgrade all 4.1 requests to 4.0 because this could be confusing to users. -- PPS: We could could potentially "downgrade" to TCP, but this could be misleading since the user might think they're using iWARP when in fact they're using TCP.
Detail of questionable value follows:
- Section 2.9.2:
-- No retries on the same connection. Personally, I think this is good policy for TCP as well. We see this problem now on NFSv3 over RDMA. If an RPC is discarded for some reason at the server and the client retries, eventually all credits will be consumed and the connection will effectively hang.
-- Reply cache commentary (also in 2.10.6.2). The reply cache does not (should not) contain transport specific information such as credit counts, RKEY/VA, etc... It doesn't for TCP and I'm not sure what they were thinking here.
- Section 2.10.3.1:
-- Paragraph 4. We just have to be certain that the combined RDMA credit count is greater than or equal the advertised slot count for the session.
-- Paragraph 5: This says that our approach of forcing the back-channel to TCP on an RDMA fore-channel is acceptable.
- Section 2.10.6.2
-- Paragraph 3: I don't think this means anything different for RDMA than for TCP.
-- Paragraph 6: This seems like documentation for a bug someone found. Regardless, transport specific data should not/is not kept in the reply cache and given that previous sections have already stated that replay cache replies MUST be on a new connection, at best, it would be considered redundant.
- Section 2.10.6.4
-- Paragraph 1: I'm not getting the continual parenthetical "especially with RDMA" allusions, however, I don't think the text means anything different for RDMA than it does for TCP.
- Section 2.10.7.1, 2.10.7.2 -- AFAIK, none of the new connection setup parameters are being honored by the RDMA transport. This needs to be implemented. -- Credit recovery on RECALL is not being done.
- Section 2.10.7.3 -- Same problem as above, i.e. we're not doing anything with these connection setup parameters.
- Section 2.10.7.4 -- Nod to the mechanism used for iWARP connection setup, perhaps to avoid confusion over TCP vs. RDMA text in the preceding sections.
Basically, we don't handle any of the struct channel_attrs4 parameters. In my opinion, I think this needs to be implemented before we should claim/offer support for RDMA.
