Client sessions Implementation Issues

From Linux NFS

The client forechannel and backchannel functionality is included as of kernel version 2.6.31. However, the version in 2.6.31 deviates from rfc 5661 in several important ways and is considered useful for developers only.

This document lists known issues in that initial implementation, including those which had to be addressed before the NFSv4.1 client could be changed from Developer Only to Experimental, allowing distros to more comfortably include the functionality in their releases.

The issues labelled (A) below have since been addressed, and the NFSv4.1 client is labelled "Experimental" as of 2.6.36.

NFSv4.1 Sessions

Backchannel

Duplicate Reply Cache
- Not yet implemented (B)
  - DRC needs to be implemented before we give the ability to reestablish the backchannel/ connection w/o breaking the session. (B)
- The backchannel currently only implements idempotent operations and operations that can be retried with no side effects.
- As specified in 2.10.6.1.3, we should cache and return NFs4ERR_TOO_BIG_TO_CACHE - on a reply where cache_this is set to TRUE or NFS4ERR_RETRY_UNCACHED_REP if cache_this is set to FALSE. This forces server to re-issue the request. Investigate if implementing is faster than returning the errors (A)
CB_NOTIFY_LOCK
- Implement Feature (C)
Alternate connection for the backchannel
- Not yet implemented (B)
- The backchannel can only currently be bound to the existing forechannel connection.
- BIND_CONN_TO_SESSION (Separate Connection) (B)
  - Not yet implemented.
  - The workaround is for the client to destroy and create a new session to reestablish the backchannel.
- BACKCHANNEL_CTL (B)
  - Not yet implemented
  - Provide alternate Backchannel program number
  - Provide Kerberos (not yet supported) Principals for Backchannel
Sequence Flag Processing
- The client does not yet implement the check on the following callback path related flags
- SEQ4_STATUS_CB_PATH_DOWN (B)
- SEQ4_STATUS_CB_PATH_DOWN_SESSION (B)
- SEQ4_STATUS_BACKCHANNEL_FAULT (B)
  - What errors are fixable? This becomes fixable when we add the ability to have multiple connections per session.
Inspect "Referring triples" to detect race with forechannel
- Section 2.10.6.3
- Not yet implemented (A)
  - Client can mark delegation state for returning - return OK to delegation recall. When the open finishes it immediately returns the delegation.
- Later we'll have to do the same thing for layout_get/ layout_return
Kerberos (B)
- Not yet implemented (B)
- Need to ensure krb5 forechannel with AUTH_SYS (or possibly AUTH_NULL) backchannel works (A)

Slot Management/ Negotiation

None of the following items have yet been implemented

Client CB_RECALL_SLOT (Handles server reducing slots) (A)
Client needs to provide indication of "highest_slotid" and comply with "target" and "enforced highest_slotid" in SEQUENCE OP (B)
Define policy to size slot table (startup, congestion, etc) (C)
Statistics to monitor (B)
Destroy Session when not in use (A-)
Ensure client checks LEASE TIMEOUT after every clientid exchange (A)
- [ Done. This is being done in nfs4_proc_create_session() ]
Adjust to correct max-cachesize? (A)
- Does open require the largest reply size? Is it sufficient to specify enough bytes for an open reply?
- Do we need to cut down our current size request?

Connection Management

Rebind session to a new connection (after loss of connection)
- BIND_CONN_TO_SESSION (B)
- Not yet implemented - we currently destroy the session and create a new one

Session Reestablishment

Need a thorough review of session and state recovery (A)

SessionID Trunking

Increases the I/O pipe and the number of slots

Bind a new connection to an existing session (B)
- BIND_CONN_TO_SESSION (B)
  - Not yet implemented
- Issue SEQUENCE with existing sessionID?

ClientID Trunking

Not yet implemented (B)

State Management

State revocation handling
- Sequence status bits processing (A)
  - SEQ4_STATUS_CB_GSS_CONTEXTS_EXPIRING (B)
  - SEQ4_STATUS_CB_GSS_CONTEXTS_EXPIRED (B)
  - SEQ4_STATUS_EXPIRED_ALL_STATE_REVOKED
    - Set LEASE_EXPIRED flag in state manager to force it to reclaim the locks (A)
    - Propagate error to app - SIGLOST (B)
    - Linux 4.0 tries to reclaim the lock instead of notifying the app - needs to be fixed altogether (B)
  - SEQ4_STATUS_EXPIRED_SOME_STATE_REVOKED
    - Set LEASE_EXPIRED flag in state manager to force it to reclaim the locks (A)
    - Propagate error to app - SIGLOST (B)
    - Linux 4.0 tries to reclaim the lock instead of notifying the app - needs to be fixed altogether (B)
  - SEQ4_STATUS_ADMIN_STATE_REVOKED
    - Same as above for now (B)
  - SEQ4_STATUS_RECALLABLE_STATE_REVOKED
    - Same as above for now (B)
  - SEQ4_STATUS_LEASE_MOVE (B?)
  - SEQ4_STATS_RESTART_RECLAIM_NEEDED (A)
- TEST_STATEID (B)
  - Not yet implemented
- FREE_STATEID (B)
  - Not yet implemented
Verify we use the correct stateid ordering (Section 8.2.4) (A)
Ensure Close with most recent stateid (not v4.1 specific) (A)
Backchannel must check for zero seqid in stateid callbacks (Section 8.2.2) (B)
Verify locks and delegations survive session reestablishment (A)

State Reclaim

RECLAIM_COMPLETE (A)
- Not yet implemented
  - Issue after establishing a new clientid even if the server didn't reboot
  - Code server or pyNFS server to accept request
  - Update wireshark to understand new OP (B)
Lock recovery when eir_server_owner is different (Section 8.4.2.1) (B)
- Only needed for migration???
- Verify client attempts lock recovery when eir_server_scope is same

State Protection

SSV Support (for trunking and reconnection) (B)
- SET_SSV
- GET_SSV
Mach creds (B)

Error Handling Review

Error mapping problems
- Code Inspection (A)
- Change the place in the kernel code where we map the errors? (B)
  - Is it doable? Are we mixing RPC errors and NFS errors?
- Code inspection (A)
pyNFS server changes to accommodate returning random errors (A)
- Framework (A)
- Operations (A)
pyNFS regression tests for ongoing development (B)

COMPOUND and CB_COMPOUND

Correct use of max sizes (A)
- Client should take care to use correct request and response max sizes
  - Known problem where max sizes does not allow for compound operation header
  - Audit client to ensure proper GETFH usage after FH modifying ops (Section 2.10.6.4) (B)
Mount negotiation
- Verify server allowed values in CREATE_SESSION is reasonable for us to proceed
- Client requests reasonable values, then checks for the bare minimum (A)

Minor Version Negotiation

Drop down to lower version if failed v4.1 mount (B)
Pass something to user-land specifying the error (A)
- [ Done. EPROTONOSUPPORT is already being returned to user-land ]

Misc Functionality

File Delegations
- OPEN4_SHARE_ACCESS_WANT_NO_DELEG (C)
  - Define Policy/ Implications/ When to use each kind? (C)
- CLAIM_DELEG_CUR_FH (C)
- WANT_DELEGATION (C)
- CB_PUSH_DELEG (C)
- CB_RECALLABLE_OBJ_AVAIL (C)
- CB_GETATTR (A)
  - [ done. Checked the spec and CB_GETATTR is the same for 4.0 and 4.1. ]
- CB_NOTIFY (B)
- CB_RECALL_ANY (A)
  - Can we track the number of processes that have the delegation so we don't return the most "popular" one. (B)

Directory Delegations
- Do they really buy us anything investigation (C)
- GET_DIR_DELEGATION (C)

Security, Kerberos and RPCSEC_GSS
- SECINFO (B)
  - Necessary for migration in the future
- SECINFO_NO_NAME (B)

Implementation ID (B)

[ AB: I have implemented a first version of this, queued up for submission upstream]

Named Attributes (C)
- Not yet supported in v4 either

Persistent Session
- OPEN - EXCLUSIVE4_1 (A)
- GUARDED (if persistent session) (A)
- Check for Persistent Session flag during CREATE_SESSION response (A)

OPEN4_RESULT_PRESERVE_UNLINKED (B)

Lock Notification
- OPEN4_RESULT_MAY_NOTIFY_LOCK (open flag) (B)
- CB_NOTIFY_LOCK (B)

ACL changes: dacl, sacl, inheritance (C)

Data Retention (C)

Client sessions Implementation Issues

From Linux NFS

Contents

Legend