This section explains how to configure your cluster for a supported export configuration and enable or disable NFSv4.1 on your cluster.
For more information about NFSv4.1 and file access permissions, see Managing File Access Permissions by Using NFSv4.1 Access Control Lists (ACLs).
- Currently, Qumulo Core 4.3.0 (and higher) supports only NFSv4.1. Mounting with version 4.0 or 4.2 isn't supported.
- The NFSv4.1 protocol requires clients to provide the server with globally unique identifiers. By default, the NFSv4.1 client for Linux uses the machine's hostname as
co_ownerid
. Because the NFSv4.1 protocol requires a unique identifier for every client, an unpredictable failure can occur if two clients have the same hostname. To configure unique identification for your NFS clients, set thenfs4_unique_id
value for them. For more information, see The nfs4_unique_id parameter in the Linux kernel user's and administrator's guide.
Configuring and Using Exports for NFSv4.1
Qumulo’s NFS exports can present a view of your cluster over NFS that might differ from the contents of the underlying file system. You can mark NFS exports as read-only, restricted (to allow access only from certain IP addresses), or configure specific user mappings. For more information, see Create an NFS Export on Qumulo Care.
While NFSv3 and NFSv4.1 share each cluster’s NFS export configuration, exports behave differently when you access them by using NFSv4.1. This section explains these differences and the new requirements for export configurations with NFSv4.1.
Differences Between NFSv3 and NFSv4.1 Exports
In the following example, a Qumulo cluster has the following export configuration.
Export Name | File System Path | Read-Only |
---|---|---|
/home |
/home |
No |
/files |
/home/admin/files |
No |
/read_only/home |
/home |
Yes |
/read_only/files |
/home/admin/files |
Yes |
NFSv3 lets you mount one of these exports by specifying the full export name, for example:
mount -o nfsvers=3 \
cluster.example.com:/read_only/home \
/mnt/cluster/home
This command gives read-only access to the /home
directory on the cluster by using the path /mnt/cluster/home
. However, the following command fails with the No such file or directory
message.
mount -o nfsvers=3 \
cluster.example.com:/read_only \
/mnt/cluster/read_only
NFSv4.1 still lets you mount exports by specifying the full export name. However, NFSv4.1 also supports navigating above exports, as if they are part of the file system. The following command succeeds.
mount -o nfsvers=4.1 \
cluster.example.com:/read_only \
/mnt/cluster/read_only
At the mount, the exports under /read_only
are visible: /mnt/cluster/read_only
displays virtual directories named files/
and home/
with the contents of the corresponding directories in the file system, for example:
/mnt/cluster/read_only/
|--- files/<file system contents>
|--- home/
|------ admin/files/<file system contents>
|------ <other file system contents>
This presentation of exports lets you view existing exports by using the file system’s own interface. It also lets you view new exports as soon as someone creates or modifies them without remounting.
Preparing Export Configurations for NFSv4.1
Qumulo’s implementation of NFSv4.1 distinguishes between navigating above exports and inside an export. To avoid confusion between paths that refer to a virtual directory above an export or a real file system directory inside an export, no export name can be a prefix of another export name when NFSv4.1 is enabled.
In the following example, a Qumulo cluster has the following export configuration.
Export Name | File System Path |
---|---|
/ |
/ |
/admin |
/home/admin |
Because /
is a prefix of /admin
, you can’t enable NFSv4.1 with this export configuration. This restriction prevents the situation where the path /admin
can refer to both the export of /home/admin
or the actual file system path /admin
.
To prepare this configuration for NFSv4.1, you can do one of the following:
- Delete the
/
export and use NFSv4.1 presentation of exports when mounting/
. - Delete the
/admin
export. - Give the
/
export a name that doesn’t use other exports as a prefix, for example:
Export Name | File System Path |
---|---|
/root |
/ |
/admin |
/home/admin |
Visibility of IP-Address-Restricted Exports
The names of exports are public to all NFSv4.1 clients, regardless of IP address restrictions. You can’t disable this behavior.
NFSv4.1 respects IP address restrictions on exports: Only clients with allowed IP addresses can access the contents of an export. However, clients without access to an export can still view the export as a directory when they traverse above exports. The restrictions apply only when a client attempts to access the contents of the export.
32-Bit Sanitization
- In NFSv3, you can configure specific exports to return 32-bit sanitized data for individual fields. NFSv3 converts any data larger than 32 bits in configured fields to 32-bit data and returns the data. For example, it can sanitize file size to 32-bit format. This truncates the field to
max_uint32
whenever the NFSv3 server returns the attribute. - NFSv4.1 doesn’t support 32-bit sanitization and ignores any sanitization configured for an export.
Enabling NFSv4.1 on a Qumulo Cluster
Currently, you can enable NFSv4.1 only by using the
qq
CLI.You can enable NFSv4.1 on your Qumulo cluster by using a single cluster-wide configuration command, for example:
qq nfs_modify_settings --enable-v4
When you enable NFSv4.1, all NFS exports are accessible through NFSv3 and NFSv4.1.
Specifying the NFS Mount Option
In Qumulo Core 7.0.0 (and higher), to greatly improve throughput, use the
nconnect=16
option to enable cross-connection write combining.Typically, NFS clients find and use the highest version of the protocol that both the client and server support. For example, the following command mounts by using NFSv4.1 (if it is enabled) or by using NFSv3 otherwise.
mount -t nfs \
-o nconnect=16 \
your.example.cluster:/mount_path \
/path/to/mount_point
Because Qumulo’s NFSv4.1 implementation currently doesn’t have full feature parity with NFSv3, you must provide the nfsvers=3
option for any mounts that require features (such as snapshot access) that only NFSv3 supports, for example:
mount -t nfs \
-o nfsvers=3,nconnect=16 \
your.example.cluster:/mount_path \
/path/to/mount_point
We recommend specifying the
nfsvers=4
or nfsvers=4.1
option for any mounts that use NFSv4.1.Checking Whether NFSv4.1 is enabled
To check whether NFSv4.1 is enabled on your cluster, run the following qq
CLI command:
qq nfs_get_settings
Disabling NFSv4.1 on a Qumulo Cluster
Disabling NFSv4.1 makes any NFSv4.1 mounts unusable immediately. We recommend switching any NFSv4.1 mounts to NFSv3 before disabling NFSv4.1.
To disable NFSv4.1 on an entire Qumulo cluster, run the following qq
CLI command:
qq nfs_modify_settings --disable-v4
Configuring Floating IP Addresses for Nodes
Currently, each Qumulo node is limited to 1,000 clients connected through NFSv4.1 simultaneously. To account for nodes going down, we recommend balancing the number of client connections across your nodes by configuring a sufficient number of floating IP addresses for each node. This prevents a node failover event from overloading the nodes to which the clients might fail over.
For example, if you configure only one IP address for each node, on a cluster with 600 clients for each node, a single node failure might overload one of the remaining nodes, preventing 200 clients from connecting. If you assign multiple floating IP addresses to each node, the clients’ connections are distributed across multiple nodes.
Listing NFSv4.1 Byte-Range Locks
Rather than lock an entire file, byte-range locking lets you lock specific portions of a file or an entire file in use. This feature is available in Qumulo Core 5.1.3 (and higher). It doesn’t require client mount configuration.
The NFSv4.1 implementation in Qumulo Core has a non-configurable lease of one minute. During each lease period, clients send a heartbeat to your Qumulo cluster. The cluster uses this heartbeat to detect lost client connections and to revoke the client leases. When the cluster revokes a lease, it releases any byte-range locks and makes them available to other clients.
- NFSv4.1 byte-range locks are interoperable with NLM (NFSv3) byte-range locks. NFSv4.1 clients view and respect locks that NFSv3 clients hold (the opposite is also true).
- NFSv4.1 and NLM locks aren't interoperable with SMB locks.
To list NFSv4.1 byte-range locks in your cluster, run the following qq
CLI command:
qq fs_list_locks \
--protocol nfs4 \
--lock-type byte-range
- Currently, Qumulo Core doesn't support revoking NFSv4.1 byte-range locks by using the CLI.
- The time to acquire or release a lock scales linearly with the number of locks that the system already holds on a specific file. If a file has a very large number of locks, system performance can degrade.
Supported and Unsupported Features in Qumulo’s Implementation of NFSv4.1
Qumulo’s implementation of NFSv4.1 currently supports:
- Authentication with Kerberos
- General file system access (reading, writing, and navigating files)
- Unstable writes
- Full use of the NFS exports configuration shared with NFSv3
- Navigation in the pseudo-file system above your exports
- NFSv3-style
AUTH_SYS
authentication (also known asAUTH_UNIX
) - Fine-grained control over file permissions by using access control lists (ACLs)
- File locking (for example, by using the
fcntl
command) - Snapshots through NFSv4.1 (Qumulo Core 5.2.4 and higher)
- Quotas through NFSv4.1 (Qumulo Core 5.2.5.1 and higher)
Qumulo Core doesn’t currently support the following NFSv4.1 features:
- Delegations