NFS Configuration: Difference between revisions

From OSNEXUS Online Documentation Site
Jump to navigation Jump to search
m QSTOR-11826: Fix CLI command bugs (live-tested against 10.0.8.110): --filter replaces --client-filter, --rdonly replaces --read-only, share-client-list/share-session-list replace enum, --disable-nfs-snap-dir fixes typo, --disable-nfs-hidden-dir-browsing replaces nonexistent --enable-nfs-snap-browsing, --share-session replaces --session, note NFS mode/Kerberos are WUI-only settings
m Rewrite from the product: correct the two server implementations and how one is selected, the NFSv4 mount path (/export/<share>, not /<share>), the root_squash and secure/insecure defaults, rule-order semantics, and the scale-out port; add the NFS Services Configuration dialog field by field, the generated export lines and their option order, reload-vs-restart, HA failover behaviour, ID mapping and Manage GIDs, and the _snaps/.zfs sub-exports (QSTOR-12352)
Line 1: Line 1:
QuantaStor provides NFS access to [[Network_Shares|Network Shares]] via two server implementations depending on the underlying storage architecture:
[[Category:admin_guide]]


* '''Scale-up (Kernel NFS)''' — Used with ZFS-based storage pools. Managed by the Linux kernel NFS server (<code>nfs-kernel-server</code>). Best for single-node and HA-paired deployments.
QuantaStor provides NFS access to [[Network Shares]] via two server implementations, and which one serves a share is decided by the storage architecture underneath it. This page covers those two servers and how one is selected, the NFS protocol versions on offer, the export configuration QuantaStor generates and where it lives on the appliance, the export options a client access rule can produce, how export changes are applied, NFS across an HA storage pool, how UIDs and GIDs resolve, and how to mount a share from Linux.
* '''Scale-out (NFS-Ganesha)''' — Used with CephFS-based storage pools. Managed by the NFS-Ganesha userspace NFS server. Enables distributed NFS access across a Ceph cluster.


Both modes are configured and managed through the same QuantaStor web interface (WUI) and CLI.
Creating and modifying shares, the '''Add NFS Client Access''' dialog and its checkboxes, quotas, aliases and snapshot management all belong to [[Network Shares]]. This page starts where that one stops: at the server and the client.


__TOC__
{| class="wikitable"
! Section !! Purpose
|-
| [[#The two NFS server implementations|The two NFS server implementations]] || Scale-up versus scale-out, and which storage architecture selects which
|-
| [[#Configuring the NFS service|Configuring the NFS service]] || The NFS Services Configuration dialog, field by field
|-
| [[#NFS protocol versions|NFS protocol versions]] || NFSv3, NFSv4 and the NFSv4 minor versions, and what changes for a client
|-
| [[#The export configuration QuantaStor generates|The export configuration QuantaStor generates]] || The files on the appliance, and the exact export lines written into them
|-
| [[#Client access rules and export options|Client access rules and export options]] || Every export option QuantaStor can emit, and what sets it
|-
| [[#Applying export changes|Applying export changes]] || Reload versus restart, and when a client is interrupted
|-
| [[#Restarting the NFS service|Restarting the NFS service]] || The Restart Storage Services operation and when it is needed
|-
| [[#NFS over an HA storage pool|NFS over an HA storage pool]] || What a client sees when a pool fails over
|-
| [[#How UIDs and GIDs resolve|How UIDs and GIDs resolve]] || ID mapping, Manage GIDs, and AD-backed ownership
|-
| [[#Snapshot access over NFS|Snapshot access over NFS]] || The <code>_snaps</code> and <code>.zfs</code> sub-exports as an NFS client sees them
|-
| [[#Mounting from a Linux client|Mounting from a Linux client]] || A worked <code>mount</code>, the options that matter, and making it persistent
|-
| [[#Ports and firewall|Ports and firewall]] || What has to be reachable, and what QuantaStor does not pin
|-
| [[#Troubleshooting|Troubleshooting]] || The failures that come up most, and what to check
|}


== Client-Side Prerequisites ==
== The two NFS server implementations ==


Install the NFS client utilities on each host that will mount QuantaStor shares.
{| class="wikitable"
 
! !! Scale-up !! Scale-out
=== RHEL / CentOS / Rocky Linux (8, 9) ===
|-
 
| '''Server''' || The Linux kernel NFS server || NFS-Ganesha, a user-space NFS server
<pre>
|-
sudo dnf install -y nfs-utils
| '''Serves shares on''' || ZFS [[Storage Pools|Storage Pools]] || CephFS pools and Ceph object gateway buckets
sudo systemctl enable --now nfs-client.target
|-
</pre>
| '''Service''' || <code>nfs-server.service</code> (<code>nfsd</code>) || <code>nfs-ganesha.service</code> (<code>ganesha.nfsd</code>)
|-
| '''Export configuration''' || {{Code|1=/etc/exports}} || {{Code|1=/etc/ganesha/ganesha.conf}}
|-
| '''Owns TCP/UDP 2049''' || Yes || Only when Scale-out is selected
|}


=== Ubuntu / Debian ===
The setting that chooses between them decides, in practice, '''which of the two servers owns port 2049'''. Selecting '''Scale-up''' leaves the kernel NFS server on 2049 serving both NFSv3 and NFSv4, and configures NFS-Ganesha for the alternate port '''2249''' in NFSv4-only mode so that the two do not contend for 2049 or for the RPC mount service. QuantaStor records that in the header of the generated exports file:


<pre>
<pre style="font-size: smaller">
sudo apt-get install -y nfs-common
## NOTE: Scale-out CephFS NAS exports are presented via NFS-Ganesha on alternate default port 2249 rather than 2049.
</pre>
</pre>


=== SUSE / openSUSE ===
Selecting '''Scale-out''' moves NFS-Ganesha to 2049 with NFSv3 and NFSv4 enabled, and stops, disables and masks the kernel NFS server. '''On a system that also has ZFS Storage Pools, that takes every share on them offline for NFS clients''' -- they remain available over SMB. There is no confirmation prompt for this, and nothing validates the choice against the pools that actually exist, so treat Scale-out as a decision about the whole appliance rather than a per-share option. On a system with no Ceph storage the Ganesha service is left masked and not running.


<pre>
QuantaStor selects Scale-out for you in one situation: when a Ceph file system or Ceph object pool is created on a system that has '''no''' ZFS Storage Pools. If ZFS pools are present it leaves the setting alone and raises a warning alert reading ''"SCALE-UP pool(s) present on local system. Not configuring SCALE-OUT NFS settings."'' Everywhere else the setting is yours to make.
sudo zypper install -y nfs-client
sudo systemctl enable --now nfs-client.target
</pre>


== NFS Protocol Version Settings ==
For the Ceph side of the picture, see [[Scale-out File Setup (ceph)]].


QuantaStor supports NFSv3, NFSv4, or both simultaneously (default). This setting is system-wide and applies to all shares on the appliance.
=== What differs between them ===


'''Navigation:''' System Management → Storage System → (right-click) → Modify Network Share Service Settings
The two servers are not interchangeable, and several things configured per share reach only one of them.
 
'''CLI — view current NFS service configuration:'''
<pre>
# View system info including "Network Share Service Config" section
qs system-get
</pre>
 
'''Note:''' Changing the NFS protocol mode (v3v4, v4, v3) is performed via the WUI — there is no CLI command for this setting.


{| class="wikitable"
{| class="wikitable"
! Mode !! Description
! !! Scale-up (kernel) !! Scale-out (Ganesha)
|-
|-
| <code>v3v4</code> (default) || Both NFSv3 and NFSv4 clients are accepted. Recommended for mixed environments.
| Export path a client mounts || <code>/export/&lt;share&gt;</code> || <code>/&lt;share&gt;</code>, or <code>/s3/&lt;bucket&gt;</code> for an object gateway bucket
|-
|-
| <code>v4</code> || NFSv4 clients only. Required for Kerberos security modes.
| NFSv3 || Available || Not available while the server is on port 2249, because the RPC mount service is left to the kernel server
|-
|-
| <code>v3</code> || NFSv3 clients only. Use when clients do not support NFSv4.
| NFSv3 byte-range locking (NLM) || Available || Disabled -- the shipped configuration sets <code>enable_nlm = false</code>
|-
| Quota reporting over NFS (RQUOTA) || Not offered -- no <code>rquotad</code> is registered with rpcbind || Not offered -- the shipped configuration sets <code>enable_rquota = false</code>, noted in the file as not supported on CephFS
|-
| Kerberos security || Available || Not available; every generated export is <code>sectype = sys</code>
|-
| Read-only, root squash, client filter || Honoured || Honoured
|-
| Full port range, async writes, subtree checks, custom export options, Filter Priority || Honoured || '''Not applied.''' These are accepted and stored, and the WUI still shows them, but they are not written into the Ganesha configuration
|-
| NFSv4 referrals for a Global Namespace || Generated || Not generated
|}
|}


== Controlling NFS Client Access ==
Neither server offers RQUOTA, so <code>quota</code> on a client does not report a share's quota. Read the share quota from the WUI or from <code>[[QuantaStor CLI Command Reference#share-get|qs share-get]]</code>, and per-user and per-group quotas from <code>[[QuantaStor CLI Command Reference#share-quota-list|qs share-quota-list]] --share=&lt;share&gt;</code>. [[Network Shares]] covers how they are set.


Each network share has an independent access control list. By default, shares are accessible from any IP address (<code>*</code>). For production environments, restrict access to specific hosts or subnets.
Hand edits to {{Code|1=/etc/ganesha/ganesha.conf}} survive only outside the <code>EXPORT</code> sections, which QuantaStor regenerates. Put additions in {{Code|1=/etc/ganesha/ganesha.conf.custom}}, which is appended verbatim on every save. Keep the <code>OSNEXUS</code> header comment in place: a <code>ganesha.conf</code> that does not contain that string is treated as foreign and replaced with the QuantaStor default.


=== Adding an NFS Client Entry ===
== Configuring the NFS service ==


'''Navigation:''' Storage Management → Network Shares → (right-click share) → Add NFS Client
{{Navigation|Storage Management &rarr; Network Shares &rarr; Network Share Services &rarr; Configure NFS ''(toolbar)''}}
 
{{Navigation|Storage Management &rarr; Network Shares &rarr; Network Share ''(select + right-click)'' &rarr; Configure NFS Services...}}
 
[[File:nfs_config.png|thumb|right|530px|NFS Services Configuration, at its defaults. The Kerberos Security Settings are greyed out until Enable Kerberos Security is ticked.]]
 
The '''NFS Services Configuration''' dialog holds every NFS setting that applies to a whole storage system. It is a per-system dialog even though it is reached from the Network Shares view -- the '''Storage System''' combo at the top selects which member of the grid you are configuring, and the fields below repopulate from that system's current configuration when you change it.


{| class="wikitable"
{| class="wikitable"
! Field !! Description !! Example
! Setting !! Default !! Notes
|-
| '''Client Filter''' || IP address, CIDR subnet, domain name, netgroup, or <code>*</code> for public access || <code>192.168.10.0/24</code>
|-
|-
| '''Access''' || Read-write (default) or read-only || <code>rw</code>
| '''NFS Server Type''' || Scale-up || Scale-up or Scale-out, as described above.
|-
|-
| '''Root Squash''' || Map root (UID 0) to the anonymous user. Enable for untrusted clients. || Disabled (<code>no_root_squash</code>) by default
| '''NFS Protocol Mode''' || NFS v3/v4 || NFS v3/v4, NFS v3, or NFS v4. See [[#NFS protocol versions|NFS protocol versions]].
|-
|-
| '''Secure Port''' || Require NFS requests to originate from ports < 1024 || Disabled (<code>insecure</code>) by default
| '''Disable NFSv4 /export Browsing''' || off || Per the dialog's own tooltip this blocks creation of the NFSv4 export root (<code>fsid=0</code>), which is what lets a client mount the top of the tree and browse every share. In the default '''NFS v3/v4''' mode QuantaStor generates no root export, so there is nothing for the setting to suppress and it has no effect there.
|-
|-
| '''Async Writes''' || Allow asynchronous write caching. Improves throughput; slight data-loss risk on unclean shutdown. || Disabled (<code>sync</code>) by default
| '''Manage GIDs''' || '''on''' || See [[#How UIDs and GIDs resolve|How UIDs and GIDs resolve]].
|-
|-
| '''Subtree Check''' || Verify that a requested file is within the exported subtree. Adds overhead; rarely needed. || Disabled (<code>no_subtree_check</code>) by default
| '''Enable Kerberos Security''' || off || Enables the two Kerberos Security Settings below and switches '''Default Security Policy''' to <code>kerberos</code>. Clearing it switches the policy back to <code>system</code> and greys both fields out again.
|-
|-
| '''Priority''' || When multiple rules match a client IP, the lowest-numbered rule wins (1 = highest priority, range 1–10) || 1
| '''Kerberos Mode''' || Kerberos Integrity (krb5i) || Kerberos Integrity (krb5i), Kerberos Privacy (krb5p), or Kerberos Client/Server Auth (krb5). These map to the NFS <code>sec=krb5i</code>, <code>sec=krb5p</code> and <code>sec=krb5</code> export options.
|-
|-
| '''Custom Options''' || Additional raw NFS export options, comma-delimited || <code>wdelay,nohide</code>
| '''Default Security Policy''' || <code>system</code> || <code>kerberos</code> or <code>system</code>. The system-wide default that a share uses unless it carries its own override. Its tooltip makes the point that matters: with Kerberos configured you can still have Network Shares that do not require Kerberos security, because each share can override this.
|}
|}


'''CLI:'''
[[File:nfs_server_type.png|thumb|right|530px|NFS Server Type offers exactly two choices. Scale-up leaves the kernel NFS server on port 2049; Scale-out hands 2049 to NFS-Ganesha and masks the kernel server.]]
<pre>
 
# Add read-write access from a subnet (root squash is NFS default)
Two details about the dialog are worth knowing before you read a value off it. The '''Default Security Policy''' actually stored is a third value, <code>inherit</code>, which the dialog never shows -- it renders the resolved policy instead, so a system with Kerberos off displays <code>system</code> while the stored setting is <code>inherit</code>. And '''NFS Server Type''' is read back from the Ganesha configuration on disk rather than from the database, so it always reflects which server currently holds port 2049.
qs share-client-add --share=share1 --filter=192.168.10.0/24
 
There is '''no <code>qs</code> command for any of these settings''' -- they are set from the WUI and the REST API only. You can read the whole configuration back with <code>[[QuantaStor CLI Command Reference#system-get|qs system-get]]</code>:
 
<pre style="font-size: smaller">
                    Nfs Mode: NFSv3+v4
        Nfs4 Kerberos Enabled: false
            Nfs Kerberos Mode: 0
      Nfs4 Browsing Disabled: false
    Default Nfs Security Policy: inherit
              Nfs Server Type: scaleup
              Nfs Manage Gids: true
          Nfsd Service Status: Running
</pre>
 
=== Kerberos ===


# Add read-only access from a second subnet
[[File:nfs_config_kerberos.png|thumb|right|530px|With Enable Kerberos Security ticked, both Kerberos fields become editable and the Default Security Policy flips to kerberos.]]
qs share-client-add --share=share1 --filter=10.0.0.0/8 --rdonly=true


# Add a specific host; disable root squash explicitly
'''Enable Kerberos Security''' turns on the GSS daemons and puts a <code>sec=</code> prefix on the export lines QuantaStor generates. It does '''not''' build the Kerberos environment for you: no keytab is created, and {{Code|1=/etc/krb5.conf}} is not written. QuantaStor says so at the time, raising an alert that reads ''"Secure NFSv4 access on '&lt;system&gt;' will not be enabled until the Kerberos configuration steps are completed including manually editing /etc/krb5.conf to setup the domain/realm(s) for your environment."''
qs share-client-add --share=share1 --filter=192.168.10.50 --options=no_root_squash


# List current NFS client rules for a share
The practical order is therefore to establish Kerberos first and enable the checkbox second. Joining the system to an Active Directory domain is the supported route -- the domain join generates {{Code|1=/etc/krb5.conf}} from a template, creates the machine keytab at {{Code|1=/etc/krb5.keytab}}, and points Samba at the system keytab. See [[Network Shares]] for the '''Join AD Domain''' operation and [[Security Configuration]] for the wider authentication picture.
qs share-client-list --share=share1


# Modify an existing rule (enable async writes)
How the policy resolves for a given share, in order:
qs share-client-modify --share=share1 --filter=192.168.10.0/24 --async=true


# Remove a rule
# Start from the system's '''Default Security Policy'''.
qs share-client-remove --share=share1 --filter=192.168.10.0/24
# If that is <code>inherit</code>, resolve it to <code>kerberos</code> when '''Enable Kerberos Security''' is on and to <code>system</code> when it is off.
</pre>
# If the share carries its own override, that wins.


=== Access Priority ===
The per-share override is the one part of this the CLI does reach: <code>[[QuantaStor CLI Command Reference#share-modify|qs share-modify]] --share=&lt;share&gt; --nfs-security-policy=&lt;inherit|kerberos|system&gt;</code>, with <code>inherit</code> the default. Two consequences follow from how the resolved policy is used:


When multiple client filter rules match a client's IP address, Linux NFS applies the most-specific (longest-prefix) match. Add a more specific rule for a single host to override a broader subnet rule:
* An effective policy of <code>system</code> emits '''no <code>sec=</code> option at all'''. QuantaStor never writes <code>sec=sys</code>; AUTH_SYS is what the server falls back to.
* A per-share policy of <code>kerberos</code> on a system where '''Enable Kerberos Security''' is off is silently ignored, because the <code>sec=</code> prefix is only emitted when Kerberos is enabled system-wide.


<pre>
Only one flavour is offered at a time. QuantaStor writes a single <code>sec=krb5</code>, <code>sec=krb5i</code> or <code>sec=krb5p</code>, never a list, so a client must match the configured mode.
# Specific host gets read-write + no_root_squash (more specific, matched first)
qs share-client-add --share=share1 --filter=192.168.10.100 --options=no_root_squash


# Broader subnet gets read-only
== NFS protocol versions ==
qs share-client-add --share=share1 --filter=192.168.10.0/24 --rdonly=true
</pre>


== NFS Export Options Reference ==
'''NFS Protocol Mode''' is a system-wide radio with three positions, and it writes the <code>vers3</code> and <code>vers4</code> keys in the <code>[nfsd]</code> section of {{Code|1=/etc/nfs.conf}}:


{| class="wikitable"
{| class="wikitable"
! Option !! Default !! Description
! Mode !! <code>[nfsd] vers3</code> !! <code>[nfsd] vers4</code> !! Use it when
|-
|-
| <code>rw</code> / <code>ro</code> || <code>rw</code> || Read-write or read-only access.
| '''NFS v3/v4''' ''(default)'' || <code>y</code> || <code>y</code> || Mixed client estates. Each client's own mount options decide which version it gets.
|-
|-
| <code>no_root_squash</code> / <code>root_squash</code> || <code>no_root_squash</code> || <code>root_squash</code> maps UID 0 on the client to the anonymous UID on the server. Enable for untrusted clients.
| '''NFS v3''' || <code>y</code> || <code>n</code> || Clients that cannot speak NFSv4 and you want to be certain nothing negotiates up.
|-
|-
| <code>sync</code> / <code>async</code> || <code>sync</code> || <code>sync</code> commits writes to disk before replying to the client. <code>async</code> improves write throughput but risks data loss if the server crashes before flushing.
| '''NFS v4''' || <code>n</code> || <code>y</code> || Kerberos, or a policy of NFSv4 only. Removes the need for rpcbind and the NFSv3 helper daemons.
|}
 
'''QuantaStor does not configure the NFSv4 minor versions.''' The <code>vers4.0</code>, <code>vers4.1</code> and <code>vers4.2</code> keys in {{Code|1=/etc/nfs.conf}} are left at their distribution defaults, which have 4.1 and 4.2 enabled. In the default mode a Linux client asking for <code>-o vers=4</code> negotiates '''4.2''', and <code>-o vers=4.1</code> is honoured if you pin it. If you need to pin or exclude a minor version, edit those keys by hand -- but note that applying the dialog rewrites <code>vers3</code> and <code>vers4</code> in the same file, and the NFS configuration is reapplied from the database each time the QuantaStor service starts.
 
=== What changes for a client between v3 and v4 ===
 
{| class="wikitable"
! !! NFSv3 !! NFSv4
|-
|-
| <code>insecure</code> / <code>secure</code> || <code>insecure</code> || <code>secure</code> requires NFS requests to originate from privileged ports (< 1024). Most modern NFS clients use unprivileged ports, so <code>insecure</code> is the default.
| Mount path (default protocol mode) || <code>&lt;ip&gt;:/export/&lt;share&gt;</code> || <code>&lt;ip&gt;:/export/&lt;share&gt;</code> -- the same
|-
|-
| <code>no_subtree_check</code> / <code>subtree_check</code> || <code>no_subtree_check</code> || Subtree checking verifies a requested file is within the exported directory. Adds overhead and can cause issues during renames. Disabled by default.
| Ports needed || 2049 plus rpcbind on 111, plus <code>mountd</code>, <code>statd</code> and <code>lockd</code> on ephemeral ports || 2049 only
|-
|-
| <code>crossmnt</code> / <code>nocrossmnt</code> || off || Allow NFS clients to follow mount points within the export. Required if the share contains nested mount points.
| Identity on the wire || Numeric UID and GID || <code>user@domain</code> strings, mapped at both ends
|-
|-
| <code>fsid=<uuid></code> || auto || Filesystem identifier used by NFSv4. QuantaStor assigns a UUID-based fsid automatically for stable exports. Do not set manually unless specifically required.
| Locking || Separate NLM protocol, via <code>lockd</code> || Part of the protocol, with server-side state
|-
|-
| <code>nohide</code> || off || Makes sub-mounts visible to NFS clients without requiring a separate mount command.
| Server state || Stateless; a server restart is transparent to a hard mount || Stateful; a server restart makes the client re-establish its session and reclaim locks
|-
|-
| <code>wdelay</code> / <code>no_wdelay</code> || <code>wdelay</code> || Delays write commits slightly to allow batching when <code>sync</code> is set. Has no effect with <code>async</code>.
| Browsing the whole tree || Not possible; each export is mounted by path || Mount <code>&lt;ip&gt;:/</code> and walk down into <code>export/</code>
|}
|}


== NFSv4 Specific Configuration ==
The mount path being identical for both versions is worth stating plainly, because NFSv4 servers commonly present shares under a pseudo-root that hides the real path. On QuantaStor in the default protocol mode the NFSv4 pseudo-filesystem is rooted at <code>/</code> and the shares appear beneath <code>/export</code>, exactly where NFSv3 finds them.
 
=== Mount Paths ===


NFSv4 uses a pseudo-filesystem root. QuantaStor exports all shares under <code>/export/</code>. When mounting with NFSv4, specify only the share name — do not include the <code>/export</code> prefix:
== The export configuration QuantaStor generates ==


{| class="wikitable"
{| class="wikitable"
! Protocol !! Mount Path Format !! Example
! File !! What it is
|-
| {{Code|1=/etc/exports}} || Generated by QuantaStor for the kernel NFS server. One line per exported share, rebuilt from the database whenever anything relevant changes and whenever the QuantaStor service starts.
|-
|-
| NFSv3 || <code>/export/<sharename></code> || <code>192.168.1.100:/export/share1</code>
| {{Code|1=/etc/exports.custom}} || Where your own export entries go. Its contents are appended verbatim to the generated file after a <code>## CUSTOM EXPORTS</code> marker line.
|-
|-
| NFSv4 || <code>/<sharename></code> || <code>192.168.1.100:/share1</code>
| {{Code|1=/etc/nfs.conf}} || The NFS daemon configuration. QuantaStor writes <code>[nfsd] vers3</code>, <code>[nfsd] vers4</code> and <code>[mountd] manage-gids</code> and preserves the rest.
|-
| {{Code|1=/etc/default/nfs-kernel-server}} || QuantaStor writes <code>RPCMOUNTDOPTS</code> and the GSS daemon switches here.
|-
| {{Code|1=/etc/ganesha/ganesha.conf}} || The scale-out server's configuration, with a QuantaStor-managed <code>EXPORT</code> section per Ceph share.
|-
| {{Code|1=/etc/ganesha/ganesha.conf.custom}} || Where your own Ganesha entries go.
|}
|}


=== NFSv4 ID Mapping ===
The generated exports file opens with its own warning:


NFSv4 identifies file owners by string-based user/group names rather than UID/GID numbers. For correct ownership display, the NFS ID mapping domain on each client must match the server's domain.
<pre style="font-size: smaller">
## WARNING: FILE GENERATED BY QUANTASTOR CORE SERVICES, MANUAL CHANGES MAY BE LOST
##
## NOTE: Scale-out CephFS NAS exports are presented via NFS-Ganesha on alternate default port 2249 rather than 2049.
##
</pre>


Edit <code>/etc/idmapd.conf</code> on the client:
Take that literally. On each regeneration QuantaStor keeps only single-<code>#</code> comments and export lines whose path is not under <code>/export</code>, <code>/mnt/storage-pools/</code>, <code>/mnt/namespaces/</code> or <code>/mnt/cloud-containers/</code>; everything else it wrote before, and everything after the <code>## CUSTOM EXPORTS</code> marker, is discarded and rebuilt. A hand-added export under <code>/export</code> will disappear. Use {{Code|1=/etc/exports.custom}} instead.


<pre>
=== The line QuantaStor writes for a share ===
[General]
 
Domain = localdomain
A share is exported at <code>/export/&lt;share&gt;</code>, which is a recursive bind mount of the share's real path under <code>/mnt/storage-pools/qs-&lt;pool-id&gt;/&lt;share&gt;</code>. Both paths are shown by <code>[[QuantaStor CLI Command Reference#share-get|qs share-get]]</code> as '''Export Path''' and '''Share Path'''.
 
A share created with default settings gets one <code>[public]</code> client rule, and produces one line:
 
<pre style="font-size: smaller">
/export/share1 *(rw,insecure,sync,no_subtree_check,crossmnt,fsid=8d13e546-ffa8-3abd-8c3f-d425443501ac)
</pre>
</pre>


Then restart the ID mapping daemon:
Every client access rule on the share becomes one <code>&lt;filter&gt;(&lt;options&gt;)</code> group on that same line, and QuantaStor assembles the options in a fixed order:
<pre>
 
sudo systemctl restart nfs-idmapd
<pre style="font-size: smaller">
[sec=krb5|krb5i|krb5p,] ro|rw , secure|insecure , async|sync , subtree_check|no_subtree_check
    [,<custom export options>] [,crossmnt] [,nohide] [,fsid=<share UUID>] [,refer=...]
</pre>
</pre>


== Kerberos / Security Mode ==
The <code>fsid=</code> value is the share's own object ID. Pinning it to something stable, rather than letting the server derive it from the underlying device, is what keeps a client's file handles valid across a service restart or a move of the pool to another node. It changes if a share is deleted and recreated under the same name, which is the usual cause of a stale file handle on a client that stayed mounted.
 
=== Rule order within a line ===
 
The kernel NFS server uses the '''first''' matching entry on a line, not the most specific one, so the order QuantaStor writes them in is what decides which rule a client gets. QuantaStor sorts most specific first:
 
{| class="wikitable"
! Order !! Filter kind !! Example
|-
| 1 || Single IP address || <code>192.168.10.44</code>
|-
| 2 || Host or domain wildcard, and netgroups || <code>*.example.com</code>, <code>@netgrp</code>
|-
| 3 || Network || <code>192.168.10.0/24</code>
|-
| 4 || Public || <code>*</code>
|}
 
'''Filter Priority''' on a rule breaks ties '''within''' a class -- it does not lift a network rule above a host rule. Leaving it on '''Auto''' sorts the rule last within its class. Five rules on one share come out like this:
 
<pre style="font-size: smaller">
/export/share1 192.168.10.44(rw,secure,sync,no_subtree_check,no_root_squash,anonuid=150,anongid=100,crossmnt,fsid=...)
    *.example.com(rw,secure,sync,no_subtree_check,crossmnt,fsid=...)
    @netgrp(rw,secure,sync,no_subtree_check,crossmnt,fsid=...)
    192.168.10.0/24(ro,secure,async,subtree_check,crossmnt,fsid=...)
    *(rw,insecure,sync,no_subtree_check,crossmnt,fsid=...)
</pre>
 
''(Shown wrapped for readability; the appliance writes one share per physical line.)''
 
=== Sub-exports for snapshot browsing ===
 
A share can produce two extra export lines, both covered under [[#Snapshot access over NFS|Snapshot access over NFS]]. Neither carries an <code>fsid=</code>.


QuantaStor supports Kerberos-based NFS security for environments that require strong authentication, data integrity checking, or wire encryption. Kerberos requires NFSv4.
{| class="wikitable"
! Line !! Appears when !! Extra options
|-
| <code>/export/&lt;share&gt;/.zfs</code> || The share has at least one snapshot and NFS browsing of the hidden snapshot directory is enabled || <code>crossmnt,nohide</code>
|-
| <code>/export/&lt;share&gt;_snaps</code> || The share has at least one '''activated''' GMT snapshot and NFS <code>_snaps</code> browsing is enabled || <code>crossmnt</code>
|}


=== Prerequisites ===
Because both are conditional on snapshots existing, the set of exports on a system grows and shrinks as snapshots are taken and expired. A client that mounted <code>/export/&lt;share&gt;_snaps</code> will find it gone once the last activated snapshot is removed.


* Active Directory or MIT Kerberos KDC must be reachable from the QuantaStor appliance.
== Client access rules and export options ==
* DNS and NTP must be correctly configured on the appliance.
* A valid keytab for the NFS service principal (<code>nfs/<hostname>@REALM</code>) must be installed. Contact OSNEXUS support for keytab deployment procedures.


=== Kerberos Security Modes ===
Access to a share over NFS is granted per client by a rule carrying its own export options. The dialog for adding and modifying those rules, the five filter kinds, and every checkbox on it are documented on [[Network Shares]] under '''NFS client access'''. What follows is the export option each control produces, and the CLI equivalents.


{| class="wikitable"
{| class="wikitable"
! Mode !! sec= option !! Description
! Export option !! Set by !! What it does
|-
| <code>rw</code> / <code>ro</code> || '''Read Only''' on the rule, or '''Read Only''' on the share -- the share-level setting wins || Whether the client may write.
|-
| <code>insecure</code> / <code>secure</code> || '''Allow Full Port Range Access''' -- ticked gives <code>insecure</code> || <code>secure</code> requires the client's requests to come from a reserved source port below 1024. Many clients do not use one by default, including macOS without <code>resvport</code> and a good number of containerised and appliance clients, so <code>secure</code> is a common cause of an unexplained permission denied.
|-
| <code>sync</code> / <code>async</code> || '''Async Writes''' || <code>async</code> lets the server acknowledge a write before it is on stable storage. Faster, and it breaks the NFS protocol's durability guarantee; the dialog's tooltip says not recommended.
|-
| <code>no_subtree_check</code> / <code>subtree_check</code> || '''Subtree checks''' || Subtree checking asks the server to verify that a file handle falls within the exported subtree. It exists for nested exports, costs performance, and can break renames. QuantaStor does not nest NFS exports, so leave it off.
|-
| <code>no_root_squash</code> || '''Allow Root Access''', which appends the literal string to the rule's custom options || Without it, <code>root</code> on the client is mapped to the anonymous user. This is one of the few options QuantaStor does not write itself -- it reaches the export line through the custom options field.
|-
| <code>crossmnt</code> || On unless '''Disable NFS cross-mount access to share''' is set on the share || Lets a client traverse from the export into file systems mounted beneath it. It is what makes snapshot directories reachable through a single mount.
|-
|-
| Kerberos (auth only) || <code>sec=krb5</code> || Authenticates the user identity via Kerberos. Data is transmitted in plaintext.
| <code>fsid=&lt;uuid&gt;</code> || QuantaStor, automatically || A stable file system identity for the export. Do not set it by hand.
|-
|-
| Kerberos + Integrity || <code>sec=krb5i</code> || Adds per-packet integrity signing. Protects against man-in-the-middle tampering. Recommended minimum for secure environments.
| <code>sec=krb5</code> / <code>krb5i</code> / <code>krb5p</code> || The system Kerberos Mode, when Kerberos is enabled and the effective policy is <code>kerberos</code> || The security flavour a client must match.
|-
|-
| Kerberos + Privacy || <code>sec=krb5p</code> || Adds full encryption of all NFS traffic. Highest security; some performance overhead.
| Anything else || '''Custom NFS Export Options''' on the rule || A comma-delimited list, appended verbatim. This is how you reach options QuantaStor has no control for -- <code>all_squash</code>, <code>anonuid=</code>, <code>anongid=</code>, <code>no_wdelay</code>.
|}
|}


=== Enabling Kerberos System-Wide ===
Options QuantaStor never writes on its own, and which therefore behave as the kernel's defaults unless you add them to the custom options field, include <code>root_squash</code> (the default, hence the '''Allow Root Access''' toggle), <code>wdelay</code>, <code>all_squash</code>, <code>anonuid=</code> and <code>anongid=</code>.
 
=== From the CLI ===
 
<pre style="font-size: smaller">
qs share-client-add    --share=share1 --filter=192.168.10.0/24 --rdonly=true
qs share-client-add    --share=share1 --filter=192.168.10.44 --options="no_root_squash,anonuid=150,anongid=100"
qs share-client-list  --share=share1
qs share-client-modify --share=share1 --filter=192.168.10.0/24 --async=true
qs share-client-remove --share=share1 --filter=192.168.10.0/24
</pre>
 
The commands are <code>[[QuantaStor CLI Command Reference#share-client-add|qs share-client-add]]</code>, <code>[[QuantaStor CLI Command Reference#share-client-list|qs share-client-list]]</code>, <code>[[QuantaStor CLI Command Reference#share-client-modify|qs share-client-modify]]</code> and <code>[[QuantaStor CLI Command Reference#share-client-remove|qs share-client-remove]]</code>. A rule is identified by its <code>--filter</code>, so modify and remove take the same filter string the rule was created with. <code>--filter</code> takes several filters in one call, comma-separated, and creates one rule per filter -- <code>--filter="192.168.10.1,192.168.10.2"</code> adds two rules, not one.


'''Navigation:''' System Management → Storage System → (right-click) → Modify Network Share Service Settings → Enable Kerberos
Three traps in the CLI surface:


'''Note:''' Enabling NFS Kerberos system-wide is a WUI-only operation — navigate to System Management → Storage System → (right-click) → Modify Network Share Service Settings → Enable Kerberos. Select the Kerberos mode (krb5, krb5i, or krb5p) from the NFS Security Policy dropdown.
* '''<code>--secure</code> defaults to <code>true</code> on the CLI and to off in the WUI.''' A rule added with <code>qs share-client-add</code> and no <code>--secure</code> comes out <code>secure</code>; the same rule added in the dialog, and the <code>[public]</code> rule QuantaStor creates with a new share, come out <code>insecure</code>. If a client works when you add its rule in the WUI and fails when you script it, this is why. Pass <code>--secure=false</code> explicitly.
* '''<code>--options</code> is not normalised on modify.''' On <code>share-client-add</code>, a recognised export option typed into <code>--options</code> is folded into the matching flag. On <code>share-client-modify</code> it is kept as free text, so <code>--options=ro</code> produces a self-contradictory <code>(rw,...,ro)</code> on the export line. Use the dedicated flags.
* '''IPv6 filters are not accepted.''' A client filter may contain letters, digits and <code>@ / * . -</code> only, so a colon in an address or prefix is rejected outright: ''"Unable to update network share with invalid client access rule '2001:db8::5', valid characters include alpha-numeric and symbols '@/*.-'."'' Restrict NFS access by IPv4 network, host name or netgroup instead.


=== Per-Share Security Policy ===
To copy a whole set of NFS rules and SMB settings from one share to another, use <code>[[QuantaStor CLI Command Reference#share-copy-settings|qs share-copy-settings]] --share=&lt;target&gt; --share-copy-settings-source=&lt;source&gt;</code>.


The system-level Kerberos policy can be overridden per share:
=== A share with no client rules is not exported ===


'''Navigation:''' Storage Management → Network Shares → (right-click share) → Modify → NFS Security Policy
A share with no NFS client rule and no SMB access is taken offline, with the state detail ''"No access via SMB or NFS has been configured, share is offline."'' and no line in the exports file. Adding at least one client rule is what puts it back. The one exception is a Kerberos system, where a share with no client rules is still exported to <code>*</code> with the Kerberos prefix.
 
== Applying export changes ==
 
Export changes and service configuration changes are applied by two different mechanisms, and only one of them touches clients.


{| class="wikitable"
{| class="wikitable"
! Policy !! Behavior
! Change !! How it is applied !! Client impact
|-
|-
| Inherit (default) || Uses the system-level default security policy
| Add, modify or remove a client access rule || The exports file is regenerated and <code>exportfs -r</code> is run || '''None.''' The server swaps its export table in place.
|-
|-
| Kerberos || Requires Kerberos authentication for this share regardless of system default
| Create, modify, delete, enable, disable, snapshot, clone or rename a share; cross-mount and snapshot browsing options; a share's NFS security policy || Same || '''None'''
|-
|-
| System (AUTH_SYS) || Uses standard Unix UID/GID authentication for this share regardless of system default
| '''Default Security Policy''' and '''NFS Server Type''' in the NFS Services Configuration dialog || Same || None from the reload itself. Changing the server type does swap which daemon is serving.
|-
| '''NFS Protocol Mode''', '''Enable Kerberos Security''', '''Kerberos Mode''', '''Disable NFSv4 /export Browsing''', '''Manage GIDs''' || Configuration files are written and <code>nfs-server</code> is restarted || '''Momentary interruption''' for every NFS client on that system
|-
| HA VIF changes and an HA pool failover || Exports regenerated plus, on the releasing node, a service restart || See [[#NFS over an HA storage pool|NFS over an HA storage pool]]
|}
|}


'''CLI:'''
The five settings in the fourth row are the only ones in the dialog that bounce the service, and the dialog restarts it only when one of them actually changes -- reapplying the dialog with nothing altered does not interrupt anything.
<pre>
 
# Force Kerberos on a specific share
A reload really is transparent: an active NFSv3 mount and an active NFSv4.2 mount both kept serving reads across repeated export regenerations while client rules were added and removed and snapshot sub-exports appeared, with no error and no remount.
qs share-modify --share=share1 --nfs-security-policy=kerberos
 
Removing a client access rule additionally unexports that one filter and path immediately, rather than waiting for the regeneration, so that client loses access at once while every other client on the share is untouched.
 
== Restarting the NFS service ==
 
{{Navigation|Storage Management &rarr; Network Shares &rarr; Network Share Services &rarr; Restart NFS/SMB Services ''(toolbar)''}}
 
{{Navigation|Storage Management &rarr; Network Shares &rarr; Network Share ''(select + right-click)'' &rarr; Restart NFS/SMB Services...}}
 
[[File:nfs_restart_services.png|thumb|right|400px|Restart Storage Services. Both services are selected by default, so untick the one you do not want restarted.]]
 
The toolbar and menu item are labelled '''Restart NFS/SMB Services''', and the dialog that opens is titled '''Restart Storage Services'''. It restarts the NFS service, the CIFS/SMB service, or both, on the storage system chosen in the combo. '''Both checkboxes are ticked when the dialog opens''', so restarting only NFS means clearing the CIFS/SMB one first.
 
The dialog carries its own warning, and it is accurate: ''"WARNING: This will momentarily interrupt access to the volumes/shares associated with these services and may effect workloads using them."'' NFS is restarted first, then SMB.
 
What it does '''not''' do is regenerate the exports file. It is purely a service bounce, so the exports the server comes back with are whatever the last regeneration left on disk. If a share is missing from the exports file, restarting the service will not add it -- fix the share's client rules or its pool state instead, which regenerates the file on its own.
 
Reach for it when:
 
* the NFS service is reported as not running, or has been left in a failed state;
* an NFSv4 client is wedged in a way that only clearing server-side state will fix;
* you hand-edited {{Code|1=/etc/nfs.conf}} and need the daemon to re-read it.
 
There is no <code>qs</code> command for this operation. It is available from the WUI and the REST API only.
 
Two appliance-side commands cover cases the dialog does not. <code>qs-util nfsstandard</code> and <code>qs-util nfsganesha</code> apply the Scale-up and Scale-out server configurations directly, and are the way back when the configuration on disk and the recorded server type have diverged. <code>qs-util nfsrdma</code> enables NFS over RDMA -- it loads the <code>rpcrdma</code> module, sets <code>rdma=y</code> and <code>rdma-port=20049</code> in {{Code|1=/etc/nfs.conf}} after backing the file up, restarts the server and verifies the RDMA port came up. RDMA has no equivalent in the WUI, and it needs lossless or PFC configured on both the appliance and the client to be worth having.
 
== NFS over an HA storage pool ==
 
An HA Storage Pool moves between nodes together with one or more virtual network interfaces. '''Clients must mount the HA VIF, never a node's own management or data address''' -- that is the whole point of the VIF, and a mount to a physical address does not follow the pool. See [[HA Cluster Setup (JBODs)]] for setting the group up.
 
What QuantaStor does during a failover, in order:
 
# '''On the releasing node''', the HA VIFs are firewalled off first, so clients stop reaching the node before anything about the exports changes.
# The pool's shares are removed from the exports file and unexported, then the pool is exported at the storage layer.
# The NFS and SMB services on that node are restarted to force session cleanup. This happens asynchronously and can land up to a few minutes after the failover otherwise looks finished, and it interrupts clients of '''other''' pools on the same node as well.
# '''On the acquiring node''', the pool is imported, the exports file is regenerated and reloaded, and the NFS and SMB services are started if they were not already running.
# The VIF firewall block is lifted last, so a client's first packet after the move arrives at a server that is already exporting the share.
 
What a client sees:
 
* '''NFSv3 with a hard mount''' -- I/O stalls while the VIF is down and resumes when it comes back on the new node. Because NFSv3 is stateless and the export keeps the same <code>fsid=</code>, the mount recovers without intervention. A soft mount returns errors to the application instead, which is usually worse.
* '''NFSv4''' -- the client's session is gone. It sees a stale client ID or a bad session, re-establishes, and reclaims what it can.
* '''Locks are not migrated.''' QuantaStor does not send <code>sm-notify</code> to NFSv3 clients and does not coordinate an NFSv4 grace period across the failover, so byte-range locks held before the move are lost. An application that depends on advisory locking for correctness has to be able to re-acquire them, and there is a window in which another client can take a lock the previous holder still believes it owns.
 
Two HA-specific settings affect NFS:
 
* '''Enforce NFS HA VIF Access''' on the HA group replaces a public (<code>*</code>) client filter on that pool's shares with one entry per HA interface subnet, written as <code>&lt;network&gt;/&lt;netmask&gt;</code>. It is a way to keep a share that is nominally public reachable only from the networks the HA VIFs live on.
* QuantaStor installs a systemd drop-in for <code>nfs-server</code> that makes its start-time <code>exportfs -r</code> failure-tolerant. Without it, the service refuses to start on a node whose export paths are absent -- which is the normal state for a pool that has failed over away.
 
== How UIDs and GIDs resolve ==
 
QuantaStor stores POSIX ownership on the file system, so what a client sees depends on how the two ends agree about identity.
 
'''NFSv3 sends numeric UIDs and GIDs on the wire.''' There is no translation: a file owned by UID 1001 shows as whatever user is 1001 on the client. Matching numeric IDs at both ends is the only thing that makes ownership meaningful.


# Override to AUTH_SYS for a share even if system default is Kerberos
'''NFSv4 sends <code>user@domain</code> strings''', which each end maps through <code>rpc.idmapd</code>. The domain is configured in {{Code|1=/etc/idmapd.conf}}, and QuantaStor leaves it unset, so it defaults to the appliance's DNS domain -- its fully-qualified name minus the host part. '''If the client's idmap domain does not match, every file shows as <code>nobody:nogroup</code>''' even though access works. This is the single most common NFSv4 surprise. Set <code>Domain</code> in <code>[General]</code> on the client to match the appliance, then restart <code>nfs-idmapd</code> or run <code>nfsidmap -c</code> to flush the cache.
qs share-modify --share=share1 --nfs-security-policy=system
</pre>


== Snapshot Access via NFS ==
Name-to-ID resolution on the appliance goes through NSS. Out of the box that is local files, then SSSD, then winbind. Joining Active Directory is what populates the last two: the domain join rewrites {{Code|1=/etc/nsswitch.conf}} and configures SSSD and winbind, at which point AD users and groups resolve on the appliance and their POSIX identities can be used for NFS ownership. Whether IDs come from the directory's RFC 2307 attributes or from an algorithmic mapping is decided by the AD join, not by anything on this page -- see [[Network Shares]] for the join and [[Security Configuration]] for directory integration.


QuantaStor exposes point-in-time snapshots to NFS clients through two mechanisms.
=== Manage GIDs ===


=== .zfs Directory (ZFS Snapshot Browsing) ===
'''Manage GIDs''' is on by default. Its tooltip describes it exactly: the appliance ignores the group list a client sends and makes its own independent check of the user's group memberships, via local <code>/etc/group</code> or the directory service.


When enabled, NFS clients can browse the <code>.zfs/snapshot/</code> directory at the root of each share to access individual snapshots directly without administrator intervention.
The reason it exists is a hard limit in AUTH_SYS: the credential a client sends carries at most '''16''' supplementary group IDs. A user who belongs to more than 16 groups has some of them silently dropped, and loses access to files whose group fell off the end -- non-deterministically, because which 16 survive is up to the client. With Manage GIDs on, the server discards the client's list entirely and resolves the user's full group membership itself.


'''Navigation:''' Storage Management → Network Shares → (right-click share) → Modify → Enable NFS Snapshot Browsing
Two consequences to plan for:


'''CLI:'''
* '''The appliance must be able to resolve the user's groups.''' If a UID resolves on the client but not on the appliance, that user loses '''all''' supplementary groups rather than gaining the full set. On an AD-backed system that means the join has to be healthy.
<pre>
* It takes effect in <code>rpc.mountd</code>, so changing it restarts the NFS service and interrupts clients.
# Enable .zfs snapshot directory browsing for a share
qs share-modify --share=share1 --disable-nfs-hidden-dir-browsing=false


# Disable .zfs snapshot directory browsing
QuantaStor writes it in two places -- <code>[mountd] manage-gids</code> in {{Code|1=/etc/nfs.conf}} and <code>RPCMOUNTDOPTS="--manage-gids"</code> in {{Code|1=/etc/default/nfs-kernel-server}} -- and reads the setting back from those files rather than from the database, so an edit made by hand is what the WUI will report.
qs share-modify --share=share1 --disable-nfs-hidden-dir-browsing=true
</pre>


Client access:
== Snapshot access over NFS ==
<pre>
ls /mnt/data/.zfs/snapshot/
# 2024-06-17-08:00:00
# 2024-06-16-08:00:00
</pre>


=== _snaps Folder (GMT Snapshot Folder) ===
Which snapshot browsing routes are open is a per-share decision made on the share's '''Advanced Settings''', and the defaults are deliberately restrictive: over NFS, both routes are closed. [[Network Shares]] documents the switches. This section is what the NFS side looks like once they are open.


Activated snapshots also appear in a <code>_snaps/</code> directory at the root of the export using GMT-formatted names, compatible with Windows Previous Versions:
{| class="wikitable"
! Route !! Export !! What a client sees
|-
| The <code>_snaps</code> folder || <code>/export/&lt;share&gt;_snaps</code>, a separate export beside the share || One directory per '''activated''' snapshot, named <code>@GMT-2026.09.03-05.10.53</code>. Mount it in its own right.
|-
| The hidden snapshot directory || <code>/export/&lt;share&gt;/.zfs</code>, a sub-export of the share || <code>.zfs/snapshot/</code> reached from inside the share's own mount, holding one read-only directory per snapshot -- named <code>GMT-2026.09.03-05.10.53</code>, without the leading <code>@</code>. On a scale-out share the equivalent directory is <code>.snap</code>.
|}


<pre>
Reading a file out of a snapshot therefore looks like this:
ls /mnt/data/_snaps/
# @GMT-2024.06.17-08.00.00
# @GMT-2024.06.16-08.00.00
</pre>


To disable <code>_snaps</code> visibility for a share:
<pre style="font-size: smaller">
<pre>
sudo mount -t nfs -o vers=4,ro 192.168.10.5:/export/share1 /mnt/share1
qs share-modify --share=share1 --disable-nfs-snap-dir=true
ls /mnt/share1/.zfs/snapshot/
</pre>
# GMT-2026.09.03-05.10.53
cp /mnt/share1/.zfs/snapshot/GMT-2026.09.03-05.10.53/report.xlsx ~/


To re-enable:
sudo mount -t nfs -o vers=3,ro 192.168.10.5:/export/share1_snaps /mnt/share1_snaps
<pre>
ls /mnt/share1_snaps/
qs share-modify --share=share1 --disable-nfs-snap-dir=false
# @GMT-2026.09.03-05.10.53
</pre>
</pre>


== Mounting on Linux Clients ==
Traversing into <code>.zfs/snapshot/&lt;name&gt;</code> from inside the share's mount depends on <code>crossmnt</code>, because each snapshot is a file system of its own. That is why '''Disable NFS cross-mount access to share''' and snapshot browsing interact: turning cross-mount off leaves the directory visible and its contents unreachable.


=== NFSv3 ===
Neither export appears until the share has snapshots, and the <code>_snaps</code> export additionally needs at least one snapshot that is '''activated'''. A snapshot that exists but is not active is not presented to clients.


<pre>
== Mounting from a Linux client ==
# One-time mount
sudo mount -t nfs -o rw,soft,timeo=300,retrans=3 192.168.1.100:/export/share1 /mnt/data


# Persistent — add to /etc/fstab
Install the NFS client package first: <code>nfs-common</code> on Debian and Ubuntu, <code>nfs-utils</code> on RHEL, Rocky and SUSE.
192.168.1.100:/export/share1  /mnt/data  nfs rw,soft,timeo=300,retrans=3,_netdev  0  0
</pre>


=== NFSv4 (Recommended) ===
The '''View Mount Command''' dialog -- right-click a share, described on [[Network Shares]] -- gives you the share path and a ready-made mount command for whichever network port you pick, which is the quickest way to get the right address for a share you are looking at.


<pre>
=== A worked mount ===
# One-time mount
sudo mount -t nfs4 -o rw,soft,timeo=300,retrans=3 192.168.1.100:/share1 /mnt/data


# Persistent — add to /etc/fstab
<pre style="font-size: smaller">
192.168.1.100:/share1 /mnt/data  nfs4  rw,soft,timeo=300,retrans=3,_netdev  0  0
sudo mkdir -p /mnt/share1
sudo mount -t nfs -o vers=4,hard,proto=tcp 192.168.10.5:/export/share1 /mnt/share1
</pre>
</pre>


=== Performance Mount Options ===
That is the whole of it for a normal share. The options worth thinking about:


{| class="wikitable"
{| class="wikitable"
! Option !! Recommended Value !! Description
! Option !! Notes
|-
|-
| <code>rsize</code> || <code>1048576</code> || Read block size in bytes. Increase to 1 MB for sequential workloads.
| <code>vers=3</code> / <code>vers=4</code> / <code>vers=4.1</code> || Pin the version rather than letting the client negotiate, so a client's behaviour does not change when the appliance's protocol mode does. <code>vers=4</code> negotiates 4.2 against a QuantaStor appliance.
|-
|-
| <code>wsize</code> || <code>1048576</code> || Write block size in bytes. Increase to 1 MB for sequential workloads.
| <code>hard</code> ''(default)'' || Retry indefinitely. This is what you want for anything holding data: an HA failover, a service restart or a network blip becomes a stall the application survives rather than an I/O error it has to handle. Use <code>soft</code> only where a hung process is worse than a failed write, and pair it with <code>timeo</code> and <code>retrans</code>.
|-
|-
| <code>timeo</code> || <code>300</code> || Timeout in tenths of a second before retrying an NFS request.
| <code>proto=tcp</code> ''(default)'' || Explicit is better than implicit, and UDP is not appropriate for NFS at any useful scale.
|-
|-
| <code>retrans</code> || <code>3</code> || Number of retransmissions before a soft-mount returns an error.
| <code>rsize</code> / <code>wsize</code> || '''Leave them alone.''' A Linux client already negotiates 1 MiB (<code>1048576</code>) in each direction against a QuantaStor appliance, over both NFSv3 and NFSv4, so the common advice to raise them to 1 MB has nothing left to raise. Confirm what you actually got with <code>nfsstat -m</code> rather than setting them on faith.
|-
|-
| <code>hard</code> / <code>soft</code> || <code>soft</code> || <code>hard</code> retries indefinitely (processes hang if server is unavailable). <code>soft</code> returns an error after <code>retrans</code> failures.
| <code>ro</code> || Worth using for a snapshot mount or an audit, and it is enforced by the client as well as the export.
|-
|-
| <code>proto=tcp</code> || recommended || Force TCP transport. More reliable than UDP for large or sustained transfers.
| <code>noresvport</code> || Use a non-reserved source port. Needed only if a rule was created with <code>secure</code>; if the rule has '''Allow Full Port Range Access''' set -- which is the WUI default -- this is unnecessary.
|-
|-
| <code>noresvport</code> || optional || Do not require a privileged source port. Use when the server export uses the <code>insecure</code> option (the default).
| <code>sec=krb5i</code> || Required when the share's effective security policy is Kerberos, and must match the appliance's Kerberos Mode. Get a ticket first with <code>kinit</code>, and confirm it with <code>klist</code>.
|}
|}


Example high-throughput NFSv4 mount:
=== Making it persistent ===
<pre>
sudo mount -t nfs4 \
  -o rw,soft,timeo=300,retrans=3,rsize=1048576,wsize=1048576,proto=tcp \
  192.168.1.100:/share1 /mnt/data
</pre>
 
=== Kerberos Mount ===


<pre>
Add a line to {{Code|1=/etc/fstab}}:
# Mount with Kerberos integrity (krb5i)
sudo mount -t nfs4 -o sec=krb5i,rw 192.168.1.100:/share1 /mnt/data


# Mount with Kerberos encryption (krb5p)
<pre style="font-size: smaller">
sudo mount -t nfs4 -o sec=krb5p,rw 192.168.1.100:/share1 /mnt/data
192.168.10.5:/export/share1 /mnt/share1  nfs  vers=4,hard,proto=tcp,_netdev  0  0
</pre>
</pre>


Ensure a valid Kerberos ticket is present on the client before mounting:
<code>_netdev</code> is the part that matters: it tells the client not to attempt the mount until networking is up, which is what stops a boot from hanging or the mount from silently failing. Leave the dump and fsck fields at <code>0</code>. On a systemd client, <code>x-systemd.automount</code> in place of a plain mount is worth considering for a share that is not needed at boot, since it defers the mount until something touches the directory.
<pre>
kinit username@REALM
klist
</pre>


== Mounting on Windows Clients ==
=== Browsing all shares over NFSv4 ===


Windows includes an NFS client (Services for NFS) available on Pro and Server editions.
An NFSv4 client can mount the top of the tree and walk down to find every share it has access to:


=== Enable the NFS Client ===
<pre style="font-size: smaller">
 
sudo mount -t nfs -o vers=4,ro 192.168.10.5:/ /mnt/qs
'''Windows Server (PowerShell):'''
ls /mnt/qs/export
<pre>
# share1  share2  share3
Install-WindowsFeature NFS-Client
</pre>
</pre>


'''Windows 10/11 Pro:'''
=== Mounting a scale-out share ===
Control Panel → Programs → Turn Windows features on or off → Services for NFS → Client for NFS


=== Mount via Command Prompt ===
A share on a CephFS pool or a Ceph object gateway bucket is served by NFS-Ganesha, and two things differ. The path is the Ganesha pseudo path -- <code>/&lt;share&gt;</code>, or <code>/s3/&lt;bucket&gt;</code> for a bucket -- not <code>/export/&lt;share&gt;</code>. And while the storage system's NFS Server Type is '''Scale-up''', Ganesha is on port 2249 rather than 2049, so the client has to be told:


<pre>
<pre style="font-size: smaller">
# Mount share1 as drive Z:
sudo mount -t nfs -o vers=4,port=2249,mountport=2249 192.168.10.5:/share1 /mnt/share1
mount \\192.168.1.100\export\share1 Z:
</pre>


# Specify soft mount type
With the server type set to '''Scale-out''', Ganesha owns 2049 and no port override is needed. NFSv3 is not an option against the alternate port.
mount -o mtype=soft \\192.168.1.100\export\share1 Z:
</pre>


'''Note:''' The built-in Windows NFS client supports NFSv2/v3. For NFSv4, use a third-party NFS client.
=== Windows clients ===


== Session Monitoring ==
Every Network Share is exported over NFS and SMB at the same time, and SMB is the right protocol for a Windows client -- it carries Windows ACLs, Active Directory identity and the ''Previous Versions'' snapshot tab, none of which an NFS mount gives you. See [[Network Shares]] for the SMB share options and the AD domain join.


Active NFS client sessions can be viewed in the QuantaStor web interface or CLI.
=== Confirming what is exported and who is mounted ===


'''Navigation:''' Storage Management → Network Shares → (right-click share) → Show Sessions
<pre style="font-size: smaller">
showmount -e 192.168.10.5    # every export the appliance is offering
showmount -a 192.168.10.5    # NFSv3 mounts currently held, as <client>:<path>
nfsstat -m                  # on the client: what each mount actually negotiated
</pre>


'''CLI:'''
<code>showmount</code> reports NFSv3 state, so an NFSv4 mount does not appear in <code>-a</code>. Both need rpcbind and the mount service reachable, which means they do not work against a scale-out share on port 2249.
<pre>
# List all active NFS and SMB sessions
qs share-session-list


# Get details for a specific session
== Ports and firewall ==
qs share-session-get --share-session=<session-id>
</pre>


For detailed NFSv4 state analysis (open files, client IPs, ZFS dataset mappings):
{| class="wikitable"
<pre>
! Service !! Ports !! Needed for
# Summary view
|-
sudo /opt/osnexus/quantastor/scripts/qs_nfsv4_sessions.sh
| NFS || 2049 TCP and UDP, 111 TCP and UDP || NFSv4 needs 2049 only. NFSv3 also needs rpcbind on 111.
|-
| NFS Ganesha || 2249 TCP || Scale-out shares while the server type is Scale-up.
|-
| NFS over RDMA || 20049 || Only after <code>qs-util nfsrdma</code>.
|}


# Verbose — includes open file details
Those are the definitions QuantaStor's own firewall management works from, in {{Code|1=/opt/osnexus/quantastor/conf/qs_services_firewall.conf}}, and the ports it monitors for reachability.
sudo /opt/osnexus/quantastor/scripts/qs_nfsv4_sessions.sh -v


# Limit output to a specific storage path
'''NFSv3 needs more than that list.''' <code>mountd</code>, <code>statd</code> and <code>lockd</code> take ephemeral ports, and QuantaStor does not pin them, so NFSv3 through a stateful firewall needs those ports fixed by hand in {{Code|1=/etc/nfs.conf}} and opened to match. Check what they currently are with <code>rpcinfo -p &lt;appliance&gt;</code>. NFSv4 has none of this problem, which on its own is a reason to prefer it where a firewall sits between the client and the appliance.
sudo /opt/osnexus/quantastor/scripts/qs_nfsv4_sessions.sh --path=/mnt/storage-pools
</pre>


== Troubleshooting ==
== Troubleshooting ==


=== Share Not Visible to Clients ===
=== A share does not appear to clients ===


# Verify the share has at least one NFS client rule: <code>qs share-client-list --share=share1</code>
# Confirm the share has at least one NFS client access rule -- <code>qs share-client-list --share=&lt;share&gt;</code>. A share with none, and no SMB access, is taken offline.
# Confirm the NFS service is running: <code>systemctl status nfs-server</code> (scale-up) or <code>systemctl status nfs-ganesha</code> (scale-out)
# List what the appliance is actually exporting: <code>showmount -e &lt;appliance&gt;</code>. If the share is not there, the exports file has not been regenerated with it; check the share's state and its pool's state rather than restarting the service.
# Test connectivity on port 2049: <code>nc -zv 192.168.1.100 2049</code>
# Confirm the pool is imported on the node you are talking to. A share on an HA pool that has failed over is exported on the other node.
# List exports visible from the server: <code>showmount -e 192.168.1.100</code>


=== Permission Denied on Mount ===
=== Permission denied on mount ===


# Verify the client's IP address matches an NFS client rule on the share.
# Check that the client's address matches a rule, and that a broader rule is not being matched first. The '''first''' matching entry on the export line wins, and QuantaStor orders host rules ahead of network rules ahead of the public rule.
# If a CIDR rule is used, confirm the client IP is within that range.
# Check whether the rule is a <code>secure</code> rule, which every rule created by <code>qs share-client-add</code> without an explicit <code>--secure=false</code> is. A <code>secure</code> rule requires the client to connect from a reserved source port below 1024, and a client mounting with <code>noresvport</code>, or one that never uses a privileged port, is refused. The fix is to set '''Allow Full Port Range Access''' on the rule, or recreate it with <code>--secure=false</code>. Confirm what the rule actually is with <code>qs share-client-list --share=&lt;share&gt;</code>, or by reading the export line.
# Check priority ordering — the highest-priority (lowest-numbered) matching rule wins.
# On a Kerberos system, confirm the client has a valid ticket and is using the <code>sec=</code> flavour the appliance is configured for.
# Confirm that port 2049 (TCP/UDP) and port 111 (portmapper, NFSv3 only) are not blocked by a firewall between client and appliance.


=== Stale NFS File Handle ===
=== NFSv4 clients hang while NFSv3 clients keep working ===


A stale file handle error occurs when the share was removed and recreated (acquiring a new UUID/fsid) while a client still held a mount. To resolve:
This pattern points at the NFSv4 client tracking daemon, <code>nfsdcld</code>, which records client state in a small database on the appliance's root file system. A full root file system can corrupt that database, at which point the daemon fails, the kernel blocks NFS server threads waiting on it, and NFSv4 mounts stall while stateless NFSv3 traffic is unaffected.


<pre>
QuantaStor detects this and repairs it automatically -- it moves the damaged database aside so the daemon rebuilds it, and restarts the NFS server -- and raises an alert saying so. Repair is rate limited and gives up after a few attempts, so if the alert persists, the thing to fix is free space on the root file system. Recovery is deliberately skipped on a system where NFS-Ganesha is serving, and on a node whose pool has failed over away, because a stale failure there is not an outage.
sudo umount -l /mnt/data
sudo mount -t nfs4 192.168.1.100:/share1 /mnt/data
</pre>


If the error persists, verify the current fsid on the server:
=== Files show as nobody:nogroup over NFSv4 ===
<pre>
sudo exportfs -v | grep share1
</pre>


=== NFSv4 Ownership Shows as "nobody" ===
An ID mapping domain mismatch. See [[#How UIDs and GIDs resolve|How UIDs and GIDs resolve]]. Access still works; only the displayed ownership is wrong.


This is an ID mapping domain mismatch between client and server. Edit <code>/etc/idmapd.conf</code> on the client:
=== Stale NFS file handle ===


<pre>
The export's <code>fsid=</code> is the share's object ID, so it changes if a share is deleted and recreated under the same name. A client that stayed mounted across that is holding handles for a file system that no longer exists. Remount:
[General]
Domain = localdomain
</pre>


Then flush the ID map cache:
<pre style="font-size: smaller">
<pre>
sudo umount -l /mnt/share1
sudo systemctl restart nfs-idmapd
sudo mount -t nfs -o vers=4 192.168.10.5:/export/share1 /mnt/share1
# Or on some systems:
sudo nfsidmap -c
</pre>
</pre>


=== Poor NFS Performance ===
=== NFSv3 mounts fail with an RPC port mapper error ===


# Increase <code>rsize</code> and <code>wsize</code> to 1 MB (<code>-o rsize=1048576,wsize=1048576</code>).
NFSv3 needs rpcbind, on port 111, in addition to the NFS server. Check that 111 is reachable and that the appliance's protocol mode has not been set to '''NFS v4''', which disables NFSv3 outright.
# Use TCP transport (<code>proto=tcp</code>) instead of UDP.
# For write-heavy workloads on ZFS pools, verify the SLOG (ZIL) device is configured on the storage pool.
# If <code>sync</code> write latency is high, consider enabling <code>async</code> on the NFS client rule (trade-off: slight data-loss risk on unclean shutdown).
# Check raw network throughput between client and appliance with <code>iperf3</code>.


=== NFSv3 Portmapper Issues ===
=== Finding out who is connected ===


NFSv3 requires the portmapper service (rpcbind) in addition to the NFS daemon. If NFSv3 mounts fail with "RPC: Port mapper failure":
QuantaStor tracks '''SMB''' sessions only. The '''SMB Sessions''' tab and <code>[[QuantaStor CLI Command Reference#share-session-list|qs share-session-list]]</code> do not report NFS clients, so do not read an empty list as nobody being mounted. For NFS, use the '''NFS Stats''' tab of the Share Dashboard for throughput and operation rates, and <code>showmount -a</code> on the appliance for NFSv3 mounts.


<pre>
=== Slow NFS writes ===
sudo systemctl status rpcbind
sudo systemctl enable --now rpcbind
</pre>


== Scale-out NFS (NFS-Ganesha) ==
Work through the storage before the protocol. Synchronous NFS writes land in the pool's ZFS intent log, so a pool without a write log device commits them to the data disks -- see [[Storage Pools]] for adding one. After that, confirm the client negotiated 1 MiB read and write sizes with <code>nfsstat -m</code>, confirm raw network throughput between the two ends independently of NFS, and only then consider '''Async Writes''' on the client rule, which trades the protocol's durability guarantee for speed.


When using CephFS-based scale-out storage pools, QuantaStor uses NFS-Ganesha instead of the kernel NFS server. Key differences from scale-up mode:
== Related pages ==


* NFS-Ganesha runs as a userspace daemon; its configuration is managed by QuantaStor at <code>/etc/ganesha/ganesha.conf</code>.
* [[Network Shares]] -- creating and modifying shares, the NFS client access dialog, SMB options, quotas and snapshots
* Mount commands and access control entries are identical to scale-up mode.
* [[Storage Pools]] -- the pools shares are provisioned from, and write log devices
* Provides distributed NFS access across multiple CephFS nodes for higher aggregate throughput.
* [[Security Configuration]] -- directory integration and authentication
* Defaults to port 2049 (same as kernel NFS).
* [[HA Cluster Setup (JBODs)]] -- HA Storage Pools and the virtual interfaces NFS clients mount
* [[Scale-out File Setup (ceph)]] -- CephFS pools, which are what the scale-out NFS server exists for
* [[QuantaStor CLI Command Reference]] -- full argument lists for the <code>qs</code> commands used here


Check the Ganesha service status on the appliance:
----
<pre>
<small>''Verified against QuantaStor 6.9.0.''</small>
systemctl status nfs-ganesha
</pre>
 
For full CephFS setup, see [[Scale-out File Setup (ceph)]].
 
== See Also ==
 
* [[Network_Shares|Network Shares]] — Creating and managing NFS/SMB shares
* [[Security_Configuration|Security Configuration]] — LDAP, Kerberos, RBAC
* [[Remote-replication_(DR)|Remote Replication (DR)]] — Replicating NFS share data to a DR site
* [[Snapshot_Schedules|Snapshot Schedules]] — Automated snapshot creation for data protection
* [[Scale-out_File_Setup_(ceph)|Scale-out File Setup (ceph)]] — CephFS-backed NFS with NFS-Ganesha
 
[[Category:admin_guide]]

Revision as of 05:30, 3 September 2026


QuantaStor provides NFS access to Network Shares via two server implementations, and which one serves a share is decided by the storage architecture underneath it. This page covers those two servers and how one is selected, the NFS protocol versions on offer, the export configuration QuantaStor generates and where it lives on the appliance, the export options a client access rule can produce, how export changes are applied, NFS across an HA storage pool, how UIDs and GIDs resolve, and how to mount a share from Linux.

Creating and modifying shares, the Add NFS Client Access dialog and its checkboxes, quotas, aliases and snapshot management all belong to Network Shares. This page starts where that one stops: at the server and the client.

Section Purpose
The two NFS server implementations Scale-up versus scale-out, and which storage architecture selects which
Configuring the NFS service The NFS Services Configuration dialog, field by field
NFS protocol versions NFSv3, NFSv4 and the NFSv4 minor versions, and what changes for a client
The export configuration QuantaStor generates The files on the appliance, and the exact export lines written into them
Client access rules and export options Every export option QuantaStor can emit, and what sets it
Applying export changes Reload versus restart, and when a client is interrupted
Restarting the NFS service The Restart Storage Services operation and when it is needed
NFS over an HA storage pool What a client sees when a pool fails over
How UIDs and GIDs resolve ID mapping, Manage GIDs, and AD-backed ownership
Snapshot access over NFS The _snaps and .zfs sub-exports as an NFS client sees them
Mounting from a Linux client A worked mount, the options that matter, and making it persistent
Ports and firewall What has to be reachable, and what QuantaStor does not pin
Troubleshooting The failures that come up most, and what to check

The two NFS server implementations

Scale-up Scale-out
Server The Linux kernel NFS server NFS-Ganesha, a user-space NFS server
Serves shares on ZFS Storage Pools CephFS pools and Ceph object gateway buckets
Service nfs-server.service (nfsd) nfs-ganesha.service (ganesha.nfsd)
Export configuration /etc/exports /etc/ganesha/ganesha.conf
Owns TCP/UDP 2049 Yes Only when Scale-out is selected

The setting that chooses between them decides, in practice, which of the two servers owns port 2049. Selecting Scale-up leaves the kernel NFS server on 2049 serving both NFSv3 and NFSv4, and configures NFS-Ganesha for the alternate port 2249 in NFSv4-only mode so that the two do not contend for 2049 or for the RPC mount service. QuantaStor records that in the header of the generated exports file:

## NOTE: Scale-out CephFS NAS exports are presented via NFS-Ganesha on alternate default port 2249 rather than 2049.

Selecting Scale-out moves NFS-Ganesha to 2049 with NFSv3 and NFSv4 enabled, and stops, disables and masks the kernel NFS server. On a system that also has ZFS Storage Pools, that takes every share on them offline for NFS clients -- they remain available over SMB. There is no confirmation prompt for this, and nothing validates the choice against the pools that actually exist, so treat Scale-out as a decision about the whole appliance rather than a per-share option. On a system with no Ceph storage the Ganesha service is left masked and not running.

QuantaStor selects Scale-out for you in one situation: when a Ceph file system or Ceph object pool is created on a system that has no ZFS Storage Pools. If ZFS pools are present it leaves the setting alone and raises a warning alert reading "SCALE-UP pool(s) present on local system. Not configuring SCALE-OUT NFS settings." Everywhere else the setting is yours to make.

For the Ceph side of the picture, see Scale-out File Setup (ceph).

What differs between them

The two servers are not interchangeable, and several things configured per share reach only one of them.

Scale-up (kernel) Scale-out (Ganesha)
Export path a client mounts /export/<share> /<share>, or /s3/<bucket> for an object gateway bucket
NFSv3 Available Not available while the server is on port 2249, because the RPC mount service is left to the kernel server
NFSv3 byte-range locking (NLM) Available Disabled -- the shipped configuration sets enable_nlm = false
Quota reporting over NFS (RQUOTA) Not offered -- no rquotad is registered with rpcbind Not offered -- the shipped configuration sets enable_rquota = false, noted in the file as not supported on CephFS
Kerberos security Available Not available; every generated export is sectype = sys
Read-only, root squash, client filter Honoured Honoured
Full port range, async writes, subtree checks, custom export options, Filter Priority Honoured Not applied. These are accepted and stored, and the WUI still shows them, but they are not written into the Ganesha configuration
NFSv4 referrals for a Global Namespace Generated Not generated

Neither server offers RQUOTA, so quota on a client does not report a share's quota. Read the share quota from the WUI or from qs share-get, and per-user and per-group quotas from qs share-quota-list --share=<share>. Network Shares covers how they are set.

Hand edits to /etc/ganesha/ganesha.conf survive only outside the EXPORT sections, which QuantaStor regenerates. Put additions in /etc/ganesha/ganesha.conf.custom, which is appended verbatim on every save. Keep the OSNEXUS header comment in place: a ganesha.conf that does not contain that string is treated as foreign and replaced with the QuantaStor default.

Configuring the NFS service

Navigation: Storage Management → Network Shares → Network Share Services → Configure NFS (toolbar)
Navigation: Storage Management → Network Shares → Network Share (select + right-click) → Configure NFS Services...
NFS Services Configuration, at its defaults. The Kerberos Security Settings are greyed out until Enable Kerberos Security is ticked.

The NFS Services Configuration dialog holds every NFS setting that applies to a whole storage system. It is a per-system dialog even though it is reached from the Network Shares view -- the Storage System combo at the top selects which member of the grid you are configuring, and the fields below repopulate from that system's current configuration when you change it.

Setting Default Notes
NFS Server Type Scale-up Scale-up or Scale-out, as described above.
NFS Protocol Mode NFS v3/v4 NFS v3/v4, NFS v3, or NFS v4. See NFS protocol versions.
Disable NFSv4 /export Browsing off Per the dialog's own tooltip this blocks creation of the NFSv4 export root (fsid=0), which is what lets a client mount the top of the tree and browse every share. In the default NFS v3/v4 mode QuantaStor generates no root export, so there is nothing for the setting to suppress and it has no effect there.
Manage GIDs on See How UIDs and GIDs resolve.
Enable Kerberos Security off Enables the two Kerberos Security Settings below and switches Default Security Policy to kerberos. Clearing it switches the policy back to system and greys both fields out again.
Kerberos Mode Kerberos Integrity (krb5i) Kerberos Integrity (krb5i), Kerberos Privacy (krb5p), or Kerberos Client/Server Auth (krb5). These map to the NFS sec=krb5i, sec=krb5p and sec=krb5 export options.
Default Security Policy system kerberos or system. The system-wide default that a share uses unless it carries its own override. Its tooltip makes the point that matters: with Kerberos configured you can still have Network Shares that do not require Kerberos security, because each share can override this.
NFS Server Type offers exactly two choices. Scale-up leaves the kernel NFS server on port 2049; Scale-out hands 2049 to NFS-Ganesha and masks the kernel server.

Two details about the dialog are worth knowing before you read a value off it. The Default Security Policy actually stored is a third value, inherit, which the dialog never shows -- it renders the resolved policy instead, so a system with Kerberos off displays system while the stored setting is inherit. And NFS Server Type is read back from the Ganesha configuration on disk rather than from the database, so it always reflects which server currently holds port 2049.

There is no qs command for any of these settings -- they are set from the WUI and the REST API only. You can read the whole configuration back with qs system-get:

                     Nfs Mode: NFSv3+v4
        Nfs4 Kerberos Enabled: false
            Nfs Kerberos Mode: 0
       Nfs4 Browsing Disabled: false
    Default Nfs Security Policy: inherit
              Nfs Server Type: scaleup
              Nfs Manage Gids: true
          Nfsd Service Status: Running

Kerberos

With Enable Kerberos Security ticked, both Kerberos fields become editable and the Default Security Policy flips to kerberos.

Enable Kerberos Security turns on the GSS daemons and puts a sec= prefix on the export lines QuantaStor generates. It does not build the Kerberos environment for you: no keytab is created, and /etc/krb5.conf is not written. QuantaStor says so at the time, raising an alert that reads "Secure NFSv4 access on '<system>' will not be enabled until the Kerberos configuration steps are completed including manually editing /etc/krb5.conf to setup the domain/realm(s) for your environment."

The practical order is therefore to establish Kerberos first and enable the checkbox second. Joining the system to an Active Directory domain is the supported route -- the domain join generates /etc/krb5.conf from a template, creates the machine keytab at /etc/krb5.keytab, and points Samba at the system keytab. See Network Shares for the Join AD Domain operation and Security Configuration for the wider authentication picture.

How the policy resolves for a given share, in order:

  1. Start from the system's Default Security Policy.
  2. If that is inherit, resolve it to kerberos when Enable Kerberos Security is on and to system when it is off.
  3. If the share carries its own override, that wins.

The per-share override is the one part of this the CLI does reach: qs share-modify --share=<share> --nfs-security-policy=<inherit|kerberos|system>, with inherit the default. Two consequences follow from how the resolved policy is used:

  • An effective policy of system emits no sec= option at all. QuantaStor never writes sec=sys; AUTH_SYS is what the server falls back to.
  • A per-share policy of kerberos on a system where Enable Kerberos Security is off is silently ignored, because the sec= prefix is only emitted when Kerberos is enabled system-wide.

Only one flavour is offered at a time. QuantaStor writes a single sec=krb5, sec=krb5i or sec=krb5p, never a list, so a client must match the configured mode.

NFS protocol versions

NFS Protocol Mode is a system-wide radio with three positions, and it writes the vers3 and vers4 keys in the [nfsd] section of /etc/nfs.conf:

Mode [nfsd] vers3 [nfsd] vers4 Use it when
NFS v3/v4 (default) y y Mixed client estates. Each client's own mount options decide which version it gets.
NFS v3 y n Clients that cannot speak NFSv4 and you want to be certain nothing negotiates up.
NFS v4 n y Kerberos, or a policy of NFSv4 only. Removes the need for rpcbind and the NFSv3 helper daemons.

QuantaStor does not configure the NFSv4 minor versions. The vers4.0, vers4.1 and vers4.2 keys in /etc/nfs.conf are left at their distribution defaults, which have 4.1 and 4.2 enabled. In the default mode a Linux client asking for -o vers=4 negotiates 4.2, and -o vers=4.1 is honoured if you pin it. If you need to pin or exclude a minor version, edit those keys by hand -- but note that applying the dialog rewrites vers3 and vers4 in the same file, and the NFS configuration is reapplied from the database each time the QuantaStor service starts.

What changes for a client between v3 and v4

NFSv3 NFSv4
Mount path (default protocol mode) <ip>:/export/<share> <ip>:/export/<share> -- the same
Ports needed 2049 plus rpcbind on 111, plus mountd, statd and lockd on ephemeral ports 2049 only
Identity on the wire Numeric UID and GID user@domain strings, mapped at both ends
Locking Separate NLM protocol, via lockd Part of the protocol, with server-side state
Server state Stateless; a server restart is transparent to a hard mount Stateful; a server restart makes the client re-establish its session and reclaim locks
Browsing the whole tree Not possible; each export is mounted by path Mount <ip>:/ and walk down into export/

The mount path being identical for both versions is worth stating plainly, because NFSv4 servers commonly present shares under a pseudo-root that hides the real path. On QuantaStor in the default protocol mode the NFSv4 pseudo-filesystem is rooted at / and the shares appear beneath /export, exactly where NFSv3 finds them.

The export configuration QuantaStor generates

File What it is
/etc/exports Generated by QuantaStor for the kernel NFS server. One line per exported share, rebuilt from the database whenever anything relevant changes and whenever the QuantaStor service starts.
/etc/exports.custom Where your own export entries go. Its contents are appended verbatim to the generated file after a ## CUSTOM EXPORTS marker line.
/etc/nfs.conf The NFS daemon configuration. QuantaStor writes [nfsd] vers3, [nfsd] vers4 and [mountd] manage-gids and preserves the rest.
/etc/default/nfs-kernel-server QuantaStor writes RPCMOUNTDOPTS and the GSS daemon switches here.
/etc/ganesha/ganesha.conf The scale-out server's configuration, with a QuantaStor-managed EXPORT section per Ceph share.
/etc/ganesha/ganesha.conf.custom Where your own Ganesha entries go.

The generated exports file opens with its own warning:

## WARNING: FILE GENERATED BY QUANTASTOR CORE SERVICES, MANUAL CHANGES MAY BE LOST
##
## NOTE: Scale-out CephFS NAS exports are presented via NFS-Ganesha on alternate default port 2249 rather than 2049.
##

Take that literally. On each regeneration QuantaStor keeps only single-# comments and export lines whose path is not under /export, /mnt/storage-pools/, /mnt/namespaces/ or /mnt/cloud-containers/; everything else it wrote before, and everything after the ## CUSTOM EXPORTS marker, is discarded and rebuilt. A hand-added export under /export will disappear. Use /etc/exports.custom instead.

The line QuantaStor writes for a share

A share is exported at /export/<share>, which is a recursive bind mount of the share's real path under /mnt/storage-pools/qs-<pool-id>/<share>. Both paths are shown by qs share-get as Export Path and Share Path.

A share created with default settings gets one [public] client rule, and produces one line:

/export/share1 *(rw,insecure,sync,no_subtree_check,crossmnt,fsid=8d13e546-ffa8-3abd-8c3f-d425443501ac)

Every client access rule on the share becomes one <filter>(<options>) group on that same line, and QuantaStor assembles the options in a fixed order:

[sec=krb5|krb5i|krb5p,] ro|rw , secure|insecure , async|sync , subtree_check|no_subtree_check
    [,<custom export options>] [,crossmnt] [,nohide] [,fsid=<share UUID>] [,refer=...]

The fsid= value is the share's own object ID. Pinning it to something stable, rather than letting the server derive it from the underlying device, is what keeps a client's file handles valid across a service restart or a move of the pool to another node. It changes if a share is deleted and recreated under the same name, which is the usual cause of a stale file handle on a client that stayed mounted.

Rule order within a line

The kernel NFS server uses the first matching entry on a line, not the most specific one, so the order QuantaStor writes them in is what decides which rule a client gets. QuantaStor sorts most specific first:

Order Filter kind Example
1 Single IP address 192.168.10.44
2 Host or domain wildcard, and netgroups *.example.com, @netgrp
3 Network 192.168.10.0/24
4 Public *

Filter Priority on a rule breaks ties within a class -- it does not lift a network rule above a host rule. Leaving it on Auto sorts the rule last within its class. Five rules on one share come out like this:

/export/share1 192.168.10.44(rw,secure,sync,no_subtree_check,no_root_squash,anonuid=150,anongid=100,crossmnt,fsid=...)
    *.example.com(rw,secure,sync,no_subtree_check,crossmnt,fsid=...)
    @netgrp(rw,secure,sync,no_subtree_check,crossmnt,fsid=...)
    192.168.10.0/24(ro,secure,async,subtree_check,crossmnt,fsid=...)
    *(rw,insecure,sync,no_subtree_check,crossmnt,fsid=...)

(Shown wrapped for readability; the appliance writes one share per physical line.)

Sub-exports for snapshot browsing

A share can produce two extra export lines, both covered under Snapshot access over NFS. Neither carries an fsid=.

Line Appears when Extra options
/export/<share>/.zfs The share has at least one snapshot and NFS browsing of the hidden snapshot directory is enabled crossmnt,nohide
/export/<share>_snaps The share has at least one activated GMT snapshot and NFS _snaps browsing is enabled crossmnt

Because both are conditional on snapshots existing, the set of exports on a system grows and shrinks as snapshots are taken and expired. A client that mounted /export/<share>_snaps will find it gone once the last activated snapshot is removed.

Client access rules and export options

Access to a share over NFS is granted per client by a rule carrying its own export options. The dialog for adding and modifying those rules, the five filter kinds, and every checkbox on it are documented on Network Shares under NFS client access. What follows is the export option each control produces, and the CLI equivalents.

Export option Set by What it does
rw / ro Read Only on the rule, or Read Only on the share -- the share-level setting wins Whether the client may write.
insecure / secure Allow Full Port Range Access -- ticked gives insecure secure requires the client's requests to come from a reserved source port below 1024. Many clients do not use one by default, including macOS without resvport and a good number of containerised and appliance clients, so secure is a common cause of an unexplained permission denied.
sync / async Async Writes async lets the server acknowledge a write before it is on stable storage. Faster, and it breaks the NFS protocol's durability guarantee; the dialog's tooltip says not recommended.
no_subtree_check / subtree_check Subtree checks Subtree checking asks the server to verify that a file handle falls within the exported subtree. It exists for nested exports, costs performance, and can break renames. QuantaStor does not nest NFS exports, so leave it off.
no_root_squash Allow Root Access, which appends the literal string to the rule's custom options Without it, root on the client is mapped to the anonymous user. This is one of the few options QuantaStor does not write itself -- it reaches the export line through the custom options field.
crossmnt On unless Disable NFS cross-mount access to share is set on the share Lets a client traverse from the export into file systems mounted beneath it. It is what makes snapshot directories reachable through a single mount.
fsid=<uuid> QuantaStor, automatically A stable file system identity for the export. Do not set it by hand.
sec=krb5 / krb5i / krb5p The system Kerberos Mode, when Kerberos is enabled and the effective policy is kerberos The security flavour a client must match.
Anything else Custom NFS Export Options on the rule A comma-delimited list, appended verbatim. This is how you reach options QuantaStor has no control for -- all_squash, anonuid=, anongid=, no_wdelay.

Options QuantaStor never writes on its own, and which therefore behave as the kernel's defaults unless you add them to the custom options field, include root_squash (the default, hence the Allow Root Access toggle), wdelay, all_squash, anonuid= and anongid=.

From the CLI

qs share-client-add    --share=share1 --filter=192.168.10.0/24 --rdonly=true
qs share-client-add    --share=share1 --filter=192.168.10.44 --options="no_root_squash,anonuid=150,anongid=100"
qs share-client-list   --share=share1
qs share-client-modify --share=share1 --filter=192.168.10.0/24 --async=true
qs share-client-remove --share=share1 --filter=192.168.10.0/24

The commands are qs share-client-add, qs share-client-list, qs share-client-modify and qs share-client-remove. A rule is identified by its --filter, so modify and remove take the same filter string the rule was created with. --filter takes several filters in one call, comma-separated, and creates one rule per filter -- --filter="192.168.10.1,192.168.10.2" adds two rules, not one.

Three traps in the CLI surface:

  • --secure defaults to true on the CLI and to off in the WUI. A rule added with qs share-client-add and no --secure comes out secure; the same rule added in the dialog, and the [public] rule QuantaStor creates with a new share, come out insecure. If a client works when you add its rule in the WUI and fails when you script it, this is why. Pass --secure=false explicitly.
  • --options is not normalised on modify. On share-client-add, a recognised export option typed into --options is folded into the matching flag. On share-client-modify it is kept as free text, so --options=ro produces a self-contradictory (rw,...,ro) on the export line. Use the dedicated flags.
  • IPv6 filters are not accepted. A client filter may contain letters, digits and @ / * . - only, so a colon in an address or prefix is rejected outright: "Unable to update network share with invalid client access rule '2001:db8::5', valid characters include alpha-numeric and symbols '@/*.-'." Restrict NFS access by IPv4 network, host name or netgroup instead.

To copy a whole set of NFS rules and SMB settings from one share to another, use qs share-copy-settings --share=<target> --share-copy-settings-source=<source>.

A share with no client rules is not exported

A share with no NFS client rule and no SMB access is taken offline, with the state detail "No access via SMB or NFS has been configured, share is offline." and no line in the exports file. Adding at least one client rule is what puts it back. The one exception is a Kerberos system, where a share with no client rules is still exported to * with the Kerberos prefix.

Applying export changes

Export changes and service configuration changes are applied by two different mechanisms, and only one of them touches clients.

Change How it is applied Client impact
Add, modify or remove a client access rule The exports file is regenerated and exportfs -r is run None. The server swaps its export table in place.
Create, modify, delete, enable, disable, snapshot, clone or rename a share; cross-mount and snapshot browsing options; a share's NFS security policy Same None
Default Security Policy and NFS Server Type in the NFS Services Configuration dialog Same None from the reload itself. Changing the server type does swap which daemon is serving.
NFS Protocol Mode, Enable Kerberos Security, Kerberos Mode, Disable NFSv4 /export Browsing, Manage GIDs Configuration files are written and nfs-server is restarted Momentary interruption for every NFS client on that system
HA VIF changes and an HA pool failover Exports regenerated plus, on the releasing node, a service restart See NFS over an HA storage pool

The five settings in the fourth row are the only ones in the dialog that bounce the service, and the dialog restarts it only when one of them actually changes -- reapplying the dialog with nothing altered does not interrupt anything.

A reload really is transparent: an active NFSv3 mount and an active NFSv4.2 mount both kept serving reads across repeated export regenerations while client rules were added and removed and snapshot sub-exports appeared, with no error and no remount.

Removing a client access rule additionally unexports that one filter and path immediately, rather than waiting for the regeneration, so that client loses access at once while every other client on the share is untouched.

Restarting the NFS service

Navigation: Storage Management → Network Shares → Network Share Services → Restart NFS/SMB Services (toolbar)
Navigation: Storage Management → Network Shares → Network Share (select + right-click) → Restart NFS/SMB Services...
Restart Storage Services. Both services are selected by default, so untick the one you do not want restarted.

The toolbar and menu item are labelled Restart NFS/SMB Services, and the dialog that opens is titled Restart Storage Services. It restarts the NFS service, the CIFS/SMB service, or both, on the storage system chosen in the combo. Both checkboxes are ticked when the dialog opens, so restarting only NFS means clearing the CIFS/SMB one first.

The dialog carries its own warning, and it is accurate: "WARNING: This will momentarily interrupt access to the volumes/shares associated with these services and may effect workloads using them." NFS is restarted first, then SMB.

What it does not do is regenerate the exports file. It is purely a service bounce, so the exports the server comes back with are whatever the last regeneration left on disk. If a share is missing from the exports file, restarting the service will not add it -- fix the share's client rules or its pool state instead, which regenerates the file on its own.

Reach for it when:

  • the NFS service is reported as not running, or has been left in a failed state;
  • an NFSv4 client is wedged in a way that only clearing server-side state will fix;
  • you hand-edited /etc/nfs.conf and need the daemon to re-read it.

There is no qs command for this operation. It is available from the WUI and the REST API only.

Two appliance-side commands cover cases the dialog does not. qs-util nfsstandard and qs-util nfsganesha apply the Scale-up and Scale-out server configurations directly, and are the way back when the configuration on disk and the recorded server type have diverged. qs-util nfsrdma enables NFS over RDMA -- it loads the rpcrdma module, sets rdma=y and rdma-port=20049 in /etc/nfs.conf after backing the file up, restarts the server and verifies the RDMA port came up. RDMA has no equivalent in the WUI, and it needs lossless or PFC configured on both the appliance and the client to be worth having.

NFS over an HA storage pool

An HA Storage Pool moves between nodes together with one or more virtual network interfaces. Clients must mount the HA VIF, never a node's own management or data address -- that is the whole point of the VIF, and a mount to a physical address does not follow the pool. See HA Cluster Setup (JBODs) for setting the group up.

What QuantaStor does during a failover, in order:

  1. On the releasing node, the HA VIFs are firewalled off first, so clients stop reaching the node before anything about the exports changes.
  2. The pool's shares are removed from the exports file and unexported, then the pool is exported at the storage layer.
  3. The NFS and SMB services on that node are restarted to force session cleanup. This happens asynchronously and can land up to a few minutes after the failover otherwise looks finished, and it interrupts clients of other pools on the same node as well.
  4. On the acquiring node, the pool is imported, the exports file is regenerated and reloaded, and the NFS and SMB services are started if they were not already running.
  5. The VIF firewall block is lifted last, so a client's first packet after the move arrives at a server that is already exporting the share.

What a client sees:

  • NFSv3 with a hard mount -- I/O stalls while the VIF is down and resumes when it comes back on the new node. Because NFSv3 is stateless and the export keeps the same fsid=, the mount recovers without intervention. A soft mount returns errors to the application instead, which is usually worse.
  • NFSv4 -- the client's session is gone. It sees a stale client ID or a bad session, re-establishes, and reclaims what it can.
  • Locks are not migrated. QuantaStor does not send sm-notify to NFSv3 clients and does not coordinate an NFSv4 grace period across the failover, so byte-range locks held before the move are lost. An application that depends on advisory locking for correctness has to be able to re-acquire them, and there is a window in which another client can take a lock the previous holder still believes it owns.

Two HA-specific settings affect NFS:

  • Enforce NFS HA VIF Access on the HA group replaces a public (*) client filter on that pool's shares with one entry per HA interface subnet, written as <network>/<netmask>. It is a way to keep a share that is nominally public reachable only from the networks the HA VIFs live on.
  • QuantaStor installs a systemd drop-in for nfs-server that makes its start-time exportfs -r failure-tolerant. Without it, the service refuses to start on a node whose export paths are absent -- which is the normal state for a pool that has failed over away.

How UIDs and GIDs resolve

QuantaStor stores POSIX ownership on the file system, so what a client sees depends on how the two ends agree about identity.

NFSv3 sends numeric UIDs and GIDs on the wire. There is no translation: a file owned by UID 1001 shows as whatever user is 1001 on the client. Matching numeric IDs at both ends is the only thing that makes ownership meaningful.

NFSv4 sends user@domain strings, which each end maps through rpc.idmapd. The domain is configured in /etc/idmapd.conf, and QuantaStor leaves it unset, so it defaults to the appliance's DNS domain -- its fully-qualified name minus the host part. If the client's idmap domain does not match, every file shows as nobody:nogroup even though access works. This is the single most common NFSv4 surprise. Set Domain in [General] on the client to match the appliance, then restart nfs-idmapd or run nfsidmap -c to flush the cache.

Name-to-ID resolution on the appliance goes through NSS. Out of the box that is local files, then SSSD, then winbind. Joining Active Directory is what populates the last two: the domain join rewrites /etc/nsswitch.conf and configures SSSD and winbind, at which point AD users and groups resolve on the appliance and their POSIX identities can be used for NFS ownership. Whether IDs come from the directory's RFC 2307 attributes or from an algorithmic mapping is decided by the AD join, not by anything on this page -- see Network Shares for the join and Security Configuration for directory integration.

Manage GIDs

Manage GIDs is on by default. Its tooltip describes it exactly: the appliance ignores the group list a client sends and makes its own independent check of the user's group memberships, via local /etc/group or the directory service.

The reason it exists is a hard limit in AUTH_SYS: the credential a client sends carries at most 16 supplementary group IDs. A user who belongs to more than 16 groups has some of them silently dropped, and loses access to files whose group fell off the end -- non-deterministically, because which 16 survive is up to the client. With Manage GIDs on, the server discards the client's list entirely and resolves the user's full group membership itself.

Two consequences to plan for:

  • The appliance must be able to resolve the user's groups. If a UID resolves on the client but not on the appliance, that user loses all supplementary groups rather than gaining the full set. On an AD-backed system that means the join has to be healthy.
  • It takes effect in rpc.mountd, so changing it restarts the NFS service and interrupts clients.

QuantaStor writes it in two places -- [mountd] manage-gids in /etc/nfs.conf and RPCMOUNTDOPTS="--manage-gids" in /etc/default/nfs-kernel-server -- and reads the setting back from those files rather than from the database, so an edit made by hand is what the WUI will report.

Snapshot access over NFS

Which snapshot browsing routes are open is a per-share decision made on the share's Advanced Settings, and the defaults are deliberately restrictive: over NFS, both routes are closed. Network Shares documents the switches. This section is what the NFS side looks like once they are open.

Route Export What a client sees
The _snaps folder /export/<share>_snaps, a separate export beside the share One directory per activated snapshot, named @GMT-2026.09.03-05.10.53. Mount it in its own right.
The hidden snapshot directory /export/<share>/.zfs, a sub-export of the share .zfs/snapshot/ reached from inside the share's own mount, holding one read-only directory per snapshot -- named GMT-2026.09.03-05.10.53, without the leading @. On a scale-out share the equivalent directory is .snap.

Reading a file out of a snapshot therefore looks like this:

sudo mount -t nfs -o vers=4,ro 192.168.10.5:/export/share1 /mnt/share1
ls /mnt/share1/.zfs/snapshot/
# GMT-2026.09.03-05.10.53
cp /mnt/share1/.zfs/snapshot/GMT-2026.09.03-05.10.53/report.xlsx ~/

sudo mount -t nfs -o vers=3,ro 192.168.10.5:/export/share1_snaps /mnt/share1_snaps
ls /mnt/share1_snaps/
# @GMT-2026.09.03-05.10.53

Traversing into .zfs/snapshot/<name> from inside the share's mount depends on crossmnt, because each snapshot is a file system of its own. That is why Disable NFS cross-mount access to share and snapshot browsing interact: turning cross-mount off leaves the directory visible and its contents unreachable.

Neither export appears until the share has snapshots, and the _snaps export additionally needs at least one snapshot that is activated. A snapshot that exists but is not active is not presented to clients.

Mounting from a Linux client

Install the NFS client package first: nfs-common on Debian and Ubuntu, nfs-utils on RHEL, Rocky and SUSE.

The View Mount Command dialog -- right-click a share, described on Network Shares -- gives you the share path and a ready-made mount command for whichever network port you pick, which is the quickest way to get the right address for a share you are looking at.

A worked mount

sudo mkdir -p /mnt/share1
sudo mount -t nfs -o vers=4,hard,proto=tcp 192.168.10.5:/export/share1 /mnt/share1

That is the whole of it for a normal share. The options worth thinking about:

Option Notes
vers=3 / vers=4 / vers=4.1 Pin the version rather than letting the client negotiate, so a client's behaviour does not change when the appliance's protocol mode does. vers=4 negotiates 4.2 against a QuantaStor appliance.
hard (default) Retry indefinitely. This is what you want for anything holding data: an HA failover, a service restart or a network blip becomes a stall the application survives rather than an I/O error it has to handle. Use soft only where a hung process is worse than a failed write, and pair it with timeo and retrans.
proto=tcp (default) Explicit is better than implicit, and UDP is not appropriate for NFS at any useful scale.
rsize / wsize Leave them alone. A Linux client already negotiates 1 MiB (1048576) in each direction against a QuantaStor appliance, over both NFSv3 and NFSv4, so the common advice to raise them to 1 MB has nothing left to raise. Confirm what you actually got with nfsstat -m rather than setting them on faith.
ro Worth using for a snapshot mount or an audit, and it is enforced by the client as well as the export.
noresvport Use a non-reserved source port. Needed only if a rule was created with secure; if the rule has Allow Full Port Range Access set -- which is the WUI default -- this is unnecessary.
sec=krb5i Required when the share's effective security policy is Kerberos, and must match the appliance's Kerberos Mode. Get a ticket first with kinit, and confirm it with klist.

Making it persistent

Add a line to /etc/fstab:

192.168.10.5:/export/share1  /mnt/share1  nfs  vers=4,hard,proto=tcp,_netdev  0  0

_netdev is the part that matters: it tells the client not to attempt the mount until networking is up, which is what stops a boot from hanging or the mount from silently failing. Leave the dump and fsck fields at 0. On a systemd client, x-systemd.automount in place of a plain mount is worth considering for a share that is not needed at boot, since it defers the mount until something touches the directory.

Browsing all shares over NFSv4

An NFSv4 client can mount the top of the tree and walk down to find every share it has access to:

sudo mount -t nfs -o vers=4,ro 192.168.10.5:/ /mnt/qs
ls /mnt/qs/export
# share1  share2  share3

Mounting a scale-out share

A share on a CephFS pool or a Ceph object gateway bucket is served by NFS-Ganesha, and two things differ. The path is the Ganesha pseudo path -- /<share>, or /s3/<bucket> for a bucket -- not /export/<share>. And while the storage system's NFS Server Type is Scale-up, Ganesha is on port 2249 rather than 2049, so the client has to be told:

sudo mount -t nfs -o vers=4,port=2249,mountport=2249 192.168.10.5:/share1 /mnt/share1

With the server type set to Scale-out, Ganesha owns 2049 and no port override is needed. NFSv3 is not an option against the alternate port.

Windows clients

Every Network Share is exported over NFS and SMB at the same time, and SMB is the right protocol for a Windows client -- it carries Windows ACLs, Active Directory identity and the Previous Versions snapshot tab, none of which an NFS mount gives you. See Network Shares for the SMB share options and the AD domain join.

Confirming what is exported and who is mounted

showmount -e 192.168.10.5    # every export the appliance is offering
showmount -a 192.168.10.5    # NFSv3 mounts currently held, as <client>:<path>
nfsstat -m                   # on the client: what each mount actually negotiated

showmount reports NFSv3 state, so an NFSv4 mount does not appear in -a. Both need rpcbind and the mount service reachable, which means they do not work against a scale-out share on port 2249.

Ports and firewall

Service Ports Needed for
NFS 2049 TCP and UDP, 111 TCP and UDP NFSv4 needs 2049 only. NFSv3 also needs rpcbind on 111.
NFS Ganesha 2249 TCP Scale-out shares while the server type is Scale-up.
NFS over RDMA 20049 Only after qs-util nfsrdma.

Those are the definitions QuantaStor's own firewall management works from, in /opt/osnexus/quantastor/conf/qs_services_firewall.conf, and the ports it monitors for reachability.

NFSv3 needs more than that list. mountd, statd and lockd take ephemeral ports, and QuantaStor does not pin them, so NFSv3 through a stateful firewall needs those ports fixed by hand in /etc/nfs.conf and opened to match. Check what they currently are with rpcinfo -p <appliance>. NFSv4 has none of this problem, which on its own is a reason to prefer it where a firewall sits between the client and the appliance.

Troubleshooting

A share does not appear to clients

  1. Confirm the share has at least one NFS client access rule -- qs share-client-list --share=<share>. A share with none, and no SMB access, is taken offline.
  2. List what the appliance is actually exporting: showmount -e <appliance>. If the share is not there, the exports file has not been regenerated with it; check the share's state and its pool's state rather than restarting the service.
  3. Confirm the pool is imported on the node you are talking to. A share on an HA pool that has failed over is exported on the other node.

Permission denied on mount

  1. Check that the client's address matches a rule, and that a broader rule is not being matched first. The first matching entry on the export line wins, and QuantaStor orders host rules ahead of network rules ahead of the public rule.
  2. Check whether the rule is a secure rule, which every rule created by qs share-client-add without an explicit --secure=false is. A secure rule requires the client to connect from a reserved source port below 1024, and a client mounting with noresvport, or one that never uses a privileged port, is refused. The fix is to set Allow Full Port Range Access on the rule, or recreate it with --secure=false. Confirm what the rule actually is with qs share-client-list --share=<share>, or by reading the export line.
  3. On a Kerberos system, confirm the client has a valid ticket and is using the sec= flavour the appliance is configured for.

NFSv4 clients hang while NFSv3 clients keep working

This pattern points at the NFSv4 client tracking daemon, nfsdcld, which records client state in a small database on the appliance's root file system. A full root file system can corrupt that database, at which point the daemon fails, the kernel blocks NFS server threads waiting on it, and NFSv4 mounts stall while stateless NFSv3 traffic is unaffected.

QuantaStor detects this and repairs it automatically -- it moves the damaged database aside so the daemon rebuilds it, and restarts the NFS server -- and raises an alert saying so. Repair is rate limited and gives up after a few attempts, so if the alert persists, the thing to fix is free space on the root file system. Recovery is deliberately skipped on a system where NFS-Ganesha is serving, and on a node whose pool has failed over away, because a stale failure there is not an outage.

Files show as nobody:nogroup over NFSv4

An ID mapping domain mismatch. See How UIDs and GIDs resolve. Access still works; only the displayed ownership is wrong.

Stale NFS file handle

The export's fsid= is the share's object ID, so it changes if a share is deleted and recreated under the same name. A client that stayed mounted across that is holding handles for a file system that no longer exists. Remount:

sudo umount -l /mnt/share1
sudo mount -t nfs -o vers=4 192.168.10.5:/export/share1 /mnt/share1

NFSv3 mounts fail with an RPC port mapper error

NFSv3 needs rpcbind, on port 111, in addition to the NFS server. Check that 111 is reachable and that the appliance's protocol mode has not been set to NFS v4, which disables NFSv3 outright.

Finding out who is connected

QuantaStor tracks SMB sessions only. The SMB Sessions tab and qs share-session-list do not report NFS clients, so do not read an empty list as nobody being mounted. For NFS, use the NFS Stats tab of the Share Dashboard for throughput and operation rates, and showmount -a on the appliance for NFSv3 mounts.

Slow NFS writes

Work through the storage before the protocol. Synchronous NFS writes land in the pool's ZFS intent log, so a pool without a write log device commits them to the data disks -- see Storage Pools for adding one. After that, confirm the client negotiated 1 MiB read and write sizes with nfsstat -m, confirm raw network throughput between the two ends independently of NFS, and only then consider Async Writes on the client rule, which trades the protocol's durability guarantee for speed.

Related pages


Verified against QuantaStor 6.9.0.